Benchmark report

Task verified

3m 25s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Nothing material: one initial fetch pulled ~300KB HTML doc pages before the agent switched to the .md variants; first success at 73s.

Where the time went

3:25 agent time
  • Research0:13
  • Attempt2:01
  • Verify1:11
3m 25sTotal elapsed
1:13Time to first success
0:00Waited on the owner

Judge reason

Saved runnable code (agentmail-task/create_delete_inbox.py, Python stdlib urllib, no mocks) calls the documented AgentMail API. Evidence 01_create_inbox_response.json shows POST https://api.agentmail.to/v0/inboxes -> HTTP 200 with real AWS API Gateway/CloudFront headers (Apigw-Requestid G7OZLiieyK4EJSA=, Date Mon, 16 Nov 2026 08:52:10 GMT) returning inbox_id and deliverable email bench-20261116085210@agentmail.to; run_output.txt prints both. 02_get shows a 200 read-back. 03_delete shows DELETE /v0/inboxes/bench-20261116085210%40agentmail.to -> HTTP 202 with empty body, matching the saved delete doc ('Deletion is asynchronous. A 202 means it has started... Poll GET until it returns 404'). 04_delete_polls.json shows poll #1 at t=0.0s HTTP 200 and poll #2 at t=2.15s HTTP 404 with body code 'not_found', 'Inbox not found' - well inside the 60s bound. Reported end state is 'deleted', not the intermediate 202.

Attempt record

  1. 0:44Answered from saved key: AGENTMAIL_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.