Benchmark report

Task verified

4m 3s elapsed

The saved attempt met the documented success criterion.

Experience score

94/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs A, Current SAInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Quickstart omitting the /v0 version prefix plus a 404 on a guessed auth page cost two extra doc fetches; otherwise first success came in ~82s.

Where the time went

4:03 agent time
  • Research0:21
  • Attempt2:09
  • Verify1:32
4m 3sTotal elapsed
1:22Time to first success
0:00Waited on the owner

Judge reason

Saved runnable code (agentmail-task/code/send_email.py) makes authenticated Bearer-token HTTPS calls to AgentMail's documented REST API via stdlib urllib. api_transcript.json/run_console_output.txt show three live 200s: two POST /v0/inboxes creating greatfigure708@agentmail.to (sender, an account-permitted @agentmail.to address returned by the API) and charmingmanager998@agentmail.to (recipient inbox created for this run, since AgentMail documents no fixed test recipient), then exactly one POST /v0/inboxes/greatfigure708@agentmail.to/messages/send. The saved request body (to/subject/text) matches the literal send_body in the code, and the live response is HTTP 200 in the documented shape {"message_id","thread_id"} with real CloudFront/Apigw response headers; printed status 200 and message_id <010001a19b295292-80fc1459-44c9-4221-b164-ff05ef1dd811-000000@email.amazonses.com> match send_response.json exactly. Nothing mocked or hard-coded; verify_delivery.py's GET /v0/inboxes/{id}/messages independently returns count=1 with the same message_id, subject and from address. API transport only, no SMTP or dashboard.

Attempt record

  1. 0:57Answered from saved key: AGENTMAIL_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.