Benchmark report

Task verified

3m 45s elapsed

The saved attempt met the documented success criterion.

Experience score

98/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current SAInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Only the PEP 668 externally-managed-environment rejection of the first pip install, resolved immediately with a venv; otherwise first-try success in ~90s.

Where the time went

3:45 agent time
  • Research0:15
  • Attempt2:14
  • Verify1:16
3m 45sTotal elapsed
1:30Time to first success
0:00Waited on the owner

Judge reason

Saved runnable code (agentmail-task/inbox_lifecycle.py) uses the official agentmail Python SDK 2.0.25 against https://api.agentmail.to with a key read from a private file. Live run (run.log, evidence/01-05) shows POST /v0/inboxes -> HTTP 200 returning inbox_id and deliverable address sparklingview738@agentmail.to (both printed), GET pre-delete -> 200, DELETE /v0/inboxes/{id} -> 202 empty body, then bounded polling: poll 1 at t=0.0s -> 200, poll 2 at t=2.1s -> 404 not_found ('Inbox not found'), well inside the 60s wait. No mocking or hard-coded responses; real server-generated ids, pod_id, organization_id and date header are present and consistent across files.

Attempt record

  1. 0:43Answered from saved key: AgentMail API key (AGENTMAIL_API_KEY, am_...)no wait
  2. —First success verified

Attempt artifacts saved.