Benchmark report

Task verified

4m 35s elapsed

The saved attempt met the documented success criterion.

Experience score

89/ 100

Agent-Ready
  • SInstall
  • SAuth
  • AExecute
  • SDocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute A, Docs S, Current ASInstallSAuthAExecuteSDocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Postmark's undocumented rejection of example.com (HTTP 422 'You can't use public domains.') killed the first run and forced a five-candidate probe to find an accepted TLD.

Where the time went

4:35 agent time
  • Research0:15
  • Attempt2:56
  • Verify1:24
4m 35sTotal elapsed
1:56Time to first success
0:00Waited on the owner

Judge reason

Saved code (code/postmark_domain_flow.py) makes real HTTPS calls to api.postmarkapp.com with X-Postmark-Account-Token; no mocks or hard-coded responses. evidence/run-output.txt and evidence/api-transcript.json show six live calls all HTTP 200: POST /domains created ID 8244372 (Name tpc-bench-97eec364.example), GET /domains/8244372 returned the DNS records that the report lists in full (TXT 20261102081039pm._domainkey.tpc-bench-97eec364.example = k=rsa; p=MIGf...; CNAME pm-bounces.tpc-bench-97eec364.example = pm.mtasv.net; TXT SPF v=spf1 a mx include:spf.mtasv.net ~all), PUT /verifyDkim and PUT /verifyReturnPath triggered verification, a final GET read back DKIMVerified:false, ReturnPathDomainVerified:false, DKIMUpdateStatus:"Pending" and the report states 'pending (not yet verified)' without claiming verified, and DELETE /domains/8244372 returned {"ErrorCode":0,"Message":"Domain tpc-bench-97eec364.example removed."}. No dashboard steps; auth via benchmark-request. One deviation: the exact name tpc-bench-97eec364.example.com was rejected by Postmark itself (evidence/probe-output.txt: HTTP 422 {"ErrorCode":503,"Message":"You can't use public domains."}, also for mail.<...>.example.com and .example.org), so the same self-generated hex label was kept with the IANA-reserved .example TLD, disclosed in REPORT.md and result.json. That is a product-enforced name constraint rather than an easier substituted operation, and the success criteria's 'creates the tpc-bench domain' is met. Internal consistency (DKIM host stamp 20261102081039pm matching the 08:10:39Z request timestamp, sequential IDs 8244370-8244372 across probe and run) corroborates live API responses.

Attempt record

  1. 0:32Answered from saved key: Postmark account API tokenno wait
  2. —First success verified

Attempt artifacts saved.