Benchmark report

Task verified

10m 35s elapsed

The saved attempt met the documented success criterion.

Experience score

93/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs A, Current AAInstallSAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Routing mismatch: the service domain's targetPort (3000) didn't match Railway's injected PORT=8080, causing ~3 min of 502s and an opaque 400 on the fix.

Where the time went

10:35 agent time
  • Research0:25
  • Attempt7:50
  • Verify2:19
10m 35sTotal elapsed
5:22Time to first success
0:00Waited on the owner

Judge reason

Saved evidence supports every criterion. Token obtained via benchmark-request and kept in /tmp/privcred; account/workspace resolved through documented GraphQL `me` (evidence/01-me.json). Project tpcbench-bf07e77d and service tpcbench-bf07e77d-svc created via projectCreate/serviceCreate; TPC_BENCH_SECRET set with Railway's documented variableUpsert mutation (04-*, response {"variableUpsert":true}), value generated locally with secrets.token_hex(32). app/server.js reads process.env.TPC_BENCH_SECRET at runtime and returns only the hex digest; deployed with railway up --detach (06-railway-up.log, deployment 1905c819). Public URL on Railway's own domain polled to HTTP 200 (07-poll.log, 404/502 -> 200 after targetPort fixed to 8080 via serviceDomainUpdate) and verify.sh printed remote sha256 c50187a0...0b342 == local sha256 (10-, 11-verify-output.txt). Captured deployment (7 lines) and build (188 lines) logs plus all saved files contain no secret; my independent scan of /workspace/home found only the expected digest and railpack image digests, and no bearer/access token (gql.sh only cats a token file). Cleanup: projectDelete true, six post-delete fetches returned HTTP 404 "Application not found" (13-), and me{projects} returned zero projects (14-).

Attempt record

  1. 0:52Answered from saved key: Railway API tokenno wait
  2. —First success verified

Attempt artifacts saved.