Benchmark report

Task verified

6m 11s elapsed

The saved attempt met the documented success criterion.

Experience score

95/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current ASInstallSAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
The first full run aborted with HTTP 400 'Minimum instance size on paid plans is Micro' after the OpenAPI enum advertised nano, forcing a patch and a second run.

Where the time went

6:12 agent time
  • Research0:23
  • Attempt3:57
  • Verify1:51
6m 11sTotal elapsed
2:26Time to first success
0:00Waited on the owner

Judge reason

Saved code (work/code/supabase_lifecycle.py) makes real urllib calls to https://api.supabase.com and shells out to stock psql; nothing is mocked. Transcript entry 64 and work/outputs show the live run: POST /v1/projects -> HTTP 201 ref=soejbijcttrtdxjifjfn, name tpcbench-2b3f9424, region us-east-1, polled to ACTIVE_HEALTHY (03/04/04b). Pooler params came from GET /v1/projects/{ref}/config/database/pooler HTTP 200 (05) giving aws-0-us-east-1.pooler.supabase.com:6543, user postgres.soejbijcttrtdxjifjfn; 06-connection-parameters.txt records sslmode=require and 07-psql-select1.txt shows psql exit 0 with 'one = 1' plus PostgreSQL 17.11 version banner (pg_stat_ssl ssl=f is the documented Supavisor behaviour the criteria allow; openssl s_client -starttls postgres on the same endpoint returned CN=*.pooler.supabase.com, file 12). DELETE /v1/projects/{ref} -> HTTP 200 (08), follow-up GET -> HTTP 404 'Project not found' (09) and GET /v1/projects no longer lists it (10), with an independent curl+jq check showing 0 leftover tpcbench-* projects. No password or password-bearing connection string appears anywhere: all saved URIs show [REDACTED], 02-create-project-request.json has db_pass '[REDACTED]', and the script redacts at write time; the PAT was read from a private file into an env var and a grep scan found it in no artifact.

Attempt record

  1. 0:41Answered from saved key: SUPABASE_ACCESS_TOKENno wait
  2. —First success verified

Attempt artifacts saved.