Benchmark report
Task verified
3m 25s elapsed
The saved attempt met the documented success criterion.
100/ 100
Agent-Ready- SInstall
- SAuth
- SExecute
- SDocs
- SCurrent
Nothing material: only the ~6.5s server-side map traversal; the whole attempt ran start-to-finish in about 90 seconds with no retries.
Where the time went
3:26 agent time- Research0:17
- Attempt1:56
- Verify1:13
Judge reason
Saved code tavily-map-task/map_postgres_docs.py makes exactly one POST to the documented URL-discovery endpoint https://api.tavily.com/map (not /search) with url=https://www.postgresql.org/docs/17/, instructions="tutorial" (Tavily's documented natural-language filter), limit=50, and server-side select_paths; no per-page fetch and no local filtering (urls = data["results"] used as-is). evidence/map_response_meta.json shows HTTP 200 from api.tavily.com/map with a Nov 11 2026 Date header; evidence/map_response_raw.json is a live response (request_id 84c4a06d-6a96-4d5b-964b-820e59813bc1, response_time 6.49, usage.credits 5) holding 22 unique URLs, all under https://www.postgresql.org/docs/17/ (verified independently). evidence/run_output.txt prints "URL count: 22" and the first 10 URLs, matching the raw response exactly. The fetched Map API reference records no beta/preview/early-access marking.
Attempt record
- 0:51Answered from saved key: Tavily API keyno wait
- —First success verified
Attempt artifacts saved.