GDPval-AA v2

What it measures

GDPval-AA v2 is an independent Artificial Analysis evaluation built on OpenAI’s GDPval dataset. It uses the 220-task public gold set across 44 occupations and nine industries, requiring actual work products rather than answers alone.

For example, an official sample asks for a one-page stage-plot PDF for a touring band: place the singers, drummer, amplifiers and microphones correctly, then list the input and output connections. Explaining how to draw the diagram is not enough; the model must produce a file whose content and layout meet the brief.

All models work through the Stirrup agent harness with shell access, file processing, web search and browsing, for up to 250 turns per task. Model identities are hidden and an LLM judge sampled from a three-model panel compares two submissions for the same task; these comparisons are aggregated into Elo ratings.

Reading the scores

Higher Elo means the deliverables are preferred more often in pairwise comparisons on this task set; human expert deliverables anchor the scale at 1,000. For example, ratings of 1,500 for A and 1,200 for B indicate stronger relative performance for A, not 150% accuracy or 25% greater efficiency. Elo can be negative, and the ± values are the published 95% confidence intervals; small ranking gaps should not be treated as certain differences. This snapshot preserves all 247 results and official model configurations checked on September 17, 2026. Entries labeled Fallback include a backup-model mechanism and are not results from a single model working entirely alone. Ratings can change as new comparisons are added, and should not be mixed with v1, launch-day snapshots or OpenAI’s own GDPval win rates; they are also distinct from the overall AA Intelligence Index.

SourceArtificial Analysis

Read the official GDPval-AA v2 methodology