Finance Agent v2.1

What it measures

Finance Agent v2.1 is Vals AI’s benchmark for the research work of junior financial analysts. Rather than answering finance trivia, agents must find public-company filings, connect information across sources and produce supported answers.

For example, comparing two companies’ inventory efficiency requires finding inventory and cost figures for the right periods, checking units and conventions, and calculating which company turns inventory over faster. Copying one number is not enough: the evidence, calculations and conclusion must hold up.

The benchmark covers nine categories, including earnings, market analysis, comparables, precedent transactions, accounting adjustments, disclosures and financial modeling. Its 926 questions comprise 27 public examples, 450 private validation questions and 449 held-out test questions; the reported results use only the test set.

Models share tools for SEC EDGAR search, web search, HTML parsing, information retrieval, calculation and historical prices, with a two-hour limit per question. Scores measure performance with this shared tool environment.

Reading the scores

The default ranking uses Partial Credit: each question has weighted checks, and failing any critical dealbreaker makes the entire question score zero. Correct dealbreakers with peripheral omissions can earn partial credit. For example, a wrong critical revenue figure earns zero even with a polished explanation, while a correct figure with a minor omission may earn partial credit. All-Pass counts a question as successful only if every check passes; the columns must not be added, and 60% Partial Credit does not mean 60% of questions were answered perfectly. A three-model jury of GPT-6.1 Sol, Gemini 3.8 Flash and Claude Sonnet 5.5 grades responses. This snapshot includes all 35 results from the v2.1 leaderboard marked updated October 29, 2026, retaining published reasoning settings. V2.1 changes the jury, fails hedged answers that give several candidate values, and revises some questions and rubrics; the source says scores are not comparable with v2, and v1.1 is kept separate too. The source reports standard errors, not 95% confidence intervals; this table does not relabel them as confidence intervals.

SourceVals AI