Science & mathematics reasoning index Beta
7 model configurations · Ordered by composite score. Similar point estimates do not establish a reliable winner. Benchmark source ↗
| Model / effortRun start dates · Full records in data sources | Beta · 0–100 | GPQADiamond · % | FrontierMathv2 · Tiers 1–3 · % | InputUSD / 1M tokens | OutputUSD / 1M tokens | Cache readUSD / 1M tokens | USD · Fixed usage |
|---|---|---|---|---|---|---|---|
| 94.9 | 95.77 | 93.68 | 10 | 50 | 1 | $20.00 | |
| 94.7 | 95.39 | 93.68 | 2 | 10 | 0.1 | $4.00 | |
| 92.9 | 95.58 | 88.77 | 2 | 10 | 0.1 | $4.00 | |
| 90.8 | 90.59 | 91.23 | 4 | 20 | 0.2 | $8.00 | |
| 84.6 | 95.39 | 68.42 | 0.75 | 3.75 | 0.075 | $1.50 | |
| 81.6 | 92.30 | 65.61 | 2 | 6 | Not published | $3.00 | |
| 76.8 | 92.74 | 52.98 | 2 | 6 | 0.5 | $3.00 |
Select a model for sources and conditions
Capability and price trade-offs
Higher: higher point score · Left: lower reference costFixed usage: 750,000 uncached input + 250,000 output tokens, including reasoning. The log cost axis clarifies the low-cost range; the score axis shows 70–100. Select a model for its exact configuration and evidence.
Sol’s reference cost is one fifth of Astra’s, with similar point scores. This comparison applies only to these two benchmarks and fixed usage; it establishes neither statistical superiority nor best value in every setting.
Scoring method & scope
Composite = 60% GPQA + 40% FrontierMath · Science questions and mathematics only · Code, literature and workflows excluded.
Data & pricing sources
Official tariffs and special conditions are grouped below. The JSON preserves each original mean, SE, Started at, record ID and complete billing conditions; results were not independently rerun. Download full data JSON ↓
Epoch ↗ · GPQA Diamond ↗ · FrontierMath v2 ↗
| Model | Official sources | Special billing · USD / 1M tokens |
|---|---|---|
| GPT-6 Astramax | API ↗ · Cache ↗ | Cache write $12.5 / 30m · USD/1M Long context: input_tokens > 272000 → $20 / $75.0 input/output Regional option ×1.1 |
| GPT-6.1 Solmax | API ↗ · Cache ↗ | Cache write $2.5 / 30m · USD/1M Long context: input_tokens > 272000 → $4 / $15.0 input/output Regional option ×1.1 |
| Claude Sonnet 5.5max | API ↗ · Cache ↗ | Cache write $2.5 / 5m / $4 / 1h · USD/1M No long-context premium Regional option ×1.1 |
| Claude Opus 5.5max | API ↗ · Cache ↗ | Cache write $5 / 5m / $8 / 1h · USD/1M No long-context premium Regional option ×1.1 |
| Gemini 3.8 Flashhigh | API ↗ · Cache ↗ | Cache storage $0.5 / 1M / h Introductory rate through 2026-12-31; future tariff in JSON |
| Qwen3.8 Max 0902xhigh | API ↗ · Cache ↗ | Cache write Explicit $2.5 / 5m; implicit $2 · USD/1M Singapore / International; regional alternatives and unpublished cache-read rates in JSON |
| Grok 4.7xhigh | API ↗ · Cache ↗ | Long context: prompt_tokens >= 200000 in main pricing table → $4 / $12 input/output Regional option ×1.1 |
Other evaluations · Research code, literature & workflows
Original evidence and historical references are retained below. They are excluded from this reasoning index; different tasks, older models and model + agent systems are not ranked together.
Research workflows · Workflow results
Research workflowsTerminal-Bench Science 0.1 · Run dates unverified
Original source ↗Current system results for GPT-6 Astra, Opus 5.5 and others exist, but public metadata lack actual run dates and immutable snapshots. They are excluded from this reasoning index.
Current official leaderboard ↗Current-model source records · Run dates unverified
All three public system results use 70 tasks × three trials. Table creation/update dates and model-release dates are kept separately and are not treated as run dates.
| Source model + agent | Original resolve rate ± SE | Actual run date | Record date |
|---|---|---|---|
| GPT-6 Astra | 68.1% ± 3.2 | Not published | 2026-09-22 Leaderboard record created |
| Opus 5.5 | 63.3% ± 3.3 | Not published | 2026-09-23 Leaderboard record created |
| Fable 5.1 | 40.0% ± 3.4 | Not published | 2026-09-13 Leaderboard record created |
Source leaderboard updated 2026-09-23. Record creation, modification and model-release dates do not prove a rerun. Codex / Claude Code system results are not scores for the underlying model alone.
Historical references · Collapsed by default
Older models and snapshots are retained as historical evidence and excluded from current-model comparisons.
Research workflowsTerminal-Bench Science 0.1 · Three older-system results
Stanford / Harbor · Leaderboard ↗| Model / provider | Agent / effort | Original resolve rate ± SE ↑ | Model release date | Source / conditions |
|---|---|---|---|---|
| Claude Code Effort: max | 30.0% ± 3.2 | 2026-07-24 Model release date | Sources & conditionsConfigurations, metrics and dates follow the original row fields. Unpublished model snapshots and run dates remain unknown. Full protocol conditions appear below the table. Official results ↗ | |
| Codex Effort: max | 22.4% ± 2.9 | 2026-07-09 Model release date | Sources & conditionsConfigurations, metrics and dates follow the original row fields. Unpublished model snapshots and run dates remain unknown. Full protocol conditions appear below the table. Official results ↗ | |
| Claude Code Effort: max | 21.4% ± 2.8 | 2026-06-09 Model release date | Sources & conditionsConfigurations, metrics and dates follow the original row fields. Unpublished model snapshots and run dates remain unknown. Full protocol conditions appear below the table. Official results ↗ |
Scroll horizontally for configurations and sources →
These are model + agent workflow results; Claude Code and Codex configurations are kept separately. SE is the source’s standard error. Actual run and leaderboard-update dates are unlisted; page-check dates are separate. The protocol is distinct from AA / Vals mini-swe-agent. Source Cost is the entire evaluation bill and is not converted to USD/task. These results do not rank underlying models alone.
Scientific code · Code evaluations
Scientific codeSciCode · Current results unverified
Original source ↗The original leaderboard’s new-result announcements remain in 2025. Recent external reruns require separate protocol, run-date and usage-permission verification.
Original protocol & historical leaderboard ↗Original SciCode, AA v1.0.1 and SciCode-Verified remain separate. Older o3-mini / o1 / Claude 3.5 scores are excluded from the current table.
Historical references · Collapsed by default
Older models and snapshots are retained as historical evidence and excluded from current-model comparisons.
Scientific codeOriginal SciCode · Six official historical results
SciCode · Official table ↗| Model / provider | Main-problem resolve rate ↑ | Subproblems ↑ | Source / conditions |
|---|---|---|---|
| 10.8% | 33.3% | Sources & conditionsConfigurations, metrics and dates follow the original row fields. Unpublished model snapshots and run dates remain unknown. Full protocol conditions appear below the table. Official table ↗ | |
| 9.2% | 34.4% | Sources & conditionsConfigurations, metrics and dates follow the original row fields. Unpublished model snapshots and run dates remain unknown. Full protocol conditions appear below the table. Official table ↗ | |
| 9.2% | 33.0% | Sources & conditionsConfigurations, metrics and dates follow the original row fields. Unpublished model snapshots and run dates remain unknown. Full protocol conditions appear below the table. Official table ↗ | |
| 7.7% | 28.5% | Sources & conditionsConfigurations, metrics and dates follow the original row fields. Unpublished model snapshots and run dates remain unknown. Full protocol conditions appear below the table. Official table ↗ | |
| 4.6% | 28.5% | Sources & conditionsConfigurations, metrics and dates follow the original row fields. Unpublished model snapshots and run dates remain unknown. Full protocol conditions appear below the table. Official table ↗ | |
| 4.6% | 26.0% | Sources & conditionsConfigurations, metrics and dates follow the original row fields. Unpublished model snapshots and run dates remain unknown. Full protocol conditions appear below the table. Official table ↗ |
Scroll horizontally for configurations and sources →
The original table remained visible on 2026-10-08. It is not a complete current-model leaderboard for 2026. SciCode-AA v1.0.1 and SciCode-Verified remain separate; scores are neither substituted nor combined.
Literature & analysis · Evaluations
Literature & analysisAstaBench · Current-model results unverified
Original source ↗The official public test-submission directory has 85 files; the latest submission is 2026-08-24. Newer submissions use custom agents and models, without verified run evidence supporting direct comparison with fixed current GPT / Claude configurations.
Later public submissions ↗August 24 is submit_time, not generation or run date. Spring judge changes and rescoring old outputs do not establish a new-model rerun.
Historical references · Collapsed by default
Older models and snapshots are retained as historical evidence and excluded from current-model comparisons.
Literature & analysisAstaBench · Two official spring snapshots
Ai2 · Official snapshot ↗| Model + framework / provider | Version / configuration | Original Overall ↑ | LLM cost / task | Source / conditions |
|---|---|---|---|---|
| suite 1.0.0 max · adaptive thinking | 58.0% | 3.54USD/task | Sources & conditionsAi2’s official 2026-04-30 update reports these ReAct results and average LLM costs. Some judges changed and results were rescored; reevaluating old outputs is not a model rerun. This snapshot is not October’s current champion. Official snapshot ↗Later public submissions ↗ | |
| suite 1.0.0 xhigh reasoning | 52.9% | 1.61USD/task | Sources & conditionsAi2’s official 2026-04-30 update reports these ReAct results and average LLM costs. Some judges changed and results were rescored; reevaluating old outputs is not a model rerun. This snapshot is not October’s current champion. Official snapshot ↗Later public submissions ↗ |
Scroll horizontally for configurations and sources →
Only clearly sourced spring records are shown; later submissions are not exhaustively aggregated. Overall is Asta’s own metric, not a cross-benchmark research score. LLM costs cover this run’s model calls, excluding total compute, retrieval, storage and human work.
Start from your search needs
| Tool / focus | When it fits | Free allowance | Monthly cost | When to upgrade | Evidence |
|---|
No candidates for this task. Switch tasks or browse the complete directory.
Checked 2026.10.08 · Monthly billing where available; quarterly/annual-only plans use the shortest term’s monthly equivalent with actual payment stated. Annual offers are in details. Selection uses official features and pricing; historical studies are cited per row. We did not test these products ourselves.
Complete tool directory · 18 entries
More entry points for search, reading, reference management, analysis, figures and writing. Inclusion does not establish a performance ranking.
| Tool | Purpose | When to use it | Learn more |
|---|---|---|---|
| Expand related papers and research networks from seed papers | After finding a key paper, use its relationship graph to discover related work and organize collections | Features, costs & sourcesFree & upgrade Billing basis Official FAQ explicitly states USD; default-country prices are used. The FAQ’s shorthand USD10/month refers to annual billing; the main card distinguishes annual $10/month from monthly $12.50. Annual $120 is 10×12 arithmetic; checkout was not entered. Official material checked: 2026.10.08; we did not test the product. Official pricing / conditions ↗Official pricing / conditions ↗ | |
| Screen literature and build evidence tables with source locations | Define study design, sample and method fields, then extract, verify and export cross-paper tables | Features, costs & sourcesFree & upgrade Billing basis Official pages show $, without an explicit currency code or verified regional checkout currency. Since 2026-09-30, Find Papers, Extract Data and Paper Chat are combined in Research Agent; old sessions are read-only. Latest quota help says only Find Papers continues after monthly pool exhaustion, despite unlimited-chat/summary marketing. Dedicated reviews require Pro/Scale/Enterprise. Optional Premium PDF parsing is off by default and increases usage for all PDFs; figure extraction requires Scale/Enterprise with additional usage. Official material checked: 2026.10.08; we did not test the product. Official pricing / conditions ↗Official pricing / conditions ↗Official pricing / conditions ↗Official pricing / conditions ↗Official pricing / conditions ↗Official pricing / conditions ↗ | |
| Paper reading / PDF Q&A and a research-tool suite | Ask about difficult passages, methods or equations, inspect PDF section citations and build cross-paper comparisons | Features, costs & sourcesFree & upgrade Billing basis The official credit guide denominates usage in USD; reference prices are dollars. The extra 30% SCI30 annual promotion ended 2026-10-07 and is excluded from this 2026-10-08 snapshot. Annual totals use monthly equivalents×12, without checkout. Editor/Typeset plans are separate; their $8 annual-billing equivalents are not research-suite Premium prices. Official material checked: 2026.10.08; we did not test the product. Official pricing / conditions ↗Official pricing / conditions ↗Official pricing / conditions ↗ | |
| Search and synthesize paper evidence around a research question | Explore relevant papers, Study Snapshots and findings across studies for a question | Features, costs & sourcesFree & upgrade Billing basis Official help shows $, without an explicit currency code or regional checkout currency. Pro messages, Deep reviews and API/MCP calls have separate allowances, not interchangeable tasks. Team plans use custom quotes, not individual Pro annual rates. Official material checked: 2026.10.08; we did not test the product. Official pricing / conditions ↗Official pricing / conditions ↗ | |
| Map paper similarity using co-citation and bibliographic coupling | Start with a representative paper and inspect clusters, Prior Works and Derivative Works | Limitations, costs & sourcesScope & limitations Costs & availability Provider Algorithmic graph discovery, not automatically classified as an LLM tool. Features are drawn from official material; no independent product test or performance ranking. Material checked: 2026.10.08 Graph methods & team ↗Plans & intended use ↗ | |
| Visualize literature relationships and monitor new papers | Maintain literature maps, expand related work with Discover and track updates with Monitor | Features, costs & sourcesFree & upgrade Billing basis Website cards show $, without a confirmed currency code. Country purchasing-power prices require the account. Older extraction mixed desktop two-map and mobile one-map copy. Current desktop and cancellation help state two maps, while old responsive copy still conflicts. Annual totals use current monthly equivalents×12; $120 also appears in older copy, without checkout. Collaboration is a Team feature, absent from Pro. Official material checked: 2026.10.08; we did not test the product. Official pricing / conditions ↗Official pricing / conditions ↗Official pricing / conditions ↗ | |
| Free academic search, TLDRs and influential-citation signals | Filter by topic, author and date, browse short summaries and citations, and build an initial bibliography | Limitations, costs & sourcesScope & limitations Costs & availability Provider AI features/product; underlying models were not uniformly verified. Official feature information, without independent product tests or performance ranking. Material checked: 2026.10.08 Search, TLDR, citations & Reader ↗Ai2 provider & free service ↗ | |
| Inspect supporting, contrasting and mentioning citation contexts | When reviewing or building an argument, inspect how later studies cite or question a key claim | Features, costs & sourcesFree & upgrade Billing basis Website cards show $, without a confirmed currency code/local tax; checkout was not entered. October corrections removed the old 25 free MCP credits/month and first ten account-free calls. The seven-day trial renews unless cancelled. Pro’s limited $60 API-credit promotion lacks confirmed duration/renewal and is not permanent monthly allowance. Annual totals use monthly equivalents×12, consistent with Save96/Save120; checkout was not entered. Official material checked: 2026.10.08; we did not test the product. Official pricing / conditions ↗Official pricing / conditions ↗Official pricing / conditions ↗Official pricing / conditions ↗ | |
| Collect, annotate and manage references, and generate formatted citations | Archive, deduplicate and annotate papers, then insert citations in Word/LibreOffice/Google Docs | Limitations, costs & sourcesScope & limitations Costs & availability Provider Core reference management is non-generative; third-party AI plugins are separate. Official feature information, without independent product tests or performance ranking. Material checked: 2026.10.08 Core features ↗Free & paid attachment storage ↗Digital Scholar & open-source identity ↗ | |
| Organize team screening, conflict resolution and review workflows | Import database results, deduplicate, screen titles/abstracts and full text by eligibility criteria, and record decisions | Features, costs & sourcesFree & upgrade Billing basis Website shows $; US App Store dollar prices do not establish regional website currency. Essential website annual total is $59.88; App Store $59.99 is another channel. Advanced’s $8.33 monthly equivalent is rounded; annual total remains $99.99, not $99.96. Business $41.67 is per-license monthly equivalent; $2,500 is the minimum five-seat annual total. An Advanced link carries quantity=2 while the calculator shows one; minimum checkout seats remain unverified and should be checked. Official material checked: 2026.10.08; we did not test the product. Official pricing / conditions ↗Official pricing / conditions ↗ | |
| Multistep Web research and reports with sources | Explore concepts, institutions and paper entry points for a new topic, then confirm against original papers | Features, costs & sourcesFree & upgrade Billing basis Public pages show $, while billing may use regional currency; references do not establish checkout currency. Max annual billing is Web-only. Pro 200/12=16.666…; $16.67 is approximate, as are Max equivalents. Consumer subscriptions exclude programmatic API fees; Computer credits differ from Research allowances. Help-center weekly limits qualify unlimited-Pro marketing, with numerical caps unreported. Official material checked: 2026.10.08; we did not test the product. Official pricing / conditions ↗Official pricing / conditions ↗Official pricing / conditions ↗Official pricing / conditions ↗Official pricing / conditions ↗ | |
| Connect scientific literature search, synthesis and data analysis | Describe a research question, find and summarize papers, then explore data analysis and visualization | Limitations, costs & sourcesScope & limitations Costs & availability Provider AI features/product; underlying models were not uniformly verified. Official feature information, without independent product tests or performance ranking. Material checked: 2026.10.08 Current Find/Summarize/Analyze capabilities ↗Product & Preview distinction ↗Official trial entry, dynamic shell only ↗ | |
| Parse complex documents into structured Markdown/JSON | Batch-parse research PDFs, tables and equations locally for downstream search or extraction | Limitations, costs & sourcesScope & limitations Costs & availability Provider Configurable parsing/OCR/VLM pipelines, not one LLM product. Official feature information, without independent tests or performance ranking. Material checked: 2026.10.08 Capabilities, models, deployment & license ↗Optional GraniteDocling model card ↗ | |
| Hosted document conversion and schema extraction, plus local tools | Convert scanned/multicolumn PDFs to Markdown/JSON or extract a specified field schema | Limitations, costs & sourcesScope & limitations Costs & availability Provider OCR/VLM with optional LLM pipelines; hosted/local modes differ. Official features, without independent tests or performance ranking. Material checked: 2026.10.08 Hosted Convert/Extract processors ↗Current free usage, metered & team/enterprise fees ↗Marker modes & separate code/weight licenses ↗Chandra implementation & model license ↗ | |
| Digitize curves/points from paper figures | When original data are absent, calibrate axes, digitize curves and export data for further analysis | Limitations, costs & sourcesScope & limitations Costs & availability Provider Core computer-vision-assisted digitization; AI Assist is a separate proprietary cloud feature. Official information, without independent tests or performance ranking. Material checked: 2026.10.08 Official code, free service & proprietary AI Assist scope ↗Official documentation ↗Exports & project files ↗ | |
| Keep code, data, figures and explanations in notebooks | Clean experimental data, run statistics and retain reproducible computations using Python/R/Julia | Limitations, costs & sourcesScope & limitations Costs & availability Provider Core non-generative computing environment; optional AI extensions need separate verification. Official features, without independent tests or performance ranking. Material checked: 2026.10.08 Interactive environments, languages & free/open-source service ↗ | |
| Create diagrams with scientific icons/templates and editable AI elements | Draw experimental workflows, mechanisms or graphical abstracts, then export consistent layouts for submission | Limitations, costs & sourcesScope & limitations Costs & availability Provider Scientific illustration environment with Leo/editable AI features, not an evidence-correctness judge. Official features, without independent tests or performance ranking. Material checked: 2026.10.08 Icons/templates & Leo editable AI ↗Plans & Free’s lack of publication/commercial rights ↗ | |
| Collaborative LaTeX writing, templates and online compilation | Collaborate with journal templates on manuscripts, equations, figures and references, using online comments/revisions | Limitations, costs & sourcesScope & limitations Costs & availability Provider Core LaTeX writing/compilation platform; AI features assist rather than judge evidence. Official features, without independent tests or performance ranking. Material checked: 2026.10.08 Editing, templates & collaboration ↗Current Free/paid plans & AI allowances ↗Provider & Digital Science relationship ↗Current operating legal entity ↗ |
Scroll horizontally for the full directory →
No matching entries. Try another model, provider or configuration.
Capability & API pricing sources
Epoch official data ↗ · The two original benchmark scores and 60 / 40 weights are unchanged. Source package updated 2026.10.08; actual run starts, configurations and record IDs appear in Data & pricing sources. Epoch’s internal score data are used under CC BY 4.0, which does not cover questions or external data. We did not independently rerun the benchmarks.
API prices come from official provider documentation, checked 2026.10.08. Fixed usage is 750,000 uncached input + 250,000 output tokens, including reasoning. Model-specific conditions appear in Data & pricing sources. Commercial API tariffs are not benchmark invoices.
Tool inclusion & scope
Five research tasks compare candidate features, free allowances and relevant paid tiers. Current pricing and public evidence for eight tools were checked. The 18-entry directory retains other entry points. Tools do not inherit their underlying model scores.
This is a static 2026.10.08 snapshot. Research code, literature and workflow evidence is retained under Other evaluations and excluded from this index.