THE EVIDENCE / MULTIMODAL
Multimodal
benchmarks.
Understanding or producing information across modalities. Results are grouped by benchmark and unit. Different tests are never merged into a raw leaderboard.
01 / RESULTS
Published results
| Benchmark | Version | Method | Configuration | Model | Result | Test date | Verified | Source type | Source |
|---|---|---|---|---|---|---|---|---|---|
| Chartography (no tools; reported best effort) | Chartography (no tools; reported best effort) | Publisher-reported configuration; see source | — | claude-sonnet-5-5 | 61.6 % | 2026-09-28 | 2026-09-29 | provider first party | Source ↗ |
| Chartography (no tools; reported best effort) | Chartography (no tools; reported best effort) | Publisher-reported configuration; see source | — | claude-opus-5-5 | 64.4 % | 2026-09-28 | 2026-09-29 | provider first party | Source ↗ |
| Chartography (no tools; reported best effort) | Chartography (no tools; reported best effort) | Publisher-reported configuration; see source | — | gpt-6-sol | 53.6 % | 2026-09-28 | 2026-09-29 | provider first party | Source ↗ |
| MMMU-Pro Mercor public leaderboard | MMMU-Pro Mercor public set | Mercor independent evaluation | 1730 public tasks; Pass@1; model effort recorded separately · Max | claude-opus-5-5 | 87.4 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MMMU-Pro Mercor public leaderboard | MMMU-Pro Mercor public set | Mercor independent evaluation | 1730 public tasks; Pass@1; model effort recorded separately · xHigh | gpt-6-astra | 86.8 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MMMU-Pro Mercor public leaderboard | MMMU-Pro Mercor public set | Mercor independent evaluation | 1730 public tasks; Pass@1; model effort recorded separately · High | gemini-3-7-flash | 85.2 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MMMU-Pro Mercor public leaderboard | MMMU-Pro Mercor public set | Mercor independent evaluation | 1730 public tasks; Pass@1; model effort recorded separately · High | gemini-3-5-flash | 84.7 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MMMU-Pro Mercor public leaderboard | MMMU-Pro Mercor public set | Mercor independent evaluation | 1730 public tasks; Pass@1; model effort recorded separately · High | gemini-3-1-pro-preview | 84.3 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MMMU-Pro Mercor public leaderboard | MMMU-Pro Mercor public set | Mercor independent evaluation | 1730 public tasks; Pass@1; model effort recorded separately · High | claude-fable-5-1 | 83.9 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MMMU-Pro Mercor public leaderboard | MMMU-Pro Mercor public set | Mercor independent evaluation | 1730 public tasks; Pass@1; model effort recorded separately · Max | gpt-5.6-sol | 83.6 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MMMU-Pro Mercor public leaderboard | MMMU-Pro Mercor public set | Mercor independent evaluation | 1730 public tasks; Pass@1; model effort recorded separately · Max | gpt-6-sol | 83 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MMMU-Pro Mercor public leaderboard | MMMU-Pro Mercor public set | Mercor independent evaluation | 1730 public tasks; Pass@1; model effort recorded separately · High | grok-4-5 | 79 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MMMU-Pro Mercor public leaderboard | MMMU-Pro Mercor public set | Mercor independent evaluation | 1730 public tasks; Pass@1; model effort recorded separately · Max | deepseek-v4-1-flash | 76.6 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor public leaderboard | MedXpertQA MM Mercor 200 public tasks | Mercor independent evaluation | 200 public tasks; Pass@1; model effort recorded separately · Max | gpt-6-astra | 86.8 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor public leaderboard | MedXpertQA MM Mercor 200 public tasks | Mercor independent evaluation | 200 public tasks; Pass@1; model effort recorded separately · Max | claude-opus-5-5 | 85.7 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor public leaderboard | MedXpertQA MM Mercor 200 public tasks | Mercor independent evaluation | 200 public tasks; Pass@1; model effort recorded separately · High | gemini-3-8-flash | 84.7 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor public leaderboard | MedXpertQA MM Mercor 200 public tasks | Mercor independent evaluation | 200 public tasks; Pass@1; model effort recorded separately · High | gemini-3-5-flash | 83.5 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor public leaderboard | MedXpertQA MM Mercor 200 public tasks | Mercor independent evaluation | 200 public tasks; Pass@1; model effort recorded separately · High | gemini-3-7-flash | 83.5 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor public leaderboard | MedXpertQA MM Mercor 200 public tasks | Mercor independent evaluation | 200 public tasks; Pass@1; model effort recorded separately · Max | gpt-6-sol | 81.3 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor public leaderboard | MedXpertQA MM Mercor 200 public tasks | Mercor independent evaluation | 200 public tasks; Pass@1; model effort recorded separately · High | claude-fable-5-1 | 79.7 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor public leaderboard | MedXpertQA MM Mercor 200 public tasks | Mercor independent evaluation | 200 public tasks; Pass@1; model effort recorded separately · Max | gpt-5.6-sol | 79.5 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor public leaderboard | MedXpertQA MM Mercor 200 public tasks | Mercor independent evaluation | 200 public tasks; Pass@1; model effort recorded separately · xHigh | grok-4-6 | 75.2 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor public leaderboard | MedXpertQA MM Mercor 200 public tasks | Mercor independent evaluation | 200 public tasks; Pass@1; model effort recorded separately · Max | deepseek-v4-1-flash | 63.5 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor public leaderboard | MedXpertQA MM Mercor 200 public tasks | Mercor independent evaluation | 200 public tasks; Pass@1; model effort recorded separately · Max | gpt-6.1-sol | 85.7 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor 51 held-out tasks | MedXpertQA MM Mercor Extended 51 held-out tasks | Mercor independent evaluation | 51 held-out tasks; Pass@1; model effort recorded separately · Max | claude-opus-5-5 | 71.2 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor 51 held-out tasks | MedXpertQA MM Mercor Extended 51 held-out tasks | Mercor independent evaluation | 51 held-out tasks; Pass@1; model effort recorded separately · Max | gpt-6-sol | 68 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor 51 held-out tasks | MedXpertQA MM Mercor Extended 51 held-out tasks | Mercor independent evaluation | 51 held-out tasks; Pass@1; model effort recorded separately · High | claude-fable-5-1 | 66 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor 51 held-out tasks | MedXpertQA MM Mercor Extended 51 held-out tasks | Mercor independent evaluation | 51 held-out tasks; Pass@1; model effort recorded separately · Max | gpt-5.6-sol | 64.7 % | not published | 2026-09-30 | independent lab | Source ↗ |
| MedXpertQA MM Mercor 51 held-out tasks | MedXpertQA MM Mercor Extended 51 held-out tasks | Mercor independent evaluation | 51 held-out tasks; Pass@1; model effort recorded separately · Max | gpt-6.1-sol | 64.7 % | not published | 2026-09-30 | independent lab | Source ↗ |
Limitations: Image, audio and video tests are distinct and are not pooled as raw results. Publication methods, prompts, sampling and model versions may differ. Check each cited source before interpreting a result.