Introducing the Scientific Superintelligence Index
A collection of scientific environments for measuring how AI researchers design methods, run experiments, and produce verifiable improvements.
Can an AI researcher build a better scientific model—and show the experiments that made it better?
The Scientific Superintelligence Index (SSI) starts with this question. We bring scientific problems into executable research environments where an AI researcher can develop a method, test it, learn from feedback, and submit a candidate for independent evaluation.
The aim is to make progress toward scientific superintelligence observable: in the quality of the resulting scientific models, the resources used to develop them, and the evidence behind each improvement.
1. The concept: evaluating AI as a researcher
Scientific research involves choices that a final score alone cannot explain. Which representation should a model learn? Which experiment should come next? Was an apparent improvement reproducible? What happened when an idea failed?
SSI makes these choices part of the evaluation. The AI researcher is the system conducting the experiments; the scientific model is the artifact it develops. An environment connects the researcher’s decisions to a specific scientific objective and a measurable outcome.
Each environment page therefore presents a researcher leaderboard, reference methods, and a research trajectory. The trajectory records the best retained result as cumulative research time increases. Failed attempts remain visible, and a promising intermediate experiment becomes a new best only when it is retained. Where compute measurements are available, a separate frontier shows the relationship between resources and performance.
The index is organized by scientific environment. Biomolecular structure quality, weather skill, and crystal stability have different meanings, so their raw scores should be interpreted within their own evaluation protocols. This first version presents those protocols and results alongside the research process, without claiming a single aggregate measure of scientific intelligence.
2. What defines an SSI environment?
An SSI environment is an executable scientific research problem with a clear objective, accessible development resources, and an evaluation that can independently assess a submitted method.
A useful environment brings together five elements:
- A scientific objective. A question that gives an improvement meaning: more accurate molecular structures, more reliable forecasts, or better candidate materials.
- Data and research tools. Development inputs, reference implementations, and the tools needed to train or experiment, with access conditions documented.
- A submission contract. A reproducible package containing the method and its inference interface, plus weights when required.
- Independent evaluation. A verifier that runs the submitted package on the environment’s evaluation inputs and reports scientific metrics. Visibility and access rules are specified for each task.
- A research record. Experiments, retained candidates, elapsed research time, and compute measurements where available, so readers can inspect how a result was reached.
The collection currently contains 26 environment pages, including tasks still in development. They span biomolecular modeling, materials and chemistry, physical systems, and forecasting. Each task retains its own data requirements and scientific scoring rules.
3. Three representative environments and their results
The examples below illustrate the range of scientific work an environment can support. The numerical results come from the project’s documented reference and integration records, captured in the website’s October 2, 2026 source snapshot. They establish reproducible starting points; they are not results from the simulated researcher leaderboards currently used to preview the site.
FoldBench: one model for molecular structures and interactions
The FoldBench environment asks a researcher to train one biomolecular folding model across nine tracks, covering proteins and their molecular partners. Its full evaluation suite contains 1,522 requests, so a candidate must work across several kinds of structures and interactions.
The integration record establishes a from-scratch, 49.2-million-parameter ESMFold2-graph starter trained for one epoch, using six H100 GPUs for 3.7 hours including a resume. The starter achieved 5.7% PPI DockQ pass and 1.7% antibody–antigen pass. Its monomer TM-score was 0.53, while the reported monomer lDDT was 0.000 because the scorer’s stereochemistry checks zeroed the score. This makes structural validity part of the research objective alongside geometric similarity.
A frozen ESMFold2-Fast reference achieved monomer lDDT 0.822, PPI DockQ pass 55%, and antibody–antigen pass 29%. These reference results provide context for the starter; the frozen model and the from-scratch training recipe are distinct conditions. The task specification and integration record document the evaluator, training setup, and reproduction fixes.
Weather forecasting: a distribution of possible futures
The weather environment asks a researcher to train a global probabilistic weather model from scratch. Its target is a 15-day ensemble forecast at 0.25° resolution, assessed through forecast accuracy, extremes, and tropical cyclones.
The reference recipe uses a 183.8-million-parameter WeatherNext2 model and an eleven-stage autoregressive training curriculum. The integration record reports a full training run of 13.1 hours on four H100 GPUs and an independently evaluated reward of 0.674377, with normalized CRPS of 0.482850. The author’s reported rewards were 0.680651 and 0.675805; the reproduction fell within the reported run-to-run spread.
This gives future research a concrete comparison point: improvements need to survive the same forecast protocol and be interpreted against existing variability. See the weather task and reproduction evidence.
Crystal generation: evaluating what a model invents
The crystal generation environment asks a researcher to generate candidate crystal structures from MP-20. Candidates undergo fixed UMA relaxation before evaluation with the LeMat SUN and MetaSUN protocol.
The reference is a 12.4-million-parameter OMatG CSPNet trained for 2,000 epochs. Its documented SUN + MetaSUN result was 21.04%, compared with 20.8% in reproduction and 10.6% for the statistical null. The reference training run used six H100 GPUs for 5.4 hours.
The stricter SUN result was approximately 0.6% in both runs. Reporting that alongside the broader measure matters: different criteria reveal different properties of the generated candidates. The integration evidence also records evaluation issues corrected during reproduction.
| Environment | Representative recorded result | What the comparison establishes |
|---|---|---|
| FoldBench | Starter PPI pass 5.7%; frozen reference 55% | Multi-track folding with structural validity checks and distinct training conditions |
| Weather forecasting | Reward 0.674377 in reproduction | A probabilistic forecast reference within reported run-to-run spread |
| Crystal generation | SUN + MetaSUN 20.8% in reproduction | A generative reference, with stricter SUN reported separately |
These environments make different scientific outcomes measurable through the same research pattern: develop a method, preserve the experiment trail, and verify the resulting artifact. As real researcher runs are added, SSI will let readers inspect both the achieved result and the path taken to reach it.
Explore the environment collection, or see how to contribute an environment or research run.