Sequential Experimental Optimization

A chemist searching for a reaction recipe cannot test every combination of temperature, solvent and catalyst. Each result changes what seems promising and what remains uncertain. The next experiment must balance exploiting a good lead with exploring an unfamiliar region.

The goal here is to learn that decision process across scientific families. Finding several strong candidates matters: the single best recipe may be impractical for reasons outside the measured objective. The benchmark rewards discovery of a valuable set, although it does not itself model laboratory cost or feasibility.

The data contain 23 landscapes: tables of candidate experiments and measured outcomes. A campaign reveals only the outcomes it chooses to measure. Twelve tasks support training, seven support validation and four unseen families test transfer.

The test families span fullerene synthesis, a palladium-catalyzed reaction, photoswitch molecules and crystallization. Some candidates are recipes, others molecular structures. A model must connect useful scientific information across these input types without seeing the hidden test outcomes. Because experiments come from fixed tables, this simulation omits many laboratory complications, such as instrument drift and unlisted recipes.

Each campaign begins with ten examples from the lower-performing half of a landscape and allows fifty further measurements. After each choice, its outcome becomes available for the next decision. The main score is the fraction of top-5% candidates discovered, averaged across twenty starting conditions per family and equally across the four families.

This differs from regret, which asks how close the best discovery is to the optimum. Finding one excellent molecule can give low regret while leaving most useful alternatives unexplored. Tied outcomes also matter: the set above the top-5% threshold can contain much more than 5% of the candidates.

A Gaussian process predicts both a candidate’s likely outcome and uncertainty about that prediction. An acquisition rule uses those estimates to decide which measurement is most valuable. Domain-specific features can describe molecular structure or reaction ingredients; text-based methods instead learn from descriptions.

The GOLLuM reference adapts text-derived features using the outcomes revealed so far, then fits a Gaussian process to them. BoChemian keeps its text features fixed. Any adaptation must use only the current campaign’s observations and reset before the next campaign.

Across the four families, the local GOLLuM reference discovers about 49% of the top set, compared with 34% for the domain-feature Gaussian process and 12% for random search. But the average hides a reversal: the domain-feature method does better on crystallization, while GOLLuM’s large advantage on photoswitches drives much of its overall gain.

Crystallization has 357 candidates tied in the top set. With fifty new measurements, even perfect choices can cover only about 14%. Random search finds an optimum in every recorded campaign yet covers only about 6% of that set. A small coverage percentage here does not mean it failed to find a good crystal recipe.

These comparisons use particular local fitting settings. They do not isolate the cause of the gain: representations and fitting procedures change together. The displayed means also do not show uncertainty across starting conditions.

A useful next comparison would keep candidate features and fitting time fixed while changing only the decision rule. Does uncertainty help discover more strong alternatives early in the campaign? Reporting discovery curves and each family separately would reveal whether a method learns a transferable search strategy or mainly excels in one kind of chemistry.