Molecular Pretraining Transfer

A molecule’s shape affects how its electrons behave and how it interacts with its surroundings. But predicting a quantum property is different from predicting solubility or how quickly a biological assay removes a compound. This task asks whether one learned representation can help across these different scientific settings.

The model first learns from molecular structures without the downstream property labels. It is then adapted to each prediction task using the same limited training recipe. This makes the comparison about reusable knowledge, rather than giving each endpoint an independently optimized model.

The permitted pretraining corpus contains about 126 million molecular records, each with atom identities and a three-dimensional arrangement. Exact molecular connectivity matches to evaluation molecules are removed. Similar chemical scaffolds can still occur, so the boundary does not guarantee wholly unfamiliar chemistry.

The evaluation covers 24 tasks and 26 scalar properties. QM9 and COMPAS concern quantum chemistry, including orbital energies and dipole moments. ExpansionRx and ASAP concern properties such as solubility, permeability and clearance—the rate of removal in an assay system.

Geometry inputs also differ. Quantum examples use calculated structures; biological endpoints use generated conformers, or different arrangements of the same molecule. Missing assay measurements are absent labels, not zero-valued properties.

Each adaptation uses at most 5,000 training molecules and fifteen passes through that data. Validation selects the model, and the evaluator measures error on the test molecules. Predictions combine the prescribed geometry inputs.

Relative absolute error divides prediction error by the spread of the test targets. In a teaching example, targets 1, 2, 3 and 4 have mean 2.5 and average distance 1 from that mean. Predictions that are each off by 0.5 have relative error 0.5. Predicting the mean for every molecule gives relative error 1.

Errors are averaged within each of the four task groups; there is no official single overall score. Ranking is also measured, because a model can order molecules correctly while systematically overestimating every value.

Rotating a molecule should not change a scalar property such as its energy. Twisting a bond can change distances and interactions. The interactive chain demonstrates this distinction: pair distances ignore whole-molecule rotation but detect changes in internal shape.

The Uni-Mol2 reference combines an atom representation with a representation of relationships between atom pairs. Training hides some atom identities and perturbs positions. Recovering the missing identities teaches chemical context; restoring coordinates and distances teaches geometric consistency.

After pretraining, each property task receives its own predictor and updates the encoder as well. The same short adaptation procedure is used throughout. This tests transfer with limited labels, not the best result obtainable through unlimited task-specific tuning.

The shortened from-scratch training improves all four groups over random initialization. Relative error falls by about 56% on QM9, 75% on COMPAS, 31% on ExpansionRx and 52% on ASAP. The benefit extends beyond quantum chemistry, although its size varies.

Compared with released pretrained weights, the reference is slightly better on QM9 and ASAP but worse on COMPAS and ExpansionRx. The released model is an external comparison, not an allowed starting point. Differences in training history prevent attributing the gap to one ingredient.

Individual endpoints also expose limitations: on one ASAP solubility endpoint, relative error is about 0.78 but rank correlation only 0.14. Reasonable average values need not provide a useful ordering of compounds. Multiple training seeds and experiments isolating the separate learning objectives are not reported.

Removing one learning objective at a time would reveal whether atom recovery and geometry recovery help different endpoints. Varying conformer diversity could test whether flexible molecules benefit most. For small biological datasets, repeated runs are especially useful: a modest gain may reflect adaptation variability rather than better molecular understanding.