Page Comparison

...

Library components and entry points can be placed in openff.benchmark.geometry_optimizationsoptimization.

openff-cli

Could introduce an entrypoint in this package for distribution. (optional, and for later)

...

Command-line interface executable from any shell preferable.

Identifier assignment

Implemented in openff.benchmark.utils. Can be as simple as a function that takes as input group/company code (3-letter), all molecules (with predefined conformers, if present). Will then produce a mapping of identifiers to molecule objects as: COM-XXXXX-YY

• three-letter company code (COM)

• molecule-index (XXXXX)

• numerical conformer-index (YY); 01, 02, 03,…

Note that with this approach, each molecule submitted in the dataset will have exactly one conformer. We would not be stacking multiple conformers into each Molecule.

Conformer generation

For molecules with fewer than 10 conformers predefined, additional conformers will be generated to give a total of 10. This can already be done with openforcefield.topology.Molecule.generate_conformers.

We will need the mapping from Identifier assignment after this, so it likely makes sense to switch the order of these workflow components.

Remaining questions

1. Do we care about easily distinguishing which conformers were provided vs. generated after the fact?

Parameterization of molecules

Parameterization of molecules will be performed with e.g.:

from openforcefield.typing.engines.smirnoff import ForceField

# Load the OpenFF "Parsley" force field
forcefield = ForceField('openff-1.0.0.offxml')

# Parametrize the topology and return parameters used
off_topology = molecule.to_topology()
molecule_labels = forcefield.label_molecules(off_topology)

The labels can then be fed directly to the Forcefield coverage report generator. An entry-point wrapping this and the coverage report can be placed in openff.benchmark.parameterization.

This step should be performed with each forcefield we are benchmarking.

Molecules that fail this step should be noted and left out of the energy minimization submission. We still want these in the coverage report that consumes the output of this step.

Forcefield coverage report

A function taking multiple sets of molecule labels from Parameterization of molecules to generate coverage reports should go into openff.benchmark.parameterization. This will the produce a report giving the counts for each parameter in the forcefield, aggregated over the molecules provided.

Although possible to provide a report for each molecule, to mitigate privacy concerns on the molecules used, it is recommended to generate a single report for the whole dataset.

Remaining questions

1. Should we enforce reports be aggregated? How Can we show how possible it is it to back-calculate a molecular structure based on the parameters used to to parameterize it?

Energy minimization with Psi4 (QM), OpenMM (MM)

...

1. High-throughput (primary)

   • QCSubmit->QCFractal(->QCEngine->GeomeTRIC->QCEngine->Psi4/OpenMM)

   • output extraction executable at any time for pulling available data

   • need error cycling process

2. High-throughput debug approach (secondary)

   • Trevor's local optimization executor

     • add this to QCSubmit; generally usable for OpenFF QCArchive users in debugging

   • components shared with (3)

   • GeomeTRIC->QCEngine->Psi4/OpenMM

   • output still usable for reporting

3. Fully-local execution (alternative)

   • Like Horton's local TorsionDrive script, minus QCFractal execution if possible

   • components shared with (2)

   • GeomeTRIC->QCEngine->Psi4/OpenMM

   • output still usable for reporting

In principle, (2) and (3) could be served via the same entrypoint.
(1) would make use of QCFractal with a persistent server to handle most of the compute orchestration.

These approaches should be given entry-points in openff.benchmark.geometry_optimization.

Once errors fail to clear in (1) and cannot be cleared in (2) or (3), these should be noted as failures in a way consumable by Analysis and report generation.

Analysis and report generation

Outputs produced in Energy minimization with Psi4 (QM), OpenMM (MM) should be directly consumable via an entry-point in openff.benchmark.geometry_optimization. We need the following included for each ID from Identifier assignment:

1. Relative energies (E_MM - E_QM)

2. Geometry comparison (RMSD or TFD, MM vs. QM)

Existing implementations should be drawn from benchmarkff. Where implementations are dependent on OpenEye Toolkit, alternatives must be put in place.

Deployment

A document describing compute stack installation, server stand up, and worker submission to queueing systems in use will need to be written and shared. This should include an upgrade pathway for the compute stack. This will likely draw on existing approaches for public QCArchive production compute.