Page Comparison

...

The labels can then be fed directly to the Forcefield coverage report generator. An entry-point wrapping this and the coverage report can be placed in openff.benchmark.parameterization.

This step should be performed with each forcefield we are benchmarking.

Molecules that fail this step should be noted and left out of the energy minimization submission. We still want these in the coverage report that consumes the output of this step.

Forcefield coverage report

...

1. High-throughput (primary)

   • QCSubmit->QCFractal(->QCEngine->GeomeTRIC->QCEngine->Psi4/OpenMM)

   • output extraction executable at any time for pulling available data

   • need error cycling process

2. High-throughput debug approach (secondary)

   • Trevor's local optimization executor

     • add this to QCSubmit; generally usable for OpenFF QCArchive users in debugging

   • components shared with (3)

   • GeomeTRIC->QCEngine->Psi4/OpenMM

   • output still usable for reporting

3. Fully-local execution (alternative)

   • Like Horton's local TorsionDrive script, minus QCFractal execution if possible

   • components shared with (2)

   • GeomeTRIC->QCEngine->Psi4/OpenMM

   • output still usable for reporting

...

These approaches should be given entry-points in openff.benchmark.geometry_optimization.

Once errors fail to clear in (1) and cannot be cleared in (2) or (3), these should be noted as failures in a way consumable by Analysis and report generation.

Analysis and report generation

...

Existing implementations should be drawn from benchmarkff. Where implementations are dependent on OpenEye Toolkit, alternatives must be put in place.

Deployment

A document describing compute stack installation, server stand up, and worker submission to queueing systems in use will need to be written and shared. This should include an upgrade pathway for the compute stack. This will likely draw on existing approaches for public QCArchive production compute.