Page Comparison

JH – Worked mostly on QCSubmit. Working on speed and compression with TG.

JH – Still having convergence problems with ANI. Getting data together to submit to Adrian Roitberg

JH – Using XTB at the moment. Has high element/chemistry coverage and speed. Running on JACS torsion fragments. Considering using this as a supported option for bespoke workflow, so currently benchmarking time/performance.

JH – Thinking about benchmarking for bespoke workflow. Will generate lots of conformers at high temperature. Then will rank energies before and after fitting torsions.

JH – Another option for testing workflow would be to benchmark against existing JACS torsiondrives. However this comparison won’t include ANI.

JW – What benchmarking strategies are available? QM minimum comparison, high-temp conformer comparison, phys props for small molecules, host-guest systems.

• DM – host molecules torsions have been identified as a problem. So refitting might improve performance.

  • Hosts are “acd” and “bcd” –

    Github link macro
    link https://github.com/slochower/host-guest-benchmarks/tree/master/taproom/systems/bcd

• JH – Could also test against freesolv

  • DM – I don’t think torsions would make a big difference on these.

Benchmarking infrastructure

• (General) We probably can NOT use QCA/QCEngine as the backend for the industry benchmarking, since it won’t allow input of custom OFFXMLs.

Pre-release benchmarking

• JW – If this can be run in less than ~an hour, we can automate it using GitHub actions.

• DM – It’d be good for this to be available for manual runs, so that individuals iterating on new FFs can quickly evaluate their changes.

JW – JH, please (re)send the code where you deduplicate redundant SMARTS

PB – To confirm, new parameter testing is currently done by manually re-running benchmarking.

• DM – Correct. For released FFs, we can use QCArchive/QCEngine. For unreleased FFs, we use benchmarkFF.

• (General) Added some thoughts to upcoming FF release meeting agenda item 2020-10-15 Meeting notes https://openforcefield.atlassian.net/wiki/spaces/MEET/pages/761856001

JA – Are we thinking of having a fixed test set?

• DM – It will probably be fixed for set periods of time, but may be improved/updated roughly annually so that we can ensure it’s meaningful and not overtrain

• JA – Can we do cross-validation?

• JW – Probably not, current fits take about 2 weeks, so CV folds would make FF fits take several months.