2021-09-29 Industry benchmarks meeting notes

OPLS executor

David Hahn

• DD – I saw email from KMeier.

• DH – There’s a quick fix that I sent to TFox and LD’Amore.

• DH - there’s a quick fix that Lorenzo, TFox has; not in repo though

  • Schrodinger changed how parameters are stores/merged between 21.1 and the most recent versions

  • easy to fix in the CLI; can just use one-liner schrodinger command. This would avoid the need to make a new release.

  • DD – One-liner command would be a good workaround.

  • JW - is there any chance folks will get errors they won’t understand

  • LD - it’s effectively swapping out a command for the corresponding schrodinger command

  • DH – there is a risk that folks don’t actually run this command, and end up without custom params to run calculations with; this was already a problem, however

  • DH – Cleanest solution would use different code depending on which Schrodinger version is installed.

    • JW – Here’s how we check for the AmberTools version in the OFF toolkit - We know that it can be printed to a terminal by running antechamber -L so we parse the output of that.

  • DH – will put together announcement to all partners, also update the protocol with a note on this workaround

Updates from team

• JW – nothing on benchmarking this week; did more long-range effort planning

  • don’t think I can get to the benchmarking effort until Jan/Feb 2022

  • DH – think it’s absolutely fine; realistically I don’t think the manuscript is done far before Christmas

  • JW – one big goal is to take what we’ve done and put into a more sustainable/robust form for a season 2

    • Other big goal is to make this part of the FF validation step – want to get it plugged into FF release process, as opposed to the current benchmark/validation step that we currently do with ForceBalance.

• DH – no other updates on benchmarking; perhaps related, protein-ligand benchmarking

  • DD – I would like some help from this group on that topic

• DD – F@H planning

  • DD – DM asked me which engines/permutations of workflows we’ll support. Also whether this is our job, or OpenFE’s

    • DD – For engines, I’ll be using pmx/Gromacs, and perses/OpenMM

    • LD – Is Perses mature?

    • DD – It’s still shifting a bit, but I have close contact with their developers

  • Who’s job is it?

    • JW – I suspect that, if OpenFF doesn’t do this, it will be OpenFE’s job. But of OpenFF does handle it, then it won’t be OpenFE’s job

    • DH – My understanding of OpenFE is that it’ll do whatever needs to be done to get free energy calcs running

    • DD – Our jurisdiction is to harden automation/infrastructure for a existing methods for switching calculations (perses and pmx)

    • DH – Similar to geometry benchmark, our job is to fill in the gaps and make it more reliable. OpenFE would ideally do stuff like this

    • JW – I’m concerned because this will make a really useful piece of automation that people will want to run locally on their own projects, but it won’t be structured for direct user access, and we won’t have time budgeted for user support.

    • DD – The only endpoint of this will be submission to F@H - It won’t be user-runnable. The outputs won’t be formatted for local/offline execution. This will be reinforced by the need for all input data to be public.

  • What does our interaction with OpenFE on this look like?

    • JW – Will OpenFE software import components from our F@H automation? Or vice versa? Who should support/develop shared components? If there’s no clear path for continued support/development, should we forbid importing/reuse of unsupported components?

    • DD – This will need continued maintenance, kinda like MolSSI with QCArchive.

  • (Question from DMobley) – How do we keep this from becoming a perpetual “fix all our problems with FE calculations”, which isn’t openFF’s job (it’s OpenFE’s)

  • DD – will schedule a call with Mobley in a morning, invite DH, LD, JW

Manuscript needs

• LD – waiting for results from partners; that’s really all we need at this point for the manuscript. I sent my email requesting results in 2-4 weeks about 2/3 weeks ago, so I’m hoping for results in the next ~10 days. So we may not get everything, but I’m hoping to get something like 6/10 in the end.

• Expecting to get Sage results from

  • TFox

  • KMeier

  • WSwope

• Already have from

  • Ian Craig (BASF)

  • Gary/Hahn/D’Amore (Janssen)

  • XLucas (Roche)

• LD – re: OPLS

  • taking into account possibility of just doing all public set optimizations with OPLS at Janssen

  • Gary working on a joint email with other partners for official approval from schrodinger on public set benchmarks

  • LD – My previous comments about emails from Robert Abel were on a different topic, it turns out – It was with regard to FEP+, not the ffbuilder by itself. So GTresadern and TFox will reach out to Schrodinger to get a blanket approval for this study.