2021-07-07 Industry benchmarks meeting notes

Presentation slides; content and structure

• DD: wrap on Season 1:

  • coordinated campaign is over, no announcement stream

• JW: No new computations(?) like OPLS or other analyses?

• DD: supplements - we provide these functionalities with instructions, but no guarantee of rapid support

  • DH: Agree with “no rapid support” wording

• DH – I think GT wants OPLS results, but I don’t think we’ll get it from every partner.

  • DD – DH, could you take the lead on this slide, and link to the protocol for running this?

  • DH – Yes.

• DD – Optimizations with Sage?

  • JW – We don’t know whether the release candidate will be the same as the release, so there’s nothing really to benchmark now.

• Alternative analysis

  • LD – No feedback from anyone yet. I think they just got back from vacation.

  • DD – LD, could you take the lead on the “feedback requested: Alternative analyses” slide?

  • LD – Yes, I can do that.

• Season 2 slide

  • JW – I like the phrasing of this as being “annual”

• On “Supplements”

  • DH: Gary would very much like to see OPLS data in the publication; probably won’t work for all partners, though (barrier to getting data out)

    • would prefer “the user may do it”

  • JW: three options for “supplement”

    • we encourage you to do this and submit

    • we allow you to do this and submit

    • we forbid you from submitting

  • JW: if we’re going with encouraging them, then we should probably put this before the publication discussion

• DD – Is schrodinger branch ready for merge?

  • DH – I haven’t looked at it for a while, so let me review, but I think it’s good to go.

• DD – Should we cut a new release?

  • Would have exact same versions+pins as previous release, except the new openff-benchmark, and .

• JW: is it accurate that this only works with OPLS3e, or does it work with OPLS4

  • DH: instructions for doing this are in the protocol on the PR. Ideally we’d get both OPLSes from everyone

• DH – We could just do the schrodinger analysis on the public set

  •  

Aggregated results

• DD – Not angling for this to be a deep scientific discussion, more of a summary of progress / celebration

Analysis of violations

• LD: will present next week on results, approach