2021-06-16 Industry benchmarks meeting notes

Updates

• DH: discussion with Gary, would be nice to have Sage included in the results

  • the paper will be very much after the Sage release

  • DD: it’s possible to do, but will want to make this as painless as possible for folks

  • DH: this can also be a topic for the partner call; we can gauge desire for adding in an openff-2.0 spec to the season

• DH: also doing a tour with Lorenzo next week of the protein-ligand-benchmarks, pmx usage

• DD:

  • Kicked off v1.1 version of benchmark on QCA-dataset-submission. It’s mostly complete since it overlaps heavily with the already-completed V1.0. So all the new stuff is the new merck set. It’s already mostly complete.

  • I was sick last week, not a ton to update on today. I’ll work with LD right after this call.

• JW: nothing too new on my end; in advisory board call, folks very excited about running openff-benchmark on Sage.

  • useful to make a list of blockers to this

    • final sage release

    • conda package

    • season 1.X expansion to include sage

  • DD – Timeline for sage release?

    • JW – Maybe around a month?

  • JW: could be useful to get RC into folks' hands

    • probably don’t want to include RC into Season results

  • DH: running the release candidate to reproduce the Lim results would be of benefit

  • JW: what’s a checklist to make this happen?

    • DD: Could make a season 1:3

    • JW: Would need a new release from the season-1 branch of openff-benchmark.

  • DD – Generalization of openff-geopt will help people to do thier own experimetns in the future

  • LD – If people run a new method and analysis, then the analysis should just run on the ADDED data, not all of it. So, adding a new method could make the shape of the final datasets different or cause an error. So we should use an intersection of the molecules/conformers in the final plot. This means that ADDING another MM method could make the final dataset for ALL methods smaller.

  • JW will check with SB to get approval to make a new conda package, and will ping DD once it’s made.

• LD: pull request for new analysis (Xavier and Bill)

  • DD: we’ll get the PR sorted in our working session after this call