2021-06-16 Industry benchmarks meeting notes

Participants

  • @David Hahn

  • @David Dotson

  • @Lorenzo D'Amore

  • @Jeffrey Wagner

Goals

  • Updates from team, topics

Discussion topics

Item

Presenter

Notes

Item

Presenter

Notes

Updates

 

  • DH: discussion with Gary, would be nice to have Sage included in the results

    • the paper will be very much after the Sage release

    • DD: it’s possible to do, but will want to make this as painless as possible for folks

    • DH: this can also be a topic for the partner call; we can gauge desire for adding in an openff-2.0 spec to the season

  • DH: also doing a tour with Lorenzo next week of the protein-ligand-benchmarks, pmx usage

  • DD:

    • Kicked off v1.1 version of benchmark on QCA-dataset-submission. It’s mostly complete since it overlaps heavily with the already-completed V1.0. So all the new stuff is the new merck set. It’s already mostly complete.

    • I was sick last week, not a ton to update on today. I’ll work with LD right after this call.

  • JW: nothing too new on my end; in advisory board call, folks very excited about running openff-benchmark on Sage.

    • useful to make a list of blockers to this

      • final sage release

      • conda package

      • season 1.X expansion to include sage

    • DD – Timeline for sage release?

      • JW – Maybe around a month?

    • JW: could be useful to get RC into folks' hands

      • probably don’t want to include RC into Season results

    • DH: running the release candidate to reproduce the Lim results would be of benefit

    • JW: what’s a checklist to make this happen?

      • DD: Could make a season 1:3

      • JW: Would need a new release from the season-1 branch of openff-benchmark.

    • DD – Generalization of openff-geopt will help people to do thier own experimetns in the future

    • LD – If people run a new method and analysis, then the analysis should just run on the ADDED data, not all of it. So, adding a new method could make the shape of the final datasets different or cause an error. So we should use an intersection of the molecules/conformers in the final plot. This means that ADDING another MM method could make the final dataset for ALL methods smaller.

    • JW will check with SB to get approval to make a new conda package, and will ping DD once it’s made.

  • LD: pull request for new analysis (Xavier and Bill)

    • DD: we’ll get the PR sorted in our working session after this call

Action items

@Jeffrey Wagner will notify @David Dotson of openff-forcefields release with Sage release candidate
@David Dotson will add in Sage as an additional, optional season spec 1:3 or similar; cut release, add instructions for use as addendum to protocol

Decisions