2021-02-24 Benchmarking for Industry Partners - Development Meeting notes

Unit tests on openff-benchmark optimize

Jeff

• We need to build out a test suite for server and non-server optimization approaches

  • DD: might pull from QCFractal’s test suite to ensure we can stand up a reasonable server process under Github Actions

  • JH: QCSubmit makes use of a fixture for standing up its server instance; inspired by the same usage in QCFractal

  • DD: we’ll use this approach to build out our openff-benchmark test suite for QCFractal interactions

    • will get skeleton in place, plus initial meat; will enlist Jeff’s help in filling in meat on other subcommands

  • JW: we should duplicate what Josh has for QCSubmit for tests, since we’ll need to introduce different molecules and JSON schema for different tests

Updates

• DH: since Friday, started up proprietary molecule dataset;

  • around 1000 unique molecules, 9000 conformers, shuffled projects drawn from to have a nice mix; can stop at any time if we run out of time; estimated 7 weeks to completion

  • Schrodinger entrypoint PR is up!

    • DD: will review

    • DH: will seek a working meeting with Bill to try it out

• JH: nothing from me; focused on openff-qcsubmit, here for support on problem areas

• JW: opened two PRs, one for mirror image deduplication, one on handling optimize export errors

  • for mirror image deduplication:

    • if there are stereocenters in the molecule, we don’t consider reflections at all

  • DD: do we want this to only apply to postprocessing for now, then PR in its inclusion into validate and generate-conformers in time for Season 2?

  • JW: could also not merge this until Season 2.

  • DH: think it’s a good suggestion to wait until Season 2.

  • for handling optimize export errors, need tests

• DD: by next week, may have up to 3/10 partners that have completed production

  • have not finished putting together proposal for data drop

Next release

Jeff

• DH: Schrodinger pathway won’t be ready for a Friday release

  • JW: should be okay; can go into the release following

• JW: want to get in proton transfer detection if possible; need test suite to do this work

  • will ask QCElemental to guess connectivity; if different from connectivity due to SMILES, will know there was a proton transfer

• JW: otherwise, just export

• So, two items above for Friday release

Season 2 (when and what)

• JH: big thing may be torsiondrives; a lot of interest from partners on this

  • ANI2x

• DH: season 2 depends on willingness and availability of partners

  • probably a good topic for discussion of next plenary call

  • DD: can reach out to Gary with list of items we’d like to hit for this call

  • JW: I think there may be a pause for paper writing, so may not be much appetite for an immediate season 2

  • DH: if it takes a while to write papers from the season 1 data, they may be interested in a dataset expansion in season 2

    • could also be a season 3 based on what we find out from results in season 2

• DD [committed]: will email Gary to ask what his plans are for next plenary agenda; lay out items along these lines for agenda items, get his thoughts