2021-02-10 Benchmarking for Industry Partners - Development Meeting notes

Updates

• DH: burn-in set complete at Janssen; looks good

  • meeting with Gary to start up production run

  • met with Jeff, DD on RMSD threshold adjustments

  • it’s okay to wrap Schrodinger tooling for OPLS benchmark component; each partner would need to get permission from Schrodinger to publish these

  • Gary would like results in ~60 days (Aprilish)

• JW: could provide engineering support for the OPLS component if I can get access to Schrodinger tools

  • no guarantee Gilson will give access; many strings attached for academic use

  • DH: will try to have a demo ready for next week so we can begin discussion, improvement

• JW: since last meeting, new RMS cutoffs discussion and implementation

• DD: production kickoff; 3 partners ready for production runs as of Friday, need to update list on who else is ready today

  • will reach out to partners we haven’t heard from on this in #benchmarks-support

Deadline extension policy

• JW: when I was thinking about the compute estimation, datasets with many large molecules may take longer than 60 days; do we want to set a policy on how we will extend deadlines, or a threshold for “complete” on a dataset (even if it isn’t 100% complete)?

  • getting a progress report from partners no more than a month after; would allow us to then decide on a reasonable policy

  • DH: even if we have some (large) fraction of the datasets, can begin working with preliminary data downstream; this would also buy the partners 1 or 2 months more to run if they need it

  • JW: if we get partial data at some point, coverage report would be for whole dataset, so not really representative of sample; could run coverage report on e.g. MM outputs and ask for that as well