Github link macro |
---|
link | https://github.com/openforcefield/protein-ligand-benchmark/pull/52 |
---|
|
IAlibay (earlier on Slack) – I won't be able to join in this week - my update on the PLB side of things is "still working on it", sorry it's been a pretty busy week back but I'll get things done soon.
IP – MBoby asked if she was needed, but is quite busy with other things. I recently gave feedback on the PR noting that it’sgood that there are some new docked ligands, but there are 4 cases where the number of ligands decreased. MB will check into this.
JC – Have you been able to tests these systems? It’ll be important to ensure that these simulate before we make the release. It’ll be good to have these checks automated.
IP – That’s still a to-do item for me.
JW – Would this be blocking to the overall project? As in, would we need to rush out a releaseif this falls behind?
JC – We can run from the branch. The big question is whether the re-preparation decreases the accuracy a lot.
IP – In that case, would we need to run these calcs to completion?
JC – Something like that, it would be fine to “spot check” without running the whole set. Do you think we are too limited to run through these in a week?
IP – We’d previously done 300 jobs in a week - Runtimes were largely system-dependent. Some take a few hours, some take a whole day.
JW – This is becoming a bit chicken-and-egg - What if we needed F@H to run enough of the dataset to have confidence in the result? How could we distinguish between regressions due to protocol vs. data?
JC – I think running a subset of the data would solve this without needing F@H. We have lots of compute at MSKCC
IP – Was seeing slower-than-anticipated performance at MSKCC, I’m talking with sysadmins. Which systems would be representative?
JC – I’d think that we should run 1 iteration of each system to test for technical issues, and then 10 edged per system to completion to assess accuracy. Does this sound appropriate?
MH – I think these tests make sense, especially the technical “1 iteration” ones.
JW – I think it’s OK to have tolerance for botched releases - Coming from the very delayed OFF Toolkit release, aiming for perfection has infinite cost, and I’d prefer to have a broken first release that we can fix than big delay.
JS (chat) – IIRC Chris Bayly had a nice plot of the benchmarking series' system sizes vs computing cost (with orion)
MH – Could we run an allocation to run this on orion?
JC – Maybe not, since they prepared systems using their own software, and they generally frown on assisting competitiors
JS – MCL1 was their fastest system
JC – Could we wrap SireMol in a protocol?
JS – Like, SOMD? I think that should be possible. JMichiel would probably be happy to be involved.