10 min | Benchmarking large systems on Folding@Home | Chapin Cavender | CC – I reached out to Sukrit Singh in the Chodera lab, asking about the process of running on F@H. Sukrit said that F@H is so distributed that it’s hard to get continuous trajectories. So while you can get aggregate millisoncs in weeks, since we need long trajectories it’ll be better to run long sims on academic clusters. CC – I think there IS an interesting scientific project that could be done regarding how to do lots of small simulations that are equivalent to a longer one. But that’s not in scope at the moment. MS – I agree that a comparison needs to be done, like it shouldn’t be assumed that they’re equivalent. Given the scales, if we find that we’d need 2-3x the aggregate sampling, that would still make F@H a worthwhile avenue. So the big question would be whether to use metadynamics or other methods to generate seeds. Or we could use seeds from previous studies (like the DE Shaw sims). CC – I wouldn’t want to rely on something like the DES dataset, since we could know that a trajectory would already exist for everything we want to run. MS – Agree, we’d want to have a way to seed our own starting points. CC – Would we want to pick a single method for these seeds, or would the experiment need to compare many methods? MG – It seems like running this experiment could add a big delay to the project, so I don’t think we want to do it in the course of making the protein FF. Maybe this could be a side project for additional personnel? MS – Yeah, it’d be necessar yto define convergence and a few other things on the path to that. CC – Especially, if we’re looking at like a 10 ms simulation lenght, we still won’t sample unfolding and other non-folded states. So it’ll be hard to figure out whether our benchmarks are suitable for comparison to previous work. MG CC – If we start from a xtal structure, we’d expect it to remain close to the xtal, as opposed to a dilute sim where we might expect unfolding or non-native states. JW – Do we have access to the clusters that we’ll need, or should we seek others? CC – I think we can start work in this direction. I can benchmark representative systems on TSCC, which I already have access to. PB – First pass will use AMBER systems? CC – Yes, ff14SB. MG – Those timings could be the basis for an XSEDE application if we need it. DM (chat) – Ours, HPC3, has V100s as I recall and is “pay for time” at a fairly low rate (hardware replacement cost only), so we can price that out if we like. These are relatively expensive chips, but on the other hand it’s ONLY hardware cost. We could run benchmarks with no cost. MG – Could we use AWS/Google cloud? MS – It’s hard to get multi-gpu instances on the cloud services. CC – I’m thinking that we can use this as a pilot project, like “this method works for a few proteins”, with the implication that we can throw more resources at it to get a larger benchmark
MS – So, one prerequisite is for Chapin to define “convergence”, and to lay out the systems. CC – From talking to sukrit about F@H, it seems really promising to run things on there once we can make use of the architecture. They have a “pilot program” where you get limited access for some time to prove that nothing explodes. Where will the trajectories be stored? JW – If they’re too large for a university cluster, we could put them on AWS in a sort of long-term data store. DM – I talked to an AWS guy at CUP who was looking for ways to connect. MS – If we can get some free time/free credit I’d love to try a refit using property calculations DM – Also, there are some lower-cost cloud providers who were looking to work with us. So that would be good to follow up on. (Woody Sherman offered to make an intro to the oracle folks)
|
|
|
| DN – Update on the implementation for vsites - JW, SB, and I talked last week about a plan for these, we’re still working on it but I will update you later. DM – What specifically are you looking at with these? DN – Like, the ordering of infra/science, and the likely release dates/deadlines. DM – I connected JW and MT to AMBER devs about vsites. JW – I’m interested to continue the discussion with them. It seems like their virtual site documentation is “read the fortran”, which I will do once, but I may need them need them to make public guarantees (eg in the manual) about the spec/function before we go too far. MS – CHARMM exports? Not a “must” for the initial release, but would be a nice “should” JW – Not clear how the ordering of priorities here would manifest - We may have a 98% accurate AMBER exporter, and the choice to
either spend time making it 99% accurate, or making an initial CHARMM exporter. I don’t know what we’d do in that case.
|