Made QCSubmit and Bespokefit releases, let me know if there’s any trouble.
BW – Already used new QCSubmit to submit a new dataset, should fix the old issue once it goes through.
LipidMaps workers seem to be slowing down
BW – Unsure what’s going wrong here. Have two types of workers submitted. I went in using an interactive shell and saw lots of psi4 jobs completing but may be finite difference inner steps or something.
JW – I wonder if we get hammered by molecule size when doing hessians since it scales quadratically
BW – I think for the opt steps it should just be doing gradient, so 3N rather than 3N^2
JW – Maybe just try workers with a lot more resources? Could deploy just a few high-resource workers, up to 32 cores and 64GB ram. Try exploring the parameter space under that.
BW – Sounds good, I tried doing parameter sweeps with a script last week. Weird because AMI said (and I saw in logs) that even large-ish mols should nly need 5-10GB of RAM. But this may scale with number of cores as well.
JW – Could also look at n_opt_steps as a funciton of molecule size?
BW – Yeah, could look at the length of opt trajectory for finished jobs.
…
BW – I’ll try some of these things, and if it still doesn’t unblock the set I’ll split by size
JW – Sounds good.
BW – Rocket dataset is finished. I told CAdams, seems like there was something wrong with their infra. Moved to scientific review and am awaiting feedback from CAdams.