BP – Some trouble with pubchem as reported by DD. I contacted our hosting provider about more storage space but I haven’t received reports of downtime.
(DD – In submitting pubchem sets 2-5 last week, I forgot to pull changes from master
, and ended up submitting them from my local box with wavefunctions still on)
DD – How is storage looking on the host? Retagged several pubchem sets.
BP – Not looking good. It went up another 200GB. At this pace we’d run out of space in a month.
DD – I’ve tagged them as openff-defunct
, but it may take a while to get that tag change propagated to all ~400k entries, and ones that are already in progress won’t get updated.
BP – I can run a script on the server itself. Also, do you have any managers without any tags? Because those could be running the defunct jobs.
PB – There was a set of 50k mols submitted over the weekend with wavefunctions and a smaller basis set. Though that’s basically already done
BP – In a pinch, I could move some datasets to tables on separate storage devices, which could give us access to mroe space. But that would be a bad long-term solution.
DD – Would it be possible to delete the results of some of the previous jobs?
DD – I’ll send BP the defunct-retagging script for local execution on the server. I’ll also set the defunct sets as low priority.
DD – Trying to resubmit them without wavefunctions would hit problems with deduplication, where since the wavefunction requests aren’t part of the submission spec, the new submissions would just trigger wavefunction calcs again.
BP will look into how to attach more storage to QCA.
DD: So for now let’s put 2-5 on the backburner and let the compute chew set 1 and other datasets until we get the issues with space resolved.