...
Using this modified script from David Dotson to run a task (name it run-local.py):
| Code Block | ||
|---|---|---|
| ||
#!/usr/bin/env python
import sys
import os
import pprint
from qcfractal.interface import FractalClient
from qcengine import compute_procedure
id = sys.argv[1]
# os.makedirs(id, exist_ok=True)
cl = FractalClient.from_file()
task = cl.query_tasks(base_result=id)[0]
print(task)
with open('task.json', 'w') as f:
f.write(pprint.pformat(task.json()))
task.spec.args[0]['input_specification']['keywords']['scf_mem_safety_factor'] = 0.5
# task.spec.args[0]['input_specification']['keywords']['df_ints_num_threads'] = 1
# task.spec.args[0]['input_specification']['keywords']['diis'] = True
# task.spec.args[0]['input_specification']['model']['method'] = 'mp2'
result = compute_procedure(*task.spec.args, local_options={"memory": "16.0", "ncores": 8, "scratch_directory": os.getcwd() })
print("Computation done.")
print(result)
with open('result.json', 'w') as f:
f.write(pprint.pformat(result.json()))
|
...
I drive this python script with the following bash script, which does the initialization and memory recording (name this run-local.sh):
| Code Block |
|---|
#!/bin/bash
id=$1
d=$PWD
finish() {
cd $d
kill $(jobs -p)
}
export -f finish
mkdir -p $id
cd $id
trap finish SIGINT
trap finish SIGCHLD
python3 -u ../run-local.py $id &
pidstat -p ALL -l -C "$id" -r -h -H -u 1 | grep python | tee stat.log |
...
I then have a similar setup for driving the plots. Here is the bash driver (name this makeplots.sh):
| Code Block |
|---|
#!/bin/bash
id=$1
d=$PWD
cd $1
awk '/psi4/ {print $1 " " $13}' stat.log | head -n -1 > psi4.dat
awk '/run-local/ {print $1 " " $13}' stat.log | head -n -1 > main.dat
awk '/psi4/ {print $1 " " $22}' stat.log | tr -d "[A-Z]" | head -n -1 > memlimit.dat
if [ -f out.log ] ; then
awk '/Cache is/ {print $1 " " $4}' out.log > cache.dat
fi
python3 $d/plot.py
cd $d |
I do the head -n -1 funny business in case buffering produced an incomplete last line, which would make the numpy loader bork.
The python plotter (name this plot.py):
| Code Block |
|---|
import matplotlib.pyplot as plt
import numpy as np
import os
fig = plt.figure(figsize=(12*4, 5), dpi=200)
ax = fig.add_subplot(111)
x, y = np.loadtxt("main.dat").T
xmin = x.min()
ax.plot(x-xmin, y / 1024 ** 2, "b", lw=0.3, label="geometric")
x, y = np.loadtxt("psi4.dat").T
ax.plot(x-xmin, y / 1024 ** 2, "r", lw=0.3, label="psi4")
x, y = np.loadtxt("memlimit.dat").T
ax.plot(x-xmin, y, "k", label="mem specified")
if os.path.exists("cache.dat"):
x, y = np.loadtxt("cache.dat").T
try:
size_x = len(x)
if size_x > 0:
ax.plot(x-xmin, y / 1024 ** 3, "g", lw=0.3, label="B cache")
except Exception:
pass
if os.path.exists("disk.dat"):
x, y = np.loadtxt("disk.dat").T
try:
size_x = len(x)
if size_x > 0:
ax.plot(x - xmin, y / 1024 ** 2, "g", lw=0.3, label="Disk space")
except Exception:
pass
ax.set_xlabel("Time (seconds)")
ax.set_ylabel("Memory (GB)")
ax.legend(loc=3)
ax.tick_params(labeltop=False, labelright=True)
ax.grid(axis='y')
fig.tight_layout()
fig.savefig("mem.png") |
...