For scf convergence issues

Related to 2021-02-10 QCArchive - PEPCONF Investigation Meeting notes

In case of molecule 34752921 there is an issue with density convergence as observed by the stagnant RMS value even though energy is converged within 1E-12. This resulted in SCF convergence failure even after 300 steps. Here adding few lines from stdout that show this behavior:

Total Energy Delta E RMS |[F,P]| @DF-RKS iter SAD: -1103.83535360608539 -1.10384e+03 0.00000e+00 @DF-RKS iter 1: -1103.18778842035908 6.47565e-01 5.26801e-03 DIIS @DF-RKS iter 2: -1086.11061814176878 1.70772e+01 1.07085e-02 DIIS @DF-RKS iter 3: -1085.83317095694201 2.77447e-01 1.13329e-02 DIIS @DF-RKS iter 4: -1104.38600962244641 -1.85528e+01 3.14371e-03 DIIS @DF-RKS iter 5: -1104.82541609006171 -4.39406e-01 1.46567e-03 DIIS . . . @DF-RKS iter 251: -1103.88828252914141 9.09495e-13 2.59338e-03 DIIS @DF-RKS iter 252: -1103.88828252914232 -9.09495e-13 2.59338e-03 DIIS @DF-RKS iter 253: -1103.88828252914254 -2.27374e-13 2.59338e-03 DIIS @DF-RKS iter 254: -1103.88828252914345 -9.09495e-13 2.59338e-03 DIIS @DF-RKS iter 255: -1103.88828252914141 2.04636e-12 2.59338e-03 DIIS @DF-RKS iter 256: -1103.88828252914095 4.54747e-13 2.59338e-03 DIIS @DF-RKS iter 257: -1103.88828252914345 -2.50111e-12 2.59338e-03 DIIS @DF-RKS iter 258: -1103.88828252914323 2.27374e-13 2.59338e-03 DIIS @DF-RKS iter 259: -1103.88828252914186 1.36424e-12 2.59338e-03 DIIS @DF-RKS iter 260: -1103.88828252914254 -6.82121e-13 2.59338e-03 DIIS . . .

Here is the sdf file and structure of the molecule:

 

Few trials with change in SCF parameters:

# 1. damping_percentage is changed from 100 to 30, still RMS stays constant after a few steps

# 2. turning off diis_rms_error as convergence metric - didn't work, lot of oscillations in energy during optimization

# 3. turn off DIIS completely - it's a fire pit

# 4. basis_guess - changed from false to true, failed

# 5. soscf - second order methods for scf convergence set to true, supposed to help with density convergence problems - THIS HELPED -

Quoting psi4 manual: ‘' Second-order convergence takes into account both the gradient and Hessian to take a full Newton step with respect to the orbital parameters. This results in quadratic convergence with respect to density for SCF methods. For cases where normal acceleration methods either fail or take many iterations to converge, second-order can reduce the total time to solution.’'

Here is the converged psi4 output:

 

Recommendation for error cycling: If possible turn on soscf in pathological cases of scf convergence.