ORCA doesn't parallelize when used as calculatiom method in CREST

[annulen]

Hello,

I've tried to make conformation sampling with CREST using ORCA. My machine is capable of running multiple ORCA calculations simultaneously while using parallelization in each of those ORCA jobs. I've set threads=96 in the input file and it seems to be used fine with GFN2, but ORCA was always running in single process mode and only one job at time. I've tried adding things like %pal nprocs 32 end to orca template, but it results in a garbage in the input file, like

orca.template:

! r2scan-3c TightSCF EnGrad
%pal nprocs 32 end

Generate ORCA.in:

! r2scan-3c TightSCF EnGrad
q@a+��[&�
�[)
��[���[8��[

*xyz 0 1  # charge and multiplicity (2S+1)
...

[pprcht]

Do not do that. ORCA is MPI parallelizd while CREST uses OpenMP, this can lead to some serious problems. Essentially, CREST and ORCA try to draw from the same pool of threads if you set it up this way. So, if you set the threads in CREST to 96, it will at some point attempt to host 96 ORCA calculations which by themselves each request 32 MPI processes. The two programs don't actively know of each other. Try setting a lower number of threads for CREST, the total number of threads would be the one requested for CREST plus all those for the parallel ORCA jobs. Getting it to run properly may requires some trial and error.

[annulen]

Is it fine to use 3 threads and 32 MPI processes for 96 CPU cores?

Can CREST actually run more than one ORCA at time? I always see it running just one ORCA job.

[pprcht]

3 Threads and 32 MPI could work, especially if the CPUs allow multi-threading, but it would probably be safer to not exceed the number of CPUs.

The code is set up to handle multiple parallel program executions, including ORCA and other MPI programs, it should work. However, for subprocess execution (scripts, ORCA, etc.) there was one bug fix in the main version some time ago (#409 ), so please make sure you use the current version.

[annulen]

Yes, I'm using 3.0.2

[pprcht]

which may be a different 3.0.2 than the current HEAD version https://github.com/crest-lab/crest/releases/tag/latest

[annulen]

With threads=3 CREST creates _1, _2, _3 subdirectories, however there is only one ORCA calculation running at time, which can be easily tracked with top and ls.

My input file:

input='KUZ-48.xyz'
runtype='imtd-gc'
optlev='tight'
multilevelopt=false
threads=3

[calculation]
[[calculation.level]]
method='gfn2'
weight=1.0

[[calculation.level]]
method='orca'
orca_cmd='/home/enine/opt/orca-6.0.1/orca_6_0_1_linux_x86-64_shared_openmpi416_avx2/orca_stdout.sh'
orca_template='orca.template'

#molecular dynamics data (activate only GFN2)
[dynamics]
active=[1]

[annulen]

which may be a different 3.0.2 than the current HEAD version https://github.com/crest-lab/crest/releases/tag/latest

Oops, I've missed that comment. Will try HEAD.

[annulen]

Same thing happens with master. There are _1, _2 and _3, but only one ORCA master process is running (and only one ORCA.out is updated)

$ LC_ALL=C ls -lt */_*/*.out
-rw-rw-r-- 1 enine enine 41261 Jun 16 16:37 calculation.level.2/_1/ORCA.out
-rw-rw-r-- 1 enine enine 82931 Jun 16 16:20 calculation.level.2/_3/ORCA.out
-rw-rw-r-- 1 enine enine 82931 Jun 16 16:20 calculation.level.2/_2/ORCA.out

[pprcht]

hm in that case there is either something wrong for the ORCA wrapping specifically which wasn't fixed in #409, or it can't handle the call to orca_stdout.sh which I assume is new in ORCA 6.x, while the interface was set up with ORCA 5.x.

I can look into that hopefully soon, but as a "right now" workaround you could try wrapping the instructions in a generic script and call that. I.e., go for

[[calculation.level]]
method = "generic"
script = "/path/to/some/generic/run.sh"
gradtype = "engrad"
gradfile = "genericinp.engrad"

where you wrap all the ORCA call in the run.sh and specify from which file the energy and gradient shall be read (and in which format). In the individual calculation directories the structure will be placed as genericinp.xyz from which you must construct the ORCA input file. I'm attaching a minimal example that can be made to work with xtb, provided that you replace the paths in the input files. ORCA probably needs the mpirun call in such a script, plus you'd need to figure out where to place the energy and gradient and in what format.

example_generic.zip

[github-actions]

This issue had no activity for 6 months. It will be closed in 1 week unless there is some new activity.