gpuElegant results not matching elegant
Posted: 06 May 2022, 08:02
After some experimentation, we're finding that the output of gpu-elegant for certain configurations seems to be failing. In particular, the particles don't seem to make it through the initial beamline elements, with subtleties as to what is going wrong. We're not quite sure where it is failing yet, but have some examples using the csbend component where this fails. TL;DR, gpu-elegant is losing particles.
I've rebuilt the newest version of elegant on our computing environment, gpu-jupyter (2021.4) and it shows the same behavior as the default install that we use (2021.1).
There's also a local build issue with elegant and CUDA 11 w/ GCC 11 that is known on other projects, likely due to a cstd versioning issue. I'm debugging that myself to build gpu-elegant locally on another system for testing as to if it is the version in our container that has issues or the code itself: c++17 vs c++14. I still need to build with c++17 instead as this requires changing some flags.
All commands were run as $ <binary> tracking.ele > <logfile>.log 2>&1
2021.1 non-GPU elegant (ele.log):
2021.1 gpu-elegant (gpu-standard-ele.log):
2021.4 gpu-elegant:
And an additional fun compilation message:
I've rebuilt the newest version of elegant on our computing environment, gpu-jupyter (2021.4) and it shows the same behavior as the default install that we use (2021.1).
There's also a local build issue with elegant and CUDA 11 w/ GCC 11 that is known on other projects, likely due to a cstd versioning issue. I'm debugging that myself to build gpu-elegant locally on another system for testing as to if it is the version in our container that has issues or the code itself: c++17 vs c++14. I still need to build with c++17 instead as this requires changing some flags.
All commands were run as $ <binary> tracking.ele > <logfile>.log 2>&1
2021.1 non-GPU elegant (ele.log):
Code: Select all
tracking 4000 particles
3 May 22 16:42:38: This step establishes energy profile vs s (fiducial beam).
3 May 22 16:42:38: Rf phases/references reset.
4000 particles present after pass 0
...
Adding OCT_K after (null)
Adding OCT_K after (null)
Adding OCT_K after (null)
Adding OCT_K after (null)
...
4000 particles present after pass 4
4000 particles transmitted, total effort of 16000 particle-turns
33776880 multipole kicks done
Code: Select all
tracking 4000 particles
3 May 22 16:41:48: This step establishes energy profile vs s (fiducial beam).
3 May 22 16:41:48: Rf phases/references reset.
4000 particles present after pass 0
...
0 particles present after pass 4
0 particles transmitted, total effort of 0 particle-turns
18720 multipole kicks done
Code: Select all
tracking 4000 particles
3 May 22 16:40:46: This step establishes energy profile vs s (fiducial beam).
3 May 22 16:40:46: Rf phases/references reset.
4000 particles present after pass 0
...
0 particles present after pass 4
Post-tracking output completed.
Tracking step completed ET: 00:00:01 CP: 1.43 BIO:0 DIO:0 PF:0 MEM:4838467
Code: Select all
/home/vagrant/jupyter/oag/apps/src/epics/extensions/include/mdb.h:599: warning: "PI" redefined
599 | #define PI 3.141592653589793
|
In file included from gpu_lsc.cu:1:
/home/vagrant/jupyter/oag/apps/src/epics/extensions/include/constants.h:35: note: this is the location of the previous definition
35 | #define PI 3.141592653589793238462643