Page 1 of 1

Wake elements with memory leak

Posted: 24 Mar 2023, 05:45
by Seb_Wilkes
Hello all,

I have just recently switched modelling the collective effects from an impedance element to a wake element. It was since then my problems have been beginning; chiefly, my simulations crashing out with out-of-memory errors. After turning on the

Code: Select all

monitor_memory_usage=1
option, I was able to see:

Code: Select all

Memory usage increased by 3511 kB in ZTRANSVERSE TRANS_RW_IMPEDANCE#1, pass 23040
Looking at the WATCH output, I was also able to get the following plot:
memory_leak.png
As you can see, it increases linearly with the number of turns. To confirm, I was able to budget twice the memory on the cluster and, sure enough, it failed about twice the number of turns later.

My colleague decided to have a look at it, so I'm going to quote some of his findings in case this helps. For clarity, we are running multi bunch simulations; I was running 2022.2 and my colleague additionally tried it on 2020.5 .

By commenting out various combinations of elements that represent collective effects, the longitudinal wake, and higher order modes modelled by FRFmode (but my simulation did not contain the latter), was found to show up on the log file. To be clear, this means he said that the memory leak effect was not observed when other collective elements were included except for the ones I just mentioned. If this observation is correct, this would indeed be quite strange; it would point to some cross talk between the different elements that results in some kind of memory leak.

Re: Wake elements with memory leak

Posted: 27 Mar 2023, 11:09
by michael_borland
That is very odd. Can you say a bit more about the simulations? Are you requesting output from the wake elements? Is this a multi-bunch simulation?

--Michael

Re: Wake elements with memory leak

Posted: 28 Mar 2023, 04:14
by Seb_Wilkes
michael_borland wrote:
27 Mar 2023, 11:09
That is very odd. Can you say a bit more about the simulations? Are you requesting output from the wake elements? Is this a multi-bunch simulation?

--Michael
Hi Michael,

Yes, this is part of our multi-bunch simulation effort. I've attached our .ele file so you can see the settings of the impedance elements. While we are not requesting output from the wake scripts, we are using a watchpoint. However the watchpoint is only designed to output every 10k turns and there is never any drop in the memory graph.

Re: Wake elements with memory leak

Posted: 28 Mar 2023, 10:41
by Seb_Wilkes
We just converted the wake sdds files into impedance files (and obviously changed the .ele file accordingly), and so far looks like no memory leak -- will update if I see otherwise!

Re: Wake elements with memory leak

Posted: 28 Mar 2023, 11:24
by michael_borland
Thanks, I'll look into this.

Impedances are the most typical implementation used in storage ring modeling, so it is possible there's a memory leak we haven't identified.

--Michael

Re: Wake elements with memory leak

Posted: 28 Mar 2023, 12:48
by michael_borland
I found the source of the memory leak and fixed it. The fix will appear in the next release.

Thanks again for reporting the problem.

--Michael