Wake elements with memory leak

Moderators: cyao, michael_borland

Post Reply
Seb_Wilkes
Posts: 15
Joined: 04 Mar 2022, 09:48

Wake elements with memory leak

Post by Seb_Wilkes » 24 Mar 2023, 05:45

Hello all,

I have just recently switched modelling the collective effects from an impedance element to a wake element. It was since then my problems have been beginning; chiefly, my simulations crashing out with out-of-memory errors. After turning on the

Code: Select all

monitor_memory_usage=1
option, I was able to see:

Code: Select all

Memory usage increased by 3511 kB in ZTRANSVERSE TRANS_RW_IMPEDANCE#1, pass 23040
Looking at the WATCH output, I was also able to get the following plot:
memory_leak.png
As you can see, it increases linearly with the number of turns. To confirm, I was able to budget twice the memory on the cluster and, sure enough, it failed about twice the number of turns later.

My colleague decided to have a look at it, so I'm going to quote some of his findings in case this helps. For clarity, we are running multi bunch simulations; I was running 2022.2 and my colleague additionally tried it on 2020.5 .

By commenting out various combinations of elements that represent collective effects, the longitudinal wake, and higher order modes modelled by FRFmode (but my simulation did not contain the latter), was found to show up on the log file. To be clear, this means he said that the memory leak effect was not observed when other collective elements were included except for the ones I just mentioned. If this observation is correct, this would indeed be quite strange; it would point to some cross talk between the different elements that results in some kind of memory leak.

michael_borland
Posts: 1927
Joined: 19 May 2008, 09:33
Location: Argonne National Laboratory
Contact:

Re: Wake elements with memory leak

Post by michael_borland » 27 Mar 2023, 11:09

That is very odd. Can you say a bit more about the simulations? Are you requesting output from the wake elements? Is this a multi-bunch simulation?

--Michael

Seb_Wilkes
Posts: 15
Joined: 04 Mar 2022, 09:48

Re: Wake elements with memory leak

Post by Seb_Wilkes » 28 Mar 2023, 04:14

michael_borland wrote:
27 Mar 2023, 11:09
That is very odd. Can you say a bit more about the simulations? Are you requesting output from the wake elements? Is this a multi-bunch simulation?

--Michael
Hi Michael,

Yes, this is part of our multi-bunch simulation effort. I've attached our .ele file so you can see the settings of the impedance elements. While we are not requesting output from the wake scripts, we are using a watchpoint. However the watchpoint is only designed to output every 10k turns and there is never any drop in the memory graph.
Attachments
multi_bunch_sim_single_excitation.ele
(3.15 KiB) Downloaded 47 times

Seb_Wilkes
Posts: 15
Joined: 04 Mar 2022, 09:48

Re: Wake elements with memory leak

Post by Seb_Wilkes » 28 Mar 2023, 10:41

We just converted the wake sdds files into impedance files (and obviously changed the .ele file accordingly), and so far looks like no memory leak -- will update if I see otherwise!

michael_borland
Posts: 1927
Joined: 19 May 2008, 09:33
Location: Argonne National Laboratory
Contact:

Re: Wake elements with memory leak

Post by michael_borland » 28 Mar 2023, 11:24

Thanks, I'll look into this.

Impedances are the most typical implementation used in storage ring modeling, so it is possible there's a memory leak we haven't identified.

--Michael

michael_borland
Posts: 1927
Joined: 19 May 2008, 09:33
Location: Argonne National Laboratory
Contact:

Re: Wake elements with memory leak

Post by michael_borland » 28 Mar 2023, 12:48

I found the source of the memory leak and fixed it. The fix will appear in the next release.

Thanks again for reporting the problem.

--Michael

Post Reply