Elegant users forum

Hi,
I am using libSDDS from the SDDSToolKit-devel RPM to read sdds files in c++ applications. It works very good - thanks again for the package.
Now I tried to use it in a multi threading application and get various SDDS errors when reading the same file with multiple threads at the same time.
Is there a way to do this thread-safe? I naively assumed reading a file parallelized is no problem.
Do you know which function(s) cause the problem? I basically create a new SDDS_TABLE in each thread and use functions like

• SDDS_InitializeInput()
• SDDS_ReadPage()
• SDDS_FilterRowsOfInterest()
• SDDS_GetValue()
• SDDS_GetParameterIndex()
• SDDS_GetParameterType()
• SDDS_GetParameterByIndex()
• SDDS_Terminate()

I never tried that. Let me look into it and see what I can do.

By the way, we usually use MPI for parallelization. There's an MPI version of the library (libSDDSmpi).

--Michael

I put together a new SDDSToolKit-devel that should solve the threading problems.
http://www.aps.anl.gov/asd/oag/download ... x86_64.rpm

Most of the important functions should now be thread safe. There are still issues with some functions which you probably aren't using:
SDDS_MakePointerArrayRecursively
SDDS_SetNameValidityFlags
SDDS_SetDefaultIOBufferSize
SDDS_InitializeAppendToPage
SDDS_SetRowLimit
SDDS_SetTerminateMode
SDDS_ComputeColumn
SDDS_FilterRowsWithRpnTest
SDDS_RegisterProgramName
SDDS_MatchColumns
SDDS_MatchParameters
SDDS_MatchArrays

There are also an issue related to setting the error values and printing them out. One thread may print another thread's error message which isn't a huge deal since there is an error anyway that needs to be fixed.

Thank you very much for looking into this.
The new version works much better, no SDDS errors!

But sometimes (about 1 of 4 runs) the program terminates unexpectedly with "realloc(): invalid next size". I have not identified the causing function, but it could be related to the issue with error values you mentioned: I use the error returned by SDDS_InitializeInput to handle a non existing file at runtime. Maybe I can try another method for this next week.

Here is a very late feedback:
All functions listed in the first post work with threads without problems. I fixed the remaining issue described in the previous post by avoiding SDDS_InitializeInput errors (I check for file existence beforehand).

Thank you very much!