Page 1 of 1

sddsprintout to csv over several pages

Posted: 28 Sep 2017, 10:55
by Faissal
good afternoon,

I have the output of a WATCH file ("output.w3" for example) that takes coordinates of the particles in the bunch at every turn. I can plot it properly and navigate between the various turns using sddsplot.

In order to post-process the data, I would like to convert the file into a csv file. I do then, as the SDDS manual recommends:

sddsprintout -spreadsheet=csv -column=* output.w3 output.csv

however, instead of having several distinct pages, the csv file has only one massive page with all the pages of the sdds file. Is there a way to separate the pages in the csv files. I have used -paginate, which doesn't seem to help.

Re: sddsprintout to csv over several pages

Posted: 28 Sep 2017, 11:07
by soliday
Try it with the -postPageLines=1 option.

Re: sddsprintout to csv over several pages

Posted: 28 Sep 2017, 11:08
by michael_borland
You can split the file into separate files first, then convert to csv. E.g., assuming use of the bash shell

Code: Select all

sddssplit output.w3 -extension=w3s -rootname=output- -digits=6
for file in output-??????.w3s ; do sddsprintout -spreadsheet=csv -column=* $file ${file/w3s/csv}; done
There are also SDDS extensions for many programming languages, so you can read SDDS files directly; this is much more efficient for large files. Finally, the SDDS toolkit can perform many postprocessing functions directly with SDDS files.

--Michael

Re: sddsprintout to csv over several pages

Posted: 29 Sep 2017, 09:05
by Faissal
Thanks Michael and Soliday for your answers, I managed to write a little piece of code to do it, with the help of the method you showed.

I am sharing it in case it may help others.

Code: Select all


#!/bin/bash


w3_all_files=$(ls *.w3)
for w3_file in $w3_all_files ; do 
    sddssplit $w3_file -extension=w3s -rootname=${w3_file%.*} -digits=6 
done

echo "split done"
w3sfiles=$(ls *.w3s)

for file in $w3sfiles ; do 
    sddsprintout -spreadsheet=csv -column=* $file ${file/w3s/csv}
    echo " ${file/w3s/csv} produced "
done




The purpose was to post-process the csv files in Python.

Re: sddsprintout to csv over several pages

Posted: 29 Sep 2017, 09:34
by michael_borland
There's a library for reading SDDS files directly into python. Look on
https://www1.aps.anl.gov/Accelerator-Op ... s/Software

--Michael

Re: sddsprintout to csv over several pages

Posted: 02 Oct 2017, 06:19
by Faissal
Excellent, thanks Michael. This will definitely be helpful.

regards,

- Faissal