How to procedurally generate parameter names from column values

Moderators: cyao, michael_borland

Post Reply
foshea
Posts: 34
Joined: 23 Jun 2009, 21:00

How to procedurally generate parameter names from column values

Post by foshea » 16 Jun 2021, 19:58

I'm trying to take values from the parameter file defined in elegant's &run_setup and create data parameters for insertion into the .twi file also created by elegant.

(As a note, the word parameter is used with 3 different meanings in this question. I'm trying to be really careful with how I use it, so I hope that this question even makes sense.)

To start, I can filter the parameter file for the rows that I want:

Code: Select all

sddsprocess elegant_run.par -pipe=out -filter=col,ElementOccurence,1,1 -match=col,ElementType=KQUAD -match=col,ElementParameter=K1 -print=column,ParameterName,%s.K1,ElementName | sddsprintout -pipe=in -columns=*
That produces the following output (note that these are data columns):

Code: Select all

    ElementName    ElementParameter  ParameterValue     ElementType    ElementOccurence    ElementGroup      ParameterName
----------------------------------------------------------------------------------------------------------------------------
        MDL-Q.00H                K1    6.317892e+00             KQUAD                 1                        MDL-Q.00H.K1
        MDL-Q.01H                K1   -7.409871e+00             KQUAD                 1                        MDL-Q.01H.K1
        MDL-Q.02H                K1    2.355447e+00             KQUAD                 1                        MDL-Q.02H.K1
        MDL-Q.03H                K1    1.428212e+00             KQUAD                 1                        MDL-Q.03H.K1
        MDL-Q.04H                K1   -2.300000e+00             KQUAD                 1                        MDL-Q.04H.K1
        TLA-Q.00H                K1    3.514338e+00             KQUAD                 1                        TLA-Q.00H.K1
        TLA-Q.01H                K1   -4.912866e+00             KQUAD                 1                        TLA-Q.01H.K1
        TLA-Q.02H                K1    4.312209e+00             KQUAD                 1                        TLA-Q.02H.K1
        TLA-Q.03H                K1   -5.817510e+00             KQUAD                 1                        TLA-Q.03H.K1
        TLA-Q.04H                K1    4.018223e+00             KQUAD                 1                        TLA-Q.04H.K1
        NLA-Q.00H                K1    3.095999e+00             KQUAD                 1                        NLA-Q.00H.K1
        NLA-Q.01H                K1   -4.286920e+00             KQUAD                 1                        NLA-Q.01H.K1
        NLA-Q.02H                K1    8.538818e-01             KQUAD                 1                        NLA-Q.02H.K1
        NLA-Q.03H                K1   -1.962864e+00             KQUAD                 1                        NLA-Q.03H.K1
        NLA-Q.04H                K1    3.700000e+00             KQUAD                 1                        NLA-Q.04H.K1
        NLA-Q.05H                K1   -3.700000e+00             KQUAD                 1                        NLA-Q.05H.K1
        NLA-Q.06H                K1    3.700000e+00             KQUAD                 1                        NLA-Q.06H.K1
        NLA-Q.07H                K1   -3.700000e+00             KQUAD                 1                        NLA-Q.07H.K1
      SWA-Q.00H.4                K1    6.773705e+00             KQUAD                 1                      SWA-Q.00H.4.K1
      SWA-Q.01H.4                K1   -2.937383e+00             KQUAD                 1                      SWA-Q.01H.4.K1
      SWA-Q.02H.4                K1    4.243246e+00             KQUAD                 1                      SWA-Q.02H.4.K1
      SWA-Q.03H.4                K1   -6.747316e+00             KQUAD                 1                      SWA-Q.03H.4.K1
      SWA-Q.04H.4                K1    6.468838e+00             KQUAD                 1                      SWA-Q.04H.4.K1
      SWA-Q.05H.4                K1   -2.636109e+00             KQUAD                 1                      SWA-Q.05H.4.K1
What I want is a set of data parameters, one for each row in the above table, that are constructed such that I have ParameterName = ParameterValue. For example: MDL-Q.00H.K1 = 6.317892e+00 or SWA-Q.05H.4.K1 = -2.636109e+00. Briefly, I want to use the rows in the data columns to procedurally generate parameters. After that I want to put these data parameters into the elegant_run.twi file, but I think I can do that with sddsxref or some such.

A possibly related comment and question is that I find parts of the SDDS toolkit guide to be incomprehensible. For example, the following text appears in the instructions for sddsprocess -define: "This and other properties of the new column or parameter may be altered by giving definitionEntries, which have the form fieldName=value; fieldName is the name of any namelist command field (except the name field) for a column or parameter, as appropriate." There are words in this block of text that make me think that these commands are exactly what I'm looking for, but I can't find any part of the manual that defines what a "fieldName" is what the "name field" is, or what "definitionEntries" are. Is there a different source where this jargon is defined?

foshea
Posts: 34
Joined: 23 Jun 2009, 21:00

Re: How to procedurally generate parameter names from column values

Post by foshea » 16 Jun 2021, 20:23

I suppose I should add what I tried to get what I want. The following appears to be one step away from ParameterName = ParameterValue:

Code: Select all

sddsexpand hebt_run.par -pipe=out | sddsprocess -pipe -filter=par,ElementOccurence,1,1 -match=par,ElementType=KQUAD -match=par,ElementParameter=K1 -print=par,ParameterName,%s.K1,ElementName | sddsprintout -pipe=in -parameters=*
But I have no idea where to go from here.

michael_borland
Posts: 1927
Joined: 19 May 2008, 09:33
Location: Argonne National Laboratory
Contact:

Re: How to procedurally generate parameter names from column values

Post by michael_borland » 16 Jun 2021, 22:41

Here's an example of how to do what you want. The command you are searching for is sddstranspose.
A possibly related comment and question is that I find parts of the SDDS toolkit guide to be incomprehensible. For example, the following text appears in the instructions for sddsprocess -define: "This and other properties of the new column or parameter may be altered by giving definitionEntries, which have the form fieldName=value; fieldName is the name of any namelist command field (except the name field) for a column or parameter, as appropriate." There are words in this block of text that make me think that these commands are exactly what I'm looking for, but I can't find any part of the manual that defines what a "fieldName" is what the "name field" is, or what "definitionEntries" are. Is there a different source where this jargon is defined?
The definition entries refer to the properties of, for example, a column as defined in the SDDS header. I agree this is a bit arcane, since we don't expect people to know the header syntax. The field names are any property of a column or parameter, e.g., units, type, symbol, and description, to name the most commonly used.

For example, assuming there's a column called x with units of meters, one could define a single-precision floating point ("float") column y=x^2 as follows

Code: Select all

sddsprocess ... -define=column,y,"x x *",units="m$a2$n",description="x squared",type=float


--Michael
Attachments
chromCorrection.zip
(2.71 KiB) Downloaded 104 times

foshea
Posts: 34
Joined: 23 Jun 2009, 21:00

Re: How to procedurally generate parameter names from column values

Post by foshea » 17 Jun 2021, 09:22

Ohh wow, thank you for the example and the explanation!

In the hopes a saving future readers some time, here is the relevant script. Very helpful for the -print command, too!

Code: Select all

#!/bin/bash

# sddsprocess: select rows for sextupole's K2 values
#              print a new string column containing a "tag" identifying the data
# sddsconvert: retain just the Tag and ParameterValue columns. We now have a table that is 2xNumberOfSextupoles,
#               where one of the columns is a string and one is a numerical value.
# sddstranspose: transpose the table so that it has a column for each sextupole
# sddsexpand: convert the per-sextupole columns into parameters
# You can inspect step[123].sdds to better understand what's happening, but you don't really need the tee commands.

sddsprocess run.param -pipe=out \
            -match=col,ElementType=SEXT -match=col,ElementParameter=K2 \
            -print=col,Tag,%s#%ld.%s,ElementName,ElementOccurence,ElementParameter \
            | tee step1.sdds \
    | sddsconvert -pipe -retain=col,Tag,ParameterValue \
            | tee step2.sdds \
    | sddstranspose -pipe \
            | tee step3.sdds \
    | sddsexpand -pipe=in temp1.sdds

# Transfer the parameters to the .twi file
sddsxref run.twi temp1.sdds -leave=* -transfer=param,*.K2

# Print the K2 values along with the chromaticities
sddsprintout -param=*.K2 -param=dnu?/dp  run.twi

# Clean up
# \rm temp1.sdds step?.sdds

Post Reply