Elegant users forum

I'm looking to build a few variants of elegant, so I thought I'd double check to see if there's a neater solution to the problem I've encountered:

I'm building this on an HPC system (a RHEL variant, so rpm-based), and the pre-requirement to install several rpms on every compute node in the cluster is counter to best practice - doing that for every application we install would lead to very large node build images, which we want to avoid. Normally we'd build the application and any dependent libraries on the cluster's network filestore. When I build the SDDS dependency for elegant, I hit a problem with the requirement for GSL - though I suspect I'll have issues with other libraries too. Ideally, I'm looking for:

1) Some way to point the elegant/SDDS build at a pre-built version of GSL outside the standard location

or

2) Some way to build GSL alongside elegant/SDDS

Option 1) doesn't look doable without some serious hacking of the Makefile's internal logic (simply adding -L and -I options isn't enough it seems)

For option 2) it looks like the Makefile has some logic to auto-detect GSL either as a subdirectory or at the same directory level as the SDDS source code, but I hit the same issue that the makefile isn't adding a suitable include path for either location even when they're detected.

Am I missing an easy option here (or some documentation)? My current workaround is to take a compute node offline, temporarily add all the necessary rpms and iteratively test the build applications on a differrent node to work out which dynamic libraries I need to copy across to the network filestore so that it will run without needing those rpms.

Regards,
Mike

Hi Mike,

You can build against dependencies installed on the cluster’s network filesystem. Installing the development RPMs on every compute node is not required.

We use this approach on Swing, a cluster here at Argonne. Both SDDS’s Makefile.rules and Elegant’s Makefile.rules contain Swing-specific settings that you can copy and adapt.

There are two parts to those settings: a block specifying library locations, and a later block adding include directories through EXTRA_INC_DIRS.

For your cluster, copy the Swing library block in both files, replace the library paths with your shared-storage locations, and change the condition that selects it. Swing is detected through BASH_ENV; an explicit make variable may be simpler for your installation. For example, a minimal GSL configuration would be:

#Makefile.rules
ifeq ($(MIKE_CLUSTER_BUILD),1)
GSL_PREFIX = /shared/apps/gsl
GSL_LIB = $(GSL_PREFIX)/lib/libgsl.a
GSLCBLAS_LIB = $(GSL_PREFIX)/lib/libgslcblas.a

# Adapt other dependency settings from the Swing block as needed:
# LZMA_LIB, HDF5_LIB, FFTW3_LIB, GD_LIB, TIFF_LIB, PNG_LIB, etc.
endif
Place this alongside the existing Swing library block, before the code that calculates EXTRA_INC_DIRS. Use the actual library filenames and change lib to lib64 where appropriate.

Specifying the library files directly is useful because the Makefiles perform their own library discovery. Adding -L alone does not change that discovery.

For a normal installed GSL layout, the current rules automatically derive /shared/apps/gsl/include from the library paths above, so no additional GSL include setting is necessary. For dependencies whose headers are elsewhere, copy and adapt the later Swing include block:

#Makefile.rules
ifeq ($(MIKE_CLUSTER_BUILD),1)
EXTRA_INC_DIRS += -I/shared/apps/some-library/include
endif
Put these additions after the automatic include-path calculations, near the existing Swing include settings. Use += to preserve the automatically generated paths.

You can then select your configuration with:

make --j MIKE_CLUSTER_BUILD=1

The example selects static GSL archives, which removes the runtime dependency on the GSL shared libraries. For any dependencies you link dynamically, make their shared-library directories available on the compute nodes through your cluster modules or runtime library paths. The Swing settings also show examples of embedding runtime search paths using -Wl,-rpath.

Regards,
Bob