Data collected at APS beamlines may be analyzed locally or it may be transferred to the experimenters home institution for remote processing (transferred via the network, on tape, or removable disk drive). APS CATs may also transfer their data to Argonne's MCS IBM SP-2 for analysis. The SP-2 is a massively parallel system made up of 128 RS6000 nodes running AIX. This represents one possible answer to the analysis dilemma. It would relieve congestion on local beamline systems and could be an answer to the problem of how to handle the reprocessing of data without further burdening local beamline systems.
The SBC-CAT is a paradigm for this strategy. SBC electronic detectors generate 18MB images. Each three-dimensional data set consists of between 250 and 1,000 images, which are recorded in a non-stop data stream (4.5 -> 18 GB), which may be acquired in as little as 15 min -> 1 hr. Although d*TREK software, by design, allows for the distribution of data processing tasks to enhance overall data analysis throughput, data analysis is currently the rate limit for the SBC facility. (Give some numbers here). Improving data processing speed is the most important ongoing development task for SBC-CAT.
At the present time, d*TREK is implemented on the SMP file and compute server, configured with four 250 MHz processors and 384 MB of memory. SBC developers have begun tests with the IBM SP-2, a massively parallel processor at Argonne. D*TREK has been compiled on the SP-2 (AIX and XlC compiler). Three-dimensional data from the SBC are transferred to the SP-2 via the ATM OC-3 fiber link. The images are subdivided into smaller sub-images and processing of the sub-images is distributed across multiple nodes of the SP-2. Without significant rewriting of the d*TREK code, this approach has resulted in substantial acceleration of diffraction data processing.
The current work focuses on developing highly parallel algorithms and methods, which can more effectively exploit the parallel architecture of the SP-2 and other Supercomputers. At present we are working on ports to the IBM SP-2 and Origin 2000. Due to the portability of the code it is possible to port the code to other Supercomputers.
(posted 21-Oct-97 jw)