Batch processing with (embarrassingly) parallel workloads affords numerous possible architectures all with their own tradeoffs and challenges. We will discuss the various architectures we've employed at the RACF/SDCC to tackle those challanges and our experiences with them
The historical implementation of scientific computing infrastructure within the Diamond Light Source has been to support the initial stages of data acquisition, from immediate sample validation, to the first stages of data processing. For many users, once the visit period had completed there was little if any continued interaction with the data stored at Diamond.
As data volumes increase it...
In order to better support the scientists using STFC Facilities (including The Diamond LightSource, the ISIS neutron source and CLF laser Facility) STFC's Scientific Computing Department has been developing the Data Analysis as a Service platform.
DAaaS brings together the facilities data already archived by SCD with a flexible, extensible platform to deploy scientific workflows on STFC's...
Abstract: We present a general overview of the challenges and opportunities in marshaling computational intensity around bursts of data generated in a (mostly) scheduled manner. Where computation “fits” in the data analytic pipeline (between detector and actionable knowledge) is an important architectural concern for advanced instruments with bursty data. Design boundary conditions include...
LCLS and SLAC have done extensive analysis to determine the facility computing needs over the next decade. Based on the set of experiments currently planned and based on today’s understanding of the computing and data requirements we estimated the computing demand from LCLS-II. This presentation describes the methodology adopted for deriving the computing rates, throughput and storage for...
This presentation will give a short overview of the current setup and operation for the DAQ and offline analysis part. The second half will focus on current activities/improvements for the storage and online data analysis services.
A short update of the Status of the NSLS-II computing situation, its challenges and futures. Including data retention, intentions, remote access requirements, remote control now and further, Real time cluster needs, experimental support and post experiment analysis.
Some discussion of the things that work well and some things that don't.
