Event ID: 37629
We are pleased to announce the next Workshop on Accelerator Operations, WAO2018, will take place at the Charles B. Wang Center of Stony Brook University in Stony Brook, New York, USA, from September 30 to October 5, 2018.
This will be the 11th WAO, since its inception in 1996 at Jefferson Lab in Virginia.
WAO2018 is hosted by Brookhaven National Laboratory’s two accelerator facilities: the Collider-Accelerator Department (C-AD), in the Nuclear & Particle Physics Directorate and the National Synchrotron Light Source II (NSLS-II), in the Energy & Photon Sciences Directorate.
The objective of the Workshop is to create an open forum for those involved in daily accelerator operation to present, discuss, and exchange ideas with their peers from around the world. Operators from facilities of all sizes gather to collaborate on topics such as accelerator operation, performance, organization, maintenance, documentation, software, control room layout, remote operation, and safety. It also serves the accelerator operations community as a means to establish benchmarks by facilitating comparisons of methods, efficiencies, costs, reliability, beam quality and other performance measures.
The workshop style atmosphere is organized with scheduled talks, a poster and demonstration session, and discussion periods; along with the opportunity for further discussion at breaks and meals, it allows for interpersonal communications not commonly found at conferences.
The International Program Committee’s preliminary workshop agenda is now published online:
The website also includes information on registration, travel and directions, accommodations, abstract submissions, and other local information.
On behalf of the International Program Committee and the Local Organizing Committee, I invite you to join us on Long Island, New York in September for WAO 2018!
Please forward this mail to new interested WAO2018 participants, or contact the Local Organizing Committee WAO2018@bnl.gov in order to subscribe to future mailings.
If you wish to unsubscribe from the WAO mailing list please inform Anna Petway from the Local Organizing Committee, WAO2018@bnl.gov .
Sincerely,
Gregory J. Marr
IPC Chair/LOC Host
A welcome address from WAO hosts, via the administration of the Collider-Accelerator Department at Brookhaven National Laboratory
A welcome address to the WAO from the administration of NSLS-II facility at Brookhaven National Laboratory.
Welcome and orientation to WAO 2018, from International Program Committee Chair
In 2011 FNAL began a transition from a multi-program colliding beams facility to a multi-program fix target facility with an emphasis on Neutrino and Muon physics. This transition is now complete...how has it affected the laboratory, and the operations department? What's in store for the future? This is "How We Do Business" now.
An electron-positron energy-asymmetric collider, SuperKEKB, is under commissioning. This project is expected to be able to elucidate the flavor physics of elementary particles with 40-fold improved luminosity compared with the previous project, KEKB. Several operational shifts are organized for individual responsibilities for beam operation, safety, or machine domains. Not only a dedicated shift crew but also most of staff members attend one of the shift groups, as it is believed that common understanding of both machines and operation would improve them better. The injector delivers the beam into two light source rings as well. Meetings are held to support coherent operation between those shifts.
The International Fusion Materials Irradiation Facility (IFMIF) aims to provide an accelerator-based, D-Li neutron source to produce high energy neutrons at sufficient intensity and irradiation volume for DEMO materials qualification. The IFMIF/EVEDA project, which is part of the Broader Approach (BA) agreement between Japan and EURATOM, has the mission to work on the engineering design of IFMIF and to validate the main technological challenges, which among a wide diversity of hardware includes a 125mA CW deuteron accelerator up to 9 MeV mainly designed and manufactured in Europe.
The Linear IFMIF Prototype Accelerator (LIPAc) under installation and commissioning at Rokkasho Fusion Institute, QST (Japan) is entering in its second commissioning phase (D+ beam, 125 mA, 5 MeV) which is to validate the design of the RFQ, the MEBT, the Beam Diagnostics and the RF System.
The organization of the LIPAc Operation Team has undergone a number of challenges to reach the main goal of full assembly, integration and operation of the entire accelerator. The different accelerator subsystems have been provided by Europe as an in-kind contribution and installed by Japan in close cooperation with the European experts, but also to manage efficiently and safely the operation of the machine in order to meet the objectives fixed by the project’s governance. An Event Management system has also been developed to support the daily operation and the management of events through preventive and corrective maintenance plan and the feedback to the design.
The Machine Protection System (MPS) of the Japan Proton Accelerator Research Complex (J-PARC) is the system that aborts a high intensity proton beam pulse to prevent unnecessary radio activation of accelerator components when it receives a failure signal of the device constituting the accelerator. Because of improvements of the first extraction (FX) device, it was recently able to abort the beam pulse immediately in the FX. Therefore, it is also necessary to speed up the MPS of each device consisting MR synchrotron. The RF System of J-PARC consists of a cavity, a driver amplifier, a final amplifier, an anode power supply and the Low-Level-RF (LLRF). And the Programmable Logic Controller (PLC) controls the MPS of RF system. When the system detects any failure signals, the corresponding system sequentially stops, and the failure information is transferred to the main MPS system in the accelerator control system. In the case of the RF system, it was effective shortening the program scan time by model update and optimizing the ladder program for the speed-up. We described the outlines of the J-PARC MPS and how to speed up the MPS of the RF system of J-PARC MR.
Cern accelerator complex is composed of many operation tools and systems. These tools are vital to achieve high performance and reliable operation. Operation Tools are interfaced with the middleware. This will allow the operator to diagnose and optimise high level accelerator machine parameters. A good operation skill is needed to use these devices. This talk will describe the role and the nature of these devices for an efficient operation.
The main tasks of SuperKEKB operators are beam tuning, safety management and operation logging.
In order to ensure these tasks, the operators may create some applications.
Several applications were created by operators under the guidance of staff.
Meanwhile, some applications, created by the operator himself, to ensure work.
In this time, we will introduce various applications created by SuperKEKB operators.
The Galactic Cosmic Ray (GCR) Simulator project at Brookhaven National Laboratory’s NASA Space Radiation Laboratory (NSRL) was developed to assess the biological risks of exposure to the wide variety of ion species (and energies) that would be experienced on manned space missions outside the protection provided in Low Earth Orbit. In recent years tools have been developed to enable rapid, reliable and fully automated switching between any combination of the wide assortment of ion beams available to NSRL. In the summer 2018 NSRL run the first full spectrum GCR (32 separate ion/energy configurations) biology experiments were successfully completed. The software tools that control and manage the rapid changes to the accelerator complex for GCR running are largely operator created. This presentation will give an overview of the NSRL/GCR project and discuss the software tools and techniques that make it possible as well as how the system is integrated with the other programs at accelerator complex.
Orbit Display is an operator-created application that displays real-time data from over 100 beam position monitors. Originally written for the Linac Coherent Light Source (LCLS) accelerator, it has also been successfully ported to a different machine and control system at the European XFEL. This talk will discuss how an operator perspective influenced the design, the architecture of the application, the performance compared to older tools, and the experience of adapting it to the needs of a different accelerator.
From its start, the NSLS-II controls system was designed in stages; Engineering controls, followed by Physics applications and pages, and finally an Operations layer for the control-room. Of the three layers, the Engineering pages have remained predominantly static, from the mindset of machine controls being “complete” and not modifying a system that is in-use. Operations pages and tools, conversely, have continuously evolved to suit a growing list of needs as new hardware comes online. This comes from active participation in development by operators, a dedicated ops-developer, and constant feedback from the operators who use the tools.
This year, several engineering issues have forced groups to double their efforts monitoring data and identifying conditions before they become problems. With legacy Engineer controls, this process was tedious, time-consuming, and some trends were easily missed. By breaking the divide between controls ‘layers’ and making Operations-styled tools for Engineers, we have streamlined that monitoring process, saving system engineers time and increasing the effectiveness of our responses to prevent those recurring issues. This talk presents the motivations, techniques, and quantified-results of using Operations-developed, user-friendly UI to meet technical Engineering-level challenges.
One of the main issues of the linac operation problems in the PLS-II is injection efficiency for top-up mode to keep radiation safety levels. The energy spread of the injection beam is the main beam parameter to effect injection efficiency. So we try to develop several operational software tools to reduce the amount of energy spread. And also beam energy jitter is another parameter for injection efficiency. We tried to develop related operational software tools about these parameters. So this report introduces these operational software tools.
Rebuilding and updating the Fermilab Main Control Room to increase efficiency and aesthetics, and to enable easier future upgrades.
The Cern Control Centre has been built to control the CERN accelerator complex in the LHC era. The main objective was having all specialists in the same room to efficiently operate the LHC beams, among all other physics programs. The main challenge of this project was
centralizing 4 different operation units working for the same objectives. The choice of the layout was paramount for an efficient human interaction, while working for several machines and accelerator studies. The excellent results of the LHC prove the quality of the project. This presentation will describe the technical and human challenges we had to face to fulfill all requirements.
INTRODUCTION: designed and built in the first decade of the new Millennium, the fully digital PSI Main Control Room has now been in Operation for over a decade.
During this period demands on infrastructure and services have constantly risen.
METHODS: Designed for an estimated thermal power of 12kW, our cooling systems today have to handle 18kW of waste heat.
The installed system with cooling ceiling and air condition has come to its limits.
Facing this challenge we we have identified three possible solutions:
• lowering the energy consuption
• improve our cooling system
• avoid heat disposal to room air
RESULTS: every one of the examined possible approaches has its own challenges.
Furthermore we came to the conclusion, that even small changes on the control room infrastructure cannot just be done by “walk-in service” but have to be well coordinated, keeping in mind, the butterfly-effect which can cause effects, affecting the big picture.
However, it became apparent that the planning of the next-generation control room must start today and iterative extension of components (as has been done up till now) is reaching its limits.
DISCUSSION & CONCLUSIONS: regardless, which of the examined solution we will apply, there is no quick&simple, and foremost cheap solution in sight.
FERMI@Elettra is the first fully seeding-based FEL. This talk is an overview of how the Control Room Operator see and control the FEL Photo Injection Laser and the FEL Seed Laser during experiment and beam preparation. Details of activities, panels and troubleshooting will be given.
The CCF at NSCL has been operating for decades. Now, the first sections of the FRIB accelerator are being commissioned and the NSCL operators are starting to take on FRIB shifts. While both facilities use the same control system tools, the difference in accelerator complexity and control system interface tools make for a very different operating experience. This contribution will look at how the NSCL operators are adapting to this change, what they like and dislike about the tools, and how they can contribute to creating an effective combination of the familiar and the new.
Operations at National Synchrotron Light Source II (NSLS-II) require the coordination of many critical systems and subsystems. The majority of graphical user interfaces (GUIs) for these systems were developed individually as needed during the early stages of commissioning as the facility was coming online. As the NSLS-II has become an operational facility, however, ease of use has necessitated condensing much of the critical information from these GUIs into a single, cohesive interface that has been named the "OpCenter." The OpCenter is a tool available to all accelerator division personnel, but is used most by Operations staff, and runs on the Control System Studio (CSS) platform. It condenses information from the most commonly used critical systems into a palatable format with links that grant quick access to the corresponding expert interfaces. It is continually optimized as more hardware and applications are integrated into day-to-day operations. The OpCenter has drastically reduced the need for copious interfaces which traditionally required a higher demand on processing power and were often cumbersome to navigate. It has optimized diagnostics and recovery in the event of failures and helped improve overall reliability of the NSLS-II.
Complex accelerator facilities necessitate complex controls. Often these complex controls impede quick and efficient responses to changing operational requests and accelerator downtime. We present an implementation of a physical interface to control the operational state of the Fermi National Accelerator Laboratory accelerator complex. By lowering the barrier of execution this interface encourages operators to more readily configure the complex into the desired state, increasing operational efficiency and promoting energy savings.
Accelerator operation requires schedule work 24/7, year around. We take advantage of website technology to create the applications for schedule work.
- Operators work schedule is an interactive system. Operators can see planed schedule for whole year, and input actual time off or change in the system. The application will generate a timesheet according to the records.
- Experiments technical support schedule system is view purpose only for operators. Support experts setup their schedule following the experiments and operators can get contact personal names, phone numbers right away when they need.
The applications are based on Linux server and Postgresql database. We created and maintain the whole system.
Defining effective organizational structure for research labs is a contextually challenging issue in the field of systems engineering. Maintaining a balance between subsystem autonomy and system cohesion has proven difficult to achieve throughout R&D industry. Multiple system perspectives are always present across divisions and project based funding contributes to competing priorities. Because of the complexity of group interactions, communication channels, and emergent behaviors, the actual system is virtually impossible to represent. These systems are often reduced to a simpler abstraction in a hope to deal with the complexity. People then hope (often naively) that the constructed policies, procedures, and organizational charts will provide adequate utility to manage the complex system.
Jefferson Lab uses a hybrid of functional and matrixed organizational structures to support reliable accelerator operations. Divisions within the laboratory include engineering, physics, and accelerator as well as largely independent cryogenics and SRF groups. Each has their own required skill sets and priorities. Constant emergent issues result from separate divisions managing their own engineering staff while sharing system responsibilities with groups from other divisions. Boundaries are often ambiguous and challenge the viability of the system’s organizational structure presenting significant challenges to work planning and management.
Key concepts of organizational theory, system theory, and viable system modeling provide insights to address these challenges and will be presented in the context of accelerator and lab operations.
Diagnostic tools in general are the eyes of the operation group to run an accelerator. It is of a fundamental importance that such instruments not only fulfil the technical requirements but also allow an efficient and fluid use in daily operation. In fact, the operation of a complex accelerator, such as the LHC, is a continuously evolving task driven by challenging obstacles that may arise, therefore the endowed instrumentation should provide the flexibility to evolve beyond design specifications. The continuous collaboration between operation and instrumentation teams is mandatory in this process. In the following, the case of the LHC synchrotron radiation monitors is chosen as an example to present the continuous upgrades the instrument went through to help operations in the challenge of the beam emittance characterisation, a crucial parameter in the performance assessment of the LHC.
Errant beam loss in the SCL slowly degrades the amount of energy the RF cavities can reliably provide. Using a Differential Current Monitor (DCM) system, SNS has been able to reduce this loss significantly. Operators perform a key role in identifying and collecting data for events captured by the DCM system. The development and future improvements to the DCM system will be discussed.
The injection efficiency is an important parameter for synchrotron light sources operating in top-up mode like the 3.0 GeV ALBA synchrotron light source located in Barcelona, as the Storage Ring (SR) is periodically refilled with the Front Ends open while the users take data continuously. A bad transmission makes injections longer and compromises data collection during refilling. At ALBA, the transfer efficiency from the Booster to the SR had always been an issue as it was not as stable as expected and to keep it at adequate levels was a time consuming operation. Hence, the Operations section together with the Beam Physics section? has successfully completed a project consisting of the improvement of the available diagnostics to be able to properly align the beam in the Booster-To storage ring (BT) transfer line, and the development of high-level software tools to control the BT transmission. Thus, now the operators have? the means to properly diagnose and correct the specific causes of the injection efficiency variations coming from the BT. Finally, a standard operating procedure has been defined to optimize and recover the BT transmission, allowing to react faster and keep it above 80% in routine operation.
Like all complex systems, the operation of an accelerator or a chain of accelerators is heavily guided by the physics that governs motions of charged particles. While this has been the theme for most of the cases, it has been also shown that an experienced operator is indispensable for achieving best beam performance for the users even if she is not at all fluent in the most sophisticated non-linear beam dynamics. So, what is the art of the accelerator operations and where is the balance between machine physics and operations? This paper will present the author’s experience and thoughts along these topics.
At SLAC; there is a very active machine development program that constantly strives to expand the capabilities of LCLS-I. Furthermore, users often require very specialized modes of running that require a high level of expertise to set up. This necessitates a high level of cooperation and knowledge-sharing among operators and physicists, especially as we begin to commission two new injectors for LCLS-II and FACET-II. Operator involvement with special machine configurations/experiments and commissioning will be discussed, as will the interaction of operators with physicists in general.
Operating a large accelerator safely and reliably on a day-to-day basis is a clearly distinct task from performing R&D studies for future accelerator upgrades or projects. Even though the required skills and culture of both worlds can be seen as contradicting at first glance, the benefits of mutual understanding, exchanges and teamwork between beam physics and operation teams has been crucial for reaching the short term and long term goals of the CERN accelerators. This contribution will describe the experience of how operators and physicists have interacted efficiently during the commissioning and routine operation of the LHC complex and the main lessons learnt from a CERN beam physicist’s perspective.
After a 2-years shutdown period, in which extensive refurbishment and modernization measures were carried out (including civil construction work in preparation of the GSI connection to FAIR), the entire GSI accelerator chain was put back into operation for the research program of FAIR Phase 0. Additionally, the largest part of the accelerator facility should be controlled with the new FAIR control system for the very first time. As the control system was renewed at all levels (frontend, timing, machine-modeling and applications), it was clear that this restart would be significantly different from the usual setup and commissioning phases after standard shutdowns. For this reason, the recommissioning was planned similar to a commissioning of a new accelerator. In this context, the first concepts for FAIR commissioning have been tested.
This talk is about the planning, the process, difficulties and successes from the dry-runs to the first beam on target.
ESS will start commissioning summer 2018. We present the process we follow in order to ensure a safe commissioning, including the license process, the Ready Review process, training of operators, and the creation of an Operations Team.
After the Long Shutdown 1 (LS1) a working group with the mandate to review the LHC injector re-commissioning strategy was created beginning of 2015. Now with 4 years experience restarting the Proton Synchrotron Complex machines, the team is preparing the restart after Long Shutdown 2 (LS2), which will take place during 2019-2020, hosting an extensive upgrade campaign for the LHC injectors. Within the framework of the LHC Injector Upgrade (LIU) project Linac4 will replace the current Linac2 as injector for the Proton Synchrotron Booster (PSB), and in addition the PSB extraction energy will be increased from 1.4 GeV to 2 GeV. This involves changing an important fraction of the machine installations and controls.
The presentation will explain the preparation and organisation of the 2020 Linac4 and PSB re-commissioning and how we plan to take up the challenge to make the machines fully operational again in time by applying our strategy.
The superconducting, recirculating electron accelerator S-DALINAC has been built in the 1980s at TU Darmstadt. The first beam was achieved in 1987 and since 1991 it is in full operation. The operation as well as further development and improvements of the machine are done mainly by students. This educational aspect leads to exchanging operators after a maximum of a few years. A lot of effort is done to conserve knowledge over the time. The training of new operators is essential, as they are involved not only in the operation but also in the maintenance of the machine. This contribution will give an overview on our way to operate an accelerator at a university and how we constantly develop our machine to keep it up-to-date with the needs of the experiments.
Accelerators and beam transport systems typically contain numerous beamline diagnostics to measure beam particles distributions, beam emittance, beam energy, to monitor beamline vacuum and beam loss etc. As with any aging beamline system, there are legacy diagnostics that are still effective, but replacements are not essential until inevitable. It was observed that diagnostics, including legacy and other diagnostics can effects the beam production time if not thoroughly evaluated prior to use. To mitigate beam down time, it becomes necessary to pre-check the devices (insertion movement, data collection, beamline vacuum etc.) during any maintenance period. Here we present several experimental examples for demonstration of beamline diagnostics pre-checks to reduce adverse impacts on production time.
*Work supported by the United States Department of Energy, National Nuclear Security Agency, under contract DE-AC52-06NA25396.
LA-UR-18-26338
The linear accelerator at the SLAC National Accelerator Laboratory has produced electron beams since 1967. Over the years, there have been many generations of upgrades, additions, removals, and reconfigurations of both the accelerator hardware and control system, but never a complete tear-down and replacement of the accelerator - bits and pieces of every generation of SLAC equipment remain in use today. These systems present many interesting challenges for accelerator operators: reliability is often low, operating the equipment depends on user interfaces that are unfamiliar. Most importantly, understanding how and why the systems work depends on understanding their history, even though many of those historical building blocks are no longer in use. This presentation will give a guided tour through some of these systems, the frustrations they cause, and tools that operators have developed to make the systems easier to operate.
A 100-MeV proton linac at the KOMAC (Korea Multipurpose Accelerator Complex) is composed of a 50-keV microwave ion source, a 3-MeV four-vane-type RFQ, a 100-MeV DTL and 10 target stations for proton irradiation on samples from many application fields. The linac was commissioned in 2013 and the user service started in July 2013 with delivering proton beam to three target stations: one for a 20-MeV beam and the other for a 100-MeV beam.
The existing control room for the linac and beam lines has been upgraded in various ways over the last 5 years, but there was a lot of interference because accelerator operators and data analysts use the same computer screen and keyboard in one space. The new control room was designed to eliminate congestion caused by accelerator operators for beam service and analysts for machine study. We named the new control room Machine Study Room (MSR). This spatial separation has provided increased flexibility within the control room to accommodate operators, physicists, and specialists. In this paper, we describe the history, concept, and status of this project.
*This work has been supported through KOMAC (Korea Multi-purpose Accelerator Complex) operation fund of KAERI by MSIT (Ministry of Science and ICT)
The Neutron and Muon source at RAL is a large scale facility, conceived over 30 years ago, consisting of a pulsed 800MeV rapid cycling synchrotron. The facility has tungsten-tantalum targets with a comprehensive suite of 36 different detectors, spread across 2 target stations.
The poster will document recent changes and its challenges to the Alarm Investigation Team (AIT). This team (consisting of accelerator Operators and Security personnel) investigate various alarms, including fire alarms.
Our poster will focus on 3 main areas:
• Procedures
Why new procedures had to be written and how this was achieved.
• Fire detection equipment
Highlighting some of the new equipment and defining why this equipment was essential to protect our accelerator from fire.
• Team Structure
A new promotion system will be explained in more detail, together with its advantages. It is called “fluid complementing”. This concept is currently in process at RAL, designed to provide a more accessible route to promotion for operators and in turn bringing increased cover for the AIT.
Within each area, details will be presented of the changes that have taken place. We will show some of the difficulties faced, solutions found and how this change was managed.
Knowing the transverse emittance through the acceleration cycle is imperative for efficient and successful operations. Relative measurements provide details of emittance growth problems while absolute measurements provide beam expectations upstream and downstream of the measurement. Both measurements are needed to fully understand the emittance as a whole. We learn the emittance by measuring the width of the beam and then translating that into an emittance knowing the beta function at the measuring instrument. Thus the knowledge of the emittance is proportional to the knowledge of the beta. One can learn the beta by distorting the equilibrium orbit of a functioning machine by adding a dipole and then measuring the resulting orbit motion. Using an ionization profile monitor local to said dipole, one attains the position-measuring capability by using the resulting centroid of the ion beam image provided. With additional standard beam measurements and parameters, operations has been able to successfully measure the local beta function at four separate locations in the Alternating Gradient Synchrotron at Brookhaven National Laboratory. The goal is to demonstrate the evolution of these measurements and compare the results to model optics.
Every RHIC physics store is analyzed, with key components updated
to a spreadsheet, to identify and monitor any improvements or degra-
dations in performance. With physics stores lasting between 50 min-
utes (optimal store length of 7.3 GeV Au in Run14) and 20 hours (store
length of 100 GeV Zr and Ru stores in Run18), manual analysis be-
comes a time consuming endeavor. To reduce the time required by
operators to update the spreadsheet data, a program was written in
Python to automatically analyze every store. An overview of the data
analyzed is given, and the programs used to analyze each store are
discussed.
A high machine availability is an important performance indicator for modern synchrotron light sources. To reach this goal of stabilizing operations by minimizing the occurrence and duration of downtime events, software tools play an important role. Reliable operation statistics are necessary, in addition to reliable tools for root-cause analysis and fault tracking. Here we describe how we do business at PETRA III and how operators are involved.
The most effective control system tools for operators are generally made by people who have experience in operations. At NSCL and FRIB, several operators are making their own CS-Studio content but are often limited by the signals available to them. One solution is to develop some degree of control system expertise within the operations group, enabling them to use the standard control system tools and processes to support their needs. This contribution will look at how operators can learn to use EPICS IOCs and Python scripts to clarify the presentation of information and streamline tasks.
Most of the world's Accelerator Laboratory provides stable beam services through numerous control devices. In addition, EPICS (Experimental Physics and Industrial Control System) software is used to acquire the normal status and data of the device through a LAN (Local Area Network). The software that plays a central role in EPICS is an Input / Output Controller (IOC) and communicates directly with the device. EPICS provides a number of tools for controlling or monitoring control devices using the Operator Interface (OPI). However, most OPIs only provide an environment that works on a PC. Therefore, when a problem occurs due to failure of a main device during operation of the beam, the person in charge of the device cannot quickly check the status of the device with the smartphone. To improve this inconvenience, we will apply the user - oriented OPI according to the redevelopment of the superconducting LLRF System at KAERI. In this paper, we propose a method to remotely control and monitor real-time device information from a smartphone as well as a PC. By observing the Web standards, we confirmed that remote devices can be controlled and monitored in various web browsers. We introduce web-based OPI using HTML5, Web Socket, and communication between IOC and web server.
Taiwan Photon Source (TPS) have been opeartoed and opened for users two years. TPS is a 3 GeV electron energy, 518 meter circumference, low-emittance synchrotron storage ring which will offer one of the synchrotron x-ray sources, provide cutting-edge experimental facilities and novel multidisciplinary scientific research. Amount of instruments and devices these must be monitored and controlled by single operator. The diagnostic tools and softwares are effective to reduce operator loading. These software will be reported in this workshop.
At the WAO2014 I presented the Operations Simulator software I have developed. The simulator provides a training and familiarisation platform for operators, students or visitors. Initially the simulator was developed to work on the main control room PCs, but this tied the ‘users’ to the main control room. To remove the control room ties the Simulator software (EPICS IOC, Cothread, and python scripts/guis) has been installed on a Raspberry PI2 (a small very inexpensive PC ). This has allowed to the simulator to run standalone and be used anywhere. A side effect of running the entire simulator on the relatively low powered Raspberry PI2 has demonstrated that using Python for Operator interfaces is a perfectly acceptable alternative to CSS as EDM is phased out at Diamond.
The first section of the ESS Linac (Ion Source and LEBT) will be commissioned during summer 2018. We will present the first results and which lessons we learned in order to prepare for commissioning the rest of the machine and how to grow the operations team.
This paper first discusses the hardware and software structure of the Hefei Light Source control system. Based upon the Hefei Light Source control system, we developed a number of high level applications for machine commissioning and operation. This dissertation reports some critical applications, including the energy matching between the injector and the storage ring, lattice calibration and optical parameter correction for the storage ring, lattice compensation for the insertion devices (IDs), and the storage ring orbit feedback.
Not so many "entry level" operators, for example with one year of experience, have enough knowledges and skills for accelerator operator. Their supervisors require high level skills as ideal operator, whereas "entry level" operators have many things to understand, gain experience and so on, and have not enough time to study. In order to eliminate the distinction between them, optimization of skills level which are required as "entry level" operators and establishment of effective training method are needed. We have been revising skills checklist, training documents and on-the-job training system.
We defined "entry level" skills checklist as knowledges and skills which should be mastered within one year, and reconfirm all the items. Then we improve our instruction material so as to cover all the items of the checklist. In addition, we carried out second phase on-the-job training about 6 months after assignment in order to compensate for the lack of knowledges and skills based on the checklist.
However it takes plenty of time to see the effect of our activity, we will continue improving our training system step by step.
In addition to shift work in the control room, the operators perform secondary activities in various specialist groups during their normal working hours.
Secondary occupation in the division Insertion Device
Outside of shift work, we are involved in the maintenance of the insertion device of SLS and SwissFEL. These maintenance can only take place in the machine shutdown, which becomes a problem of manpower allocation.
The service, which is determined by fixed maintenance intervals, includes the lubrication of the guide rails, the drive spindles and oil changes on the drive motors or the motor gearbox. This guarantees smooth operation and eliminates any deposits. Likewise, possible damage and/or wear and tear can be detected in good time during regular maintenance.
An analysis of the oil we use for the gearboxes has shown that it is possible to extend the maintenance interval by one year and then a new analysis provides information on whether the interval can be even further extended or changed. Through this analysis, the PSI saves approx. 5500 francs per saved oil change on all insertion devices.
A further task is the assembly of the new insertion devices for SwissFEL and their maintenance during operation. In the course of this work, we will also be involved in smaller projects of the department and will be able to work on and accompany them independently.
Operations at National Synchrotron Light Source II (NSLS-II) require the coordination of many critical systems and subsystems. The majority of graphical user interfaces (GUIs) for these systems were developed individually as needed during the early stages of commissioning as the facility was coming online. As the NSLS-II has become an operational facility, however, ease of use has necessitated condensing much of the critical information from these GUIs into a single, cohesive interface that has been named the "OpCenter." The OpCenter is a tool available to all accelerator division personnel, but is used most by Operations staff, and runs on the Control System Studio (CSS) platform. It condenses information from the most commonly used critical systems into a palatable format with links that grant quick access to the corresponding expert interfaces. It is continually optimized as more hardware and applications are integrated into day-to-day operations. The OpCenter has drastically reduced the need for copious interfaces which traditionally required a higher demand on processing power and were often cumbersome to navigate. It has optimized diagnostics and recovery in the event of failures and helped improve overall reliability of the NSLS-II.
The Isotope Separator On Line Device (ISOLDE) can profit from up to 60% of the protons the Proton Synchrotron Booster (PSB) can deliver. The protons are essential, to deliver the wide range of isotopes required. The machine supervisor main task is to guide the beam through the different sections of the facility and deliver it to the users. The team utilizes a variety of tools to perform the set up. In this poster a machine supervisor view of the main segments of the facility will be presented.
The Shanghai Synchrotron Radiation Facility (SSRF) has opened to user for eight years, and the performance has been improved continuously to satisfy the users’ needs. The operation performance and the reliability of SSRF have been kept in high level to provide users sufficient and stable synchrotron radiation laser. The report will focus on the operation performance of SSRF in last operation season, and the analysis of reliability is also shown in the report. In addition, the upgrade program of SSRF accelerator will be brief introduced in the report, including superbend, double-waist lattice, and electric power stabilization, the phase II beamline project with more than 20 new beamlines will be constructed with next 5 years.
. At KOMAC (Korea Multi-purpose Accelerator Complex), a 100-MeV proton linac has been started to operate since 2013. Nowadays, the constructions of the total six beamlines are completed. Among them, 20 MeV or 100 MeV proton beam has been provided to various users through the three proton beam irradiation facilities.
The proton beam irradiation facilities could provide the pulsed proton beam and the available proton energy range is from 20 MeV to 100 MeV. And also, could provide the various beam spot size according to user’s demand. The proton irradiation is performed in the air, therefore, the intensity and profile and position of provided proton beam should be measured in the air. To assure the beam quality, we have been constructed own QA procedure and has been acquired the ISO9001:2015 at 2015.
In this paper, we will introduce the operation status of the proton beam irradiation facility at KOMA, and the details on the beam diagnostics tool in these facilities will be given. In addition to, the future upgrade plan for the improved beam quality will be discussed.
FLASH is DESY's free-electron laser user facility providing ultra-short femtosecond laser pulses in the EUV and soft-X ray wavelength.FLASH is also a pilot facility for the European XFEL project and a test bed for further research and development for linear collider related superconducting accelerator technologies. Since 2014 the FLASH Linac operates two seperate beamlines, FLASH 1 and FLASH 2, in parallel. Currently a third beamline, FLASHforward, is under construction and installation. To keep the accelerator running and to provide time for installation work regularly maintenance and shutdown periods are needed. On behalf of the whole FLASH Coordination team, we describe the organization, the work flow of those periods and the hence resulting difficulties and pending issues.
Carbon-ion radiotherapy using Heavy Ion Medical Accelerator in Chiba (HIMAC) has been carried out since 1994. Over 11000 cancer patients have been treated with carbon beams having energies of between 56-430 MeV/u. The HIMAC has two injectors and provides heavy-ion beams for medical and experimental purposes. The first injector consists of two Electron-Cyclotron-Resonance (ECR) type ion-sources, one Penning-Ion-Gauge (PIG) type ion-source, the Radio Frequency Quadrupole (RFQ) linac and the drift tube linac (DTL). This injector provides carbon-ion beam for cancer therapy, and concurrently provides various ion beams such as H, He, Fe, Xe for biological and physical experiments. The 10 GHz NIRS-ECR ion-source produces the carbon ion for cancer therapy, while the 18 GHz NIRS-HEC ion-source produces C to Xe ions for experimental use. Light ions such as H and He, moreover, B and Si ions are produced from solid materials with sputtering method by the NIRS-PIG ion-source. The second injector consists of the compact ECR ion-source with all permanent magnets, the RFQ linac and the Alternating-Phase-Focused Interdigital H-mode Drift-Tube-Linac (APF IH-DTL). This injector mostly provides carbon-ion beams for experimental use. In this paper, present status as well as recent development of the injectors will be described.
At National Institute of Radiological Sciences (NIRS), carbon-ion radiotherapy has been carried out since 1994, and more than 11,000 patients were treated by now. Recently, the world's first superconducting rotating-gantry was developed at NIRS, and treatments using the rotating-gantry began since 2017. This rotating-gantry composed of ten superconducting magnets, and superconducting coils of these magnets are cooled down below 4K using compact cryocoolers. By using the cryocoolers, no liquid helium is necessary for cooling of the coils. However, these cryocoolers would require periodic maintenance. We employed the so-called “cold maintenance method”; the temperature increase of the coils can be minimized by using this method. Although the maximum temperature of the coils during the maintenance can be kept below approximately 100K, coil quench during magnet operation after the maintenance may occasionally occur in some of the magnets. Once the coil quench occurs, it would take 1~2 hours to recover the coil temperature, causing delay in treatment schedule. To overcome this issue, training of the magnets was performed in biweekly maintenance day. Having performed the training, we see almost no quench issue during treatment operation. In this presentation, we will show present status of operation and maintenance of the rotating-gantry.
Synchrotron SOLEIL is the 3rd generation French synchrotron light source. It has been in operation since 2007 providing photon beams to 29 beamlines with a maximum intensity of 500 mA, 5000 hours a year.
Since the beginning of 2018, the operation group has been migrating to JIRA Altassian Software as the unique tool for reporting failures. The tool was already used by the computing division for managing user requests, software evolutions, problems, etc. On the operation level, JIRA is already recording all to the demands of interventions and access to the tunnels. It provides a better interaction between the reporter and the support groups that are involved in the process of resolution. Anyone can report a failure related to the accelerators by creating a ticket. Here we will describe the workflow to manage an incident during its full lifetime and give a first operational feedback.
In order to make the reporting more convenient, a simplified web page has been developed by an operator. It procures an interface to anyone without knowledge of JIRA and allows us to free ourselves from errors by skipping unnecessary steps.
Automatic reporting is also made in case of failure occurring on injection or insertion devices. It is programmed in Python thanks to a JIRA plugin. Dashboards are available for all support groups reporting incident by severity level, which is a major asset compared to the previous logbook we used in terms of quality, interaction with people and review. It improves also integration between support, developers and operations.
American culture has accustomed us to working a third of a day for 5 out of 7 days. Certain times are preferred over others. Due to the fixed cost of cryogenic operations, running all day, every day is preferred for a given Run (ie the time between long maintenance periods). Staff load analysis recommends a crew of 2 Operators with a Crew Chief for oversight. How should the coverage be arranged? How should the schedule be disseminated?
At Jefferson Lab, we run three shifts a day: Owl (midnight - 8am), Day (8am - 4pm), Swing (4pm - midnight). For a given Run a Crew works the same shift. Starting on Saturday for a given fortnight (ie two week period) a Crew works 7 days on shift, takes 2 days off (Saturday & Sunday), and then spends three of the next five days training, working on projects, or filling in for other shift personnel. A second Crew works the same schedule offset 7 days. The Crew Chiefs' schedules are offset -25 hours from their Operators.
Scheduling can be chaotic. To manage the chaos, powerful, easy-to-use software is very helpful. We have chosen the commercially available, web-interfaced software, WhenToWork.
American culture has accustomed us to working a third of a day for 5 out of 7 days. Certain times are preferred over others. Due to the fixed cost of cryogenic operations, running all day, every day is preferred for a given Run (ie the time between long maintenance periods). Staff load analysis recommends a crew of 2 Operators with a Crew Chief for oversight. How should the coverage be arranged? How should the schedule be disseminated?
At Jefferson Lab, we run three shifts a day: Owl (midnight - 8am), Day (8am - 4pm), Swing (4pm - midnight). For a given Run a Crew works the same shift. Starting on Saturday for a given fortnight (ie two week period) a Crew works 7 days on shift, takes 2 days off (Saturday & Sunday), and then spends three of the next five days training, working on projects, or filling in for other shift personnel. A second Crew works the same schedule offset 7 days. The Crew Chiefs' schedules are offset -25 hours from their Operators.
Scheduling can be chaotic. To manage the chaos, powerful, easy-to-use software is very helpful. We have chosen the commercially available, web-interfaced software, WhenToWork.
Synchrotron SOLEIL is the third generation French synchrotron light source. It has been in operation since 2007, providing photon beams to 29 beamlines with a maximum intensity of 500 mA, 5000 hours a year.
2017 has been the second best year for SOLEIL with 98.7 % beam availability (98.9% in 2015) and 92 hours “mean time between failures” (MTBF) (105h in 2015). The target remains 99 % beam availability and 100 h MTBF despite the forthcoming obsolescence and aging of the facility.
We generate statistics that allow us to measure the machine efficiency. We then need tools such as online applications to monitor these statistics at anytime from anywhere. DJANGO (Python web framework) has been used to develop dedicated web pages. Moreover, following the discussions within the community on common metrics to compare similar facilities together, we present in this poster how we generate and use these common metrics.
The IFMIF/EVEDA Linear Prototype Accelerator (LIPAc) is a prototype of the projected IFMIF accelerator. The installation and commissioning of LIPAc is progressing at Rokkasho Fusion Institute, QST (Japan). This prototype is aiming at accelerating a deuteron beam of 125 mA to 9 MeV in continuous wave mode (CW). Currently, the beam commissioning up to 5 MeV, so-called ‘Phase-B’, is underway at low duty cycle. One of the important features of this accelerator is that most of the components were developed individually and delivered by Europe as in-kind contributions. While QST is responsible for the installation, the operation is done by a joint operation team Japan-Europe. In the Phase-B operation, we need to operate several sub-systems together delivered by different institutes, i.e. the injector from CEA (France), RFQ from INFN (Italy), MEBT and the RF system from CIEMAT (Spain) with utilities (cooling system, HVAC) supplied by QST. The beam instrumentation has been developed by CEA, CIEMAT and INFN and some devices are assembled on Diagnostics Plate made by CIEMAT. These sub-systems are integrated in the central control system (TS, MPS, PPS) prepared by QST. We started the beam commissioning in June 2018 after the successful integration and management of the interfacing of such a different subsystem into one accelerator system. We will present the details of the integration of different subsystems, the procedure developed to reach the first beam commissioning and some improvements needed considering the feedback of this integration phase.
The free-electron laser FLASH at DESY in Hamburg provides high brilliance FEL radiation in the XUV and soft X-ray wavelength regime for user experiments. Starting in 2016 two separate undulator beamlines (FLASH1 and FLASH2) are being operated simultaneously, driven by a common superconducting LINAC. A third beamline is in use as alternative to FLASH2 to provide high current-density electron beams for the plasma-wakefield acceleration experiment FLASHForward. The accelerator is operated 24 hours a day, 7 days a week by a shift crew working in a three-shift system. In addition a machine coordination team decides on the schedule, machine upgrades, maintenance tasks, etc. The coordination team also handles the follow-up of technical issues which appear during operation. It is mandatory to have a steady exchange of information between the operating crew and the coordination team to achieve the most efficient and stable machine operation. On behalf of the FLASH coordination team we will describe the work flow and the tools we use to keep the machine operators informed about technical and operating issues and how the FLASH documentation and bug tracking system works. We will present the advantages and disadvantages of those tools and work flows and we will explain our ideas about future upgrades to combine different tasks in one common tool.
The information on one system and along time are very important for operation effectively. The information on one system includes system schematic, manual, wiring table, parameter setting, etc. The information along time includes trip information, maintanence information, etc. Collecting these information effectively can make us know what happend and what we should do next.
This presentation will show essential tools to collect these information.
The Australian Synchrotron has always required extensive operational statistics for the accelerators documenting the frequency and severity of faults that interrupt user beam, as well as demands on the operators' time. For years this has been accomplished through the Systems Operations Log (SOL) a web based elog created for fault accounting and as general logging software developed entirely by operators. Since last year senior management has made it a priority for beamlines to similarly account for their downtime - so we've adapted SOL to allow the beamlines to have analogous downtime accounting, among other upgrades.
The Isotope Separator On Line Device (ISOLDE) is one of the oldest experiments at CERN. In 2017 ISOLDE celebrated 50 years of life. The latest addition to the ISOLDE facility is the HIE-ISOLDE project. A super conducting LINAC with 4 cryomodules, capable of reaching energies up to 10 MeV/u. Different isotopes are set up and delivered on time, almost on a weekly basis. In several cases even parallel delivery of different isotopes at different experimental stations is possible. In this talk, the day to day activities of the machine supervisor team will be presented.
The MedAustron Particle Therapy Accelerator (MAPTA) is a synchrotron-based medical accelerator providing 60-250 MeV proton and 120-400 MeV/u carbon ion beams for tumour treatment and non-clinical research.
Clinical operation of MAPTA has started in December 2016 on one horizontal beamline supplying protons to one of the three foreseen treatment rooms. Since then, two additional beam lines – one horizontal, one vertical – supplying a second treatment room with protons have been added to the clinically used accelerator configuration. Outside of the clinical use, the commissioning of new clinical and non-clinical beams (in particular C6+) and the development of functional upgrades for MAPTA is still ongoing.
This contribution focuses on the current challenges encountered during MAPTA operation. Many of them are related to the parallel operation of MAPTA for clinical use, for non-clinical research and for commissioning purposes, which require different machine configurations and quality assurance measures to be applied. Additional challenges are mainly related to the clinical workflow and involve the reduction of technical and administrative downtime, the information flow between users and operators, the handling of delays, and the processing of short-notice machine time requests from the medical user.
Beam stability is an important issue when operating a medical accelerator. To obtain an early indicator of the machine condition we have established a morning beam performance check taking place prior to the formal quality assurance of the medical physics experts. On the basis of the recorded data we have observed that the beam position in the HEBT, represented by the position on a profile grid shortly downstream the synchrotron extraction, depends on the operating conditions of the previous night. This suggests assigning the beam position variations to a temperature effect. For further investigations we distributed several self-made climate data loggers in our building and connected them to our control system. Equipped with this new set of data loggers and the existent access on cooling water data we dedicated a complete night shift to an accelerator stress test by running half a shift (4 hours) high energy (430 MeV/u) carbon beam to the gantry (heating phase) and half a shift idle (cooling phase). We found strong correlations between room temperature and beam position. The causal relationship, however, is not so obvious and still under investigation.
With experience at next generation electron and ion accelerator facilities that spans his career, Dr. Willeke offers his perspective on the history of, and outlook to the future for, the efficient operation of accelerators presently in the design phase, such as the proposed electron-ion collider at Brookhaven National Laboratory.
The recently formed Drivers Operation Group at TRIUMF has merged E-LINAC operations into the pre-existing 520 MeV Cyclotron Operations Group. The end goal is to operate both accelerators in one control room with one group of cross-trained operators. Despite challenges, most cyclotron operators have now been trained to operate the E-LINAC for low energy commissioning activities and the control rooms are in the process of being combined. This talk will discuss how operators are coping with the changes to their responsibilities and work environment. Struggles encountered, as well as the positive outcomes, will be presented from an operator's perspective.
ALBA is a 3 GeV storage ring running 6000 h per year. While in operation the control room is manned 24/7, while in shut downs only the morning shift is covered. The operators ensure efficient operation of the machine and provide support for the coordination of the tasks during the shut-down by working between 50 and 60% of their time in shift. The other 40 to 50% is dedicated to work in areas other than operation or to improve operation.
According to their background and preferences, the operators have the opportunity to contribute to other sub-systems under the leadership of the accelerator division (e.g. RF group, pulsed elements, linac group...). Each operator is assigned a specific sub-system. In addition contributions related to their job in the control room are encouraged, mostly taking place in the form of programming or improving diagnostics.
This scheme has resulted in having a crew of operators, with very specific and detailed knowledge on different sub-systems areas as well as very knowledgeable with Python and the TANGO control system. The additional work helps to keep operators motivated and provides room for further formation and training.
The Spallation Neutron Source is a user-oriented accelerator complex with a fully subscribed users list each run period. Most shutdowns are relatively short and packed with carefully coordinated activities to keep the accelerator running and occasionally integrate machine or instrument upgrades.
Accelerator Operators are asked to propose their shutdown projects and pick groups they will collaborate with during these shutdown periods. Once those projects are approved by management, operators are expected to report weekly for time spent while simultaneously maintaining current status in training activities. If the operator does not opt to generate a project, there are operations-related projects that are assigned by management.
At the end of the year 2017, SNS began the longest shutdown in its operating history. My project for this shutdown would be to write a proposal for discretionary beam time, requesting Neutron Beam Analysis of the Cesium cartridges we use in the Ion Source for Neutron Production. This would be the first time any operator had been a user as well as the first time any part of the source of the neutron beam would be analyzed by neutron beam, completing a full circle in the science and materials analysis performed at SNS.
The accelerator-driven spallation neutron and muon source at Rutherford Appleton Laboratory in the United Kingdom is manned 24/7 all year round. The key to the success of the accelerator is the operators, who ensure the safe and efficient operation of the machine, working around the clock in shift crews of 3 people. In 2018 a new system has been introduced allowing crew members to spend some of their time working outside their role as operators.
This presentation will highlight the benefits of the transition to a regime which gives the operators the opportunity to spend some time with other groups and further their training, and will consider how the first months after changing to this new regime have progressed.
The transition was gained by changing the shift pattern from 5 crews of 3-4 people to 6 crews of 3 people. Every operator could choose tasks they were most interested in to enhance their knowledge from a pool of different groups (e.g. RF group, Electrical Group, Controls Group, Performance Improvement). After several months of running this new regime and some bumps on the road it seems that mostly this additional work is indeed benefitting the operators and the groups alike.
Will be described how the rotation of the operator shift has changed during the years in order to allowed a better cooperation into project not directly connected to machine operation and the impact of how this cooperation involve to the operator when in charge to the control room.
An overview about some projects realized by operator will be given.
Machine Learning has proven itself as a useful technique in a variety of applications. We have used machine learning techniques to provide an RF feedforward system to the NSLS-II linac to correct for long term drifts in the system. Prior to this, the operator needed to do the correction manually. The operators participated in every aspect of the process from generating the necessary controls, to data collection, and verification. In this paper, we discuss have machine learning was used to correct drifts in the NSLS-II linac and how the operators participated in its development.
From safe self driving cars to computers that can beat a human at the TV game Jeopardy; Machine learning algorithms are infiltrating many aspects of our lives. This paper will present an overview of algorithms being used at SLAC and elsewhere. The main focus will be on design and performance of programs used by operations to reduce tuning time.
The WAO workshops have traditionally focused on how Accelerator Operations Staff (the Control Room Operators) are organized and how they conduct business.
At LBNL's Advanced Light Source an Electronic Systems support group is also staffed around the clock and provide immediate repair services whenever any of the power supplies or control subsystems break down.
This talk will look at how the Electronic Maintenance Technologists matrixed from LBNL's Engineering Division are trained for both the complex technical and safety aspects of their work.
At the Workshop on Accelerator Operations 2004 (WAO2004), Michael Epps, then the Accelerator Operations Deputy Group Leader at Jefferson Lab gave a talk called Accelerator Operations Training and Development Program which described a proposed training program which JLab was just beginning to develop at that time. So some 14 years down the road how has it worked out? This talk is a retrospective of what was done, what worked and what did not. What has been done and what still needs to be done to finish implementing JLab's plan to have a fully functioning, sustainable training program.
Moderators summarize the results of breakout session discussions
A survey was sent out to registrants of all institutes, to get a global overview on our daily working environment. How does the Operator “see” and “control” the accelerator? Is it a black box with few parameters to tune, or is it rather a long list of several hundred magnets and other devices, all with their individual settings? How often do you actively manipulate the accelerator settings, and which tools to you make use of? How do you monitor the machine? How do you troubleshoot? If you were in the position to start from scratch, which features of your operator interfaces would you keep, which would you change?
An overview of the compiled responses will be presented.
Final remarks and recognition on WAO 2018
A look forward to the next Workshop, 2 years hence; other workshop announcements
WAO attendees will travel via bus to BNL to tour facilities at NSLS-II, as well as NSRL, RHIC and MCR facilities for C-AD