Choose timezone
Your profile timezone:
RHIC DAP Round Table
→
US/Eastern
3-191 (BNL)
3-191
BNL
Minutes AI-generated from Zoom transcript
Summary of the RHIC DAP roundtable held on October 02, 2025
Agenda
· Follow-up from the review report and on the ChatBot– Eric
· Status of MetaData – Vincent
The meeting focused on the development and validation of a chatbot, as well as the critical need for data preservation and metadata management, particularly for the PHENIX and sPHENIX experiments.
Key Discussion Points:
- Chatbot Validation and Ethics:
- Christine Nattrass suggested making the chatbot live for users (e.g., graduate students) to test and provide feedback, as they are accustomed to using such tools.
- Eric Lancon agreed with the live testing idea but emphasized the need for a curated set of expert questions and answers to validate and calibrate the chatbot's accuracy, especially for "generic answers about the physics and environment of RHIC."
- Jerome Lauret highlighted a "chicken and egg issue," stating that while a live interface is good, the answers will only be as accurate as the documented and indexed information. He suggested a two-stage approach: first, subject matter experts rate answers, and then casual users provide feedback for refinement.
- David Morrison raised concerns about the ethics of academic research, particularly regarding plagiarism, if the technology encourages it. He also questioned the nature of the "thousands of questions" needed for validation, especially if the chatbot generates elaborate answers or code, which would be difficult to verify for correctness.
- Jerome Lauret acknowledged the "credit" issue for code generation, noting that while the chatbot provides references, users might not properly credit the original authors. He suggested thinking about this aspect more carefully.
- Eric Lancon stressed the importance of the chatbot being plugged into the official code repository of the experiment and having knowledge of data structure, conditions, and provenance to ensure validated and meaningful code generation.
- Yasuyuki Akiba emphasized the crucial need for good documentation to train the AI correctly and enable data preservation, stating it's the "highest priority."
- Authentication Issue:
- Ankush Reddy Kanuganti reported an authentication issue where internal information (hyper news, chats) was visible in guest mode without authentication. Jerome Lauret confirmed this is a critical condition to address before public release.
- Data Preservation and Metadata (Vincent's Presentation):
- Vincent (Garvin) presented on data preservation, focusing on what would be needed for an analysis in 5-10 years. Key needs include discovering datasets, finding software versions, and understanding post-processing states.
- STAR Experiment: STAR has a "pretty good" metadata catalog, financing almost 100 petabytes of data, with a key-value logic for annotating data, naming conventions that include metadata, and a complete lineage from publication to data conditions and versions.
- PHENIX Experiment: The situation for PHENIX is less centralized.
- There is no centralized metadata, and users rely on top-level macros to find dataset files.
- Basic information like event count and quality assessment is missing.
- File locations are "opaque" to end-users, which Christopher Pinkenburg clarified is a deliberate design to avoid users focusing on physical locations, as the analysis is hooked into a file catalog.
- Maxim suggested deploying a simple database to integrate different components, including event production location, file provenance, and analysis notes.
- Christopher Pinkenburg disagreed with some of Maxim's claims, stating that PHENIX analysis is active and everything needed is available for someone to start an analysis, utilizing an "analysis taxi" system.
- sPHENIX Experiment: The goal is to set up a similar "analysis taxi" system for sPHENIX due to the large amount of data.
- Challenges: Identified challenges include undocumented generation techniques, potential loss of expert and tacit knowledge, and person-dependent condition databases.
- Solutions/Suggestions:
- Introduce minimal discovery with minimal personal support.
- Move to persistent storage for long-term accessibility.
- Define and implement open standards for metadata catalogs (e.g., using tools like "Justin").
- The chatbot prototype could be used to answer questions about calibration procedures.
- Analysis Taxi System:
- David Morrison elaborated on the "analysis taxi" system, explaining its success in coordinating multiple analyses to process large datasets efficiently. It integrates requests over time and feeds data to analyses in a coordinated fashion, which is crucial for throughput and avoiding severe I/O demands.
The meeting concluded with a plan for Vincent to revise his slides based on Chris's feedback and to continue discussions, with Eric Lancon summarizing the meeting notes.
There are minutes attached to this event.
Show them.
