4.9 Development Processes and Infrastructure

Database and data management development in ATLAS follows the procedures and processes of the Software Project's Software Infrastructure Team. Software is maintained in the ATLAS offline software repository. Software coupled to the ATLAS offline framework, such as Athena persistency software, is incorporated and distributed with ATLAS offline releases. Software without this coupling, such as the distributed-data-management system DQ2, or the software configuration system (CMT), is not included in offline releases, but is distributed using the same Pacman-based distribution mechanisms as other ATLAS software, and will appear as ATLAS software `project' components when the project partitioning of ATLAS software is completed.

ATLAS databases share a pattern of discrete, regular releases with the experiment software, but it is a distinct release schedule not necessarily coupled to software releases. Accordingly, the DB project is putting in place a `database release coordinator' role and activity to manage the complex coordination of database releases that are consistent both internally and with prevalent software releases.

Testing and quality assurance must have a high priority in this domain. Databases and data management are heavily exposed to the sheer scale of ATLAS computing, so scalability and performance testing are vitally important. Data storage is also key to long term readability of data, so regression testing is essential. In coming scalability tests during the computing commissioning and service-challenge programs, as well as in routine production, ATLAS DDM will be deploying systems designed and intended for good performance at full startup scales, and will be evaluated and improved accordingly based on these smaller scale deployments - while smaller scale, we will evaluate whether system scalability shows comfortable headroom to reach data-taking scales, and adjust design and development accordingly. Testing of data storage and database software systems are integrated with the nightly and larger scale (RTT) automated testing programs of ATLAS software. Dedicated regression tests monitor readability of old formats and check for inadvertent schema changes.

4.9.1 Milestones, Planning, Manpower

Milestones and deliverables for the DB project are maintained here [4-14]. Project manpower is tracked at the subproject level. Currently about 42 people are contributing materially to the project, with a very approximate 25 FTEs of total effort. An essential factor in being able to deliver ATLAS needs with available manpower is leveraging external and collaborative work, in particular the LCG AA efforts in POOL, COOL, ROOT and SEAL; the CERN IT Physics Database team; the 3D project in LCG GD; and the Grid projects. While Grid projects are well staffed, the LCG AA, IT Physics DB, and 3D teams are not; we rely on robust support for all these projects to complete the ATLAS system. Within ATLAS, manpower is adequate in most areas but there are two very important exceptions. Database services is a large and rapidly growing activity, essential to ATLAS data processing, and is presently severely understaffed. Oracle database administrator (DBA) expertise is almost completely lacking in ATLAS and must be addressed with the hire of at least one ATLAS Oracle professional DBA. We expect these issues to be addressed, respectively, by the establishment of an ATLAS Computing Operations organization and by the availability of computing infrastructure support funds through M&O following the MoU process.