---

Computing Technical Design Report

5.3.1 The ATLAS Virtual Organization

It is assumed that all members of the ATLAS Collaboration are authorized to become members of the ATLAS Virtual Organization (VO), and therefore are allowed to submit jobs and gain access to Grid resources. While currently it is not a trivial operation to submit jobs to the Grid, and it has effectively been done only by production managers, we have to foresee facilitating access to the Grid for analysis users.

In order to open access to Grid computing facilities to all members of the ATLAS Collaboration, we are setting up an internal system of priority allocation for the usage of CPU and data storage resources. The problem is multi-dimensional, as one can define at least three dimensions that can affect the priority of a job on a given site:

1. The role of each person in the computing structure:
• Grid software administrator (who installs software and manages the resources);
• production manager, who will have higher priority than normal users for "official" group productions and will be able to place files in commonly accessible areas;
• any normal user.
2. The activity (physics, combined performance, detector groups and central computing operations);
3. The funding agency or institute affiliation of the person submitting the job.

The VOMS (Virtual Organization Management Service [5-2]) middleware package allows the definition of user groups and roles within the ATLAS Virtual Organization. We are initially setting up a VOMS group for each Physics Working Group, Combined Performance group and Detector System, as well as a generic ATLAS group and other groups for software testing, validation and central production activities.

At the time of registration with the ATLAS VO, a user will indicate the group (or groups) with which he or she is going to work, and in which capacity. The VO manager will check with the group leaders that the assignments and the proposed roles are correct.

In order to implement an efficient allocation sharing and priority system in the Grid access tools, it is necessary that the use of the resources dedicated to ATLAS be prioritized according to the Collaboration's policy. This could be achieved in principle via a central job distribution system to fully-dedicated clusters, but such a solution may suffer from bottlenecks and single points of failure. In a distributed submission system, local resource providers will have to be involved in defining and enforcing policies on priorities (in agreement with the Collaboration) through pseudo-dynamic updates to the local resource management systems fair-share policies. These options are under investigation as part of this year's upgrade plan (see Section 5.4.7 ).

In this context, one important issue to be addressed in the near future is the mapping of Grid users to local group accounts, through which tools it happens and how local priorities are taken into account. Identity mapping is necessary when a site's resources do not use Grid credentials natively, but instead use a different mechanism to identify users, such as UNIX accounts or Kerberos principals. In these cases, the Grid credential for each incoming job must be associated with an appropriate site credential.

The Open Science Grid (OSG) in the US has set up the GUMS (Grid User Management System [5-3]) project to develop a Grid Identity Mapping Service. The GUMS server performs this mapping and communicates it to the gatekeepers. The gatekeepers are charged with enforcing the site mapping established by GUMS. The GUMS client (gatekeeper) portion consists of an administration tool for querying and/or changing the state of the server. While this tool is not yet of general use outside OSG, it could in the future be adopted more widely to manage local heterogeneous environments with multiple gatekeepers; it allows the implementation of a single site-wide usage policy, thereby providing better control and security for access to the site's Grid resources. Using GUMS, individual resource administrators are able to assign different mapping policies to different groups of users and define groups of hosts on which the mappings will be used. GUMS was designed to integrate with the site's local information services (such as HR databases or LDAP).

5.3.2 Data Management on the Grid

The data management system fulfils a set of distinct functions: global cataloguing of files, global movement of files, and space management on ATLAS-owned storage resources. In ATLAS we opted initially to realize the global catalogue function by building on the existing catalogues of the three Grid deployments (the Globus RLS in the case of NorduGrid and Grid3, the EDG-RLS in the case of the LCG). The data management system acted as a thin layer channelling catalogue requests to the respective Grid catalogues and collecting/aggregating the answers. At the same time it presented the users with a uniform interface on top of the native Grid data-management tools, both for the catalogue functions and the data movement functions. The current implementation of the data management system used in ATLAS [5-4] and its foreseen evolution are described in Section 4.6 .

Following the experience of Data Challenge 2 in 2004, we have re-examined the basic assumptions of this system. A large fraction of job failures were due to lack of robustness of the native Grid tools used for data cataloguing and data transfer, as well as to a lack of appropriate Storage Element space-management tools. We had to implement many work-arounds to offset the middleware problems (such as retries and time-outs in file transfers, resubmission of jobs to different sites when input data became unavailable etc.), and at the same time we asked middleware providers to develop more robust data-management systems.

In order to build a robust Distributed Data Management system for ATLAS, we request that:

• all sites deploy Storage Elements presenting the same interface (SRM), backed up by a reliable disk pool or mass storage management system, such as DPM, dCache or Castor;
• Grid middleware implements an infrastructure for storage quotas, based on user groups and roles;
• Grid middleware implements hooks for file placement policies depending on users' roles;
• all mass-storage-based SE's provide information on file stage-in status (disk or tape) to be used for job scheduling and file prestaging.

The data management system based on native Grid catalogues has proven not to scale to the current needs of the ATLAS experiment. We are therefore in the process of redesigning the system, basing it on a multi-tiered cataloguing system. The basic concept is to have each dataset (a coherent set of events, recorded or selected according to the same procedure) subdivided into a number of "data blocks". Each data block can consist of many files, but is considered a single indivisible unit in terms of data storage and transfer. Central catalogues need to know only the location of the Storage Elements where each data block is present (in addition of course to the relevant metadata); the mapping to the physical filenames is known only to local catalogues. All local catalogues are required to present the same interface to the ATLAS data management system, but could in principle be implemented using different Grid-specific technologies. More information on the new ATLAS Distributed Data Management System (DDM) can be found in Section 4.6.6 .

5.3.3 Job Submission to the Grid

ATLAS has gained experience of job submission through Data Challenges 1 and 2, and through its preparations for the Rome Physics Workshop in June 2005. At present the job-submission system works with LCG2, Grid3, NorduGrid, and is starting to gain experience with gLite. It has also worked with the precursors to these Grid deployments. The system is now able to describe the function of a generic job submission, and to draw out some additional requirements.