Brian's Notes on Joint DM/SM TEG Meeting (ATLAS)

January 6, 2012 (first meeting of the year!)

Graeme will be presenting the following answers from ATLAS: https://twiki.cern.ch/twiki/bin/view/LCG/WLCGTEGDataManagement_ATLAS.

Indico link: https://indico.cern.ch/conferenceDisplay.py?confId=169143

Daniele Bonacorsci chairing.

Attending:

• Paul Millar
• Elisa
• Graeme Stewart
• Andrew Hanushevsky
• Fabio Hernandez
• Daniele Bonacorsci
• Dirk Duellmann
• Vincent Garonne
• Alina Grigoras

Presentation from Graeme

• Primary focus: standardization, simplification
• If anything, current system does too much: do a small number of things better.
• Things added: Datasets, data lifecycle management. "Robust transfer service" on top of FTS.
• SRM functionality used is very restricted (list, read, write).
  • Doesn't do overwrite properly! (What does that mean?)
  • Asynchronous calls cause delays.
  • ATLAS uses space tokens, but only for very crude space partitioning (has more elaborate functionality inside the experiment).
    • Space tokens are partitions, too crude to use for space accounting. For example - may want to ask "how much is physics group X using" or "how much of physics stream Y is stored at the site"? The data in question may overlap.
  • Fairly unique, useful things done via SRM: tape recall.
• Maintains a transfer system on top of FTS: implements priority, longer queues on top of FTS. Does not keep long-term queues inside FTS.
  • Needs source switching (common with CMS).
• FTS provides some source protection. Architecturally, this is the wrong place perhaps (but it is important to keep working, as SRM doesn't!)
• The storage system should have a better means for "saying no", and have the larger ecosystem adopt accordingly.
• Relatively disinterested in the discussion of protocols. Look for something that (a) works and (b) gives the lowest overall costs. Not aware of any specific tests of HTTP support for ROOT.