DOMA Contribution to the LHCC Computing Review 2021

The 2021 review is over and this twiki space is now closed.

The DOMA report has been published and can be found here - https://doi.org/10.5281/zenodo.5839866

The Review report has been published and can be found here - https://cds.cern.ch/record/2803119

Basics

• The charge - https://indico.cern.ch/event/1009737/contributions/4237201/attachments/2195035/3710994/HL-LHC-CommonSoftware-v4.pdf
• The document, r/w access - https://www.overleaf.com/6395194873bdxdvnckftvt
• The mailing list - wlcg-doma-lhcc@cernNOSPAMPLEASE.ch
  • Subscribe here - https://e-groups.cern.ch/e-groups/EgroupsSubscription.do?egroupName=wlcg-doma-lhcc.

The first draft was released at the end of June.

A round of feedback is underway and will finish 3rd September.

The document should be ready on the 1st October.

Feedback

Structured feedback has been received from WLCG/HSF, Atlas and CMS, alongside numerous other comments and suggestions.

Editorial Advice

Section templates

The following 4 points are taken directly from the charge.

• Include a description of the project and present plans and timelines to deliver the agreed functionality and performance.
• Describe how the project is managed, including how it will set priorities, monitor progress, and communicate with stakeholders.
• Present the current status of the development teams and note any gaps in skills or effort.
• Describe any major risks, potential functionality gaps, and dependencies on other projects.

For storage systems, and other sections considering it relevant, please make sure to mention the following themes explicitly.

• Data access via distributed caches
• TPC
• Exploitation of tape (inc HTTP interface)
• QoS and cost savings
• Token support
• Storage diversity (HPC / Cloud)
• Packet flow marking and connections to the network section

General advice and decisions

• We will factor out as much as possible from individual entries - common plans, risks etc
• Statements about dependencies should prioritise those which themselves are common software projects in our community (e.g. canl)
• The report should ensure it's consistent with what was reported at the WLCG storage workshop - https://indico.cern.ch/event/941278/
• We can assume the reviewers are experts and can include a glossary of terms if necessary
• Highlight the intersections between the network and other sections (e.g. provisioning by orchestration services)
• We should adopt a common definition of FTE - we should count purely development effort and respect the "Full Time Equivalent" definition, i.e. 2 people at 50% is 1FTE
• Include info about long term R&D plans/hopes and main scaling concerns

Open sessions for experiment feedback

Two open sessions were organised, inviting feedback from experiments and facilities.

• Tue 1st June, 16hr CEST.
  • Rucio, FTS and network - https://indico.cern.ch/event/1043829/
• Tue 8th June, 16hr CEST.
  • Storage - https://indico.cern.ch/event/1043848/

Editorial Team meeting Tue 25 May 2021

Decisions:

• We will organise 2 x 2hr sessions to collect feedback from experiments and facilities
  • 16hr CEST Tue 1st June dedicated to Rucio, FTS and network
  • 16hr CEST Tue 8th June dedicated to storage
  • To be announced at DOMA General 26th May
• Editorial approach, conclusions merged with general advice section in this doc

Editorial Team meeting Mon 10 May 2021

Decisions:

• We will operate a two weekly iteration cycle until end of June
  • Next meeting Tue 25th May 17hr CEST
• For the first iteration, each contributing project should follow the skeleton and produce a first draft (~1 page) ready for 25th May
  • For Rucio and FTS, follow the template and reference any of the storage themes that are relevant
  • For the storage systems, follow the template provided in the charge and also address explicitly each of the identified storage themes
  • For networking, follow the agreed template but adapt it where necessary
  • We will have to converge on a common interpretation of the template and themes, use the mailing list or explicitly identify points for further discussion in your draft
  • At this early stage we should all be ready to change the structure of the document if necessary
  • At the end of this cycle we will examine common questions and/or inconsistencies and take them to the WLCG management and the experiments for input.

Further info

Here are some links to further info from recent DOMA General meetings:

Feb - First discussion of the review

March - Update

April - First presentation of plan