Week of 240415

WLCG Operations Call details

• The connection details for remote participation are provided on this agenda page.

General Information

• The purpose of the meeting is:
  • to report significant operational issues (i.e. issues which can or did degrade experiment or site operations) which are ongoing or were resolved after the previous meeting;
  • to announce or schedule interventions at Tier-1 sites;
  • to inform about recent or upcoming changes in the experiment activities or systems having a visible impact on sites;
  • to provide important news about the middleware;
  • to communicate any other information considered interesting for WLCG operations.
• The meeting should run from 15:00 Geneva time until 15:20, exceptionally to 15:30.
• The SCOD rota for the next few weeks is at ScodRota
• Whenever a particular topic needs to be discussed at the operations meeting requiring information from sites or experiments, it is highly recommended to announce it by email to the wlcg-scod list (at cern.ch) to allow the SCOD to make sure that the relevant parties have the time to collect the required information, or invite the right people at the meeting.

Best practices for scheduled downtimes

• Best practices for scheduled downtimes

Monday

Attendance:

• local: Panos (WLCG), Laurence (computing)
• remote: Kate (DB, chair), Julia (WLCG), Ville (NDGF), Ivan (BNL), Xavier (KIT), Matteo (CNAF), Darren (RAL), Dave (FNAL), Peter L (), Henryk (NCBJ), Petr (ATLAS)

Experiments round table:

• ATLAS reports (raw view) -
  • Nine IGTF core Root CA still rely on SHA1 causing troubles again GGUS:166154
    • new issue with fetch-crl which doesn't work on EL9 ( issue#4, no response)
    • existing issue with XRootD ( issue#2150)
    • dCache also can't load SHA1 signed CRLs, because default Java configuration also doesn't allow SHA1 (advertised by dCache developers last week in the mailing list)
    • can WLCG ask IGTF to provide timeline replacing SHA1 on supported root CAs
    • does IGTF (WLCG? EGI? OSG?) plans to ask sites to move back to "DEFAULT" crypto policy once this gets fixed
      • is it OK if our sites runs with updated configuration in 2032 when EL9 reaches EOL?
      • OSG officially advetise `DEFAULT:SHA1` workaround for affected services
      • some site even apply `LEGACY` to "solve EL9 issues"
  • Asked BNL FTS to use libcurl instead of libneon Davix HTTP backend library GGUS:166234
    • some transfers terminated after 300s when FTS rely on libneon GGUS:166204
  • Tier-0 and computing infrastructure working well
    • ATLAS data taking often in special configurations (e.g., big event sizes) and with sizeable rates that exceeds 8GB/s

• CMS reports (raw view) -

• ALICE
  • NTR

• LHCb reports (raw view) -

Sites / Services round table:

• BNL: NTR
• CNAF: The endpoints whose configuration has been asked via GGUS:165796 cannot be used since are not working, in particular the .well-known endpoint is not public
• EGI:
• FNAL: NTR
• IN2P3:
• JINR:
• KISTI:
• KIT: NTR
• NCBJ: NTR
• NDGF: NTR
• NL-T1:
• NRC-KI:
• OSG:
• PIC:
• RAL: NTR
• TRIUMF: NTR

• CERN computing services: NTR
• CERN storage services:
• CERN databases:
• GGUS: NTR
• Monitoring:
• Middleware: NTR
• Networks:
• Security:

AOB: