Summary of October GDB, November 8, 2017 (CERN)

Agenda

https://indico.cern.ch/event/578992/

Introduction - I. Collier

presentation

Notes

- February pre GDB following CernVM Workshop, not immediately preceding GDB.

- GDB at ISGC - saw many participants from Asia Pacific not normally connected.

- WLCG Workshop confirmed, not provisional.

- HEPiX has the HTCondor workshop the week after.

- WLCG/HSF Joint Workshop. Starts first thing Monday so be there Sunday; ends Thursday to allow travel time before holiday on Friday for some.

- putting agenda together, informed by CWP.

SOC Hackathon - D. Crooks

presentation

Notes

- Note that provision should be made for people that need visitor passes.

HEPiX Fall - KEK Workshop Report - H. Meinhard

presentation

Notes

- Summarising a full agenda.

- special role of KEK in foundation of HEPiX (see later in talk).

- Abstracts normally accepted.

- new WG.

- network function virtualisation.

- good to see closer contacts between network and system admins; keen to repeat at suitable occasion.

- Common registration HEPiX /LHCONE, ~110-115 HEPiX attendees, Asia Pacific particularly strong.

- Numerous network contributions, not surprising - working group launched looking at archival, not too late to join, contact Helge.

- Virtualisation/cloud becoming commodity, mentioned in other presentations rather than explicit talks.

- good progress in benchmarking.

- number of configured slots has a strong effect on HS06. Next gen long benchmark. New SPEC now out, being examined. Much more memory/core than we would expect/use.

- would be surprised to see new large Grid Engine sites.

- HPC/HTC overflow, interesting work.

- questions of how to replace commercial software.

- At CHEP organised by Tsukuba, at a plenary, HEP Unix coordination group proposed -> HEPiX.

- much of what was proposed 25 years ago was still valid, recognising people involved for this - spring 2018 -> HTCondor workshop the week after (4/9-7/9).

- have proposals for next meetings comfortably until 2021-22, but this should not dissuade sites from volunteering.

LHCONE/LHCOPN Workshop Report - E. Martelli

presentation

Notes

- ATCF (Asia Tier Centre Forum).

- TEIN network connecting different centres in Asia.

- Thailand, China, Korea, Taiwan close to connecting to Japan.

- traffic doesn't have to cross Pacific.

- Missing connection TEIN -> NA. While Japan, Korea, Taiwan connect OK, smaller countries will only have TEIN.

- Sites without full meshes may not have full connection - routing table important tool.

- Science DMZ (identify servers connected to bypass).

Pete Clarke: What is the situation with other activities than experiments - astronomy on LHCOPN.

Eduardo: Discussed, no conclusion.

Pete: This is very important.

Eduardo: Subject of next meeting, not everyone in favour.

Pete: Policy as well as a technical issue; policy, funding agencies should be represented.

Ian Collier: Sites involved pay for links. Sites/funding agencies can decide if this is something they want to do. Pragmatic mix.

Eduardo: Nothing decided on wider usage.

Ian: No-one anticipates Belle II degrading overall bandwidth, which might not be universally true.

- Agreed next meeting 6-7th March 2018, Cosener's house.

Ian Collier: Observation that quality of discussion of Belle II traffic was good, including wider implications, good collaborative discussion.

Storage Accounting update - D. Christidis

presentation

Notes

- page 4 chart: Blue, Dmitri's area, Grey, someone else.

- Monitoring team provide transport + storage.

- page 7 chart: Blue, implemented, tested, deployed, Grey, future plans.

- Future path, leftmost, CRIC, SRR (collect data) -> json, or non SRM protocol.

- Now -> AGIS -> collected with SRM, ATLAS custom method, closely resembles SRR. ATLAS first, falls back to SRM.

- allows for gradual transition.

- add more plugins if required, applies to topologies as well.

- DIRAC in between.

- implementation tested at 3 sites, also looking for CMS/ALICE.

- Dashboard mature enough.

- see link in talk.

- Auth via SSO.

- New collector deployed last week, new schema, don't migrate old data.

- (screenshot) red line is pledge.

- thinking of having API available if desired.

- LHCb really close, ALICE/CMS asap.

- CRIC, SRR.

- progress good, data from CRIC well known, don't have to wait for full deployment.

- CPU validation measuring different things; expect 100% agreement for Storage.

- Tape might be more complicated, but smaller number of sites.

Questions

Dave Britton: Presentation of data needed to report to funding agencies every 6 months. CPU accounting is useless, can't bookmark particular view. Can't bookmark plot, hard to reproduce (aside from visual issues with colour scheme). For storage, need to be able to bookmark view.

Dmitri: Grafana does this, filters URL changes. For graph, can export to more permanent image.

Alessandra: How to log in?

Dmitri: SSO.

Maarten: Could be opened more widely, not everyone has SSO.

Dmitri: Can't answer that, need to take it up with monitoring infrastructure.

Q: Devs informed of lack of functionality (CPU)?

Dave: Filled in feedback 9-12 months ago, then asked John; he said he would talk to developers (John said he hadn't made any progress). I and others have tried, not successful.

Julia Andreeva: Discuss after the meeting.

[Update from after meeting, Dmitri noted that access without SSO should be possible in due course]

EGI Checkin - N. Liampotis

• SP only knows one IdP, EGI Checkin
  • Checkin propagates the level of assurance associated with the authentication source
• EGI Checkin integrates/aggregates several sources of identity, using possibly different protocoals (SAML, X509, OAuth2/OIDC...)
  • A unique Checkin identity can be associated with several "accounts" from different sources
• Can be coupled with a community management service: CoManage and Perun currently supported
  • A community can also run its own group management service, e.g. VOMS
• Token translation service to provide short-lived certificated using the RCAuth online CA
  • Proxy can be retrieved with OIDC or SSH (keys)

2 deployment modes

• Multi-tenant: limited customization, integration with community management services
• Dedicated: more customizations of user-facing parts and group aggregation/definition rules, possible integration with the main Checking instance

Checkin interoperable (or about to be) with many RIs

• Will be one of the pillar of EOSC-Hub AII

Checkin doesn't have to be the only IdP supported by a SP: using standard tokens, can interwork with any other similar service issuing standard tokens

• Need to agree on the token details (profile)

Potential overlap with INDIGO IAM: agrement to join the forces in EOSC Hub

• Already based on the same model (AARC blueprint) and sharing several upstream solutions for the implementation

Data Preservation at CMS - K. Lassila-Perini

see slides

Auth WG pre-GDB Summary - H. Short

(https://indico.cern.ch/event/578976/).

Supporting ML in the grid - E. Hernandez

see slides

ML libraries, in particular the popular TensorFlow, are not deployable easily on a CentOS 6 platform: Ubuntu is the preferred one

• They also potentially require access to GPUs

Solution deployed at OSG: Singularity containers with a TensorFlow (Ubuntu) image stored in CVMFS

Recently, also made a container for CMS based on CentOS Nvidia container and containing the CMS-specific dependencies (e.g. Hadoop)

Difficulty: update of device GPU drivers and ML libraries

• Problem: container is generally maintained by the VO when the drivers are maintained by the site
• No perfect solution found so far...
  • Liz: may the HSF packaging WG work benefit to this problem. Goal is to provide an easy packaging for tools common to several VOs.