WLCG Containers Working Group
Mandate
The mandate discussed at the first meeting and agreed at the
11 July 2017 GDB is:
To understand and track the use of containers within WLCG for compute deployment, and to make recommendations to sites on potential deployment patterns that they may wish to adopt. To ensure that Singularity and its support services are deployable for the LHC experiments.
We agreed two main goals:
- Singularity is sensibly deployable in WLCG and used by all experiments, and we’ve documented and understand the operational aspects.
- We should have surveyed and made recommendations on potential deployment patterns (and technologies) for container-based compute deployment and provisioning, with an aim to reduce operational effort on the sites, and to meet the needs of the ongoing analysis reproducibility work going on in the experiments. At the end, we should have clearly documented a few reasonable paths for a site (should it wish to) to migrate their compute to these technologies.
Work areas
Singularity deployment
The group's current focus is coming up with a baseline recommendation for the deployment of Singularity for sites, while trying to ensure as much commonality as possible between the experiments in the way they use it.
This is being tracked in a Google doc at
http://cern.ch/go/Q6rP (view and comment link).
Container-based compute deployment recommendations
Not much work here yet, though some initial presentations at the
kick-off pre-GDB).
Actions
Action |
Who |
What |
Status |
WC8 |
Olga / Gavin |
Test the new underlay feature for CERN and comment on pull-request |
Open |
WC9 |
Olga / Gavin |
CERN to setup HammerCloud based CI test for epel-testing including Singularity based test |
Open |
WC10 |
Ben |
CERN to enable unprivileged namespaces and disable SUID after tests |
Attempted, though config issue - still to try again. |
DONE |
Who |
What |
Status |
WC1 |
Maarten, Alessandra |
Contact David Love in Manchester (the current Singularity EPEL maintainer) to discuss our community's needs |
Done. Brian now maintains EPEL. |
WC2 |
Andrew |
Describe more fully the potential workaround for ATLAS' PWD bind-mounting |
Done / Superseded with WC6. |
WC3 |
Andrej |
Understand whether WC6 solution would work better for ATLAS |
Yes, thought that underlay should help a lot. |
WC4 |
Dave, Jakob |
Understand the issue with bind-mount overlay on CVMFS unpacked images, and whether a CVMFS workaround is appropriate |
Understood: CVM-1434. Workaround is to use underlay. |
WC5 |
Gavin |
Start google doc to track emerging baseline recommendation and issues |
Done, see http://cern.ch/go/Q6rP |
WC6 |
Vincent, Dave |
Track feature request with upstream Singularity for unprivileged overlay trick |
Now supported by EPEL 2.6 & upstream 3.0, |
WC7 |
Alessandra / Andrej |
Test the new underlay feature for ATLAS and comment on pull-request |
Reported https://docs.google.com/spreadsheets/d/1SGKyja47Veu_8IUXlXWOOEferuFoD62O4m64pgTNgSk/edit |
Meetings
The Indico event area is
https://indico.cern.ch/category/9769/.
Reports
Mailing lists
The mailing list is
wlcg-containers@cernNOSPAMPLEASE.ch - you can subscribe from egroups.cern.ch.