CREAM Migration Task Force

Mandate

The support of the CREAM CE will come to its end by the end of 2020. By this time WLCG sites which are currently running CREAM CEs need to have migrated to different CE implementations or No-CE solutions. Recommended CE implementations are HTCondor and ARC. There are several No-CE solutions which can be also considered as a valid option for WLCG sites. The goal of the task force is to support sites in the migration process and to follow the overall progress at the WLCG level.

We foresee supporting sites in their migration by:

• Providing necessary documentation and keeping it up to date. This includes sharing recipes and experiences in the migration process and troubleshooting
• Mailing list where people can get help and answers to their questions
• Contributing to the work aiming to facilitate deployment using containers and configuration management tools like Puppet and Ansible.

Areas of work

Following up on migration progress

• Following up on migration progress

Development ideas

• Containers for ARC / HTCondor
• Lightweight deployment systems
• Sharing of Puppet / Ansible / ... code
• ...

Membership

Initial members:

• Maarten Litmaath (CERN)
• Mayank Sharma (CERN)
• Julia Andreeva (CERN)
• Eraldo Silva Junior (CBPF)
• Pepe Flix (PIC)
• Stephen Jones (Liverpool University)
• Stefano Dal Pra (CNAF)
• David Bouvet (IN2P3-CC)
• Vanessa Hamar (IN2P3-CC)
• Jean-Michel Barbet (Subatech)
• Matt Viljoen (EGI)
• Alessandro Paolini (EGI)
• Baptiste Grenier (EGI)

If you are interested, please subscribe to the wlcg-ops-coord-tf-cream-migration mailing list (or we can do that for you).

CMS Timeline for CREAM CE migration

The CMS glideinWMS factory is distributed with the OSG software stack. The current OSG software version 3.4, and its end of critical bug/security support is planned for November 2020. After that, the condor version distributed with OSG 3.5 will remove the support for CREAM CEs.

CMS sites have been reminded of that at the CMS Service and Facilities meeting that the CREAM-CE support in CMS ends September 1st, 2020. CMS is now entering the CREAM-CE Phase-Out stage. This means the CMS factory operations team will soon upgrade the UCSD factory software stack to OSG 3.5, which will bring the "upcoming" glideinWMS series (3.8.x) with numerous new features CMS is eager to test, like the usage of tokens in place of x509 proxies for authentication. However, pilots for CREAM CEs will no longer be sent from the UCSD factory. The CERN factory will stay at OSG 3.4 and will keep sending pilot jobs to CREAM CEs, but sites that are just relying on CREAM CEs might experience reduced pilot pressure and service interruptions.

Once the OSG 3.4 November end of life is reached, CMS will have to upgrade all the production factories to OSG 3.5. Therefore, starting November 1st the CMS factories will stop sending pilot jobs to CREAM CEs sites.

Useful links

• To compare different options:
  • Cream Migration Workshop 1st part
  • Cream Migration Workshop 2nd part

• Information on ARC
  • NorduGrid 2019 Conference, incl. ARC Technical Workshop
  • NorduGrid / ARC home page
  • please report your experiences on the wlcg-arc-ce-discuss mailing list

• Information on HTCondor
  • HTCondor week May 2019
  • HTCondor home page
  • HTCondor CE Accounting
  • you can use the wlcg-htcondor-discuss list to report your experiences

• Information on lightweight deployments
  • The SIMPLE Framework
  • Vac, Vcycle

-- JuliaAndreeva - 2019-05-29