Reducing WLCG dependencies on BDII

Introduction

The Information System TF is working towards stopping WLCG dependencies in the BDII. This twiki presents the different proposals that the TF is evaluating. Please, note that this is all being discussed and still needs to be approved by the WLCG MB.

Storage

WLCG has no dependencies on storage information published in the BDII. Each LHC experiment has its own recipe to collect and maintain storage endpoints and storage capacity information. For this reason, WLCG is investigating the possibility of stopping storage information being published in the BDII for dedicated WLCG resources.

A proof of concept has been done with dCache at pic which has a dedicated service for three LHC VOs: ATLAS, CMS and LHCb. The recipe described in the next section has been succesfully implemented and it is recommended to sites with dedicated WLCG storage.

Recipe to stop publishing Storage Services in the BDII

This recipe is for sites with dedicated WLCG storage services.

• Remove from GOCDB the tag EGI for the storage service and set it to Local(x) (this may take some time to propagate to the SAM OPS servers)
• Remove SE Endpoints from Site-BDII and wait until this is propagated to top BDIIs (this may take several hours or days depending on how long it is configured the top BDII cache)
• Stop BDII service running on the hostname where the storage service is running.

These steps will imply:

• The SAM OPS tests will not be executed on that service.
• The storage service won’t be published anymore in the BDII.
• The storage service will still be tested by ETF since storage services are taken from experiment VOfeeds that do not rely on the BDII to get the list of services.

Computing

All LHC VOs rely on the BDII to discover CE queues and related information. However, the amount of information that is actually used by the VOs is very little compared to what the BDII publishes. Moreover, OSG is moving out of the BDII. For this reason, there is a general agreement that the static information needed by LHC VOs could be moved to GOCDB and OIM instead. This would allow WLCG to rely only on GOCDB/OIM to get information from EGI and OSG resources. This would also allow sites not belonging to EGI or OSG and not running a BDII to provide the set of needed attributes in an easy way, avoiding to run a service like the BDII.

The following set of attributes are considered to be sufficient to cover experiment use cases:

The IS TF will investigate how to move this information from the BDII to GOCDB/OIM taking the following actions:

• The GLUE2 name of the attributes will be used
• The GLUE Enumerated types values will be used for the types
• Identify a set of volunteer sites to define the new attributes in GOCDB and OIM
• Using the web UI, use extension properties to define key/value pairs matching the list of attributes
  • GOCDB: a writeable API has been requested and is under discussion
• Ask AGIS to consume this information from GOCDB/OIM instead of the BDII

After this exercise is carried out, IS TF will be in a position to evaluate whether it's a feasible choice and plan for a wide deployment if necessary.

Monitoring

WLCG monitoring relies on the BDII to be able to get the queue names for the CEs that need to be tested according to the experiments VOfeeds. There is a proposal to improve the current VOfeed so that it directly includes the queue name on it. However, with the new CRIC, it seems reasonable to get this information directly from CRIC, as shown in the picture below.

The IS TF proposal would be the following:

• Freeze any developments to provide the VOfeed V2 containing queue information.
• Wait for CRIC to be ready and then implement an experiment topology API that would allow applications and other services to get WLCG and experiment topology related information in a uniform way.
• In the meantime, WLCG Monitoring keeps on relying on the BDII to get the queue names until CRIC is ready.

-- MariaALANDESPRADILLO - 2016-04-28