VO tags Validation

Introduction

On 07.03.2016, GOCDB released a new feature to be able to define WLCG tags associated to WLCG resources. See Release notes for more details. VO tags are in fact used by WLCG to be able to identify which resources declared in GOCDB can actually be used by LHC VOs.

This is a very useful mechanism to be able to query WLCG resources per VO, per Tier, etc. However, the tags were originally defined per site and not per resource. In order to correct this, sites have been asked to look at the tags and doublecheck whether they are correct for each production resource, cleaning the wrong tags accordingly. To verify that the current tags are correct, a validation effort is on going comparing the tags in GOCDB for each resource with the resources declared in each experiment VOfeed. It could also be that the VOfeed is wrong, see the next table to understand the scenarios we are trying to identify.

Note for sites: If the site doesn't have a certain VO tag and that VO tag is needed in a resource, the GOCDB admin needs to add the tag at site level first.

This task is carried out by Aleksandr Berezhnoi.

Experiment VOfeeds

• ALICE
• ATLAS
• CMS
• LHCb

GOCDB

The following method can be used to retrive all sites that support a specified site-scope tag. i.e.

https://goc.egi.eu/gocdbpi/public/?method=get_site&scope=cms,wlcg

This will return a list of sites. This list can be iterated using the get_service method to get the site's services. i.e:

https://goc.egi.eu/gocdbpi/public/?method=get_service&sitename=FI_HIP_T2

This can then be compared to the experiment VOfeed information for the site.

Monitoring of sites that need to fix their scope tags

Site name	Status	Verify	GGUS ticket	Comments
FZK-LCG2	waiting for reply		121344
egee.irb.hr	assigned		121345
UNIBE-LHEP	waiting for reply		121352
IEPSAS-Kosice	waiting for reply		121353
PSNC	assigned		121354
MY-UPM-BIRUNI-01	in progress		121355
T3-TH-CHULA	in progress		121357
UKI-SCOTGRID-ECDF	waiting for reply		121360
UA-BITP	assigned		121362
JINR-LCG2	closed		121366
IL-TAU-HEP	assigned		121369
NIKHEF-ELPROD	in progress		121370
DESY-HH	assigned		121378
UKI-LT2-Brunel	waiting for reply		121388
RO-13-ISS	closed		121390
FMPhI-UNIBA	closed		121392
INFN-CATANIA	closed		121395
IFCA-LCG2	assigned		121402
NZ-UOA	in progress		121403
ZA-CHPC	assigned		121404
BY-NCPHEP	assigned		121406
INFN-BOLOGNA-T3	assigned		121408
UA-ISMA	in progress		121410
WEIZMANN-LCG2	assigned		121416
INFN-TRIESTE	in progress		121419
UKI-LT2-RHUL	waiting for reply		121421
RU-SPbSU	in progress		121422
INFN-NAPOLI-ATLAS	assigned		121432
ICN-UNAM	assigned		121433
RO-11-NIPNE	in progress		121439
TECHNION-HEP	assigned		121446
SE-SNIC-T2	closed		121449
UKI-SOUTHGRID-BHAM-HEP	in progress		121450
ICM	in progress		121452
GRIF	closed		121454
CERN-PROD	in progress		121456
INFN-CAGLIARI	assigned		121460
Hephy-Vienna	solved		121358	- Remove 'alice' tag in GOCDB for the following services not present in ALICE VOfeed: hephyse.oeaw.ac.at(SRM).
INFN-T1	solved		121377	- Remove 'lhcb' tag in GOCDB for the following services not present in LHCB VOfeed: storm-fe.cr.cnaf.infn.it(SRM). - Add 'lhcb' tag in GOCDB for the following services present in LHCB VOfeed: ce01-lhcb-t2.cr.cnaf.infn.it(CE), ce02-lhcb-t2.cr.cnaf.infn.it(CE).
UKI-SOUTHGRID-BRIS-HEP	solved		121386	- Remove 'atlas' tag in GOCDB for the following services not present in ATLAS VOfeed: lcgce01.phy.bris.ac.uk(CE).
INFN-FRASCATI	solved		121396	- Add 'lhcb' tag in GOCDB for the following services present in LHCB VOfeed: atlasce1.lnf.infn.it(CE), atlasce3.lnf.infn.it(CE).
TRIGRID-INFN-CATANIA	should be opened			- Remove 'alice' tag in GOCDB for the following services not present in ALICE VOfeed: cream-ce-01.ct.trigrid.it(CE).
INFN-CNAF-LHCB	should be opened			- Remove 'lhcb' tag in GOCDB for the following services not present in LHCB VOfeed: ce01-lhcb-t2.cr.cnaf.infn.it(CE), ce02-lhcb-t2.cr.cnaf.infn.it(CE).

How to renew the information in the table with tickets

• ssh root@vo-tag-validationNOSPAMPLEASE.cern.ch
• kinit youraccount@CERNNOSPAMPLEASE.CH
• aklog
• cd /root/vofeed_vs_gocDB
• ./renew.sh

After several minutes you should obtain a new file twiki.txt with a new twiki formatted table.

renew.sh runs tree python files:

• gocbd_info.py (extracts all host from COGDB and saves the data to the shelve file gocdb_info.db)
• vofeed_vs_gocDB.py (compares the information from COGDB and VOfeed saves the results to the shelve file gocdbvofeed.db)
• tickets_submission.py (compares the gocdbvofeed.db with the initial data file gocdbvofeed_initial.db, monitors and creates tickets, creates new twiki table)

Now the scripts does not create new tickets. To begin a new campaign of tickets submission your should:

• in the file tickets_submission.py replace the string "out=ggus_monitor_gocdb_vofeed.ggus_monitor(site,template,'','False','prod')" to "out=ggus_monitor_gocdb_vofeed.ggus_monitor(site,template,'','True','prod')"
• mv gocdbvofeed_initial.db gocdbvofeed_initial.db.old
• cp gocdbvofeed.db gocdbvofeed_initial.db
• ./renew.sh

The file patterns.py contains the patterns used for parsing the COGDB and VOfeed xml files and host names, which are should be removed from the final table for some reasons.

-- MariaALANDESPRADILLO - 2016-04-28