Motivation

WLCG Ops Coordination is organizing a review of the WLCG procedures and tools for accounting and pledge management. We are collecting input from the experiments, sites, WLCG Project Office and WLCG ops coordination team. The goal is to understand whether existing tools and procedures satisfy the needs of the community and to identify possible areas for improvements.

Experiments input

ALICE

• Please, provide your feedback regarding the overall procedure for definition of the VO requirements and yearly pledges.
  • Do you believe it is well defined and allows you to manage your requests for resources conveniently?
  • Do you have suggestions for how it can be improved?
  • More comments
• Please, provide your feedback regarding CRIC UI for definition of the VO requirements and yearly pledges.
  • Does the UI satisfy your use cases?
  • Do you have suggestions for how it can be improved?
  • Is anything missing?
  • More comments
• Please, provide your feedback regarding central accounting systems: APEL, EGI accounting portal and WAU.
  • We know that experiments are mainly relying on their custom accounting systems. Do you have any use of the central systems?
  • What about information quality?
  • What about UI / UX?
  • Is there any missing functionality?
  • More comments
• Do you believe WLCG needs to invest in the improvement of the central accounting systems? Justify your answer, please.
• Please, provide your feedback regarding monthly accounting reports.
  • Are you checking them on a regular basis?
  • What about information quality?
  • What about report layout?
  • Is anything missing?
  • More comments

ALICE input

ATLAS

• Please, provide your feedback regarding the overall procedure for definition of the VO requirements and yearly pledges.
  • Do you believe it is well defined and allows you to manage your requests for resources conveniently?
  • The C-RSG process is well established, clearly defined, and up to now allowed us to appropriately define our requirements.
  • Do you have suggestions for how it can be improved?
  • We frequently get comments / requests from site managers asking about the pledges for the upcoming year. It seems that the communication of the approval of pledges to the sites could be improved (up to now, this is meant to be the duty of the FA representatives).
  • More comments
• Please, provide your feedback regarding CRIC UI for definition of the VO requirements and yearly pledges.
  • Does the UI satisfy your use cases?
  • We do not define the pledges in CRIC.
  • Do you have suggestions for how it can be improved?
  • Is anything missing?
  • It would be useful to have more information. For instance, it is important to know the % of the pledge as a function of the year committed by a specific FA for an experiment — i.e. an FA providing T2 to ATLAS, going from 2% to 1.5%, can do this without telling the experiment, and it is very difficult to see this from the experiment point of view.
  • More comments
• Please, provide your feedback regarding central accounting systems: APEL, EGI accounting portal and WAU.
  • We know that experiments are mainly relying on their custom accounting systems. Do you have any use of the central systems?
  • We often cross-check the overall data also in the EGI accounting portal, to make sure that what the sites declare is roughly in line with the ATLAS monitoring and accounting systems.
  • What about information quality?
  • We often find discrepancies. CERN for instance is not correct 2 times out of 3 (experience over the past 5+ years). Other sites have misleading/incorrect information, e.g. we saw last year CPU/WALL efficiencies going up to 250% in some sites. Such errors make the trustability of the overall system very low.
  • What about UI / UX?
  • Not very user friendly.
  • Is there any missing functionality?
  • The granularity in the EGI portal is bigger than the one we have in the experiments, i.e. if there are sites that are composed of several parts, e.g. standard Grid site plus come cloud resources, it is not possible to see them separately (or it might be possible but it is some config that some of the sites are not aware on how to do it).
  • More comments
  • One of the main problems is that the EGI accounting portal is not used “in production” by the experiments, but the information is taken from there once/twice a year. This means that also small errors can be there for several months. Having ways to correct the numbers is useful, but that’s yet another place where the information is copied, edited and updated, i.e. the results are not then fed back to EGI Accounting.
• Do you believe WLCG needs to invest in the improvement of the central accounting systems? Justify your answer, please.
• Yes, because having a trustable, user friendly, solid accounting portal would simplify the work done by all the WLCG experiments. We understand that this is not a quick process, i.e. the developments and testing might be only a few months, but the testing and integration, to move from the experiment-specific monitoring to the global accounting will take longer.
• Please, provide your feedback regarding monthly accounting reports.
  • Are you checking them on a regular basis?
  • Yes, we look at them, and, if some sites are very low, we try to understand what happened.
  • What about information quality?
  • What about report layout?
  • Is anything missing?
  • More comments

CMS

• Please, provide your feedback regarding the overall procedure for definition of the VO requirements and yearly pledges.
  • Do you believe it is well defined and allows you to manage your requests for resources conveniently?
    • Yes.
  • Do you have suggestions for how it can be improved?
  • More comments
• Please, provide your feedback regarding CRIC UI for definition of the VO requirements and yearly pledges.
  • Does the UI satisfy your use cases?
    • Yes.
  • Do you have suggestions for how it can be improved?
    • It would be nice to be able select two years in CRIC when listing pledges, for example 202[23] not just "2023" or "2022"
  • Is anything missing?
  • More comments
• Please, provide your feedback regarding central accounting systems: APEL, EGI accounting portal and WAU.
  • We know that experiments are mainly relying on their custom accounting systems. Do you have any use of the central systems?
    • Not used by CMS Facilities & Services (Site Support).
  • What about information quality?
  • What about UI / UX?
  • Is there any missing functionality?
  • More comments
• Do you believe WLCG needs to invest in the improvement of the central accounting systems? Justify your answer, please.
• Please, provide your feedback regarding monthly accounting reports.
  • Are you checking them on a regular basis?
    • Yes, and following up outlier results, which are often reporting errors.
  • What about information quality?
  • What about report layout?
  • Is anything missing?
  • More comments

LHCb

• Please, provide your feedback regarding the overall procedure for definition of the VO requirements and yearly pledges.
  • Do you believe it is well defined and allows you to manage your requests for resources conveniently?
  • Do you have suggestions for how it can be improved?
    • no
  • More comments
    • perhaps a bit outside scope, but it would be great to have an historical view of requirements / pledges
• Please, provide your feedback regarding CRIC UI for definition of the VO requirements and yearly pledges.
  • Does the UI satisfy your use cases?
    • yes
  • Do you have suggestions for how it can be improved?
  • Is anything missing?
    • a page of pledge vs requirement per country would be nice to have
  • More comments
• Please, provide your feedback regarding central accounting systems: APEL, EGI accounting portal and WAU.
  • We know that experiments are mainly relying on their custom accounting systems. Do you have any use of the central systems?
    • we use the DIRAC accounting systems for operations.
    • we use WAU for reporting CPU accounting to the C-RSG, as the "official" WLCG system.
  • What about information quality?
    • We cross-check WAU with the DIRAC accounting; the two systems use different metrics for CPU accounting and they tend to give systematically different results for some sites. We are currently calibrating the DIRAC benchmark against the WLCG HepScore benchmark to address this discrepancy. We acknowledge support from CERN IT on this topic.
  • What about UI / UX?
  • Is there any missing functionality?
  • More comments
• Do you believe WLCG needs to invest in the improvement of the central accounting systems? Justify your answer, please.
  • we would be in favour of a central accounting system, but we understand this requires a lot of development and validation work vs the experiments' accounting systems.
• Please, provide your feedback regarding monthly accounting reports.
  • Are you checking them on a regular basis?
    • not every month, rather a few times a year. We check for significant inconsistencies with respect to our accounting system and follow up with sites if needed.
  • What about information quality?
  • What about report layout?
  • Is anything missing?
  • More comments

WLCG Project office input

• Please, provide your feedback regarding the overall procedure for definition of the VO requirements and yearly pledges.
  • Do you believe it is well defined and allows you to manage your requests for resources conveniently?
    • the overall procedure is well defined
  • Do you have suggestions for how it can be improved?
  • More comments
• Please, provide your feedback regarding CRIC UI for definition of the VO requirements and yearly pledges.
  • Does the UI satisfy your use cases?
  • Do you have suggestions for how it can be improved?
  • Is anything missing?
  • More comments
• Please, provide your feedback regarding central accounting systems: APEL, EGI accounting portal and WAU.
  • What about information quality?
  • What about UI / UX?
  • Is there any missing functionality?
  • More comments
• Do you believe WLCG needs to invest in the improvement of the central accounting systems? Justify your answer, please.
• Please, provide your feedback regarding monthly accounting reports.
  • Does interface for generating of the accounting reports serve your needs and is user friendly?
  • Are you checking reports on a regular basis?
  • What about information quality?
  • What about report layout?
  • Is anything missing?
  • More comments
• Please, provide your input regarding monthly accounting validation
  • Do you think that it did improve the quality of the accounting data?
  • More comments

WLCG Operations Coordination input

• Please, provide your feedback regarding central accounting systems: APEL, EGI accounting portal and WAU.
  • What about information quality? *According to our crosschecks with the experiment accounting systems , data is rather trustworthy . WAU application which provides functionality to compare ATLAS and CMS data, shows pretty good consistency for ATLAS. We plan to enable comparison for LHCb and ALICE.
  • What about UI / UX?
    • No particular concerns
  • Is there any missing functionality?
    • System is not flexible. Changing/adding any reported metric takes too much effort and too much time
  • More comments
    • Slow processing of tickets and requests
• Do you believe WLCG needs to invest in the improvement of the central accounting systems? Justify your answer, please.
  • • We understand that improving flexibility of APEL and EGI portal would require full redesign and might not be possible given current priorities (enabling accounting for new architectures), however we believe, it is required in a long term
• Please, provide your feedback regarding monthly accounting reports.
  • Does interface for generating of the accounting reports serve your needs and is user friendly?
    • No suggestions for improvements
  • Are you checking reports on a regular basis?
    • They are checked by WLCG Ops before generation of the official version which is announced and published in the WLCG repository. We chaise all suspicious or missing information contacting the sites. Response form the sites is some time extremely slow which delays report generation
  • What about information quality?
    • WLCG Ops tries to follow up on eventual problems, in general we believe it is good
  • What about report layout?
    • Fine
  • Is anything missing?
  • More comments
• Please, provide your input regarding monthly accounting validation
  • Do you think that it did improve the quality of the accounting data?
    • We are convinced that introducing of validation considerably improved data quality. Since monthly validation has been introduced no major long lasting problems. Still we strongly encourage all sites put effort in enabling automatic generation of the accounting metrics, check them during validation and fix automatic generation if it provides wrong results.
  • More comments

Site survey

-- JuliaAndreeva - 2023-05-03