April 2016 GDB notes

DRAFT

Agenda

http://indico.cern.ch/event/394781/

Startup

Introduction.

Data preservation: Status and Strategy of Certification in WLCG

• Should we record videos of people using the software and the data from the current projects? Probably a good idea.
• We can't hide from it any more.
• Review of T0 and T1 should be done,starting with T0.
• Self-certification is the first step.
• Data or the organization? All of it should get documented and certified. Not all has to be done at once.

APEL Accounting in INDIGO DataCloud

• REST vs SSM, why? Vague claims that performance is bad with SSM. The problem now is the EGI message broker system itself. The cloudy people are more comfortable with REST than message brokers. (REST API is a specific requirement placed upon it by INDIGO-DataCloud)
• Adrian (remote): ARGO messaging might replace EGI message brokers.
• EGI message brokers scale extremely well.
• No problem with 1000 Hz updates.
• REST looks a bit like the older APEL protocols, that was scrapped since they were good.
• SSM is not going away.
• EGI message brokers are run by Greece and Croatia.

Traceability WG Update

• Needs a kickoff meeting/PreGDB with invited people with a proposal for people to argue about.
• WLCG operations should also be involved.

Federating storage NDGF style - when does it make sense?

• Network problems (asymmetric routing) is a problem for distributed system.
• Compute? It requires some extra caching, which can be done in dCache or ARC.
• The cache is always for the whole file.
• Slovenia does not have to run a storage frontend.
• Slide 9: Pledges are a bit messy with loads of countries behind one endpoint. You can't federate T1's and T2's currently, perhaps the procedures and REBUS should be updated?
• Should T2-merged into T1 have the same SLA?

LHCOPN-LHCONE meeting, Taipei

• No questions.

APEL Repository development plans (Stuart Pullinger)

Slides

• Alessandro: a schema extension would be desirable to support a list of benchmarks that were run, instead of just one
• Stuart: that should be possible with the UR2 format
• John: there could also be multiple records, one per benchmark

• Miguel: how will cASO get the benchmark?
• Stuart: from a lookup table
• John: at a GDB last year Ulrich presented how this could be done

• Julia: is the problem on page 7 with aggregation of the data?
• Stuart: in a way - we want to avoid averaging over gaps

• Julia: we have experience with accounting in the Dashboards;
  we went for Hadoop, first with MapReduce, then Spark;
  as we have it in production, we should exchange knowledge
• Stuart: agreed

• Jeff: the issue of locked cores (page 12) can be solved by policy:
  high-memory jobs shall reserve multiple cores and get accounted for those

Accounting portal development plans and demo (Ivan Diaz)

Slides

• Ivan: the old portal will be stopped when the new one is OK for all stakeholders

• Alessandro: is it necessary to keep detailed usage data for old VM images?
• Ivan: so far it does not seem to be a worry
• Vincent: such data are also very useful to see where vulnerable images are still being used!

• Maarten: we should watch out for clients doing crazy queries that cause load on the server
• Ivan: not worried about that either

• Ian C: IRISGrid certificate details?
• Maarten: put a link on the portal welcome page?
• Ian C: how should feedback be given?
• Ivan: bugs and urgent operational matters via GGUS, RFEs via EGI RT;
  questions may also be sent via e-mail to the CESGA grid admin team
  (the old portal has the link under the CESGA logo, the new one not yet)
• Julia: WLCG mainly uses GGUS, developers can transfer/copy RFEs to their development tracker

• Julia: when might be a dedicated UI for WLCG be ready?
• Ivan: it is expected in 1 or 2 months

Summary from the pre-GDB discussion on CPU accounting (Julia Andreeva)

Slides

no comments

Summary of the pre-GDB discussion on space accounting (Alessandro Di Girolamo)

Slides

• Julia: the proposed storage providers WG should be involved in the high-level accounting objective
• Ian B: first we need to write down what should be done and
  get it agreed in the MB, then present it to the storage providers

Discussion

• Ian C: can we already define an action w.r.t. the storage accounting?
• Oliver: when we have the requirements document, I can involve the storage providers
• Alessandro: we will try to get the experiment and WLCG requirements collected and written up in the next two weeks
• Ian C: we may have some news by the May GDB then

Accounting Actions

• Proposals for next steps for accounting derived from the summary slides to be prepared in time for May MB
  • storage accounting - Alessandro & Oliver
  • CPU accounting - Julia