Quarterly plan for February 2012-April 2012

Job monitoring area

Historical view

ATLAS

There is a long list of feature requests, but they will have lower priority compared to the new version of the historical view for CMS. New version will be released mid-April:

• On the Resource Utilisation page, add a line with the 2 coefficients for Tier1s and Tier2s. Manual input can be made from https://espace.cern.ch/WLCG-documen... . Store the history of those coefficients and manually or automatically fetch the values from http://www3.egee.cesga.es/accountin... -- this should be verified. Multiply the total capacity times the ATLAS share in the http://gstat-wlcg.cern.ch/apps/capa... to get the possible ATLAS deployed resources.
• Patch/ extend the graphtool python graphics library to accept variable values for the ATLAS Tier 0, Tier 1 and Tier 2 Federation Pledges in order to plot historical data for the used HEPSPEC hours over the pledged resources. DONE
• Start monitoring of the job input and output size of PANDA in the ATLAS Historical views to be able to check the evolution of job inputs sizes and outputs sizes over time.
• Unification of plot names: make sure that the titles of all the available plots are consistent.

CMS

• Adapt new version of the historical view redesigned for ATLAS to CMS

This include:

1). changing smry tables adding new sorting attributes (data type, CMSSW version) done but needs validation aftre UI is enabled

2).add # of users metric 3).add site/application failure on the interactive-like plot. 4).Redoing UI 5) Refactoring the code (removing duplicate code) (Deploy prototype for validation during mid-March)

Interactive interface

CMS

• Evaluate interactive UI performance quering denormalized DB object (table or materialized view) (first checks done, next steps poponed were waiting for migration to 11g and new hardware in the end of January, now can be performed on the new hardware) end of March
• In case of positive results of evaluation (point 2) rewrire interactive UI DAO part using denormalized DB object end of April

Task monitoring

ATLAS

• Develop stored procedures for aggergating data of the analysis task for increasing of the UI performance ( Done)
• Prototype first version of the alanysis task monitoring (end of February)

Transfer monitoring area

ATLAS DDM Dashboard

• Modularization of UI code (expected Feb-2012)
  • Allow parallel development of UI for DDM Dashboard and Global Transfer Monitoring.

• 2.0 M3.2 release (expected Mar-2012)
  • Links to monitoring data via Web API in JSON/XML formats.
  • Help page for Web API.
  • Live preview of endpoint selection.

• 2.1 release (expected Apr-2012)
  • Registration statistics.
  • Registration details.

WLCG Transfers Dashboard

• Support wiki
• VO-specific site naming in UI including curl service to retrieve VO topologies.
• Add server (aka fts endpoint) as filter.
• Automated consistency checking.
• Port to new modular UI (see ATLAS DDM Dashboard tasks above).
• Deploy to production server / database.
• Perform full consistency checks following deployment of FTS 2.2.8 to all T1 sites.
• Define new features and development plan following consultation with VOs.

Google Earth

• Remove statistics for local Phedex site transfers in CMS (Done)

Handling of the Dashboard cluster

SSB

Pablo, please , draft a plan here

SiteView

• Improve performance for the 24 hour history (end of February)
• Remove statistics for local Phedex site transfers in CMS (Done)
• Correct job processing and data transfer historical plots (mid March)
• Add a possibility to create numeric plots for job processing and data transfer history (mid March)
• Following the outcome of the WLCG TEG adapt SiteView for the set of agreed metrics (dependng on when list of new metrics will be agreed)

SUM

Finishing validation and switching to a new version for operations (end of February) Bug fixes following user input

Preparation for CHEP and EGI forum

• Prepare and present the ' User-centric monitoring of the analysis and production activities within the ATLAS and CMS Virtual Organisations using the Experiment Dashboard system' EGI Community Forum 2012 talk that will be conducted in the end of March in Munich, Germany.
• Prepare the 'Designing and developing portable large-scale JavaScript web applications within the Experiment Dashboard framework' CHEP 2012 presentation for May 2012 New York, USA.

Detailed input from team members

-- JuliaAndreeva - 16-Feb-2012