JRA1.1 - Work Package Coordination

JRA1 performs a regular telephone conference. Please find the links to the relevant Indico pages on the following page:

JRA1 Task Leader Telcons

The following graphic was created during the Architecture F2F meeting of the PTB:

• 2010-11-15 Architecture
  • Adobe Illustrator (ai)
  • EPS Graphic (eps)

We have established technical and political contacts to the following DCI projects:

VENUS-C

• Political (MoU, etc.)
  • Political Contact: Ake Edlund [ ake.edlund at gmail.com ]
  • Political Contact in EMI for VENUS-C: Alberto Di Meglio (CERN) [Alberto.Di.Meglio at cern.ch]

• Technical (collaboration, know-how exchange, etc.)
  • Technical Contact: Zeeshan Ali Shah (KTH) [ zashah at pdc.kth.se ]
  • Technical Contact in EMI for VENUS-C: Shahbaz Memon (JUELICH, Cloud Task Force Member) [m.memon at fz-juelich.de]

StratusLab

• Political (MoU, etc.)
  • Political Contact: Charles Loomis (LAL/CNRS) [loomis at lal.in2p3.fr]
  • Political Contact in EMI for StratusLab: Alberto Di Meglio (CERN) [Alberto.Di.Meglio at cern.ch]

• Technical (collaboration, know-how exchange, etc.)
  • Technical Contact: Vangelis Floros (GRNET) [efloros at grnet.gr]
  • Technical Contact in EMI for StratusLab: Shahbaz Memon (JUELICH, Cloud Task Force Member) [m.memon at fz-juelich.de]

IGE

• Political (MoU, etc.)
  • Political Contact: Helmut Heller (LRZ) [Helmut.Heller at lrz.de]
  • Political Contact in EMI for IGE: Alberto Di Meglio (CERN) [Alberto.Di.Meglio at cern.ch]

• Technical (collaboration, know-how exchange, etc.)
  • Technical Contact: Rafal Lichwala (PSNC) [syriusz at man.poznan.pl ]
  • Technical Contact in EMI for IGE: n/a - investigation started

EGI

• Political (MoU, etc.)
  • Political Contact: Steven Newhouse (EGI.eu) [steven.newhouse at egi.eu]
  • Political Contact in EMI for EGI: Alberto Di Meglio (CERN) [Alberto.Di.Meglio at cern.ch]

• Technical (collaboration, know-how exchange, etc.)
  • Technical Contact: Michel Drescher (EGI.eu) [michel.drescher at egi.eu]
  • Technical Contact in EMI for EGI: n/a - investigation started

EDGI

• Political (MoU, etc.)
  • Political Contact: EDGI Technical Director Jozsef Kovacs [smith at sztaki.hu]
  • Political Contact in EMI for EDGI: Alberto Di Meglio (CERN) [Alberto.Di.Meglio at cern.ch]

• Technical (collaboration, know-how exchange, etc.)
  • Technical Contact: EDGI Technical Director Jozsef Kovacs [smith at sztaki.hu]
  • Technical Contact in EMI for EDGI: Christian Ulrik Soettrup (Copenhagen) [soettrup at nbi.dk]

See JRA1 main page currently being a hot topic in the moment.

The following notes are essential pieces of information relevant for the developments within JRA1.

About backwards compatibility

• Once a year the project will produce a major release of the EMI distribution, characterized by a well-defined interface and behavior for each of its components. Interface and behavior are allowed to change within a major release of the distribution only in a backwards-compatible way.
  • Source: DSA1.1 - Software Maintenance and Support Plan

• The term interface is intended in a broad sense. Interfaces include, but are not limited to, APIs, ABIs, WSDLs, DataBase schemas, network protocols, authentication and authorization mechanisms, logging formats, packaging and other deployment characteristics of a component.
  • Source: DSA1.1 - Software Maintenance and Support Plan

• A strong constraint of the software maintenance activity is that backwards-incompatible changes can be introduced in a production environment only in a strictly controlled way.
  • Source: DSA1.1 - Software Maintenance and Support Plan

About EMI Releases

• The EMI distribution will be organized in periodic major releases, tentatively delivered once a year (corresponding to milestones MSA1.2.2, MSA1.2.2 and MSA1.2.4 - “EMI Reference Releases”), providing a good balance between the conflicting requirements of stability and innovation.
  • Source: DSA1.1 - Software Maintenance and Support Plan

• An EMI major release is characterized by well-defined interfaces, behavior and dependencies for all included components, available on a predefined set of platforms. What is included in a new EMI major release is defined by the PTB and the implementation of the plan is coordinated by JRA1 (corresponding to milestones MJRA1.19.1, MJRA1.19.2 and MJRA1.19.3, “Integrated EMI Major Release Candidates”).
  • Source: DSA1.1 - Software Maintenance and Support Plan

• Backward-incompatible changes to the interface or to the behavior of a component that is part of the EMI distribution can be introduced only in a new EMI major release. Changes to interfaces that are visible outside the node where the component runs (e.g. a WSDL or a network protocol) need to be preserved even across major releases, according to end-of-life policies to be defined on a case-by-case basis.
  • Source: DSA1.1 - Software Maintenance and Support Plan

• The availability of a new major release of EMI does not automatically obsolete the previous ones and multiple major releases may be supported at the same time according to their negotiated end-of-life policies.
  • Source: DSA1.1 - Software Maintenance and Support Plan

About Component Releases

• An EMI distribution includes all the components that are developed within the project and that have reached production quality. Within an EMI major release, only one version of a given component is maintained.
  • Source: DSA1.1 - Software Maintenance and Support Plan

• Four types of releases have been identified for a given component
  • Source: DSA1.1 - Software Maintenance and Support Plan

• (1) Major Release : A major release for a component is characterized by a well-defined interface and behaviour, potentially incompatible with the interface or behaviour of a previous release. New major releases of a component can be introduced only in a new major release of EMI. The contents of a new major release are endorsed by the PTB and included in the project technical plan. The implementation is coordinated by JRA1.
  • Source: DSA1.1 - Software Maintenance and Support Plan

• (2) Minor Release : A minor release of a component includes significant interface or behaviour changes that are backwards-compatible with those of the corresponding major release. New minor releases of a component can be introduced in an existing major release of EMI. The contents of a new minor release are endorsed by the PTB and included in the project technical plan. The implementation is coordinated by JRA1. If the release is going to be introduced in an existing major release of EMI, the implementation is also supervised by SA1 in order to guarantee that the production quality and the backwards-compatibility are preserved.
  • Source: DSA1.1 - Software Maintenance and Support Plan

• (3) Revision Releases : A revision release of a component includes changes fixing specific defects found in production and represents the typical kind of release of a component during the lifetime of an EMI major release.
  • Source: DSA1.1 - Software Maintenance and Support Plan

• (4) Emergency Releases : An emergency release of a component includes changes fixing only Immediate-priority defects found in production, typically security-related.
  • Source: DSA1.1 - Software Maintenance and Support Plan

• The type of release is reflected in the version of the corresponding package(s) and will be described in DSA1.2, “Software Release Plan”.
  • Source: DSA1.1 - Software Maintenance and Support Plan

About changes

• The role of Change Manager, i.e. following the process of controlling the life-cycle of approved changes, is taken either by the SA1 Maintenance task leader or by the JRA1 leader, depending on whether that RfC is going to be applied to a component release to be delivered within an existing EMI major releases or within the next one.
  • Source: DSA1.1 - Software Maintenance and Support Plan

About Support timelines

• It is foreseen that only the latest two EMI major releases are supported at a time. Within an EMI major release only the latest version of a component is supported. More extensive coverage will be evaluated on a case-by-case basis together with the users requesting it.
  • Source: DSA1.1 - Software Maintenance and Support Plan

About Testing

• PTs are expected to provide test suites and improve, where necessary, their software components test suites in order to meet the quality metrics defined by SA2.
  • Source: DJRA1.7.1 - Software Development Quality Control Report

• provide a test plan document that complies with the guidelines defined by SA2 (being part of the minimum required documentation).
  • Source: DJRA1.7.1 - Software Development Quality Control Report

• improve unit test coverage so that the thresholds defined by SA2.3 are met.
  • Source: DJRA1.7.1 - Software Development Quality Control Report

• augment functional test suites so that each component functionality is covered; if possible, measure the coverage also arising from new features and functionality introduced in EMI
  • Source: DJRA1.7.1 - Software Development Quality Control Report

• integrate the test suites in the EMI continuous build and integration process so that QA tools provided by SA2 can be used to monitor the execution and coverage of the testing
  • Source: DJRA1.7.1 - Software Development Quality Control Report

• when applicable, consider using a standard compliance test suite to assess the compliance of your software with respect to the standards listed in standardization task deliverables. This activity should be carried out in collaboration with JRA1.7 integration and interoperability and under the supervision of the JRA1.6 standardization task leader.
  • Source: DJRA1.7.1 - Software Development Quality Control Report

• test the interaction and interoperability of your software component with other related EMI components, following the integration and interoperability matrices provided by JRA1.7. This task should be part of the standard release certification activities and should be carried out leveraging the EMI distributed testbed.
  • Source: DJRA1.7.1 - Software Development Quality Control Report

About middleware versions

• For each middleware release SA2.6 assumes to have two available versions per software product: a Release Candidate (RC) version and a Production version, respectively the version which is going to be tested and the last version passing the acceptance certification process for that release.
  • Source: DSA2.4 - Continous Integration and Certification Testbeds

About using the testbed from the PT perspective

• As mentioned before, the certification testing resources of a service in insulation before integration is in charge of PT (see certification and testing guidelines for overview of certification process) so SA2.6 will focus on making available the testing infrastructure needed by each PT to test the interaction of its product with other PT's software products.
  • Source: DSA2.4 - Continous Integration and Certification Testbeds

• Therefore, in agreement with JRA1 and, SA1 leaders, SA2.6 expect collaboration from PT members involved in certification concerning:
  • 1. Providing effort for service installation on HW provided by SA2.6
  • 2. Keeping Service Status Logbook documenting all updates, configuration and information useful for the service usage from other PT
  • 3. Providing effort for support and debugging on issues related to the service they are in charge of.
    • Source: DSA2.4 - Continous Integration and Certification Testbeds

• Three types of access to testbed resources are expected:
  • 1. Tester users access to perform tests. This type of access will require common grid services user configuration, which may include both personal account for PT testers on User Interface services and dummy user accounts to simulate concurrent usage stress test on software products.
  • 2. Tester user access to collect information useful for tests debugging: access to logs, network configuration etc. This use case will not necessarily require root privileges to be granted to testers and could be implemented just making needed files available through https to users with specific certificates, or via gridftp whenever possible. The mentioned approaches would have the advantage of reducing the number of direct access to servers, which would involve matching the local sites (the sites hosting the testbed servers) security access policies for each participant site.
  • 3. Root access to server for service installation and configuration. Since granting root access to remote users poses a security problem, it will be regulated following local security access policies for each participant site.

• Account requests will be regulated according to communication procedures described in section 4.1.6 (see the following relevant bullets)
  • Procedures: the following support requests cases (from PTs, SW Area Manager, SA1, JRA1) are expected and treated as described below:
  • Requests for configuration support of existing services. These requests may include enabling VO/users, making services to talk to each other, custom bdii setup etc. Procedure:
    • Send either a request by mail to SA2.6 mailing list explaining the user's testing and configuration needs or open a savannah task on the testbed squad. The request will be then evaluated and tracked into a savannah task on the testbed squad.
    • If needed a PT members of services involved in the test will be contacted and his contribute is tracked by savannah tasks.
  • Requests for new services setup (particular RC Service versions):
    • Send a request by mail to SA2.6 mailing list or by savannah task on the testbed squad, explaining:
      • Your testing needs and the type and version of services you need to be installed and the PT producing that service
      • Please also specify if you need the service to be included in the permanent EMI testbed.
    • The request will be then evaluated and tracked into a savannah task on the testbed squad. If needed, the involved PT members will be then contacted and his contribute tracked by savannah tasks.
  • Requests for specific testbed (in this category: performance tests, security tests, data management tests, etc.):
    • Send a request by mail to SA2.6 mailing list or by savannah task on the testbed squad, explaining:
      • Your testing needs and an estimate of HW and SW requirements for your test
      • PTs involved in the setup and suggestions on possible sites/PT/NGI that may help in the setup. The request will be then evaluated and tracked into a savannah task on the testbed squad.
      • The period of time you expect to have the testbed on for
    • The involved PT members will be then contacted and his contribute is tracked by savannah tasks.

• Source: DSA2.4 - Continous Integration and Certification Testbeds

• EMI Internal Testbed documentation, https://twiki.cern.ch/twiki/bin/view/EMI/TestBed, reporting:
  • Coordination Procedures and overview
  • Testbed Inventory with a list of provided instances specifying: Middleware Suite, Service Deployed, Platform, Server Hostname, Site Location, reference Product Team, Status Logbook
  • Status Logbook field in previous table is a link to a instance-specific web page describing the hardware details of instance, software version installed, configuration information and history of updates. The maintenance of this page is in charge of people performing installations, configurations or updates (can be PT members).

Phase-out of components

• Security Area
  • UVOS --> SAML-based VOMS
  • glExec --> PAM Modules and OS
  • SCAS --> ARGUS

• CERN Data Management (FTS, DPM, LFC, GFAL, lcg_util ) - Contacted Oliver on 2011-01-31
• gLite MPI PT (mpi-start, MPI_utils) - Contacted Enol on 2011-01-31
• Logging and Bookkeeping PT (L&B server and clients ) - Contacted Zdenek on 2011-01-31
• gLite Security PT (Trustmanager/Util-java, LCAS/LCMAPS, glexec, SCAS (not EMI-1), LCMAPS-plugins-c-pep, Hydra, STS, Delegation Java, SLCS, Pseudonymity) - Contacted John on 2011-01-31
• VOMS PT (VOMS, VOMS-admin) - Contacted Vincenzo on 2011-01-31
• StoRM PT (StoRM) - Contacted Riccardo on 2011-01-31
• gLite Information System PT (BDII, Glue model, Service info provider, site info provider, Infosys test suit, two cli (lcg-info and lcg-infosites)) - Contacted Laurence on 2011-01-31
• gLite Job Management PT (BLAH, CREAM, WMS, CEMon) - Contacted Massimo/Marco on 2011-01-31
• Cesnet Security PT (glite-security-gss, glite-security-gsoap-plugin, glite-security-proxyrenewal and org-gridsite ) - Contacted Zdenek on 2011-01-31
• ARC PTs - Contacted Balazs on 2011-01-31
• UNICORE PTs - Contacted UNICORE team on 2011-01-31

DONE:

• VOMS PT (VOMS, VOMS-admin) - Contacted Vincenzo on 2011-02-04
• CERN Data Management (FTS, DPM, LFC, GFAL, lcg_util ) - Contacted Oliver on 2011-02-04
• gLite Information System PT (BDII, Glue model, Service info provider, site info provider, Infosys test suit, two cli (lcg-info and lcg-infosites)) - Contacted Laurence on 2011-02-04
• gLite Security PT (Trustmanager/Util-java, LCAS/LCMAPS, glexec, SCAS (not EMI-1), LCMAPS-plugins-c-pep, Hydra, STS, Delegation Java, SLCS, Pseudonymity) - Contacted John on 2011-02-04
• gLite Job Management PT (BLAH, CREAM, WMS, CEMon) - Contacted Massimo/Marco on 2011-02-04
• dCache PT (dCache Server and Client) - Contacted Patrick on 2011-02-04
• Cesnet Security PT (glite-security-gss, glite-security-gsoap-plugin, glite-security-proxyrenewal and org-gridsite ) - Contacted Zdenek on 2011-02-04

UNDONE:

• gLite MPI PT (mpi-start, MPI_utils) - Contacted Enol on 2011-01-31
• Logging and Bookkeeping PT (L&B server and clients ) - Contacted Zdenek on 2011-01-31
• StoRM PT (StoRM) - Contacted Riccardo on 2011-01-31
• ARC PTs - Contacted Balazs on 2011-01-31
• UNICORE PTs - Contacted UNICORE team on 2011-01-31

EMI JRA1 Deliverable M12 Tracking

EMI JRA1 GANTT Charts

EMI 1 Architecture Work

EMI 1 Release Work