MILESTONE TARGET as FROM EMI DOW:

• SA2.6 ROLE: "In addition, SA2 is responsible to negotiate and coordinate the availability of larger-scale testbeds for executing scalability tests and the final acceptance tests required to validate release candidates in conditions as close as possible to real production environments. SA2 will seek the collaboration of EGI, PRACE and individual NGIs to get shares of resources from qualified resource centres participating in pre-production or staged roll-out activities."
• KPI: Number of CPUs available for distributed testing through collaborations with external providers (NGIs sites, commercial providers, other projects, etc) Participating sites monitoring tools Year 1: 50 CPUs Year 2: 200 CPUs Year 3: 500 CPUs

PROBLEM AT MATCHING THE TARGET AS WRITTEN IN THE DOW: after EGEE PPS experience and preliminary negotiation with EGI operations, a permanent infrastructure devoted to Release Candidates testing is not considered effective nor feasible. In fact:

• very difficult to achieve: No official Commitment from EGI Operations in providing testing resources. Moreover already difficult for EGI to have NGI sites available for the staged roll-out
• also very unefficient: something similar has been experienced in the old EGEE-PPS stable infrastructure which was: * expensive in effort to mantain * never fit for any test in particular to simulate the production environment at the right scale, or with specific key indicators being tested.

WORK DONE UP TO NOW:

1. Contacted both EGI and Technical director in EMI.
2. Change of model with respect to EMI DOW. New approach: "demand and supply" testbed approach, meaning:
   • DEMAND: A request of test describing the testing scenario (resources needed, services involved, number of instances, number of users involved etc.etc.). This request can come from EMI technical area (PT, Technical area managers, Release manager) or from EMI user community (EGI, NGI...)
   • SUPPLY: A community of partners (NGI, EGI, PRACE, EMI internal partners) available to participate to specific tests campaigns with X effort, Y resources for Z time to test P1, P2, PN product. Declaring availability just for specific products testing will enhance the chance of having motivated contributors, which is a key aspect for the success of the model.
     • NOTICE THAT: * The role of EMI SA2 to implement this model is then to provide operational resources (coordination, communication channels, expertise in testbed setup, general purpose utilities (fake certificates or fake CA generation utility)) to match DEMAND and SUPPLY. * In this model the KPI are matched "statistically", i.e. as an average on the year: we setup a specific testbed on demand and measure the overall availability of all testbeds (Amount and TIME) put in place over the year. The target of 50 CPU in this new approach seems not so meaningful. A better KPI would be: number of setup testbed and overall availability. * Finally, notice that this model found the agreement of both EMI Technical director B.Konia and EGI operations staff.
3. Partial setup of a twiki reporting the model above, need feedbacks to be completed: https://twiki.cern.ch/twiki/bin/view/EMI/LargeScaleEMITestbed

WHAT IS MISSING, so that the Milestone cannot be considered achieved:

• Up to now, we have no partners wishing to give us 50 cpu permanently. I just contacted EGI, cause of historical proximity with EGI people. Don't know about PRACE or other partners. Moreover negotiation with external partner in my opinion should involve both EMI board representatives and EMI outreach and dissemination unity. Ask for resources under the hat of EMI SA2.6 task leader is less effective.
• More in practice, we miss the formal agreement of community declaring their availability at participating in testing campaign. EGI position (after negotiation) can be summarized as:

"EGI periodically collects requirements from the user community and the operations community to
foster innovation. These are provided to third-party software providers - including EMI - to ensure
that the deployed technology meets the needs of these communities.

The partners providing the EGI requirements may participate in pre-release testing activities either
with direct participation to testing, or with their own resources to make sure that the software 
developed meets their requirements, and to verify its usability and scalabilityin a production 
environment, as applicable. This activity is voluntary relationship between EMI and the group 
involved and does not form any part of the software verification or validation activities undertaken 
by EGI in formally accepting the software from external technology providers.

The Chief Operations Officer and Chief Community Officer can be contacted by EMI and third party 
software when involvement of EGI partners is felt to be needed in testing activities. In any case, EGI 
requirements will be collected and tracked in a transparent way to ensure that the interested 
partners are recorded."

CONCLUSION and QUESTION FOR ECB:

• Do you agree on the model and KPI change?
• About the delay:
  • Either we consider the milestone achieved without having the certainty that we'll have the resources when somebody asks for the testbed or we look for resource providers within EMI partners: PT resources? They must be somehow committed to do so. Although using EMI internal resources could be good for enlarging the scale of testing resources but less effective at simulating the production environment.
  • We postpone the milestone and try to negotiate with other partners other than EGI. On this regard I suggest the EMI board and outreach and dissemination WP to be involved in the negotiation.