EMI Beta & Acceptance Testing

In order to improve the quality of its products and services and to increase the users confidence, EMI is providing access to interested parties to earlier stages/phases of the software development:

• beta phase - through participation in EMI Beta Testing activity
• release candidate phase- through participation in EMI Acceptance Testing activity

EMI Beta Testing

Access to *beta versions* of EMI products is provided to beta testers through access to EMI testing repository:

• beta version
  • bug fixes and/or new features/improvemets are already implemented
  • basic testing was done by the PT, enough to be confident it can be reccomended to beta testers
  • performance and/or stability tests have still to be done
• beta testers
  • experienced sites interested in having an early access, a technical preview of future releases
• EMI testing repository:
  • structure:
    • dist/EMI/testing/{1|2|3}/<platform\>/<basearch\>
    • for EMI 1 - http://emisoft.web.cern.ch/emisoft/dist/EMI/testing/1/sl5/x86_64/
    • for EMI 2
      • SL5 - http://emisoft.web.cern.ch/emisoft/dist/EMI/testing/2/sl5/x86_64/
      • SL6 - http://emisoft.web.cern.ch/emisoft/dist/EMI/testing/2/sl6/x86_64/
    • for EMI 3
      • SL5 - http://emisoft.web.cern.ch/emisoft/dist/EMI/testing/3/sl5/x86_64/
      • SL6 - http://emisoft.web.cern.ch/emisoft/dist/EMI/testing/3/sl6/x86_64/
  • content:
    • unsigned packages that will become part of the next stable distribution; in the certification stage, available for technical preview. Please DISABLE the gpgcheck (gpgcheck=0 in /etc/yum.conf) when using this repository.
  • yum repository files to be used.
    • emi-1-testing.repo
    • emi-2-testing-sl5.repo
    • emi-2-testing-sl6.repo
    • emi-3-testing-sl5.repo
    • emi-3-testing-sl6.repo
  • see Repository Configuration for more details
• notification
  • beta testers are notified of the availability of packages in the testing repository through a GGUS tkt opened by the EMI RM - see GGUS template for more info
  • beta testing duration is approximately 5 days, but it can be extended depending on the tested services.
  • eventual problems should be recorded in the ticket
  • developers are involved to provide workaround or eventual bug fixes
  • at the end of this period beta testers fill a small report, see report template for details, and recommend or not the versions tested by attaching the report to the GGUS ticket.

EMI Acceptance Testing

Access to *release candidate versions* of EMI products is provided to acceptance testers through access to EMI deployment repository:

• release candidate version
  • version with potential to be a final product, ready to be release unless critical bugs emerge
  • Certified by the PT and possibly by beta testers
  • available during an EMI Update Cycle; under EMI QC verification and testbed deployment
• acceptance testers
  • sites interested in having an early access to the stable versions of future updates
• EMI deployment repository:
  • structure:
    • dist/EMI/deployment/{1|2|3}/<platform\>/<basearch\>
  • content:
    • signed, tested, certified packages
  • yum repository files to be used:
    • emi-1-updates.repo
    • emi-2-sl5-updates.repo
    • emi-2-sl6-updates.repo
    • emi-3-sl5-updates.repo
    • emi-3-sl6-updates.repo
  • see Repository Configuration for more details
• notification
  • acceptance testers are notified of the availability of packages in the deployment repository through a GGUS tkt opened by the EMI RM - see GGUS template for more info
  • default acceptance testing duration is 5 days, but it can be extended to the Update Cycle duration, depending on the tested services
  • eventual problems should be reported in the GGUS ticket
  • developers are involved to provide eventual workarounds
  • at the end of this period acceptance testers file a small report, see report \template for details, and recommend or not the versions tested by attaching the report to the GGUS ticket.

Repositories Configuration

• base OS (Sl5/x85_64), EPEL and EGI Trustanchors repositories should already be enabled
• middleware repositories (EMI/UMD) should be enabled
• for Beta Testing - the EMI testing/ repository, mentioned above, should be enabled
  • IMPORTANT NOTE - testing/ repository contains not-signed packages. To avoid errors "gpgcheck=0" should be added to the /etc/yum.conf file
• for Acceptance Testing - the EMI deployment/ repository, mentioned above, should be enabled
• IMPORTANT NOTE - once the testing phase ended it is recommended to disable the testing/ or depoyment/ repositories

GGUS Template

Subject: 	EMI XXXX beta/acceptance testing
Notification Mode: on every change
CC to:  or 
Describe your problem providing the information listed here: 
Please assign to EMI Release Management SU

Dear Beta/Acceptance Testers,

The EMI   beta version is available in the testing/deployment repository:
_link_to_repository_
YUM .repo file:
_link_to_repo_file_

Please use the information provided in the task:
_link_to_task_in_emi_release_tracker_

Please let me know if you need something else.

Type of problem: Middlewaew
Priority: Less Urgent

Final Report template

*********************************
EMI Beta/Acceptance Testing Report Template 
*********************************

Product: 
Release Task:
Date 
===========================================
 Site Name:XXX-XXXXX 
 Site Admin: 

===========================================
Comment on the release notes:

===========================================
Comment on the installation/upgrading of the service
(Missing dependencies, services which should be restarted, etc.)

===========================================
Comment on the (re)-configuration (with YAIM)
(Issues/errors found when configuring) 

===========================================
Comment about the functionality of the service (info in service reference card)
(Have the daemons started/restarted OK, does it pass the Nagios tests, if a test is not iimplemented in Nagios do it yourself)

===========================================
Any other comment

-- DoinaCristinaAiftimiei - 18-Nov-2011