Testing and certification

Testing and patch certification

Testing is mainly done in the course of patch certification. The certification process is being described in various EGEEDocuments (Developers Guide and Software Process).
For patch certification an How-To is available which explains the steps involved in the patch certification process. Depending on the services affected by the patch a defined set of functionality tests has to be executed. The page SA3Testing lists, for each service, the test plans, available tests, tests reports and has specific information for certifiers.

These tests can all be executed on the command line and the results must be made available (pasted as a comment to the patch, attached as a file to the patch, link to some result page). A comment on the results of the tests must also be put into the patch. If possible verification of a bug should result in a regression test for the bug. Whether this is possible or not depends on the nature of the bug. If in doubt contact Gianni Pucciani. Regression tests should be deliverd soon after patch certification.

For the current status of certification have a look at

Patches In Certification

and

Patches Ready for Certification.

Test writing

Tests should be written to be executed on the command line. Guidelines on how to write tests are provided on the page

Test writing guidelines

For guidance on how to write regression tests have a look at the README provided with the regression tests. Eventual integration into other frameworks (SAM, Nagios, ETICS) will be considered in due course. An evolving list of missing tests is available

Missing tests

When writing new tests first provide a test plan:

• List which functionality should be tested
• List which command line interface (CLI) commands should be tested
• List which functions/methods in the API should be tested

Write the tests and check if the test plan is fulfilled. Don't forget to provide documentation for the tests. Tests will be checked in in our CVS repository

Test repository: org.glite.testsuites.ctb

Test reports

Links to all available test reports, listed by component, are available as part of the SA3testing page.

Test infrastructure and framework

A stable testbed providing all services (including a dedicated CA) is maintained at CERN. Partners can add/remove resources to this testbed by adding/removing entries in the top level BDII. We don't ask partners to maintain a stable site but rather emphasize on providing exactly the resources necessary for certifying a patch or doing dedicated tests. Details about the testbed can be found on the testbed page.

The certification testbed will be monitored by Nagios (work in progress). Eventually tests will be integrated into Nagios. The SAM test framework is no longer being used in certification. Currently we investigate the possibility to move certain tests into the ETICS framework. More information about testing within ETICS can be found here.

Testing of gLite components is also being done independently from patch certification, e.g. general stress testing of components. Links to different test results are available on the test results page.

Expertise table

This table together with the who's who table should be enough to find out who's who, who's doing what, and who's willing to do what.

Component	Patch development	Patch certification	Test development	Extra
Accounting (APEL)	Cristina Del Cano Novales (STFC)	John Walsh (TCD)	John Walsh (TCD)	-
	-	Nikos Voutsinas (GRNET)	-	-
	-	Andew Elwell (CERN)	-	-
AMGA	Asterios Katsifodimos (UCY)	Asterios Katsifodimos (UCY)	Asterios Katsifodimos (UCY)	-
	-	Nicholas Loulloudes (UCY)	Nicholas Loulloudes (UCY)	-
BDII(also other info system)	Laurence Field (CERN)	Christian Neissner (IFAE)	Christian Neissner (IFAE)	-
	-	Nikos Voutsinas (GRNET)	Felix Lee (ASGC)	-
	-	Andew Elwell (CERN)	-	-
BLAH		Nikos Voutsinas (GRNET)	Elisabetta Molinari (INFN)	-
	-	Andew Elwell (CERN)	-	-
CREAM CE		Nikos Voutsinas (GRNET)	GRNET	-
	-	Andew Elwell (CERN)	Alessio Gianelle (INFN)	-
	-	-	Paolo Andretto (INFN)	-
Condor	Christian Neissner (IFAE)	Dennis van Dok (FOM)	Christian Neissner (IFAE)	-
dCache	Owen Synge (DESEY)	Gianni Pucciani (CERN)	-	-
DPM	-	Ricardo Mendes (CERN)	-	-
	-	Dimitar Shiyachki (UoS)	-	-
Data management tools	-	Gianni Pucciani (CERN)	Fidel Almagro (CERN)	-
	-	Ricardo Mendes (CERN)	Victor Galaktionov (JINR)	-
FTS	-	Gianni Pucciani (CERN)	Dimitar Shiyachki (CERN)	-
	-	-	Gianni Pucciani (CERN)	-
GFAL	-	Ricardo Mendes (CERN)	-	-
	-	-	Gianni Pucciani (CERN)	-
LCG CE	Andew Elwell (CERN)	Andew Elwell (CERN)	Felix Lee (ASGC)	-
	-	-	Nikolai Klopov (PNPI RAS)	-
LB	-	Lorenzo Sbolgi(CERN)	Zdenek Sustr (CESNET)	-
	-	Andrew Elwell (CERN)	Jan Pospisil (CESNET)	-
LFC	-	Ricardo Mendes (CERN)	John Walsh (TCD)	-
	-	Dimitar Shiyachki (UoS)	-	-
LSF	-	-	-	-
MDM	-	Kalle Kapponen (HIP)	Kalle Kapponen (HIP)	-
MyProxy	-	-	-
RGMA	-	John Walsh (TCD)	-	-
SGE	Alvaro S.Garcia (CESGA)	Esteban Freire(CESGA)	-	-
	Esteban F. Garcia (CESGA)	Dennis van Dok (FOM)	-	-
SCAS	-	Gianni Pucciani (CERN)	-	-
	-	Andrey Kiryanov	-	-
Glexec	-	Gianni Pucciani (CERN)	-	-
	-	Andrey Kiryanov	-	-
SD	-	-	Steve Fisher (STFC)	-
	-	-	Antony Wilson (STFC)	-
SLCS	-	Nikos Voutsinas (GRNET)	GRNET	-
Torque	-	Nikos Voutsinas (GRNET)	GRNET	-
	-	Jan Just Keijser (FOM)	-	-
UI	-	Lorenzo Sbolgi (CERN)	Felix Lee (ASGC)	-
	-	Andrew Elwell (CERN)	-	-
WMS	-	Lorenzo Sbolgi (CERN)	Alvaro Fernandez (CSIC)	-
	-	Andrew Elwell (CERN)	Gabriel Amoros (CSIC)	-
	-	-	Alessio Gianelle (INFN)	-
	-	-	Paolo Andretto (INFN)	-
	-	-	Nikolai Klopov (PNPI RAS)	-
WMProxy	-	Alvaro Fernandez (CSIC), Lorenzo Sbolgi (CERN)	Lyudmila Stepanova (SINP MSU)	-
	-	Andrew Elwell (CERN)	-	-
VOBOX	-	Christian Neissner (IFAE)	Christian Neissner (IFAE)	-
VOMS	-	Dimitar Shiyachki (CERN)	Dimitar Shiyachki (CERN)	-
	-	Andrew Elwell (CERN) (for client)	-	-
YAIM	-	Felix Lee (ASGC)	-	-
	-	Lorenzo Sbolgi (CERN)	-	-
	-	Andrew Elwell (CERN)	-	-

Contact

Gianni Pucciani Gianni.Pucciani@cernNOSPAMPLEASE.ch

-- AndreasUnterkircher - 24 June 2008