Appendix: EMI Test Plan Template

1. Introduction

This twiki page presents a template for the EMI Test Plan. PTs must use the template to make sure the following sections are present in the test plan for their EMI software products.

2. EMI Product Description and Version

The Test Plan must include a description of the EMI product. It could be a link to a service reference card, when it exists.

Sice the Test Plan is a document that will evolve and change with newer versions of the product, it must include the most recent product version for which the Test Plan is suitable. This means that the Test Plan contains all the necessary tests to test such version.

3. Unit tests

The Test Plan must describe the tools used to execute unit tests. Note that there is no need to describe individual tests, only give an overview of the strategy choosen to run unit tests. Unit tests must be automatically executed for any product release.

4. Deployment tests

The Test Plan must describe deployment tests. Deployment tests include installation and configuration tests.

Deployment tests must be done for at least rpm packages for EMI 1. For EMI 2, this document will be updated in the upcoming months.

4.1. Installation tests for EMI 1

Installation tests must define:

• YUM repositories needed to install the product version under testing.
• YUM command needed to test a clean installation, that is an installation of the product version under testing a clean machine.
• YUM command needed to test an upgrade installation, if applicable. That is an installation of the product version in production, using the EMI production repository, on a clean machine. And then an upgrade to the product version under testing.

4.2. Installation tests for EMI 2

This section will be updated in the upcoming months.

4.3. Configuration tests

Configuration tests must define the configuration commands and configuration files needed to configure the product under testing. The configuration test must be done for both clean and upgrade installations.

5. System tests

The Test Plan must describe system tests. The following sections describe some of the system tests to be included in the test plan.

5.1. Basic functionality tests

The Test Plan must describe basic functionality tests in the following way:

• For each functionality present in the product:
  • Description of the functionality to be tested (describing what it does and not how) must be included. If a test is available to verify and validate this functionality, this must be clearly stated as implemented, otherwise it should say not implemented.
  • In case the functionality is covered by a test:
    • Test for normal workflow: description of the test that proves that the functionality is present and works as expected. The following must be also included:
      • Input needed for the test.
      • Criteria for considering the test succesful (PASS) or failed (FAIL).
    • Test for error cases: description of the test that proves that the proper error messages are returned in anomalous scenarios.The following must be also included:
      • Input needed for the test to trigger the anomalous scenario.
      • Criteria for considering the test succesful (PASS) or failed (FAIL).

Implementing the missing functionality tests is an area for improvement that the PTs should consider in future versions of the test plan.

5.2. Regression tests

The Test Plan shall describe regression tests for software defects (bugs) tracked in RfCs relevant to the product.

Regression tests must contain at least the following information:

• Regression Test unique ID.
• If the Regression Test is not the RfC unique ID, then, include as well the RfC unique ID.
• Description of the test that will prove that the RfC has been properly implemented and that the problem it fixes is indeed not present any more in the product.
• Input needed for the test.
• Criteria for considering the test succesful (PASS) or failed (FAIL).

5.3. Performance and scalability tests

The Test Plan shall describe the strategy to implement load, stress, performance and scalability tests. If these tests are present, they should contain at least the following information:

• Description of the test.
• Description of the scenario under which the product will be tested. This should include parameters like time, load, stress conditions, etc.
• Input needed for the test. This should include scripts or any other tools used to recreate the special conditions needed to execute the test.
• Log files, memory usage, average response time or any other mean that proves the expected behaviour of the product.
• Criteria for considering the test succesful (PASS) or failed (FAIL).

5.4. Standard compliance and conformance tests

The Test Plan shall describe the strategy to implement standard compliance and conformance tests. If these tests are present, they should contain at least the following information:

• Description of the test.
• Description of the standard to be tested.
• Input needed for the test.
• Criteria for considering the test succesful (PASS) or failed (FAIL).

5.5. Inter-component tests

The Test Plan shall describe the strategy to implement inter-component tests. If these are present, they should contain at least the following information:

• Description of the test.
• EMI products involved in the test.
• Input needed for the test.
• Criteria for considering the test succesful (PASS) or failed (FAIL).