Deliverable Review Form

Attach the reviewed document to the deliverable page, put here a link

General comments

Jon

It looks much better now, I believe all my comments were answered in v1.2. The deliverable is now acceptable for me.

Alberto

Hi, thanks for the comment, you are right about your points. The KPIs were wrong and I changed them, see in the document if is ok what I have put as comment.

For SLOC I removed then section as the KPI is not there anymore as a KPI.

25/05/12 Jon

I cannot accept this document in its current state. As far as I can see, none of my comments concerning SLOC and KPIs were taken into account.

Before I can accept this document, please provide answers on what is done to address each of my comments (both on the reviewer page and in the document), one by one, and if not addressed, give a justification on why.

25/05/12 Claudio

About the in-line comments: JN1: added reference to development tasks JN2: changed accordingly

About the KPI: I understand your point of view, but Quality Control cannot relies directly on DoW (where is the last version?), but it needs to comply with SA2 Quality Assurance policies and in particular the metric definition document (https://twiki.cern.ch/twiki/pub/EMI/QualityModel/EMI_SA2_MetricsSpecification_v1.4.doc), which does not reflect yet the changes you mentioned, so there was some communication problem between SA1/JRA1 and SA2. Anyway I have added some lines explaining that the SLOC metric will be revised for the next updates/release. By the way even the KPI wiki page is outdated: https://twiki.cern.ch/twiki/bin/view/EMI/EmiKPIs .

23/05/12 Jon

There seems to be a major confusion about JRA1 KPIs. These were changed in the revised DoW. The old KJRA1.1-4 are now replaced with the following:

Name: Number of EMI service interfaces and libraries passing standard compliance tests

Description: The metric measures how many EMI service interfaces and libraries are successfully tested for standard compliance. Standard compliance is defined broadly and also includes compliance with EMI internal agreements.

How to measure: The number is taken by checking the available test reports generated during the quarter by the Product Teams.

Target: JRA1 aims to increase the number of successfull tests at least by two per reporting quarter.

KJRA1.2 - "Product integration KPI":

Name: Number of passed inter-product tests

Description: The metrics shows how EMI products can be used together by passing inter-product tests based on real-life use case scenarios.

How to measure: The number should be taken from the available test reports produced during the quarter. Each passed test should be assigned to a numeric value corresponding to the number of products involved in the test. The metric itself is the sum of the assigned test numbers.

Target: JRA1 aims to increase the metrics value at least by four per reporting quarter

KJRA1.3 - "Harmonization KPI":

Name: Number of EMI products implementing EMI agreements

Description: This metrics shows how the EMI harmonization is progressing by measuring the commitment and actual work to implement EMI agreements.

How to measure: The number of ongoing or completed development tasks of the EMI development tracker targeting EMI agreement implementation will be counted.

Target: Based on the currently known agreements, approximately 20 by the end of the project.

The last time the old KPIs were reported was halfway into the reporting period (in Q6), but this is not reflected in the report. I've tried to highlight the sections/parts were old KPIs are used, but not sure if I found all places that need change/rephrasing. I suggest you go through the document and revise the parts were the old KJRA1.1-4 are used, referenced or being a motivation.

Other than that, only minor comments, see attached tracked version.

Additional recommendations (not affecting the document content, e.g. recommendation for future work)

Detailed comments on the content

Note 1: The reviewers must list here any observation they want to track explicitly and that require interaction with the authors
Alternatively all changes must be listed in the document itself using Word change tracking features (if you use Word)
Note 2: These comments have to be explicitly addressed by the authors and the action taken must be clearly described

Any other modification, spelling or grammatical corrections, etc must be done directly in the document using tracked changes or similar mechanisms that allows the authors to identify which correction is suggested.

-- FloridaEstrella - 24-May-2012