Attach the reviewed document to the deliverable page, put here a link

General comments

24/05/12 Florida

-Requirements policy not included?

It is mentioned in 2.2.3 Requirements Management, but I forgot to add a reference.

-Can you provide additional info to explain (or give an excuse ) for this "During the release of EMI 1 products, including EMI 1 updates, only the 9% of products released were compliant with the Quality Assurance policy." Why? When exactly - EMI 1 and all EMI 1 updates? What have we done to fix or improve the situation?

Already changed.

-Only if this is easy to do: Can you be consistent with some of the figure colors example Fig5 Pass has Green, Fig 6 Pass has Blue.

Sorry, not enough time to get new figures.

-Unlike Fig5-8, Fig9-10 are using actual numbers and not %s. Is this intentional?

Probably not, but I have not the original excel used to generate the charts, so it would take time to me to regenerate them.

-In nearly all cases an analysis of the metrics is included. It would be great we can do that for all, ex increase in SLOC after EMI 1.

Right, in the case of SLOC, the main reason are new features, I haven't had time to address this comment in the deliverable.

-SLOC graph - effort: Where did you get this?

Sorry, I have not generated this chart and I do not know.

-Under Major nonconformities, would it be possible to include the why? and what SA2 can do to improve the situation in Y3?

Some suggestion for improvements are in the conclusions.

-". This is a clear lack of commitment by PTs in fixing RPMLint errors, although other factors are influencing the measurements (i.e. miss configurations, dependency problems, etc.)" - is this the intended message? It may help if we include what other more urgent, important things they have to do.

Already changed.

-Terminology chapter should not be at the end (see template)

Moved.

25/05 Florida

1. Why is the SLOC graph only up to Q5? Because of the last available data form the SLOC monitoring page: https://twiki.cern.ch/twiki/bin/view/EMI/Emi1Slocs . Jon Kerr Nilsen mentioned that after that time the KPI changed and they (JRA1) stopped to calculate it, but this change has not been reflected by SA2 metric definition document, so Quality Control is forced to consider it to comply with SA2 QA policies. Anyway I added some lines explaining that the metric definition will be revised.

2. Where do you get the information about effort. For instance 9,863PM for EMI 1 was taken from where? what does it mean [support, maintenance, development?], what does 9863 EMI 1 PM effort mean? The data source is here: https://twiki.cern.ch/twiki/bin/view/EMI/Emi1Slocs and I have added a reference in the deliverable, but not all the details are provided.

3. Fix section 3.4.2 diagrams Done.

4. Please remove the 2 occurrences of "probably" • A general improvement on all metrics occurred moving from EMI 1 to EMI 2. Best results were achieved for EMI 1 Updates as PTs had most probably more time to fulfil SA2 policy. Done.

5. Still no SLOC analysis? Added some lines explaining that the present increment is due to the addition of new features.

6. References Replace all twiki deliverables URLs with their CDS counterparts. These are the official ones. Done.

Minor: please do a spell and grammar check. There are a few which can be easily fixed. Done.

26/05 Florida

I agree with Andrea about the security assessment, although there is nothing confidential or damning in the text. In the spirit of not divulging "security analysis" publicly [DSA233 is public], I suggest to move the text to another document which can be added/handed to whoever we want to give it.

Twiki refs to deliverables are still there, but don't worry, I will fix that when I produce the final version.

I give the green light to this doc after the Security part is moved to another document [please upload this other document to twiki as well, for now].

Additional recommendations (not affecting the document content, e.g. recommendation for future work)

Detailed comments on the content

Note 1: The reviewers must list here any observation they want to track explicitly and that require interaction with the authors
Alternatively all changes must be listed in the document itself using Word change tracking features (if you use Word)
Note 2: These comments have to be explicitly addressed by the authors and the action taken must be clearly described

Any other modification, spelling or grammatical corrections, etc must be done directly in the document using tracked changes or similar mechanisms that allows the authors to identify which correction is suggested.

-- FloridaEstrella - 24-May-2012