Deliverable Review Form

Attach the reviewed document to the deliverable page, put here a link

General comments

• The document does not provide page numbers. This makes reviewing and working with the document more difficult (e.g. when negotiating KPI targets with actual SLAs), but had cost me noticeable time sorting out pages in conjunction with section numbering glitches.
  
  Sorry about this. The template contains the page numbers. I do not know why they are missing and did not noticed it before sending the doc out.
  -- AlbertoDiMeglio - 14-May-2012
  
  

• The document provides as a new SLM element to EMI an OLA to regulate interactions and service descriptions between EMI and external software providers whose software products are agreed to be part of an EMI distribution release (e.g. EMI-2). OLAs are, however, designed to be used to coordinate internal teams and divisions to support an SLA agreed with a customer. an Underpinning Contract ("UC") is the equivalent to the OLA, and is agreed with external entities for the same reason, thus being the better SLM construct to regulate collaboration between EMI and, for example, EDGI. (see also: http://itsm.certification.info/uc.html http://itsm.certification.info/slavola.html http://www.hdiconnect.com/blogs/servicemanagement/030911/1300/service-level-management-basics-the-operational-level-agreement-ola.aspx)
  
  Your comment is formally correct. The reasoning behind using an OLA instead of a UC is that the EMI distribution contributors do not provide services to EMI, they still provide services to EGI (or Customers in general), but we want to make sure they also respect the standard SLA. We have considered the EMI distribution as an Organization and the EMI and EDGI or other providers are units within this larger organization. It's probably stretched, but it seemed to fit better than a UC in this case. I've explained the choice in more details in the document
  -- AlbertoDiMeglio - 14-May-2012
  
  

Additional recommendations (not affecting the document content, e.g. recommendation for future work)

Detailed comments on the content

Note 1: The reviewers must list here any observation they want to track explicitly and that require interaction with the authors
Alternatively all changes must be listed in the document itself using Word change tracking features (if you use Word)
Note 2: These comments have to be explicitly addressed by the authors and the action taken must be clearly described

N°	Page	Section	Observations and Replies	Is Addressed?
1	7	1.2	If deciding to use UC: Reword to reflect this change Please see replies to the general comments above. -- AlbertoDiMeglio - 14-May-2012	yes
2	9	2	3rd paragraph:_ EGI was _not considering all bank holidays and weekends as working days; this statement as is is wrong. Weekends were always considered being weekends. Bank holidays, however differ from member state to member state (EU), and therefore SLM has to find a way to unify this into a situation that is a compromise for all. Statement corrected in the text. -- AlbertoDiMeglio - 14-May-2012	no. The relevant section still states that EGI was considering public holidays as working days, insinuating that EGI expects members of EMI working on their public holidays. This is not true.
3	9	2	If deciding to use UC: Reword to reflect this change. SLAs are scoped around services delivered to the customer without the customer knowing how they are delivered (internal or external). OLAs and UCs define how and who delivers these services, i.e. internal (OLA) or external (UC). Please see replies to the general comments above. -- AlbertoDiMeglio - 14-May-2012	yes
4	10	3	_2nd paragraph:_ Contracts are referred to but no relationship between SLA and contract is given or described. Added better description of the relationship between service contracts and SLA. The SLA defines the service levels to be provided for the service that are the object of the contract. -- AlbertoDiMeglio - 14-May-2012	yes
5	11	3.2	This section is a bit unclear and diffuses the concepts between SLA, OLA and UC. While the SLA (or a set of) exposes the union of all services delivered from the provider to the customer, the OLA and UC support this SLA as contracts with other functional units. Whether visible or invisible services (as service chains), OLAs are agreed with functional units of the same administrative domain (i.e. organisation-internal divisions and groups), and UCs are set up with external entities outside of the administrative domain. Thus OLAs and UCs are very similar, except that they may differ in terms of legal aspects. Therefore it is suggested to change the constructs from OLAs to UCs with external suppliers (e.g. EDGI). Please see replies to the general comments above. The OLA is considered a better mechanism when thinking of EMI as a single organization of which the EMI Project and the additional projects are both contributors. This is also more in line with the goal of making the EMI distribution a community endeavour beyond the duration of the EMI project -- AlbertoDiMeglio - 14-May-2012	yes
6	16	5.1	Table 1 and Table 2_ The number of tickets _assigned to EMI, and the number of tickets closed by EMI cannot be directly compared in any given timeframe, because not all tickets assigned in in a given time period are not closed in the same time period (e.g. a ticket assigned on 27 March may be closed on 15 April) - these tickets either need to be taken out of the figures, or explained why they are included. The tables report the tickets submitted to the 3rd level and closed by 3rd-level in the same period of time. The number are comparable. The glitch in the reasoning if whether any of the closed ticket was submitted before the reference period. I need to check this and remove from the figures all closed tickets that were not submitted within the same timeframe -- AlbertoDiMeglio - 14-May-2012	yes
7	17	5.1	_Table 3 and Figure 1:_ Correlating # of violations with # of closed tickets over a full year does not make sense and blurs the picture. SLA violations and # of closed tickets vary over the months in the year (as Figure 1 demonstrates), so the actual # of violations should be correlated with # of closed tickets on a monthly bases, IMHO - or explained why not. Both figures are meaningful in my opinion. The figures over a full year period give the general trend and help identifying any structural issue (for example if the number of violation were really high). The monthly figures help assessing whether time-bound conditions were affecting the service provision (for example the code freeze, as shown). The two sets have to be taken together. Giving only monthly figures would not help in understanding the performance over the full period. In addition, we are changing the info to show the number of violation over all escalated tickets, not only the closed ones. There may be tickets that are in compliance or in violation of the SLA and still open, since the resolution is still in progress. -- AlbertoDiMeglio - 14-May-2012	yes
8	17	5.2	2nd paragraph: The total number of violations increases with the ticket priority, while the relative number drops from Top Prio to <5%. Please clarify in document. The # of violations correlate with the average response delay (perhaps a better term than deviation as deviation already has a statistical meaning). IMHO there is no correlation with the relative number of closed tickets - this is a mere side-effect of there being less numbers of tickets with high priorities, and many more with low(er) priorities. The explanation is left to the author. :-). Right, indeed I meant "relative number" in percentage, not actual number. Thanks for spotting this, I've fixed it in the text. For the second part of your comment, I think we agree. I do not correlate the number of violations with the number of tickets, but with their severity levels and the priority they get when they enter into the EMI support system. The more stringent the service level, the more chances you have to be late because of the short time; the lower the priority, the more chances of being late because of other activities to divert effort. I don't think there is any correlation between the # of violation and the response DELAY, though. -- AlbertoDiMeglio - 14-May-2012	yes
9	18	5.2	last paragraph:_ This statement is worring as it indicates a structural problem that needs to be addressed, not a glitch/outlier that need not be worried about.BR% Well, no objections here. The fact is that effort is fixed. It's a structural problem in the sense that given the current levels of work, the available effort and the service levels, there are periods of time were we cannot provide the agreed level. This means to me that _in the current conditions it would be better to change the service level and increase the max response time. I've written this in the document now. -- AlbertoDiMeglio - 14-May-2012	yes
10	19	6	_2nd paragraph:_ This contradicts the 2nd paragraph in section 2. The SLA was signed with EGI in April 2011, at least the version I know about.BR% True, v. 1.0 was submitted to EGI in September for evaluation, but not signed until v. 1.3 in April. I've revised the text. -- AlbertoDiMeglio - 14-May-2012	yes
11	24	SLA ToC	Contains the ToC of the OLA. Should be removed. Alternatively maintain the actual template in a separate document.BR% This is a glitch in the Word ToC functionality (or at least there is no way I know to make it work). The ToCs contain all headings in the doc, there is no way of bounding a ToC to a section. This is fixed in the PDF version, but will always be present in the DOC version. -- AlbertoDiMeglio - 14-May-2012	yes
12	28	5.1 (6.1)	Section numbering got garbled up (is 6.1 but should be 5.1) - all subsections of section 5 are affected by this.BR% Fixed. -- AlbertoDiMeglio - 14-May-2012	no, still errorneous numbering in PDF version.
13	31	5.3.2 (6.3.2)	Link http://www.eu-emi.eu/retirement_calendar is a dead link (HTTP/1/1 404 NOT FOUND error message)BR% This page seems to have been lost in the web site redesign a few months ago. It is being recreated. -- AlbertoDiMeglio - 14-May-2012	yes
14	33	5.4.1 (6.4.1)	The link to GGUS should be updated to https://ggus.euBR% Fixed. -- AlbertoDiMeglio - 14-May-2012	yes
15	49	5.1 (11.1)	Section numbering got garbled up (is 11.1 but should be 5.1) - all subsections of section 5 are affected by this.BR% Fixed. -- AlbertoDiMeglio - 14-May-2012	yes
16	51	5.3.2 (11.3.2)	Link http://www.eu-emi.eu/retirement_calendar is a dead link (HTTP/1/1 404 NOT FOUND error message)BR% This page seems to have been lost in the web site redesign a few months ago. It is being recreated. -- AlbertoDiMeglio - 14-May-2012	yes
17	53	5.4.1 (11.4.1)	The link to GGUS should be updated to https://ggus.euBR% Fixed. -- AlbertoDiMeglio - 14-May-2012	yes
18	55	6.1 (12.1)	Section numbering got garbled up (is 12.1 but should be 6.1) - all subsections of section 6 are affected by this.BR% Fixed. -- AlbertoDiMeglio - 14-May-2012	yes
19	55	6.2 (12.2)	The response times given in this document are the identical as the ones given in the SLA template, allowing on EMI-internal operational margin. The OLA (UC) response times should be lower/shorter (in general better) than the ones agreed to with the customer. This observation may not be applicable for the templates as such, but when negotiating this may become relevant.BR% This is indeed one of the reasons we prefer to use an OLA rather than a UC. The service levels are not to provide a service to EMI so that EMI can provide a service to EGI. Rather is to make sure that the contributor respects the same service levels as EMI. In practice, we say "if you want to be in the EMI distribution, you must comply with this service levels". We expect Top Priority issues to be acknowledged in up to 4h, Very Urgent in up to 2 days and so on. However, I agree that this depends on how the tickets are escalated. If the contributor has a standard GGUS SU, then the escalation is (can be) done in the same way as standard EMI SU. If the contributor has a different user support system, then there may be a further escalation and in this case it is the chain of escalations that must support the final SLA. I agree it must be taken into account in specific OLA instances. -- AlbertoDiMeglio - 14-May-2012	yes
20	57	9.1 (15.1)	Section numbering got garbled up (is 15.1 but should be 9.1) - all subsections of section 9 are affected by this.BR% Fixed. -- AlbertoDiMeglio - 14-May-2012	yes