JRA1.1 - EMI-1 Development and Test Plans

EMI 1 Component dCache

System for storing and retrieving huge amounts of data, distributed among a large number of heterogenous server nodes, under a single virtual filesystem tree with a variety of standard access methods.

• EMI File Access Solution
• Managed by data area
• Part of Product Team dCache [ Details ]
• Component in ETICS : emi.dcache [ Details ]
• Supported Platforms : All platforms supported by Java : Linux, Windows, Mac [ 2010-10-29 ]
• EMI 1 Release
  • dCache Version 1.9.12 (aka 'dCache Golden Release' - Long supported)
  • Elements
    • dCache Server
    • dCache Client
  • Tracker Item : https://savannah.cern.ch/task/?18583
  • Most known bug fixes provided in EMI 1
    • TBD
• SA2 Testbed
  • 2011-02-24 : Production Version deployed [ More Details ]
  • New features not already deployed, Maven building issues
• Verification and Validation Plan
  • not available yet

Overall Component Status

Mandatory Features for EMI 1 - Addressed Technical Objectives

• EMI technical objective Data Area 1 : All storage elements publishing initial GLUE 2.0 storage information and one storage client (library) is capable consuming that. (M12)
  • Addressed by developers : [In Testing and Certification]
• EMI technical objective Data Area 2 : Using https instead of httpg for the SRM protocol as a prototype implementation in one storage element and client (library). (M12)
  • Addressed by developers: [In Testing and Certification]
• EMI technical objective Data Area 3 : All storage elements offering support for the http(s) protocol (M12)
  • Addressed by developers: [In Testing and Certification]
• EMI technical objective Data Area 4 : All storage elements offering at least a prototype-level support for the "file://" access protocol. (M12)
  • Addressed by developers : [In Testing and Certification]

Mandatory Features for EMI 1 (ready)

The following features are implemented and considered to be part of the EMI 1 release 'Kebnekaise'.

GLUE 2.0 : The Information Provider in dCache will publish data using the GLUE 2.0 and GLUE 1.3 schema in parallel. However, initially the content will be the same. Over time the information published with GLUE 2.0 will become richer.

• Status: Developed; now in testing and certification phase
• Verification : Testing Strategy 1 - GLUE2 Information [ More Details ]
  • 2011-02-18: Patrick: Tests from the infrastructure area are used for this
• Milestone 1: 12/2010 - Development in beta quality present [Achieved]
• Milestone 2: 01/2011 - Tests in place for continous internal JRA1 testings during 02/2011 (JRA1.7, JRA1.8) [Achieved]
• Feature Deadline : 02/2011 [Achieved]
• Addressing EMI technical objective Data Area 1 : All storage elements publishing initial GLUE 2.0 storage information and one storage client (library) is capable consuming that. (M12)
• Development Task Tracker : [ Details ]

HTTPS : dCache provide http(s) access right now

• Status : Developed; now in testing and certification phase
• Verification : Testing Strategy 2 - SEs Access Protocols [ More Details ]
  • TBD : How is testing done exactly (Morris, Patrick)
• Milestone 1: 12/2010 - Development in beta quality present [Achieved]
• Milestone 2: 01/2011 - Tests in place for continous internal JRA1 testings during 02/2011 (JRA1.7, JRA1.8) [Achieved]
• Feature Deadline : 02/2011 [Achieved]
• Addressing EMI technical objective : Data Area 3 : All storage elements offering support for the http(s) protocol (M12)
• Development Task Tracker : [ Details ]

NFS 4.1 : dCache will provide a production ready NFS 4.1 (pNFS) implementation with standard Unix security. Kerberos security is envisioned but not guarantied

• Status: Developed; now in testing and certification phase
• Description: dCache fully implements NFS4.1/pNFS including kerberos security in EMI 1.
• Verification : Testing Strategy 2 - SEs Access Protocols [ More Details ]
  • TBD : How is testing done exactly (Morris, Patrick)
• Milestone 1: 12/2010 - Development in beta quality present [Achieved]
• Milestone 2: 01/2011 - Tests in place for continous internal JRA1 testings during 02/2011 (JRA1.7, JRA1.8) [Achieved]
• Feature Deadline : 02/2011 [Achieved]
• Addressing EMI technical objective Data Area 4 : All storage elements offering at least a prototype-level support for the "file://" access protocol. (M12)
• Development Task Tracker : [ Details ]

SRM security : dCache server and the dCache SRM client to communicate using the X509/SSL protocol instead of GSI (httpg) without delegation functionality

• Status: Developed; now in testing and certification phase
• Description: Code will be in EMI 1. Only functional testing will have been done. Not production or stress tests.
• Verification : Testing Strategy 3 - SRM Compliance [ More Details ]
• Milestone 1: 12/2010 - Development in beta quality present [Achieved]
• Milestone 2: 01/2011 - Tests in place for continous internal JRA1 testings during 02/2011 (JRA1.7, JRA1.8) [Achieved]
• Feature Deadline : 02/2011 [Achieved]
• Addressing EMI technical objective Data Area 2 : Using https instead of httpg for the SRM protocol as a prototype implementation in one storage element and client (library). (M12)
• Development Task Tracker : [ Details ]

Optional Features for EMI 1 (ready)

Web access, WebDav : For EMI 1 dCache will provide WebDav access. WebDav authentication via User/Password.

• Status: Developed; now in testing and certification phase
• Verification: TBD: Tests for WebDAV (Morris, Patrick)
• Milestone 1: 12/2010 - Development in beta quality present [Achieved]
• Milestone 2: 01/2011 - Tests in place for continous internal JRA1 testings during 02/2011 (JRA1.7, JRA1.8) [Achieved]
• Addressing EMI component improvements

Optional Features for EMI 1 (planned and/or in development)

Web Interface : Authenticated/Authorized web interface to manage pools and other system components.

• Status: In development
• Addressing EMI component improvements
• TBD: Verify from Patrick for EMI 1

Data re-balancing : Re-balancing data on pools after new pools are added or filled unbalanced.

• Status: In development
• Addressing EMI component improvements
• TBD: Verify from Patrick for EMI 1

Feature Summary for EMI - 1

P = Production Quality
experimental = Pre-Production Quality

Events

5. dCache Workshop" in Goettingen (16/17 March 2011)

• Description : Tutorials and hands-on will be provided during the workshop
• Link: [ More Details ]

Status Update and Tracking

...
2011-02-15 - Morris : Added tracker item
2011-02-15 - Morris : Requesting updates from Patrick about GLUE2
2011-02-24 - Morris Updated with information send from Patrick on status of technical objectives and features (2011-02-18)
2011-02-24 - Morris asked Patrick for some development clarifications of optional features