EMI Data Activity Leader Phone Conference

Time	Wednesday Aug 24, 11:00
Participants	Jon, Oliver, Patrick, Riccardo Z., Michele, Christian excused due to technical difficulties

Minutes

(Patrick)

EU Review and some consequences (Details already sent around by Alberto d.M.)

• EMI - Data Deliverable 1.2.2.

  Deliverable has been accepted. Reviewers asking for more technical details for the next deliverable, e.g. UML diagrams whenever possible. Patrick needs more help next time.

• The new EMI project managment structure.

  Details will certainly be presented by Alberto in Padova, Oct 17-19. Important for us (emi-data) is that Jon is now our JRA leader together with Balazs.

• Activity tracking with Savannah (See the new EMI wiki page ). * Jon is in the process of creating one savannah ticket for each EMI development objective and all the dependent sub-objectives. Tickets are assigned to the PT or activity leader whenever possible. Generic EMI-Data objectives are assigned to Patrick. High level objectives to Balazs or Jon. Ticket owners are responsible for updating their tickets when necessary. If you are not updating your tickets, Jon will chase you down.
  • Jon will inform everyone when the update of the task tracker is complete.

Quick reports from the different Data activities (Oliver et al., Christian, Riccardo Z. et al., me et al.)

• Accounting Record.

  The common part for the accounting record is done. (Thanks to Jon). Now DPM, StoRM and dCache PT's have to present a suggestion on how to implement the design into their product. This is now closely watched by Jon as JRA leader.

• ARC and gLite data client library consolidation. (Jon)

  Jon : Plans are expected to be finalized in October. There might be slight delayes because of the vacation period. People now what to do an the bext f2f is planned for Padova.

• GLUE 2.0 migration

  Oliver : First step is done. All SE's report information via GLUE 1.3 and 2.0. Now the clients have to be adopted. This is followed by adding more information to the GLUE 2.0 branch, which the clients are supposed to make use of. It is not yet clear what that will be. There is a request to add details on the component version/release numbers. However this is opposed by some system-administrators based on security concerns. (Security by obscurity: Which is a bit funny as the security people have been asking for this additional information). We are planing to make extended version information available technically but site admins might decide not to pubish them. Details will be discussed in Padova.

• SRM over https instead of httpg plus delegation.

  This section will be filled after Paul is back from vacation.

• UNICORE and GRID (LFC and SRM)

  Christian reported the current status on the UNICORE activity wiki page.

• UNICORE and AMGA

  The first step of this objective is the evaluation of necessary efforts and expected benefits. As it seems now, the interest in this activity, by our customers, is not that overwhelming. Christian will finish a short evalution document within the next two weeks.

• POSIX file access (NFS 4.1/pNFS):

  dCache

  Production version is part of EMI-1. Used in production for Light Source experiements at DESY.

  DPM

  Beta version available in EMI-1. Testing done by 'friendly' sites in UK and in Taipei.

  StoRM

  No statement from IBM on when pNFS will be available for GPFS. They claim, they are rewriting the entire NFS stack for GPFS which is causing some delay. But as StoRM is providing a mountable file system anyway, this is not really an issue. It could become an issue as soon as we would like to build NFS 4.1/pNFS federations.

  General

  • DPM and dCache are still evaluating x509 integration. (Kerberos is done).
  • Industry vendors are very late. NetApp postoned the release to 'end of this year'. IBM delayed without further estimate on availability.
  • Linux client availability looks good. Full pNFS in 2.6.38.
  • Patrick will present the EMI pNFS efforts at ACAT (Uxbridge)

• Support of file:// in SRM
  • Technically this is not difficult. The client code has to offer the 'file:' protocol and the server simply has to accept it, if configured. The problem is that the server doesn't know how the file system (NFS 4.1 or GPFS) is mounted. Each workernode cluster could have a different mount point for the same file system. Here we need an agreement which covers future global NFS 4.1 as well. However, this problem is not a show stopper for implementing this feature in DPM and dCache already.
  • dCache: not done yet.
  • DPM: not done yet.
  • StoRM: done.
• Catalogue synchronization (including permission synchronization).
  • The catalogue sysnchroniztion wiki pages are up to date.
  • The message passing framework is done. DPM and LFC provide a pluggin already.
  • DPM doesn't automatically report missing files to the catalogues. (but could technically). We need to build some trust before we try convince system administrators to enable this feature. Right now, reporting 'missing files' is a manual operation.
  • Permission propagation with sync cat is implemented and can now be verified by the other SE people.
  • Fabrizio has a detailed description of the framework and how to write pluggins.
  • dCache and StoRM work on this hasn't started yet. dCache people will meet Fabrizio in Lyon to discuss on how to proceed further. Not clear yet if StoRM people will make it to Lyon.
• HTTP/WebDAV support in EMI SE's
  • dCache: done
  • DPM: well in time for EMI-2.
  • StoRM: well in time for EMI-2.
  • http/WebDAV is discussed later in the document again.

Activities

New

• FTS next generation, FTS3. (SALICHOS, Michail)
• Investigation of a common WebDAV interface for the LFC and SE's / SE suite (Leader: Oliver)
• Improve user satisfaction (very long list of things we have to do) (tracking : Jon and Patrick)
• Investigating into EMI Data Clouds based on recommendations of the EMI Cloud Strategry group. (Patrick)
• Persistent data ID's (Leader: Oliver)

From EMI-1

• NFS/pNFS: Patrick
• SRM over https and delegation: Paul
• GLUE 2: Oliver
• UNICORE related: Christian
• Accounting Record: Jon (but now individual PTs)
• Client library consolidation: Jon

Conferences

• The Lyon conference was already sorted out by e-mail, as the deadline was Aug 22. Accepted presentations on Monday morning:
  • Ricardo: DPM and LFC
  • Michail: FTS3, the next generation File Transfer Service
  • Jon: Common Data Library
  • Fabrizio: Catalogue Synchronization
  • Paul: dCache ...
  • Patrick: EMI the next year or something
• EMI All Hands meeting in Padua : Participation and presentations not further discussed.
  • As usual : Participation of EMI funded members is mandatory.
  • Patrick will ask for working group meeting rooms for at least 'cat sync', 'future of http', 'glue2' and the 'common data library'.

Special Topics

Catalogue synchronization in dCache and StoRM (How to proceed and timeline).

• dCache and StoRM people will make themselves familiar with Fabrizios documentation.
• dCache and hopefully StoRM people will meet in Lyon (Sep 19) and discuss details.
• If necessary we will have to arrange for a meeting (CERN or DESY) to discuss technical details of the dCache StoRM implementation.
• There might be issues with setting the permissions in the storage elements as Fabrizio took DPM as a prototype for doing the permisssion settings. Oliver stated that the cat sync framework is flexible enought to adjust for dCache or/and StoRM requirements if not too exotic.

General HTTP strategy in EMI.

Providing http(s) and WebDAV for all EMI storage elements is of course only a start in moving towards standard protocols for the EMI Data stack. We have to work on the following topics in order to proceed:

• We http/WebDAV clients which provide all the funtionality we need and work together with all our storage elements. Especially the WebDAV third part copy is of interest. Although there are plenty of open source and other clients around, there not a single one which does everything we need. One posibility is pick a client and to contribute to its developement.
• Even more important is to make the ROOT framework work with our SE's using http. Tigran already improved the ROOT http driver but authentication is not yet available. We would need to contibute to ROOT to get this fixed.

Patrick will bring this up in PTB the will find an activity leader who tracks those sub-tasks. (Not necessary an technical expert in this field.)

WLCG experts working groups.

The WLCG management has decided that the future of the LHC computing should be planed and decided within working groups, composed of people from various areas. This essentially replaces the GDB. For DATA, three working groups will be created

• Data Management
• Data Access
• Data Bases

EMI is asked to suggest members from EMI for those groups. EMI Data is suggesting Paul and Jon to contribute to Data Managment and Data Access. As EMI doesn't provide a database component, we are not proposing a person for the Database Working group.

Other

• People would prefer a more frequent EMI Data meeting. Patrick will arrange for those.
• Participation of the Data Access Federation workshop in Lyon Nov 21/22: Hasn't been discussed.

-- PatrickFuhrmann - 18-Aug-2011