The Pre-Production Service

• gLite 3.0 timeline and high-level pps plans (presentation to Feb 28 MB).

The final gLite 3.0.0 Beta, including the new CE and WMS/LB, has been frozen and will now be passed to PPS for further validation.

The distribution includes components from gLite 1.5 and LCG-2_7_0 along with numerous updates. Patches and configuration updates will be released periodically after they have been tested.

Scope and Requirements from Experiments

The currently planned initial deployment of gLite 3.0 services on the pre-production service (PPS) is shown in the spreadsheet linked here. Those boxes marked with a "?" show where the service does not currently exist at a particular site but where we expect one or more experiments will want the service to be made available. At the meeting we will ask the experiments to give their updates to this spreadsheet. Please bear in mind that once gLite 3.0 has been deployed on the PPS, it will be working more or less at the maximum capacity of the people and nodes involved. Thus any additional services required by an experiment (including those marked with "?") will require that experiment to negotiate with the relevant site for extra PPS resources.

The assumption is that testing of new releases will be done in a number of stages, including:

• Initial testing of the software/services, including any new interfaces / functionality;
• Feedback loop of bug-fixing, documentation, deployment of fixed versions as required.

Once a reasonable level of confidence is reached, 'stress-testing' at an agreed level, e.g.

• Defined rate of job submission;
• Agreed number of sites;
• Agreed data rates / volumes across specified number of sites.

It is clear that there will never be sufficient resources to fully duplicate the production environment, but both basic functionality and limited scale tests need to be performed in as close-to-production-like conditions as is feasible.

This may well require some reconfiguration of some of the services and scheduling of test activities.

There was general agreement that the pre-production services will run on a separate (or factorised) infrastructure from the production services. e.g. separate BDII, distinct FTS channels, etc.

ACTION: Experiments to provide feedback on the list provided (whether there are any missing services) before next week's meeting.

ACTION: Ask for clarification if there is anything which is unclear, before next week's meeting.

ACTION: Experiments to provide feedback on metrics for success of pre-production. For meeting after next.

Timeline

• 2nd half of March: initial testing by experiments, resolution of critical bugs;
• 1st half of April: build up in scope / scale of tests, resolution of critical bugs;
• 2nd half of April: 'final' testing, shake-down at as large a scale as possible, final fixing and preparation of distribution kits.

It is inevitable that some problems will still slip through this procedure - which we need to continuously improve - and will require scheduled quick-fixes of the production system.

Experiment Production Plans and Issues

The monthly outline listed below is based on the SC4 Workshop in Mumbai and subsequent discussions.

We will update the schedule as required. Detailed resource allocation / coordination will be handled by Harry.Renshall@cernNOSPAMPLEASE.ch in conjunction with the sites / experiments and Bernd.Panzer@cernNOSPAMPLEASE.ch for the Tier0.

This will be discussed in more detail at the Management Board on March 7th, including level of detail required by sites etc.

March

• ALICE: bulk production at T1/T2; data back to T0
• GSSDATLAS: 3-4 weeks Mar/Apr T0 tests
• CMS: PhEDEx integration with FTS
• LHCb: start generation of 100M B-physics + 100M min bias events (2-3 months; 125 TB on MSS at Tier-0)

April

• ALICE: first push out of sim. data; reconstruction at T1s.
• GSSDATLAS: see above
• CMS: 10TB to tape at T1s at 150MB/s
• LHCb: see above
• dTeam: T0-T1 at nominal rates (disk); 50-75MB/s (tape)
• Sites: setup of FTS (or equivalent) services (see under Tier1-Tier1 transfers) & end-points; testing of all combinations of T1<->T1 and T1<->supported T2 transfers

• Detailed timeline for T0->T1 throughput tests can be found here.

May

• Deployment of gLite 3.0 at major sites for SC4 production
• Sites: see above

June

• ALICE:
• GSSDATLAS: Tier-0 test (Phase 1) with data distribution to Tier-1s (3 weeks)
• CMS: 2-week re-run of SC3 goals (beginning of month)
• LHCb: reconstruction/stripping: 2 TB/day out of CERN - 125 TB on MSS @ Tier1’s

July

• ALICE: Reconstruction at CERN and remote centres
• GSSDATLAS:
• CMS: bulk simulation (2 months)
• LHCb: see above
• dTeam: T0-T1 at full nominal rates (to tape)

August

• ALICE:
• GSSDATLAS:
• CMS: bulk simulation continues
• LHCb: Analysis on data from June/July … until spring 07 or so…

September

• ALICE: Scheduled + unscheduled (T2s?) analysis challenges
• GSSDATLAS:
• CMS:
• LHCb: see above

• all: Celebration of end of Service Challenges!

October

• Start of WLCG production service

Status of Production Services

We have agreed with the experiments how to configure their dedicated Castor-2 instances, and are deploying them. Status:

• Alice: in production since a few weeks
• CMS: ready
• GSSDLHCB, Atlas: to be delivered this week

We also want to deliver a setup for SC4 throughput tests this week.

Lastly, we are working on setting up an SRM endpoint for the throughput tests (v1.1), and a v2.1 SRM endpoint for interoperability tests. With the discussions about durable/permanent storage classes ongoing, these endpoints will not be the Mother Of All SRM Endpoints.

Jan van Eldik IT-FIO

Discussions about how to support durable and permanent storage classes in SRM v1 is still ongoing. It is likely that Castor and dCache will use different mechanisms to do this - dCache will use specific paths (taken from distinct SA roots) while Castor will use the makePermanent flag (currently unused flag of SRM v1 interface). The client tools will need to support both, guessing which behaviour to use based on the SRM type and contents of the information system.

Oracle Server Deployment at Tier1s

• What backend database services are required at which sites according to the agreed plan?
• What are the licensing and support issues if Oracle is the backend database?
• Which Tier1 sites do not currently have appropriate Oracle licenses and corresponding support contract?

• More info...