Week of 051107

Open Actions from last week:

• Look at queries in FTS for locking problem (Gavin/Kris) IN PROGRESS

• move VOBOXes to Quattor (Simone)
• check possiblility of using LCG Quattor WG componets for LFC/DPM/... (Jan/Vlado/Sophie) TO TEST
• Escalate problem with lost packets on loopback I/F to linux.support (Vlado) ESCALATED

• QF for FTS memory leak code - beginnning of this week (Paolo) IN PROGRESS

Chair: Maarten

On Call: Sophie + James

Monday:

Log: Nothing

New Actions:

• Gonzalo wants to talk to someone about castor/srm on IA32/SLC3 instabilities (Vlado) DONE
• ASCC reports poor throughput after upgrade - to test DONE
• Lyon wants to switch to SRM Copy (fts-support)

Discussion:

• QF is done. Waiting on Alberto to build it - sometime this week.

Tuesday:

Log:

New Actions:

Discussion:

Wednesday

Log: Nothing

Actions:

• redraft OPM procedure for LFC to deal with LFC_DB_ERROR (James) DONE

Discussion:

• Problem from yesterday with LFC_DB_ERROR alarm understood - it will automatically go away after 30 min.
• ASCC problem was a problem at their end (bad castorsrm config)

Thursday

Log: Problem with LFC Pilot backend - machine was accidentally re-installed. DB team reinstalled from backup - lost 9 minutes worth of data. Service now running again

Actions:

• Check for updates after last backup on pilot (James/Sophie)

Discussion:

Friday

Log: thread exhaustion on lfc008 (lfc-atlas). This is not understood fully - seems to be a problem perhaps in the TCP layer of the kernel. Importantly, we didn't have an alarm to find this. Also cleanup is not good, since we need to kill the process explicitly.

Actions:

• New LFC sensor to detect current thread usage, and external service availability via CLI tools (James)
• check what LFC init.d script does if lfc-shutdown doesn't work (Sophie)

Discussion:

• Gavin reported that 1.4.1 is still not out, and our QF for the memory leak will not be out before end of next week - This is very late, and too long a delay for a "quick" fix.