CASTOR dumps at CERN

Dear Flavia and all,
 As just discussed,
 I temporarily put a couple of the scripts that we use at CERN in

http://castor.web.cern.ch/castor/DIST/dumpDiskPool.pl

http://castor.web.cern.ch/castor/DIST/dumpDiskPoolavail

- dumpDiskPool.pl is the one generally used where you get a sorted listing
  of the paths of the files in the disk pool with fileid, size, status,
  diskcopy creation time and file last access time.

- dumpDiskPoolavail is a version developed for a special case where we
  list all diskcopies (so you may have several entries for the same 
path and fileid) with their availability information (based on the
  filesystem+diskserver one).

Both scripts assume that there is a one to one mapping of Castor diskpool 
to service class.

We run the scripts once a day early in the morning on weekdays (not in 
the weekend) to minimize impact on the stager DB.

They both put the dump in a directory in AFS that is accessed through 
http://cern.ch/castor Website.

We have a convention where the URLs are 
http://castor.web.cern.ch/castor/DiskPoolDump/..last
this is pointing to the latest instance of
http://castor.web.cern.ch/castor/DiskPoolDump/atlas..
for each pool so that you can bookmark the reference. Please note that 
older outputs are removed when creating the latest one.
For instance
http://castor.web.cern.ch/castor/DiskPoolDump/cms.t0input.last
or
http://castor.web.cern.ch/castor/DiskPoolDump/atlas.default.last

As you get a sorted list of the paths in the pool with their associated
information, you can exploit them in your own framework or use simple
scripts to do resource consumption queries
For instance
- The total file size of files in atlt3 in GB
cat atlas.atlt3.last | awk 'SUM+=$4 {printf("%f\n",SUM/1.e9)}' |tail -1
905.836689
- The total file size of the files in atlt3 in the hierarchy under
/castor/cern.ch/grid/atlas/atlt3/scratch in GB
cat atlas.atlt3.last | grep /castor/cern.ch/grid/atlas/atlt3/scratch | awk
'SUM+=$4 {printf("%f\n",SUM/1.e9)}' |tail -1
248.180882

-- FlaviaDonno - 08 Jul 2008