(placeholder to see whether CASTOR can be made to support (and benefit from that support) immutable files).

POSIX file semantics says that a file can be updated in place, via concurrent writers (with undefined behaviour, but still...). This is difficult as soon as replication is introduced (invalidate other copies on write, read-write locks, transparent versioning etc - nice material for CS thesis-es but lots of error-prone code). And CASTOR as an HSM is inherently about replicating data, at least to tape..

Now, if everybody is using readonly files anyway (why would anybody edit their raw data? oh wait..), we could just simplify life for CASTOR, or at least allow for otherwise-too-expensive operations like persistent client-side caching or other "throwaway" replicas.

CASTOR changes

would need a new file attribute (suitably visible), would need to modify behaviour on write. Might try to use the existing POSIX flags (r--r--r-- means "can replicate"), but this would give us all kind of nasties whenever these permission change (e.g. make this a very slow update, to give time to all users to call back and invalidate?)

ideas of possible benefits

much easier code path

(i.e. no late checks for changes before writing to tape, no tape invalidation on write,consistent copies across service classes)

P2P CPU server "CASTOR"s - for performance

If we track which (CPU) machine kept a copy of the file, we could use that instead of going to a real disk server. We could even go towards bittorrent to spread the load across several machines.. with disk servers as "permanent seeders". Vast scalability improvement for "hot" files. Initial startup delay makes this attractive only above a certain size, of course. Extra copies would be throwaway, i.e. clients decide when to delete them.

stochastically reliable "cheap" hardware

(LOCKSS - "lots of copies keep stuff safe"-idea) - distribute files widely over cheap hardware that the chance of losing all (at least permanently) is comparable to MTBF of a RAID5, i.e. "good enough". Just requires tracking the copies and in-time replication if too many copies disappear.

Experiments

ATLAS

from "Managing ATLAS data on a petabyte-scale with DQ2" (pdf avail from IOP). ATLAS has two concepts: data sets (can be open/close/frozen) and files.

2.2. Dataset properties
Datasets as used for ATLAS also have certain distinct properties:
     [..]
Immutability Datasets hold a mutability state which can be open, closed or frozen.  [..]
     A dataset is said to be
     frozen when the latest version is closed and no new versions can be added. The dataset is
     completely immutable and cannot be versioned anymore to support discrete changes in its
     content.

2.4. Files
Files are the basic system units and they are immutable. 

Sounds good, we would just need to know that a file is part of DQ2-managed data.

ALICE

CMS

LHCb

everybody else

-- JanIven - 01-Apr-2010