---

Computing Technical Design Report

3.4.1 Data Objects and Algorithms

Using computing science terminology, the Gaudi software architecture belongs to the blackboard family: data objects produced by knowledge modules (called Algorithms in Gaudi) are posted to a common in-memory database (or "blackboard") from where other modules can access them and produce new data objects.

This model greatly reduces the coupling between the Algorithms containing the algorithmic code for analysis and reconstruction, in that one Algorithm does not need to know which specific Algorithm can produce the information it needs nor which protocol it must use to obtain it (the "interface explosion" problem described in component software systems). Algorithmic code is known to be the least stable component of software systems and the blackboard approach has been very effective at reducing the impact of this instability, from the Zebra system of the FORTRAN days to the InfoBus architecture for Java components. The trade-off of this separation between data and algorithm objects is the need for algorithm objects to identify data objects to be posted on or retrieved from the blackboard. It is crucial to develop a data model optimized for the required access patterns and yet flexible enough to accommodate unexpected ones.

3.4.2 StoreGate: the ATLAS Transient Data Store

The Transient Data Store (TDS) is the blackboard of the Gaudi architecture: an Algorithm creates a data object and posts it on to the TDS to allow other Algorithms to access it.

Once an object is posted on to the store, the TDS takes ownership of it and manages its lifetime according to preset policies, removing, for example, a TrackCollection when a new event is read. The TDS also manages the conversion of a data object from/to its persistent form and provides therefore an API to access data stored on persistent media.

StoreGate is the ATLAS implementation of the TDS. It manages the data objects in transient form, it steers their transient/persistent conversion and it provides a dictionary allowing to identify and retrieve data objects in memory. The StoreGate design and implementation was largely driven by a few design concepts.

• Work with User Types . The success of the STL [3-10] and of other public domain template libraries means that it has become vital to design an open system that can work with generic types that export an interface, in particular the STL containers, rather than forcing data objects to import a common interface. StoreGate adapts its behaviour to the functionality each data object exports. The only StoreGate-imposed constraint on a data object is to be an STL Assignable type.
• Avoid User-Defined Keys . The disadvantage of the data/algorithmic object separation is the need for algorithmic objects to identify data objects to be posted on or retrieved from the blackboard. It is crucial to develop a data model optimized for the required access patterns and yet flexible enough to accommodate the unexpected ones.

StoreGate addresses this problem with a two-step approach: it defines a natural identifier mechanism for data objects and it transparently associates to each data object a default value of this identifier allowing developers to register and retrieve data objects without having to identify them explicitly.

The first component of the identifier is the data object type. Experience shows that HEP developers tend to group the objects they work on into collections, most often STL vectors. As a result the TDS will often contain a single instance of a data object type (e.g. a TrackCollection) or several related ones (e.g. a TrackCollection for each component of the Inner Detector). The StoreGate retrieve interface covers these two use cases.

Type-based identification is not always sufficient. For example the TDS may contain several equivalent instances of a TrackCollection produced by alternative tracking algorithms. Therefore a second component has been added to the identification mechanism: the identifier of the Algorithm instance that produced the desired data object. In the spirit of working with user types, StoreGate allows developers to augment this history identifier with a generic key optimized for their access patterns.

• Control Object Access and Creation . The TDS is the main channel of communication among modules. A data object is often the result of a collaboration among several modules.

StoreGate allows a module to use transparently a data object created by an upstream module or read from disk. A Virtual Proxy defines and hides the cache-fault mechanism: upon request1, a missing data object instance can be transparently created and added to the TDS, presumably retrieving it from a persistent database or, in principle, even reconstructing it on demand.

To ensure reproducibility of data processing, a data object should not be modified after it has been published to the store. The same handle/proxy scheme is used to enforce an ``almost const'' access policy: modules downstream of the publisher are only allowed to retrieve a constant iterator to the published object.

• Support Inter-Object Relationships . StoreGate supports unidirectional inter-objects relationships, or links. A link is a persistifiable pointer. If the linked object is a data object then the handle/proxy mechanism described above is also used to implement the link. But typically links will refer to objects that are not data objects but are contained within a data object. StoreGate knows how to get to the container and the container knows how to return an element given its index. The job of the link is to find out the value of the index, persistify it and, later on, pass it on to the container and get back the linked object.

3.4.3 Data Objects

As mentioned earlier, StoreGate is designed to work with user types rather than requiring them to implement a C++ interface. Basically any STL Assignable can be stored into StoreGate and hence is a Data Object.

A Data Object is a struct or class that encapsulates and "publishes" the result of some arbitrarily complex processing performed by one or more Algorithms. A Data Object should present a predictable, stable and efficient interface to client Algorithms.

StoreGate uses a compact, technology-independent mechanism to describe object types with two integer identifiers: a CLID and a VERSION . CLID is a 16-bit integer which uniquely identifies an object type across all ATLAS software for the lifetime of the experiment. If an object changes in a non-backward-compatible way a new VERSION number must be assigned to it2.

Data Objects recorded to StoreGate are subsequently owned by StoreGate and the creator must not delete them.

3.4.4 Accessing Data Objects

StoreGate provides the following support for access to Data Objects managed by it.

• Recording a Data Object . In order to record a Data Object, StoreGate must be provided with a pointer to the object instance (from which the type is derived), and with a key. The combination of type and key must be unique. Once a Data Object has been recorded, StoreGate takes ownership of it.
• Locking a Data Object. Once a Data Object has been recorded it is locked by default such that it cannot be modified by downstream Algorithms. This can be overridden if necessary.
• Retrieving a Data Object . Data Objects in StoreGate are retrieved by type and key. StoreGate sets a pointer to the requested object(s). StoreGate supports the concept of automatic retrieval of Data Objects from a persistency service: if the requested Data Object has not yet been read from disk, StoreGate will message the persistency service and the appropriate converter4 will create the Data Object.

StoreGate supports the retrieval of a single keyed instance of an object as well as providing iterators to allow retrieval of all objects of a given type.

• Checking if a Data Object is in the store . StoreGate provides methods to check whether a given Data Object has already been stored.

3.4.5 Using DataLinks to Persistify References

In C++ associations among objects are described using pointers or references. For example, a Cluster object may refer to its list of associated Cells by holding a vector of Cell pointers. Unfortunately a plain C++ pointer can not be simply written out and read back from disk; it is valid only within the context of a running job since it refers to a virtual memory location.

To address this limitation DataLink and ElementLink have been introduced, being two class templates which can be dereferenced like a pointer and can be read and written using various persistency mechanisms. The DataLink template allows one to reference a single Data Object, using its unique type/key combination. ElementLink is used to reference an element of a container recorded in StoreGate.