A prepare method for Ganga applications: Developers Guide

This page documents the 'prepared' state for Ganga Core applications (introduced in Ganga 5.7.0) for Ganga application developers.

For a user's perspective of the Prepared state, see GangaPreparedApplications

Enabling the prepare functionality

Using the example of the Root application, we will demonstrate how to develop an application such that it can exploit the full functionality of the prepared 'state'.

Inheritence and Schema definition

First off, inherit from the IPrepareApp module

from Ganga.GPIDev.Adapters.IPrepareApp import IPrepareApp

class Root(IPrepareApp):
<Application class definition>

and add the is_prepared attribute to the application schema:

'is_prepared' : SimpleItem(defvalue=None, strict_sequence=0, visitable=1, copyable=1,\
      typelist=['type(None)','str','bool'],protected=0,comparable=1,doc='Location of shared resources.\ 
      Presence of this attribute implies the application has been prepared.')

Export at least the following two methods:

    _exportmethods = ['prepare','unprepare']

At this point, a decision should be made over which of the existing attributes should be read-only after the prepare method has executed. This is usually what we desire for attributes which hold file-type objects, and which will consequently be copied into the users SharedDir during preparation. For such attributes, the is_prepared schema attribute requires its preparable meta-attribute setting to True, as above.

The prepare() method

At the very least, you will want to define a prepare() method along these lines:

    def prepare(self, force=False):
        """
        A method to place the Root application into a prepard state.
        """
        if (self.is_prepared is not None) and (force is not True):
            raise Exception('%s application has already been prepared. Use prepare(force=True) to prepare again.'%(self._name))
        self.is_prepared = ShareDir()
        logger.info('Created shared directory: %s'%(self.is_prepared.name))
        send_to_sharedir = self.copyPreparables()
        #add the newly created shared directory into the metadata system if the app is associated with a persisted object
        self.checkPreparedHasParent(self)
        return 1

Interesting points of note here include

• self.is_prepared = ShareDir(), where the call to ShareDir() instantiates a 'shared directory' object with a randomly generated sub-directory name in the user's gangadir/shared space.
• copyPreparables() is inherited from IPrepareApp, and iterates over all attributes in an application whose meta-attribute parameter preparable = 1. If this attribute references a File() object, or a string which resolves to a file, that file is copied into the ShareDir.
• checkPreparedHasParent(). This method checks to see whether the application is associated with a persisted Ganga object, such as a job, or box. If the application isn't associated with such an object, it will not persist beyond the end of the current Ganga session, and nor will the associated ShareDir object.

The unprepare() method

This is inherited from IPrepareApp, but may be overridden here if necessary. The purpose of this method is to simply reset the is_prepared attribute to the default value of None, and decrement the reference counter (which keeps track of the number of applications using a particular ShareDir.)

The calc_hash() method

This method is not exposed to the GPI, but is used internally by Ganga to calculate a MD5 checksum of the application after it has been prepared (see bug #93682 for a detailed description). If the preparable application implements the post_prepare() method (which should most likely be the final step in the prepare() method), then calc_hash() will be called automatically.

In summary, this method generates a text string representation of the application's preparable attributes, and stores the corresponding MD5 checksum in the application's schema. Thus, to implement this functionality, developers need to include the following schema entry in their application

    'hash': SimpleItem(defvalue=None, typelist=['type(None)', 'str'], hidden=1, doc='MD5 hash of the string representation of applications preparable attributes')

and remember to call self.post_prepare() at the end of the application's prepare() method.

The result of implementing the above is that every time the prepared application is written to the Ganga repository, an on-the-fly calculation of the checksum is compared to that stored in the schema. If the two differ, an error is printed along the lines of:

Ganga.Core.GangaRepository         : ERROR    Protected attributes of Executable application, associated with Job #1305 have changed!
Ganga.Core.GangaRepository         : ERROR    If you knowingly circumvented the protection, ignore this message (and, optionally,
Ganga.Core.GangaRepository         : ERROR    re-prepare() the application). Otherwise, please file a bug report at:
Ganga.Core.GangaRepository         : ERROR    http://savannah.cern.ch/projects/ganga/

Note that this does not prevent the user from submitting/copying etc the application/job; they simply receive the above message when the object is written to the repository. Ganga determines whether the hash has changed by calling

application.calc_hash(verify=True)

when writing the prepared application to disk. If this returns False, then the above error is printed, otherwise, operation continues as normal. Note that the application's hash schema attribute is not updated during this verification step, thus, once an application's hash has changed, calc_hash(verify=True) will always return False unless the application/job is re-prepared:

j.prepare(force=True)

-- MikeKenyon - 24-Nov-2011