Submission mechanism in Ganga

The job submission, j.submit(), is considered in two main cases:

1. simple jobs (no splitting)
2. master/subjobs (splitting enabled i.e. j.splitter defined)

In general job submission involves 3 main steps, because we want to keep the application handler independent from the backend handlers. The runtime handler acts as a bridge between the two and provides appropriate translation of the configuration objects.

1. configuration of application: app config object is produced by j.application object, this step does not depend on the submission backend
2. preparation in the runtime handler: app config in tranlated into the job config object by the runtime handler, this step depends both on the application type and the submission backend
3. submission in the backend handler: job config is used to submit the job, this step does not depend on the application

For general-purpose submission backends (such as LSF, LCG, etc) the job config objects are very similar: they define the executable, sandbox etc. For specialized submission backends (such as DIRAC) the interface may be very different.

The app config and job config objects may be arbitrarily defined. However the following pair of config objects is always considered: (master_config, sub_config). The distinction is important for splitting, however the same mechanism is also used for the simple job to avoid having two different frameworks. The master_config refers to the shared aspect of the job configuration (so to the master job) while sub_config is a list of configuration objects which refer to each individual subjob. In case of simple job both master_config and sub_config refer to the same job but the config objects are different!

The detailed process is described below. For more information on the interface look at the description of the interfaces: CVS:ganga/python/Ganga/GPIDev/Adapters

Simple jobs

1. configure the application in two steps (following the same pattern as used for splitting): the result of this operation are the app config objects
   1. app_master_config = j.application.master_configure()
   2. app_sub_config_ = j.application.configure(app_master_config)
2. create a runtime handler object instance rth: select the runtime handler by j.application and j.backend name in the registered handlers dictionary
3. prepare the job with runtime handler using the job and app config objects from the previous step
   1. job_master_config = rth.master_prepare(j.application,app_master_config)
   2. job_sub_config = rth.prepare(j.application,j, app_master_config, job_master_config)
4. submit the job using the job config objects
   1. j.backend.master_submit([j], job_sub_config, job_master_config)

Split jobs

1. split job: j.subjobs = j.splitter.split()
2. configure the application in two steps: the result of this operation are the app config objects
   1. app_master_config = j.application.master_configure()
   2. app_sub_config_ = [s.application.configure(app_master_config) for s in j.subjobs]
3. create a runtime handler object instance rth: select the runtime handler by j.application and j.backend name in the registered handlers dictionary
4. prepare the job with runtime handler using the job and app config objects from the previous step
   1. job_master_config = rth.master_prepare(j.application,app_master_config)
   2. job_sub_config = [rth.prepare(j.application,s, app_master_config, job_master_config) for s in j.subjobs]
5. submit the job using the job config objects
   1. j.backend.master_submit(j.subjobs, job_sub_config, job_master_config)

Notes on the submission backend and input sandboxes

Class IBackend defines the default (emulated) bulk submission behavior in the method master_submit(). It submits subjobs in the loop using the submit() method which should be implemented in the subclass. The implementation of submit() should create the specific part of the input sandbox (from the job_config). The shared part of the sandbox is created by the internal method master_prepare() (which is used by master_submit()). This method has nothing to do with the runtime handler master_prepare (this is a name clash that we will remove)!

If you consider overriding the default bulk submission by overriding master_submit() method you must also call the self.master_prepare() in order to create the input sandbox correctly. From master_submit() you should also either call submit() in case of a simple job or implement the support in for simple jobs in master_submit() directly.

PENDING: These notes will go into the IBackend module desciption.

-- JakubMoscicki - 10 Jan 2007

• Another view on the Ganga submission model: