Overview

As I'm working my way through building simple build and test examples, I'm trying to document my work and capture my lessons learned, such that others don't have to fight with the same issues.

Eventually, I'd like to see this example to grow into something we can reuse this for training, on the NMI aspects of the ETICS service.

NMI have a "getting started" web page: http://www.cs.wisc.edu/condor/nmi/builds/usersguide/

Here's the NMI progress page, where you can follow the status of your NMI submissions: http://grandcentral.cs.wisc.edu/build/Run-main.php. You can find my examples, by selecting "gLite" in the project search field.

Documentation Complement

Here's a list of things I didn't find clear reading the NMI getting started page, and are worth explaining a little before you start working with NMI:

• Something I didn't find clear reading the NMI getting started page was the relationship between "remote_declare" and "remote_task". As far as I understand it now, "remote_declare" allows you to create "dynamic" tasks. To do that, in remote_declare, you have to create a file called "tasklist.nmi". For each line in that file, the NMI framework will call the "remote_task" (so one needs to be define), with the environment variable "_NMI_TASKNAME" set to the corresponding line in the "tasklist.nmi" file. In the example of the gLite build, I'm using it to have "remote_task" for each gLite moduled that needs to be built.

• The "nmi_submit" command, although reminds me of edg_job_submit and glite_job_submit, doesn't have the same concept of input sandbox. At first, I was assuming that the different scripts refered to in the glue submit file would be automatically copied on to the remote machine. This is not the case! You have to use the "inputs" parameter in the submit file to do that. In my examples I've used the "scp" method to copy the scripts from the "submit machine". But the "cvs" method would be more appropriate in production.

Things to think about...

1. How do we get from building a module/component/service to perform functional testing on it? Unit testing is probably part of (or close to) the build, but functional testing requires a minimum of nstalation, configuration and deployment of services. Or is this out-of-scope?
2. Once the targeted middleware has been deployed, how do we bridge the information system of NMI (Condor) and the deployement model? In other words, how do we get NMI to select the right machine to run the different tests on, based on which machines different services, components and tools are installed? Further, this "bridge" will have to also work with different target middleware - e.g. GT4, OMII, VDT, UniCore, etc.

Example: simple gLite build

This example builds gLite, taking as input a gLite module xml file and as an option the CVS tag (the default is "HEAD").

...

Example: simple gLite functional tests

This example shows how to execute functional tests, using the TestManager, taking as input the location of the tests to be executed. The missing element for this test to properly work is solving the issue of specifying the "prereqs" or another mechanism for NMI to select the right machine to execute the test on (e.g. UI, WMS or CE).

...

Questions

- Were can I find a list of error codes (as used in the web report)... Condor?

Bugs (AFAIK)

1. In an submit file:
   • if you put a space at the end of the "method = scp ", the nmi_submit command doesn't recognise the method and reports an error
   • the scp_file attribute requires an absolute file path, which is not very portable, and not documented
2. The local submit command "nmi_run_local" doesn't accept the "inputs" parameter in the submit file. It only accpets the sources (deprecated) parameter
3. The macro (_NMI_FOO -> $(FOO)) doesn't work inside the files used with the sources/inputs parameter of the submit file
4. Documentation:
   • The link at the bottom of the usersguide page contains a broken link about "cvs server".

-- MebSter - 26 Sep 2005