LHCb Software Training: Basics

Software organisation, environment setup and CMT

Logging in to Dortmund cluster

To enable X11 connections when logging in  as a remote machine, use ssh with the -X option: ssh -X username@leopardenhai Alternatively, omit the -X option and include the line: ForwardX11 yes in the ssh configuration file, ${HOME}/.ssh/config.

X11 connectivity is needed to be able to open graphics windows and windows for the emacs editor.

Choice of shell

These particular tutorials are configured to use the tcsh shell, an enhanced version of csh, but LHCb setup scripts are also maintained for the Bourne family of shells: sh, bash, ksh, zsh. There are subtle and not-so-subtle differences in the functionality of the different shells, and for scripting there are major differences between (t)csh and Bourne-type shells. A useful short summary can be found in:

A user may redefine the default shell on lxplus by going to https://cra.cern.ch and changing the appropriate field. In general unless you have a strong preference stick to tcsh as this is better tested.

In what follows, <script>.[c]sh is used as shorthand for:

LHCb software environment

To has aceess to LHCb software on Uni-Dortmund cluster one need to source the following script using your userid at lxplus.cern.ch cluster:

source /software/lhcb/LHCbSoftwareSetup.csh ibelyaev 

This sets a number of environment variables, mainly identifying directories relevant for working with LHCb software, and creates some aliases. Also it defines some additional environment variables and aliases, and performs the setup for the Configuration Management Tool (CMT), used in LHCb for building software and for environment setup.

LHCb flavour of emacs editor

Customisations to emacs designed to help LHCb users have been written, and are described in the LHCb emacs user guide. Even for users who prefer a different editor (vim, nedit, Joe's own editor, pico, nano, or other), the LHCb flavour of emacs is useful for creating skeleton algorithms and similar.

Exercise 1

This is a simple exercise just to check your environment setup.
  • Check that X11 connections are enabled, for example by displaying a clock: xclock & Note time zone is European!
  • Check which shell you are using, for example using: echo $0 or echo $SHELL
    •  and optionally change your default shell by going to https://cra.cern.ch (you will need to login with your CERN NICE username and password)
  • Check that the standard LHCb environment has been set. If it has, then you will have seen a message at login time about your CMT settings, and the result of the command: env
    • should contain a number of variables with prefix LHCB (LHCBHOME, LHCBRELEASES and so on), and with prefix CMT (CMTROOT, CMTCONFIG and so on). If this isn't the case then you need to setup the environment yourself.
  • Setup the LHCb flavour of emacs and (optionally) the EDT keyboard emulation by adding the following lines to ~/.emacs: (load (expand-file-name "$EMACSDIR/lhcb")) (load (expand-file-name "$EMACSDIR/edt")) or cp $EMACSDIR/.emacs ~ LHCb specific emacs commands are documented in the LHCb emacs user guide.

Organisation of LHCb software

LHCb software is based on a framework called Gaudi. This framework provides functionality useful in a wide range of contexts, such as file access, histogramming, message printing, and run-time configuration. The run-time configuration is based on job options.

Software in LHCb is developed in the context of packages. Here, a package is essentially a set of files, organised according to some directory structure, and providing some well-defined, circumscribed functionality. Most LHCb packages contain code that can be compiled into (shared) libraries. Packages with related functionality are collected in groups (or are said to be placed under a hat), and package groups are arranged in projects. Projects and packages (but not package groups), have an associated version number, of the form vXrY, with X and Y integers, and this is incremented when changes are made in package contents. The hierarchy of the software organisation is then: PROJECT/PROJECT_version/package group/package/package version. For example ls $DaVinci_release_area/DAVINCI/DAVINCI_v17r5/Phys/DaVinci/v17r5/

Sub-directories typically found in an LHCb package include:

Package development in LHCb is carried out using the Concurrent Versions System (CVS). This defines a code repository, provides dedicated commands for moving code in and out of the repository, and keeps a record of all changes made. With the standard LHCb environment, the repository is identified by the variable CVSROOT, and a user can obtain a copy of any package in the repository using the command: getpack <package group>/<package> <version> If no arguments are specified, a list of all available packages is printed, and if the version is omitted a list of all available versions for the specified package is given. The latest version of a package, (not necessarily a working version!) is always available as the package head.

New versions of packages, and new packages, are placed in the release area after testing, or may be available in the development area as works in progress. The standard LHCb environment includes the variables:

Instructions for making package modifications and new packages available from the release area can be found in the documentation for How to use Gaudi under CMT.

Web access is available to both the repository and the release area.

LHCb applications

The LHCb applications are sets of packages used within the Gaudi framework to perform specific tasks. The main applications are:

There is a project specific to, and with the same name as, each application. Additional projects are shared across applications.

Exercise 2

This exercise is designed to give you some familiarity with the LHCb software organisation. It's fairly open-ended, in that you could spend a very long time exploring all of the LHCb packages. Until you're trying to find something - which is when knowing the organisation is a big help - it's probably not useful to spend more than 5-10 minutes looking around.
  • Move to the LHCb release area: cd ${LHCBRELEASES}
  • Note the different project directories, and enter the project directory for any one of the LHCb applications.
  • Note the different project versions available, and move to the directory for the latest.
  • Note the different package groups, and have a look in two or more of these at what packages are available.
  • Choose any two packages, and for each in turn: move to the directory for the latest version, look at which sub-directories are defined, and have a look at the files that are in each.

Configuration Management Tool (CMT)

CMT is used to specify how the environment setup for a package is to be performed, and how the package is built (for example, how to produce executables or libraries from source code). To allow this, a CMT package must have a cmt sub-directory containing a requirements file, where the relevant instructions are given. All LHCb packages developed in the Gaudi framework are CMT packages.

For a package with name myPackage, CMT itself defines an environment variable MYPACKAGEROOT that contains the package location.

In the context of environment setup, there are two path variables that tend to be very important:

The full set of commands that can be used in a requirements file is described in the CMT manual. Some of the most useful commands are shown below.

# Lines beginning with the symbol # are treated as comments. # Blank lines are ignored. #========================================================================== # The following commands describe a package. # Package name. package myPackage # Package version, should correspond to CVS tag version v1r0 # Structure, i.e. directories to process. # A package must always have: # a cmt directory (for the requirements file) # a doc directory (for the release.notes file) # It may have: # a src directory for source files (.h, .cpp) # a options directory for job options (.opts) branches cmt doc src options #============================================================================ # The following command specifies packages which the current needs for # compilation and linking. Put as many lines as necessary # The keyword is followed by package name and version # and, where applicable, the hat under which the package is placed. use DaVinciTools v13r* Phys #======================================================================== # The following tell CMT what to build and how # Build an application with the specified file application myApp ../src/myApp.cpp # Build library with the specified files library myLib ../src/*.cpp # Define link options and environment variables for loading component library apply_pattern component_library library=myLib # Define link options and environment variables for loading linker library apply_pattern linker_library library=myLib #======================================================================== # The following commands define symbols (variables and aliases). # They consist of a keyword followed by a symbol name, # and possibly by a value. # Previously defined symbols, including environment variables, can be # used in value assignments by placing brackets - () or {} - around # them and a $ in front. # Treat symbol as a Make macro definition. # This can be used to define a symbol as a local variable (i.e. one # that won't be visible from the shell). macro MYCMTDIR $(HOME)/cmtuser # Treat symbol as an environent variable. # The symbol will be visible from the shell after sourcing setup.[c]sh. set LCGDIR /afs/cern.ch/sw/lcg # Treat symbol as a path variable. # The symbol will be visible from the shell after sourcing setup.[c]sh. # When appending or prepending an item to a path variable not initialised # in the requirements file, it's a good idea to first remove the item, # to ensure it occurs only once. # Set path. path PYTHONPATH $(HOME)/python # Append item to path. path_append PYTHONPATH $(HOME)/myPythonPackage # Remove item from path. path_remove LD_LIBRARY_PATH $(HOME)/lib # Prepend item to path. path_prepend LD_LIBRARY_PATH $(HOME)/lib # Create alias. # The alias will be visible from the shell after sourcing setup.[c]sh. alias myApp $(MYPACKAGEROOT)/$(CMTCONFIG)/myApp.exe

Most CMT commands, but not all, operate on requirements files, and must be given from a package's cmt directory. If a requirements file includes lines indicating that other packages are used, then these packages will be searched for in the locations specified by the path variable CMTPATH, and the requirements files of each used package is processed in turn.

The CMT manual should be consulted for the full set of CMT commands, but a useful subset is given below.

Exercise 3

This exercise introduces basic CMT functionality demonstrates the use of the LHCb flavour of emacs to create different kinds of files.

Exercise 4

This exercise demonstrates the use of CMT for environment setup.


Environment setup for using Python
Python is one of the external packages defined for use with Gaudi. There is therefore a CMT interface package associated to it, which will automatically select the Python version compatible with your current environement. Try the following:
  • Remove the requirements file you created for Method 1, and create a new one
  • Add the following use statement: use GaudiKernel v* use Python v* LCG_Interfaces
  • Set the environment to use the default version of Python by sourcing the setup file: source setup.[c]sh
  • Start Python (from any directory) by giving the command: python The version number will be printed.

As a general rule, it is recommended to let CMT choose the version of external packages for you according to your environment, rather than setting it yourself. The list of available interface packages is in the LCG_Interfaces directory of the current configuration: ls $LCG_SETTINGSROOT/../LCG_Interfaces

Running LHCb applications

The basic steps for running an LHCb application are as follows:

Running an LHCb application then amounts to giving the three commands indicated above. Controlling what happens when the application is run means knowing how to add new code and understanding the job options.

Exercise 5

In this exercise we just execute the released DaVinci application, with default job options.
  • Set the CMT environment for DaVinci version v17r5 DaVinciEnv v17r5
  • Set up the run time environment for DaVinci v17r5 source ${DaVinci_release_area}/DAVINCI/DAVINCI_v17r5/Phys/DaVinci/v17r5/cmt/setup.[c]sh
  • Run the excutable ${DAVINCIROOT}/${CMTCONFIG}/DaVinci.exe ${DAVINCIROOT}/options/DaVinci.opts In fact, DaVinci sets up an alias for the above command, so you could also type: DaVinci ${DAVINCIROOT}/options/DaVinci.opts
Notice that you can execute these commands from any directory. The job output goes to your current directory, so you should choose a sensible current directory. Once you have chosen a version and set up the enviroment, the behaviour of the application is determined by the job options file that you supply as the argument of the DaVinci command.

Notice also that you can keep any number of different job options files in your own area, each describing a different configuration of the application (e.g. different cuts), without having to rebuild the standard application. This is recommended if your analysis can be carried out simply by modifying job options (i.e. if you do not need to write your own C++ algorithms).

Exercise 6

In this exercise we address the case where you also need to modify the application executable, typically by adding your own component library package. The steps are the same as in exercise 5, but now you build your own executable.
  • Set the CMT environment for DaVinci version v17r5 DaVinciEnv v17r5
  • Get your own version of the application cd ~/cmtuser; getpack Phys/DaVinci v17r5 ;  cd Phys/DaVinci/v17r5/cmt
  • Modify the requirements file to add your package, then build the application and setup the run time environment make source setup.[c]sh
  • Run the excutable DaVinci ../options/DaVinci.opts
As with Exercise 5, you can now run the same executable from any directory, with any number of different options files
Sometimes, it may be necessary to work with several different versions of an application. You should be very careful when you do this: if you have packages in your ~/cmtuser directory that were built against a version of DaVinci, and then you move to a different version, you will still pick up these packages but the results may be unpredictable (if you are lucky you will get a core dump...) because the packages have not been recompiled for the new version of the application.

Exercise 7

This exercise shows how to have several cmtuser directories, one per environment
  • Choose an application, and setup the environment for the associated project: <project>Env
  • Examine the value of CMTPATH: echo ${CMTPATH}
  • Create a second cmtuser working directory:
    • mkdir ~/cmtuser2
  • Set the environment variable $CMTPATH to ~/cmtuser2 setenv CMTPATH ~/cmtuser2
  • Repeat the command: <project>Env
    • and look again at the value of CMTPATH. What you have done is to tell CMT to search for packages in your ~/cmtuser2 directory and not in the ~/cmtuser directory which is the default at log in. In fact, CMT now knows nothing about the directory ~/cmtuser, and will not see any packages that you might have put there.
  • Restore the previous value of CMTPATH by setting setenv CMTPATH ~/cmtuser
    • and again giving the command: <project>Env

Last modified by Vanya BELYAEV, 28-Nov-06