WARNING! This page is obsolete. As of 2015, CORAL and COOL are being built with CMake instead of CMT.

This page remains relevant for the CMT builds of the software, either for ROOT5-based stacks, or for CORAL_SERVER tests in ad hoc ATLAS HLT stacks. For more details, see PersistencyGettingStarted and PersistencyCMake.

In the following, we describe how to get started with the CORAL and COOL software, either using one of the releases pre-installed for ATLAS and LHCb on AFS, or rebuilding the software from scratch against the external libraries installed on AFS and CVMFS. CORAL and COOL are supported on Linux SLC5/SLC6 and Mac OSX using a variety of different compilers. To set up the runtime environment (and a fortiori to rebuild from scratch), the CMT configuration management tool is needed (see further below on this twiki page). The recipes described in the following are given for the tcsh shell, but equivalent recipes also exist for the bash shell family. A simple script CMT_env.csh bundled with both CORAL and COOL is used to set up CMT for the software: before sourcing it, please remember to choose a build platform and compiler by setting the CMTCONFIG environment variable. All of the following examples are described for gcc46 64bit debug builds on SLC6 (i.e. using setenv CMTCONFIG x86_64-slc6-gcc46-dbg): please logon lxplus6 or another SLC6 machine with AFS and CVMFS to follow these recipes.

Introduction to CMT and LCGCMT

CMT is the configuration management tool that is used internally by the CORAL and COOL development team to build and test the software. It is also used by the developers and end users of ATLAS and LHCb to build their own software frameworks, integrating into them CORAL, COOL and many other external dependencies.

The 'glue' integrating the CMT setups in CORAL and COOL with those used in ATLAS and LHCb is provided by a special package called LCGCMT. The LCGCMT packages defines a set of common configuration policies which apply also to the way CMT is used internally by the CORAL and COOL development team.

CMT is used by CORAL and COOL developers for three purposes primarily:

• to define (and eventually analyze) the internal and external dependencies of the Persistency components on one another and on other software packages
• to define the build rules and targets (and eventually to build the software)
• to define the runtime environment (and eventually to set up all environment variables)

Use a pre-installed release of CORAL or COOL on AFS

In the context of the CORAL and COOL release process, whenever new software versions are released for ATLAS and LHCb, both packages are built and tested with CMT by the SPI team using the nightly build infrastructure. The resulting libraries, binaries and scripts are then installed in the AFS release area on /afs/cern.ch/sw/lcg/app/releases. ATLAS and LHCb take the CORAL and COOL software already built from the AFS release area and integrate it into their build and runtime infrastructure using CMT. The process is different in CMS, where the the CORAL source code is built internally using another configuration management tool, scram (this is the reason why scram BuildFile's can be found in CORAL).

If you are a member of ATLAS or LHCb and want to set up CMT to use the CORAL and COOL software as an external user (i.e. without rebuilding the code from scratch!), you may refer to the documentation provided by your own experiment:

• CMT for ATLAS users
• CMT for LHCb users

Alternatively, you may use the following recipe to setup the CMT runtime environment directly from the CORAL or COOL releases installed on AFS (these releases also exist on CVMFS, but the CMT setup scripts preinstalled in these release areas point to AFS and are not relocatable - while all libraries and binaries are perfectly relocatable if the environment is setup externally, e.g. from the ATLAS and LHCb software releases). The following shows how to use the CORAL_2_3_28 and COOL_2_8_19 versions that were released for the latest LCGCMT_66 configuration.

Set up the CMT runtime environment for a CORAL release on AFS

To set up CORAL:

  ssh lxplus.cern.ch
  tcsh
  setenv CMTCONFIG x86_64-slc6-gcc48-dbg
  cd /afs/cern.ch/sw/lcg/app/releases/CORAL/CORAL_2_3_28/src/config/cmt
  source CMT_env.csh
  source setup.csh    

You may then execute a simple test

  test_unit_CoralBase_CoralBaseTest

Set up the CMT runtime environment for a COOL release on AFS

To set up COOL:

  ssh lxplus.cern.ch
  tcsh
  setenv CMTCONFIG x86_64-slc6-gcc48-dbg
  cd /afs/cern.ch/sw/lcg/app/releases/CORAL/COOL_2_8_19/src/config/cmt
  source CMT_env.csh
  source setup.csh    

You may then execute a simple test

  test_CoolKernel_RecordSelection

Note that setting up COOL automatically sets up the correct LD_LIBRARY_PATH for using the corresponding CORAL libraries, however it will include COOL-specific tests in the runtime PATH. If you want to execute CORAL tests, please set up CORAL instead so that those tests end up in the runtime PATH.

Rebuild CORAL and/or COOL from scratch using the externals installed on AFS or CVMFS

The following describes all steps necessary for downloading, building and running CORAL and COOL from scratch using CMT on Linux. These recipes were tested using an account that is not normally used for CMT-based developments, so that no hidden assumption should be present. We will use the CORAL_2_3-patches and COOL_2_8-patches tags in SVN to download the current release candidate as tested in the nightlies (feel free to use a different tag or the trunk if you know what you are doing). We assume that CORAL and COOL are installed in subdirectories of a private directory $MYLCG (that you may for instance configure by adding setenv MYLCG /home/valassi to your .cshrc), either on AFS or (better) on a local disk.

WARNING: the following instructions need to be fixed. The CORAL_2_3-patches and COOL_2_8-patches tags may be evolving tags, where the LCGCMT version is no longer equal to LCGCMT_66 in the cmt/project.cmt file. This is ok, as long as that version has already been installed on AFS or cvmfs. This is normally the case after a release, but it is not just before a release (the tag is changed BEFORE the release is built and installed). In that case you may need to change manually the cmt/project.cmt file to point to a version that exists.

Rebuild CORAL from scratch on AFS against LCGCMT_66

To download CORAL from SVN:

  ssh lxplus.cern.ch
  tcsh
  mkdir -p $MYLCG/CORAL/CORAL_2_3-patches
  cd $MYLCG/CORAL/CORAL_2_3-patches
  svn co svn+ssh://svn.cern.ch/reps/lcgcoral/coral/tags/CORAL_2_3-patches src

You should then set up CMT for building (you only need to do this once) by sourcing the CMT_env.csh script (after having chosen your preferred CMTCONFIG). By default this script will look for external software packages under AFS. For faster builds, it is better to to look for external software packages under CVMFS, in particular from /cvmfs/sft.cern.ch/, which you can achieve by having a file called usecvmfs in your config/cmt directory (WARNING: this presently fails because of issue SPI-328). The version of the externals depends on the LCGCMT configuration for your build (determined by file $MYLCG/CORAL/CORAL_2_3-patches/src/cmt/project.cmt=, in this case LCGCMT_66) and by the location of the LCGCMT package itself (determined by the CMTPROJECTPATH environment variable, configured by the CMT_env.csh script. The following commands set up CMT using externals from CVMFS, build the CORAL-specific setup scripts and rebuild the whole of CORAL from scratch:

  setenv CMTCONFIG x86_64-slc6-gcc48-dbg
  cd $MYLCG/CORAL/CORAL_2_3-patches/src/config/cmt
  ###\cp usecvmfsSFT usecvmfs # This presently leads to failures because of SPI-328
  source CMT_env.csh
  cmt config
  ./cmtBroadcast.sh cmt make all_groups

Having done that, you can execute the CORAL-specific setup script and try to execute a simple test:

  setenv CMTCONFIG x86_64-slc6-gcc48-dbg
  cd $MYLCG/CORAL/CORAL_2_3-patches/src/config/cmt
  source setup.csh
  test_unit_CoralBase_CoralBaseTest

Rebuild COOL from scratch on AFS against LCGCMT_66

To download COOL from SVN:

  ssh lxplus.cern.ch
  tcsh
  mkdir -p $MYLCG/COOL/COOL_2_8-patches
  cd $MYLCG/COOL/COOL_2_8-patches
  svn co svn+ssh://svn.cern.ch/reps/lcgcool/cool/tags/COOL_2_8-patches src

As explained above for CORAL, the following commands set up CMT using externals from CVMFS, build the COOL-specific setup scripts and rebuild the whole of COOL from scratch:

  setenv CMTCONFIG x86_64-slc6-gcc48-dbg
  cd $MYLCG/COOL/COOL_2_8-patches/src/config/cmt
  ###\cp usecvmfsSFT usecvmfs # This presently leads to failures because of SPI-328
  source CMT_env.csh
  cmt config
  ./cmtBroadcast.sh cmt make all_groups

Having done that, you can execute the COOL-specific setup script and try to execute a simple test:

  setenv CMTCONFIG x86_64-slc6-gcc48-dbg
  cd $MYLCG/COOL/COOL_2_8-patches/src/config/cmt
  source setup.csh
  test_CoolKernel_RecordSelection

Note that in this example COOL_2_8-patches is being rebuilt from scratch against all externals from LCGCMT_66, including CORAL! In other words, the official CORAL_2_3_28 release is used instead of the private build of CORAL_2_3-patches described above. To use a different version of CORAL or other externals (such as ROOT), the CMT configuration files must be tweaked, for instance using the approach described in the next section for building COOL against the nightlies.

Rebuild COOL from scratch using the externals from the nightly builds

The following describes all steps necessary for downloading, building and running COOL from scratch, using externals from the ROOT6 slot of the nightly builds. One motivation for documenting this recipe is to allow developers from the ROOT team to build and test PyCool against the beta version of ROOT6 (see task #47955). While this can be done in a variety of way, the following recipe is based on a few specific choices:

• Externals (here CORAL, LCGCMT and ROOT6) are copied from AFS to a private directory (because the nightly builds are erased every seven days and this approach guarantees longer availability).
• A new platform x86_64-slc6-gcc48-dbg-root6 is introduced and used (to allow parallel builds for the default x86_64-slc6-gcc48-dbg in the same directory). Note that gcc48 is used here because ROOT6 is only built for this compiler in the nightlies.
• All CMT tweaks are configured using the CMTUSERCONTEXT feature of CMT.

First, download COOL from SVN. We use the COOL-preview tag here instead of the COOL_2_8-patches tag to reproduce the results of the dev2 nightly slot testing the beta version of ROOT6.

  ssh lxplus.cern.ch
  tcsh
  mkdir -p $MYLCG/COOL/COOL-preview
  cd $MYLCG/COOL/COOL-preview
  svn co svn+ssh://svn.cern.ch/reps/lcgcool/cool/tags/COOL-preview src

Then, copy the latest CORAL, LCGCMT and ROOT6 nightlies to a private directory (here /opt/nightlies/dev2/20131029Tue is assumed on a private SLC6 node). If you do not want to change your cmt/project.cmt file (for instance because you are using the same installation also against the official externals for that LCGCMT release), you must also create a new LCGCMT_66 directory. Alternatively, you can also work with the nightlies on AFS directly, in which case please change all paths accordingly in your CMT configuration files below (and you may need to change the cmt/project.cmt file too).

  \rm -rf /opt/nightlies/dev2/20131029Tue
  mkdir -p /opt/nightlies/dev2/20131029Tue/LCGCMT
  rsync -rlptDPu --exclude=InstallArea /afs/cern.ch/sw/lcg/app/nightlies/dev2/Tue/LCGCMT/LCGCMT_root6/ \
    /opt/nightlies/dev2/20131029Tue/LCGCMT/LCGCMT_root6
  mkdir -p /opt/nightlies/dev2/20131029Tue/LCGCMT/LCGCMT_66
  pushd /opt/nightlies/dev2/20131029Tue/LCGCMT/LCGCMT_66; ln -sf ../LCGCMT_root6/* .; mkdir cmt.new
  cat cmt/project.cmt | sed "s/LCGCMT_root6/LCGCMT_66/" > cmt.new/project.cmt
  \rm -f cmt; \mv cmt.new cmt; popd
  mkdir -p /opt/nightlies/dev2/20131029Tue/CORAL/CORAL-preview
  rsync -rlptDPu /afs/cern.ch/sw/lcg/app/nightlies/dev2/Tue/CORAL/CORAL-preview/include/ \
    /opt/nightlies/dev2/20131029Tue/CORAL/CORAL-preview/include
  rsync -rlptDPu /afs/cern.ch/sw/lcg/app/nightlies/dev2/Tue/CORAL/CORAL-preview/src/ \
    /opt/nightlies/dev2/20131029Tue/CORAL/CORAL-preview/src
  rsync -rlptDPu /afs/cern.ch/sw/lcg/app/nightlies/dev2/Tue/CORAL/CORAL-preview/x86_64-slc6-gcc48-dbg/ \
    /opt/nightlies/dev2/20131029Tue/CORAL/CORAL-preview/x86_64-slc6-gcc48-dbg
  mkdir -p /opt/nightlies/dev2/20131029Tue/ROOT/ROOT_today
  rsync -rlptDPu /afs/cern.ch/sw/lcg/app/nightlies/dev2/Tue/ROOT/ROOT_today/x86_64-slc6-gcc48-dbg/ \
    /opt/nightlies/dev2/20131029Tue/ROOT/ROOT_today/x86_64-slc6-gcc48-dbg

You should then tweak the CMT configuration so that COOL uses the CORAL, LCGCMT and ROOT6 copied from the nightlies. The COOL CMT_env.csh script assumes that your private configuration files are in a subdirectory config/cmt/USERCONTEXT/$USER (defined by environment variable $CMTUSERCONTEXT) whose name depends on your username. You should create this directory and the three files requirements, CMT_userenv.csh, CMT_userenv.sh as follows.

  bash
  mkdir -p $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER
  \rm -f $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/CMT_userenv.csh
  cat << EOF > $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/CMT_userenv.csh
if ( "\$CMTCONFIG" == "x86_64-slc6-gcc48-dbg-root6" ) then
  setenv CMTPROJECTPATH /opt/nightlies/dev2/20131029Tue
endif
EOF
  \rm -f $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/CMT_userenv.sh
  cat << EOF > $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/CMT_userenv.sh
if [ "\$CMTCONFIG" == "x86_64-slc6-gcc48-dbg-root6" ]; then
  export CMTPROJECTPATH=/opt/nightlies/dev2/20131029Tue
fi
EOF
  \rm -f $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/requirements
  cat << EOF > $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/requirements
tag x86_64-slc6-gcc48-dbg-root6 target-x86_64 target-slc6 target-gcc48 target-dbg ROOT_GE_6_00
macro LCG_external \$(SITEROOT)/sw/lcg/external target-gcc48 \$(SITEROOT)/sw/lcg/experimental
macro CORAL_config_version \$(CORAL_config_version) x86_64-slc6-gcc48-dbg-root6 CORAL-preview
macro CORAL_base \$(CORAL_base) x86_64-slc6-gcc48-dbg-root6 /opt/nightlies/dev2/20131029Tue/CORAL/\$(CORAL_config_version)
macro ROOT_config_version \$(ROOT_config_version) x86_64-slc6-gcc48-dbg-root6 ROOT_today
macro ROOT_base \$(ROOT_base) x86_64-slc6-gcc48-dbg-root6 /opt/nightlies/dev2/20131029Tue/ROOT/\$(ROOT_config_version)
EOF
  exit
  cat $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/CMT_userenv.csh
  cat $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/CMT_userenv.sh
  cat $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/requirements

You can then set up CMT, build the COOL-specific setup scripts, rebuild the whole of COOL from scratch and source the COOL-specific setup script:

  \rm -rf $MYLCG/COOL/COOL-preview/x86_64-slc6-gcc48-dbg-root6
  \rm -rf $MYLCG/COOL/COOL-preview/src/*/x86_64-slc6-gcc48-dbg-root6
  \rm -rf $MYLCG/COOL/COOL-preview/src/*/*/x86_64-slc6-gcc48-dbg-root6
  setenv CMTCONFIG x86_64-slc6-gcc48-dbg-root6
  cd $MYLCG/COOL/COOL-preview/src/config/cmt
  source CMT_env.csh
  cmt config
  ./cmtBroadcast.sh cmt make all_groups
  source setup.csh

Having done that, you are now ready to try and execute some simple tests, such as:

  test_CoolKernel_RecordSelection

or

  python -c 'import PyCool'

or

  python $MYLCG/COOL/COOL-preview/src/PyCool/tests/test_coralAttributeList.py    

or

  python $MYLCG/COOL/COOL-preview/src/PyCool/tests/test_coralBlob.py

or

  setenv COOLTESTDB "sqlite://;schema=/tmp/sqliteTest-${USER}-COOLTEST.db;dbname=COOLTEST"
  python $MYLCG/COOL/COOL-preview/src/PyCool/tests/test_IFolder.py

If you just need to rebuild PyCool (e.g. because you made changes to ROOT), it is enough to execute the following:

  cd $MYLCG/COOL/COOL-preview/src/PyCool/cmt
  cmt make all_groups

In the following, we describe how to get started with the CORAL and COOL software, either using one of the releases pre-installed for ATLAS and LHCb on AFS, or rebuilding the software from scratch against the external libraries installed on AFS and CVMFS. CORAL and COOL are supported on Linux SLC5/SLC6 and Mac OSX using a variety of different compilers. To set up the runtime environment (and a fortiori to rebuild from scratch), the CMT configuration management tool is needed (see further below on this twiki page). The recipes described in the following are given for the tcsh shell, but equivalent recipes also exist for the bash shell family. A simple script CMT_env.csh bundled with both CORAL and COOL is used to set up CMT for the software: before sourcing it, please remember to choose a build platform and compiler by setting the CMTCONFIG environment variable. All of the following examples are described for gcc46 64bit debug builds on SLC6 (i.e. using setenv CMTCONFIG x86_64-slc6-gcc46-dbg): please logon lxplus6 or another SLC6 machine with AFS and CVMFS to follow these recipes.

Introduction to CMT and LCGCMT

CMT is the configuration management tool that is used internally by the CORAL and COOL development team to build and test the software. It is also used by the developers and end users of ATLAS and LHCb to build their own software frameworks, integrating into them CORAL, COOL and many other external dependencies.

The 'glue' integrating the CMT setups in CORAL and COOL with those used in ATLAS and LHCb is provided by a special package called LCGCMT. The LCGCMT packages defines a set of common configuration policies which apply also to the way CMT is used internally by the CORAL and COOL development team.

CMT is used by CORAL and COOL developers for three purposes primarily:

• to define (and eventually analyze) the internal and external dependencies of the Persistency components on one another and on other software packages
• to define the build rules and targets (and eventually to build the software)
• to define the runtime environment (and eventually to set up all environment variables)

Use a pre-installed release of CORAL or COOL on AFS

In the context of the CORAL and COOL release process, whenever new software versions are released for ATLAS and LHCb, both packages are built and tested with CMT by the SPI team using the nightly build infrastructure. The resulting libraries, binaries and scripts are then installed in the AFS release area on /afs/cern.ch/sw/lcg/app/releases. ATLAS and LHCb take the CORAL and COOL software already built from the AFS release area and integrate it into their build and runtime infrastructure using CMT. The process is different in CMS, where the the CORAL source code is built internally using another configuration management tool, scram (this is the reason why scram BuildFile's can be found in CORAL).

If you are a member of ATLAS or LHCb and want to set up CMT to use the CORAL and COOL software as an external user (i.e. without rebuilding the code from scratch!), you may refer to the documentation provided by your own experiment:

• CMT for ATLAS users
• CMT for LHCb users

Alternatively, you may use the following recipe to setup the CMT runtime environment directly from the CORAL or COOL releases installed on AFS (these releases also exist on CVMFS, but the CMT setup scripts preinstalled in these release areas point to AFS and are not relocatable - while all libraries and binaries are perfectly relocatable if the environment is setup externally, e.g. from the ATLAS and LHCb software releases). The following shows how to use the CORAL_2_3_28 and COOL_2_8_19 versions that were released for the latest LCGCMT_66 configuration.

Set up the CMT runtime environment for a CORAL release on AFS

To set up CORAL:

  ssh lxplus.cern.ch
  tcsh
  setenv CMTCONFIG x86_64-slc6-gcc48-dbg
  cd /afs/cern.ch/sw/lcg/app/releases/CORAL/CORAL_2_3_28/src/config/cmt
  source CMT_env.csh
  source setup.csh    

You may then execute a simple test

  test_unit_CoralBase_CoralBaseTest

Set up the CMT runtime environment for a COOL release on AFS

To set up COOL:

  ssh lxplus.cern.ch
  tcsh
  setenv CMTCONFIG x86_64-slc6-gcc48-dbg
  cd /afs/cern.ch/sw/lcg/app/releases/CORAL/COOL_2_8_19/src/config/cmt
  source CMT_env.csh
  source setup.csh    

You may then execute a simple test

  test_CoolKernel_RecordSelection

Note that setting up COOL automatically sets up the correct LD_LIBRARY_PATH for using the corresponding CORAL libraries, however it will include COOL-specific tests in the runtime PATH. If you want to execute CORAL tests, please set up CORAL instead so that those tests end up in the runtime PATH.

Rebuild CORAL and/or COOL from scratch using the externals installed on AFS or CVMFS

The following describes all steps necessary for downloading, building and running CORAL and COOL from scratch using CMT on Linux. These recipes were tested using an account that is not normally used for CMT-based developments, so that no hidden assumption should be present. We will use the CORAL_2_3-patches and COOL_2_8-patches tags in SVN to download the current release candidate as tested in the nightlies (feel free to use a different tag or the trunk if you know what you are doing). We assume that CORAL and COOL are installed in subdirectories of a private directory $MYLCG (that you may for instance configure by adding setenv MYLCG /home/valassi to your .cshrc), either on AFS or (better) on a local disk.

WARNING: the following instructions need to be fixed. The CORAL_2_3-patches and COOL_2_8-patches tags may be evolving tags, where the LCGCMT version is no longer equal to LCGCMT_66 in the cmt/project.cmt file. This is ok, as long as that version has already been installed on AFS or cvmfs. This is normally the case after a release, but it is not just before a release (the tag is changed BEFORE the release is built and installed). In that case you may need to change manually the cmt/project.cmt file to point to a version that exists.

Rebuild CORAL from scratch on AFS against LCGCMT_66

To download CORAL from SVN:

  ssh lxplus.cern.ch
  tcsh
  mkdir -p $MYLCG/CORAL/CORAL_2_3-patches
  cd $MYLCG/CORAL/CORAL_2_3-patches
  svn co svn+ssh://svn.cern.ch/reps/lcgcoral/coral/tags/CORAL_2_3-patches src

You should then set up CMT for building (you only need to do this once) by sourcing the CMT_env.csh script (after having chosen your preferred CMTCONFIG). By default this script will look for external software packages under AFS. For faster builds, it is better to to look for external software packages under CVMFS, in particular from /cvmfs/sft.cern.ch/, which you can achieve by having a file called usecvmfs in your config/cmt directory (WARNING: this presently fails because of issue SPI-328). The version of the externals depends on the LCGCMT configuration for your build (determined by file $MYLCG/CORAL/CORAL_2_3-patches/src/cmt/project.cmt=, in this case LCGCMT_66) and by the location of the LCGCMT package itself (determined by the CMTPROJECTPATH environment variable, configured by the CMT_env.csh script. The following commands set up CMT using externals from CVMFS, build the CORAL-specific setup scripts and rebuild the whole of CORAL from scratch:

  setenv CMTCONFIG x86_64-slc6-gcc48-dbg
  cd $MYLCG/CORAL/CORAL_2_3-patches/src/config/cmt
  ###\cp usecvmfsSFT usecvmfs # This presently leads to failures because of SPI-328
  source CMT_env.csh
  cmt config
  ./cmtBroadcast.sh cmt make all_groups

Having done that, you can execute the CORAL-specific setup script and try to execute a simple test:

  setenv CMTCONFIG x86_64-slc6-gcc48-dbg
  cd $MYLCG/CORAL/CORAL_2_3-patches/src/config/cmt
  source setup.csh
  test_unit_CoralBase_CoralBaseTest

Rebuild COOL from scratch on AFS against LCGCMT_66

To download COOL from SVN:

  ssh lxplus.cern.ch
  tcsh
  mkdir -p $MYLCG/COOL/COOL_2_8-patches
  cd $MYLCG/COOL/COOL_2_8-patches
  svn co svn+ssh://svn.cern.ch/reps/lcgcool/cool/tags/COOL_2_8-patches src

As explained above for CORAL, the following commands set up CMT using externals from CVMFS, build the COOL-specific setup scripts and rebuild the whole of COOL from scratch:

  setenv CMTCONFIG x86_64-slc6-gcc48-dbg
  cd $MYLCG/COOL/COOL_2_8-patches/src/config/cmt
  ###\cp usecvmfsSFT usecvmfs # This presently leads to failures because of SPI-328
  source CMT_env.csh
  cmt config
  ./cmtBroadcast.sh cmt make all_groups

Having done that, you can execute the COOL-specific setup script and try to execute a simple test:

  setenv CMTCONFIG x86_64-slc6-gcc48-dbg
  cd $MYLCG/COOL/COOL_2_8-patches/src/config/cmt
  source setup.csh
  test_CoolKernel_RecordSelection

Note that in this example COOL_2_8-patches is being rebuilt from scratch against all externals from LCGCMT_66, including CORAL! In other words, the official CORAL_2_3_28 release is used instead of the private build of CORAL_2_3-patches described above. To use a different version of CORAL or other externals (such as ROOT), the CMT configuration files must be tweaked, for instance using the approach described in the next section for building COOL against the nightlies.

Rebuild COOL from scratch using the externals from the nightly builds

The following describes all steps necessary for downloading, building and running COOL from scratch, using externals from the ROOT6 slot of the nightly builds. One motivation for documenting this recipe is to allow developers from the ROOT team to build and test PyCool against the beta version of ROOT6 (see task #47955). While this can be done in a variety of way, the following recipe is based on a few specific choices:

• Externals (here CORAL, LCGCMT and ROOT6) are copied from AFS to a private directory (because the nightly builds are erased every seven days and this approach guarantees longer availability).
• A new platform x86_64-slc6-gcc48-dbg-root6 is introduced and used (to allow parallel builds for the default x86_64-slc6-gcc48-dbg in the same directory). Note that gcc48 is used here because ROOT6 is only built for this compiler in the nightlies.
• All CMT tweaks are configured using the CMTUSERCONTEXT feature of CMT.

First, download COOL from SVN. We use the COOL-preview tag here instead of the COOL_2_8-patches tag to reproduce the results of the dev2 nightly slot testing the beta version of ROOT6.

  ssh lxplus.cern.ch
  tcsh
  mkdir -p $MYLCG/COOL/COOL-preview
  cd $MYLCG/COOL/COOL-preview
  svn co svn+ssh://svn.cern.ch/reps/lcgcool/cool/tags/COOL-preview src

Then, copy the latest CORAL, LCGCMT and ROOT6 nightlies to a private directory (here /opt/nightlies/dev2/20131029Tue is assumed on a private SLC6 node). If you do not want to change your cmt/project.cmt file (for instance because you are using the same installation also against the official externals for that LCGCMT release), you must also create a new LCGCMT_66 directory. Alternatively, you can also work with the nightlies on AFS directly, in which case please change all paths accordingly in your CMT configuration files below (and you may need to change the cmt/project.cmt file too).

  \rm -rf /opt/nightlies/dev2/20131029Tue
  mkdir -p /opt/nightlies/dev2/20131029Tue/LCGCMT
  rsync -rlptDPu --exclude=InstallArea /afs/cern.ch/sw/lcg/app/nightlies/dev2/Tue/LCGCMT/LCGCMT_root6/ \
    /opt/nightlies/dev2/20131029Tue/LCGCMT/LCGCMT_root6
  mkdir -p /opt/nightlies/dev2/20131029Tue/LCGCMT/LCGCMT_66
  pushd /opt/nightlies/dev2/20131029Tue/LCGCMT/LCGCMT_66; ln -sf ../LCGCMT_root6/* .; mkdir cmt.new
  cat cmt/project.cmt | sed "s/LCGCMT_root6/LCGCMT_66/" > cmt.new/project.cmt
  \rm -f cmt; \mv cmt.new cmt; popd
  mkdir -p /opt/nightlies/dev2/20131029Tue/CORAL/CORAL-preview
  rsync -rlptDPu /afs/cern.ch/sw/lcg/app/nightlies/dev2/Tue/CORAL/CORAL-preview/include/ \
    /opt/nightlies/dev2/20131029Tue/CORAL/CORAL-preview/include
  rsync -rlptDPu /afs/cern.ch/sw/lcg/app/nightlies/dev2/Tue/CORAL/CORAL-preview/src/ \
    /opt/nightlies/dev2/20131029Tue/CORAL/CORAL-preview/src
  rsync -rlptDPu /afs/cern.ch/sw/lcg/app/nightlies/dev2/Tue/CORAL/CORAL-preview/x86_64-slc6-gcc48-dbg/ \
    /opt/nightlies/dev2/20131029Tue/CORAL/CORAL-preview/x86_64-slc6-gcc48-dbg
  mkdir -p /opt/nightlies/dev2/20131029Tue/ROOT/ROOT_today
  rsync -rlptDPu /afs/cern.ch/sw/lcg/app/nightlies/dev2/Tue/ROOT/ROOT_today/x86_64-slc6-gcc48-dbg/ \
    /opt/nightlies/dev2/20131029Tue/ROOT/ROOT_today/x86_64-slc6-gcc48-dbg

You should then tweak the CMT configuration so that COOL uses the CORAL, LCGCMT and ROOT6 copied from the nightlies. The COOL CMT_env.csh script assumes that your private configuration files are in a subdirectory config/cmt/USERCONTEXT/$USER (defined by environment variable $CMTUSERCONTEXT) whose name depends on your username. You should create this directory and the three files requirements, CMT_userenv.csh, CMT_userenv.sh as follows.

  bash
  mkdir -p $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER
  \rm -f $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/CMT_userenv.csh
  cat << EOF > $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/CMT_userenv.csh
if ( "\$CMTCONFIG" == "x86_64-slc6-gcc48-dbg-root6" ) then
  setenv CMTPROJECTPATH /opt/nightlies/dev2/20131029Tue
endif
EOF
  \rm -f $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/CMT_userenv.sh
  cat << EOF > $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/CMT_userenv.sh
if [ "\$CMTCONFIG" == "x86_64-slc6-gcc48-dbg-root6" ]; then
  export CMTPROJECTPATH=/opt/nightlies/dev2/20131029Tue
fi
EOF
  \rm -f $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/requirements
  cat << EOF > $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/requirements
tag x86_64-slc6-gcc48-dbg-root6 target-x86_64 target-slc6 target-gcc48 target-dbg ROOT_GE_6_00
macro LCG_external \$(SITEROOT)/sw/lcg/external target-gcc48 \$(SITEROOT)/sw/lcg/experimental
macro CORAL_config_version \$(CORAL_config_version) x86_64-slc6-gcc48-dbg-root6 CORAL-preview
macro CORAL_base \$(CORAL_base) x86_64-slc6-gcc48-dbg-root6 /opt/nightlies/dev2/20131029Tue/CORAL/\$(CORAL_config_version)
macro ROOT_config_version \$(ROOT_config_version) x86_64-slc6-gcc48-dbg-root6 ROOT_today
macro ROOT_base \$(ROOT_base) x86_64-slc6-gcc48-dbg-root6 /opt/nightlies/dev2/20131029Tue/ROOT/\$(ROOT_config_version)
EOF
  exit
  cat $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/CMT_userenv.csh
  cat $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/CMT_userenv.sh
  cat $MYLCG/COOL/COOL-preview/src/config/cmt/USERCONTEXT/$USER/requirements

You can then set up CMT, build the COOL-specific setup scripts, rebuild the whole of COOL from scratch and source the COOL-specific setup script:

  \rm -rf $MYLCG/COOL/COOL-preview/x86_64-slc6-gcc48-dbg-root6
  \rm -rf $MYLCG/COOL/COOL-preview/src/*/x86_64-slc6-gcc48-dbg-root6
  \rm -rf $MYLCG/COOL/COOL-preview/src/*/*/x86_64-slc6-gcc48-dbg-root6
  setenv CMTCONFIG x86_64-slc6-gcc48-dbg-root6
  cd $MYLCG/COOL/COOL-preview/src/config/cmt
  source CMT_env.csh
  cmt config
  ./cmtBroadcast.sh cmt make all_groups
  source setup.csh

Having done that, you are now ready to try and execute some simple tests, such as:

  test_CoolKernel_RecordSelection

or

  python -c 'import PyCool'

or

  python $MYLCG/COOL/COOL-preview/src/PyCool/tests/test_coralAttributeList.py    

or

  python $MYLCG/COOL/COOL-preview/src/PyCool/tests/test_coralBlob.py

or

  setenv COOLTESTDB "sqlite://;schema=/tmp/sqliteTest-${USER}-COOLTEST.db;dbname=COOLTEST"
  python $MYLCG/COOL/COOL-preview/src/PyCool/tests/test_IFolder.py

If you just need to rebuild PyCool (e.g. because you made changes to ROOT), it is enough to execute the following:

  cd $MYLCG/COOL/COOL-preview/src/PyCool/cmt
  cmt make all_groups

-- AndreaValassi - 2013-10-03