CMake in LHCb

Introduction

CMake is one of the most used configuration and build tools for C/C++ projects. There is plenty of resources available online to learn the basics of the CMake configuration language, like

As it’s a common practice in large projects to wrap low level CMake code into high level functions, we introduced some Gaudi and LHCb helpers to simplify the writing of the build configuration of our software projects.

It is also important to understand the difference between building instructions and packaging instructions. Although CMake gives you control on both of them, it is a good practice to refrain from dictating packaging and deployment rules so that the project is more portable. A great presentation given at CppCon 2018 explains the issue in details.

Environment

The standard LHCb User Environment includes a recent version of CMake, as the version available on CentOS 7 is too old for our software configuration (the minimum required version is 3.15).

Project Configuration

The configuration of a project is divided in multiple files called CMakeLists.txt, one in the top directory of the project and others in the project subdirectories, plus other modules that can be imported to have access to useful custom functions (CMake itself provide some commands as builtin functions written in C++ and other as importable modules written in the CMake language).

The top level CMakeLists.txt file contains the global configuration options for a project. A stripped down example of a top level configuration file is (from the project Lbcom):

 1cmake_minimum_required(VERSION 3.15)
 2
 3project(Lbcom VERSION 32.0 LANGUAGES CXX)
 4
 5include(CTest)
 6
 7list(PREPEND CMAKE_MODULE_PATH ${PROJECT_SOURCE_DIR}/cmake)
 8
 9# -- Dependencies
10set(WITH_Lbcom_PRIVATE_DEPENDENCIES TRUE)
11include(LbcomDependencies)
12
13# -- Subdirectories
14lhcb_add_subdirectories(
15    Associators/Associators
16    Calo/CaloAssociators
17    # ...
18)
19
20lhcb_finalize_configuration(NO_EXPORT)

Lets’s break it down to understand what it does.

Preamble (lines 1-5)
cmake_minimum_required(VERSION 3.15)

Make sure we use a version of CMake that supports all the features needed in our configuration, and make sure that newer versions of CMake are internally configured to be compatible with the behavior we expect (see cmake_minimum_required() documentation)

project(Lbcom VERSION 32.0 LANGUAGES CXX)

Declare project name, version, languages used and other optional details. It is mandatory that it appears as early as possible in the top level configuration file as it set some internal CMake variables (see project() documentation for details)

include(CTest)

The CTest module initializes the CMake subsystem for testing.

Dependencies (lines 7-11)

Project dependencies are collected in a separate file that is installed together with the other build artifacts to be used by downstream projects. The special variable WITH_<Project>_PRIVATE_DEPENDENCIES set to true is used to declare that we want to use both public and private dependencies of the current project (see Project Dependencies for details).

Subdirectories (lines 13-18)

At this point we have to tell CMake which subdirectories of the current directory we want to consider for the configuration of the project. Each of the listed subdirectories must contain a CMakeLists.txt file for the subdirectory configuration. The command lhcb_add_subdirectories() is a thin helper wrapping multiple calls to the standard add_subdirectory() command.

The order of the subdirectories in the list is not relevant, but it’s a good practice to keep them in alphabetical order.

Finalization (line 20)

At the end of the configuration of all the subdirectories there are a few tedious tasks that have to be performed for a regular LHCb project. The command lhcb_finalize_configuration() takes care of those details in a semiautomated way.

Project Dependencies

A typical project uses some external projects (libraries and tools) which we refer to as dependencies. There are two main category of dependencies:

public dependencies

These are the external projects that are needed to build the current project, but also to build projects that are using our project. For example Gaudi uses external projects like Boost to simplify the internal implementations, but some of Gaudi public headers require the inclusion of Boost headers so the project LHCb, to be able to use those Gaudi headers has to implicitly depend on Boost even if it does not use it explicitly.

private dependencies

These are the external projects that are used only internally to the project and not needed by downstream projects. In this category fall both external projects that are not appearing in public headers and tools that are used for special tasks like code generation and documentation.

The declaration of all dependencies are collected in one file called cmake/<Project>Dependencies.cmake which is installed with the build artifacts. This file may look like this (from the project Rec):

 1if(NOT COMMAND lhcb_find_package)
 2    # Look for LHCb find_package wrapper
 3    find_file(LHCbFindPackage_FILE LHCbFindPackage.cmake)
 4    if(LHCbFindPackage_FILE)
 5        include(${LHCbFindPackage_FILE})
 6    else()
 7        # if not found, use the standard find_package
 8        macro(lhcb_find_package)
 9            find_package(${ARGV})
10        endmacro()
11    endif()
12endif()
13
14# -- Public dependencies
15lhcb_find_package(Lbcom 33.0 REQUIRED)
16
17find_package(ROOT REQUIRED
18    MathCore
19    RIO
20    Tree
21)
22
23find_data_package(ChargedProtoANNPIDParam 1.0 REQUIRED)
24find_data_package(TMVAWeights REQUIRED)
25
26# -- Private dependencies
27if(WITH_Rec_PRIVATE_DEPENDENCIES)
28    find_package(AIDA REQUIRED)
29    find_package(Boost REQUIRED
30        container
31        filesystem
32        headers
33        iostreams
34    )
35    find_package(Eigen3 REQUIRED)
36    find_package(fmt REQUIRED)
37    find_package(GSL REQUIRED)
38    find_package(Rangev3 REQUIRED)
39    find_package(ROOT REQUIRED
40        Core
41        GenVector
42        Hist
43        Matrix
44        TMVA
45    )
46    find_package(Vc REQUIRED)
47    find_package(VDT REQUIRED)
48    find_package(XGBoost REQUIRED)
49
50    if(BUILD_TESTING)
51        find_package(Boost REQUIRED unit_test_framework)
52    endif()
53endif()
preamble (lines 1-12)

This section tries to find the module LHCbFindPackage to load the lhcb_find_package() command. If not found we add a trivial implementation wrapping the standard command find_package().

public dependencies (lines 14-24)

The declaration of dependencies has the form of a list of calls to find_package(), lhcb_find_package() (for LHCb projects) or find_data_package() (for LHCb data packages).

private dependencies (lines 26-53)

The private dependencies section is not much different from the public one expect for the fact that it is protected in a if-endif block so that it is activated only when inside the project it belongs to, thanks to the conventional variable WITH_<Project>_PRIVATE_DEPENDENCIES. 1

Subdirectory Configuration

Each subdirectory (sometimes called package) is in general different from the others, but there are conventions it is better to follow and other that are mandatory.

Layout

A typical subdirectory contains the mandatory file CMakeLists.txt (described in the next section) and some of these directories:

include

directory for public headers

src

local source files

dict

files to generate ROOT dictionaries

python

Python modules

tests

source and configuration files for tests

Configuration file

The structure of the CMakeLists.txt file of a subdirectory follows roughly this model:

 1# ... copyright ...
 2#[========================================[.rst:
 3Group/Subdirectory
 4------------------
 5Lorem ipsum dolor sit amet...
 6#]========================================]
 7
 8gaudi_add_library(Subdirectory
 9    SOURCES src/A.cpp src/B.cpp
10    LINK PUBLIC Gaudi::GaudiKernel
11         PRIVATE Boost::headers
12)
13
14gaudi_add_dictionary(SubdirectoryDict
15    HEADERFILES dict/dict.h
16    SELECTION dict/selection.xml
17    LINK Subdirectory
18)
19
20gaudi_install(PYTHON)
21
22if(BUILD_TESTING)
23    gaudi_add_tests(QMTest)
24    gaudi_add_executable(unit1
25        SOURCES tests/src/unit1.cpp
26        LINK Subdirectory
27        TEST
28    )
29endif()

At the top of the file (lines 1-6) we have the copyright statement followed by a comment block documenting the subdirectory features and roles. The title of the documentation section must be the path of the subdirectory relative to the project top directory.

The first non-comment part of the file (lines 1-20) contains instructions to build the various parts of the subdirectory and instructions to install public files like Python modules or scripts.

Following the instructions for the public part of the subdirectory we have the section for the tests. It is a good practice to enclose this part in a if(BUILD_TESTING)-endif() block to avoid wasting build time if the tests are not needed (see CTest module documentation).

Common commands

The full list of CMake commands con be found in the CMake manual and LHCb custom commands can be found in the index.

The most common commands found in a subdirectory configuration are:

gaudi_add_library()

creates a library of common classes and functions that can be reused in other libraries and modules

gaudi_add_module()

creates a plugin library, i.e. a shared object file containing Gaudi components like algorithms and tools

gaudi_add_header_only_library()

similar to gaudi_add_library(), but all the code is in header files (so there is no real build step)

gaudi_add_executable()

creates an executable command; it is most often used with the flag TEST to declare that the executable being built is a unit test and should be run with the other tests by the ctest command

gaudi_add_tests()

declare tests to be run with the ctest command; it knows how to handle QMTest, nosetest and pytest test suites

gaudi_install()

useful to install non C++ (or non compiled) files, like Python modules, CMake modules or scripts (shell, Python, etc.)

Reference Documentation for LHCb Modules

Footnotes

1

The reason why we use a protected block, instead of hiding the private dependencies in a file that is not installed, is for interoperability with satellite projects created by lb-dev.