Input samples with different attributes

The LHCb software has different data formats for different stages in the simulation or the data collection and reconstruction. These formats are briefly described in this page.

Running on (X)DIGI

This is the data type that stores raw information right after simulation. It does not contain reconstruction information, as it has not been performed yet; Instead, you need to run it. This is described in the Enable real-time reconstruction tutorial. input_type should be set to ROOT and input_raw_format, to 0.5. In summary, these lines should be added to your script:

from Moore import options
options.input_type = "ROOT"
options.input_raw_format = 0.5

from RecoConf.reconstruction_objects import reconstruction

with reconstruction.bind(from_file=False):
    run_moore(options, make_lines, public_tools)

Running on MDF/RAW

These data types are used to store raw information right after data taking. Their use as input also requires running reconstruction. It is mostly similar to the DIGI case, but here input_type must be set to MDF:

from Moore import options
options.input_type = "MDF"
options.input_raw_format = 0.5

from RecoConf.reconstruction_objects import reconstruction

with reconstruction.bind(from_file=False):
    run_moore(options, make_lines, public_tools)

Note

The default behavior of set_input_and_conds_from_testfiledb is to set input_type to ROOT for almost every file type, except for MDF files, in which case it is set to MDF.

IO optimizations for MDF

For MDF inputs, IO may be optimized by setting options.use_iosvc = True and options.event_store = 'EvtStoreSvc'. However, this requires that the input files are already in memory or at least on the local disk. It won’t work on mounted storage such as EOS. One possibility is to use the xrdcp function, as explained in the Downloading a file from the grid Starterkit lesson.

As an example, assume you had been setting the input with options.set_input_and_conds_from_testfiledb('MiniBrunel_2018_MinBias_FTv4_MDF'). You may obtain the url’s to the input files by using an intractive python session or consulting the test file database. After downloading them with xrdcp, you may enable the optimizations:

options.set_input_and_conds_from_testfiledb('MiniBrunel_2018_MinBias_FTv4_MDF')
options.input_files=[paths_to_downloaded_files]
options.use_iosvc = True
options.event_store = 'EvtStoreSvc'

Note

If LHCb__MDF__IOSvcMM errors arise, try reverting to the default options: options.use_iosvc = False and options.event_store = '"HiveWhiteBoard"'.

Running on (L)DST

This is the data type used to store the output of reconstruction and selection algorithms. The Writing an HLT2 line tutorial uses this format as input. The input_type should be set to ROOT and the input_raw_format depends on the settings of the previous reconstruction step. The different options are documented at RawEventFormat. If the file has been processed by Brunel (mainly samples produced before 2022), it should usually be set to 4.3 (see https://gitlab.cern.ch/lhcb-datapkg/RawEventFormat/blob/master/python/RawEventFormat/__init__.py for more information). In summary, the following lines have to be added to your script if your input file is a (L)DST:

from Moore import options
options.input_type = "ROOT"
options.input_raw_format = 0.5 #must be set to 4.3 if old sample processed by Brunel

Note

Since (L)DST files already contain reconstruction information, they may be used as input with either from_file=true or from_file=false. The pros and cons are discussed in the Enable real-time reconstruction tutorial.

FT decoding version

NOTE: Text below is valid if working with older releases of Run 3 code. Per the current master (as of 04/2024) the SciFi decoding properly auto-detects the version. The detector simulation is constantly changing to reflect the latest developments. One part which is still in development is the format in which sub-detectors save their data. The detector output is part of the simulation and, thus, samples from different simulation campaigns can have different formats. One example is the decoding version of the SciFi-Tracker (FT). If you encounter an error connected to FTRawBankDecoder, try change the decoding version. This is done by adding the following lines to your options file before the run_moore or run_reconstruction call:

from RecoConf.decoders import default_ft_decoding_version
ft_decoding_version=2 #4,6
default_ft_decoding_version.global_bind(value=ft_decoding_version)

Alternatively, one can include the files ft_decoding_v2.py or ft_decoding_v6 from Moore options.

A general rule of thumb is that for MC that was produced before 2018, 2 should be the version used. For MC produced between 2018 and early 2019, version 4 is generally correct, and for MC produced since 2019, version 6 is the most appropriate. There is more information about FT raw event formats and decoding here.

If you are experiencing further error messages, please make sure that you are using the right database tags. Information how to obtain the right database tags can be found amongst others in the starterkit pages.