Writing an HLT2 line

An HLT line is a sequence of steps that collectively define whether an event contains an object of interest which should be kept for later analysis. This object is typically a reconstructed candidate physics process, such as an exclusive particle decay.

This page will walk you through defining an HLT2 line step by step. We’ll reconstruct candidate \(\Lambda_{b}^{0} \to \Lambda_{c}^{+} \pi^{-}\) and \(\Lambda_{b}^{0} \to \Lambda_{c}^{+} \mu^{-} \bar{\nu}_{\mu}\) decays with \(\Lambda_{c}^{+} \to p K^{-} \pi^{+}\), explaining the details of how to encode this within Moore.

To follow along, it’s expected that you have a development setup built and ready to go.

File structure

Each HLT2 line is defined by a single small Python function, a ‘line definition’. Line definitions for HLT2 live in files under this directory in the Moore project:

Hlt/Hlt2Conf/python/Hlt2Conf/lines

First look there to get a sense of how things are structured. Some files are further organised into sub-folders. For us, we can just create a file directly under lines:

touch Hlt/Hlt2Conf/python/Hlt2Conf/lines/hlt2_line_tutorial.py

Open the newly-created file in your text editor of choice.

A possible final result also exists already as:

Hlt/Hlt2Conf/python/Hlt2Conf/lines/hlt2_line_example.py

It can be run with:

Moore/run gaudirun.py '$HLT2CONFROOT/options/run_hlt2_line_example.py'

Prototyping

We focus first on the \(\Lambda_{b}^{0} \to \Lambda_{c}^{+} \pi^{-}\) with \(\Lambda_{c}^{+} \to p K^{-} \pi^{+}\) and add the \(\Lambda_{b}^{0} \to \Lambda_{c}^{+} \mu^{-} \bar{\nu}_{\mu}\) decay later to highlight certain aspects of the framework.

Think about how you would reconstruct/make candidates for our decays of interest. This is always the first step before writing any code. How could we do it?

1. Reconstruct and filter/select protons, kaons, pions. They are the basic building blocks for this module.

2. Reconstruct \(\Lambda_{c}^{+} \to p K^{-} \pi^{+}\) candidates.

3. Reconstruct pion candidates needed for the \(\Lambda_{b}^{0}\) candidates.

4. Reconstruct \(\Lambda_{b}^{0} \to \Lambda_{c}^{+} \pi^{-}\) candidates.

Each (sub-)step, like ‘reconstruct and filter proton’, and ‘reconstruct \(\Lambda_{c}^{+}\)’, represents the running of an algorithm, a C++ component defined within the LHCb selection framework. We want to configure these algorithms, so they behave in a way that creates the candidates we want. We have consolidated the reconstruction and filtering sub-steps to define module-specific input particles. Those could also be common to your working-group or the entire experiment, in which case they would be imported from a shared module.

But there is one step missing here, which historically has been implicit: reconstruct primary vertex (PV) candidates! These are necessary if we want to cut on quantities related to PVs such as the impact parameter and flight distance.

So, let’s outline a function that does these steps:

def lb0_to_lcpim_line():
    pvs = make_pvs()
    protons = protons_for_charm()
    kaons = kaons_for_charm()
    pions = pions_for_charm_and_beauty()
    lcs = make_lambdacs(protons, kaons, pions, pvs)
    lbs = make_lambdabs(lcs, pions, pvs)

    return lbs

This is a step-by-step encoding of what we want our line to do. Of course, this version doesn’t run yet because we haven’t defined the various make_ and h_for_charm functions yet, and it’s not clear what will happen to the return value lbs, but this function is already very close to what our final function will look like.

Note

We ‘skipped’ step 3 in our outline because we’ll assume that for this line the pions used for the \(\Lambda_{c}^{+}\) also meet our criteria for \(\Lambda_{b}^{0}\) pions.

What would our function look like if this wasn’t the case?

There are just a few changes we need to make to our file and function to be consistent with what Moore expects.

1. Return an object that Moore understands from the function.

2. Define a ‘line registry’ object that will hold all the lines defined within this file/module. Moore will expect this object to be present, and will use it to discover all lines it should run.

3. Add our line definition function to this registry.

Line declaration

The first step means returning a Moore.lines.Hlt2Line object. This contains some metadata about the information that Moore will use, such as a name, in addition to the control flow defining the line. Let’s return that object first and then discuss it:

from Moore.lines import Hlt2Line

def lb0_to_lcpim_line(name="Hlt2Tutorial_Lb0ToLcpPim_LcpToPpKmPip", prescale=1):
    pvs = make_pvs()
    protons = protons_for_charm()
    kaons = kaons_for_charm()
    pions = pions_for_charm_and_beauty()
    lcs = make_lambdacs(protons, kaons, pions, pvs)
    lbs = make_lambdabs(lcs, pions, pvs)

    return Hlt2Line(
        name=name,
        algs=[lbs],
        prescale=prescale,
     )

There are three new things going on:

1. The Moore.lines.Hlt2Line object needs to be created with a name, so we’ve parametrised this as a function argument, with a default value, and passed it to Moore.lines.Hlt2Line.

2. The Moore.lines.Hlt2Line object can be created with a prescale, so we’ve parametrised this similarly as for the name.

3. Finally, we defined the control flow of the line as the algs parameter of Moore.lines.Hlt2Line. The control flow specifies how Moore should evaluate whether this line made a positive decision or not. The filters of a line are executed in the order defined by the control flow, and the execution of a line is aborted if a filter fails. The control flow order can be used to optimize the execution time of a line. For our line, the decision is decided solely by the presence of \(\Lambda_{b}^{0}\) candidates: if we created a non-zero number of \(\Lambda_{b}^{0}\) candidates, this line should be considered as having ‘passed’ (also called ‘fired’).

Parametrising the function in the way we have allows for two things:

1. For development purposes we can easily create multiple lines with different names, prescales just by calling the function with different arguments. Of course, we are free to add further arguments, like cut values. We’ll see later how to run multiple instances of the line with different cuts.

2. For bookkeeping purposes, we will decorate the function. The decorator expects the name argument to register the line.

Tip

How should you decide what name to give your line? The conventions are outlined among the best practices below, Moore#60 , or your WG might provide a dedicated naming scheme. If you’re still unsure, just open your merge request and someone will make suggestions.

Control and data flow

We didn’t need to specify that the other algorithms should run, like the creation of the \(\Lambda_{c}^{+}\) candidate. Why is this? It’s because Moore makes the distinction between control flow and data flow.

Control flow defines whether decisions are positive (passed) or negative (failed). The control flow may, for example, specify that several algorithms should be run and that the decision should be positive if at least one algorithm reports as passing.

The data flow defines which inputs are necessary to create a given output. In order to evaluate the control flow in our line, Moore needs to run the \(\Lambda_{b}^{0}\) making algorithm. Before doing that it will automatically deduce what other algorithms it needs to run in order to satisfy the inputs to the \(\Lambda_{b}^{0}\) algorithm. One input is the output of the \(\Lambda_{c}^{+}\) algorithm, and Moore will likewise automatically deduce what algorithms need to run to produce the required inputs (that is: the proton, kaon, and pion makers). This automatic data flow resolution goes all the way up through the reconstruction to the raw event.

We could choose to impose additional requirements on the control flow if it makes physics sense for our line. For example:

1. At least one PV must be present in the event; or

2. Intermediate selection steps.

Because we already have a PV making algorithm in our prototype, we could include this in our control flow already. Including the \(\Lambda_{c}^{+}\) selection in our case won’t have any effect, as we chose to use the exact same pion candidates for the \(\Lambda_{c}^{+}\) and \(\Lambda_{b}^{0}\) selections. For our semileptonic decay however, we could add the muons to the control flow.

from RecoConf.event_filters import require_pvs


    return Hlt2Line(
        name=name,
        algs=[require_pvs(pvs), muons, lbs],
        prescale=prescale,
     )

Moore will define the control flow for this line to be:

“First require the pvs algorithm to pass, then require the muons algorithm to pass, and finally require the lbs algorithm to pass; if all pass then the line decision is positive.”

Note

The entries in our algs list define the entries in a single control flow node. This node has a ‘logic’ of ‘lazy and’. All possible node logics are defined by PyConf.control_flow.NodeLogic.

The total HLT2 decision is defined by a control flow node containing each individual line’s node, with a logic of ‘non-lazy or’. Most line authors don’t have to worry about the different types.

Line registration

The last two steps we need to make to our prototype function are rather straightforward. The line registry object is just a Python dictionary, and this is populated by using the Python decorator syntax with the Moore.config.register_line_builder helper:

from Moore.config import register_line_builder
from Moore.lines import Hlt2Line
from RecoConf.legacy_rec_hlt1_tracking import require_pvs

all_lines = {}

@register_line_builder(all_lines)
def lb0_to_lcpim_line(name="Hlt2Tutorial_Lb0ToLcpPim_LcpToPpKmPip", prescale=1):
    pvs = make_pvs()
    protons = protons_for_charm()
    kaons = kaons_for_charm()
    pions = pions_for_charm_and_beauty()
    lcs = make_lambdacs(protons, kaons, pions, pvs)
    lbs = make_lambdabs(lcs, pions, pvs)

    return Hlt2Line(
        name=name,
        algs=[require_pvs(pvs), lbs],
        prescale=prescale,
     )

The decorator just adds the line function to the dictionary. We’ll see later how this dictionary is used to run the line.

Note

It’s now worth taking a step back to look at the function we’ve written, because this contains all the core ideas we’ll need. All other HLT2 lines you’ll see look very similar to what we have now.

Standard objects

In HLT2, several maker functions are already defined for general usage. These ‘standard makers’ take the output of the reconstruction and make objects common to many HLT2 lines. These standard makers produce objects such as:

• Charged tracks with predefined mass hypotheses and associated PID objects

• Neutral objects such as photons and neutral pions

• Composite objects, representing candidate particle decays, such as \(J/\psi \to \mu^{+} \mu^{-}\) and \(\Lambda^{0} \to p \pi^{-}\).

Before writing your own maker, you should always first browse the list of standard makers to see if something already exists that suits your needs. Having all HLT2 lines re-use the same makers reduces the number of unique algorithms that Moore has to run, reducing the total time-per-event.

For our use case, we can see there are already makers we can use for charged non-composite inputs:

• Protons: make_has_rich_long_protons

• Kaons: make_has_rich_long_kaons

• Pions: make_has_rich_long_pions

• (Muons: make_ismuon_long_muon)

We’ve chosen the has_rich variant because in this example we will apply PID cuts to all non-composite particles, so it makes sense to first require that the objects have passed the requirements needed to assign PID likelihood values.

Primary vertices are also part of the set of standard objects, produced by the make_pvs function.

Given this information, we can flesh out our function a bit more:

from Moore.config import register_line_builder
from Moore.lines import Hlt2Line
from RecoConf.legacy_rec_hlt1_tracking import require_pvs

from ..standard_particles import (
    make_has_rich_long_kaons,
    make_has_rich_long_pions,
    make_has_rich_long_protons,
)
from RecoConf.reconstruction_objects import make_pvs

all_lines = {}

@register_line_builder(all_lines)
def lb0_to_lcpim_line(name="Hlt2Tutorial_Lb0ToLcpPim_LcpToPpKmPip", prescale=1):
    pvs = make_pvs()
    protons = protons_for_charm()
    kaons = kaons_for_charm()
    pions = pions_for_charm_and_beauty()
    lcs = make_lambdacs(protons, kaons, pions, pvs)
    lbs = make_lambdabs(lcs, pions, pvs)

    return Hlt2Line(
        name=name,
        algs=[require_pvs(pvs), lbs],
        prescale=prescale,
     )

Note

The functions to create reconstruction objects like PVs, tracks or protoparticles should only be imported from the module RecoConf.reconstruction_objects. More explanation can be found in the tutorial Enable real-time reconstruction.

Filters and combiners

We’re nearly there! What’s left is to define the various h_for_charm and make_ placeholders.

Think about what these functions should do. They need to take input, as we’ve written it in our prototype, configure the correct type of algorithm, and then return something. We’ll be using selection algorithms which use ThOr functors.

Let’s start with the basic building blocks of our module h_for_charm:

from GaudiKernel.SystemOfUnits import GeV
import Functors as F

from ..algorithms_thor import ParticleFilter


def protons_for_charm():
    pvs = make_pvs()
    cut = F.require_all(
        F.PT > 0.5 * GeV,
        F.MINIPCHI2(pvs) > 9.,
        F.PID_P > 5.,
    )
    return ParticleFilter(make_has_rich_long_protons(), F.FILTER(cut))

We’ve used the require_all helper to define the cut expression here.

Note

The function has no arguments. This is on purpose for production-ready selections. In this way we have re-defined a custom basic building block for our lines.

The return value is the configured algorithm. This can be used as an ‘input’ to other algorithms as the framework knows how to extract the (single) output the Hlt2Conf.algorithms_thor.ParticleFilter algorithm produces.

Define similar functions for the remaining particle filters, for kaons and for pions.

Next is a function which combines its input to a composite \(\Lambda_{c}^{+}\) candidate:

from Functors.math import in_range
from GaudiKernel.SystemOfUnits import (GeV, MeV, mm)

from ..algorithms_thor import ParticleCombiner

def make_lambdacs_for_beauty(protons, kaons, pions, pvs):
    two_body_combination_code = F.require_all(
        F.MAXDOCACHI2CUT(9.), F.MAXDOCACUT(0.1 * mm))
    combination_code = F.require_all(
        in_range(2080 * MeV, F.MASS, 2480 * MeV),  # mass of the combination
        F.PT > 1.4 * GeV,  # pT of the 3-track combination
        F.SUM(F.PT) > 2 * GeV,
        F.MAXDOCACHI2CUT(9.),
        F.MAXDOCACUT(0.1 * mm),
    )
    vertex_code = F.require_all(
        in_range(2100 * MeV, F.MASS, 2460 * MeV),  # mass after the vertex fit
        F.PT > 1.6 * GeV,  # pT after the vertex fit
        F.CHI2DOF < 10.,
        F.BPVFDCHI2(pvs) > 25.,
    )
    return ParticleCombiner(
        [protons, kaons, pions],
        DecayDescriptor="[Lambda_c+ -> p+ K- pi+]cc",
        name="Tutorial_Lcp_Combiner",
        Combination12Cut=two_body_combination_code,
        CombinationCut=combination_code,
        CompositeCut=vertex_code,
    )

The concepts here follow on from the protons_for_charm example. However, this combiner is not written as a basic building block of our selection, so that we pass reconstructed objects as positional arguments to make the data-flow explicit in the function using this combiner.

Combiners are always instances of ParticleCombiner, and length of the input list determines whether a 2-, 3- or 4-body combiner is called on the C++ side. There is detailed documentation for combiners in our codebase. Most notably for the configuration are:

1. The order of particles in the decay descriptor and the input list must be the same; there is no mix and match unlike Run2!

2. Another change w.r.t. Run2 is that particles of the same type are passed explicitly ([pi, pi, pi], DecayDescriptor="[D+ -> pi+ pi+ pi-]cc",)

3. Multiple child particles with the same ID must be grouped together (D+ -> pi+ pi+ pi- is good, D+ -> pi+ pi- pi+ is forbidden).

4. For performance purposes, the algorithm logic assumes that the rarest children are listed first in the decay descriptor. In case you are unsure what is rarest, checking counters in the log file can help.

Note

When adding the line for \(\Lambda_{b}^{0} \to \Lambda_{c}^{+} \mu^{-} \bar{\nu}_{\mu}\), we may choose to use the same make_lambdacs_for_beauty function to build \(\Lambda_{c}^{+}\) candidates as in the hadronic decay. If this function is also called with the exact same inputs (protons_for_charm, kaons_for_charm, pions_for_charm_and_beauty) the combiner will run only once; Because the configuration framework resolves two identically-configured algorithms to the same underlying object. This is an important point for optimization and will be discussed at several stages of this tutorial.

On the other hand this means, that if you change a cut slightly in one of the combiners or its inputs, another instance of the algorithm is created and work is (almost) doubled.

Finally, can define a maker function for the \(\Lambda_{b}^{0}\) candidates based on what we have learned so far.

Running

We now have a first prototype for a line selecting \(\Lambda_{b}^{0} \to \Lambda_{c}^{+} \pi^{-}\) decays. The remaining piece is an options file that configures Moore with our line maker function. You can place this in a file called test_line.py in your working directory.

test_line.py

from Moore import options, run_moore
from Hlt2Conf.lines.hlt2_line_tutorial import lb0_to_lcpim_line
from RecoConf.global_tools import stateProvider_with_simplified_geom


def all_lines():
    return [lb0_to_lcpim_line()]

public_tools = [stateProvider_with_simplified_geom()]

options.set_input_and_conds_from_testfiledb('Upgrade_MinBias_LDST')
options.input_raw_format = 4.3
options.evt_max = 100

run_moore(options, all_lines, public_tools)

Most of the pieces we’ve used here are explained in the Running Moore page. All we’ve done is tell Moore to run with its default configuration, using our line definition function to create the only line that it should run, and defined an input to use from the test file database.

The cache in the TrackStateProvider is not compatible with the new scheduler used by Moore as it relies on incidents for clearing. Until a final solution is implemented, a temporary workaround has been put in place, which is activated by stateProvider_with_simplified_geom. See LHCBPS-1835 and Rec!1584 for more details.

Moore needs to know the input file when running, so we’ll just an Upgrade minimum bias input data options file that comes with Moore:

./Moore/run gaudirun.py '$MOOREROOT/tests/options/default_input_and_conds_hlt2.py' test_line.py 2>&1 | tee logs/test_line.log

With any luck this will run, but it will soon fail with an error.

Upfront reconstruction

The example we’ve made will run, but upon inspecting the logfile you’ll notice that no candidates are produced by any algorithm.

This is because the HLT2 reconstruction is not fully defined in a way that allows us to use it with our example as-is. Instead, we need to modify our line definition only slightly to explicitly create the reconstruction before we start building our candidate:

from RecoConf.reconstruction_objects import upfront_reconstruction


@register_line_builder(all_lines)
def lb0_to_lcpim_line(name="Hlt2Tutorial_Lb0ToLcpPim_LcpToPpKmPip", prescale=1):


    return Hlt2Line(
        name=name,
        algs=upfront_reconstruction() + [require_pvs(pvs), lbs],
        prescale=prescale,
     )

This is just a temporary measure until the full HLT2 reconstruction is defined in Moore. You should include it for now.

Re-running

Run again and you’ll see the command complete successfully. Look at the log and see how many candidates were created. Seeing as we’re running over minimum bias data, you should expect to see very few candidates (ideally zero).

Hint

You can now also add the line for the \(\Lambda_{b}^{0} \to \Lambda_{c}^{+} \mu^{-} \bar{\nu}_{\mu}\).

What differences do you see in the log files?

Use the instructions in Analysing the output section to find the commands for generating and inspecting the control and data flow graphs that are produced when the options were run. The data flow for our example looks like this:

Whilst the control flow looks like this:

The control flow node for our HLT2 line contains quite a few steps. Most of these are the upfront reconstruction we added earlier. Again, the presence of these is just a detail for now; in the near future the control flow for a line will look much simpler, for our line being just the \(\Lambda_{b}^{0}\) combiner algorithm.

Inspecting the log-file

Some aspects of reading and understanding log files, like reading off Moore configuration options and the control flow table are documented in the “Running Moore” section.

Here, we focus on counters of filters and combiners, and the timing table. If you did not pass a name to ParticleFilter, the default will be ParticleRangeFilter. Every time a new instance of ParticleFilter is created, the name changes automatically to ParticleRangeFilter#i for the i+1 st instance in the sequence. In a similar manner combiners have as default names TwoBodyCombiner, ThreeBodyCombiner and FourBodyCombiner.

Note

As mentioned in the discussion on filters and combiners, the framework de-duplicates algorithms with the exact same configuration. For filters of basic particles, the likelihood of “accidentally” sharing the configuration with any line in Moore is relatively large. If there are multiple instances with different names, PyConf will raise an exception at compile time. Therefore we recommend to only give names to combiners.

These names appear in counters, the timing, and control flow table (if they are part of the control flow). We can search for them, for example with:

grep -A 7 'Tutorial_Lcp_Combiner' logs/test_line.log

and find something like:

151:Tutorial_Lcp_Combiner                                      INFO Number of counters : 6
152- |    Counter                                      |     #     |    sum     | mean/eff^* | rms/err^*  |     min     |     max     |
153- |*"# passed"                                      |       100 |          0 |(  0.000000 +-  0.000000)% |
154- |*"# passed Combination12Cut"                     |       128 |          4 |( 3.125000 +- 1.537892)% |
155- |*"# passed CombinationCut"                       |        10 |          0 |(  0.000000 +-  0.000000)% |
156- | "Input1 size"                                   |       100 |        152 |     1.5200 |
157- | "Input2 size"                                   |       100 |        133 |     1.3300 |
158- | "Input3 size"                                   |       100 |        244 |     2.4400 |
--
212: | "Tutorial_Lcp_Combiner"                                              |             100 |           0.045 |          454.773 |
213- | "FunctionalParticleMaker#3"                                          |             100 |           0.010 |          109.340 |
...

Let’s go through this line by line.

• The first line (151) prints the name of this GaudiAlgorithm instance and the number of counters.

• Line 152 shows the header; as there are different types of counters, the header contains columns that are not filled for all counters of this combiner.

• Line 153 counts the number of events in which the decision has been positive. In our case 0 out of 100 for which this combiner has been run.

• Line 154 counts the number of candidates in which the two-body combination cut decision of is positive. Here, there were 128 combinations (which implicitly passed the combiner’s decay descriptor), 4 out of which passed the two-body combination cut. As the Combination12Cut is a very cheap operation, and helps to reject background early on, you should always consider applying one.

• A similar counter in line 155 shows the positive combination cut decisions. In this case, we combined 4 candidates passing the Combination12Cut with the Input3 (pion) container of this event. This leads to 10 candidates, out of which 0 are selected.

• Lines 156 to 158 display the sizes and average per event multiplicities of input containers. They are in the same order as we passed them to the combiner, i.e. proton, kaon and pion candidates.

• You might have noticed that there is no counter for the CompositeCut of our combiner. It has been suppressed, since there was no input to be processed.

Tip

Understanding counters is extremely useful for developing and debugging your selection.

More on counters in this talk.

The second part of our grep result shows an except of the timing table where the header:

| Name of Algorithm | Execution Count | Total Time / s  | Avg. Time / us   |

is further up in the log.

Tip

To speed up your selection, you are mainly interested in the Total Time of your filters and combiners. This can be reduced by a

1. tighter selection on the inputs;

2. tighter selection in the Combination(12[34])Cut;

3. ordering cuts by efficiency and functor evaluation speed;

4. well chosen configuration of the control flow;

5. globally by sharing selections (not builders).

Further hints are given in the Timing and Performance section of the line authoring guidelines.

Full example

A full implementation example of the line described here can be found at Hlt/Hlt2Conf/python/Hlt2Conf/lines/hlt2_line_example.py. Have a look at this and see how it differs from yours. In particular, see how the imports have been organised near the top of the file, and everything has a consistent look.

You may have noticed that the make_ and h_for_charm functions have a leading underscore in their name. This is a detail that we will follow up on when discussing code design guidelines.

Hint

The full example runs 3 lines. One for the \(\Lambda_{b}^{0} \to \Lambda_{c}^{+} \pi^{-}\) decay, and two instances of the \(\Lambda_{b}^{0} \to \Lambda_{c}^{+} \mu^{-} \bar{\nu}_{\mu}\) line with a slightly modified pion \(p_\text{T}\) cut. Have a close look at the counters that the example produces. Can you understand all of them?

• We have two counters for the Tutorial_pions_for_charm_and_beauty filter. Which one is which?

• Why does one of them have fewer inputs?

• We have two counters for the Tutorial_Lcp_Combiner and three for Tutorial_Lb0_Combiner combiners, two of which look identical. Why is that?

• Some combiners don’t seem to run on all 100 events. Why is that?

• There is no Combination12Cut counter for Tutorial_Lcp_Combiner#1, even though the input containers for one event are not empty. What happened?

Try to follow elements from the example in your own line. Remember that it is your line, and you should feel free to really own it. Show off and make it nice!

Modifying thresholds

A common task when developing lines and when running the trigger is to modify cut values. This can be to increase or decrease the rate of the selection when running, or to fix a bug.

There are two ways to do this. The first is simple: edit the source! The values in the source signal your intent, and if your intent changes, so should the source. (The alternative is to have the values in the source and the ‘actual values’ that are used by running elsewhere, which can be confusing.)

What if you wanted to run a couple of instances of this line, but one with the standard cuts and one with some thresholds slightly modified? This can be achieved by using the @configurable decorator.

In the full example Hlt/Hlt2Conf/python/Hlt2Conf/lines/hlt2_line_example.py, we made use of this functionality

run_hlt2_line_example

def make_lines():
    standard_lines = [line_builder() for line_builder in all_lines.values()]

    # This is to demonstrate how `configurable`/`bind` works. We could also pass the function arguments directly lb0tolcpmum_line()
    with lb0tolcpmum_line.bind(
            name="Hlt2Tutorial_Lb0ToLcpMumNu_LcpToPpKmPip_Pip_pt450MeV",
            pi_pt_min=450 * MeV):
        modified_line = lb0tolcpmum_line()

    return standard_lines + [modified_line]


run_moore(options, make_lines, public_tools)

This configuration will run 3 lines: the \(\Lambda_{b}^{0} \to \Lambda_{c}^{+} \pi^{-}\) and \(\Lambda_{b}^{0} \to \Lambda_{c}^{+} \mu^{-} \bar{\nu}_{\mu}\) lines with default configuration, and the \(\Lambda_{b}^{0} \to \Lambda_{c}^{+} \mu^{-} \bar{\nu}_{\mu}\)

When running with:

Moore/run gaudirun.py '$HLT2CONFROOT/options/run_hlt2_line_example.py' 2>&1 | tee hlt2_line_example.log

Hint

1. You don’t need to modify the source code (but it is often better to do that!).

2. You don’t need to ‘expose’ everything you want to change on the top-level line maker, you just modify the behaviour of @configurable functions directly.

You can read a lot more about the @configurable decorator in the Tonic documentation.

Code design guidelines

These guidelines are not set in stone, and up for debate.

Important

Summary of best practices

1. Basic building blocks of the selections should be identified and declared locally if they cannot be taken from a shared location.

2. Builder functions (_make_*) should only be used if called more than once. DataHandles of reconstructed objects need to be passed as positional arguments. Optional arguments like names, decay descriptors or bool/enum-style variables should also be passed as positional arguments. Using cut values as arguments or passing *args or **kwargs is discouraged.

3. Selection steps with “rare” outputs should be part of the control flow. Best efforts on the order of objects should be made based on speed and rarity of the selection step.

4. Functions should never be imported from a module which registers lines. Functions that are only used in one module should start with an underscore, see the PEP 8 Style Guide for Python Code

5. A consistent naming scheme for lines and combiners can help with code readability and debugging.

6. A docstring at the beginning of a file can act like a table of contents and help to navigate through it. Searching a line name from a list in that comment can help to jump to the right place in the code.

Corollary: selection cuts are exposed once in the configuration, and cannot be overwritten.

The reasoning for these choices is as follows: The principles follow those of the RecoConf package, most notably

1. Reconstructed objects “flow” through the make_* functions.

2. Functions define a single “logical” step. The global data flow is configured in as “flat” as possible functions, where the logical steps are pieced together.

We make some distinctions for the selection configuration, as selections sit on top of a relatively long data flow. Exposing this to adhere to rule 1. would be too explicit, as the starting point would be the raw event. It thus makes sense to re-define starting points or basic building blocks for selections. In the example case these are high level objects like _protons_for_charm, _kaons_for_charm and _pions_for_charm_and_beauty.

Defining these building blocks is a design choice that each line-author has to make. In our case, we might even want to choose the \(\Lambda_{c}^{+}\) candidate as a point of entry. The basic building blocks can be declared locally in the module with the lines, or shared within a working group or with everybody (standard_particles).

Another distinction to the reconstruction configuration is that the majority of the selection uses exactly two algorithms: ParticleFilter and ParticleCombiner, but will create a vast number of instances of them to express the various selections. On the other hand, the reconstruction mostly uses dedicated algorithms, most of them with their own default parameter tuning. We would like to express our production-ready selections similarly, i.e have well tuned cuts for specific selection purposes (_make_lambdacs_for_beauty). Note that we have the flexibility to tune cuts to the last digit for every specific purpose separately, but the price of such an approach should be made clear:

Attention

Every call to ParticleFilter or ParticleCombiner with different inputs or different selection cuts will create a new instance of the algorithm.

To be explicit, imagine the following: After inspecting first data from both our example decays, we found that the hadronic selection would profit from a tighter \(p_\text{T}\) cut on the \(\Lambda_{c}^{+}\) candidate. To avoid boilerplate code, we could add an argument to the _make_lambdacs_for_beauty function that defines the \(p_\text{T}\) cut value and passes it to the functor. We would then call this function with different values from the line-defining function. But this will create 2 instances of ParticleCombiner, meaning that the full algorithm will run twice with slightly different cuts.

This kind of duplication should be avoided as much as possible, especially for relatively expensive operations like 3-body combinations with highly abundant inputs. A faster way in such a case might be to run a Particlefilter on the common \(\Lambda_{c}^{+}\) candidate, that only performs the tighter \(p_\text{T}\) cut.

Can you modify the example to confirm that this really speeds up the selection?

Selections live at the end of the trigger-food-chain, and the data-flow determines which steps need to be taken to make candidates for our line. However, selection steps often have more than one input, and the configuration offers the possibility to create artificial barriers in the data-flow. Taking \(\Lambda_{b}^{0} \to \Lambda_{c}^{+} \mu^{-} \bar{\nu}_{\mu}\) as an example, we can for example decide to run the combiner for \(\Lambda_{c}^{+} \to p K^{-} \pi^{+}\) only after a muon with large \(p_\text{T}\) and high \(\chi^2_\text{IP}\) has been found. Such a configuration of course only brings an advantage if the majority of events does not contain such a muon.

See also

To find out if a certain control flow configuration speeds up the selection, in most cases it is sufficient to look at the timing tables and counters of a log file when running on HLT1-filtered minBias data.

Importing a selection or function from a module that defines Hlt2 lines is discouraged. That is because the line authors of the module might not be aware that their function is used elsewhere and modify their selection, rename the function or change its behaviour. If the function to import defines a selection that should be common to both modules, the question is if this function should be moved to a shared file within the WG or LHCb, or if both selections should be part of the same module.

Line names have been briefly discussed in the section on Line declaration. There is no strict common set of best practices, but it makes sense to think about consistent names of lines beforehand, as we would like to avoid re-naming lines during data-taking. For debugging purposes, it has proven useful to overwrite the default names of combiners (e.g. TwoBodyCombiner#123 to Tutorial_Lb0_Combiner). For (machine-)readability it is useful to have names like MyWG_MyModule_MyCombiner. We don’t recommend naming filters, as it can easily lead to clashes during the automatic code-deduplication stage. See also Moore#378 and Moore#380 . The naming of builder-, filter- and line-defining functions itself is, apart from the leading underscore for local functions, not of great concern. We recommend keeping them in snake_case, short and descriptive.

Adding comments to the code is recommended. The comments should add information on the selection, provide pointers to further documentation, or remind the authors and others of future steps (# TODO). Commenting out code is discouraged.

Monitoring your line

Monitoring your line during data-taking is important for spotting errors as early as possible. We make a distinction between online- and offline-monitoring. While the former is histogram-based and happens in parallel with the Hlt2 processing, the latter can be done with a regular automatized offline production, as proposed by the Early Measurements Task Force here .

Default monitoring

The documentation here concerns the online-monitoring that can be configured in Moore. If you choose to not configure any monitoring, a set of default monitors will run. This means that every Hlt2 line will fill a set of histograms by default. Currently these default histograms are \(p_\text{T}\), \(\eta\), mass, vertex \(\chi^2\), \(\chi^2_\text{IP}\) w.r.t. the “best” PV and the candidate multiplicity. The monitors can be configured line-by-line with the monitoring_variables argument of the Hlt2Line instance. The argument is a tuple, and the current default is ("pt", "eta", "m", "vchi2", "ipchi2", "n_candidates"). More variables for automatic monitoring are, and can be, defined in monitoring.py. For switching off the default monitors for a line entirely, simply pass an empty tuple: (Hlt2Line(name=..., algs=[...], monitoring_variables=(), ...)). Default monitoring can also be switched off globally with

from Moore.monitoring import run_default_monitoring
...
with run_default_monitoring.bind(run=False):
  ...

Custom monitoring

Custom monitoring of particles can be configured individually for each line with a functor-based Algorithm called Monitor__ParticleRange. As an example, we can set up a mass monitor for the \(\Lambda_{c}^{+}\) that is an intermediate step of our tutorial line Hlt2Tutorial_Lb0ToLcpPim_LcpToPpKmPip_Line

from PyConf.Algorithms import Monitor__ParticleRange
...
# in the lb0_to_lcpim_line(name=...) function
lc_mass_mon = Monitor__ParticleRange(
  Input=lcs,
  Variable=F.MASS,
  HistogramName=f"/{name}/lc_m",
  Bins=60,
  Range=(2080 * MeV, 2480 * MeV),
)
...
return Hlt2Line(
          name=name,
          algs=upfront_reconstruction() + [require_pvs(pvs), lc_mass_mon, lbs],
          prescale=prescale,
          monitoring_variables=(),
       )

There are a few things to note:

• The monitor has to be included in the control flow, takes a datahandle of the particle to monitor (lc) and a functor of what should be monitored (F.MASS).

• In this case, it will plot all \(\Lambda_{c}^{+}\) that are used as input to the \(\Lambda_{b}^{0}\) combiner. If put after lbs in the control flow, we would only get a plot of \(\Lambda_{c}^{+}\) for which a \(\Lambda_{b}^{0}\) candidate is found (the ones you’d see offline, plus combinatorial or opposite side \(\Lambda_{c}^{+}\) candidates).

• For being able to benchmark monitors, when a name is given, it should start with Monitor_. This is implicit in the example (the algorithm will be called Monitor__ParticleRange#; you can modify it with name=Monitor_)

• The HistogramName property takes a full path. In this case there will be a histogram called lc_m in the directory {name} (the trigger line name) in the output file. It is recommended to stick to this naming, which is also used for the default monitoring (there, the histogram m would exist and be filled with the \(\Lambda_{b}^{0}\) mass).

• In the example, monitoring_variables is an empty tuple. If this property would not be set explicitly, the default monitoring histograms would be filled as well.

Test your monitors

To test the monitoring, add the following to your Moore options file:

options.histo_file = 'my_histograms.root'

For the time being, this will create two files, and the histograms from Monitor__ParticleRange will end up in my_histograms_new.root (while histograms created with older algorithms will be put to my_histograms.root).

Next steps

An important aspect of authoring an HLT2 line is stepping back and spotting instances of code duplication. Multiple instances of the same intent can be refactored into a common function. This reduces any maintenance burden and decreases the likelihood of two implementations slowly drifting apart over time (if someone changes one but does not know about the existence of the other).

When writing lines it’s extremely useful to be able to be able to analyse the output files. It’s also helpful to refer to the documentation on Debugging in case something isn’t working. Once you’re ready to start physics performance studies, you can start Analysing HLT2 output. The Running with Ganga page has instructions for writing Ganga-compatible options.

Guidelines on selection rate, output bandwidth, and timing are given in Line authoring guidelines.

Differences from Run 2 configurations

This section is yet to be written. If you have some tips and tricks on porting Run 2 Stripping and HLT2 lines to Moore, please open a merge request!