Flexible configuration with Tonic
Tonic is a small Python package that aims to allow modular configuration whose behaviour can be overridden in a way that is easy to comprehend and debug.
You might have seen uses of the @configurable decorator in Moore code. This
is part of Tonic. If you want to understand the motivation behind this, and
when you might want to use it, read on. The detailed documentation of the
Tonic API follows after that.
Designing flexible configuration
We’ve structured the Moore configuration in a way that packages each small step in separate functions. In this way, your HLT2 line calls a function that makes pions, say, and this calls a function which produces tracks, which calls a functions that loads raw data. Your line creates a stack of function calls:
hlt2_line()
calls make_pions()
calls make_tracks()
calls make_hits()
calls make_raw()
This separation of code helps to organise things.
But now you want to modify some behaviour of some function in the middle, say make_tracks. How can you do this? There are a couple of ways you might come up with.
Modify the source code of the function directly.
Copy the function as some new version and use that.
Option 1 is perfectly valid, and is how many Moore developers work, as explained in Developing Moore.
Option 2 brings a problem. The caller of the function you copied is still calling the original version. So, now you need to modify or copy that function. And now you have to repeat this for the caller of that function! This gets very cumbersome very quickly.
The @configurable decorator
The configurable decorator is designed to help in these situations. It allows
you to override the arguments of a function wherever it happens to be called.
So even if it’s called deep down the call stack, you still have some ability to
override its behaviour.
Take this representative example:
from PyConf import configurable
from PyConf.Algorithms import HitMaker, TrackMaker, SelectPions, ParticleCombiner
from Moore.lines import DecisionLine
def make_raw():
pass
def make_hits():
return HitMaker(RawEvent=make_raw())
@configurable
def make_tracks(pT_threshold=200 * MeV):
return TrackMaker(Hits=make_hits())
@configurable
def make_pions(max_pidk=5):
return SelectPions(MaxPIDK=max_pidk, tracks=make_tracks())
def dipion_line(name="Hlt2DiPionLine", prescale=1.0):
pions = make_pions()
dipions = ParticleCombiner(Decay="B0 -> pi+ pi-", Particles=[make_pions])
return DecisionLine(name, algs=[dipions], prescale=prescale)
You can see the full chain going from the line to the raw event.
Now we want to study what effect changing the PIDK cut has on the rate of
our line. We could just modify the call to make_pions directly. This
requires modifying the source of the dipion_line function:
def dipion_line(name="Hlt2DiPionLine", prescale=1.0):
# Remember to uncomment this back when we're done!
# pions = make_pions()
pions = make_pions(max_pidk=0)
dipions = ParticleCombiner(Decay="B0 -> pi+ pi-", Particles=[make_pions])
return DecisionLine(name, algs=[dipions], prescale=prescale)
Instead of doing this, we can use the bind method that’s made available on
all functions decorated with configurable:
with make_pions.bind(max_pidk=0):
line = dipion_line()
It’s as easy as that! When you use bind with a context manager like
this, any calls to the ‘bound function’ (make_pions in this case) will be
intercepted, and the argument value you’ve specified will override the original
value. In the example, the value of the max_pidk argument will be 0
inside the with block rather than the default of 5.
Multiple calls to bind can be used in the same with statement. This means
we could also modify the tracking threshold along with the PID cut:
with make_pions.bind(max_pidk=0), make_tracks(pT_threshold=500 * MeV):
line = dipion_line()
So, when should you use configurable to
decorate your functions? There are some cases when it never makes sense:
Functions with no arguments, as there’s nothing to override.
Functions that you don’t expect to be buried inside a call stack. Line functions, like our
dipion_line, are an example. They are usually called at the top level of script, so if we wanted to override some argument values we would just do so directly:line_standard = dipion_line() line_prescaled = dipion_line(name="Hlt2DiPionPrescaledLine", prescale=0.5)
Outside of these, it depends how you expect the function to be used. It’s
generally safe to add configurable, but you can
also just omit it. We can always add it later if it turns out it’s needed.
Remember that the standard development flow has the full source code checked
out; it’s often easier just to modify it directly rather than jumping through
bind calls!
Note
Overriding of deeply nested components was something quite common when we
used objects called Configurables. These could be retrieved inside any
scope based on a name: if you knew the name of the Configurable, you could
retrieve it and modify its properties. This permits ultimate flexibility.
The trouble with this approach is that any one can modify any Configurable at any time. It becomes tricky to keep track of exactly who is modifying what, and what piece of code sets the final value the Configurable ends up with. In an application like the trigger, it’s very important to be able to understand exactly what’s going on!
Using the configurable decorator is an
alternative that tries to make overriding more explicit. Everything
happens in the callstack, and nothing outside it can mess around inside
it. Using bind only modifies things within a very specific scope.
There are a couple of other useful features you can use when using
configurable, such as tonic.debug and substitute. This are described in
the PyConf.tonic documentation below.
Tonic API
Wrappers for defining functions that can be configured higher up the call stack.
Tonic provides the @configurable decorator, which allows the
default values of keyword arguments to be overridden from higher up the
callstack with bind.
>>> from PyConf import configurable
>>> @configurable
... def f(a=1):
... return a
...
>>> with f.bind(a=2):
... f()
...
2
>>> f()
1
This allows for high-level configuration of behaviour deep within an
application; all that’s needed is a reference to the configurable function
that one wishes to modify the behaviour of.
The idiomatic way of using tonic is define small, self-contained functions
which construct some object of interest. These functions should call other,
similarly self-contained functions to retrieve any components which are
dependencies. Like this, callers can override the behaviour of any function in
the call stack with bind. Each function should expose configurable parameters
as keyword arguments.
To help debugging, bindings can be inspected using the debug context manager.
>>> from PyConf import tonic
>>> with tonic.debug():
... f()
... f(a=3)
... with f.bind(a=2):
... f()
...
1
3
2
Functions marked configurable can also be substituted entirely with
substitute.
>>> @configurable
... def echo(arg=123):
... return arg
...
>>> def echo_constant():
... return 456
...
>>> with echo.substitute(echo_constant):
... echo()
...
456
>>> echo()
123
tonic is named for Google’s gin configuration framework 1 which served as inspiration.
- add_cache_serializer(f, *args, **kwargs)[source]
Insert a function that can serialize custom objects.
The function f is tried and if it raises TypeError, the previous global serializer function is attempted.
- Parameters
f (callable) –
f(obj, *args, **kwargs)must convert obj to a serializable representation or raise TypeError.
- bound_parameters(configurable)[source]
Return the parameters bound to configurable in the current stack scope.