NP04 Photon Detection System

Make an Entire PD Run Using the DAQ (Editing in Progress)

Quick outline:

1. Make sure ALL Lights are OFF, including cameras 201-203, using: =NP04_DCS_01/CAMERA CONTROL. First check here
   • Turn off Camera LEDs
     • Click LED Camera 1-5
     • Click off, bottom left
     • For Cameras 201-203, go to here click HERE
       • Scroll down to bottom: click IR LED off, then save settings
     • Check ALL cameras to make sure there is no light in cryostat, click HERE
2. Turn on SSP Power Supply: LV, & HV, using: NP04_DCS_01/PD Power Setup/
   • Turn on in this order
     • Power (Power supply for powering the SSPs, LV)
     • Bias (Voltage supply for biasing SiPMs/MPPCs, HV)
3. Bias SiPMs/MPPCs, using: NP04_DCS_01/SSP Info/
   • Click Set Voltage
   • Enter correct bias voltage/channel/SSP
     • Note: units are in mV
     • Make sure to not confuse "standard" & "modified" SSPs
   • Click SET
4. Start DAQ setup
   • Currently Partition 1 (and maybe 2) is setup for PDs (SSPs)
   • Current working configurations for PD system
     • dune-artdaq_artdaq_v3_03_00_beta
     • =np04_WibsReal_Ssps000111=
     • for configuration updates, check configurations here
   • Follow shifter instructions here: Start/Stop a run with Run Control
5. After finished PD data taking,
   • Bias SiPMs/MPPCs back to zero BEFORE turning off any SSP power supply
     • Click on Set Voltage, using: NP04_DCS_01/SSP Info/
     • Enter 0mV for ALL channels/SSP
     • Click SET
   • Turn off HV, via NP04_DCS_01/PD Power Setup/
     • Click off for SSP Bias

You have now completed taking PD data!

Hardware and DAQ configuration

Quick Reference for magic numbers

dsp_clock_control: 0x0 #[internal clock] 0x431 #[pdts clock]

ALL_channel control: 0x0 #[disabled] 0x1 #[pdts trigger only] 0x401 #[pdts+internal triggers]

ARR_pdts_cmd_control: [0xFF000000, 0x00000000, 0x00000FFF] #[normal operation] [0x00000000, 0x00000000, 0x00000000] #[No PDTS trigger]

Note on SSP triggering scheme, data format and nomenclature

The SSPs consist of 12 individual channels, each connected to a SiPM photosensor. Each channel is individually triggerable, where triggers can come from a periodic timestamp trigger, an internal trigger based on a leading-edge discriminator local to the individual channel, or an external trigger from thre timing system. Upon seeing any trigger, a channel will produce a data packet consisting of a header, and optionally waveform information, which comprises a series of ADC values. The header contains book-keeping information (module and channel numbers, timestamps, trigger word etc.) and also some calculated values for the waveform, making the assumption that the waveform contains a peak - please see the SSP user manual (DUNE DocDB 1571) for full documentation of these calculations. Note that because the SSP trigger system is distinct from the global trigger, we refer to the unit of data produced when an SSP channel triggers as a "packet", and try and reserve the term "trigger" to refer to the global triggers only.

Because the SSP readout is not inherently synchronous with the global DAQ (except insofar as a trigger from the timing system will generate packets from all SSP channels), the data format is a little different from, e.g. the RCEs. The SSP boardreader generates a fragment when a trigger produced by the timing system is observed, and this fragment will contain all packets received from the SSP with timestamps in a window +/- 2.5ms from the timestamp of the trigger. In general, therefore, an SSP fragment will contain a fixed 12 packets with identical timestamps corresponding to the trigger time, one for each channel, and also an arbitrary number of additional packets generated when a channel's discriminator fires. It should be expected that a different number of packets will be observed for each channel within a given fragment. In the case that only the external trigger is enabled, exactly 12 packets should be present in a fragment. Note that any packets recieved by the SSP which do not fall within the window around a timing system trigger will be dropped and never included in a fragment.

Hardware configuration

All configuration of the SSP is performed by defining the values of hardware registers directly in the boardreader FHiCL. These register values are then loaded into the SSP when a config transition is made by the DAQ. The boardreader looks for the hardware configuration in its FHiCL at daq.fragment_receiver.HardwareConfig. In our present workflow, edits to the SSP hardware config are made by the user in the file common_code/ssp_standard.fcl, in the section ssp_standard.fragment_receiver.HardwareConfig, which gets aliased into daq.fragment_receiver.HardwareConfig for all SSP boardreaders.

Within the relevant FHiCL block, a register is set by simply setting a FHiCL parameter with the same name as the register, to the value which one wishes to assign. One complication is that some of the registers in the SSP form arrays (e.g. channel_control[0..11]). To set all registers in an array to the same value, one would set a FHiCL parameter with the register name preceded by "ALL_", e.g. ALL_channel_control. To set each element of an array to a separate value, do:

ARR_channel_control: [val0, val1, ... val11].

The most commonly-edited settings will be documented here (requests to Jennifer Haigh for more particulars are welcome). For further information, please refer to the SSP user manual and register map (DUNE DocDB 910), though it should be noted that these documents are no longer entirely up-to-date and are missing relevant information, particularly regarding the interface of the SSP to the timing system.

[NB for experts. The name/address mapping in the register map document is manually reproduced in the dune_artdaq source, in dune_artdaq/Generators/anlBoard/RegMap.cxx, and this is the version used to convert the FHiCL block into concrete register settings. Due to issues with documentation and asynchronous updates, there may be small differences between the map produced by ANL and the map in the source file, so some cross-referencing may very occasionally be needed.]

[NB an exception to the described scheme is SiPM bias voltage settings, which are set using a separate slow control application, so that DAQ transitions can be maintained as "safe" operations. Internally the bias levels are actually registers, so this is a safety/policy requirement, not a technical one.]

Timing and trigger configuration

Clock source

To set the clock used to drive the SSP data processing, set the register dsp_clock_control. A value of 0x0 will use the internal clock, 0x431 will use the timing system clock. When isolated from the timing system, the internal clock must be used. When using the timing system, the SSP multiplies the 50MHz input clock up to 150MHz, so that the sampling rate is 150MHz regardless of clock source. Note that in order for correct external timestamps to be generated for the SSP, at least one sync pulse must be sent to the endpoints by the timing system after configuration of the SSPs is complete and before any triggers are issued.

Internal discriminator

To disable generation of packets on internal discriminator events, set ALL_channel_control to 0x1. To enable, set ALL_channel_control to 0x401. To completely disable all channels, set ALL_channel_control to 0x0. The threshold of the discriminator can be set using ALL_led_threshold. Experience will be needed to decide on useful values, but I'd suggest:

7..10: Somewhere in this range will give you ~a few packets from electronics noise within a 5ms window, in the absence of any extrinsic noise. Maybe try a value of 9 first and experiment if necessary.

~15: Above the intrinsic channel noise level, this should trigger on dark noise pulses if SiPM voltages are above breakdown. It might be necessary to increase this number if the environment is noisy.

Global triggers

To disable generation of packets based on timing system triggers, set ARR_pdts_cmd_control to 0x0. To enable, assuming that the top 8 PDTS command codes represent triggers, set ARR_pdts_cmd_control: [0x00080000, 0x00000000, 0x00000FFF]. Note that since the boardreader uses packets corresponding to triggers from the timing system to control fragment generation, this is not recommended for non-experts.

If it is necessary to change which pdts signal(s) correspond to a trigger, start using the in-spill/out-of-spill functionality etc., please get in touch with Jen for now.

Window sizes

Set ALL_readout_window equal to the number of waveform samples you wish to store for each packet (0: header only, 2000: max). You can also choose how many of these samples are before the trigger/discriminator time, by setting ALL_readout_pretrigger.

DAQ configuration

Similarly to the SSP hardware, the boardreader software configuration is done entirely in FHiCL. The boardreader looks for its configuration in daq.fragment_receiver.DAQConfig, but edits to the file common_code/ssp_standard.fcl, in the section ssp_standard.fragment_receiver.DAQConfig, make it here by aliasing. Changing the configuration is largely an expert operation but some documentation is provided here.

DummyTriggerPeriod: How long, in ticks, to go in between tagging dummy triggers onto packets, in order to generate fragments in the absence of real global triggers. Default to -1, or off, which is how it should be in normal running.

PreTrigLength: Window length in ticks for packets to be included in a fragment. This is the length of the window before the trigger timestamp.

PostTrigLength: Length of the window after the trigger timestamp.

TriggerWriteDelay: This is used to determine when to build a fragment. When a trigger from the timing system is seen, the DAQ works out the window boundaries, and waits until it sees a packet with timestamp of at least (triggerTime+PostTrigLength+TriggerWriteDelay) before actually building a fragment. This is to make sure we don't build a fragment when not all the data has arrived.

UseExternalTimestamp: Whether to use the external timestamp to determine whether to include packets in fragment. Both timestamps are stored in the packets anyway.

HardwareClockRate: in MHz. Doesn't change it, just tells the DAQ what it is. Should always be 150MHz.

TriggerMask: Which bits in packet header word 1, 16:31 define a global trigger which should be used to build a fragment around. Sorry for horrible nomenclature, but the thing reported as "Trigger type" by the header dump method in the code, is the top 8 bits of this field. Please leave this at 0x2000 for now.

SSP Slow control and monitoring

A slow control system for the SSPs allows to set the bias voltages of individual channels, and exposes a subset of the SSP register values for monitoring purposes. This functionality is now available via the standard slow control GUI and for most purposes it is no longer necessary to use command-line tools once the slow control backend is running.

Slow control backend

The slow control backend for the SSPs is a DIM server process. Each SSP has its own DIM server, but a script exists which starts up all of the servers, and periodically restarts any which has crashed. This script is set up to run only on np04-srv-012, to avoid having multiple instances of the servers running on different machines. To start up the servers, log onto np04-srv-012 and do:

• source /nfs/sw/work_dirs/sspQuery/setup.sh
• source /nfs/sw/work_dirs/sspQuery/initDimSrv.sh

This step only needs to be taken if the slow control GUI indicates a communications error when the SSPs are otherwise responsive. Under normal circumstances, this will only happen when np04-srv-012 is rebooted.

Controlling SiPM Voltage

Log into np04-srv-012,

source /nfs/sw/work_dirs/sspQuery/setup.sh

, and then:

sspBiasControl.exe IP_address will read back the currently set bias voltages,

sspBiasControl.exe IP_address Voltage_in_mV will set all channels to the selected voltage, rejecting if it is >27000mV.

To manually set voltages via DID:

ssh -Y np04daq@np04-srv-010

export DIM_DNS_NODE=cs-ccr-mpdds1

did

Checking timing endpoint on SSP

Go onto machine np04-srv-012 as np04daq

source /nfs/sw/work_dirs/sspQuery/setup.sh

then to check the sync do sspPDTSStatus.exe [IP address of SSP]

New SSP

SSP IPs are available via LanDB. Their names are of the form: np04-ssp-XXX (starting 102)

Old SSP

SSP MAC address = 32:02:02:02:20:01

Must use arping to probe:

arping -I eth0 192.168.200.51

This is the current IP : 192.168.200.51

Run PD SSP Online Monitor on np04:

Manual:

• Login on np04daq@np04-srv-001
• Version built on /nfs/sw/om/ProtoDUNE_OM_artdaq_v3_02_dev
• source /nfs/sw/om/ProtoDUNE_OM_artdaq_v3_02_dev/setupONLINEMONITORING
• cd your_output_dir
• art -c /nfs/sw/om/ProtoDUNE_OM_artdaq_v3_02_dev/srcs/dunetpc/dune/Protodune/singlephase/RawDecoding/OnMon.fcl /data0/np04_raw_run_runnumber.root > output.log &
• Look at your output in RawDecoder_hist.root

Paolo Franchini ROOT script:

• cd /nfs/home/np04daq/pfranchini/DUNE/PD
• source /nfs/sw/om/ProtoDUNE_OM_artdaq_v3_02_dev/setupONLINEMONITORING
• /om.sh run_number (e.g. 001983_5)
• it creates a sub directory with all the plots and a pdf including all of them

List of available plots (not all are in MONET):

• packets frequency
• packets per event
• peaks height distribution

• pedestal vs event (per channel)
• area vs event (per channel)
• peak vs event (per channel)
• area vs peak (per channel)
• persistent waveform (per channel)

• area distribution (per channel)
• peak distribution (per channel)
• FFT (per channel)
• trigger type (per channel)

-- KarolH - 2017-06-29

• ssp-map.png:
  

• PDChannel_Map_Final_Aug2018.pdf: Final PD Channel Map-Aug2018