Introduction

This page describes how the MRTG monitoring for the WLCG Squids is configured and how to make changes.

Setup

Cron jobs:

As squidmon user:

% crontab -l
# Updating of GOCDB and OIM squid records
40 */3 * * * [ "`cl_status rscstatus`" == "all" ] || exit; /home/squidmon/scripts/grid-services/get_squids.py >> /home/squidmon/logs/get_grid_squids.log 2>&1
44 */3 * * * [ "`cl_status rscstatus`" == "all" ] || exit; /home/squidmon/scripts/all/makeConfig.py >> /home/squidmon/logs/all/makeConfig.log 2>&1

# Updating MRTG monitoring that uses GOCDB and OIM records
*/5 * * * * [ "`cl_status rscstatus`" == "all" ] || exit; /home/squidmon/scripts/all/mrtg_plots.py >> /home/squidmon/logs/all/mrtg_plots.log 2>&1

Retrieving information from GOCDB / OIM

A cron job run every 3 hours:

40 */3 * * * [ "`cl_status rscstatus`" == "all" ] || exit; /home/squidmon/scripts/grid-services/get_squids.py >> /home/squidmon/logs/get_grid_squids.log 2>&1

This creates a file called /home/squidmon/www/grid-squids.json which contains a list of the squid services, the site name and the monitoring port:

% head /home/squidmon/www/grid-squids.json
{
  "cvmfs-px4.cr.cnaf.infn.it": {
    "source": "egi", 
    "name": "INFN-T1", 
    "port": 3401
  }, 
  "squid1.ppgrid1.rhul.ac.uk": {
    "source": "egi", 
    "name": "UKI-LT2-RHUL", 
    "port": 3401 

The information is taken from the GOCDB and OIM. The script then checks and exception list and modifies the list if necessary before saving the file. The field source will either be egi if the squid came from the GOCDB, osg if the squid came from OIM and exception if it came from the exception list.

Adding exceptions

Exceptions can be added to /home/squidmon/conf/all

% cat exceptions.txt 
#######################################################################
#
# This file contains a list of exceptions for the squid monitoring
#
# ADD <site> <alias> <port>
# Will add (or overwrite if alias already exists) squid
#
#
# REMOVE <alias>
# Will remove the listed alias from being monitored
# Will print a warning if alias doesn't exist
#
#######################################################################
ADD CERN-PROD ca-proxy.cern.ch 3401
REMOVE lcg0679.gridpp.rl.ac.uk

Creating config files

A cron job run every 3 hours and 4 minutes after the information has been updated from the GOCDB/OIM:

44 */3 * * * [ "`cl_status rscstatus`" == "all" ] || exit; /home/squidmon/scripts/all/makeConfig.py >> /home/squidmon/logs/all/makeConfig.log 2>&1

This creates the .cfg files and puts them in:

% ls /home/squidmon/etc/all/
ANLASC-squid.cfg       Australia-ATLAS_1.cfg  CA-VICTORIA-WESTGRID-T2.cfg  DukeT3_SQUID.cfg  GRIF.cfg        INDIACMS-TIFR.cfg  JINR-T1.cfg            NGI_IT.cfg      RAL-LCG2_1.cfg  RWTH-Aachen.cfg    SARA-MATRIX_1.cfg  SBU_T3_Squid.cfg
ANLASC-squid.ok        Australia-ATLAS_1.ok   CA-VICTORIA-WESTGRID-T2.ok   DukeT3_SQUID.ok   GRIF.ok         INDIACMS-TIFR.ok   JINR-T1.ok             NGI_IT.ok       RAL-LCG2_1.ok   RWTH-Aachen.ok     SARA-MATRIX_1.ok   SBU_T3_Squid.ok
Australia-ATLAS_0.cfg  Australia-ATLAS.cfg    DESY-ZN.cfg                  FMPhI-UNIBA.cfg   IN2P3-LAPP.cfg  JINR-LCG2.cfg      Nevis_cvmfs_squid.cfg  RAL-LCG2_0.cfg  RAL-LCG2.cfg    SARA-MATRIX_0.cfg  SARA-MATRIX.cfg
Australia-ATLAS_0.ok   Australia-ATLAS.ok     DESY-ZN.ok                   FMPhI-UNIBA.ok    IN2P3-LAPP.ok   JINR-LCG2.ok       Nevis_cvmfs_squid.ok   RAL-LCG2_0.ok   RAL-LCG2.ok     SARA-MATRIX_0.ok   SARA-MATRIX.ok

MRTG plots

A cron job run every 5 minutes:

# Updating MRTG monitoring that uses GOCDB and OIM records
*/5 * * * * [ "`cl_status rscstatus`" == "all" ] || exit; /home/squidmon/scripts/all/mrtg_plots.py >> /home/squidmon/logs/all/mrtg_plots.log 2>&1

The script is relatively straight forward, it

Webpage

The index.html file is created by hand.

-- AlastairDewhurst - 2015-12-09