Gridview Administration Guide

This document describes the steps involved in installing, configuring and administration of the Gridview service.

Gridview Related Links:

1. Gridview Twiki Page
2. Gridview Service Dash Board Notes
3. Gridview Deployment at CERN
4. Gridview Admin Guide
5. Quattor Installation of Gridview
6. Gridview Software Release Status
7. Installation and Configuration of Gridview Publisher
8. Installation and Configuration of Gridview Web Service Clients
9. Using Gridview XML Interface
10. URL for Excel Report generation
11. Downloads
12. New Gridview Interface
13. Gridview Monthly Availabilities and Reports

Contents

Gridview components and rpms:

More information about gridview components and their functions can be found in Gridview Deployment Page. The following table lists out the different modules which are packaged in rpm format.

Module	Description	Name of rpm
Common	Common module needed in all instances	gridview-common
Frontend	Frontend modules (web interface)	gridview-frontend
Summarizers	Summarizer modules for different metrics	gridview-summarizer
Synchronizers	Synchronizer modules for GOCDB and CICDB	gridview-synchronizer
Archivers	R-GMA and web service archvers	gridview-archiver

Gridview Installation Pre-requisites:

The following middleware components should be available in the machines before installing Gridview.

Module	Pre-requisite middleware components	Remarks
Common	Oracle 10g Instantclient library	Should be available on every Gridview server
Frontend	Apache Web Server, PHP, PHP-gd	PHP-gd is the GD library support module for PHP
Summarizers and Synchronizers	PHP and CRON	These are sets of PHP programs run as Cron jobs
Archivers	Java, Tomcat, R-GMA client	All archivers are written in Java, the Web service Archivers run under Tomcat

Grid certificates for Gridview machines:

Standard grid certificates issued by the CERN Trusted CA are required to be installed on each gridview system. These certificates are typically stored in /etc/grid-security directory, from which they need to be copied to different directories for gridview.

User IDs for Gridview:

Different modules of Gridview run under different user credentials because of their environment as listed below:

Module	Runs under	User ID
Frontend	Apache	apache
Summarizers	Standalone	gridview
Synchronizers	Standalone	gridview
Archivers	Both standalone and under tomcat	gridview and tomcat

A new user 'gridview' with group 'gridview' needs to be created in the system. The group 'gridview' should contain the users 'gridview', 'apache' and 'tomcat' in order to be able to read the configuration file and write logs.

Configuration:

Configuration of gridview and pre-requisite rpms can be either performed manually or by Gridview NCM Component under Quattor.

Oracle Database:

It is required to have an Oracle Database account before proceeding with Gridview configuration. Presently this will have to be done manually. The Oracle database should be accessible with 3 different user accounts

• Owner/Administrator account: Create tables, indexes and table synonyms
• Reader account: For read-only access
• Writer account: For read-write access

The following operations should be performed on the database:

• Create all required tables by means of SQL scripts in /opt/gridview/scripts
• Create synonyms for read/write access
• Grant accesses to Service Availability related tables to SAM service
• Populate some of the tables with initial values

The following table describes the different scripts and their function:

Script	Function
create_general_tabs.sql	Create some general tables
create_gridftp_tabs.sql	Create tables related to data transfer
populate_gridftp_tabs.sql	Populate certain data transfer tables with initial values
create_job_tabs.sql	Create tables related with job status
populate_job_tabs.sql	Populate certain job status tables with initial values
create_same_tabs.sql	Create tables related with service availability monitoring
populate_same_tabs.sql	Populate certain service availability monitoring tables with initial values

The create_same_tabs.sql script also creates synonyms to certain tables created by the SAM service.

Gridview Configuration File

All modules of Gridview share a common configuration file "/opt/gridview/etc/gridview-common.conf". Configuration consists of a set of key/value pairs. The file is informally divided into different sections purely for readability, otherwise the order of the lines do not make any difference. Each gridview module parses the file and picks up the configuration it needs.

A sample configuration file is shown below:

# Gridview configuration file. This file could be automatically
# generated in Quattorized machines. Otherwise feel free to edit
# this file as long as you know what you are doing!

# Section 1: Gridview-Common configuration (Oracle database)
# There should be a separate reader and writer accounts to the
# single Gridview repository. Information to be filled in here
# should be made available by the DBA.
#gridview.common.db.reader.database =
#gridview.common.db.reader.hostspec =
#gridview.common.db.reader.options =
#gridview.common.db.reader.password =
#gridview.common.db.reader.port =
#gridview.common.db.reader.protocol =
#gridview.common.db.reader.username =
#gridview.common.db.writer.database =
#gridview.common.db.writer.hostspec =
#gridview.common.db.writer.options =
#gridview.common.db.writer.password =
#gridview.common.db.writer.port =
#gridview.common.db.writer.protocol =
#gridview.common.db.writer.username =

# Section 2: Gridview Frontend configuration
# Path name for the directory containing bitstream vera truetype fonts.
# By default jpgraph searches in /usr/X11R6/lib/X11/fonts/truetype
#gridview.frontend.ttf_path =

# Webroot for this installation. Typically if Gridview is installed with URl
# http://gridview.cern.ch/GRIDVIEW/  the webroot will be /GRIDVIEW/
#gridview.frontend.webroot =

# Section 3: Gridview Synchronizer Configuration
# Here we have currently 2 synchronizers - one for GOCDB and the other for CICDB
# For both we define oracle db parameters.
# CICDB
#gridview.synchronizer.cicdb.db.database =
#gridview.synchronizer.cicdb.db.hostspec =
#gridview.synchronizer.cicdb.db.options =
#gridview.synchronizer.cicdb.db.password =
#gridview.synchronizer.cicdb.db.port =
#gridview.synchronizer.cicdb.db.protocol =
#gridview.synchronizer.cicdb.db.username =

# GOCDB
#gridview.synchronizer.gocdb.db.database =
#gridview.synchronizer.gocdb.db.hostspec =
#gridview.synchronizer.gocdb.db.options =
#gridview.synchronizer.gocdb.db.password =
#gridview.synchronizer.gocdb.db.port =
#gridview.synchronizer.gocdb.db.protocol =
#gridview.synchronizer.gocdb.db.username =

Gridview Frontend Configuration:

The Gridview frontend modules have dependencies on Apache, PHP and Jpgraph.

Apache Configuration for Gridview

This involves the following actions:

• Creating an alias for Gridview and permanent redirection to this alias, if needed
• Configuring SSL certificates and parameters
• Copying of certificates

A sample /etc/httpd/conf.d/httpd-gridview.conf is shown below:

# Application gridview
Redirect permanent /index.php http://gridview.cern.ch/GRIDVIEW/

Alias /GRIDVIEW/ "/opt/gridview/frontend/"

The following parameter should be modified in the standard /etc/httpd/conf.d/ssl.conf for gridview:

SSLVerifyClient optional
SSLVerifyDepth  10

PHP Configuration for Gridview

The PHP configuration file is named

/etc/php.ini

. The following parameters in the file need to be configured for Gridview.

max_execution_time = 0
memory_limit = 256M
error_reporting = E_ALL
display_errors = Off

Bitstream Vera truetype fonts for jpgraph

Gridview uses jpgraph library for generating graphs. Vera and Vera Mono fonts are used for the lettering inside the graphs. By default jpgraph library looks for these font files in

/usr/X11R6/lib/X11/fonts/truetype

directory whereas bitstream fonts are usually stored in /usr/share/fonts/bitstream-vera directory. Therefore jpgraph needs to be configured to use the correct directory. For this, the following line should be added to /opt/gridview/frontend/lib/jpgraph/src/jpg-config.inc.php :

DEFINE("TTF_DIR","/usr/share/fonts/bitstream-vera/");

Please note that this line should be added to the file before the closing

?>

tag.

Gridview Summarizer Configuration

The PHP configuration described above in the frontend configuration will be required for summarizers too.

There are no specific configuration related to the summarizer modules. All summarizer modules reside under the subtree

/opt/gridview/summarizer

In this subtree are directories for data transfer(dt), job status (js) and service availability(sa) summarizers. The 'sbin' directories of each summarizer contains the summarizer startup shell script and the libexec directory contains the actual summarizer php script.

Summarizer programs need to be activated as cron jobs under the user name 'gridview'. Typically these should be run with a frequency of an hour, but higher frequencies (multiple times in an hour) are also possible. The summarizer scripts create lock files in /var/run/gridview directory so that multiple cron jobs do not clash with each other. An example cron job for data transfer summarizer is shown below.

25,55 * * * * gridview /opt/gridview/summarizer/dt/sbin/gridview-summarizer-dt.sh >> /var/log/gridview/gridview-summarizer-dt.log 2>&1

This table shows all summarizer scripts for the different metrics:

Module	Summarizer script
Data transfer	/opt/gridview/summarizer/dt/bin/gridftpSummarize.sh
Job Status	/opt/gridview/summarizer/js/bin/JobStatusSummarize.sh, JobMonitorSummarize.sh, JobStatusCorrect.sh
Service Availability	/opt/gridview/summarizer/sa/bin/sam_summarization.sh, sam_con_summarization.sh

Gridview Synchronizer Configuration

The PHP configuration described above will be required for synchronizers too.

Presently, two synchronizers are operational in Gridview - one, the GOCDB synchronizer and the second, the CICDB synchronizer. Both these remote databases are hosted in Oracle and Gridview has direct database read only access to these tables. To enable accesses to these databases, the /etc/tnsnames.ora file should be updated with Oracle access information to the databases. Moreover the gridview configuration file should contain the login and password information for these databases.

Similar to the summarizers described above, the synchronizer programs too need to be activated as cron jobs with a frequency of once in an hour. A sample cron job is shown below.

10 * * * * /opt/gridview/synchronizer/gocdb3/bin/SyncGocDB3/syncdb3.sh >> /opt/gridview/logs/SyncGocDB3.log 2>&1

Gridview Archivers Configuration

Gridview archiver code is completely written in Java and requires Java and Tomcat components to run. There are two kinds of archivers.

1. R-GMA archivers: These archivers read tuples from R-GMA and archive them to Gridview DB. These programs run as standalone Java applications run as independent services from the Linux init.d subsystem.
2. Web Service archivers: These archivers are implemented as web services running under Apache Axis which in turn runs under Tomcat. The Axis jar files are bundled in the gridview archiver rpm.

R-GMA Archiver configuration

There are two R-GMA archivers, each available as a separate service in the init.d subsystem.

1. gridview-archiver-rgma-dt (Data transfer archiver)
2. gridview-archiver-rgma-js (Job Status archiver)

Each service could be started and stopped using standard init.d syntax

         service gridview-archiver-rgma-dt [start | stop | status | restart ]

'pid' files are written to /var/run directory.

R-GMA configuration for gridview archivers

R-GMA client rpm needs to be installed on the machine. For configuring R-GMA, the following information should be configured in /opt/glite/etc/rgma/rgma.conf.

Consumer=https://monb001.cern.ch:8443/R-GMA/ConsumerServlet
PrimaryProducer=https://monb001.cern.ch:8443/R-GMA/PrimaryProducerServlet
SecondaryProducer=https://monb001.cern.ch:8443/R-GMA/SecondaryProducerServlet
OnDemandProducer=https://monb001.cern.ch:8443/R-GMA/OnDemandProducerServlet
Registry=https://lcgic01.gridpp.rl.ac.uk:8443/R-GMA/RegistryServlet
Schema=https://lcgic01.gridpp.rl.ac.uk:8443/R-GMA/SchemaServlet

Gridview archiver configuration files for R-GMA:

The following configuration files and their entries are required for configuring the R-GMA archivers:

/opt/gridview/etc/gridview-common.conf

gridview.archiver.common.rgmaarch_props = /opt/gridview/etc/rgmaarch.conf

The above entry points to the file containing configuration for r-gma archivers.

/opt/gridview/etc/rgma-authentication.conf

This file contains following lines and is used for authenticating the archiver to R-GMA. This is done by means of grid certificates. This file contains information about locating grid certificates and keys for authenticating into R-GMA. A sample file is shown below:

sslCertFile = /opt/gridview/certificates/hostcert.pem
sslKey = /opt/gridview/certificates/hostkey.pem
sslKeyPasswd =
crlFiles = /etc/grid-security/certificates/*.r0
sslCAFiles = /etc/grid-security/certificates/*.0

The certificate and key files should be readable by user 'gridview' under whose credentials, the R-GMA archivers run.

/opt/gridview/etc/rgmaarch.conf

This file contains configuration specific to the Gridview R-GMA archivers. A sample file is given below:

gridftpmonitor.handler=gview.arch.GridftpHandler
jobstatusraw.handler=gview.arch.JobStatusHandler
jobmonitor.handler=gview.arch.JobMonitorHandler
cmr.props=/var/run/gridview/consumer.properties

Each line in the above file is an entry specifying the java classes which handle each different archiver. The first line 'gridftpmonitor.handler' is the configuration line for the data transfer archiver. The next two lines are configuration for job status archivers. The last line contains the path of the file where the R-GMA client stores its state information.

Web Service Archiver Configuration

Gridview Web Service Archivers run under Apache Axis which in turn runs under Tomcat. Essentially starting and stopping a web service archivers is just starting and stopping the tomcat service. There are two web service archivers for Gridview:

1. Data transfer archiver
2. Job status archiver

Gridview web service archiver configuration files

The following configuration files are required for configuring web service archivers under Tomcat.

/opt/gridview/gridview-common.conf

gridview.archiver.common.wsarch_props = /opt/gridview/etc/wsarch.conf

The above entry just points to the location of the web service archiver config file.

/opt/gridview/etc/wsarch.conf This file contains specific settings for the different web service archivers. The format is similar to the R-GMA archiver and a sample file is given below:

WSDumpLogFlag=1
WSDumpLogDir=/opt/gridview/logs/ws_dump
gridftpmonitor.handler=gview.arch.GridftpHandler
gridftpmonitor.columns=19
jobstatusraw.handler=gview.arch.JobStatusHandler
jobstatusraw.columns=14

The first line decides whether the archiver should dump the data of every tuple received from the clients (0=off, 1=on). The 'WSDumpLogDir' directory points to the directory where these dumps are stored. The remaining lines gives the mapping between the table handler name and the java class which implements the handler.

Gridview Operations

This section describes how to start and stop Gridview services on each machine, the processes to be monitored and the log files to look out for.

Gridview Frontend

The Gridview frontend module works under Apache web server. Therefore the operation is similar to a typical web server machine.

Starting and stopping the service

/sbin/service httpd start
/sbin/service httpd stop

Processes to be monitored

httpd

Log files to be watched

/var/log/httpd/access_log
/var/log/httpd/error_log
/var/log/httpd/ssl_access_log
/var/log/httpd/ssl_error_log

Gridview Summarizer

The Gridview summarizer runs as cron jobs controlled by files in /etc/cron.d.

cron job files

/etc/cron.d/gridview-summarizer-*

Log files

/var/log/gridview/gridview-summarizer-*.log

Gridview Synchronizer

The Gridview synchronizer runs as cron jobs controlled by files in /etc/cron.d.

cron job files

/etc/cron.d/gridview-synchronizer-*

Log files

/var/log/gridview/gridview-synchronizer-*.log

Gridview Archiver

Gridview R-GMA Archiver

The Gridview R-GMA archivers runs as independent services under the init.d subsystem.

Starting and stopping the services

/sbin/service gridview-archiver-rgma-dt start           # To start R-GMA data transfer archiver
/sbin/service gridview-archiver-rgma-dt stop           # To stop R-GMA data transfer archiver

/sbin/service gridview-archiver-rgma-js start           # To start R-GMA job status archiver
/sbin/service gridview-archiver-rgma-js stop           # To stop R-GMA job status archiver

Processes to be monitored

java

Log files

/var/log/gridview/gridview-archiver-*.log

Web service archivers

The web service archivers run under Tomcat.

Starting and stopping the services

/sbin/service tomcat5 start
/sbin/service tomcat5 stop

Process to be monitored

java

Log files

/var/log/gridview/gridview-archiver-wsarch.log