Installing frontier-tomcat

The frontier-tomcat package is intended for use on a central Frontier "launchpad" server. This page has instructions for installing it.

This is what is on this page:

Hardware

The hardware requirements depend on the application. In general, a typical 2-core virtual machine is sufficient for an LHC Conditions data server. However, if there are many clients that request different database queries at around the same time, more cores may be needed in order to keep up. The bottleneck tends to be the CPU, not the network or memory. Very little disk access is done.

Software

The instructions below are for the latest version of the frontier-tomcat rpm on a Scientific Linux version 6 based system. The rpm includes the standard Apache Tomcat distribution, an Oracle JDBC interface library, a !MySQL JDBC interface library, and the Frontier servlet. Please see the Frontier servlet Release Notes and rpm Release Notes for details on what has changed in recent versions.

Preparation

1. If you want to be able to support digital signature security on the frontier responses, make sure the server has an unexpired grid host certificate and key in /etc/grid-security. If there are any DNS aliases that the server will be operating under, they will need to be listed as either the primary name in the subject of the certificate or one of the subject alternate names.
2. By default the user id that that the tomcat process will run under is "tomcat". This can be changed by creating a file /etc/tomcat/tomcat.conf before installing the frontier-tomcat rpm and putting in the following line with the user id you choose, for example:

       export FRONTIER_USER=dbfrontier
   

3. It is recommended to pre-create the above user id, but if it doesn't exist the rpm will create the user id along with a matching group id.
4. Make sure that email sent to the chosen user id will be forwarded to the person or people who administer the frontier software.
5. By default, the logs for the tomcat process go in /var/log/tomcat. They can get quite large (CMS frontier servers typically use about 25GB) and might not fit in /var, so if /var has limited space it may be best to put them somewhere else. If you want to change the directory that the logs will be in, add the following to /etc/tomcat/tomcat.conf with the directory you want, for example:

       export FRONTIER_TOMCAT_LOGS=/data/tomcat_logs
   

   The directory will be created if it is missing but the parent directory has to exist.

If you are going to be configuring frontier-tomcat to read from an Oracle database, do these additional steps of preparation:

1. Make sure that the Oracle database descriptor(s) are available at /etc/tnsnames.ora and make sure that all the descriptors that will be used by Frontier servlets include the option (ENABLE=BROKEN). Without this option the Frontier servlet will not always be able to recover from database node failures. Here's an example of a complete tnsnames.ora entry:

   cms_orcon_adg=(
           DESCRIPTION=
           (ADDRESS=(PROTOCOL=TCP)(HOST=cmsonradg1-s.cern.ch) (PORT=10121))
           (ADDRESS=(PROTOCOL=TCP)(HOST=cmsonradg2-s.cern.ch) (PORT=10121))
           (ENABLE=BROKEN)
           (CONNECT_DATA=
                   (SERVER=DEDICATED)
                   (SERVICE_NAME=cms_orcon_adg.cern.ch)
           )
   )
   

2. In order to avoid using up all the shared memory over time, a PL/SQL script like the following needs to be run in the Oracle database for each username used by frontier:

   create or replace trigger cursor_sharing_USERNAME
   AFTER LOGON ON USERNAME.SCHEMA
   BEGIN
           execute immediate 'alter session set "cursor_sharing"=FORCE';
   END;
   /
   

   Replace both occurrences of USERNAME with the specific username.
3. In order to efficiently support caching, Frontier needs to have table modification times stored in a special table. Here is an example dbms_scheduler script to accomplish that. Have your database administrator ask phydb.support@cernNOSPAMPLEASE.ch if they need any help with it.

Installation

First, if you have not installed any frontier rpm before, execute the following command as the root user:

    # rpm -Uvh --replacefiles http://frontier.cern.ch/dist/rpms/RPMS/noarch/frontier-release-1.2-1.noarch.rpm

(If you are not upgrading a previous installation done without the frontier-release package you may leave out --replacefiles).

Install frontier-tomcat and its dependencies with the following command:

    # yum install frontier-tomcat

Enable starting automatically at boot time:

    # systemctl enable frontier-tomcat

Configuration

The two configuration files that still need to be set up are /etc/tomcat/servlets.conf and /etc/tomcat/servlets.passwd.

servlets.conf

The main configuration file is /etc/tomcat/servlets.conf. This consists of several sections that each begin with a name in square brackets. The first section is

    [defaults]

and it contains the default settings that apply to each servlet if they're not overridden in the specific servlet's section. Lines beginning with '#' and blank lines are ignored.

The following sections begin with servlet names in square brackets. Within each section there can be settings that get translated into different pieces of the underlying tomcat system configuration, as detailed below.

servlets.conf JDBC settings

The following settings become part of the JDBC (which stands for Java DataBase Connectivity) configuration for each servlet. These are put into /usr/share/frontier-tomcat/tomcat/conf/Catalina/localhost/<servlet>.xml where <servlet> is the servlet name. These settings are required when not using FileBaseDirectory:

• tnsName: the name of an oracle database as defined by a connection string name in a tnsnames.ora file. See the useCopyOfTnsnames setting below for details about how to find the file.
• mysqlHost: the name of the host running a MySQL database server. One and only one of tnsName or mysqlHost is required when not using FileBaseDirectory.
• username: the username of the database account.
• maxDbConnections: the number of parallel connections to the database allowed for this servlet.

servlets.conf Servlet settings

The following settings are directly for the frontier servlet. They are put into /usr/share/frontier-tomcat/tomcat/webapps/<servlet>/WEB-INF/classes/config.properties.

• LongCacheExpireSeconds (required): the number of seconds that most responses should be cached before they expire.
• ShortCacheExpireSeconds (required): the number of seconds until cache expiration for the responses marked by the application to expire in a relatively "short" amount of time. Often this time is set to be the same number of seconds as LongCacheExpireSeconds.
• ForeverCacheExpireSeconds (optional): the number of seconds until cache expiration for the responses marked by the application to never expire with the "forever" time-to-live. The default is a year of seconds.
• ValidateLastModifiedSeconds (optional): enables "If-Modified-Since" processing and specifies the number of seconds that the frontier servlet should cache modification timestamps. A typical value is 300 for 5 minutes.
• LastModifiedTableName (required if ValidateLastModifiedSeconds is set and using oracle): the database account and table name that stores table modification times, for example CMS_FRONTIER_MODS.LAST_MODIFIED.
• ExpireEmptyQueriesLikeErrors (optional): when set to true, sets the cache expiration time on empty queries (0 rows) to be as short as errors, which is the lesser of 5 minutes and ShortCacheExpireSeconds. Default is false.
• FileBaseDirectory (optional): path to a directory of files or an http:// prefix to serve through the servlet instead of serving database queries. If this is set, it doesn't make sense to specify JDBC settings or the ValidateLastModifiedSeconds setting. http:// prefixes should end in slash (/).
• MaxFileConnections (default 5): the maximum number of simultaneous file or http connections allowed before queuing requests. Applies only when FileBaseDirectory is set.
• MaxThreads (default 100): the maximum number of parallel request threads that may exist (that is, the number of active connections to the back end plus the number of queued requests). If the number goes beyond this value and a connection needs to be acquired, the servlet rejects the request with a SERVICE_UNAVAILABLE code instead of queuing the request.
• MaxDbAcquireSeconds (default 300): the maximum number of seconds that the servlet will wait to acquire a connection to the database, file, or http server. During this period the servlet will send a keepalive message every 5 seconds to keep the client from timing out.
• MaxDbExecuteSeconds (default 10): the maximum number of seconds that the servlet will wait for the database or http server to begin to respond to a query. While waiting, the servlet will send a keepalive message every 5 seconds to keep the client from timing out. The total time allowed for the response then is this time plus the client read timeout (which is also typically 10 seconds). For http servers that do not supply a Content-Length header, this time includes reading the entire response, not just the beginning.
• CertFileName (optional): path to the host certificate readable by the frontier user. If this is not set, the default is to copy it from /etc/grid-security/hostcert.pem to /etc/tomcat/hostcert.pem at start time, make the copy owned by the frontier user, and set this parameter to /etc/tomcat/hostcert.pem.
• KeyFileName (optional): path to the host key readable by the frontier user. If this is not set, the default is to copy it from /etc/grid-security/hostkey.pem to /etc/tomcat/hostcert.pem at start time, make the copy owned by the frontier user, and set this parameter to /etc/tomcat/hostkey.pem.

servlets.conf Other setting

The following setting can only be in the [defaults] section:

• useCopyOfTnsnames (optional): By default the Oracle database description file that frontier-tomcat uses is in /etc/tnsnames.ora. If that is a file that can be updated without the intervention of the frontier software administrator, set this option to a value of "1". This will make a copy of /etc/tnsames.ora in /etc/tomcat/tnsnames.ora, tell frontier-tomcat to read from that file instead, and install a daily cron job to compare the two. If there is a difference between the two files, the cron job will email a warning to the user id that frontier-tomcat runs under.

servlets.conf Example

Here is an example servlets.conf:

[defaults]
useCopyOfTnsnames: 1
username: CMS_COND_FRONTIER_R
maxDbConnections: 2
LastModifiedTableName: CMS_FRONTIER_MODS.LAST_MODIFIED

[FrontierPrep]
tnsName: CMS_ORCOFF_PREP
ValidateLastModifiedSeconds: 300
LongCacheExpireSeconds: 1200
ShortCacheExpireSeconds: 300

[FrontierProd]
tnsName: cms_orcon_adg
maxDbConnections: 5
ValidateLastModifiedSeconds: 300
MaxDbExecuteSeconds: 30
MaxThreads: 200
LongCacheExpireSeconds: 21000
ShortCacheExpireSeconds: 21000

[smallfiles]
FileBaseDirectory: /data/fileserve/smallfiles
LongCacheExpireSeconds: 21600
ShortCacheExpireSeconds: 21600

servlets.passwd

Every servlet that connects to a database needs a database password. These come from /etc/tomcat/servlets.passwd and are written into the same JDBC configuration file as the servlets.conf JDBC settings described above. The file must be owned by the user id that tomcat is running under and be mode 600. The format of the file is sections beginning with a name in square brackets followed by the setting name password: followed by the password value. The name in square brackets may be either a tnsName or a username matching the servlets.conf JDBC settings. If a tnsName matches, it takes priority, otherwise the username is used. For example:

[CMS_COND_FRONTIER_R]
password: a_hard_to_guess_password
[cms_orcon_adg]
password: a_harder_to_guess_password

This will use the second password for the FrontierProd servlet in the servlets.conf example above and the first password for the FrontierPrep servlet.

Changing the port number

The port number that tomcat listens on can be changed from the default 8080 by setting FRONTIER_TOMCAT_PORT in /etc/tomcat/tomcat.conf. For example:

  export FRONTIER_TOMCAT_PORT=8000

Testing

Start the tomcat process as root with the command

    # systemctl start frontier-tomcat

Test it using a similar procedure as the frontier-squid test procedure except don't set http_proxy, change cmsfrontier.cern.ch to the name of your server, change 8000 to 8080, and change FrontierProd to the name(s) you chose for your servlet(s).

Debugging

A lot of debugging information is written to the log file catalina.out in the log file directory. The default log directory is /data/tomcat_logs unless you changed it as described in the Preparation section above. This log records events and conditions that affect Tomcat's operation and also interactions between the Tomcat server and the client.

For details on common errors and messages in the log, see Catalina.out messages.