LogsStatisticsPlugin

Introduction

The LogsStatisticsPlugin handles the tag STATISTICSLOGS that inserts a summary of data from the TWiki log files based on user criteria.

Besides standard statistical usage the LogsStatisticsPlugin can be used to identify problems with oversize numbers of requests or to present usage statistics based on TWiki logs.

Syntax Rules

%STATISTICSLOGS{...}%

date

today

today

date="today"

yesterday

date="yesterday"

month

date="month"

last X days

date="last 13 days"

dd MMM yyy

date="01 Jun 2010"

dd MMM yyy - dd MMM yyyy

date="31 Mar 2010 - 01 Jun 2010"

type

top5 users

topX (users|webs|topics|crawlers)

type="top10 webs"

unique (users|webs|topics|crawlers)

type="unique webs"

output

table

table

output="table"

list

output="list"

description

output="table, description"

It is also possible to narrow data to the records containing specific values. Different parameters of the same type should be divided with commas, spaces will be automaticly deleted.

user

user="WikiName"
user="WikiName1, WikiName2"

web

web="SandBox"
web="SandBox, Main"

topic

topic="WebHome"
topic="WebHome, TestPage"

action

action="view"
action="view, edit"

ip

ip="127.0.0.1"
ip="127.0.0.1, 192.168.1.1"

Notes and hints:

• The more parameters, the slower it will work.
• If the summary period covers the current day then the result data may vary every time when the LogsStatisticsPlugin is being called, this is because the log files are continuously updated.
• One can define a period covering future dates, but the plugins will analyze data only up to the servers date.
• If the log file for a specific month does not exists then the plugin will not report any errors.
• Output parameter should be divided with commas, spaces will be automaticly deleted.
• If the output attribute with the table value is used then one can customize it using for example TablePlugin or ChartPlugin.

  %TABLE{ name="tableAL1" sort="on" tableborder="0"  headerbg="#D5CCB1" headercolor="#666666" databg="#FAF0D4, #F3DFA8" headerrows="2" footerrows="1" }%%STATISTICSLOGS{date="last 10 days" type="top3 users"}%

  or

  %TABLE{ name="tableAL"} }%%STATISTICSLOGS{date="last 10 days" type="top3 webs"}%
  %CHART{ type="bar" name="bar" table="tableAL" data="R2:C2..R4:C2"  legend="R1:C2..R1:C2" xaxis="R2:C1..R4:C1" }%

Example

• Which 5 users are making the most requests today?

  %STATISTICSLOGS{date="today" type="top5 users"}%

• How many unique topics have been viewed in the Atlas web in the current month?

  %STATISTICSLOGS{date="month" type="unique topics" action="view" web="Atlas"}%

• Customizing tables:

  %TABLE{ name="tableAL1" sort="on" tableborder="0"  headerbg="#D5CCB1" headercolor="#666666" databg="#FAF0D4, #F3DFA8" headerrows="2" footerrows="1" }%%STATISTICSLOGS{date="last 10 days" type="top3 users"}%

• Creating charts from reports

  %TABLE{ name="tableAL" }%%STATISTICSLOGS{date="last 10 days" type="top3 webs"}%
  
  %CHART{ type="bar" name="bar" table="tableAL" data="R2:C2..R4:C2"  legend="R1:C2..R1:C2" xaxis="R2:C1..R4:C1" }%

Plugin Settings

Plugin settings are stored as preferences variables. To reference a plugin setting write %<plugin>_<setting>%, for example, %LOGSSTATISTICSPLUGIN_SHORTDESCRIPTION%

• One line description, shown in the TextFormattingRules topic:
  • Set SHORTDESCRIPTION = Create statistics from the log files

• Set DEBUG to 1 to get debug messages in data/debug.txt. Default: 0
  • Set DEBUG = 0

• Set a path to the log files (if no option then by default set to $TWiki::cfg{LogFileName}). Path should containt %DATE%
  • Set LOGSDIRECTORY =

• Set a list of groups and/or users allowed to use the plugin. If the user name and the group name applys for the current user then the user name parameters will be set. Names should be divided with semicolons, spaces will be automaticly deleted. One can specify a list of webs for the user/group UserName: web="Sandox, Main".
  • Set ALLOWUSAGE = TWikiAdminGroup; TWikiSupport

• Set to yes if you want the plugin to cache the results for queries which don't contain server's current date. The load generated by the plugin will be significantly reduced, if this option is enabled then the plugin will need to analyze log files only once for each unique query (which doesn't include the server's current date).
  • Set CACHEOLDRESULTS = yes

• Set to all or months if you want the plugin to cache the results for queries which contains a server's current date. The load generated by the plugin will be significantly reduced. Cache stored in this way can be reused for the same queries for different time periods, if this option is enabled then the plugin will need to analyze every time only the current month's log files. This option can enable caching results in two different ways.
  • If the option months is enabled then the plugin will store results for all the past months.
  • If the option all is enabled then the plugin will store results for all the past months and every day of the current months besides the current server's date.
  • Set CACHECURRENTDATERESULTS = all

• Set a directory where the plugin can store it's cache files (if no option then by default set to the plugin's working area). Files will be written only if caching is enabled
  • Set CACHEDIR =

Plugin Installation Instructions

• Download the ZIP file from the Plugin web
• Unzip the zip file in your root ($TWIKI_ROOT) directory.
• Visit configure in your TWiki installation, and enable the plugin in the {Plugins} section.

Notes and hints

• If any manual changes were made to the server's log files (editing, adding, deleting log files) and if caching results is enabled then the administrator should remove all the files from the plugin's working area.
• It is advisable to remove daily files (starting wit letter 'd') from the plugin's working area at the beginning of month. Daily files from the previous month will not be used anymore.

Plugin Info

Plugin Author:	TWiki:Main/ZbigniewGrzybek
Copyright:	© 2010, TWiki:Main/ZbigniewGrzybek
License:	GPL (GNU General Public License)
Plugin Version:	2010-08-11 (V1.000)
Change History:
2010-08-11:	Initial version
TWiki Dependency:	$TWiki::Plugins::VERSION 1.1
CPAN Dependencies:	none
Other Dependencies:	Date::Calc; Storable; Data::Dumper; Date::Manip; List::Util; Time::Local
Perl Version:	5.005
Benchmarks:	GoodStyle nn%, FormattedSearch nn%, LogsStatisticsPlugin nn%
Plugin Home:	http://TWiki.org/cgi-bin/view/Plugins/LogsStatisticsPlugin
Feedback:	http://TWiki.org/cgi-bin/view/Plugins/LogsStatisticsPluginDev
Appraisal:	http://TWiki.org/cgi-bin/view/Plugins/LogsStatisticsPluginAppraisal

Related Topics: TWikiPlugins, DeveloperDocumentationCategory, AdminDocumentationCategory, TWikiPreferences