StatViz - Visual Clickstream Analysis

Goals

StatViz is a simple program to help visualize the way visitors use your web site. This is sometimes called clickstream or path analysis. Unlike traditional web statistics, the output of StatViz is a graph. The file format of the graph file is DOT. These files can be visualized using GraphViz.

Aggregate Clickstream Analysis ("Pairs" report)

Example of Aggregate Clickstream Analysis (Referrer -> Hit "Pairs")

StatViz analyzes the log by creating a histogram of referrer->hit "pairs". The most popular of these pairs are then expressed as a graph. The graph is colorized and labelled to aid in interpretation.

StatViz "pairs" graph explained: in a nutshell, run the program, and look at the output. The stuff that catches your eye is how most people are moving to and through your site.

• The graph shows movement through your site as links from one page to another. Each node is a web page, each line is a "click" from one page to the other as indicated by the arrow.
• The lines are color coded based on the relative popularity of that "click". Popular clicks are shown in red, less popular pages in blue. The number of times that click occurred is shown next to the line.
• Pages that aren't on your site (external referrers) are shown as brown boxes. The relative frequency of referrals from that is indicated by the darkness of the brown. The darker the box, the more referrals come from that page.
• Pages on your site are shown as ovals. Once again, these are color-coded by relative frequency of "hits". More popular pages are red, less popular pages are blue.
• Each page also shows the number of times the page was a referrer to another page, and the number of hits that page got.

Individual Session Tracks

Example of Individual Session Tracks

StatViz keeps track of the movements of each individual session in the logfile. The way the session is determined is configurable. By default it now simply uses the IP address. However, you can easily configure StatViz to use a different column as the "unique session ID", for instance from mod_usertrack or a custom session id logged via the Apache notes mechanism. The session track reporter will graph the exact clickstream for the longest (by clicks not time) sessions in the log. This will result in several graphs, and is designed to give a good sampling of how visitors move around the site. If the "Pairs" report above is the 50,000 foot view, this is the on-the-streets view.

The individual session tracks allow you to really understand how people successfully or unsuccessfully navigate through your site. Studying them is a great way to understand how you could better present information on your site to make your visitors get what they want more efficiently.

StatViz "session tracks" graph explained:

• The graph shows movement through your site as links from one page to another. Each node is a web page, each solid line is a "click" from one page to the other as indicated by the arrow.
• Each line has a number next to it representing which "click" it is in the session track. The time of the "click" is also shown.
• In some cases you will also see a dashed line with the same number as a click. These "BACK" links indicate that the visitor went back to that page before proceeding.
• Pages that aren't on your site (external referrers) are shown as brown ovals.
• The "entry" page is colored green. The "exit" page is colored red. If the entry and exit pages are the same, that page will be red.

Live Analysis

StatViz can actually show you these reports LIVE... If you configure the logfile to "php://stdin" then StatViz will read the log lines from STDIN. It will then periodically generate reports as the log is processed, thus allowing you to watch your traffic patterns in real-time. I am not sure how useful this is yet, but it's definitely fun!

Setting up live mode is quite simple. Let's say for instance you create a config to analyze your log files normally (offline) and use the following command to process the logs:

statviz.php --config mydomain.conf

To switch to live view, simply do the following:

tail -f /path/to/mylogfile.log | statviz.php --config mydomain.conf --continuous=20 --logfile=php://stdin

You can even analyze the logfiles remotely with a command like:

ssh -C user@domain.com "tail -f /path/to/mylogfile.log" | statviz.php --config mydomain.conf --continuous=20 --logfile=php://stdin

I am not sure about all of the GraphViz apps, but the one for Mac OS X actually will automatically redraw as soon as the DOT file changes, so in this setup you can just sit back and watch your site traffic live.

License

StatViz is licensed under the MIT open source license.

Links and Resources

• GraphViz home page.
• Apache's mod_usertrack for generating unique session ID's.
• Pathalizer is similar tool, also hosted on SourceForge.

Well, that's about all we've got for now! You can DOWNLOAD the software from our SourceForge StatViz Project Page.