WWWSTAT

Description

wwwstat is a simple program written in C which provides an analysis of the popularity of your web pages.

wwwstat parses access_log files which list the time, date, and domain of each instance a web document is hit.

To use wwwstat, you do not need a web counter, or perl, or java, or cgi-bin access, or anything else. You only need to compile the C program and run it on an access_log file. Access log files are usually world-readable files located in a directory like /www/httpd/logs/ so anyone can get them.

Using wwwstat

Typical access_log files are very large because they contain a complete history of all accesses to a all web pages on a server.

If you are only interested in generating statistics for a particular web page, you should filter out access_log entries which you are interested in, to a separate file, then run www_stat on that file.

If you have never seen an access_log file, it is just a big list of web page hit data listed like this:

155.250.128.1 - - [06/Nov/1997:04:02:14 -0500] "GET /%7Esvei/tifaq/small/fountain.html HTTP/1.0" 200 42740
155.250.128.1 - - [06/Nov/1997:04:02:15 -0500] "GET /%7Esvei/tifaq/small/cirque.gif HTTP/1.0" 200 17736
155.250.128.1 - - [06/Nov/1997:04:02:15 -0500] "GET /%7Esvei/tifaq/small/marquard.html HTTP/1.0" 200 25422

Example: If you only care about hits to "mypage.html", you should filter out lines from your access_log into a file called mypage_log. In UNIX, you can do something like this:

 cat access_log | grep mypage.html > mypage_log

wwwstat can then process this mini-access_log file, and dump the results to a report file:

 wwwstat < mypage_log > mypage_report

If you are unable to filter out an interesting subset of web pages from the access_log file, then wwwstat will simply generate a report for all documents on the entire web server

Reports generated

wwwstat outputs 3 kinds of reports from the log file, and dumps them to stdout:

• Popularity report. This report lists all web files which were listed in the access_log, sorted by popularity. The report simply lists the document's name, and the number of times it was hit
• Unique Users report. This report lists all of the unique domains which hit any of your web pages, sorted by frequency. The report lists the domain along with the number of times someone from that domain hit any of your documents.
• Chronology history report. This report shows a table listing your top 10 most popular web documents, and the number of hits they each received over the history of the log file. Use the "-chrono n" option to change the report to tally every n days. The default is 3, meaning that each row of the table reports 3 days worth of hits for each document. Monthly totals are also listed for each document, so you can see how each page's popularity has varied over time.

Other stuff

• Here is the C src code: wwwstat.c
  
• Here is an example access_log data file: svei_log
• Here is the report generated by wwwstat: svei_report Note that this log only constitutes 1 month of data, so the chrono report isn't very interesting in this example.

Disclaimer: This program is offered "as-is". All binaries and src files are freeware, and be re-used for any purpose. Binaries may be posted here in the future.

• wwwstat has some hard-coded limits on the size of the data file it can process. You can change the #define's in the src to adjust these limits. Currently, it only handles access_log files with less than 150 different documents, and less than 20,000 unique users hitting all documents. You could, however, change these limits to be anything you want.

• wwwstat assumes all hits are already given in chronological order. Therefore you cannot concatenate two greps together, otherwise, the temp file will have two ranges of dates, not in order.

• Changed Chrono report to report #new hits for each period rather than list the running total of hits up to the end of each period.
• Fixed Chrono report monthly summaries
• Fixed compiler warnings