![]() Sudo aptitude install libgeo-ipfree-perl Step 2 : Configure Squid Logging The first step was just to install AWStats on Ubuntu using these apt commands (note: the last 2 are only needed if you want to track stats by country): So my plan was to end up with something like this: ![]() My goal was to get this AWStats set up on my Ubuntu box to read the Squid logs and provide some nice HTML output - updated on a regular basis so I could monitor usage through the day Looking at the various log analysis tools available - I decided to give AWStats a try - it's free, seems well documented and commonly used This is the only place that knows about all the traffic (as this will log all access, whether delivered from the cache, or from the backend servers)Īlthough Squid provides the cachemgr tool - this is more geared to monitoring cache access, than actual detail of what is delivered, where and to who (which is the kind of info I'm more interested in) However, the mobile apps access the web services directly - so a client based tracker like Analytics cannot track these requests.Īs all requests are routed through Squid - all access data sits in the squid 'access.log'. I can track website visitors fine using Google Analytics. What I want to do is be able to track visitors across all web sites and mobile apps. The Problem - How to Analyse Web Service Traffic? So my basic setup is similar to this diagram: This caches certain html, image, json, php, etc content (as defined in /etc/squid3/nf) - and if it can't find the request in the cache (or has been told not to cache it), will hand off to one of the upstream servers to service the request - Tomcat (on port 8080 - serving my Java web services, and admin App) or Apache (on port 9898 - serving my web sites, PHP/HTML). There is a complexity overhead, but part of the reason for doing this is to play with the technologies - so thats all part of the fun!Īt the moment, all this is running on a single EC2 Ubuntu server instance.Īll HTTP traffic comes into the Squid server, running on Port 80. This lets me run a pretty mixed environment, which suits my current purposes for delivering speedy content, removes load on my main app server and also doesn't tie me to closely to any particular technical implementation in the back end. It also takes more load off the Tomcat server - which is also running scheduled tasks to update the data in the system.) (I'm also caching in the application layer using Ehcache and Spring - but adding the Squid cache in front of this lets me cache more content that just the JSON, and lets me take advantage of Squids proxy functions to manage back end services easily. I'm running a Squid server as a reverse proxy caching in front of the web services, to cache defined content at the HTTP layer. Logrotate is configured as (/etc/logrotate.d/apache2): /var/log/apache2/*.I've got a Java application running on Tomcat on Amazon EC2, which exposes RESTful web services delivering JSON to a variety of mobile apps and web sites. My awstats configuration is: LogFile="/usr/share/awstats/tools/ /var/log/httpd/-sslaccess.log /var/log/httpd/-access.log |" The problem might be caused by the logresolvemerge tool (I have seperate logs for regular and SSL traffic). I assume it is a problem with log rotation but it's hard to tell. To be clear the data was being shown for these days up until the end of the month but then on the next month (not always on the 1st) about 1-2 weeks of data suddenly goes missing from the previous month. I am seeing a problem on my server where awstats is deleting data from previous days.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |