Summary

Web Analytics Tutorial

 

Lesson 12 – Investigating Troublemakers

IN THIS LESSON
* Unusual Access Patterns
   Drilling Down
* Bad Robots
   Denial of Service Attacks
   Worms
   Content Mirroring
* Digging Deeper
* What to Do About Troublemakers
   Responsible Parties
   Validity of Information
   Counter Measures
   Limiting Robots
   Limiting Mirroring Tools

Digging Deeper

In the previous sections we discussed how to use subreports and filters to create investigative reports for analyzing traffic from troublemakers. In order to apply most of these filters to subreports you need to have Summary SP. In Summary and Summary Plus, you can only apply filters on a global basis and you may not want to interrupt your reports to do this. Even if you do have Summary SP, the sheer volume of data in web logs can still lead to lengthy reports, especially the Host Report. Fortunately, Summary provides some tools to help you search the reports and logs to find information that may be buried. Using these searches allows you dig deeply into the data in your logs and find patterns that may not be immediately apparent in the greater summaries.

Figure 9. Sample Agent Report
Figure 9. You can search the
Agent Report for a particular
pattern, like “wget.”
All editions of Summary include search fields in some reports. For example, you can use the search field in the Agent Report to find all user agents that contain the phrase ‘wget.’ Summary’s search is case-insensitive, so that search will bring up matches like the ones in Figure 9. You can use this same search technique to find particular requests in any of the content reports. For example, to find if a Nimda worm has accessed your server, you could search the All Requests report for “cmd.exe.” To see what might be possible failed intrusion attempts, your could search the Failed Request report for “.exe”.

Sometimes you really need to dig into the logs themselves and find specific requests that match particular patterns. You could do this with a text editor or with at Unix tool such as ‘grep,’ but that could take a significant amount of time. Summary Plus and SP include a Search Logs tool that can do this for you, across all logs, relatively quickly. Using the Search Logs tool you can enter a pattern match for the host, requested file, server (if you run reports for several web sites) or user name. The results will show you the exact time and date of each matching request. When analyzing unusual access patterns from particular hosts, this can help give you insight into the behavior of the individual or robots trying to enter your site.



Table of Contents | 1: What is Web Analytics? | 2: Where are My Visitors Coming From? | 3: Search Engines | 4: Advertising | 5: Revenue Modeling | 6: Design Considerations | 7: Determining Visitor Behavior Patterns | 8: Examining Subsets of Traffic  | 9: Incorporating Business Goals | 10: Bandwidth Management | 11: Site and Server Diagnostics | 12: Investigating Troublemakers | Appendix A: Making Reports More Usable | Appendix B: Technical Details of Metric Accuracy

Copyright 2002 by Summary.Net - Updated 16.Apr.2002