Re: [Summary-Talk] Incremental Processing Questions

[Jason Linhart <jason@xxxxxxxxxxx>]

On 1/14/04 9:35 AM John May (jmaymailing@pointinspace.com) wrote:

>- Is there any way to automate rebuilding the report file, instead of 
>manually turning off and on incremental processing?

That depends on what you are actually trying to do. In normal usage you 
should not have to be turning incremental on and off. Perhaps what you 
want is "Incremental every hour, regular on schedule", an option on the 
Time Units configuration page. That will reset the database according to 
the schedule you configure.

>- If the initial run is set to limit to the past 60 days, then 
>incremental is enabled, what happens with the older log lines?

First, nothing that is in the database before incremental mode is turned 
on has any effect. Summary starts over at the moment when incremental 
mode is turned on.

Second, incremental mode honors starting dates, but not ending dates. So 
if you have Summary configured to report on the last sixty days then it 
will start reading log entries sixty days before the moment that 
incremental mode was turned on and then keep adding in everything after 
that.

>- We roll our logs weekly into a .gzipped file.  That said, when the 
>log rolls will incremental pprocessing double-count hits since this 
>may appear as a "new" file to it?  Or will it only use newer log 
>lines than the last in memory?  And will it recognize that the 
>access_log file is now different (since it was emptied when rolled)?

Summary will properly track the access_log file, noticing when it has 
been reset. It can also handle log rolling, where the log file gets 
renamed, and correctly read the last few log entries out of the rolled 
log. It will not, however, properly deal with rolled logs that are 
compressed when they are rolled. They will not look like the uncompressed 
version of themselves and will be treated as a new file, resulting in 
double counting.

One good approach is to only compress the logs the second time they are 
rolled, that is roll it once uncompressed and then at the next log 
rolling interval compress it. Then you only show Summary the current log 
and the previous log and everything will be fine. Another approach is to 
never show Summary the current log file. Then it will only see compressed 
logs and again everything will be fine. Another approach is to use 
"Incremental every hour, regular on schedule" and set the schedule to 
reread everything shortly after the logs are rolled.

Jason

-----------------
Jason@Summary.Net
-----------------
Dr. Seuss books . . . can be read and enjoyed on several levels. For
example, 'One Fish Two Fish, Red Fish Blue Fish' can be deconstructed
as a searing indictment of the narrow-minded binary counting system.
  -- Peter van der Linden, Expert C Programming, Deep C Secrets
-------------
Go to <http://summary.net/list.html> to update subscription info.