Re: [Summary-Talk] Multiple Log File Locations?

[Jason Linhart <jason@xxxxxxxxxxx>]

On 4/9/04 9:09 PM Lyn (lynlist@nowdata.com) wrote:

>Are you saying that Summary will analyze compressed logs during it's 
>nightly run?

Yes. Summary will automatically read logs which are compressed with Zip, 
GZip, or BZip2.

>Does it first have to unzip them? (I think I recall 
>reading this in the documentation, but your explanation would be 
>appreciated.)

No, compressed logs are not expanded on disk. Summary expands the logs 
into memory bit by bit as it goes, without requiring any additional disk 
space. It is actually faster to read Zipped or GZipped logs from disk 
than uncompressed logs because the expansion time is shorter than the 
extra disk I/O time for an uncompressed log.

There is one limitation with log compression. If you are running Summary 
in incremental mode you don't want to give it an uncompressed log and 
subsequently give it the same log file compressed. Summary will think 
that they are two different log files and count them twice. Also, if you 
move a log file to a different folder, Summary will read and count it 
again. This is only an issue in incremental mode.

Of course, in incremental mode this isn't really much of an issue because 
you can compress the log files and put them somewhere that Summary won't 
see them. Summary will retain the information from reading the 
uncompressed copy of the log, even after the file is gone, and you will 
have the compressed copy as an emergency backup.

Zip and GZip compress most logs to about 1/10th of their original size or 
better. BZip2 is significantly slower, but it compresses things to more 
like 1/20th or 1/40th of their original size. I have seen BZip2 compress 
some logs to 1/100th of their original size. Actual compression results 
vary by log file.

Jason

-----------------
Jason@Summary.Net
-----------------
Dr. Seuss books . . . can be read and enjoyed on several levels. For
example, 'One Fish Two Fish, Red Fish Blue Fish' can be deconstructed
as a searing indictment of the narrow-minded binary counting system.
  -- Peter van der Linden, Expert C Programming, Deep C Secrets
-------------
Go to <http://summary.net/list.html> to update subscription info.