Re: [Summary-Talk] Incremental config lockdown

[Jason Linhart <jason@xxxxxxxxxxx>]

The brief answer is to compress and then keep your log files. The
largest installs of Summary I know about report on approximately
1,000,000,000 log entries. Log entries with most of the optional fields
average perhaps 250 characters each. BZip2 can compress log files by
about a factor of 30. 250 * 1,000,000,000 / 30 = 7.76 Gig, which is
easily within the capacity of modern hard disks.

Then you run in incremental mode. When you need to make a change that is
locked in incremental mode, turn off incremental mode, make the change,
and turn incremental mode back on again. Even with really large log
files, Summary should be able to reprocess all of your log files with
the new configuration overnight (on a reasonably modern machine).

---

Changing settings in incremental mode is possible, by directly editing
the configuration file by hand, but we don't recommend or support doing
so. The result of such changes is that log entries processed before the
change are based on the old settings, and those processed after the
change will be based on the new settings. Do that a couple of times and
there won't be any way to keep track of what your results mean.

If we kept enough information to be able to recalculate what the results
of configuration changes should be retroactively on already processed
log entries then the Summary database would have to be larger than the
original log files. Some of our competition do exactly that. We figure
it is simpler to just keep the old log files and keep Summary's database
relatively small.

---

I can imagine situations where it would be nice to run some sub-reports
in incremental mode and others in regular mode. I will add this request
to our wish list. Of course this is possible now by purchasing and
running two different copies of Summary.

Jason

-- 
Jason@xxxxxxxxxxx
--
Dr. Seuss books . . . can be read and enjoyed on several levels. For
example, 'One Fish Two Fish, Red Fish Blue Fish' can be deconstructed
as a searing indictment of the narrow-minded binary counting system.
   -- Peter van der Linden, Expert C Programming, Deep C Secrets
-------------
Go to <http://summary.net/list.html> to update subscription info.