[Summary-Talk] Regular kernel panic in Mac OS 10.2

[John Burwell <john.burwell.lists@xxxxxxxxxxxx>]

Our Summary machine crashes almost weekly, and I'm running out of 
ideas. It'll run for about six days before it crashes with a kernel 
panic. I'm trying to decide if, based on our setup: are we asking 
Summary to do too much, is there an issue with software or hardware, 
are our log files broken, or what.

If anyone willing to look over some details could give me some feedback 
or ideas, I'd appreciate it. Here are some details:

It's Summary SP 2.3.10, Mac OS 10.2.8, on a graphite dual G4 500 with 
1.5 GB RAM.

At first, I was concerned about vnodes and RAM size, and we upgraded 
the RAM from 768 MB to 1.5 GB. All the RAM is the same Kingston DIMMs. 
Upgrading the RAM relieved some vnode pressure, but it didn't stop the 
crashes.

I fiddled with Summary's options a bit to reduce the complexity of the 
processing, but I couldn't find anything that substantially lightened 
things up. In any case, we're starting to need the more complicated 
reports for some of our sites, so I'm going to want total freedom on 
this.

The log files are all in Apache's basic log format, no referrers or 
user-agent logging -- though we do enable and disable that in apache 
for various sub reports when we want to sample the browsers. They are 
currently not gzip-compressed at any point, but they do roll once a day.

We process incrementally on the hour, and rarely but sometimes it takes 
more than an hour to process. As you can see below, it takes the better 
part of a day to do a complete run. We don't require lightning speed on 
this though... it can take as long as it wants as long as it gets it 
done. The full processing is scheduled to begin on Sunday night in time 
to be ready Monday morning. The crashes almost always occur on Thursday 
or Friday.

We are running 25 sub-reports, and we add more on a slow but regular 
basis as the company grows.

Here's a look at our program stats:

Log files                     9,754
Log file lines          290,407,541
Corrupt log lines             3,921
Skipped log lines         1,075,198
Valid log entries       289,328,423
Filtered log entries    178,113,816
Not in any report           140,556
Log size in MBytes           25,294M
Logs on disk in MBytes       34,746M
Time to crunch             15:17:17
Log processing                 27.6 Meg/min
Log processing                5,257 lines/sec

Memory used                 133,888K
Physical memory           1,572,864K


I want to do all I can to keep Summary happy on this machine, but I 
want to evaluate all my options. I'd have thought this would be well 
within Summary's capacity to handle, but as the frequency of the 
crashing seemed to increase with the size of our logs, I'm beginning to 
wonder if we're overloading it. I've also put in as much RAM as we can, 
and Summary's not even using it all.

Thoughts?

j
-------------
Go to <http://summary.net/list.html> to update subscription info.