Much of the challenge in sizing and planning Centralized Log Management (CLM), Security Intelligence Systems and Security Information and Event Management (SIEM) solutions is determining an adequate amount of storage for storing logs for real-time analysis and archiving the log data to meet long-term retention requirements. The biggest challenge most customers face is determining the required metrics needed in sizing a solution. My post “Basic Log Storage Calculations” http://www.buzzcircuit.com/?p=208 can assist in the sizing and variables needed and my post “Guessing Game – Planning & Sizing SIEM Based on EPS” http://www.buzzcircuit.com/?p=231 can help with guessing the EPS averages for each device types. Finally, I have a couple of cool calculators at http://www.buzzcircuit.com/?p=408 and http://www.buzzcircuit.com/?p=378 that can actually assist with the final calculations.

At this point you have probably guessed that log storage calculations and storage planning is somewhat of an art, rather than a science – there’s a lot of guesswork involved, especially if you don’t have access to the systems or network devices hosting the logs. While I have done a good job (I think) in helping you dispel some of the myths and “guesstimating” an overall log capacity in previous posts, one area that is often overlooked in planning log management or SIEM is the concept of Normal EPS (NE) vs Peak EPS (PE) and ensuring your daily calculation provide a necessary contingency for consistent peaks in your event logging throughout the day.

Normal vs Peak Logging

There are two basic calculations when combining normal + peak EPS, which by no means is a hard rule. The idea is that there is the NUMBER_OF_PEAKS multiplied by the DURATION_OF_EACH_PEAK, which is then multiplied by the DEVIATION_FACTOR. To describe each of these points:

Hope you enjoy!