Log Management

AIO WP Security Firewall Log Hacks

BuzzCircuit Lock Logo

Ever since my WP site was hacked and defaced back in 2012 by who claimed to be the “Kuwait Hackers” group, I became especially paranoid of using WP plugins wildly on my site and in fact started installing plugins that would aid in hardening and scanning the site for security issues. I started a routine of checking for plugin updates, backing up my DB and, at first, used .htaccess to blacklist IP addresses that showed up with significant numbers of failed logins or 404 errors. It became a taunting endeavor  and I spent countless hours trying to stay ahead of the bad guys on my site and the three other sites I was managing as sub-domains for customers.

So, when the All-In-One WordPress Security & Firewall plugin became available to the WP community two years ago, I installed it and was amazed at the plethora of security features it offered. The plugin was easy to set up, with clear instructions. I get an email when someone suspicious tries to login to my site or generates too many 404 errors, so I can counter threats fairly proactively. The options I implemented negatively affect any of my performance or other plugins (some of which are additional security plugins). Check out the link or see the video below for details on all the features.

Continue reading


Are you a Security PreSales Ninja?

Security Ninja Quiz

Take this quiz and determine if you are a Security PreSales Ninja.

NOTE: this quiz has a 20 minute time limit to complete.

Enter your full name and email address in the results table to save to the leaderboard!

ScreenHunter_06 Jul. 28 11.07

You must specify a text.
You must specify an email address.

Event Log Convergence = Business Intelligence


I have come across many prospects over the last 6 years that are only trying to acquire a SIEM solution to satisfy a compliance requirement, or what we call in the industry, “check-box purchasing” – they have a minimum set of requirements, specific to only one business unit or compliance mandate that is completely siloed from the rest of the organization.

Here is how this conversation usually goes:

Client: “we would like a SIEM tool that will help us monitor our 200+ Windows Servers for PCI-DSS compliance”

Me: “What other event sources are you going to be monitoring with the solution?”

Client: (Stunned look) “we only need to monitor our servers.”

Me: “PCI-DSS requirement 10 states you have to monitor the logs from all of your security devices and servers that are deemed critical assets.”

Client: “our department is only responsible for the servers we listed.”

Me: “To get value out of a SIEM solution and monitor all 12 PCI requirements you need audit logs from all of your devices and contextual information regarding your network, asset and vulnerability data – and that will just get you started.”

Client: “Perhaps we need to increase the scope – we’ll get back to you.”

While the Centralized Log Management (CLM) and Security Information and Event Management (SIEM) vendors will be lined up around the block to influence the sale, the vendor you choose should be a trusted advisor. They will be interested in providing you the most value from your investment and assist you in designing a solution to satisfy many business problems that goes beyond a traditional security-centric SIEM. This is why you will need to identify key device types and the value that can be derived by cross correlating the log data with business context to align monitoring with your governance, security and compliance initiatives.

The SIEM Value Derived from Heterogeneous Device Logging

While each of the event sources you collect events from will provide distinct reporting and alerting value, combining many different “types” of event sources will derive immediate intelligence about the business and help analysts establish baselines of threat activity. One of the other benefits is that incidents can be prioritized by business value, threat classification and the additional context can help reduce the plethora of false-positives or false-negatives that plagues every CLM solution.

Additionally, the multitude of the various technologies have their own management and reporting solutions that become “silos of information” that only the black-belts responsible for each of the device types are able to decipher. This makes security intelligence and investigations near impossible when an analyst has to request log data from the owners or log in to many different systems to find the evidence to support their cause. Essentially, they would have to piece together the clues and manually normalize the data using a technique called “Spreadsheet Kung Fu”, which would be fraught with assumptions and inference.

Below is a list of different device types and the value that can be derived from each when correlated together. The list isn’t exhaustive and I’m not suggesting you need everyone mentioned to successfully deploy a SIEM, but the more data feeds you can correlate, the more intelligence you will have available in the future to expand and grow with your business (click “more…” for complete article):

Continue reading


Determining Peak EPS Calculations in Logging

Much of the challenge in sizing and planning Centralized Log Management (CLM) and Security Information and Event Management (SIEM) solutions is determining an adequate amount of storage for storing logs for real-time analysis and archiving the log data to meet long-term retention requirements. The biggest challenge most customers face is determining the required metrics needed in sizing a solution. My post “Basic Log Storage Calculations” http://www.buzzcircuit.com/?p=208 can assist in the sizing and variables needed and my post “Guessing Game – Planning & Sizing SIEM Based on EPS” http://www.buzzcircuit.com/?p=231 can help with guessing the EPS averages for each device types. Finally, I have a couple of cool calculators at http://www.buzzcircuit.com/?p=408 and http://www.buzzcircuit.com/?p=378 that can actually assist with the final calculations.

At this point you have probably guessed that log storage calculations and storage planning is somewhat of an art, rather than a science – there’s a lot of guesswork involved, especially if you don’t have access to the systems or network devices hosting the logs. While I have done a good job (I think) in helping you dispel some of the myths and “guesstimating” an overall log capacity in previous posts, one area that is often overlooked in planning log management or SIEM is the concept of Normal EPS (NE) vs Peak EPS (PE) and ensuring your daily calculation provide a necessary contingency for consistent peaks in your event logging throughout the day.

Normal vs Peak Logging

There are two basic calculations when combining normal + peak EPS, which by no means is a hard rule. The idea is that their is the NUMBER_OF_PEAKS multiplied by the DURATION_OF_EACH_PEAK, which is then multiplied by the DEVIATION_FACTOR. To describe each of these points:
  • NUMBER_OF_PEAKS: calculating Peak EPS (PE) is required to factor in Normal EPS with Peaks (expressed as NE+PE) to ensure their is sufficient licensing and storage to accommodate periods of deviation from the normal EPS throughout the day. The default in the calculator below assumes there will be at least 3 peaks a day (morning logins, lunchtime web surfing, evening logoffs/backups). This value will vary based on network throttling, congestion, attacks, etc.
  • DURATION_OF_EACH_PEAK: This setting works in conjunction with the previous PE setting and assumes that each peak lasts for approximately 1 hour (3600 seconds) – this may vary given many factors such as how congested the network is, how busy the logging device is or other scenarios such as DDoS attacks.
  • DEVIATION_FACTOR: is generally 2-5x the average EPS for that period. While in reality the EPS spikes almost 20x the average EPS for only seconds, we are building in contingency for attacks such as perimeter devices under DDoS or excessive IT Operational errors that go unnoticed for hours.  NOTE: again, this is an art, not a science and we’ll sound like we know more than our competitors if we think to include contingency into our calculations!

Hope you enjoy!


NetCerebral’s Device EPS Calculator

Hi folks, this post is another form I created using the Calculated Fields Form plugin for WordPress. Basically, this simple form calculates the number of devices input in the form fields and multiplies the number of devices by the designated Events Per Second (EPS) average for each device type. It then provides a live calculation of total number of devices, total average EPS and total average Events Per Day (EPD).

This handy calculation can then be used on my other calculator NetCerebral’s Simple Log Storage Calculator as the average EPS, need as the primary input to calculate amount of storage and IOPs required for the EPD and retention periods defined.


NetCerebral’s Simple Log Storage Calculator

The following form assumes you have done the preliminary math of determining your number of devices and the total anticipated Events Per Second (EPS) you will be collecting from all of your logging devices. The calculator uses EPS to determine the Events Per Day (EPD), amount of raw and normalized log data you will generate daily and then use retention and compression values you set to determine the required amount of storage as well as IOPs required.

Stay tuned and I will be building another calculator that will allow you to specify number of devices by device type, which will give you your estimated EPS (needed as a starting point for this calculator).


Guessing Game – Planning & Sizing SIEM Based on EPS

Many of the competing log management and SIEM tools on the market these days use some variation 0f the Events Per Second (EPS) metric to determine the licensing, sizing and storage requirements for scalable solution. Unfortunately, none of the devices that are to be monitored have a specification associated with the amount of logging which will be generated per second (or volume for day, for that matter!) by the device. Moreover, many of the same device type from the same vendor will generate varying amounts of log volume daily and it’s more of an art than a science when determining what the total volume all of the corporate devices will generate daily.

Determining EPS isn’t a problem for existing log management or SIEM customers looking to upgrade to a new solution as they can generate reports from the old log management/SIEM tool and provide a break-down of device type and the daily volumes generated by each device category. However, prospects looking for a proposal for a net-new solution are plagued with the following tasks to properly design a log management or SIEM solution:

  • Complete inventory of all assets they plan on monitoring
  • Determining average, sustained event rates expressed as an EPS metric
  • Understanding how logging levels impact the volume of logs that are generated
  • Retention periods, storage options, use cases, regulatory requirements, ad infinitum

Fortunately, once you have a device count and can determine the EPS generated on average by each of the different device categories you need to monitor, the math is easy to determine the licensing, storage, system performance and archiving needs. My post “Basic Log Storage Calculations” http://www.netcerebral.com/?p=208 can assist in the sizing, as this post is geared more towards guessing the EPS averages for each device types.

In my roles as a presales SE that sold log management and SIEM we often were asked by prospects for budgetary quotes, proposals and architecture with little to no empirical data. In most cases the best we could get out of the prospect is an itemized inventory of the number and types of systems they would like to monitor. Without an understanding of the log volumes generated by devices, unique to every customer’s environment, we had to come up with a system of determining the EPS for the different device classes and using this as a starting point for calculating daily storage (EPS * Event_Size * 84600 / Compression Ratio).

The list below is an example of lessons learned in the field from actual customer environments and a document provided by SANS (sponsored by NitroSecurity – now McAfee) called “Benchmarking Security Information Event Management (SIEM)” (found at http://www.sans.org/reading_room/analysts_program/eventMgt_Feb09.pdf). With the information we collected we devised a list, which is a cross-section of averages per event source.

I hope you find this helpful:

Continue reading


Basic Log Storage Calculations

Determining the sizes of log management systems requires knowledge of the number of devices being monitored and the anticipated event rates for each class of system. In many customer engagements, Professional Services time may be required to measure the event rate calculations from all of the monitored devices. This is important since there are too many variables to predict the average or peak Events Per Day (EPS) of any given system. I would caution any customer that if the vendor they are working with gives them “magic” calculations and pricing without gathering the necessary information regarding customer-specific speeds and feeds, they can expect to spend a lot more money later once the vendor gets their foot in the door. Basically, poor planning will result in unavoidable OP/EX costs later.

EPS is one metric used by many log management and SIEM vendors to determine such factors as licensing, storage and peak system loads. Another variable used could be Events Per Day (EPD), especially when it relates to storage sizing and license enforcement. This is why it’s imperative that accurate device counts and product types are audited when planning a centralized log management or SIEM solution.

Continue reading