Some Generic SIEM Requirements to Ponder

Being tasked with selecting a Security Information and Event Management (SIEM) tool for your organization can be a bit overwhelming. I’ve been there and chosen poorly (in my last life)! The questions you need to ask the SIEM vendor you are buying from are limitless as every customer’s needs are different and the business drivers range from “check-box” compliance to actual enterprise incident handling and response.

Numerous customers have approached me with what they thought were straight Log Management (LM) requirements, since they have only ever had the luxury of manual log review using the “Grep”, “Awk”, “Sed” approach or “spreadsheet Kung Fu”, while others have the budget and want to “boil the oceans”. There are hurdles with both approaches, while the former may be the way to “grow” into a mature concept such as a SIEM tool and the latter will never be outgrown.

In fact, before you can perform real-time analysis on all of the logs to detect threats as they occur, you need to capture all of the event data from the plethora of heterogenous event sources and store the logs in a centralized location. Therefore, I believe log management is an essential part of SIEM because, with the right tool, 100% of your logs are readily available with automated archiving and retention. Additionally, since you have mandated all of the logs from the various technologies to be sent to your central facility, the teams that manage the devices will need an easy-to-use tool that will allow them to do their day-to-day tasks such as troubleshooting network issues, application development debugging, long-term investigations and possibly the last six months of an employees activity for HR or litigation purposes.

Regardless, you should have a strong command of what it is you need SIEM for and use vehicles such as Request For Information (RFI) or Request For Proposal (RFP) to rate each vendor on the top mandatory requirements vs. the “nice-to-have’s”. For this purpose, I have compiled a list of questions that you may determine to be useful when creating your vendor ratings criteria. Here are what I believe to be essential 70+ requirements for the ultimate SIEM and Log Management tool:

SIEM General Requirements

1. Log Collection Automation: The vendor’s product must provide a agent-less solution that will automatically accept events and start to monitor devices without any administrator intervention and can automatically scan Active Directory for the list of Windows devices to be monitored. I mention the Windows devices because in my experience, this represents the majority of monitored operating systems used by large organizations.
2. Log Management Automation: The vendor’s product must provide a log management solution that would require very little post-deployment effort for tasks such as introducing new event sources, managing retention policies and archiving of log data. Some of the greatest minds in the SIEM space believe that log management is a major requirement and should exist as a separate entity from the correlation engine; Log Management represents historical log data while the correlation engine provides real-time threat analysis and baseline anomaly detection.
3. SIEM Analysis Automation: The vendor’s product must provide the ability to start analyzing and correlating activity out-of-the-box. The product must assist security analysts by reducing false-positives automatically without configuring any rules or filters to do so. Generally this is done by using frequent vulnerability scanner reports to overlay context about the targeted asset and it’s susceptibility to known attacks.
4. SIEM Workflow Automation: The vendor’s product must provide a SIEM solution that can initiate workflow that will automatically open tickets either locally or on third-party ticketing systems and assign the tickets to the appropriate team members while maintaining a complete audit trail and metrics for the incident handling process.
5. SIEM Compliance Automation: The vendor’s product must provide the ability to reduce compliance auditing efforts by monitoring and alerting on non-compliance events in real-time and provide the necessary reports and dashboards to assist auditors in gathering the necessary data, alleviating staff from being taken off task during an audit. This is a key differentiator for compliance in initiatives as it goes beyond simple compliance reporting and provides immediate notification or even responsive action when a critical asset becomes non-compliant.

SIEM Architecture and Scalability

1. Bandwidth Throttling: The vendor’s product must provide the ability to limit bandwidth used for transmitting event data from remote sites. A combination of methods are provided such as batching and sending events in periods of lower bandwidth utilization, committed bit-rate, caching, shaping and scheduling different configurations throughout a 24 hour period.
2. Transaction Assurance: The vendor’s product must provide some mechanism that guarantees delivery of events to the SIEM tool and that no events will get lost if the SIEM system is unavailable.
3. Federal Security Requirements: The vendor’s product must provide the ability to enable advanced security to allow FIPS 140-2 level cryptographic modules and Common Access Card (CAC) authentication in accordance to US Federal Agency requirements. Every country around the globe has different federal standards and the vendor should be able to provide some adherence to the different cryptographic standards.
4. Collection High-Availability: The vendor’s product must provide options for log collection high-availability without the need for additional hardware. An example would be a syslog agent, whereby the two agents could be deployed to send events to the same SIEM manager, offering diverse destinations for every event source.
5. Storage Flexibility: The vendor’s solution must be able to store a year’s worth of log data locally on disk or provide log storage through SAN integration. This is usually done with methods such as elevated compression algorithms, event reduction (filtering / aggregation) and warm archiving.
6. Storage Volume: The vendor’s solution must be able to store at least 40 TB of log data in a highly compressed format on one appliance without requiring any costly external storage devices. I wouldn’t regard this as mandatory considering the cost of SAN storage is increasingly falling, but it’s definitely lowers the administration costs.
7. Log Management Scalability: The vendor’s solution must scale to larger environments and the increase of additional event sources without requiring additional hardware. This is generally achieved by sizing the solution up front with a 30-40% margin for anticipated growth. This is not so much a product feature as it is a design element.

SIEM Event Collection

1. Device Support: The vendor’s product must provide a comprehensive out of the box coverage across all types of event sources. The easiest method of event collection is using a passive component such as a syslog daemon which communicates over UDP 514. This may represent a majority of Unix or network devices but what about systems such as Windows, Databases or proprietary API’s such Checkpoint OPSEC LEA or SourceFire eStreamer? Monitoring the perimeter is a legacy approach to security, now insider threat, fraud, application security and identity monitoring are all fodder for SIEM and require the vendor to not only collect the events but also understand the proprietary vendor IDs associated with each event.
2. Distributed Event Processing: The vendor’s product must collect logs in a distributed manner, offloading the processing requirements of the log management or SIEM system for tasks such as normalization, filtering, aggregation, compression and encryption.
3. Custom Collection API: The vendor’s product must have a toolkit to allow customers to create integration with unsupported legacy or home-grown event sources. The toolkit must allow customers to integrate with Syslog, log files and databases and support the ability to parse multi-line log files.
4. Normalized Event Data: The vendor’s product must normalize all collected event data into a consistent format suggested by NIST 800-92. This prepares the data in advance to real-time correlation or ad hoc reporting and allows the data from heterogeneous event sources to be viewed in one consistent manner.
5. Categorized Event Data: The vendor’s product must categorize log data into a humanly-readable format to eliminate having to know vendor-specific event IDs. This lends it’s self to the point about “device support” in that the vendor has done the painstaking task of interpreting the security relevant events from each vendor’s products into a common taxonomy that eliminates having to write rules or queries specific to event IDs.
6. Event Reduction: The vendor’s product must provide the ability to reduce event data through filtering or aggregation before it is sent to the log management or SIEM system. Event aggregation is also know as event de-duplication which involves merging identical events (with the exception of the timestamps) generated by the same device into one log record. An example would be a Cisco Router with a flapping interface – many events will be generated repeatedly until the problem is fixed.
7. Secure Transport: The vendor’s product must provide encrypted transmission of log data to the log management system. This is more than just security paranoia, this is generally the requirement by compliance regulations and a court of law to eliminate the possibility of log tampering, log siphoning or any other questions regarding confidentiality or integrity.
8. Collection Health Monitoring: Any failures of the event collection infrastructure must be detected immediately and operations personnel must be notified. Health monitoring must include the ability to validate that original event sources are still sending events. A hacker will attempt to stop logging services on a compromised systems and auditors will not give you a pass if you have gaps in your compliance reporting, no matter how small.
9. Time Correction: The vendor’s product must be capable of correcting event time for systems with incorrect timestamps. This allows integrity for forensic analysis to determine the original time on the event source and what the system time was for each vendor component processing the event.

SIEM Log Management, Storage and Retention

1. Centralized Log Storage: The vendor’s log management system must be appliance-based with integrated storage. While the correlation engine may very well be software-based, the LM component should be a distinct system and not an all-in-one solution.
2. Autonomy / Integration: The vendor’s log management product must be capable of providing an autonomous log management platform without the need for correlation or a management server. Conversely, the log management platform must be capable of integrating with an existing SIEM platform without considerable effort. My analogy on this topic is remember those old TVs with the built in VCR? Your VCR would fail and would have to go in for repair and guess what happened to your TV? Same goes with a LM and SIEM – if they are integrated on a Swiss-army appliance then you create a single point of failure. At a minimum, if the LM component fails, the collection tier can fail-over to the SIEM component and not stop the real-time analysis of data.
3. Log Management Performance: The vendor’s log management product must be capable of handling peak event rates beyond 80,000 events per second (eps). This is important, especially during an attack scenario when event rates could be 100 times the sustained event rate. This means if you anticipate the event sources in your public facing DMZ to generate 800 events per second (eps) then a tribal attack against the perimeter will not knock over your logging infrastructure.
4. Storage Indexing: The vendor’s log management system must be capable of indexing terabytes of normalized log data and provide performance in both indexed and table scans that exceeds search results of 1 million records a second.
5. Logical Data Segregation: The vendor’s log management system must provide logical segregation of log data that can be viewed by different teams. Various operating teams can only see “their” device event data but no other event data beyond that, which provides separation of duties.
6. Retention Policies: The vendor’s log management system must provide the ability to create multiple policies for the automated retention and disposal of log data.
7. Log Data Integrity: The vendor’s log management system must provide audit quality integrity mechanisms in accordance with NIST 800-92. This is usually some method of a hash digest and/or digitally signing the event data.
8. Search Interface: The vendor’s log management system must provide a simple, intuitive search interface usable by different users with varying skill sets. Take a search engine as an example, the general population wouldn’t use it if it were too complex and the power users wouldn’t use it if it didn’t allow advanced search methods with logic structures.
9. Search Drilldown: The vendor’s log management system search interface must provide the ability to drilldown on output data.
10. Search Patterns: The vendor’s log management system search interface must provide support for simple Boolean-style search patterns as well as complex regular expressions.
11. Search Performance – Structured Data: The vendor’s log management system search performance must be capable of searching through millions of structured (indexed) log messages in less than a minute.
12. Search Performance – Unstructured Data: The vendor’s log management system search performance must be capable of searching through millions of unstructured (raw) log messages in less than a minute.
13. Search Method Combination: The vendor’s log management system search interface must provide the option to allow combined search queries using a combination of methods such as indexed and non-indexed event data and regular expression and full unstructured text search simultaneously without impacting search performance.
14. Search Criteria: The vendor’s log management system search interface must provide the option to search for any element in a log message such as strings (i.e. Microsoft) or integers (i.e. IP Address) and provide the ability to perform Regular Expression wildcard boundary matches (i.e. d{1-3}.d{1-3}.d{1-3}.d{1-3}).
15. Search Logic: The vendor’s log management system search interface must provide the ability to combine Boolean search operators (i.e: “ANDs” and “ORs” and “NOTs”) into a single search expression.
16. Search Time Range: The vendor’s log management system search interface must provide the option to search across time ranges using either a custom time (date / time start, end) or dynamic time variables ($Now – 2h).
17. Real-Time Alerts: The vendor’s log management system must be capable of generating alerts based on filter pattern matches for operational health monitoring.

SIEM Analysis and Workflow

1. Correlation Rules: The vendor’s correlation engine must provide many correlation rules out-of-the-box to automate the incident detection and workflow process.
2. Cross-Device Correlation: The vendor’s product must be capable of correlating activity across multiple devices out-of-the-box to detect authentication failures, perimeter security, worm outbreaks and operational events in real-time without the need to specify particular device types.
3. Statistical Correlation: The vendor’s product must be capable of keeping a statistical baseline of “normal” monitored activity. This includes attacker, target, ports, protocols and session data.
4. Historical Correlation: The vendor’s product must be capable of monitoring attack history against critical asset or by particular users.
5. Session Correlation: The vendor’s solution must provide the ability to correlate DHCP, VPN and Active Directory events to provide session tracking for every user in the enterprise. This is essential for pinpointing who was using a particular workstation historically during an incident investigation.
6. Correlation Tracking: The vendor’s product must be able to correlate event data against static lists of items that the customer either allows or doesn’t allow on the network (i.e. list of insecure protocols). Additionally, lists should be automatically populated by the system for tracking things such as attacks, user sessions and other policy violations.
7. Data Modeling: The vendor’s solution must provide the ability to model incoming event data into logical groups such as domains, networks, applications, criticality of target devices, etc. This data modeling should then be available to assist in filtering and logically segregating data from view.
8. False-Positive Reduction: The vendor’s solution must be capable of leveraging information pertaining to a targeted device and alter the criticality of the event based on the target’s susceptibility to an attack.
9. Pattern Detection: The system must be capable of detecting patterns of activity that would otherwise go unnoticed by real-time correlation.
10. Correlation Performance: The vendor’s product must be capable of efficiently presenting categorized data to the correlation engine to allow real-time detection and response.
11. Rule Chains: The vendor product must provide the ability to allow rules to be triggered in a series, matching various correlation activity before an alert is generated.
12. Alert Thresholds: The vendor’s product must provide the ability to aggregate and suppress alerting with granular options and use conditional logic to determine if an alert should be generated.
13. Re-Usable Content: The vendor must provide a solution that allows customers to create objects such as filters or search queries that are reusable throughout the system. I once used a SIEM tool that had a different interface for every resource you created from scratch with no way to levarage content I had created in another part of the system (i.e. a filter I created for firewall events couldn’t be used in a report query) – how frustrating!
14. Application Integration: The vendor’s solution must provide an interface to allow investigators to integrate with other security or network management tools and launch commands and tools to aid in the investigation. If your SIEM tool becomes the single interface for security investigations then it should include the ability to run the basic ping, whois, nslookup and nbtstat commands against event data. Additionally, wouldn’t it be nice to invoke on-demand packet captures or vulnerability scans during an incident triage? Many SIEM vendors boast alliance partnerships with other vendors, ask for a demonstration of how “integrated” the tools really are.
15. Case Management: The vendor’s solution must provide an integrated case management system which allows investigators to collaborate on incidents without  ticketing system administrators viewing case information. This feature supports the separation of duties that the security or forensic teams must have to conceal internal investigations. Imagine how fast your cover would be blown if you were investigating misconduct of the guy administering the enterprise ticketing system!
16. Workflow: The vendor’s solution must provide a complete audit trail and accountability during the incident handling or forensic investigation process. This called “track the tracker”. What is the Mean Time To Recovery (MTTR) on an incident? How fast are cases triaged by the tier 1 security analysts?
17. Incident Tracking: The vendor’s product must provide customers with the necessary tools to identify, isolate and remediate incidents as they occur.

SIEM Reporting and Visualization

1. Report Creation: The vendor’s solution must provide an intuitive reporting interface that can leverage existing reports or the creation of new reports that does not require complex SQL queries.
2. Future Proofing: The vendor’s solution must provide a level of confidence that reporting will continue to work and not have to be modified if a particular technology, such as a Firewall or IDS product, is replaced with a newer product or vendor. The reports should continue to run and include the new technology into the report criteria automatically.
3. Cross-Compliance Reporting: The vendor’s solution must provide the framework to report on ISO or NIST compliance items that can be mapped directly to any regulatory standard or enterprise security policy.
4. Report Trending: The vendor’s solution must provide the ability to profile data for use with baselines and increase the performance of ad-hoc reporting.
5. Logical Diagrams: The vendor’s product must provide the ability to import graphics from applications such as Visio and overlay chart objects to provide logical visualization of the enterprise network architecture, business processes or application specific components. This functional will provide a visual mapping of alerts to enterprise specific components.
6. Attack Visualization: The vendor’s product must provide the ability to visually represent event data into a dynamically updated graph. This will assist analysts in determining the expanse of attacks and pinpoint the original attacker during incident response and remediation.

SIEM Advanced Use Cases

1. Compliance Automation: The vendor’s solution must provide value in assisting in adhering to audit requirements, alerting of non-compliance and providing necessary reports that can be used during an audit.
2. Business Process Monitoring: The vendor’s product must provide the ability to monitor and visually display statistics for all dependent components used by business applications from start to end of a transaction. This includes the ability to monitor latency between components excessive resource usage, errors during process flow and other business logic required to troubleshoot business applications.
3. Worm Outbreak: The vendor’s product must provide automatic detection of a 0-day worm outbreak across the enterprise when IDS or Antivirus signatures are unable to detect the incident. The system must then immediately send alerts and automatically start the incident triage and workflow.
4. Physical / Logical Convergence: The vendor’s solution must be capable of collecting log data from physical access devices such as card readers, biometrics and security cameras and correlate this information with logical network and security devices to detect such patterns as building access after hours by contractors or users logged in through VPN and physically accessing the building within the same time period.
5. User Role Monitoring: The vendor’s solution must provide the ability to synchronize with authentication directories to collect information regarding user roles and responsibilities and correlate this data with all user activity. Users that violate their roles within the organization should be recorded for alerting and reporting purposes.
6. User Activity Monitoring: The vendor’s solution must be able to track user activity and ultimately bind an individual to an action. Analysts must be able to generate ad hoc reports that will detail what a particular user or group of users has accessed in the enterprise for a defined period of time.
7. Generic Account Monitoring: The vendor’s solution must provide the ability to correlate information regarding users that are logged into the domain and their generic or service account usage within the enterprise. The vendor must provide a mechanism whereby in the event of generic account violations, the solution can contain the threat in real-time using quarantine methods such as disabling the user’s switch port, adding filters to firewalls, disabling user accounts, etc.
8. Insider Threat Detection: The vendor’s solution must be able to detect suspicious activity, such as printing large numbers of files outside of business hours, emailing large attachments to personal email accounts, employee communication with competitors or the clearing of system audit logs to cover up malicious activity.
9. Forensic Investigations: The vendor’s product must be capable of allowing investigators to restore a year’s worth of historical log files to a single appliance and then perform complex pattern searches and reporting against terabytes of data in a short period of time. The entire process from restoring the data to reporting results must take less than two days.
10. Monitor Source Code: The vendor’s solution must be capable of correlating activity between enterprise users and source code repositories. Users accessing the repositories that are not developers or developers that are extracting sensitive intellectual property from the systems must be detected and alerted upon in real-time.
11. Fraud Detection: The vendor’s product must provide the ability monitor online banking applications, banking infrastructure devices and user transaction activity. The product must use this data to detect anomalous transactions such as simultaneous user transactions from multiple geo-spatial locations, fraudulent activity and breaches.
12. Threat Response: The vendor’s solution must be capable of not only detecting attacks but must also provide a mechanism to respond and mitigate the attacks in real-time using various quarantine methods while providing the necessary audit trail.