Linux SecOps – Look Who’s Knocking
This is a tutorial I posted on Anti-Online back in 2006 – just thought I’d update it and pass it along to the SecOps community to show you how easy it is in 2021 to do something as mundane with a SIEM that required lots of scripting back before SIEMs were common in the security field. It makes me laugh when I see some of this old scripting “Kung Fu” I had to do with Grep, Awk, Sed in order to do something that takes seconds with a good CLM or SIEM tool!
DISCLAIMER: This is a tutorial of sorts that takes you through a day-to-day problem and solution that I was often faced with in my Security Planning / Operations role for a large Telecommunications company. I am not making any assumption as to where in the curve people reading this will be situated and I don’t even guarantee this will be a good read. In fact, given my exposure and expertise of the tools used in this article, I may be missing the plot and some may find an easier, softer way of doing what I was tasked to do. Having said all of this, for those I’ve confused, sorry, I tried to provide links for further reading. For those I’ve disgusted with my simplicity or seeming Lamer approach, well, like you, I’m always learning and I’m open to criticism and advice.
Introduction
Why is it when you Google for something you absolutely need you can never find it? Well, case in fact, I had a Squid proxy server left over from a decommissioning project that was still seeing tons of traffic when it shouldn’t be seeing any! The Linux server was locked down using sudo and no one knew the root password so we had very little choices as to what programs we could run to view activity. The server was flaky and Netstat would never finish outputting the current activity. So the server folks approached me and asked if there was any way to find out what unique IP addresses internally were connecting to the five pre-configured proxy ports (8080, 8082, 8084, 8086, 8888).
As it turns out, the Squid admin user had access to the Tcpdump application and could run the application against Eth0. I got him to run Tcpdump and output it to a dump file for three hours worth of activity during the lunch hour web traffic spike. This produced a 470MB text file that I had to SFTP from his server to my Linux box.
Alrighty then! What do I do with a honkin’ text file that repeats the same info endlessly? We have hits from employees and internal servers hitting the proxy ports, the proxy itself establishing connections to the web, the foreign sites replying to the proxy and then, finally, the proxy returns the data to the corporate host. One conversation from an internal host connecting to the homepage of their favorite security tutorial site could warrant four times the number of HTTP flows. I needed to strip out extraneous information and narrow down the million+ lines of data to something sensible. So, I started thinking of the commands that would be required so that eventually I could write a shell script.
Calculating Peak EPS for Security Log Monitoring
Much of the challenge in sizing and planning Centralized Log Management (CLM), Security Intelligence Systems and Security Information and Event Management (SIEM) solutions is determining an adequate amount of storage for storing logs for real-time analysis and archiving the log data to meet long-term retention requirements. The biggest challenge most customers face is determining the required metrics needed in sizing a solution. My post “Basic Log Storage Calculations” http://www.buzzcircuit.com/?p=208 can assist in the sizing and variables needed and my post “Guessing Game – Planning & Sizing SIEM Based on EPS” http://www.buzzcircuit.com/?p=231 can help with guessing the EPS averages for each device types. Finally, I have a couple of cool calculators at http://www.buzzcircuit.com/?p=408 and http://www.buzzcircuit.com/?p=378 that can actually assist with the final calculations.
At this point you have probably guessed that log storage calculations and storage planning is somewhat of an art, rather than a science – there’s a lot of guesswork involved, especially if you don’t have access to the systems or network devices hosting the logs. While I have done a good job (I think) in helping you dispel some of the myths and “guesstimating” an overall log capacity in previous posts, one area that is often overlooked in planning log management or SIEM is the concept of Normal EPS (NE) vs Peak EPS (PE) and ensuring your daily calculation provide a necessary contingency for consistent peaks in your event logging throughout the day.
Normal vs Peak Logging
There are two basic calculations when combining normal + peak EPS, which by no means is a hard rule. The idea is that there is the NUMBER_OF_PEAKS multiplied by the DURATION_OF_EACH_PEAK, which is then multiplied by the DEVIATION_FACTOR. To describe each of these points:
- NUMBER_OF_PEAKS: calculating Peak EPS (PE) is required to factor in Normal EPS with Peaks (expressed as NE+PE) to ensure their is sufficient licensing and storage to accommodate periods of deviation from the normal EPS throughout the day. The default in the calculator below assumes there will be at least 3 peaks a day (morning logins, lunchtime web surfing, evening logoffs/backups). This value will vary based on network throttling, congestion, attacks, etc.
- DURATION_OF_EACH_PEAK: This setting works in conjunction with the previous PE setting and assumes that each peak lasts for approximately 1 hour (3600 seconds) – this may vary given many factors such as how congested the network is, how busy the logging device is or other scenarios such as DDoS attacks.
- DEVIATION_FACTOR: is generally 2-5x the average EPS for that period. While in reality the EPS spikes almost 20x the average EPS for only seconds, we are building in contingency for attacks such as perimeter devices under DDoS or excessive IT Operational errors that go unnoticed for hours. NOTE: again, this is an art, not a science and we’ll sound like we know more than our competitors if we think to include contingency into our calculations!
Hope you enjoy!
Event Log Convergence = Business Intelligence
Problem
I have come across many prospects over the last 15 years that are only trying to acquire a SIEM solution to satisfy a compliance requirement, or what we call in the industry, “check-box purchasing” – they have a minimum set of requirements specific to only one business unit or compliance mandate that is completely siloed from the rest of the organization.
Here is how this conversation usually goes:
Client: “we would like a SIEM tool that will help us monitor our 200+ Windows Servers for PCI-DSS compliance”
Me: “What other event sources are you going to be monitoring with the solution?”
Client: (Stunned look) “we only need to monitor our servers.”
Me: “PCI-DSS requirement 10 states you have to monitor the logs from all of your security devices and servers that are deemed critical assets.”
Client: “our department is only responsible for the servers we listed.”
Me: “To get value out of a SIEM solution and monitor all 12 PCI requirements you need audit logs from all of your devices and contextual information regarding your network, asset and vulnerability data – and that will just get you started.”
Client: “Perhaps we need to increase the scope – we’ll get back to you.”
While the Centralized Log Management (CLM) and Security Information and Event Management (SIEM) vendors will be lined up around the block to influence the sale, the vendor you choose should be a trusted advisor. They will be interested in providing you the most value from your investment and assist you in designing a solution to satisfy many business problems that goes beyond a traditional security-centric SIEM. This is why you will need to identify key device types and the value that can be derived by cross correlating the log data with business context to align monitoring with your governance, security and compliance initiatives.
The SIEM Value Derived from Heterogeneous Device Logging
While each of the event sources you collect events from will provide distinct reporting and alerting value, combining many different “types” of event sources will derive immediate intelligence about the business and help analysts establish baselines of threat activity. One of the other benefits is that incidents can be prioritized by business value, threat classification and the additional context can help reduce the plethora of false-positives or false-negatives that plagues every CLM solution.
Additionally, the multitude of the various technologies have their own management and reporting solutions that become “silos of information” that only the black-belts responsible for each of the device types are able to decipher. This makes security intelligence and investigations near impossible when an analyst has to request log data from the owners or log in to many different systems to find the evidence to support their cause. Essentially, they would have to piece together the clues and manually normalize the data using a technique called “Spreadsheet Kung Fu”, which would be fraught with assumptions and inference.
Below is a list of different device types and the value that can be derived from each when correlated together. The list isn’t exhaustive and I’m not suggesting you need everyone mentioned to successfully deploy a SIEM, but the more data feeds you can correlate, the more intelligence you will have available in the future to expand and grow with your business (click “more…” for complete article):
Chronology of a Ransomware Attack
This chronology is a fictional characterization of an actual ransomware attack on some unsuspecting IT administrators with names and programs that have been changed to protect the innocent (or the unlucky).
Monday, December 7, 2020 – 18:35 (IT Crew)
Dave, Jack, Cathy and Rashid all work in the IT Department of a large Law Firm and are required to stay late daily, after all of the firm’s employees have left for the day, to administer systems and perform the daily backups of computer systems when they are not in use. This after-hours, unsupervised work period gives the IT crew the opportunity to play action/shooter, multi-player games over the company Local Area Network (LAN) while they wait for menial IT tasks to complete for the day. Their game of choice today is called “Solar Commando,” and they all downloaded the same “cracked” version of the game using BitTorrent and have been “fragging” each other over the network for the last hour.
SIEM Storage Calculator
AIO WP Security Firewall Log Hacks
Ever since my WP site was hacked and defaced back in 2012 by who claimed to be the “Kuwait Hackers” group, I became especially paranoid of using WP plugins wildly on my site and in fact started installing plugins that would aid in hardening and scanning the site for security issues. I started a routine of checking for plugin updates, backing up my DB and, at first, used .htaccess to blacklist IP addresses that showed up with significant numbers of failed logins or 404 errors. It became a taunting endeavor and I spent countless hours trying to stay ahead of the bad guys on my site and the three other sites I was managing as sub-domains for customers.
So, when the All-In-One WordPress Security & Firewall plugin became available to the WP community four years ago, I installed it and was amazed at the plethora of security features it offered. The plugin was easy to set up, with clear instructions. I get an email when someone suspicious tries to login to my site or generates too many 404 errors, so I can counter threats fairly proactively. The options I implemented negatively affect any of my performance or other plugins (some of which are additional security plugins). Check out the link or see the video below for details on all the features.
Essential Firewall Rules for Internet Facing Firewalls
1. Introduction
In a not-too-distant past I worked for a large telco company, first as a network firewall administrator and eventually made my way into the security team as a network security specialist, responsible for developing and auditing the network security standards. While I wrote many network-related standards and best-practices documents, the following was definitely one of my favorites (or favourites here in Canada, eh!). I had to convert and sanitize the content and while it is very lengthy (and not suitable for a blog), I figured I give it a shot at posting to my site. Please note, we had a mix of Check Point and Cisco PIX firewalls when this document was first authored. The newer, Next-Gen Firewalls (or Application Firewalls – layer 7) may conflict with the following rule order.
A special thanks to Yuri Kopylovski who prodded me, moderated and, otherwise, helped me co-author this guide and to the many folks at www.anti-online.com who benefited from the content over the years. This was originally published in 2007 under my alias “aciscorouter” and has since been edited to include suggestions from the Anti-Online community.
You will note that the rule order is identified in the first column I provide with the samples under all descriptions (i.e. the very first rule is a drop rule against the firewall and the very last should be a “clean-up” rule). However, this is a best-practice – order your rules any way you see fit (…and report back to me and let me know how that works for you)!
SIEM-as-a-Service: do the survey and let me know if you’re an early adopter…
Are you a Security PreSales Ninja?
Security Ninja Quiz
Take this quiz and determine if you are a Security PreSales Ninja.
NOTE: this quiz has a 20 minute time limit to complete.
Enter your full name and email address in the results table to save to the leaderboard!
|
You must specify a text. |
|
|
You must specify an email address. |
SCAM: Call from Computer Maintenance Department
This scam sucks and they seem to be getting out of control. Two years ago my Mom got scammed out of $300 on her credit card and we had to reimage her laptop after she had gotten a call from “Windows Tech Support” and, before checking with her son, the security guy, went ahead and let them remote to her laptop and enter her credit card information under the guise of helping rid her viruses. She got her money back from VISA and we reported to RCMP.
I get the calls often in the evenings and always someone with an Anglo-Saxon name with an obvious South Asian accent. I recently got a call from 1-989-686-3652 (reverse lookup shows Michigan, US) and here’s the dialog:
Aggressive South Asian Accent: “Sir, this is Mark from computer maintenance department and the computer is making errors and problems on the Internet!”
Me: “Oh, really. The computer?”
Aggressive South Asian Accent: “Do you have have a computer connected to the Internet?”
Me: “Yes, many. Which one is it?”
Aggressive South Asian Accent: “Many? How many PCs do you have?”
Me: “Sorry, I can’t tell you that. Tell me which one is causing the problem.”
Aggressive South Asian Accent: “Your PC is making big problems for us and you will get in trouble.”
Me: “Wow, we don’t want that to happen – which IP address or hostname are you referring to?”
Aggressive South Asian Accent: “Yes, that’s right so I can help you.”
Me: “Sir, you missed my question. Which IP address or hostname is causing you the problem?”
Aggressive South Asian Accent: “You know all your PC address on the Internet?”
Me: “Yes, I have an internal network connected through a range of NAT addresses on the Internet, I would know if the address is mine.”
Aggressive South Asian Accent: <CLICK> hung up…
Microsoft, RCMP, FBI and many other agencies are aware of the scam and have education and warnings that indicate “just hang up the phone” and I wouldn’t engage in baiting the perpetrator (the way I did) as these callers can get very aggressive and can go as far as death threats (as this British Columbia man found out here).
In Canada at least, you can report any suspicious calls to the Canadian Anti-Fraud Centre by calling 1-888-495-8501 or by going to antifraudcentre.ca. However, it seems the agencies are more focused on dealing with the millions of dollars that have be bilked from Canadians that fall victim to the scam.
- S3 Ep138: I like to MOVEit, MOVEit
- Firefox 114 is out: No 0-days, but one fascinating “teachable moment” bug
- Chrome and Edge zero-day: “This exploit is in the wild”, so check your versions now
- MOVEit zero-day exploit used by data breach gangs: The how, the why, and what to do…
- Researchers claim Windows “backdoor” affects hundreds of Gigabyte motherboards
- S3 Ep137: 16th century crypto skullduggery
- Serious Security: That KeePass “master password crack”, and what we can learn from it
- Serious Security: Verification is vital – examining an OAUTH login bug
- S3 Ep136: Navigating a manic malware maelstrom
- Ransomware tales: The MitM attack that really had a Man in the Middle