In my daily security strategy conversations with clients I’m often asked “what’s the one thing I should really focus on?” Now that may seem like a pretty big question to ask, but when you consider how thinly spread most security functions are within many organizations (if they exist at all) it’s not a surprising one.
If you’ve worked with me or read my blogs at any point, you’ll know that I’m a pretty big proponent of building an underlying security framework that takes into account the unique security context of the organization; so there are a LOT of ways to answer this kind of a question. Do you start with executive support, risk definition, policies? What?
My usual response goes something like this: “You can spend weeks, months, or even years putting your security program in place, but an incident could happen tomorrow. If it did, how would you know and what would you do?” More often than not I get a blank, worried look as a reaction to my inquiry, followed by a conversation that essentially boils down to “I’m not sure”.
Now, there’s nothing wrong with this per se. Obviously admitting you have a problem is the first step towards solving it. The people sitting on the other side of the table are always smart, capable individuals who are usually a) unsure where to begin, b) overwhelmed by the enormity of the security conversation, c) under-budgeted and under-staffed, or d) all of the above. They just need a little help.
I’m not a fan of holding my audience captive waiting for ‘the big reveal’, so I’ll let you know up front that, personally, I think nearly ALL small to medium sized business (and most enterprises) are best served by what I define as an ‘outsourced tier-1 managed security monitoring service’; coupled with a simple incident response plan and a bit of awareness/training. Let me explain.
It is ABSOLUTELY ESSENTIAL that you have visibility into the security events occurring within your information ecosystem. If you aren’t keep an eye on what’s actually happening, then EVERYTHING else you do to address security within your organization will fail! That may sound harsh, but it’s true. You WILL NOT be successful in defending your information assets it you don’t know what’s happening to them.
Even if you DO have a reasonable level of visibility into security events occurring within your information ecosystem, the next part is just as important. Do you have a plan in place to deal with a suspected security incident WHEN (not if) it occurs? Do people know about the plan? Do they understand their role in it? Do they know what to do? It’s FAR better to create a plan in advance – not during an event.
I’ll deal with the incident planning and awareness/training pieces in a separate article. This post is really about the visibility side of the equation. It’s about coming up with the most comprehensive, most cost effective way of gaining insight into the security events happening within your information ecosystem so you can address them as they occur (instead of months later, AFTER the damage has been done).
First, let’s establish that visibility is made possible by event logging and log monitoring. Nearly every network device, operating system, application, etc. has some level of logging capability. In some cases security logs need to be turned on – or they need to be ‘tuned’ to get the kind of alerts that are important to you. Do you want to know about all failed login attempts (for instance), or just big issues?
Side note: Deciding what types of alerts should be generated is a topic in its own right. You need to make a lot of decisions about what you want monitored, why you want it monitored, what level of detail you want to achieve, how you want to see alerts generated, where you want them to go. And so on. Again, probably a topic for its own separate article later on.
As you can imagine – all of those log files being generated can be way way too much for a human being to get through (especially when you start talking about hundreds or even thousands of systems). It’s not uncommon for MILLIONS of alerts to be generated per week, day, hour, minute, or even second (depending on the size and complexity of your environment).
So how do you weed through all those alerts to identify the things that really need your time and attention? Well, some folks try to just focus on the event logs and alerts that come from their security devices – things like firewalls and intrusion detection/prevention systems. Sure, those are useful, but they aren’t going to catch everything – in fact, they are likely to miss a lot of important event data.
One of the most basic principles of security incident response is to have as much data as possible. If you are trying to piece together an attack against one of your key financial systems, chances are you’re going to want to know about it quickly, deal with it quickly, and understand how it was done so you can prevent it from happening again. A firewall alert may tell you there’s a problem, but is that enough?
Let’s talk briefly about the importance of understanding both the breadth and depth of a security event. If someone were to attack the financial system alluded to before, an alert at the firewall might signal the start of an event. If your IDS/IPS also flags it as suspicious, now you’ve got something more substantial to work with. If the target server also generates an alert, now you have a MUCH better idea of what’s going on.
When you have multiple event logging mechanisms working together in unison, you also decrease the chance of getting false positives (i.e. something suspicious looks like it may be happening, but it really isn’t) – or worse yet, false negatives (i.e. when something suspicious IS happening and no alerts are generated). In other words, you increase the effectiveness of your monitoring efforts overall.
Side note: If the idea of greater visibility and effectiveness doesn’t move you to agree how important this is, there’s also the compliance perspective to take into account. Almost EVERY compliance requirement and almost EVERY ‘best practices’ framework includes one or more specifications for log monitoring, event management, incident response, etc. There’s really NO getting away from it.
This increased effectiveness (and compliancy) comes at a price though – efficiency. If you are aggregating and correlating log files from multiple devices (and you are facing stacks and stacks of log files DAILY), how can you possibly hope to find that ‘needle in a haystack’? There are really just two answers here – you either do it yourself, or you get someone else to do it.
Because the term ‘outsourcing’ has become such a dirty word within the IT industry, and because people are increasingly conscientious of their long-term employment, and because many security professionals are ‘paid to be paranoid’, there’s a general perception that dealing with security monitoring in-house is preferable. I’d argue that this is a bit of a misnomer and actually creates more problems than it solves.
The #1 reason I tend to shy away from in-sourcing security monitoring is COST. Not just the up-front cost of investing in a few key technologies, but the TOTAL COST OF OWNERSHIP that comes with trying to do everything yourself. Reading log files, identifying security issues, analyzing suspicious activity, and dealing with incidents is a HIGHLY specialized skill set (which usually means expensive).
Not to mention you’ll need at least one dedicated, full-time employee who’s doing NOTHING but managing log files and investigating alerts. Now this may be okay for a larger organization, but I can’t think of too many small business owners who want to keep a 70k to 90k resource on staff that just does this stuff all day – ON TOP OF the half-a-million dollars it will take to put the tools in place.
Side note: Yes, it may be a slight over-exaggeration to say that you need all of this – but remember the basic premise here. If you don’t know what’s happening within your information ecosystem, you CAN NOT PROTECT IT. That visibility is essential, and you need to have the tools and the people in place to keep an eye on things. You may be able to get away with less – sure. Risk is a personal decision after all.
There’s a better option though – and one that I HIGHLY recommend for almost EVERYONE. There are a number of Managed Security Service Providers (MSSPs) in the marketplace who specialize in security monitoring. For an initial set-up fee and an annualized contract of some sort, you can have someone ELSE do all of the work – and for WAY WAY WAY less money (and effort).
Here’s how most of them work. The service provider will add one or more event collectors (i.e. a security device that collects all of the log files from all of the devices you identify) to your network. This box will take all of those log files, run a few basic algorithms, compress everything, and send it (via a secure connection) to the service provider for further analysis.
After that, your log data is fed into additional event correlation and analysis engines to filter through all of the events and identify the ones that look anomalous or suspicious in some way. This may take 10,000,000 alerts and drop them down to 500. At that point, a trained and qualified human being will look at the remaining 500 alerts to see if there’s anything to really worry about.
If the analyst does find that a suspected incident has occurred, there’s usually a classification and categorization process and a validation process. Then you get a phone call. Once you get called, and depending on the provider (and the service level you subscribed to), the provider will help you understand what’s occurred and provide recommendations on what should be done about it.
So, in essence, for far less money, less time, and less effort, you’ve had VERY qualified individuals use HIGHLY specialized tools to whittle through ALL of your log files, taking thousands or even millions of alerts per day and reducing them down to what could end up being one or two phone calls a week or so. These numbers are really just examples (because it’s a lot more complicated than this), but you get the idea – more [security] for less [money].
So again, when my client’s ask me what to focus on – my recommendation boils down to “let’s get someone in here to talk to you about a managed monitoring service. At the same time, we’ll start working on building that baseline incident response plan I mentioned (which is also greatly simplified if you have a managed services partner) and getting some basic awareness/training in place.”
As for WHO to get as your managed security services provider, that’s a different topic entirely and leads into a whole new series of articles on gathering technical requirements, establishing clear selection criteria, evaluating options, etc. If you want to learn more though – I’m always here to help. Just give me a call at SecureITExperts: 425.877.0919.
*NOTE: SecureITExperts is NOT an MSSP. We do NOT provide managed monitoring services. We offer a completely independent third-party perspective on the issues and challenges surrounding log monitoring and management (among a wide variety of other security topics). We can also recommend service providers and broker service arrangements – However, we value our independence and our objectivity above all else. Our focus will be on helping YOU find the right solution for YOUR business.