Author: Ravi Devireddy, Founder & CTO
As the waiter cleared our plates to make room for afternoon coffee at the local restaurant, the topic of discussion with Andy turned into machine learning and analytics in cybersecurity — the buzz surrounding all things security nowadays — and how can it be applied or used. You see, Andy is no ordinary security guy. He is leading the charge for security operations at a global company that employs over 20,000 people, he is a US Army veteran, and has worked in cyber operations at various three letter agencies. I deeply empathize with Andy’s cause; he inherited a messy compliance-oriented ‘tick-the-box’ security practice a couple of years back, and since then has built a security operations center, hired smart and dedicated individuals who work very hard each day on the mission, all while navigating through decades-old organizational neglect and bureaucracy. Andy is not only a security domain expert, his interest in machine learning led him to undertake graduate-level study on machine learning at a local university. He is the real deal when it comes to being on the cutting edge in the cyber security profession.
Andy is expanding the coverage of security visibility across his organization, upgrading endpoint technology, replacing perimeter security tools, implementing security audit logging across his environment to gain visibility into all servers, endpoints, applications, users, data stores, network devices, access points, etc. — a massive undertaking for any organization, and even more drawn out for an organization of his size. Looking ahead, his team also realizes that the volume and variety all of this data that they are collecting needs a big data platform for cost-effective storage & processing. They also realize that alternate approaches to mining this data is required to meaningfully make use of the investments they’ve already made. This isn’t anything earth-shattering. This is the pattern we have seen across the country, across verticals, and across organizations of all sizes.
In 2012, Gartner predicted that by the year 2016, 40% of enterprises will actively analyze at least 10 terabytes of data for information security intelligence, up from less than 3% in 2011.
From our experiences on the field working with several customers, we can attest to this trend. Most of our customers are dealing with data in tens of terabytes, collected from their networks, servers, endpoints, users, applications, etc.
This situation presents unique challenges for security operations teams. To illustrate this point, let’s say that you are trying to find your lost car keys in a large parking lot at night, and all you have is a flashlight. Your visibility is limited to where you shine the flashlight. This is a situation akin to a pre-big-data security world, where what you can “see” in your organization is limited to the small number of perimeter access controls you had setup. Now, getting back to the parking lot, you realized you needed better visibility to find those damn keys, so you turn on the floodlights. The parking lot is now brightly lit up, but you realize that although you have light on everything now, you’re still no closer to finding your keys because now there’s too much flooding your vision to actually focus. I call this the blinding effect of all of the “visibility” you suddenly have. Many security teams are in this situation after beefing up their logging and monitoring. They’re drowning in data, and no closer to solving their challenges.
Andy, however, is thinking ahead and exploring various technologies to help make sense of the security data at his disposal. One of his team’s primary drivers for security analytics is their need to gain meaningful insight into targeted attacks that bypassed preventive controls and have penetrated the organization’s defenses. Like myself, Andy subscribes to the philosophy that to identify such targeted attacks, in addition to looking for known bad via threat intelligence feeds, malware sandboxes, next-gen endpoint protection, and carefully curated event correlations, he also needs to invest in anomaly detection methods that identify unknown things for which there are no signatures, rules or threat intelligence. We share a vision of intelligent machines (our robot overlords) discovering, learning, and identifying patterns of behaviors that assist our security operations teams in managing the cybersecurity mission.
Having gone through few proofs of concept for this promising new tech, Andy is a bit disillusioned. Machine-learning-based anomaly detection in its current stage is non-deterministic in nature, largely due to the lack of real labeled data (ground truth) in most enterprises, to train the behavior models. There are shades of grey between the binary poles of true positive and false positive. Over our steaming post-lunch espressos, Andy and I brainstorm this situation, coming to the following guidelines for how to succeed with analytics in cybersecurity operations:
Set the right expectations upfront to internal stakeholders and users of security analytics technology regarding its purpose:
- A data-analytics mindset in the organization.
A bias towards exploratory analysis, rather than a reactionary tool providing alerts. An active defense rather than passive monitoring, where identifying and investigating suspicious patterns and early signs of compromise are part of the job. A willingness to develop a strong culture of inquiry where questions are just as valuable, if not more so, than getting to the quickest answer to complete the work-item at hand.
- Not looking for an “easy button,” i.e. IDS type of alerts
If the evaluation criteria is to assess what threats the analytics technology can find, it subscribes to the old-school IDS mentality of looking for an “easy button.” The unfortunate truth is that it isn’t sustainable anymore. Instead of what is found, perhaps better criteria in evaluating analytics is “what is discovered.” How did it help security teams become smarter, faster, or better in identifying meaningful patterns from mountains of data as they go about their jobs to detect, hunt and respond to security incidents? Did the technology meaningfully impact key performance indicators (KPIs), such as mean-time-to-detect (MTD) or mean-time-to-resolve (MTR) security incidents?
- Security analysts are both users and part of the overall solution.
Machine learning based analytics solutions in cyber security comprise three essential components:
- A learning system that can be trained to discover and learn various aspects of the organization’s infrastructure and activities — things that security analysts, themselves, do not know or cannot keep track of.
- A detection system that can identify patterns of deviations or anomalies from this automated learning.
- An analyst or operator that can reinforce or provide feedback to automated learning engines within the context of the organizational structure and the mission.
Setting the right expectations to the technology/vendor community emphasizing that the grey areas inherent in ML/anomaly-based technologies, are precisely the reasons why the solution must be easy to use by security operators (not data scientists), easy to interpret (no blackbox algorithms), and easy to interact with (aiding analysis and decision making).
Andy and I are on two sides of the same coin, solving some of the most complex challenges vexing cybersecurity communities. Empathy for each other’s challenges and a common understanding of the solution enables my team at E8 Security and Andy’s team to bring advances in analytics and machine learning to the forefront of cybersecurity operations.