Attack Simulations for your SOC

 Introduction

SOC analysts spend 99 percent of their time looking at the same data and patterns over and over again. They develop muscle memory that helps them use their platforms without much effort but also in grains in them potentially poor practices. Most analysts are new to the industry and are taking certifications like CompTIA Cysa+ and Azure SC200 but none of these certifications teach them what genuine malicious actions look like. So how do you train a team of people how to detect malicious activity and respond in the ever-morphing threat landscape? You throw them through the ring of fire. You make them triage real (as close as possible) incidents and appropriately monitor their behavior and progress to tune the processes perpetually.

Analysts are arguably the single most important role within a SOC, they carry the burden of triaging alerts and correctly identifying whether malicious activity is occurring. They are the first, middle and last line of defense. However, despite this fact being very true there is an unfortunate reality. Analysts are considered beginner roles which leads to a lot of them simply being unfit to meet their objectives. This in turn places a requirement on leadership to actively and rigorously invest in their personal/professional development. Any decent leader should welcome this challenge head on and if you're an analyst who feels like that isn't happening then you have bad leadership.

Attack Simulations

Attack simulations are the execution and use of tools, techniques and procedures commonly deployed by genuine adversaries to generate telemetry, data and often alerts so that certain questions can be asked such as: Am I protected by X control, Will the analyst identify the activity as malicious, Will the activity be captured in my telemetry etc. The complexity used in simulations can vary a lot and simulations are often confused for emulations. When running a simulation there are a few rules or objectives you want to keep too:

• Plan your simulation with the purpose in mind. Simulations to train analysts look very different (a lot quieter and more specific) to simulations that test tools.
• Unless you're retesting something that failed keep them unique from each other
• Try to explore multiple steps in the kill chain so that a bigger picture than just one operation can be made.
• Rigorously record details about your simulation. Maintain items such as what and where the simulation was run, what are the associated entities and the associated timeframe
• Maintain a database of simulations and their outcomes so you can use historic data to guide future simulations and increase their value. 

The above points are important for conducting high-quality simulations, but you should also consider your target audience. If you're just trying to train a new starter don't push them too much.

Inside a SOC

Using attack simulations to train analysts requires more thought than when your testing controls. You will have a broad range of skills across your team and the overall preparedness of your team will vary. You need to consider how you can articulate value over time through established metrics and ensure that the simulations you run are targeted toward the knowledge gaps in your team.

Metrics

You can use metrics to highlight the value of the simulation program and identify areas of weakness within the team. Below are some metrics you will want to think about:

• Rate of successful identification
• Time to triage
• Types of attacks simulated

With these, you can monitor how well each analyst performs on an individual basis and across time. These metrics can also be fed back to the start of the simulation process and allow you to target areas your analysts are worst at.

Preparedness

As mentioned, before you need to take into consideration how prepared your team is going to be because if your already aware that your analysts will immediately fail or panic then it's best not to put them through that. Instead, you need to approach simulations in stages and ensure that analysts have "buy-in" on the process. Below I created three phases you may want to use:

Phase 1: Workshops and exercises for analysts to explore and gain confidence in tooling and alert types. Some really cool examples of workshops are incident replays and onboarding training. 

Phase 2: Pre-coordinated and announced simulations using testing environments with senior team members guiding analysts through triage.

Phase 3: Live simulations indistinguishable from real incidents.

By deploying the above phases over an extended period of time you reduce any risks associated with running live simulations and gain the opportunity to work out any issues that you don't want to arise. Finally, once you have deployed all three phases you can continue them in conjunction with each other. I have created a diagram that highlights what this may look like:

Technology

The technology used when conducting simulations really depends on the skills available. You can use a collection of custom scripts or manual instructions, utilize open-source tooling, or deploy paid toolsets. So long as you can achieve the objectives and rules set out above the technology used does not matter much. If you're looking to test analysts, the three main things your technology needs to achieve are 

• Data, telemetry and alerts generated by the simulations are indistinguishable from genuine incidents.
• Each simulation is unique from the next, so analysts don't pick up on commonalities and instantly regard it as a simulation.
• Ensure your simulations reflect genuine activity observed in the wild. 

I'm going to write a separate post on individual pieces of technology in particular the use of custom atomics in red canaries atomic red team framework, paid breach attack simulation tools (they aren't that great), and using attack flows to record the simulations you do.

Attack simulations will look different in each organization, but the above content should give you plenty to think about and make a start.