Intro to Detection Engineering

Note: This room is a refreshed version of Intro to Detection Engineering. The original may be retired in the coming months, so we recommend using this new version instead. Don't forget to leave feedback to help us shape future updates.

As a SOC analyst, you deal with alerts every day: triaging, investigating, escalating. But have you ever stopped to ask where those alerts actually come from? Who decides what gets detected, and what gets missed? These questions can be answered with the Detection Engineering process in security teams. Before moving on to rule creation, you will learn concepts to develop an engineering mindset and understand the Detection Engineering process. This knowledge equips you to spot gaps better and develop consistent detections.

You don't want to be that guy.

Learning Objectives

By the end of this room, you will be able to:

• Learn what Detection Engineering is and its role within a SOC.
• Understand the Detection Engineering Life Cycle and recognize common frameworks used to structure it.
• Understand the mindset a detection engineer must carry when building detections.
• Identify the main challenges that Detection Engineering teams face in practice.

Prerequisites

Before starting this room, we recommend you have previous knowledge of what a SIEM is and how SOC alerts work:

• SOC L1 Alert Triage
• Introduction to SIEM
• Splunk: Exploring SPL

Your SIEM is ingesting millions of events a day. You have 100 alerts firing every hour, and all analysts are drowning in them. Most of the alerts are noise, but somewhere in that pile, a real attacker has been sitting undetected in your network for three weeks.

Sound familiar? This scenario plays out in real SOCs every day. But here's the thing: the alerts that failed were built by someone. Someone decided what to detect, how to detect it, and what to ignore.
That "someone" is a detection engineer, and their discipline is Detection Engineering (DE).

At its core, Detection Engineering is the systematic process of designing, building, testing, and maintaining the detections that power a SOC. It sits at the intersection of threat intelligence, data engineering, and security analysis, translating knowledge of attacker behavior into actionable signals.

Unlike a SOC analyst, who responds reactively to what the tooling surfaces, a detection engineer works proactively, studying attacker techniques and building detections before threats materialize. As an L2, understanding this discipline means you can contribute to improving the system you work in, not just operate within it.

Where Is the Detection Engineering Team in a SOC?

There's not a singular answer. The placement of the Detection Engineering subject varies significantly across organizations. This happens for several reasons, such as the total budget a company has to invest in a dedicated team and the maturity of its SOC processes.

In some cases, all the detections are handled by L2 and L3 analysts from the SOC team. The diagram below illustrates three different ways in which Detection Engineering is implemented in companies based on their resources and maturity:

Regardless of the team structure, detection engineers rely on a common set of tools to do their work:

What Does a Detection Engineering Team Do?

The core activities of a Detection Engineering team are based around three pillars:

Beyond these three pillars, Detection Engineering teams often also take on automation, threat intelligence, and threat hunting. The exact scope varies widely because there is currently no industry-wide standardization of what a Detection Engineering team owns.

Practice: The Detection Guy

Senna is a SOC manager looking to improve his SOC detection maturity. He hired you, an experienced SOC L2 analyst, to help him decide how to build this high-maturity team.

By answering his questions correctly, you will receive flags that will let you complete each task in this room.
Use the View Site button below to interact with Senna. Help him!

Let's revisit the word "Engineering" for a moment, because you are seeing it a lot.

In software engineering, you don't just write code and ship it. You design, build, test, review, deploy, monitor, and iterate. There's a process with a structured life cycle that separates a reliable, maintainable system from a fragile one.

Detection Engineering is no different. Writing a SIEM query that catches a known-bad behavior isn't engineering. Engineering is what ensures that the query is built on solid threat research, validated against real data, tested for accuracy, documented, deployed with version control, monitored for drift, and revised when attacker behavior changes. Without that structure, your mistake appetite will be high, leading to tons of poor alerts (or missing attacks), making the SOC analyst's life a nightmare. This is why the "Engineering" in Detection Engineering matters: it's a commitment to treating detections as products, not scripts.

The Detection Engineering Life Cycle

While there is no single universally adopted standard, most Detection Engineering teams operate around a common life cycle. The exact stages and their names vary between organizations and frameworks, but the underlying logic is consistent:

Note that there are two feedback loops in the diagram above: if a detection fails testing, it goes back to design, not to deployment, with fingers crossed. And when attacker TTPs change significantly, the life cycle restarts from threat research.

Detection Life Cycle Frameworks

Some great detection engineers developed frameworks to bring structure to Detection Engineering and help the community. You don't need to memorize them, but you should know they exist to use as a reference to enhance your own pipeline.

• CREDO (opens in new tab) (Creation, Review, Evaluation, Deployment, and Operations): Breaks the detection life cycle into five explicit phases, emphasizing that detection work doesn't end at deployment. It pushes teams to treat ongoing evaluation and operations as first-class activities rather than afterthoughts.
• ADS (opens in new tab) (Alerting and Detection Strategy): Gives detection engineers a consistent template to follow every time they build a detection. It covers the detection goal, the ATT&CK technique it targets, the data sources it relies on, and the expected false positives — ensuring every rule is built and documented consistently so anyone on the team can understand, reproduce, and maintain it.

These frameworks all share a common premise: good detections are engineered, not improvised. The specific framework you use (if any) matters less than having a structured, smoother process.

Writing a detection is a technical task. Thinking like a detection engineer is something else entirely. The tooling and querying skills are learnable, but the mindset is what separates engineers who build high-quality detections from those who build detections that are easily bypassed. In this task, you will explore some concepts that build a strong detection engineering mentality.

The Detection Gap

There's an uncomfortable truth at the heart of detection engineering: you are always playing catch-up. Attackers move first. They research, develop, and deploy a technique, and only after it's observed in the wild does the defender community begin building detections and documenting it. By the time a detection is mature and widely deployed, a motivated attacker has often already moved on to a variation or an entirely different approach. This dynamic is visualized as the detection gap:

The gap between the evasion capability curve and the defender's detection curve never fully closes. A Detection Engineer's job is not to eliminate that gap; it's to keep it as narrow as possible. This leads to a classic question: Since defenders are always one step behind, will attackers win every time?

Detecting Everything

No, you will not detect everything, and you don't need to. If we revisit the concept of an attack chain, an attacker does not perform a single isolated technique; they execute a subset of techniques.

This means an attacker will use multiple known techniques in their attack chain. That's the opportunity the detection engineer has: attackers need to succeed at every step of a kill chain, while defenders only need to catch one to start investigating and acting. So, even detecting a minority of techniques can be sufficient to disrupt an attack.

Note in the diagram above that even known techniques can be bypassed by an attacker (Masquerading). This may happen due to different ways of implementing a technique, or because the DE team's strategy did not prioritize that specific detection (Step 0 in the life cycle).

Stealth is the attacker's primary goal, and regardless of your role in a SOC, you should know that. Every choice they make, from how they execute code to how they move laterally, is optimized to avoid generating alerts. They study the same detection logic that defenders publish. They test their tools against commercial EDRs. They know which event IDs are forwarded to SIEMs and which ones aren't. They deliberately operate in the margins of what looks normal.

Durable detections are built on what an attacker must do, the actions that are so fundamental to a technique that they can't be easily avoided without abandoning the technique entirely. This is the core insight behind the Pyramid of Pain. The higher up the pyramid you anchor your detections, from hash values and IP addresses at the base, up through tools and behaviors at the top, the more it costs the adversary to evade you. Detections built on TTPs are harder to break when an attacker rotates infrastructure or recompiles a binary. They break the attacker's operation. We will explore how to create and research durable detections in the Detection Rules Development room (coming soon).

Precision and Recall

Writing a detection that fires is easy. Writing one that fires correctly, reliably catching real threats without burying analysts in noise. When creating detections, you should always prioritize their performance, which can be challenging.

To reason about that challenge, detection engineers borrow two concepts from statistics: precision and recall.

• Precision: of all the alerts this detection fires, how many are actual threats? Low precision means analysts are drowning in false positives.
• Recall: of all the real attacks that occurred, how many did this detection catch? Low recall means threats are slipping through silently with no alert.

The missed ones, called false negatives, are the most dangerous outcome, because they produce no alert at all. No noise, no complaint, just a silent gap an attacker can walk through.

Keeping this tension consciously in mind, and measuring it over time through real SOC outcomes, not just gut feel, is what separates engineers who build detections that hold up from those who just ship queries and move on.

"The better you research and test a detection, the better your precision and recall values."

Detection Engineering is a discipline that looks clean on a diagram and gets complicated fast in practice. Most challenges aren't about writing detection logic; they're about everything surrounding it.

The Main Detection Challenges

These challenges are real, but they are not unique to your team; every Detection Engineering practitioner faces them. The difference lies in how much investment the team has to address them. A single analyst absorbing DE responsibilities alongside SOC work will feel all of these simultaneously. A mature team with dedicated roles, tooling, and processes will face the same challenges but with the resources to systematically tackle them one by one.

Two examples of how this plays out in practice:

• Tackling missing log sources starts with building a data inventory, which is a living document mapping every detection to its required log source and flagging gaps. Teams that invest in this early avoid the painful discovery of a broken detection after an incident. As the team matures, this process becomes automated: pipelines validate the availability of the log source before a detection is even deployed.
• Managing detection library decay starts with ownership. Assigning each detection a clear owner and a review date prevents orphaned rules. Mature teams go further, tracking detection performance metrics over time and automatically flagging rules that haven't fired in months or are generating disproportionate noise, making retirement decisions data-driven rather than guesswork.

The more the team grows and matures, the more these challenges shift from fires to managed processes.

Detection Engineering is the discipline that sits behind every alert your SOC depends on. Throughout this room, you explored what Detection Engineering is and where it lives in a security organization, how the detection life cycle structures the work from threat research all the way through to maintenance, the mindset required to build detections that hold up against real attackers, and the practical challenges every DE team faces regardless of size or maturity.
Before moving on, make sure you are comfortable with the following:

• The proactive mindset: Detection Engineers work ahead of threats, not in response to them. Every detection is built on research, tested before deployment, and maintained over time, not written once and forgotten.
• Precision and recall: A detection that fires on everything is as dangerous as one that fires on nothing. Understanding this trade-off is fundamental to building detections that add value rather than noise.
• The life cycle never truly ends: Deployment is not the finish line. Attacker techniques evolve, environments change, and detections degrade. Maintenance is as important as creation.

You now have the foundation. The next step is putting it into practice. Head over to the Detection Rules Development room (coming soon) to start building your first detections from scratch!