Agent Discovery

AI agents can analyse information, use tools, retrieve knowledge, and complete multi-step work. These capabilities make them useful for some problems, but not every difficult or repetitive task requires an agent.

A successful agent project begins before a model, framework, or tool is selected. The team must first understand the work, the people affected by it, and the result the organisation needs.

In this room, you will take the role of an AI agent engineer brought in to help NorthStar Fashion. The company is considering an AI agent that assists with security-alert investigations. Your responsibility is to examine the idea and determine which type of solution, if any, is appropriate.

You will not build the system in this room. You will recommend one of five practical paths:

• Build a traditional workflow or automation.
• Build an AI workflow.
• Build an AI assistant.
• Build an AI agent.
• Do not build a new system.

Learning Objectives

By the end of the room, your completed discovery work will include:

• Learn why agent discovery is necessary.
• Define NorthStar Fashion’s business problem.
• Map the current alert-investigation workflow.
• Separate the workflow into individual tasks and decisions.
• Determine which tasks suit a traditional workflow and which benefit from an LLM-based system.
• Distinguish between an AI workflow, AI assistant, and AI agent according to who controls the next step.
• Define what the proposed system can and cannot do.
• Make a final recommendation to solve NorthStar's problem.

Prerequisites

Before starting this room, you should be familiar with the following:

• A foundational understanding of AI and machine learning concepts, but no experience building AI agents or using agent frameworks is required (The Building Blocks of AI room)
• Basic familiarity with SIEM concepts and alert analysis, as the room uses a security-alert investigation scenario (Introduction to SIEM room)
• General knowledge of Security Principles, including why high-impact actions require clear authority and human oversight (Security Principles room)

Scenario

NorthStar Fashion is a small online boutique clothing company that sells its products through an e-commerce storefront. The company depends on a small technical environment to support its website, internal services, employee accounts, and customer-facing operations.

NorthStar Fashion does not have a dedicated security operations team. One engineer is responsible for security, infrastructure, and code.

In addition to maintaining the company's systems, the engineer reviews approximately 50 security alerts every day in a SIEM (Security Information and Event Management system).

Fifty daily alerts may be manageable for a dedicated security team. For one engineer with several other responsibilities, however, investigating them can consume a significant part of the working day.

The assigned severity is not always reliable, and an alert may change meaning when it is compared with external evidence or related events. Task 4 will examine these problems and their business impact in detail.

NorthStar Fashion is considering an AI agent that investigates alerts alongside the engineer. The company wants to reduce manual evidence gathering so that the engineer can focus on other work.

NorthStar Fashion has not yet decided what the system should contain or how autonomous it should be. It might become:

• A traditional workflow which uses small scripts or code.
• An AI assistant that answers investigation questions and prepares summaries.
• An AI workflow that follows a fixed investigation sequence.
• An AI agent that uses approved tools and documentation.
• A combination of these approaches.

If an agent is eventually selected, its tools must provide reliable feedback about the environment, and it must stop when it lacks enough evidence. Human checkpoints and clear stopping conditions prevent an open-ended investigation from continuing without control.

The proposed system must not close alerts independently, block IP addresses, disable accounts, change firewall rules, alter email settings, modify other security controls, or perform containment. The engineer will remain in the loop, review the supporting evidence, and make the final security decision.

Your Role

NorthStar Fashion has asked for your help to recommend an appropriate way forward. You must not assume that an agent is required simply because the company proposed one. Instead, you will examine the business problem and the existing workflow before making a recommendation.

Tasks 3 through 5 will produce the business problem and current workflow needed for the interactive exercise. In Task 6, you will classify the workflow tasks, decide whether they require an LLM, distinguish between an AI workflow, AI assistant, and AI agent, and define the system’s permitted and prohibited actions, then make the final recommendation.

The quality of your reasoning is more important than recommending an agent. Discovering that traditional automation, an AI assistant, or a process change is more appropriate is a valid outcome.

The terms workflow, assistant, and agent are sometimes used interchangeably. For discovery, it is more useful to distinguish them by who controls the next step and whether an LLM is involved. A traditional workflow is not an LLM-based system because it uses fixed code and rules without an LLM.

Traditional Workflow

A traditional workflow uses code, rules, scripts, or integrations without an LLM. The developer defines the sequence, and the same conditions produce the same actions.

For NorthStar Fashion, a traditional workflow could extract alert fields, compare timestamps, or query an IP reputation API using fixed logic. This could be via a simple script or code.

AI Workflow

An AI workflow combines an LLM with tools inside a predefined sequence controlled by code. The model may interpret notes or prepare a summary, but the developer decides the order of the steps and the conditions used to continue, branch, or stop.

For example, a predefined workflow could extract alert details, retrieve IP reputation, search historical notes, and then ask an LLM to produce a structured summary. Every investigation would follow that designed path.

AI Assistant

An AI assistant helps by answering queries. The engineer asks a question or requests a summary, and the assistant responds using the information available to it. The human decides what to ask, what to investigate next, and whether to accept the response.

An assistant can only answer questions and sometimes may use tools, but it does not independently control the complete investigation.

AI Agent

An AI agent is given a goal and allowed to dynamically decide how to proceed. Based on the evidence it receives, the model may choose which approved tool to use, when to look up documentation and internal knowledge, maintain memory, decide whether another search is needed, and adjust its plan.

The agent controls parts of the investigation process, but its authority must still be limited by permissions, human checkpoints, and stopping conditions.

An AI assistant describes a human-led interaction, while an AI workflow describes how multiple steps are orchestrated. An assistant may use a predefined workflow internally, so these approaches can be combined.

Anthropic describes AI workflows and AI agents under the broader term Agentic Systems, while distinguishing them by who controls the process. Its engineering guidance recommends starting with simple, composable approaches and adding complexity only when it creates a meaningful improvement. AI workflows provide greater consistency for well-defined tasks. AI agents are more suitable when the required steps cannot be predicted in advance, but they may increase latency, cost, and the possibility of errors accumulating across steps. You can read the Anthropic’s article on building effective agents (opens in new tab) to learn more about this.

NorthStar Fashion's alert investigation challenge has five possible paths: a traditional workflow, an AI workflow, an AI assistant, an AI agent, or no new system. You must now examine why discovery is required before one of those paths is selected.

An AI agent is not the goal of an agent project. The goal is to improve a real workflow in a useful, reliable, and safe way.

Agents can interpret unstructured information, use tools, and adapt their next steps as new evidence is discovered. These capabilities are valuable when work requires flexibility, but they also introduce uncertainty, cost, and operational risk.

Agent discovery is the investigation performed before deciding to build an agent. It helps a team determine whether agentic capabilities are necessary or whether a simpler solution would be more appropriate.

Start With the Problem

Consider the following statement: "NorthStar Fashion needs an AI agent to investigate security alerts."

This is a proposed solution, not a description of the problem. Here is the problem with it: it does not explain which part of the investigation is difficult, how the difficulty affects the company, or why an agent would improve it.

Starting with a preferred technology can cause a team to select models, frameworks, and tools before understanding the work. The result may be technically impressive while failing to address the real bottleneck.

Discovery reverses that order:

• Understand the business problem.
• Examine how the work is currently performed.
• Identify the repetitive, difficult, valuable, and risky tasks.
• Compare manual work, traditional workflows, AI workflows, AI assistants, and AI agents.
• Decide where human involvement is required.
• Recommend the simplest approach that can safely deliver value.

In Task 4, you will define NorthStar Fashion's business problem without assuming that an agent is the answer.

Repetitive Does Not Mean Agentic

The engineer's investigation work is repetitive, but repetition alone is usually a reason to consider a traditional workflow, not necessarily an AI agent.

Some tasks have structured inputs and stable rules. Extracting an IP address, comparing identifiers, or ordering events by timestamp can be implemented as a traditional workflow. These are deterministic tasks: the same type of input follows predefined rules to produce a predictable result.

Other tasks involve unstructured language or incomplete evidence. For example, an LLM could compare a previous incident record with a new alert and explain whether the earlier investigation provides useful context.

How the capability is delivered determines the system type. If code controls a predefined sequence, it is an LLM workflow. If the engineer requests the work and decides what happens next, it is an AI assistant. If the model chooses among approved next steps based on the evidence, it is an AI agent. All three use an LLM, but only the final approach gives the model control over the investigation path.

Agentic behaviour becomes relevant when the next step cannot always be selected in advance. For example, a system might need to choose another approved investigation tool because the first source returned insufficient evidence.

Discovery separates these kinds of work instead of treating the entire investigation as one large AI task.

Compare the Available Approaches

Agent discovery is not a choice between a human and a fully autonomous agent. A workflow may combine several approaches.

These approaches are not maturity levels. A fully autonomous agent is not automatically better than a script, search tool, or assistant.

The best design is the simplest one that can safely achieve the required result. A simpler component is usually easier to test, explain, operate, and maintain.

Why Agents Require Additional Scrutiny

A traditional or AI workflow follows paths defined in advance. An AI agent may be allowed to choose a tool, form a plan, or change direction based on information it receives.

This flexibility creates questions that must be answered before implementation:

• Which tools may the agent access?
• What information may it send to those tools?
• How should it handle an unavailable or untrusted source?
• How can the engineer verify the evidence behind a recommendation?
• When must the agent stop and request human input?
• What happens if a tool returns incorrect or contradictory information?
• What is the operational impact if the agent's conclusion is wrong?

These questions matter because a plausible answer is not necessarily a correct answer. LLMs are good at sounding confident even when their evidence is incomplete or wrong.

The more authority a system receives, the greater the consequence of an error. A system that prepares an investigation summary has a smaller impact than one that closes alerts, blocks IP addresses, or disables accounts.

For NorthStar Fashion, investigating and acting must therefore remain separate. The proposed system may support an investigation, but the engineer remains responsible for the final decision.

What Discovery Prevents

Skipping discovery creates several common failure modes.

• Automating the wrong bottleneck: rewriting an alert description would save little time if the engineer still had to search every source manually.
• Using AI where rules are sufficient: a model would add variability and cost to tasks that structured code could perform reliably.
• Assuming the data is ready: existing documentation may be incomplete, inconsistent, outdated, or difficult to retrieve.
• Failing to define improvement: if the team cannot describe what should become better, it cannot determine whether the project succeeded.

Discovery makes these risks visible while the solution can still be changed cheaply. It prevents the team from applying the same technology to every step merely because the complete process has been labelled an agent project.

Valid Discovery Outcomes

The final recommendation will be to either:

• Build a traditional workflow or automation.
• Build an AI workflow.
• Build an AI assistant.
• Build an AI agent.
• Continue with or improve the existing process without building a new system.

Choosing a traditional workflow, AI workflow, AI assistant, or no-build outcome is not a failed agent project. It means discovery prevented unnecessary complexity and directed the company towards a more appropriate response.

You will now turn NorthStar Fashion's request into a clear business problem. The aim is not to decide between a traditional workflow, LLM workflow, AI assistant, AI agent, or no new system. That decision comes later.

By the end of this task, you will have a business problem statement supported by evidence from the current situation.

Begin With the Request

NorthStar Fashion's initial request can be summarised as: Build an AI agent to help investigate security alerts.

This tells us what the company is considering, but not what the company needs. It does not identify the most difficult part of the investigation, the effect on the business, or the result that should improve.

A discovery conversation should instead ask who performs the investigation, which parts consume the most effort, what evidence is required, what happens when context is missed, what must remain under human control, and how NorthStar Fashion would recognise an improvement.

These questions keep the investigation focused on operational value rather than a preferred technology.

Examine the Evidence

The business problem must be supported by evidence from the current situation. NorthStar Fashion has already provided several useful observations.

Alert Volume Meets Limited Capacity

The engineer reviews approximately 50 alerts per day while also maintaining security, infrastructure, and code. The volume may be manageable for a dedicated security team, but NorthStar Fashion has only one engineer performing all of these responsibilities.

The alert count is therefore relevant because of the company's limited capacity, not because 50 is automatically a high number for every organisation.

High Severity Can Be a False Positive

The engineer has reported cases where the SIEM classified repeated network scanning as high severity. After checking previous documentation, the engineer found that the source belonged to an approved vulnerability scanner.

The alert appeared dangerous without context, but the assigned severity overstates the real risk.

Low Severity Can Hide Real Risk

The opposite has also happened. An alert was assigned low severity, but an IP reputation check showed that its source address was associated with malicious activity.

This shows that the engineer cannot rely on SIEM severity alone and must gather evidence from another source.

Related Alerts Can Change the Meaning

Consider two alerts involving the same user:

• A login occurs from an unusual location.
• Ten minutes later, an external mail-forwarding rule is created.

Each event may appear low or medium risk by itself. Together, they may indicate an account compromise and an attempt to redirect future email.

The engineer must therefore compare users, IP addresses, devices, timestamps, and behaviours across alerts rather than investigate every event in isolation.

Evidence Is Distributed

The engineer may need the SIEM alert, IP reputation results, related events, previous incident documentation, and personal investigation notes before reaching a conclusion.

The problem is not simply that evidence exists. The problem is that the engineer must find, compare, and interpret it manually while handling other responsibilities.

Separate Symptoms, Causes, Impacts, and Solutions

Discovery becomes clearer when each statement is placed in the correct category.

"NorthStar Fashion receives too many alerts" is incomplete because it focuses only on volume. Hiding or discarding alerts could reduce the count without making investigations safer.

"NorthStar Fashion does not have an AI agent" is also not a business problem. It describes missing technology rather than an operational need.

The underlying problem is that the engineer must repeatedly gather and interpret fragmented evidence before alerts can be prioritised with confidence. That is the real problem NorthStar is facing.

Business Problem Statement

The evidence can now be combined into a business problem statement:

NorthStar Fashion's only engineer must manually gather and interpret evidence from several sources while reviewing approximately 50 security alerts each day. SIEM severity may misrepresent the real risk, and related events may change an alert's meaning. This repetitive investigation takes time away from the engineer's other security, infrastructure, and development responsibilities.

The statement describes the stakeholder, difficulty, cause, and operational impact. It does not prescribe a traditional workflow, LLM workflow, AI assistant, or AI agent.

In the next task, you will map the current investigation workflow and verify where this problem appears within the individual steps.

Task 4 identified NorthStar Fashion's business problem: its only engineer must repeatedly gather and interpret fragmented evidence before security alerts can be prioritised with confidence.

The next step is to understand exactly how that work is performed today. A business problem explains what is wrong; a workflow map explains where and how the problem occurs.

In this task, you will map the current human-led investigation. You are not designing an AI agent or improved process yet.

Why Map the Current Workflow?

The phrase "investigate an alert" makes the work sound like one task. In practice, it contains several actions, searches, comparisons, and decisions.

If NorthStar Fashion treats the investigation as one large task, it may apply the same technology to every step. Mapping the workflow reveals which steps are predictable, which depend on context, and where the engineer must make a judgement.

A useful current-state map records the trigger, order of work, information and tools used, output of each step, decisions that change the path, repeated work, and the point at which the workflow ends.

This process is sometimes called workflow archaeology because some important steps may exist only in the engineer's habits or personal notes rather than in an official procedure.

Set the Workflow Boundary

A workflow map needs a clear beginning and end. Without a boundary, the map can expand into unrelated security operations.

For this exercise:

• Start: A SIEM alert is available for the engineer to review.
• End: The engineer records an investigation conclusion and decides how the alert should be prioritised or handled next.

Containment actions such as blocking an IP address, disabling an account, or changing security controls are outside this workflow. NorthStar Fashion has already stated that the proposed system must not perform those actions.

The map describes what the engineer currently does, including inefficient or repetitive work. Do not replace a manual step with a proposed tool while mapping it.

Step 1: Review the SIEM Alert

The workflow begins when the engineer opens an alert in the SIEM.

The available information may include the alert type, timestamp, assigned severity, user, IP address, device, destination, and event description. The exact fields depend on the alert.

The engineer reviews the alert and notes the details that can guide the investigation, such as the user, IP address, device, and timestamp. These details are then used to search for supporting evidence in other sources.

Step 2: Decide What Evidence Is Needed

Not every alert contains the same information. An alert may contain a source IP address, a user account, a device, or several of these identifiers.

The engineer decides which information can help verify the alert. If a source IP is available, its reputation can be checked. If a user or device is available, related activity can be searched. If useful identifiers are missing, the engineer must work with the remaining context.

The investigation path therefore depends on the contents of the alert.

Step 3: Check IP Reputation

When an alert contains an external source IP, the engineer checks it using an IP reputation service.

The result may show reported malicious activity, no known abuse, or insufficient information. This evidence must be compared with the SIEM alert rather than treated as a final answer.

An IP with no known abuse history is not automatically safe, and a reported address still requires context.

Step 4: Search Previous Documentation

The engineer searches previous incident documentation and personal investigation notes for matching users, IP addresses, devices, alert types, or behaviours.

Historical context may explain that an address belongs to an approved scanner or that similar activity was previously investigated. The records may also be incomplete, outdated, or written using different terms.

The output of this step is not simply a matching document. The engineer must decide whether the record is relevant to the current alert.

Step 5: Look for Related Alerts

The engineer searches the SIEM for other alerts involving the same user, IP address, device, or time period.

This step is necessary because separate events may appear low risk until they are connected. An unusual-location login followed by an external mail-forwarding rule is one example.

The engineer places potentially related events into a timeline and checks whether their identifiers and behaviours form a meaningful sequence.

Step 6: Compare and Correlate the Evidence

The engineer now compares the original alert with the IP reputation result, historical documentation, personal notes, and related events.

The evidence may agree, conflict, or remain incomplete. For example, a high-severity alert may match an approved scanner IP, while a low-severity alert may involve a malicious IP address.

This is the most judgement-heavy part of the current workflow. The engineer must decide which evidence is relevant and what the combined information means.

Step 7: Make the Investigation Decision

Once the available evidence has been reviewed, the engineer decides whether the alert can be deprioritised, requires further investigation, or should be escalated.

If the evidence is insufficient or contradictory, the engineer may return to an earlier step, perform another search, or seek additional context. The workflow is therefore not always a straight line.

The engineer remains accountable for the decision.

Step 8: Record the Outcome

The engineer records the conclusion and the evidence used to reach it. This may include why an alert was considered expected activity, why it remains suspicious, or which related events changed its priority.

The record becomes a note or is documented as a context for future investigations. Poor or incomplete documentation at this stage can create problems when a similar alert appears later.

The current workflow ends when the conclusion has been recorded and the next handling priority has been decided.

Current-State Workflow Map

The main path can be represented as follows:

Build the Task Map

Convert the workflow into individual tasks before evaluating possible solutions.

Do not label a task as traditional, LLM-based, assistant-led, or agentic yet. Task 6 will evaluate the tasks after the current workflow has been captured.

Identify Bottlenecks and Decision Points

The map reveals several areas that deserve later evaluation. The engineer moves between several sources, SIEM severity requires context, historical records require a relevance judgement, and related alerts must be correlated across identifiers and time. Missing or contradictory evidence may repeat earlier steps, while the final conclusion requires human accountability.

These observations describe the workflow; they do not prove that an AI agent is required.

In the next task, you will enter this map into the Agent Discovery Canvas, evaluate task frequency, complexity, and risk, apply the Human Agency Scale, and compare the five solution paths.

Click the View Site button at the top of the task to launch the static site and complete the activity. Work through each stage to identify the real business problem, build the current alert-investigation graph, and classify which tasks suit a traditional workflow or require an LLM-based system. You’ll then need to distinguish between an AI workflow, AI assistant, and AI agent according to who controls the next step, and define what the proposed system can and cannot do.

Throughout this room, you moved from NorthStar Fashion’s proposed solution to an evidence-based recommendation. You distinguished traditional workflows, AI workflows, AI assistants, and AI agents; identified the real business problem; mapped the current alert-investigation process; and used the Agent Discovery Canvas to evaluate its individual tasks. The result is a recommendation for a controlled investigation-support AI agent that can gather approved evidence, correlate alerts, prepare summaries, and suggest the next investigation step, while the engineer retains responsibility for the final decision and all containment actions.

Key Takeaways

• Agent discovery starts with the problem, not the proposed agent; understanding the business need, the people affected, and the current workflow prevents teams from automating the wrong bottleneck or adding AI where a simpler solution would work better.
• Who controls the next step determines the type of system; fixed rules belong in traditional automation, code-controlled LLM steps form an AI workflow, engineer-led interactions form an AI assistant, and evidence-dependent decisions made by the model create an AI agent.
• Investigation support and security action must remain separate; a controlled agent may gather evidence, correlate alerts, prepare summaries, and recommend the next step, but clear boundaries, stopping conditions, and human review keep final decisions and containment actions with the engineer.