Skip to main content
Back to all walkthroughs
Room Icon

Agent Design

Design a safe AI agent by defining its scope, tools, oversight, stop conditions, and output.

easy

45 min

55

User profile photo.
User profile photo.
User profile photo.

To access material, start machines and answer questions login.

In the previous room, you completed Agent Discovery for NorthStar Fashion. You examined the business problem, mapped the current alert investigation workflow, considered human oversight, and compared possible solution paths.

This room moves from discovery to design.

NorthStar Fashion is now exploring a controlled -assisted investigation system for security alerts. The system should help the engineer gather evidence, correlate context, prepare summaries, and recommend investigation priority.

However, the system must not replace the engineer’s accountability. It must not block IP addresses, disable accounts, close alerts, modify controls, delete evidence, or perform containment.

The goal of this room is to define the agent design before implementation begins.

You will decide what the agent should do, what it must not do, what tools it may use, what context and state it needs, when it should stop, and where human review is required.

Learning Objectives

  • Convert discovery findings into design requirements.
  • Define an agent role, goal, and workflow boundary.
  • Identify allowed and disallowed agent actions.
  • Select appropriate tools for an investigation-support agent.
  • Define context, state, memory, and output requirements.
  • Add human review points and stop conditions.

Prerequisites

  • Completed the Agent Discovery room, as this room builds directly on its NorthStar Fashion scenario, workflow map, human oversight analysis, and final recommendation.
Answer the questions below

I'm ready to learn!

discovery and design stages

Before an agent is implemented, it needs a clear design.

Agent design is the process of defining how an agent should behave before any code, model, framework, or tool integration is selected. A good design explains what the agent is responsible for, what it is allowed to access, what it must never do, how it should handle uncertainty, and where human oversight is required.

This step is important because agents can make decisions during execution. They may choose tools, interpret evidence, update state, follow branches, and continue working across several steps. Without a clear design, this flexibility can become unsafe or unpredictable.

For example, an alert investigation agent that is only meant to support evidence gathering should not accidentally be designed with permission to block IP addresses, disable accounts, close alerts, or modify security controls. Those are containment actions, and they require human accountability.

Design turns a broad idea like this:

Build an agent to investigate alerts.

Into a safer and more specific specification:

Build an -assisted investigation agent that gathers approved evidence, correlates context, prepares a structured summary, recommends an investigation priority, and sends the result to the engineer for review.

The second version is better because it defines the agent’s purpose, boundaries, output, and human review requirement.

Why Agent Design Matters

A vague agent design creates risk.

If the role is unclear, the agent may be expected to do too much. If the tool boundary is too broad, the agent may gain access to actions that should stay under human control. If state is not defined, it becomes harder to understand what evidence was checked or why a recommendation was made. If stop conditions are missing, the agent may continue when it should ask for help.

A strong agent design helps answer questions such as:

  • What is the agent trying to achieve?
  • What actions are allowed?
  • What actions are disallowed?
  • Which tools can the agent use?
  • What information should be provided as context?
  • What information should be tracked in state?
  • What memory rules are needed?
  • When should the agent ask for human review?
  • When should the agent stop?
  • What should the final output look like?

These questions reduce ambiguity before implementation begins.

The goal is not to make the agent as autonomous as possible. The goal is to design the simplest agent behaviour that safely supports the workflow.

The Design Methodology

This room breaks the agent specification into five parts. Each part focuses on one design area and one design principle that helps keep the agent useful, controlled, reviewable, and safe.

The main design principle is that the agent should support the investigation without exceeding its authority. The specification should define the agent’s role, tools, information flow, human review points, stop conditions, and final output before implementation begins.

Each component will be introduced in a separate task, together with the design principle it represents, before you complete the final Agent Spec Builder challenge. You will begin by defining the agent’s role and scope, followed by its tool boundary. Next, you will determine the context, state, and memory it requires, add human review points and stop conditions, and finally define its structured output.

These principles will help you decide which design choices are safe, which are incomplete, and which should be rejected.

Design the agent before building it

Answer the questions below

What defines agent behaviour before implementation?

What do these questions reduce before implementation begins?

What can a vague agent design create?

The role and scope of an agent

Before selecting tools, memory, or output fields, an agent needs a clear role and scope.

  • The role describes what the agent is responsible for.
  • The scope defines where that responsibility begins and ends.

This matters because vague agent roles can create unsafe expectations. If an agent is described as “investigating alerts”, that could mean many different things: Does it only gather evidence? Does it recommend a priority? Does it close alerts? Does it block IP addresses? Does it disable accounts?

A safe designA safe design removes that ambiguity before implementation begins.

For NorthStar Fashion, the agent should support the engineer during alert investigations. It may help gather approved evidence, correlate context, prepare a structured summary, and recommend an investigation priority for human review.

It must not make the final decision or perform containment actions.

Why Role and Scope Matter

A well-scoped agent is easier to build, test, evaluate, and secure.

If the role is too broad, the agent may be given tools or authority that exceed the actual need. This can lead to unsafe behaviour, such as modifying systems, closing alerts automatically, or making decisions without human accountability.

A good agent role should answer four questions:

  1. What is the agent helping with?
  2. What information can it use?
  3. What output should it produce?
  4. What decisions remain with a human?

For this room, the agent should be designed as an investigation-support agent rather than a containment agent.

That means the agent can support the process, but it does not control the final security outcome.

Good Role Definition

A safe role definition for this scenario could be:

  • The agent assists the engineer by gathering approved evidence for a security alert.
  • Correlating relevant context.
  • Preparing a structured investigation summary.
  • Recommending a priority for human review.

This role is strong because it defines:

  • Purpose: assist the engineer
  • Evidence boundary: approved evidence
  • Main activity: correlate relevant context
  • Output: structured investigation summary
  • Decision boundary: recommendation for human review

The agent is useful, but its authority is limited.

Poor Role Definition

A poor role definition would be:

The agent investigates and handles security alerts automatically.

This is unsafe because it is vague and gives the agent too much authority.

An agent with too much authority

The phrase “handles security alerts automatically” could suggest that the agent can close alerts, escalate incidents, block IP addresses, disable accounts, or perform containment actions without human review - all of which fall outside the intended scope.

Allowed Agent Responsibilities

For NorthStar Fashion, the agent may:

  • Read alert details.
  • Extract relevant entities such as users, IP addresses, devices, and timestamps.
  • Query approved evidence sources.
  • Search previous investigation notes.
  • Find related alerts.
  • Correlate evidence.
  • Prepare a structured investigation summary.
  • Recommend an investigation priority for human review.
  • Ask for human input when evidence is missing or contradictory.

These responsibilities support the investigation without giving the agent control over security actions.

Disallowed Agent Responsibilities

The agent must not:

  • Block IP addresses.
  • Disable accounts.
  • Close alerts automatically.
  • Modify , , email, or controls.
  • Delete or alter evidence.
  • Invent missing evidence.
  • Continue indefinitely when evidence is insufficient.
  • Present a recommendation without supporting evidence.
  • Make the final security decision.

These actions are outside the agent’s scope because they involve containment, system modification, unsupported reasoning, or final accountability.

The engineer remains responsible for deciding how the alert should be handled.

Scope Boundary

The agent’s workflow boundary should be clear.

For this room, the boundary is:

  • Start: A security alert is available for review.
  • End: The agent produces a structured investigation summary and priority recommendation for the engineer.

The workflow does not include containment or remediation.

Actions such as blocking an IP address, disabling an account, resetting a password, changing a rule, or closing an alert should happen outside the agent’s authority.

A safe design keeps investigation support and containment separate.

Design Principle: Scope Boundary

The role and scope should follow this principle:

The agent supports the investigation, but the human owns the decision.

This principle should guide every remaining design decision. The selection of tools, context, state, memory, stop conditions, and output fields must preserve the same boundary: if any choice allows the agent to act beyond investigation support, it should be rejected.

Answer the questions below

What describes what the agent is responsible for?

What should happen to a design choice if it allows the agent to act beyond investigation support?

An agent’s tools define what it can actually do. While a prompt describes the agent’s role, its tools determine its real capabilities. If the agent has access to a dangerous tool, the system may be only one bad decision, prompt, input, or routing step away from unsafe behaviour.

For that reason, tool design is not just an engineering decision. It is also a security boundary.

In this task, you will review a proposed tool catalogue for NorthStar Fashion’s -assisted investigation agent. Your goal is to decide which tools belong inside the agent boundary and which tools must stay outside it.

The Tool Boundary Review

NorthStar Fashion’s engineer wants the agent to help with alert investigations.

The agent should be able to:

  • Read alert details.
  • Collect approved evidence.
  • Search previous notes.
  • Find related alerts.
  • Summarise findings.
  • Recommend a priority for review.

However, the agent must not perform containment or modify systems.

This means the tool boundary should separate investigation-support tools from action-taking tools.

A useful way to think about this is:

  • Can this tool help the agent understand the alert?
    • It may belong inside the agent boundary.
  • Can this tool change the environment or affect a user, system, alert, or control? 
    •  It should stay outside the agent boundary.

The safest agent is not the one with the most tools. The safest agent is the one with the minimum tools required to complete its role.

Candidate Tool Catalogue

The development team proposes the following tool catalogue:

Tool Description
get_alert_details Retrieves alert fields from the SIEM
check_ip_reputation Checks an approved IP reputation source
search_previous_notes Searches previous investigation notes
find_related_alerts Finds related alerts by user, IP, device, or time
disable_account Disables a user account
close_alert Closes an alert in the SIEM

At first glance, all of these tools are security-related. That does not mean they all belong in the agent.

The question is not:

Is this tool useful for security?

The better question is:

Is this tool safe and necessary for an investigation-support agent?

Investigation-Support Tools

Some tools help the agent gather or organise evidence without changing the environment.

These tools fit the agent’s role:

  • get_alert_details
  • check_ip_reputation
  • search_previous_notes
  • find_related_alerts

These tools are appropriate because they support investigation. They help the agent answer questions such as:

  • What happened?
  • Which entities are involved?
  • Has this IP appeared before?
  • Are there related alerts?
  • Is there previous context?

They do not directly contain, remediate, or modify anything.

That makes them safer candidates for this agent.

Action-Taking Tools

Other tools cross the boundary from investigation into response.

These tools should not be available to the agent:

  • disable_account
  • close_alert

These actions may be valid during a real incident response process, but they should not be performed by this agent.

disable_account affects a user and could interrupt business operations.

close_alert changes the investigation record and could hide a real issue if the agent is wrong.

Both actions require human accountability: an engineer may choose to perform them later, but the agent should not have direct access to them.

Design Principle: Minimum Safe Access

A safe tool boundary follows this principle:

Give the agent enough access to gather evidence, but not enough access to take irreversible or high-impact action.

For NorthStar Fashion, that means the agent can retrieve and summarise investigation evidence, but it cannot act on the environment.

This separation is important because the agent may be wrong, a tool may return incomplete data, or the available evidence may be contradictory. If the agent only gathers evidence, mistakes are easier to catch during human review. If the agent can act directly, the same mistake may cause operational impact.

Tool Design Checklist

Before approving a tool for an agent, ask these questions:

  • Does the tool support the agent’s role?
  • Does the tool have a clear and narrow purpose?
  • Does the tool return useful evidence?
  • Can the tool change systems, users, alerts, or controls?
  • Could misuse of the tool cause business impact?
  • Should a human approve this action instead?

A tool should be rejected or moved outside the agent boundary if it can modify systems, affect users, delete evidence, close records, or perform containment.

Safer Alternative Pattern

Rejecting an action-taking tool does not mean ignoring the action completely; instead of giving the agent a tool like disable_account, the design can allow it to recommend: 

Human review required: account compromise indicators were found. The engineer should review whether account containment is needed.

Similarly, instead of giving the agent a tool like close_alert, the design can allow it to recommend: 

Recommended priority: low. Evidence suggests this may be expected scanner activity. Engineer review required before closure.

This pattern keeps the agent useful while preserving human control: the agent can recommend, but the engineer decides.

Tool Boundary Decision

For this room, the approved tool boundary is:

Inside the agent boundary Outside the agent boundary
get_alert_details disable_account
check_ip_reputation close_alert
search_previous_notes  
find_related_alerts  

This boundary gives the agent enough capability to support the investigation while preventing it from performing containment or changing investigation records.

Agent tool boundaries

Answer the questions below

What extends agent capability?

Should the agent be able to modify systems (yea/nay)?

The right information at the right time

An agent’s design is not limited to what it can do. It must also define what the agent can see, what information it tracks during execution, and what it remembers from previous work. These three concepts are often confused:

  • Context: information available during the current run.
  • State: information tracked as the workflow progresses.
  • Memory: information retained or retrieved beyond the current run.

For a security-alert investigation agent, this distinction matters. The agent may need enough information to support the investigation, but it should not receive unnecessary sensitive data, hide important workflow details, or treat old notes as automatically correct.

A safe design gives the agent the right information at the right time, while keeping the investigation traceable and reviewable.

Context: What the Agent Sees Now

Context is the information provided to the model during a specific run. For NorthStar Fashion, this may include the current alert details, extracted entities, approved evidence results, relevant previous notes, related alerts, and any human instructions for the investigation. Good context helps the agent understand the task, but excessive or poorly selected context can introduce risk. Unrelated employee data, unnecessary logs, or stale notes may cause the agent to rely on information that is irrelevant, sensitive, or misleading.

For this room, context should be limited to the information required to support the current investigation. A safe context design may include:

  • Current alert details.
  • Extracted entities.
  • Approved evidence results.
  • Relevant previous notes.
  • Related alert timeline.
  • Human instructions for the current investigation.

A poor context design may include:

  • Unrelated employee data.
  • All historical alert data.
  • Raw system logs not needed for the investigation.
  • Unverified information without limitations.

The goal is not to give the model everything. The goal is to give it enough relevant information to produce a useful and reviewable result.

State: What the Workflow Tracks

State is the structured information carried through the workflow while the agent runs.

In a simple prompt-response workflow, state may not be necessary. However, investigation workflows usually involve multiple steps. The agent may retrieve alert details, extract entities, check reputation, search notes, find related alerts, and then produce a recommendation.

Without state, it becomes difficult to answer basic questions:

  • Which evidence was checked?
  • Which tools returned results?
  • What information was missing?
  • Why did the agent recommend human review?
  • What confidence level was assigned?

A safe agent design should make this information visible.

For NorthStar Fashion, useful state fields may include:

alert_id alert_type confidence device evidence_checked
final_summary missing_evidence recommended_priority related_alerts requires_human_review
severity source_ip timeline timestamp user

State makes the workflow easier to debug, test, audit, and explain by showing which evidence influenced each decision. For example, if the agent recommends a high priority, the engineer should be able to trace that recommendation back to the supporting evidence. A design that fails to track evidence, confidence, missing information, or human-review requirements creates hidden state and may produce outputs that appear confident without being adequately supported.

Memory: What Can Be Reused Later

Memory is information retained or retrieved beyond a single run. It can support an investigation by revealing that an IP address belongs to an approved scanner, that a user has triggered similar alerts before, or that an earlier investigation identified benign activity. However, because historical records may be incomplete, outdated, or written using different terminology, they should be treated as supporting context rather than absolute truth.

For this room, memory should be designed as controlled retrieval from approved previous notes, not unlimited long-term storage.

A safe memory design may allow:

  • Search approved previous investigation notes.
  • Retrieve records relevant to the current alert.
  • Show the source of retrieved notes.
  • Mark historical notes as supporting context.
  • Track limitations when notes may be stale or incomplete.

A poor memory design may allow:

  • Store all alert data indefinitely.
  • Use unrelated employee information.
  • Treat previous notes as always correct.
  • Retrieve data without source visibility.
  • Use memory without showing limitations.

The agent should use memory carefully. Previous notes can help explain context, but the current evidence still needs to be reviewed.

Together, context, state, and memory ensure that the agent has relevant information, tracks its investigation clearly, and uses previous records only as supporting evidence.

Design Principle: Evidence Visibility

The context, state, and memory design should follow this principle:

Give the agent enough information to support the investigation, but keep the evidence trail visible.

The agent should receive only relevant data, maintain visible state, and treat memory as supporting context rather than automatically trusted information. Every important recommendation should be traceable to the current context, recorded state, or retrieved evidence.

Context, design, and state for IA agents

Answer the questions below

What tracks workflow progress?

What information is available to the model during a specific run?

Human oversight and stop conditions

A safe agent design defines not only what the agent can do, but also when it must stop. This is especially important in security workflows, where an investigation-support agent may gather evidence, compare signals, and recommend a priority, but should stop when evidence is missing, contradictory, or outside its scope. At NorthStar Fashion, the engineer remains accountable for the final decision: the agent supports the investigation without replacing human judgement. Human oversight and clear stop conditions therefore keep the agent within its intended boundary.

The Autonomy Problem

Agents are useful because they can make decisions during execution - such as selecting a tool, inspecting its result, updating state, and determining the next step - but that same flexibility also creates risk.

If an agent is not given clear limits, it may:

  • Keep retrying when evidence is unavailable.
  • Treat missing evidence as safe.
  • Ignore contradictory results.
  • Produce a confident answer without support.
  • Recommend actions outside its authority.
  • Continue working outside the original scope

A well-designed agent should not attempt to complete a task at any cost; when evidence is insufficient, risk is too high, or the situation requires human judgement, the safest action is to stop and request human review.

What Human Oversight Means

Human oversight

Human oversight means that a person remains responsible for reviewing the agent’s output and making the final decision. This does not make the agent less useful: it can still reduce manual effort by collecting evidence, organising context, identifying missing information, and preparing a structured summary. The key distinction is that the agent supports the decision-making process, while the engineer retains ownership of the decision.

For NorthStar Fashion, the agent may say:

Related alerts and reputation data suggest this alert may require higher priority. Human review is required before any response action.

But it must not say:

This is malicious. The account has been disabled.

The first response supports the engineer’s decision-making, while the second performs - or implies - the kind of action that falls outside the agent’s intended scope.

Human Review Points

A human review point is a condition that requires the engineer’s involvement before the workflow can continue or the agent’s output can be accepted. For this design, human review should be required when:

  • Evidence is missing.
  • Evidence is contradictory.
  • The alert may indicate account compromise.
  • The recommended priority is high or critical.
  • The agent has low confidence.
  • A required source is unavailable.
  • The agent cannot explain its recommendation

These conditions matter because unsupported or automated conclusions could create security or business risk. For example, if an IP reputation source is unavailable, the agent should not assume the IP is safe; it should record the evidence as missing and require human review. Similarly, if related alerts indicate a possible account compromise, the agent should escalate the finding to the engineer rather than disabling the account itself.

Stop Conditions

A stop condition tells the agent when it must stop the workflow instead of continuing automatically. Stop conditions are different from human review points, but they often work together.

A human review point says:

A person must review this.

A stop condition says:

The agent must not continue past this point.

For NorthStar Fashion, the agent should stop when:

  • Required evidence cannot be retrieved.
  • Repeated tool errors occur.
  • Evidence is contradictory.
  • The request is outside scope.
  • The maximum retry limit is reached.
  • Human review is required.
  • The agent lacks enough evidence to support a recommendation

This prevents the agent from guessing, looping indefinitely, or producing a recommendation that appears more certain than the evidence allows.

Retry Limits

Retries can help recover from temporary tool failures, such as a timeout during an IP reputation lookup, but they must have clear limits. Without a retry limit, the agent may waste time, increase costs, fill logs with repeated failures, and delay the engineer from reviewing the alert manually.

A safer pattern is:

Try the approved tool. If the tool fails, retry a limited number of times. If the tool still fails, mark the evidence as unavailable. Stop or request human review.

The agent should never retry forever.

Uncertainty Is a Valid Outcome

An unsafe design assumes that the agent must always produce a final answer, whereas a safer design allows it to acknowledge when the available evidence is incomplete.

For example:

The IP reputation source could not be reached. Related alerts were found, but the reputation evidence is missing. Human review is required before assigning final priority.

This is safer than inventing a conclusion or treating an unavailable source as evidence that no risk exists. In security workflows, uncertainty should remain visible, so the final output should clearly identify missing evidence, known limitations, the confidence level, and any human-review requirements.

Safe vs Unsafe Behaviour

Situation Unsafe Behaviour Safer Behaviour
IP reputation source unavailable Treat the IP as safe Mark evidence as missing
Related alerts conflict Pick one result without explanation Require human review
High-risk recommendation Automatically act Send for engineer review
Tool keeps failing Retry indefinitely Stop after retry limit
Scope is unclear Continue anyway Stop and request clarification
Evidence is weak Produce confident conclusion Include limitations

The agent should be designed to fail safely. A safe failure is one where the agent stops, explains what happened, and asks for review instead of continuing with unsupported assumptions.

Design Principle: Safe Stop

Human oversight and stop conditions should follow this principle:

The agent may support investigation, but it must stop when the evidence no longer supports safe continuation.

This principle keeps the agent’s autonomy within clear boundaries: it may collect and organise information, but it must not continue blindly, conceal uncertainty, or make final security decisions.

Collect, organise information, and safe stop

Answer the questions below

Who owns the final decision?

Should agents retry forever (yea/nay)?

What should missing evidence trigger?

An investigation-support agent should not return only an unrestricted paragraph. It should produce a consistent, structured review package that shows what it found, which evidence it used, what information is missing, and why it made its recommendation.

For example, an unstructured response might say:

"This alert is probably malicious and should be escalated".

This sounds useful, but it leaves important questions unanswered. The engineer cannot easily see which sources were checked, whether any tools failed, what evidence supports the conclusion, or what the word "probably" means.

A structured output separates these details into defined fields. This makes the result easier for the engineer to review and allows the system to validate that required information is present before accepting the output.

State and Output Are Different

State and output may contain some of the same information, but they serve different purposes.

  • State is the working record maintained while the investigation runs. It may include extracted entities, intermediate tool results, retry counters, missing evidence, and the next planned step.
  • Output is the final review package handed to the engineer. It should contain the evidence, limitations, recommendation, and review status needed to understand the result without exposing unnecessary internal details.

The agent may use state to reach its result, but the final output must be designed as a separate and reliable contract.

Required Output Fields

For NorthStar Fashion, the structured output can contain the following fields:

Field Purpose
alert_id Identifies the alert being investigated
run_status Shows whether the investigation completed or stopped safely
summary Provides a concise description of the findings
extracted_entities Lists relevant users, IP addresses, devices, and timestamps
evidence Records findings together with their sources and limitations
related_alert_timeline Shows related events in time order
contradictions Identifies evidence that does not agree
missing_evidence Identifies evidence that could not be obtained
recommended_priority Provides a suggested investigation priority or undetermined
recommendation_rationale Explains how the evidence supports the recommendation
requires_human_review Makes the mandatory engineer review visible
stop_reason Explains why the agent stopped when it could not complete normally

Each evidence item should also record its provenance. At minimum, it should show the source or tool, the finding, when the information was retrieved, and any known limitation.

For example:

{
  "source": "check_ip_reputation",
  "finding": "The source IP has recent malicious reports",
  "retrieved_at": "2026-07-24T09:30:00Z",
  "limitations": []
}

This is more reviewable than placing the finding into a summary without identifying where it came from.

Use Controlled Values

Some output fields should use a small set of allowed values. Controlled values prevent the agent from inventing inconsistent labels such as "fairly urgent" or "medium-high".

For this design, run_status should be one of:

  • ready_for_review - The investigation completed and the review package is ready for the engineer.
  • needs_human_input - The agent requires information or a decision before it can continue.
  • insufficient_evidence - Bounded evidence gathering finished without enough support for a recommendation.
  • tool_failure - A required evidence source remained unavailable after the retry limit.
  • out_of_scope - The request falls outside the investigation-support role.

recommended_priority should be one of:

  • low
  • medium
  • high
  • critical
  • undetermined

The agent must use undetermined when the available evidence does not support a priority recommendation. It must not guess simply to complete the field.

Example Completed Output

The following example shows a completed investigation ready for engineer review:

{
  "alert_id": "ALERT-1042",
  "run_status": "ready_for_review",
  "summary": "The source IP has a malicious reputation and appears in related alerts involving the same user",
  "extracted_entities": {
    "users": ["user-17"],
    "source_ips": ["203.0.113.24"],
    "devices": ["device-08"]
  },
  "evidence": [
    {
      "source": "check_ip_reputation",
      "finding": "The source IP has recent malicious reports",
      "retrieved_at": "2026-07-24T09:30:00Z",
      "limitations": []
    }
  ],
  "related_alert_timeline": [
    "09:10 - Unusual-location login",
    "09:20 - External mail-forwarding rule created"
  ],
  "contradictions": [],
  "missing_evidence": [],
  "recommended_priority": "high",
  "recommendation_rationale": "The reputation result and related account activity provide consistent indicators of possible account compromise",
  "requires_human_review": true,
  "stop_reason": null
}

The recommendation remains advisory. The output does not claim that the account was disabled, the alert was closed, or any other response action was performed.

Represent Missing Evidence Safely

The structured output must also support incomplete investigations. A tool failure or evidence gap is not permission to invent a result.

For example:

{
  "alert_id": "ALERT-1043",
  "run_status": "insufficient_evidence",
  "summary": "Related alerts were found, but an approved reputation result could not be retrieved",
  "extracted_entities": {
    "users": [],
    "source_ips": ["198.51.100.18"],
    "devices": []
  },
  "evidence": [],
  "related_alert_timeline": [],
  "contradictions": [],
  "missing_evidence": [
    "IP reputation result was unavailable after the retry limit"
  ],
  "recommended_priority": "undetermined",
  "recommendation_rationale": "The available evidence does not support a reliable priority recommendation",
  "requires_human_review": true,
  "stop_reason": "Required evidence remained unavailable"
}

This output is still useful because it tells the engineer what was attempted, what is missing, and why the agent stopped.

Confidence Requires a Definition

A confidence field should not be included merely because the agent can generate one. A model-produced percentage may appear precise without being calibrated or supported.

If the design uses confidence, it must define how the value is determined and how it relates to evidence completeness, source reliability, and contradictions. Otherwise, the design should show uncertainty through missing_evidencecontradictionsrun_status, and recommended_priority instead.

Confidence must never replace supporting evidence.

Validate the Output

Before an output is accepted, the system should check that:

  • All required fields are present.
  • Controlled fields contain only allowed values.
  • Every important finding identifies its evidence source.
  • Missing and contradictory evidence is visible.
  • requires_human_review is always true.
  • An unsupported recommendation uses undetermined instead of guessing.
  • A stopped run includes a clear stop_reason.
  • The output does not claim that containment, remediation, alert closure, or another prohibited action was performed.

These checks make the output more consistent and reduce the chance that a confident-looking but incomplete result reaches the engineer.

Design Principle: Reviewable Output

The structured output should follow this principle:

The agent's recommendation must be traceable to evidence, explicit about limitations, and presented in a format the engineer can reliably review.

A safe output does not hide uncertainty or imply that the agent made the final security decision. It gives the engineer a clear investigation package and preserves human accountability.

Answer the questions below

What must the agent use when the available evidence does not support a priority recommendation?

What should not replace supporting evidence?

You have now reviewed the key design areas needed for a safe -assisted investigation agent:

  • Role and scope
  • Tool boundary
  • Context, state, and memory
  • Human review and stop conditions
  • Structured output

In this final task, you will apply these principles in the Agent Spec Builder by assembling a safe and complete agent specification for NorthStar Fashion before implementation begins. This is not a coding task, but a design challenge in which you must determine which choices belong in the specification and which would introduce unsafe autonomy.

Agent Spec Builder

NorthStar Fashion wants an -assisted agent to support security-alert investigations by gathering approved evidence, correlating relevant context, preparing a structured summary, and recommending an investigation priority for human review. However, the agent must remain within its intended boundary and must not:

  • Perform containment
  • Modify systems
  • Close alerts automatically
  • Disable accounts
  • Block IP addresses
  • Invent missing evidence
  • Hide uncertainty
  • Continue when the evidence is insufficient
  • Make final security decisions

The engineer remains responsible for reviewing the evidence and deciding what should happen next.

Design Principle: Complete and Safe Specification

A design is ready only when the agent’s role, tools, context, state, memory, human review points, stop conditions, and output are complete, appropriately scoped, and safe. Completeness means that every required part of the specification is defined, while safety means that the agent is not given authority beyond investigation support. A complete but unsafe design is not ready - for example, it may define tools, state, and outputs while still allowing the agent to disable accounts or close alerts. Likewise, a safe but incomplete design is not ready if it avoids containment actions but fails to define stop conditions, human review points, or required output fields.

The final specification must do both:

  • Include the required design areas
  • Reject unsafe autonomy
Answer the questions below

What is the flag revealed?

You've completed the design phase for the NorthStar Fashion Security Investigation Agent!

Throughout this room, you have learned that successful agents are defined not by the language model they use, but by the decisions that shape how they operate. A well-designed agent begins with a clear objective, remains within a defined scope, uses only the tools it needs, maintains appropriate context and state, and knows when to stop and involve a human. These choices determine whether the agent becomes a reliable assistant or an unpredictable source of risk.

The next step is implementation

Answer the questions below

All done!