It's easy to measure how quickly AI produces an answer. Measuring what happens afterwards is harder. Someone still has to verify the findings, correct mistakes, escalate difficult cases and keep the integrations running.
For SOC leaders, a productivity claim should answer a straightforward question: how much work has the team actually stopped doing, and what has happened to the time saved?
Here's how to find out.
Start Where the Investigation Ends
AI might produce an alert summary in ten seconds. That tells you how quickly the summary was generated.
It says very little about the effort required to complete the investigation.
Assume an analyst reviewing suspicious activity involving a cloud storage resource. The AI retrieves relevant events, prepares a timeline and recommends closing the case as authorised administrative activity.
The analyst still needs to verify the identity involved, examine the supporting records and establish whether the activity matches an approved change.
If that review takes longer than the original investigation, the tool may have improved drafting speed without reducing the team's overall workload.
Start by defining the task you're measuring, from the moment work begins to the point where it reaches a supported disposition.
Separate two useful measurements:
Active handling time is the time people spend investigating, reviewing, correcting and completing the case.
Elapsed case time includes waiting for approvals, data retrieval and responses from other teams.
Both matter, but they answer different questions. A workflow can reduce analyst effort while leaving cases open for the same length of time.
Include work performed outside the immediate SOC queue, too. Detection engineers, senior reviewers and service owners may inherit responsibilities that weren't part of the previous process.
As we've explored in our article on improving SOC efficiency through automation, operational improvements depend on how the complete workflow changes. That is the level at which AI's contribution should be measured.
Before putting another productivity figure in the board deck, find out what your SOC actually gained.
Don't Let Easy Cases Flatter the Results
A pilot that handles routine alerts will naturally appear more efficient than a baseline containing complicated investigations.
Before comparing performance, decide which cases the AI-assisted workflow is expected to handle.
Define the alert families, data requirements and investigation tasks within scope. Keep that population consistent across the baseline and evaluation periods.
If an AI tool is intended to help analysts triage cloud identity alerts, compare its performance against equivalent identity investigations. Don't mix straightforward administrative events with complex incidents requiring cross-platform investigation and expect a single average to tell the whole story.
Also count cases where AI assistance fails. A workflow that handles 70 investigations successfully but sends 30 back to analysts has a different operating cost from one that completes all 100.
Those handovers consume time and belong in the result. Keep changes in the environment visible.
New detection rules, staffing differences or changes in alert volume can influence performance independently of AI.
The Minutes That Quietly Come Back
The first place to look for hidden workload is human verification.
An AI-generated investigation may be fast to produce but require an analyst to examine its evidence, challenge unsupported conclusions and correct the final account.
Those activities can be worthwhile. They may improve investigation quality.
They still consume time.
Track the effort required to complete the assisted workflow, including:
- Investigation: Reviewing the activity and gathering additional evidence.
- Verification: Checking the AI's findings against the underlying records.
- Correction and escalation: Fixing material errors or transferring unresolved work.
- Recurring support: Maintaining the service, repairing integrations and resolving operational failures.
Use clear categories so the same minutes aren't counted twice. Then consider what happens after an initial closure.
If a reviewer discovers missing evidence and reopens a case, the additional effort belongs in the evaluation. The original closure may have looked efficient, but the work continued.
This is particularly important when an AI system produces convincing explanations.
In our experiment with an agentic SOC, five AI agents worked through controlled tabletop investigations. The experiment showed how confident reasoning could coexist with questionable decisions when the evidence was incomplete.
What a 19% Saving Looks Like
Suppose a team handles 100 comparable investigations each month.
Before AI assistance, each investigation requires an average of 20 minutes of human effort, giving the team a baseline of 2,000 minutes.
After introducing AI, the team measures an average of eight minutes investigating, five minutes verifying and two minutes correcting each case.
That brings direct handling effort down to 15 minutes per investigation.
However, the AI-assisted workflow also requires 120 minutes of recurring engineering and service support during the month.
The result is a reduction of 380 minutes, or 19% of the original workload.
That's a useful improvement, assuming the quality of the investigations remains acceptable.
But notice how the figure changes. Looking only at direct case handling suggests a 25% improvement. Including the support required to operate the new workflow reduces the measured gain to 19%.
The calculation also assumes there was no equivalent support overhead in the baseline. If there was, that effort needs to be included for a fair comparison. Implementation costs, licensing and model usage charges remain separate considerations.
Before putting another productivity figure in the board deck, find out what your SOC actually gained.
Faster Cases Can Leave Expensive Mistakes Behind
Speed needs to be measured alongside the quality of the resulting investigation. A case closed five minutes earlier offers little benefit if the analyst overlooked evidence that would have changed its disposition.
Define what counts as a material error before reviewing performance. An incorrect asset identity, unsupported closure or missed escalation may affect the response. A minor wording correction usually has a different consequence.
Track both the frequency and significance of those errors. Review cases accepted without correction as well as those that required changes. A low correction rate could indicate accurate AI output, but it could also mean reviewers are overlooking mistakes.
Apply the same quality standard to the original human-led process. Analysts make errors too, and the comparison should establish whether the assisted workflow changes the outcome.
For consequential failures, investigate the individual cases rather than relying entirely on averages.
Six Numbers Worth Bringing to the Review
For most initial SOC evaluations, six measures provide a useful starting point:
Keep the case count, period and relevant scope alongside the figures. Look beyond the average when performance varies considerably between cases.
Give the Team Something Better to Do With the Time
There is a human side to this measurement question.
In TryHackMe's research report, The Hard Part Is Still Human, skills, judgement and operational capacity emerged as recurring challenges in discussions about AI adoption across 43 security organisations examined in depth.
These findings reinforce a practical consideration for SOC leaders: introducing AI changes the work analysts perform, and teams need the capability to handle that change.
Analysts may spend less time gathering routine evidence while taking on more responsibility for verifying findings, resolving ambiguity and deciding how to respond.
Those are skills worth measuring and developing independently of any production AI deployment.
Practical SOC training gives analysts opportunities to work through realistic investigations, review evidence and improve their decisions with feedback. Managers can also use team capability and progress reporting to identify development needs.
So, Where Did the Hours Go?
AI may reduce the work required to investigate security alerts. Some workflows will benefit more than others, and a useful deployment can release substantial analyst capacity.
The worthwhile question is where that capacity goes.
Has the SOC reduced its backlog? Are difficult investigations receiving more attention? Are analysts making better-supported decisions? Or has the work simply moved into verification, corrections and service maintenance?
Measure the full workflow, and those answers become easier to defend.
Before putting another productivity figure in the board deck, find out what your SOC actually gained.
Ayomide Joseph A.