Incident Response

What Is a Lessons Learned Session in IR?

HuntRule Team · · 8 min read

A steel depth-stop collar clamped tight on a drill bit, the tool now physically unable to cut deeper than the set limit
On this page

Two lines came out of the same incident. The first: "improve communication with the network team." The second: "on-call had no out-of-hours route to network engineering, so the rota now lists a named engineer and a number, and that line is step 2 of the isolate-host runbook." Six months later, only one of those changed anything.

Both were written in a lessons learned session. Only one was encoded anywhere a person would trip over it again.

Lessons learned session or post-incident review

These get used as synonyms and they are not the same meeting.

The post-incident review is the formal one. Stakeholders in the room, an agreed timeline, a root cause statement, and an output contract: owned action items with names and dates. It has a template, a distribution list, and often a compliance obligation behind it. We covered its shape in what a post-incident review is.

The lessons learned session is the working-level conversation that feeds it. Practitioners only. The people who ran the containment, pulled the logs, and wrote the queries at 03:00. An hour, no management, no template to satisfy. Its output is raw observations, and the review is where those get argued into commitments.

The frameworks name it. SANS makes Lessons Learned the sixth step of its handling process, after Preparation, Identification, Containment, Eradication and Recovery. NIST SP 800-61 Rev. 2 folded it into Post-Incident Activity. Rev. 3, published April 2025, superseded Rev. 2, dropped the four-phase lifecycle and restructured the guidance as a CSF 2.0 Community Profile, where improvement sits under Identify as ID.IM. None of them tell you what happens to the observation after somebody says it out loud.

What the session produces

Run it within five business days, while people still remember which tab they had open. Put the timeline on screen and walk it. Three questions per gap:

  1. What did we see, and what artifact told us? An alert ID, a log line, a message with a timestamp.

  2. What did we not see, and where would it have been? A log source, an event ID, a rule that did not exist.

  3. What slowed us down between seeing and acting? Missing access, an unknown owner, a tool nobody had run before.

The output is observations, each tied to an artifact. No owners yet, no dates. If an observation cannot name an artifact, it is an opinion and it goes in a separate list.

The half-life of a lesson

A lesson recorded in a document nobody reads is not learned. This is the failure mode, and it is boring: the session runs well, the notes are good, the wiki page gets one edit dated the week of the incident and none after. Eighteen months later the same gap reopens, the analyst who held the knowledge has left, and the action item was assigned to a team rather than a person.

What predicts survival is whether the lesson stayed a sentence or became a thing that executes.

Evaporates

Survives

"We should communicate better with the network team."

"On-call had no out-of-hours route to network engineering. The rota names an engineer and a number, and it is step 2 of the isolate-host runbook."

"Improve domain controller monitoring."

"Process creation events were not arriving from DC03. Log source onboarding now verifies Event ID 1 from every DC before the ticket closes."

"Our credential dumping coverage is weak."

"Nothing matched ntdsutil ... ifm. Rule shipped, and Atomic Red Team T1003.003 Test #3 runs against it monthly."

The right column is longer, more specific, and harder to write. That is the cost. It is also the only column anyone can act on twelve months later.

Encode it or lose it

Four places a lesson can live where it is enforced by the system instead of remembered by a person. Ranked weakest to strongest.

  1. A runbook step. Still needs a human to read it at the right moment, so it is the weakest of the four. It beats a wiki page because a runbook is actually open during the next incident.

  2. A detection rule. The lesson now fires on its own. Nobody has to recall the technique, and the rule text is the documentation.

  3. A validation test. Guards the rule. If a field mapping changes or an agent stops reporting, the test fails before the next intrusion does the testing for you.

  4. A default configuration. Strongest, because it removes the condition instead of watching for it. A rebuilt domain controller arrives with the logging already on.

Most sessions produce observations that fit more than one of these. Take all of them.

Worked example: ntdsutil IFM

The example below is constructed for this post. We did not respond to this incident. The commands are the published Atomic Red Team test, and the log fields are the ones Sysmon writes.

Observation from the session: an operator with domain admin dumped Active Directory using the built-in install-from-media path, and nothing alerted. The command:

ntdsutil "ac i ntds" "ifm" "create full C:\Windows\Temp\ntds_T1003" q q

IFM exists so a new domain controller can be promoted from media without pulling a full replication over the network. It also writes a complete copy of Ntds.dit plus the SYSTEM hive to a directory of the operator's choosing, which is offline credential access with no third-party binary. ATT&CK tracks it under T1003.003. APT28 has used ntdsutil.exe to export the AD database, per the 2021 NSA and CISA advisory on GRU brute-force campaigns.

The soft version of this lesson is "improve DC monitoring." Here is the encoded version.

The rule. SigmaHQ already ships 2afafd61-6aae-4df4-baed-139fa1f4c345, which fires on any ntdsutil.exe execution at level medium with "NTDS maintenance" listed as the false positive. On a domain controller that is noisier than it sounds. This narrows to the IFM path:

title: Ntdsutil IFM media creation
id: a1c7d3e2-9b5f-4e18-8f2a-6d0c41b7e9a3
status: experimental
description: Detects ntdsutil.exe creating install-from-media output, which writes a full copy of Ntds.dit and the SYSTEM hive to an operator-chosen path
references:
    - https://attack.mitre.org/techniques/T1003/003/
author: HuntRule
date: 2026-07-31
tags:
    - attack.credential-access
    - attack.t1003.003
logsource:
    category: process_creation
    product: windows
detection:
    selection:
        Image|endswith: '\ntdsutil.exe'
        CommandLine|contains|all:
            - 'ifm'
            - 'create'
    condition: selection
falsepositives:
    - Staging a new domain controller from IFM media
    - Backup tooling that wraps ntdsutil for scheduled AD exports
level: high

The runbook change. Log source onboarding gains one verification step, run before the ticket can close:

index=wineventlog EventCode=1 host IN (dc01, dc02, dc03)
| stats count, max(_time) as last_seen by host

Every DC in the inventory returns a row, or the source is not onboarded. That step is what would have caught DC03 sitting silent for a year.

The validation test. Atomic Red Team T1003.003 Test #3, "Dump Active Directory Database with NTDSUtil", runs monthly against a lab DC. The rule fires or the pipeline fails.

The default configuration. Command line capture in Security Event ID 4688 is off by default. The Group Policy setting is Computer Configuration, Administrative Templates, System, Audit Process Creation, "Include command line in process creation events", which writes:

HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System\Audit
    ProcessCreationIncludeCmdLine_Enabled = 1 (REG_DWORD)

Applied to the Domain Controllers OU, a rebuilt DC arrives already logging the argument the rule depends on. That is the lesson moved into the system.

What the rule catches and what it misses

It catches the IFM path regardless of output directory, and regardless of how the sub-commands are quoted, because Sysmon Event ID 1 records the full command line as typed.

It misses a renamed ntdsutil.exe, since the match is on Image. It misses every volume shadow copy route to the same file (vssadmin, diskshadow, WMI, esentutl, and the fsutil and MFT-parsing variants in the same Atomic set), which need their own rules. It misses DCSync, T1003.006, because that never touches the DC's disk. And it misses any domain controller whose process creation events are not collected, which is the gap the runbook step closes.

False positives are real. Promoting a domain controller from media is legitimate, and some backup products shell out to ntdsutil. Both are rare and both should map to a change ticket, which makes this a good high-severity rule and a bad auto-containment trigger.

Summary

A lessons learned session is the practitioner-level conversation after an incident, and the post-incident review is the formal meeting it feeds. The session surfaces observations tied to artifacts. The review turns them into owned commitments. Neither step makes the lesson durable, and that is where the value leaks out. Encode each finding into a runbook step, a detection rule, a validation test, or a configuration default, and it is enforced rather than recalled.

Collect these before you need them:

Sysmon Event ID 1        Process creation with full command line, all domain controllers
Security Event ID 4688   Process creation, requires ProcessCreationIncludeCmdLine_Enabled=1
Security Event ID 4624   Successful logon, to attribute the session to an account
Security Event ID 4672   Special privileges assigned to new logon

Credential access rules for Windows hosts are in the catalog at /rules?domain=windows, and the ntdsutil coverage discussed here is at /rules?q=ntdsutil.

Related articles