What Is Detection Engineering?

vssadmin.exe delete shadows /all /quiet maps to T1490, Inhibit System Recovery. Writing the Sigma rule for it takes about ten minutes. Everything after those ten minutes is detection engineering.

Detection engineering is the practice of treating detections as engineered artifacts with a lifecycle: an input, a build, a test, a deployment, a measurement, and an end of life. The discipline grew out of work like Palantir's Alerting and Detection Strategy framework and Roberto Rodriguez's open telemetry projects (OSSEM, the Threat Hunter Playbook). Most teams get the build step right and skip the rest. That is where programs quietly rot.

The failure modes are structural, not individual

Every failure below is a process gap, not a bad analyst.

Data source validation comes first

Before authoring, answer one question: do we collect the telemetry this rule needs, on the hosts that matter, with the fields populated. For T1490 that means Sysmon Event ID 1 or Windows Security 4688 with command line auditing on. 4688 without CommandLine gives you the image name and nothing else, which makes the rule above unwritable.

Check coverage as a percentage of in-scope hosts, not as a yes or no on the pipeline.

index=win EventCode=1 earliest=-24h
| stats dc(host) as reporting
| appendcols [| inputlookup cmdb_windows_assets | stats dc(hostname) as expected]
| eval coverage_pct = round(reporting/expected*100, 1)

That is stats over raw events, which is slow over 24 hours on a busy index. tstats is the fast version and you cannot use it here, because EventCode is a search-time extraction in the Windows TA rather than an indexed field. Accelerate a data model first if you want this cheap.

DeTT&CT is the tool built for this. It scores data sources across device completeness, data field completeness, timeliness, consistency and retention, then renders visibility as an ATT&CK Navigator layer. A data source scored 1 out of 5 for device completeness will not support a high-confidence rule no matter how good the logic is.

Authoring

Write in Sigma if you run more than one backend. The rule is the abstract logic, the pipeline handles the field mapping.

title: Volume Shadow Copy Deletion Via Vssadmin
id: 8b1a9e1f-3c2d-4c6a-9f10-6e4b7c2d5a31
status: experimental
description: Detects deletion of all volume shadow copies via vssadmin.exe, a common precursor to ransomware encryption.
references:
    - https://attack.mitre.org/techniques/T1490/
author: huntrule.com
date: 2026-07-31
tags:
    - attack.impact
    - attack.t1490
logsource:
    category: process_creation
    product: windows
detection:
    selection_img:
        - Image|endswith: '\vssadmin.exe'
        - OriginalFileName: 'VSSADMIN.EXE'
    selection_cli:
        CommandLine|contains|all:
            - 'delete'
            - 'shadows'
    condition: all of selection_*
falsepositives:
    - Backup agents pruning shadow copies on a schedule
    - Imaging and provisioning jobs run by IT
level: high

Validate and convert before it goes anywhere near a SIEM.

sigma check rules/proc_creation_win_vssadmin_delete_shadows.yml
sigma convert -t splunk -p sysmon rules/proc_creation_win_vssadmin_delete_shadows.yml

Note the OriginalFileName selection. It survives a renamed binary, which Image|endswith alone does not. That is the kind of decision the ADS framework wants recorded under Blind Spots and Assumptions, alongside the obvious one here: this rule sees nothing if the operator calls the VSS COM API directly instead of shelling out to vssadmin.

Proving it fires

A rule nobody validated against real telemetry is a guess. Run the emulation, then confirm the event landed with the fields you assumed.

Invoke-AtomicTest T1490 -TestNames "Windows - Delete Volume Shadow Copies" -GetPrereqs
Invoke-AtomicTest T1490 -TestNames "Windows - Delete Volume Shadow Copies"

Two checks, not one. First, did the raw event arrive. Second, did the deployed search return it. Those fail independently, and the second failing while the first passes is the classic field mapping bug. Caldera covers the same ground for chained behaviour when a single atomic is not enough.

Record the result on the rule itself. Test date, test ID, host, outcome. A rule with no validation date is untested regardless of what the heatmap says.

What "tuned" means, numerically

Tuned is a number, not a feeling. For this rule, ours are:

• Alert volume: 5 or fewer per week across the estate, measured over a rolling 30 days

• Precision: at least 0.5 true positive rate over the last 20 dispositioned alerts

• Concentration: no single host contributing more than 40 percent of alerts

• Triage cost: median analyst time under 10 minutes with the runbook

The vssadmin rule almost always breaches the concentration threshold first, because one backup server generates the entire volume. The fix is a scoped exclusion on that host and parent process, not a global exclusion on vssadmin.exe, which is how teams end up with a rule that cannot fire anywhere.

If a rule cannot meet its volume target and cannot be scoped, it is not a rule. It is a hunt query, and it belongs in a scheduled hunt, not an alert queue.

Coverage is not a heatmap

ATT&CK Navigator heatmaps count rules. They do not measure detection. A green cell for T1003.001 (OS Credential Dumping: LSASS Memory) can mean a validated Sysmon Event ID 10 rule scoped to lsass.exe with a tested GrantedAccess mask, or it can mean one keyword search on the string "procdump" that nobody has run since 2023. Both render the same colour.

Score cells instead of colouring them. We use four levels per technique: none, untested rule exists, validated against emulation, validated plus tuned to threshold. Publish the layer with that scale and the picture changes fast. Sub-technique granularity matters too. Claiming T1218 as covered because you detect T1218.011 (Rundll32) is a reporting error, not a detection.

Rules fail silently

The most expensive failure is a rule that stops firing. An agent upgrade renames a field, a log source drops a channel, a forwarder is decommissioned. Nothing errors. The search returns zero rows, and zero rows looks exactly like a quiet environment.

Monitor the input, not the output.

| tstats count where index=win EventCode=1 earliest=-14d by host, _time span=1d
| eventstats avg(count) as baseline, stdev(count) as sd by host
| where _time >= relative_time(now(), "-1d@d") AND count < (baseline - 3*sd)

Volume monitoring catches the missing host. It does not catch a field going null while events keep flowing, which is the sneakier version. For that, alert when a field's non-null ratio drops below a threshold on a source that previously populated it. We do this for CommandLine, ParentImage and OriginalFileName and nothing else, because per-field health checks are expensive to maintain.

Retirement

Rulesets only grow because nothing is set up to remove anything. Give every rule a review date and three possible outcomes at that date: keep, rewrite, retire.

Retire on evidence. The technique no longer applies to your estate (the vulnerable software is gone). The rule is fully subsumed by a broader one. Zero true positives in 12 months combined with a validation test that still passes, meaning the rule works and the behaviour simply does not occur here. That last case is a judgement call and we keep a handful of zero-hit rules for high-impact, low-frequency techniques like T1490.

Rules that fail validation and have never fired are the easy deletes. Most teams have more of them than they expect.

Summary

Detection engineering is the loop, not the rule. Validate the data source before writing anything, author in a portable format, prove the rule fires against emulated activity, tune to a stated number, monitor the input for silent failure, and retire on evidence. Coverage heatmaps are a communication tool and a poor measurement tool, so score cells by validation state rather than by rule count. The gap we still have not solved well is field-level health monitoring at scale, and we run it on three fields only because the maintenance cost is real.

Start with what you already collect. Browse /rules?domain=windows for process creation coverage, or pull the impact set with /rules?severity=high and check each one against the gates below.

detection_gates:
  - id: data_source
    check: telemetry present on >=90% of in-scope hosts, required fields non-null
    blocks: authoring
  - id: syntax
    check: sigma check passes, converts cleanly on all target backends
    blocks: testing
  - id: validation
    check: emulation executed, raw event confirmed, deployed search returns it
    blocks: deployment
  - id: tuning
    check: <=5 alerts/week, precision >=0.5, no host >40% of volume
    blocks: alert_queue
  - id: runbook
    check: triage steps and escalation path documented
    blocks: alert_queue
  - id: health
    check: input volume baseline registered, 3-sigma drop alerts
    blocks: none
  - id: review
    check: review_date set, outcome in [keep, rewrite, retire]
    blocks: none