What Is a Data Source in Detection Engineering?

Windows event ID 4688 fires on every process creation. The Process Command Line field inside it is empty by default. Microsoft documents this plainly: the field arrived in event version 1 on Windows 8.1 and Server 2012 R2, and it stays blank until someone enables Administrative Templates\System\Audit Process Creation\Include command line in process creation events.

So a SOC can have 4688 flowing from every endpoint, tick "process creation" on a coverage matrix, and still have zero rules that work. Every rule that reads CommandLine matches nothing. Not fewer things. Nothing.

The contract

A data source is the contract between a detection and reality. The rule promises to fire when a behavior happens. That promise only holds if something out there records the behavior, records it with the fields the rule reads, delivers it before the rule is useless, and keeps it long enough to hunt in. Break any clause and the rule still parses, still deploys, still shows green in the console, and still never fires.

Coverage claims fail because nobody reads the contract. They count rules, or they count technique IDs, and neither counts telemetry.

Log source, data source, data component

Three terms get used interchangeably and they answer different questions.

ATT&CK built that middle and right column out explicitly. Process is DS0009. Command is DS0017. Under Process sit components including Process Creation, Process Metadata and Process Termination, and techniques cite the component, not the vendor product.

What ATT&CK changed in v18

Data sources were deprecated in ATT&CK v18, released October 2025. The pages stay up for reference and no new data sources will be added. Do not build a new program around DS numbers.

The replacement is more useful for this exact problem. A Detection Strategy carries an ID, a name, and one or more Analytic IDs. Each analytic carries an ID, a platform, a detection statement, a log source, and mutable elements. Data components survive and now hang log source permutations off themselves, each permutation naming a log source name and a channel. Mutable elements are the thresholds, tool lists and time windows you are expected to tune locally without changing the behavior being detected.

That is ATT&CK conceding the point. The old model said which subject to watch. The new model forces you to name the channel and the tunables, which is where coverage actually breaks.

Five clauses in the contract

DeTT&CT, the Rabobank CDC framework built to score exactly this, splits data quality into five dimensions: device completeness, data field completeness, timeliness, consistency and retention. Its scoring table runs 0 to 5. Device completeness 3 means the source reaches 51 to 75 percent of devices. Retention 5 means you store 100 percent of the desired period.

Read those as clauses rather than scores:

1. Present on the hosts that matter. Not the estate average. The jump boxes, the domain controllers, the build agents.

2. Populating the fields the rule reads. CommandLine present is a different question from 4688 present.

3. Standardised in name and type, so one rule works across the fleet instead of per-collector variants.

4. Arriving inside a usable latency. Data that lands the next morning is retrospective, not detective.

5. Retained long enough to hunt in. A 7 day window does not answer a question about a 40 day dwell.

Prove it per host

Do not ask whether the pipeline works. Ask which hosts it works on. Start from the asset inventory and left join the telemetry, so absent hosts appear as rows instead of vanishing.

| inputlookup asset_inventory.csv where os_family="windows"
| fields host, business_unit, criticality
| join type=left host
    [ search index=win sourcetype="WinEventLog:Security" EventCode=4688 earliest=-7d
      | eval has_cmdline=if(isnotnull(CommandLine) AND CommandLine!="", 1, 0)
      | stats count as events, sum(has_cmdline) as with_cmdline, max(_time) as last_seen by host ]
| fillnull value=0 events with_cmdline
| eval pct_cmdline=if(events>0, round(with_cmdline*100/events, 1), 0)
| eval status=case(events=0, "no 4688 at all",
                   pct_cmdline<95, "4688 without command line",
                   1=1, "ok")
| table host, business_unit, criticality, events, pct_cmdline, last_seen, status
| sort status, -criticality

Two rows of output matter. Hosts with events=0 have no agent, no forwarder, or no audit policy. Hosts with a low pct_cmdline have the policy off. Both look identical on a technique heatmap.

Latency and retention need their own checks, and both are one line each.

| eval lag=_indextime-_time | stats p50(lag) as p50, p95(lag) as p95 by host
| tstats min(_time) as oldest where index=win sourcetype="WinEventLog:Security" by host

We run these against a lab collector, not against a customer estate. Field names differ per platform. In Sentinel the same idea is a leftouter join from a Heartbeat or asset table against SecurityEvent, and the CommandLine column is nullable there too.

The gaps that keep surprising teams

Command-line auditing off is the classic. Two settings, two registry values, and both default to disabled.

auditpol /get /subcategory:"Process Creation"
HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System\Audit\ProcessCreationIncludeCmdLine_Enabled = 1
HKLM\SOFTWARE\Policies\Microsoft\Windows\PowerShell\ScriptBlockLogging\EnableScriptBlockLogging = 1

PowerShell script block logging is the second. Without it there is no 4104 in Microsoft-Windows-PowerShell/Operational, and every rule written against deobfuscated script content is inert. Sysmon event ID 1 sidesteps the 4688 command-line problem entirely, which is why teams running Sysmon often do not realise their 4688 baseline is hollow.

Cloud audit logging is per service, not per account. AWS CloudTrail logs management events by default and does not log data events: S3 object-level GetObject, PutObject and DeleteObject, and Lambda Invoke, are off until you add a data event selector, and they bill separately. An Azure resource writes nothing to a workspace until a diagnostic setting on that resource says so. The account looks logged. The bucket is not.

Agent gaps follow ownership, not geography. The servers with an exception are usually the ones with an owner who argued, which correlates with the ones that matter.

A rule that states its own contract

title: Encoded PowerShell command line
status: experimental
description: Detects PowerShell launched with an encoded command payload
logsource:
  category: process_creation
  product: windows
detection:
  selection_img:
    Image|endswith:
      - '\powershell.exe'
      - '\pwsh.exe'
  selection_enc:
    CommandLine|contains: ' -enc'
  condition: all of selection_*
falsepositives:
  - Configuration management agents wrapping payloads in encoded commands
  - Vendor installers that re-launch themselves elevated
level: medium
tags:
  - attack.execution
  - attack.t1059.001
  - attack.t1027.010

What it catches: powershell.exe or pwsh.exe with an encoded payload. Sigma string matching is case-insensitive by default, so the single string ' -enc' also covers ' -EncodedCommand' and ' -EncodedCOMMAND'.

What it misses: PowerShell accepts any unambiguous prefix of a parameter name, so ' -ec' and ' -e' both resolve to EncodedCommand and neither matches this rule. It also misses anything hosted through System.Management.Automation without powershell.exe.

What it requires: process_creation with a populated CommandLine. On a Sysmon fleet that is event ID 1 and it works. On a fleet backed only by 4688 with the group policy unset, this rule cannot fire, and no amount of tuning changes that.

Summary

A data source is not a checkbox on a matrix. It is a claim that a specific component of telemetry reaches specific hosts, with specific fields populated, inside a specific latency, for a specific retention period. ATT&CK deprecated its data source objects in v18 and replaced them with detection strategies whose analytics name the log source and channel outright, which is the more honest model. DeTT&CT gives you five dimensions to score against and a table to score with. Validate the source before you write the rule, because a rule sitting on absent telemetry is worse than no rule at all. No rule reads as a gap. That rule reads as coverage.

Browse rules by the telemetry they need at /rules?domain=windows, and see /blog/what-is-detection-engineering for where source validation sits in the wider workflow.

Validate before writing a Windows process rule:
  auditpol /get /subcategory:"Process Creation"          -> Success enabled
  ProcessCreationIncludeCmdLine_Enabled                  -> 1
  EnableScriptBlockLogging                               -> 1
  Sysmon config includes ProcessCreate (event ID 1)      -> present
  4688 host count vs asset inventory count               -> delta = 0
  p95 ingest lag                                         -> under your triage SLA
  oldest event in index                                  -> beyond your hunt window