What Is a Detection Signal vs Noise?

powershell.exe -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -EncodedCommand SQBu...

That is a malicious command line in one estate and an Ansible task in the next one over. Ansible's PowerShell shell plugin sets _common_args = ['PowerShell', '-NoProfile', '-NonInteractive', '-ExecutionPolicy', 'Unrestricted'] and base64 encodes the module body into -EncodedCommand. Nothing about the process creation event distinguishes the two. The event is identical. The environment is not.

Signal and noise are not properties of an event. They are properties of an event in an environment. A rule that is precise in a 900-seat law firm can be the loudest thing in a 12,000-seat estate with configuration management, and no change to the YAML caused it.

The same command line in two estates

Estate A runs 900 Windows endpoints. No Ansible, no Configuration Manager, no Jenkins agents on desktops. Encoded PowerShell fires three times a day, all of it from one backup wrapper with a stable parent. Ninety events in a thirty-day window.

Estate B runs 12,000 endpoints under Ansible over WinRM and Configuration Manager. Every Ansible task lands as a powershell.exe process under wsmprovhost.exe, the WinRM plugin host. Encoded PowerShell fires roughly 38,000 times a day. That is 1,140,000 events in the same thirty-day window.

Same behaviour. A base rate that differs by four orders of magnitude. Every number in this post is illustrative, chosen to make the arithmetic legible. The shape is what matters.

Precision, recall and the base rate

Three numbers, defined at the event level.

• Precision is true positives divided by all alerts. It answers: if this fires, what are the odds it matters.

• Recall is true positives divided by all malicious events that existed. It answers: what fraction of the real thing did we see.

• The base rate is malicious events divided by all candidate events. It is the prior, and it is invisible to the analyst.

Put one intrusion into estate B. It produces six encoded PowerShell executions across its dwell time. Base rate is 6 in 1,140,000, or one in 190,000.

That prior is what kills rules. A rule with recall of 0.83 sounds like a good rule. Applied against a prior of one in 190,000, a false positive rate of four percent per candidate event yields 45,600 alerts and five true ones. Precision is 0.011 percent. At six minutes of honest triage each, 1,520 alerts a day is 152 analyst hours a day. The queue does not get worked, it gets closed. That failure mode is covered in more detail in what is alert fatigue.

Level one: the behaviour on its own

title: Encoded PowerShell command line
id: 0f2b9d3a-7c41-4a1e-8f5b-2c9d61a4e77b
status: experimental
description: Any powershell.exe or pwsh.exe start carrying an encoded command argument.
logsource:
    category: process_creation
    product: windows
detection:
    selection_img:
        - Image|endswith:
              - '\powershell.exe'
              - '\pwsh.exe'
        - OriginalFileName:
              - 'PowerShell.EXE'
              - 'pwsh.dll'
    selection_enc:
        CommandLine|contains: ' -e'
    condition: all of selection_*
falsepositives:
    - Configuration management agents
    - Software packaging and deployment
level: low
tags:
    - attack.execution
    - attack.t1059.001
    - attack.t1027.010

' -e' is the standard trick for catching every abbreviation of -EncodedCommand down to -e, because PowerShell accepts any unambiguous prefix. It also matches -ExecutionPolicy. Every Ansible task in estate B hits this rule on the -ExecutionPolicy Unrestricted argument alone, before the encoded payload is even considered.

This is why the SigmaHQ rule ca2092a1-c273-4878-9b4b-0d60115bf5ea pairs -e with a list of base64 prefixes ( JAB, SQBFAFgA, aQBlAHgA and friends) that correspond to UTF-16LE encodings of $, IEX and iex. That is already a precision technique, not a detection technique. It does not detect anything new.

Level two: a context condition

The instinct at this point is to write ParentImage|endswith: '\wsmprovhost.exe' into a filter and move on. That is an allowlist, and it is a permanent hole. Anything that runs under that parent is now invisible.

Add a condition instead of removing a population. Malicious encoded PowerShell in estate B arrives from a document handler, a script host or an interactive shell. Automation arrives from a service session.

title: Encoded PowerShell from an interactive or document parent
id: 3d81c6ee-5b2f-4a0c-9a17-4e0f8b1c22d5
status: experimental
logsource:
    category: process_creation
    product: windows
detection:
    selection_img:
        - Image|endswith:
              - '\powershell.exe'
              - '\pwsh.exe'
        - OriginalFileName:
              - 'PowerShell.EXE'
              - 'pwsh.dll'
    selection_enc:
        CommandLine|contains:
            - ' -enc'
            - ' -EncodedCommand'
    selection_b64:
        CommandLine|contains:
            - ' JAB'
            - ' SQBFAFgA'
            - ' aQBlAHgA'
            - ' IAB'
            - ' cwB'
    selection_parent:
        ParentImage|endswith:
            - '\explorer.exe'
            - '\winword.exe'
            - '\excel.exe'
            - '\outlook.exe'
            - '\wscript.exe'
            - '\cscript.exe'
            - '\mshta.exe'
    condition: selection_img and selection_enc and selection_b64 and selection_parent
level: medium

The recall cost is real and should be written down. Two of the six malicious executions in our scenario came from a parent that was already a service process, and this rule misses both. Recall drops from 0.83 to 0.67. The rule also assumes command line auditing is on, which means Sysmon Event ID 1 or Security 4688 with ProcessCreationIncludeCmdLine_Enabled set to 1.

Level three: a second signal

Encoded PowerShell that is going to matter usually talks to something. Demand the second signal rather than tightening the first one further.

title: PowerShell outbound to a public address
name: powershell_public_egress
id: 5e7c1a02-9f34-4b8d-a6c1-0d3e9f7b4411
logsource:
    category: network_connection
    product: windows
detection:
    selection:
        Image|endswith: '\powershell.exe'
        Initiated: 'true'
    filter_internal:
        DestinationIp|cidr:
            - '10.0.0.0/8'
            - '172.16.0.0/12'
            - '192.168.0.0/16'
            - '127.0.0.0/8'
    condition: selection and not filter_internal
level: informational

title: Encoded PowerShell followed by public egress
id: 9c4a0b71-2e58-4f6d-bb03-7a1d5e2f8c90
status: experimental
correlation:
    type: temporal
    rules:
        - 3d81c6ee-5b2f-4a0c-9a17-4e0f8b1c22d5
        - powershell_public_egress
    group-by:
        - ComputerName
        - ProcessId
    timespan: 60s
level: high
tags:
    - attack.execution
    - attack.t1059.001
    - attack.command_and_control
    - attack.t1071.001

Grouping on ComputerName and ProcessId ties Sysmon Event ID 3 back to the exact process from Event ID 1. The 60 second window is the assumption doing the work, and it is the assumption that breaks. A loader that sleeps for ten minutes before its first check-in produces both events and never correlates. Recall drops again, to 0.50 in our scenario.

What the three levels cost

Illustrative figures for estate B. Note what level one buys with its extra recall: one additional malicious event, at a cost of 45,511 additional alerts. Note also that none of this applies to estate A, where level one on its own runs at roughly 5 percent precision and a single parent-process filter takes it close to 1.0. Identical YAML, opposite verdict.

Baselines beat allowlists

A static allowlist is a decision made once against an environment that keeps moving. A baseline is the same decision recomputed.

Build the prevalence of the tuple (parent image, first token of the command line, signer) over a rolling 30 days, and alert when the tuple is new or present on fewer than five hosts. Automation is high prevalence by construction, because it is deployed to everything. Intrusion tooling is low prevalence by construction, because it is not.

Two honest costs. A baseline needs a state store the SIEM may not have, and it goes loud for a week after any deployment wave, which is exactly when nobody wants to look at it. Rarity is also gameable: an attacker who executes through an already-common tuple sits under the threshold permanently.

Scope to asset roles

Ship one detection twice rather than one detection with an exception list. Build agents, jump hosts and packaging servers get the level three correlation and nothing else. User workstations get level two, where the parent context assumption holds. Domain controllers get level one unfiltered, because encoded PowerShell from an interactive parent on a DC should be zero.

The dependency is an asset inventory that is current. A host that changed role six months ago and still carries the old tag is a blind spot with a rule sitting on top of it.

Behaviours that will not go precise

Some behaviours have no context condition that separates the malicious instance from the benign one. certutil.exe -urlcache -split -f is genuinely used by administrators. Unsigned DLLs loading from user-writable paths are genuinely produced by half the software estate. rundll32.exe calling an ordinal export is normal.

For those, precision engineering is the wrong instrument. Put them in a hunt queue instead: a ranked list, run weekly, sorted by rarity, reviewed in batches, with no ticket and no SLA. The output of a hunt is a hypothesis or a new rule, not a closed alert. Trying to force those behaviours into the alert queue produces the level one situation on purpose.

Summary

Precision and recall are measured against a base rate, and the base rate lives in the environment rather than in the rule. The same encoded PowerShell rule is high precision in an estate without configuration management and produces 1,520 alerts a day in one with it. Raise precision by adding context conditions, baselining rarity, correlating a second signal and scoping to asset roles, and record the recall each of those costs. Behaviours that resist all four belong in a hunt queue, not in the alert queue. Start with the Windows process creation rules in the catalog at /rules?domain=windows and the PowerShell rules at /rules?q=powershell.

Telemetry required
  Sysmon Event ID 1    process creation with CommandLine and ParentImage
  Sysmon Event ID 3    network connection with Initiated and DestinationIp
  Sysmon Event ID 22   DNS query, for destination rarity scoring
  Security 4688        requires ProcessCreationIncludeCmdLine_Enabled = 1
  PowerShell 4104      script block logging, decoded payload content

ATT&CK
  T1059.001   Command and Scripting Interpreter: PowerShell
  T1027.010   Obfuscated Files or Information: Command Obfuscation
  T1140       Deobfuscate/Decode Files or Information
  T1071.001   Application Layer Protocol: Web Protocols

Fields to record per rule
  expected_alerts_per_day
  known_blind_spots
  asset_roles_in_scope