What Is Patch Management in Security?

Log4j 2.17.1 on disk does not fix anything if the JVM that loaded 2.14.1 has been up for two hundred days. The file is patched. The process is not. Every vulnerability scanner in the building will report the host as remediated, because the scanner reads the filesystem and the package database, and the exploit reads memory.

That gap is the whole subject. Patch management is a release engineering problem. Someone else builds the artifact, you test it, you stage the rollout, you keep a way back, and you verify the result. Security owns exactly one thing in that pipeline: the deadline. Everything else is change management, and treating it as a security task run by security tooling is how organizations end up with compliance reports that are technically accurate and operationally false.

The pipeline

Six stages, and each one fails differently.

1. Inventory. You cannot patch what you do not know runs. This includes the JAR shaded inside another JAR, the vendor appliance nobody logs into, and the container image built in 2023 that CI still pulls by tag.

2. Source and authenticity. Where the patch comes from and whether it is really from there.

3. Test ring. A population that looks like production and is allowed to break.

4. Staged rollout. Rings, not a switch.

5. Rollback plan. Written before deployment, not during the outage.

6. Verification. The part almost everyone skips.

Inventory drives everything downstream, and it is the stage where most programs are weakest. A CVE lands against a library, and the question "do we run it" takes three days to answer. Those three days are the entire exploitation window now. Google Threat Intelligence tracked 138 vulnerabilities disclosed in 2023 and exploited in the wild, and measured an average time-to-exploit of five days, down from 32 days across 2021 and 2022 and 63 days across 2018 and 2019.

Where the patch comes from

Fetch over TLS from the vendor, and verify the signature separately. On Windows the update package is Authenticode signed and the servicing stack validates it. On Linux the failure case is a repository added with gpgcheck=0 years ago for a one-off install.

rpm -qi openssl | grep Signature
grep -r 'gpgcheck' /etc/yum.repos.d/
apt-config dump | grep AllowUnauthenticated

Vendor portals that ask you to disable signature checks to install a hotfix are supply chain incidents waiting for a date.

Test ring and rollback

Rings are what make an emergency patch survivable. A workable shape is a canary ring of IT and engineering hosts, a broad ring of general users, and a last ring holding domain controllers, database primaries and hypervisors. Domain controllers go last, and never all at once.

The rollback plan has to name the actual mechanism. For Windows that is wusa /uninstall /kb:5009557 or dism /Online /Remove-Package, and for a domain controller already in a boot loop it is Safe Mode plus offline package removal, which needs console access you may not have on a remote site. For a container it is redeploying the previous image digest, not the previous tag.

Failure mode one: patched on disk, still resident

The patch replaced the file. The process still holds the old inode. On Linux the deleted-but-mapped library is visible directly.

lsof +c0 -d DEL 2>/dev/null | grep -E '\.so'
dnf needs-restarting
needrestart -b

dnf needs-restarting walks running processes and reports those using files from packages updated after the process started. needrestart does the same job on Debian derivatives. On Windows the same problem shows up as a loaded DLL that cannot be replaced until reboot, which is why the servicing stack queues the swap through PendingFileRenameOperations instead of failing.

Log4Shell was the mass-scale version. Teams upgraded the JAR, the application server kept running, and the vulnerable class stayed loaded until someone restarted the service. The safe assumption is that no library patch takes effect without a process restart.

Failure mode two: supersedence

Windows quality updates are cumulative. The March LCU contains everything in the February LCU, so after March the February KB number is not present as an installed hotfix. A scanner or script that asks "is KB5009557 installed" gets a negative answer on a host that is fully patched.

Get-HotFix makes this worse. It queries the WMI Win32_QuickFixEngineering class, which returns only the updates supplied by Component Based Servicing and omits those supplied by Windows Installer or the Windows update site. Do not build a compliance check on it. Query the build revision instead.

Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion' |
  Select-Object ProductName, CurrentBuild, UBR

CurrentBuild plus UBR gives the exact patch level, and it maps one-to-one to a published KB. A host reporting 17763.2458 is running the January 2022 out-of-band update for Server 2019, and no amount of missing KB entries changes that.

The inverse failure is nastier. Product families where updates are not cumulative across baselines let a newer install roll you backwards. Exchange security updates are built against a specific cumulative update, so installing a newer CU returns the server to that CU's baseline until the matching SU is installed again. The inventory says "updated". The build number says otherwise.

Failure mode three: installed but not effective

Two documented cases, both worth knowing because both are silent.

Microsoft's Exchange security updates, installed by double-clicking the .msp on a server with UAC enabled, do not fully apply. Some files are not updated correctly and no error is shown. OWA and ECP break, and the fix is reinstalling from an elevated prompt and running UpdateCas.ps1 and UpdateConfigFiles.ps1 from an elevated Exchange Management Shell. An operator following the obvious procedure produces a server that reports the SU as installed and is not protected by it.

PrintNightmare (CVE-2021-34527) is the configuration variant. The July 6, 2021 updates contain the code fix, but the fix only holds when RestrictDriverInstallationToAdministrators is set to 1, which became the default only with the August 10, 2021 updates. Point and Print settings override it. A host with NoWarningNoElevationOnInstall set to 1 stays exploitable with the patch installed.

HKLM\SOFTWARE\Policies\Microsoft\Windows NT\Printers\PointAndPrint
  RestrictDriverInstallationToAdministrators = 1
  NoWarningNoElevationOnInstall = 0
  UpdatePromptSettings = 0

Failure mode four: reboot deferral

The update installed, the user clicked "remind me later" for six weeks, and the compliance dashboard has been green the whole time. This is the most common way patch reporting lies, and the check is cheap.

Test-Path 'HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Component Based Servicing\RebootPending'
Test-Path 'HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\RebootRequired'
(Get-CimInstance Win32_OperatingSystem).LastBootUpTime

test -f /var/run/reboot-required && cat /var/run/reboot-required.pkgs
uptime -s

Compare last boot time against patch install time. If boot is older, the reported state is a claim about the filesystem, not about the running kernel.

Emergency patching, honestly

Sometimes the patch is worse than the bug. On January 11, 2022, Microsoft's Patch Tuesday updates for Windows Server (KB5009624 for 2012 R2, KB5009557 for 2019, KB5009555 for 2022) put domain controllers into reboot loops driven by LSASS, prevented Hyper-V from starting, and left ReFS volumes inaccessible. Out-of-band updates followed on January 17 and 18, including KB5010791 for Server 2019, and reporting at the time indicated the ReFS problem was not fully resolved by the OOB alone.

That is the tradeoff in concrete form. An organization that pushed everything on day one lost its directory. One that held the DC ring for seven days did not. A CVSS 9.8 with no observed exploitation does not justify skipping the canary. A CISA KEV entry under active exploitation might. That call should be made by a named person against a written threshold, not improvised at 22:00.

Verification means reading the binary

Patch state is a claim until you check the artifact. Three checks that actually answer the question:

(Get-Item C:\Windows\System32\ntoskrnl.exe).VersionInfo.FileVersion
Get-Command ExSetup.exe | ForEach-Object { $_.FileVersionInfo.FileVersion }
dism /Online /Get-Packages /Format:Table

rpm -q openssl
openssl version -a
systemctl show sshd -p ActiveEnterTimestamp

The last one matters more than it looks. It tells you when the service started, which tells you whether it started before or after the package changed.

The detection has to land first

Patching lags exploitation by definition, and the gap for a zero-day is the whole window. The defensive answer is to have the exploitation behavior detected before the fix exists, so the patch cycle is a cleanup rather than the only control. This is the same argument as behavior over indicators in detection engineering, applied to the vulnerability lifecycle.

Most public-facing exploitation (T1190) converges on one observable: a web server worker process spawning a shell. It does not matter which CVE got the attacker there.

title: Suspicious child process spawned by a web server worker
status: experimental
description: Detects shells and discovery binaries spawned by IIS, Apache, nginx or Tomcat worker processes, the common post-exploitation step after a web vulnerability or web shell drop
references:
    - https://github.com/SigmaHQ/sigma/blob/master/rules/windows/process_creation/proc_creation_win_webshell_susp_process_spawned_from_webserver.yml
tags:
    - attack.initial-access
    - attack.persistence
    - attack.t1190
    - attack.t1505.003
logsource:
    category: process_creation
    product: windows
detection:
    selection_parent:
        ParentImage|endswith:
            - '\w3wp.exe'
            - '\httpd.exe'
            - '\nginx.exe'
            - '\php-cgi.exe'
            - '\tomcat.exe'
            - '\UMWorkerProcess.exe'
    selection_child:
        Image|endswith:
            - '\cmd.exe'
            - '\powershell.exe'
            - '\pwsh.exe'
            - '\wscript.exe'
            - '\cscript.exe'
            - '\net.exe'
            - '\net1.exe'
            - '\whoami.exe'
            - '\nltest.exe'
            - '\certutil.exe'
            - '\bitsadmin.exe'
    condition: selection_parent and selection_child
falsepositives:
    - Web applications that shell out by design, such as monitoring agents and some management consoles
    - Deployment and build tooling running under the app pool identity
level: high

This is a trimmed version of SigmaHQ rule 8202070f-edeb-4d31-a010-a26c72ac5600. What it catches: the interactive step after a web shell lands, across every CVE that produces one. What it misses: in-process exploitation that never forks, .NET payloads executed inside w3wp.exe itself, and any attacker who goes straight to a C2 implant without touching cmd.exe. Those need image load and network telemetry from the worker process.

The false positives are site-specific. Run it in audit for two weeks, catalog the applications that legitimately shell out, then filter on the parent command line or the app pool identity rather than dropping a child binary from the list.

Summary

Patch management is a delivery pipeline with a security SLA on top, and the pipeline fails in ways that reporting does not show. A patch on disk with the service still running, a superseded KB that reads as missing, an update installed without the elevation it needed, and a deferred reboot all produce a green dashboard over a vulnerable host. Verify against the build revision, the file version and the service start time, not against the deployment tool's own status field. Because exploitation now averages days rather than weeks, the detection for the exploitation behavior has to be live before the patch ships, not after.

Verification checks worth automating
  Windows:  CurrentBuild + UBR from HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion
            RebootPending and RebootRequired keys
            LastBootUpTime vs update install time
  Linux:    dnf needs-restarting -r  /  needrestart -b
            lsof +c0 -d DEL | grep '\.so'
            /var/run/reboot-required
  Services: systemctl show <unit> -p ActiveEnterTimestamp

ATT&CK techniques referenced
  T1190      Exploit Public-Facing Application
  T1505.003  Server Software Component: Web Shell
  T1068      Exploitation for Privilege Escalation
  T1210      Exploitation of Remote Services

Exploitation detections that cover the window before a patch lands are in the catalog under /rules?domain=windows, and the web-facing set is at /rules?domain=web.