In-depth analysis of uevent_helper container escape vector

In-depth analysisuevent_helperContainer escape vector

Introduction: Recently, I encountered uevent_helper escape when doing WIZ-CTF questions, but I have never understood its detailed principle, so I wrote this article.

Q&A

Q: How do conventional detection tools such as linpeas detect the existence of uevent_helper container escape?

A：

linpeas中的的响应代码:
uevent_helper_breakout="$( (echo -n '' > /sys/kernel/uevent_helper && echo Yes) 2>/dev/null || echo No)"

it tries to write/sys/kernel/uevent_helperdocument:

• If it can be written successfully, the variableuevent_helper_breakoutThe value isYes,representThere is a loophole in the system 😱。

• If the write fails, the value isNo,representCurrent path is safe 👍。

Q: Are there any automated internal and external scanning and automated security patching solutions or device alarm rules and supervision solutions for this vulnerability?

A: Next, we will introduce the key points, analyze the principles behind it, and use ideas to explore protection solutions.

Part One: Linux Uevent Subsystem anduevent_helperhistorical legacy

1.1 Kernel to user space communication: Kobjects, Netlink and Uevents

Linux kernel adoptionuevents(User events) As a mechanism for communication between the kernel and user space, it is usually used for changes in device status (such as addition, removal, change. These events are related to the basic data structures in the kernel device modelkobjectslife cycle is closely linked. The modern and recommended way of event delivery is vianetlinksocket(NETLINK_KOBJECT_UEVENT), user space daemons (such asudevd) will listen to this socket to receive events. A typical uevent message contains key-value pairs, e.g.ACTION、DEVPATH、SUBSYSTEMandSEQNUM。

Understanding this framework is crucial to recognizinguevent_helperWhy it is an abnormal existence is crucial. A Netlink socket is an efficient, asynchronous, many-to-many communication channel. It allows multiple userspace listeners (such asudevd) receives events without the kernel needing to care who is listening. This decoupled design is at the heart of modern Linux device management

1.2 uevent_helperThe original role of: From hot-swappable scripts to potential security risks

uevent_helperIs a user-mode helper program (user-mode helper), the kernel willevery ueventThe event forks a process of the program. Historically, the path usually points to/sbin/hotplug, was the primary mechanism for handling device events before netlink-based systems became a standard. The path to this helper can be accessed at runtime by specifying/sys/kernel/uevent_helperFile is written to configure.

This "fork a process per event" model proves to be extremely inefficient and can result in extremely high system load or even out-of-memory conditions during system startup or device discovery storms. It is this performance bottleneck that became the main reason why it was replaced by the netlink mechanism and gradually abandoned.

1.3 ModernudevDaemon process anduevent_helperof abandonment

udev（userspace /dev) is the device manager of the modern Linux kernel and has becomesystemdpart of.systemd-udevd.serviceThe service listens for kernel uevents from the netlink socket and responds accordingly/etc/udev/rules.d/The rules defined in the directory are processed. In most modern distributions, the kernel configuration optionUEVENT_HELPER_PATHThe default value of has been changed to an empty string (""), thus disabling this feature by default. The kernel documentation clearly states that this mechanism "should no longer be used today".

Although disabled by default, its underlying kernel code (CONFIG_UEVENT_HELPER) is usually still compiled into the kernel (default y), and if the system is not properly hardened,/sys/kernel/uevent_helperThe interface is still writable. This creates a dangerous gap: a feature that is considered obsolete and disabled by default can be re-enabled at runtime by a privileged process. This phenomenon can be considered an "architectural residue". The evolution of the kernel—from a simple, synchronous process-fork model to a complex, asynchronous event-driven model—bequeathed a legacy of powerful, high-privilege execution primitives. This primitive is retained for backward compatibility, but it is not part of modernudevpart of the threat model and therefore often overlooked in container security configuration. The attackers exploited not a vulnerability in the code logic, but an architectural decision that allowed an abandoned, high-privilege "backdoor" to remain active and configurable at runtime.

Part 2: Anatomy of Container Escape

2.1 Attack surface analysis:

This attack is essentially impossible without high privileges within the container. The main triggering condition is--privilegedFlags to run a container, or more specifically, to give to a containerCAP_SYS_ADMINability.-privilegedThe flag grants all abilities and removes many safety restrictions, making it the most dangerous configuration.CAP_SYS_ADMINOften referred to as the "new root" because it grants broad administrative rights, including executingmountsystem calls and/sysand/procWrite permissions for many paths. The core of the attack is to/sys/kernel/uevent_helperWrite data./sysThe pseudo file system is an interface to kernel data structures. By default, withCAP_SYS_ADMINIn the container, this file system is mounted in read-write mode, allowing modification. In addition, the attack process within the container must beroot(UID 0) user because even if there isCAP_SYS_ADMIN, non-root users may also face additional Discretionary Access Control (DAC) restrictions. The container must also share the same user namespace as the host, because the helper program being executed will run in the context of the host.

2.2 Attack chain: step-by-step walkthrough

1. Step 1: Create a malicious payload

1. Step 2: Parse the host file system path (OverlayFS)

1. Step 3: Overwrite the uevent_helper path

1. Step 4: Trigger kernel execution from user space

Part 3: Comparative analysis of user-mode helper program escapes

3.1 cgroupfs release_agentVectors: a parallel technology

cgroupofrelease_agentIs another user-mode helper program. The path to the executable can be written to therelease_agentin the file. When a cgroup in the hierarchynotify_on_releaseflag is set and the cgroup becomes empty (the last process exits), the kernel executesrelease_agentThe specified binary file. This technology has been known for years and has been used in real-world attacks to deploy cryptocurrency miners.

3.2 Decisive factor: Why user namespace can trigger CVE-2022-0492, but cannot be used foruevent_helperescape

Historically,release_agentanduevent_helperEscapes all need to be in the initial user namespace.CAP_SYS_ADMIN. However, CVE-2022-0492 demonstrates that one is inside the containerNo CAP_SYS_ADMINAn attacker can create a new user namespace, which would give themwithin this new namespaceofCAP_SYS_ADMINpermissions. This is enough to mount a writablecgroupfsand setrelease_agent 。

The same technology pairuevent_helperbut doesn't work because trying to mount in a new user namespacesysfswill be explicitly blocked by the kernel. Kernelmount_too_revealingChecked bysysfsandprocfsonSB_I_USERNS_VISIBLEflag triggers, this check prevents these file systems from being mounted in the new user namespace if they contain masked paths (this is standard practice in container environments).

cgroupfsThis flag and corresponding check are missing, which is the crux of this CVE. This technical difference is crucial and explains why in modern environmentsrelease_agentis considered a more severe vulnerability and received a CVE number, whileuevent_helperStill a problem that relies on misconfiguration.

3.3 core_pattern、modprobewith a common threat model

The References material lists other user-mode helpers that follow the same pattern:/proc/sys/kernel/core_pattern(execute helper program when process crashes) and/proc/sys/kernel/modprobe(Executes a helper program for loading kernel modules). All of these represent a common threat model: a privileged kernel execution mechanism controlled by a writable file in a pseudo file system.

The following table distills a complex analysis of different but related escape vectors into a clear, concise reference for use by security architects and penetration testers. It enables rapid risk assessment by comparing the preconditions and exploitability of each vector.

Part 4: Detection Framework: From Configuration to Runtime

This section will provide security teams with actionable guidance to detectuevent_helperEscape conditions and active exploitation behavior.

4.1 Static analysis and compliance scanning

CIS Docker Benchmark provides hardening guidance for container environments. Although it does not specifically nameuevent_helper, but its controls map directly to the prerequisites for an attack. Key controls include a recommendation not to run privileged containers and a recommendation to remove all non-essential capabilities. Tools like Docker Bench for Security can automate these checks.

Trivy is a versatile scanner that audits Kubernetes resources for misconfigurations. Security teams can write custom Rego policies for Trivy or use its built-in checks to scan Pod security policies or security contexts for allowedprivileged: trueor insecurityContext.capabilities.addThe array containsSYS_ADMINConfiguration of capabilities.

4.2 Runtime threat detection using Falco

Falco is a runtime security tool that monitors kernel system calls to detect anomalous behavior. The core logic to detect this attack is to monitor the processes running inside the container (container.id!= host) to open a specific file for writing/sys/kernel/uevent_helperbehavior.

Based on the above logic, a specific Falco rule can be formulated. This rule should have high priority (e.g.CRITICAL), because any write to this file from within the container is extremely suspicious and almost certainly malicious.

rule: Container Write to uevent_helper 
desc: "Detects a container process writing to /sys/kernel/uevent_helper, a known container escape vector." 
condition: > 
  open_write and fd.name = "/sys/kernel/uevent_helper" and container.id!= "host" 
output: > 
  Container escape attempt detected. A process in a container wrote to /sys/kernel/uevent_helper. 
  (user=%user.name command=%proc.cmdline container_id=%container.id image=%container.image.repository:%container.image.tag) 
priority: CRITICAL 
tags: [container, escape, mitre_execution_t1059]

The logic for this rule is adapted from the one used to detectrelease_agentThoughts of escape. The detection of this escape vector perfectly illustrates the symbiotic relationship between static "shift left" safety and runtime safety. Static analysis of configuration (IaC, manifest files) acts as a policy gate to prevent the creation of vulnerable conditions. Detect the exploitation of these conditions if they are gated by policy or created dynamically. .

Part 5: Hardening Principles and Mitigation Strategies

This section details specific, layered defenses to proactively preventuevent_helperEscape vector.

5.1 Enforcing minimum privileges:

The most effective and fundamental mitigation is to follow the principle of least privilege. Don't run containers as root. Do not use--privilegedlogo. Explicitly remove all abilities (-cap-drop=all) and only add the specific capabilities necessary for the application's functionality. For Kubernetes, Pod security standards should be used such asRestricted), disabling privileged containers and limiting capabilities.

5.2 File system hardening: mount in read-only mode/sys

Although not the default in most container runtimes, it is technically possible to/sysMount read-only within the container. This will be done by making/sys/kernel/uevent_helperIt cannot be written to directly prevent attacks. This is an advanced hardening technique that can be implemented through custom security profiles.

5.3 Active blocking using Mandatory Access Control (MAC)

AppArmor is a Linux kernel security module that can restrict programs to a limited set of resources using per-application configuration files. It is possible to create a custom AppArmor profile that explicitly denies/sys/kernel/uevent_helperwrite permissions even if the container starts withCAP_SYS_ADMINOperation can also provide precise defense.

Sample AppArmor configuration file：

#include <tunables/global>
profile custom-container-deny-uevent-helper flags=(attach_disconnected) {
  #include <abstractions/base>
  # 继承默认的 docker 配置文件规则
  #include <abstractions/docker>
  # 明确拒绝向 uevent_helper 的写入权限
  deny /sys/kernel/uevent_helper w,
}

This profile will useapparmor_parserLoaded to the host node. In Docker, you can pass-security-opt apparmor=custom-container-deny-uevent-helperApplication. In Kubernetes, it is applied through Pod annotations:container.apparmor.security.beta.kubernetes.io/: localhost/custom-container-deny-uevent-helper

Mandatory access control (MAC) systems like AppArmor provide a critical "defense in depth" layer that decouples file access permissions from the Linux capability model. This allows administrators to create “privileged but constrained” workloads by precisely blockingCAP_SYS_ADMINTo mitigate the risk of the most dangerous applications of overbroad capabilities. Although the primary mitigation is "Do not grantCAP_SYS_ADMIN”, but some legitimate system-level workloads (e.g. monitoring agents, network plug-ins) may require this capability to work properly. AppArmor operates in another dimension: file path access control. It does not care what capabilities a process has, but rather enforces its own set of rules based on the files being accessed. By creating an AppArmor profile like this, we can allow the container to retain what it legitimately needsCAP_SYS_ADMIN, while blocking this specific known escape vector. Therefore, AppArmor becomes a crucial compensatory control that complements the coarse-grained control of the capability model with path rule-based accuracy.

References

• Kubernetes AppArmor Security Tutorial: https://kubernetes.io/docs/tutorials/security/apparmor/

• Research paper on container escape detection using dependency graphs: https://www.mdpi.com/2079-9292/13/23/4773

• Research on improving Kubernetes security using Trivy and Falco together: https://www.theseus.fi/handle/10024/891495

• Linux kernel documentation: device-mapper uevent: https://docs.kernel.org/5.17/admin-guide/device-mapper/dm-uevent.html

• Technical analysis of "HotPlugEater" attack using uevent_helper: https://stigward.github.io/posts/fiio-m6-exploit/

• How the uevents and udev daemons work: https://sid-project.github.io/context.html

• CrowdStrike: Block container escape attempts: https://www.crowdstrike.com/en-us/blog/preventing-container-escape-attempts-falcon-cloud-runtime-security/

• Stack Exchange: How udev/uevent works: https://unix.stackexchange.com/questions/550037/how-does-udev-uevent-work

• Microsoft Azure Documentation: Use AppArmor to secure container access in AKS: https://learn.microsoft.com/en-us/azure/aks/secure-container-access

• CIS Benchmarks List: https://www.cisecurity.org/cis-benchmarks

• Linux Kernel Labs: Device Models: https://linux-kernel-labs.github.io/refs/pull/189/merge/labs/device_model.html

• Palo Alto Networks Unit 42: Container Escape Technology: https://unit42.paloaltonetworks.com/container-escape-techniques/

• AWS Blog: CVE-2019-5736 runc container escape analysis: https://aws.amazon.com/blogs/compute/anatomy-of-cve-2019-5736-a-runc-container-escape/

• SUSE Linux documentation: udev: https://documentation.suse.com/en-us/sles/12-SP5/html/SLES-all/cha-udev.html

• Sysdig: Using Falco to detect container escape capabilities: https://sysdig.com/blog/container-escape-capabilities-falco-detection

• Trivy Operator Quick Start Guide: https://aquasecurity.github.io/trivy-operator/v0.11.0/getting-started/quick-start/

• Docker Documentation: AppArmor Security Profiles: https://docs.docker.com/engine/security/apparmor/

• Palo Alto Networks: “Leaky Vessels” Container Escape Vulnerability: https://www.paloaltonetworks.com/blog/cloud-security/leaky-vessels-vulnerabilities-container-escape/

• Wiz Blog: “Leaky Vessels” Container Escape Vulnerability: https://www.wiz.io/blog/leaky-vessels-container-escape-vulnerabilities

• Linux Kernel Documentation: Fallback Mechanism for Firmware Loading: https://www.kernel.org/doc/html/v4.14/driver-api/firmware/fallback-mechanisms.html

• Wiz Academy: Docker Bench for Security Overview: https://www.wiz.io/academy/docker-bench-for-security-overview

• Infosec Writeups: Container Escape Technology: https://infosecwriteups.com/container-escape-techniques-breaking-out-of-the-digital-jail-ad06962c5292

• Sysdig Documentation: Docker/CIS Benchmarks: https://docs.sysdig.com/en/docker-cis-benchmarks/

• "Open Source For You" magazine: uevent mechanism: https://issuu.com/hibadweib/docs/open_source_for_you_-_october_2012/s/13663276

• GitHub repository: InSpec configuration file for CIS Docker Benchmark: https://github.com/dev-sec/cis-docker-benchmark

• Linux Kernel Documentation: AppArmor: https://www.kernel.org/doc/html/v4.15/admin-guide/LSM/apparmor.html