flawopen.com/Teardowns/cve-2024-5321-kubernetes-windows-container-log-permissions

● CVE-2024-5321 · CVSS 6.1 · Medium
FlawOpen Security Research

CVE-2024-5321: Kubernetes Windows Node Container Log Permissions

CVE-2024-5321 (CVSS 6.1, Medium): on Windows nodes the kubelet created pod log directories with os.MkdirAll, which ignores the permission mode on Windows, so the logs inherited an ACL that let BUILTIN\Users read them and Authenticated Users modify them.

💡 Plain English Explainer (ELI5)

A hospital keeps each patient's chart in a filing cabinet. When staff add a new cabinet they write "doctors only" on the order form, but in this building the form's lock field is silently ignored, so the cabinet just takes the room's default rule: any employee may read it and anyone with a badge may write in it. The fix is a clerk who, after every new cabinet arrives, fits its own lock and throws away the room's default rule for that cabinet.

Core Concepts & Subsystem Terms

DACL / ACE
A Windows discretionary access control list: the list of access control entries that say which accounts may read, write or execute a file or directory.
ACL inheritance
By default a new Windows file or directory copies the inheritable entries of its parent's DACL. Setting a protected DACL stops that.
BUILTIN\Users / Authenticated Users
Broad Windows groups: every local user account, and every account that has logged on at all. Kubernetes maps containers running as ContainerUser onto BUILTIN\Users when they touch host files.
os.MkdirAll on Windows
Go's os.MkdirAll(path, perm) only uses perm on Unix. On Windows the mode is ignored and the directory gets the inherited ACL.

Root Cause Analysis

setupDataDirs() in pkg/kubelet/kubelet.go created the pod log root (C:\var\log\pods) with os.MkdirAll(dir, 0750). Go ignores the mode on Windows, so the directory, and the log files written under it, inherited the parent's ACL, which granted BUILTIN\Users read and NT AUTHORITY\Authenticated Users modify access. PR #126104 added MkdirAll() and Chmod() to pkg/util/filesystem that translate the Unix mode into an explicit DACL and apply it with PROTECTED_DACL_SECURITY_INFORMATION, and switched the kubelet's log, plugin and checkpoint directories to them.

Step-by-Step Attack Flow

Step 1

Kubelet creates the log root

On a Windows node, setupDataDirs() calls os.MkdirAll(kl.getPodLogsDir(), 0750). The 0750 is ignored and the directory inherits the ACL of its parent.

Step 2

Broad groups inherit access

The inherited entries give BUILTIN\Users read access and NT AUTHORITY\Authenticated Users modify access to the pod log directories and the files the runtime writes into them.

Step 3

Reading other workloads' logs

Any low-privileged local account on the node, or a workload with the log path mounted through hostPath, reads other pods' stdout and stderr, which often contain tokens, connection strings or customer data.

Step 4

Tampering with the evidence

The same accounts can edit or truncate log files before a log shipper collects them, hiding activity from central logging.

Source Code: Flaw vs. Secure Implementation

UNPATCHED FLAW
// pkg/kubelet/kubelet.go (kubelet v1.30.2)
func (kl *Kubelet) setupDataDirs() error {
	if err := os.MkdirAll(kl.getRootDir(), 0750); err != nil {
		return fmt.Errorf("error creating root directory: %v", err)
	}
	// BUG on Windows: os.MkdirAll ignores the 0750 mode. The directory inherits
	// its parent's ACL, which lets BUILTIN\Users read and
	// NT AUTHORITY\Authenticated Users modify every pod's log files.
	if err := os.MkdirAll(kl.getPodLogsDir(), 0750); err != nil {
		return fmt.Errorf("error creating pod logs root directory %q: %w", kl.getPodLogsDir(), err)
	}
	if err := os.MkdirAll(kl.getPodsDir(), 0750); err != nil {
		return fmt.Errorf("error creating pods directory: %v", err)
	}
	return nil
}
HARDENED SECURE PATCH
// pkg/kubelet/kubelet.go (fixed in v1.30.3, PR #126104)
func (kl *Kubelet) setupDataDirs() error {
	if err := os.MkdirAll(kl.getRootDir(), 0750); err != nil {
		return fmt.Errorf("error creating root directory: %v", err)
	}
	// FIX: utilfs.MkdirAll is plain os.MkdirAll on Linux. On Windows it also
	// calls utilfs.Chmod, which replaces the inherited ACL with an explicit one.
	if err := utilfs.MkdirAll(kl.getPodLogsDir(), 0750); err != nil {
		return fmt.Errorf("error creating pod logs root directory %q: %w", kl.getPodLogsDir(), err)
	}
	if err := os.MkdirAll(kl.getPodsDir(), 0750); err != nil {
		return fmt.Errorf("error creating pods directory: %v", err)
	}
	return nil
}

// pkg/util/filesystem/util_windows.go
func MkdirAll(path string, perm os.FileMode) error {
	if err := os.MkdirAll(path, perm); err != nil {
		return fmt.Errorf("Error creating directory %s: %v", path, err)
	}
	return Chmod(path, perm)
}

func Chmod(path string, filemode os.FileMode) error {
	// Maps owner, group and other mode bits to ACEs for the file's owner SID,
	// its group SID and BUILTIN\Users (BU). For 0750 the result is
	// "D:(A;OICI;FA;;;<owner>)(A;OICI;FRFX;;;<group>)(A;OICI;;;;BU)".
	// (Abridged: the real function builds this string inline.)
	dacl, err := daclForMode(path, filemode)
	if err != nil {
		return err
	}
	newSD, err := windows.SecurityDescriptorFromString(dacl)
	if err != nil {
		return fmt.Errorf("Error creating new security descriptor from DACL string: %v", err)
	}
	newDACL, _, err := newSD.DACL()
	if err != nil {
		return fmt.Errorf("Error getting DACL from new security descriptor: %v", err)
	}
	// PROTECTED_DACL_SECURITY_INFORMATION blocks inheritance from the parent,
	// so the BUILTIN\Users and Authenticated Users entries no longer apply.
	return windows.SetNamedSecurityInfo(path, windows.SE_FILE_OBJECT,
		windows.DACL_SECURITY_INFORMATION|windows.PROTECTED_DACL_SECURITY_INFORMATION,
		nil, nil, newDACL, nil)
}

Engineering & System Hardening Checklist

Sources