Container Security: Multiple Kubernetes Misconfigurations in YAML

[Container Security] Abuse of CAP_DAC_READ_SEARCH leads to container escape

docker:{"cap_add":["DAC_READ_SEARCH"],"command":"sleep 9999"}

k8s:

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  containers:
  - name: example-container
    image: busybox  # 请根据需要替换为实际的镜像
    command: ["sleep", "9999"]
    securityContext:
      capabilities:
        add: ["DAC_READ_SEARCH"]

[Create POD] pod with SYS_ADMIN permissions

apiVersion: v1
kind: Pod
metadata:
  name: sys-admin-pod
  labels:
    app: sys-admin-pod
spec:
  serviceAccountName: service_account
  containers:
    - name: sys-admin-containernode_proxy
      image: registry_url/busybox:latest
      imagePullPolicy: IfNotPresent
      securityContext:
        capabilities:
          add:
            - SYS_ADMIN
      command:
        - sh
        - '-c'
        - sleep 1h

docker:

{"cap_add":["SYS_ADMIN"],"command":"sleep 9999"}

Containers with CAP_SYS_MODULE permissions

{"cap_add":["SYS_MODULE"],"command":"sleep 9999"}

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  containers:
  - name: example-container
    image: busybox  # 请根据需要替换为实际的镜像
    command: ["sleep", "9999"]
    securityContext:
      capabilities:
        add: ["SYS_MODULE"]

k8s-backdoor_daemonset backdoor

{
    "metadata": {
        "name": "daemonset",
        "labels": {
            "app": "daemonset"
        }
    },
    "labels": {
        "app": "daemonset"
    },
    "spec": {
        "selector": {
            "matchLabels": {
                "app": "daemonset"
            }
        },
        "template": {
            "metadata": {
                "labels": {
                    "app": "daemonset"
                }
            },
            "spec": {
                "serviceAccountName": "service_account",
                "containers": [
                    {
                        "command": [
                            "sleep",
                            "999"
                        ],
                        "image": "registry_url/busybox:latest",
                        "name": "daemonset-container"
                    }
                ]
            }
        }
    }
}

k8s_shadow_apiserver

The process of creating a shadow API server in a Kubernetes cluster can be divided into the following steps:

1. Getkube-systemunder namespacekube-apiserver Pod

First, we need to findkube-apiserverPod. existkube-systemRun the following command under the namespace:

kubectl get pods -n kube-system -l component=kube-apiserver

This will list allkube-apiserver Pod。

2. Getkube-apiserverPod configuration information

turn upkube-apiserverAfter the Pod, get its details:

kubectl get pod <kube-apiserver-pod-name> -n kube-system -o yaml > kube-apiserver-pod.yaml

this will putkube-apiserverPod configuration information is saved tokube-apiserver-pod.yamlin the file.

3. Reconstruct Pod configuration information

editkube-apiserver-pod.yamlfile and make the following changes:

• deleteaudit-logrelated configuration.

• Revisesecure-portto 9444.

turn upcontainerssection, make the following modifications:

containers:
- name: kube-apiserver
  image: k8s.gcr.io/kube-apiserver:v1.20.0
  command:
  - kube-apiserver
  - --advertise-address=10.0.0.1
  - --allow-privileged=true
  - --authorization-mode=Node,RBAC
  - --client-ca-file=/etc/kubernetes/pki/ca.crt
  - --enable-admission-plugins=NodeRestriction
  - --enable-bootstrap-token-auth=true
  - --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
  - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
  - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
  - --etcd-servers=https://127.0.0.1:2379
  - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
  - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
  - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
  - --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt
  - --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key
  - --requestheader-allowed-names=front-proxy-client
  - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
  - --requestheader-extra-headers-prefix=X-Remote-Extra-
  - --requestheader-group-headers=X-Remote-Group
  - --requestheader-username-headers=X-Remote-User
  - --secure-port=9444  # 修改 secure-port
  - --service-account-issuer=https://kubernetes.default.svc.cluster.local
  - --service-account-key-file=/etc/kubernetes/pki/sa.pub
  - --service-account-signing-key-file=/etc/kubernetes/pki/sa.key
  - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
  - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
  # 删除 audit-log 相关配置项
  ports:
  - containerPort: 9444  # 确保端口一致
    name: https
    protocol: TCP

4. Deploy Shadow Pod

In order to avoid conflicts with existingkube-apiserverPod conflicts, it is recommended tometadata.nameChange it to a new name, for examplekube-apiserver-shadow：

metadata:
  name: kube-apiserver-shadow
  namespace: kube-system

Then deploy the new Shadow Pod using the following command:

kubectl apply -f kube-apiserver-pod.yaml

Things to note

1. Permissions and certificates: Ensure that the newly created Shadow Pod has permissions to access the necessary resources and that the certificate path is correct.

1. resource conflict: Ensure that the new Pod’s ports and resources are not identical to those of existingkube-apiserverPod conflict.

1. security: modifiedsecure-portand other configurations should comply with the cluster's security policy.

Through the above steps, you can create a shadow API server in the Kubernetes cluster for testing or other purposes.

k8s_backdoor_cronjob

{
    "apiVersion": "batch/v1",
    "kind": "CronJob",
    "metadata": {
        "name": "cronjob"
    },
    "spec": {
        "schedule": "*/1 * * * *",
        "jobTemplate": {
            "spec": {
                "template": {
                    "spec": {
                        "serviceAccountName": "service_account",
                        "containers": [
                            {
                                "name": "cronjob-container",
                                "image": "registry_url/busybox:latest",
                                "imagePullPolicy": "IfNotPresent",
                                "command": [
                                    "sleep",
                                    "1234"
                                ]
                            }
                        ],
                        "restartPolicy": "OnFailure"
                    }
                }
            }
        }
    }
}

k8s_node_proxy

Node Proxy in Kubernetes is a proxy mechanism used to access nodes in a Kubernetes cluster. Improper configuration can cause security and performance issues. Here are some common misconfigurations:

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  ports:
    - port: 80
      targetPort: 80
  selector:
    app: my-app

mount-docker-sock

docker:

version: '3.8'

services:
docker-cli:
image: docker:latest
container_name: docker-cli
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /usr/bin/docker:/usr/bin/docker
command: ["sh", "-c", "while true; do sleep 3600; done"]


mount-host-etc

apiVersion: v1
kind: Pod
metadata:
  name: risky-pod
spec:
  containers:

• name: risky-container
image: alpine:latest
command: ["sh", "-c", "while true; do sleep 3600; done"]
volumeMounts:

  • name: etc-volume
mountPath: /etc
volumes:

• name: etc-volume
hostPath:
path: /etc


mount-host-procfs

docker

{
"command": "sleep 1000",
"volumes": [
"volume_path",
"/host_proc"
],
"binds": {
"volume_path": {
    "bind": "volume_path",
    "mode": "rw"
},
"/proc": {
    "bind": "/host_proc",
    "mode": "rw"
}
}
}

k8s:

apiVersion: v1
kind: Pod
metadata:
  name: insecure-procfs-pod
spec:
  containers:

• name: insecure-container
image: ubuntu:latest
command: ["/bin/sh", "-c", "sleep 3600"]
securityContext:
privileged: true # 以特权模式运行
volumeMounts:

  • name: procfs
mountPath: /host/proc
mountPropagation: HostToContainer
volumes:

• name: procfs
hostPath:
path: /proc # 将宿主机的 /proc 文件系统挂载到容器中
type: Directory