# Kubernetes Hardening

{{#include ../../../banners/hacktricks-training.md}}

## Tools to analyse a cluster

### [Steampipe - Kubernetes Compliance](https://github.com/turbot/steampipe-mod-kubernetes-compliance)

It's will **several compliance checks over the Kubernetes cluster**. It includes support for CIS, National Security Agency (NSA) and Cybersecurity and Infrastructure Security Agency (CISA) Cybersecurity technical report for Kubernetes hardening.

```bash
# Install Steampipe
brew install turbot/tap/powerpipe
brew install turbot/tap/steampipe
steampipe plugin install kubernetes

# Start the service
steampipe service start

# Install the module
mkdir dashboards
cd dashboards
powerpipe mod init
powerpipe mod install github.com/turbot/steampipe-mod-kubernetes-compliance

# Run the module
powerpipe server
```

### [**Kubescape**](https://github.com/armosec/kubescape)

[**Kubescape**](https://github.com/armosec/kubescape) is a K8s open-source tool providing a multi-cloud K8s single pane of glass, including risk analysis, security compliance, RBAC visualizer and image vulnerabilities scanning. Kubescape scans K8s clusters, YAML files, and HELM charts, detecting misconfigurations according to multiple frameworks (such as the [NSA-CISA](https://www.armosec.io/blog/kubernetes-hardening-guidance-summary-by-armo) , [MITRE ATT\&CK®](https://www.microsoft.com/security/blog/2021/03/23/secure-containerized-environments-with-updated-threat-matrix-for-kubernetes/)), software vulnerabilities, and RBAC (role-based-access-control) violations at early stages of the CI/CD pipeline, calculates risk score instantly and shows risk trends over time.

```bash
curl -s https://raw.githubusercontent.com/kubescape/kubescape/master/install.sh | /bin/bash
kubescape scan --verbose
```

### [**Popeye**](https://github.com/derailed/popeye)

[**Popeye**](https://github.com/derailed/popeye) is a utility that scans live Kubernetes cluster and **reports potential issues with deployed resources and configurations**. It sanitizes your cluster based on what's deployed and not what's sitting on disk. By scanning your cluster, it detects misconfigurations and helps you to ensure that best practices are in place, thus preventing future headaches. It aims at reducing the cognitive \_over_load one faces when operating a Kubernetes cluster in the wild. Furthermore, if your cluster employs a metric-server, it reports potential resources over/under allocations and attempts to warn you should your cluster run out of capacity.

### [**Kube-bench**](https://github.com/aquasecurity/kube-bench)

The tool [**kube-bench**](https://github.com/aquasecurity/kube-bench) is a tool that checks whether Kubernetes is deployed securely by running the checks documented in the [**CIS Kubernetes Benchmark**](https://www.cisecurity.org/benchmark/kubernetes/).\
You can choose to:

- run kube-bench from inside a container (sharing PID namespace with the host)
- run a container that installs kube-bench on the host, and then run kube-bench directly on the host
- install the latest binaries from the [Releases page](https://github.com/aquasecurity/kube-bench/releases),
- compile it from source.

### [**Kubeaudit**](https://github.com/Shopify/kubeaudit)

**[DEPRECATED]** The tool [**kubeaudit**](https://github.com/Shopify/kubeaudit) is a command line tool and a Go package to **audit Kubernetes clusters** for various different security concerns.

Kubeaudit can detect if it is running within a container in a cluster. If so, it will try to audit all Kubernetes resources in that cluster:

```
kubeaudit all
```

This tool also has the argument `autofix` to **automatically fix detected issues.**

### [**Kube-hunter**](https://github.com/aquasecurity/kube-hunter)

**[DEPRECATED]** The tool [**kube-hunter**](https://github.com/aquasecurity/kube-hunter) hunts for security weaknesses in Kubernetes clusters. The tool was developed to increase awareness and visibility for security issues in Kubernetes environments.

```bash
kube-hunter --remote some.node.com
```

### [Trivy](https://github.com/aquasecurity/trivy)

[Trivy](https://github.com/aquasecurity/trivy) has scanners that look for security issues, and targets where it can find those issues:

- Container Image
- Filesystem
- Git Repository (remote)
- Virtual Machine Image
- Kubernetes


### [**Kubei**](https://github.com/Erezf-p/kubei)

**[Looks like unmantained]**

[**Kubei**](https://github.com/Erezf-p/kubei) is a vulnerabilities scanning and CIS Docker benchmark tool that allows users to get an accurate and immediate risk assessment of their kubernetes clusters. Kubei scans all images that are being used in a Kubernetes cluster, including images of application pods and system pods.

### [**KubiScan**](https://github.com/cyberark/KubiScan)

[**KubiScan**](https://github.com/cyberark/KubiScan) is a tool for scanning Kubernetes cluster for risky permissions in Kubernetes's Role-based access control (RBAC) authorization model.

### [Managed Kubernetes Auditing Toolkit](https://github.com/DataDog/managed-kubernetes-auditing-toolkit)

[**Mkat**](https://github.com/DataDog/managed-kubernetes-auditing-toolkit) is a tool built to test other type of high risk checks compared with the other tools. It mainly have 3 different modes:

- **`find-role-relationships`**: Which will find which AWS roles are running in which pods
- **`find-secrets`**: Which tries to identify secrets in K8s resources such as Pods, ConfigMaps, and Secrets.
- **`test-imds-access`**: Which will try to run pods and try to access the metadata v1 and v2. WARNING: This will run a pod in the cluster, be very careful because maybe you don't want to do this!

## **Audit IaC Code**

### [**KICS**](https://github.com/Checkmarx/kics)

[**KICS**](https://github.com/Checkmarx/kics) finds **security vulnerabilities**, compliance issues, and infrastructure misconfigurations in the following **Infrastructure as Code solutions**: Terraform, Kubernetes, Docker, AWS CloudFormation, Ansible, Helm, Microsoft ARM, and OpenAPI 3.0 specifications

### [**Checkov**](https://github.com/bridgecrewio/checkov)

[**Checkov**](https://github.com/bridgecrewio/checkov) is a static code analysis tool for infrastructure-as-code.

It scans cloud infrastructure provisioned using [Terraform](https://terraform.io), Terraform plan, [Cloudformation](https://aws.amazon.com/cloudformation/), [AWS SAM](https://aws.amazon.com/serverless/sam/), [Kubernetes](https://kubernetes.io), [Dockerfile](https://www.docker.com), [Serverless](https://www.serverless.com) or [ARM Templates](https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/overview) and detects security and compliance misconfigurations using graph-based scanning.

### [**Kube-score**](https://github.com/zegl/kube-score)

[**kube-score**](https://github.com/zegl/kube-score) is a tool that performs static code analysis of your Kubernetes object definitions.

To install:

| Distribution                                        | Command / Link                                                                          |
| --------------------------------------------------- | --------------------------------------------------------------------------------------- |
| Pre-built binaries for macOS, Linux, and Windows    | [GitHub releases](https://github.com/zegl/kube-score/releases)                          |
| Docker                                              | `docker pull zegl/kube-score` ([Docker Hub)](https://hub.docker.com/r/zegl/kube-score/) |
| Homebrew (macOS and Linux)                          | `brew install kube-score`                                                               |
| [Krew](https://krew.sigs.k8s.io/) (macOS and Linux) | `kubectl krew install score`                                                            |

## Tools to analyze YAML files & Helm Charts

### [**Kube-linter**](https://github.com/stackrox/kube-linter)

```bash
# Install Kube-linter
brew install kube-linter

# Run Kube-linter
## lint ./path/to/yaml/or/chart
```

### [Checkov](https://github.com/bridgecrewio/checkov)

```bash
# Install Checkov
pip install checkov

# Run Checkov
checkov -d ./path/to/yaml/or/chart
```

### [kube‑score](https://github.com/zegl/kube-score)

```bash
# Install kube-score
brew install kube-score

# Run kube-score
kube-score score ./path/to/yaml
# or
helm template chart /path/to/chart | kube-score score -
# or if the chart needs some values
helm template chart /path/to/chart \
  --set 'config.urls[0]=https://dummy.backend.internal' \
  | kube-score score -
```

### [Kubesec](https://github.com/controlplaneio/kubesec)

```bash
# Install Kubesec
## Download from https://github.com/controlplaneio/kubesec/releases

# Run Kubesec in a yaml
kubesec scan ./path/to/yaml
# or
helm template chart /path/to/chart | kubesec scan -
# or if the chart needs some values
helm template chart /path/to/chart \
  --set 'config.urls[0]=https://dummy.backend.internal' \
  | kubesec scan -
```


## Scan dependency issues

### Scan images

```bash
#!/bin/bash
export images=$(kubectl get pods --all-namespaces -o jsonpath="{range .items[]}{.spec.containers[].image}{'\n'}{end}" | sort | uniq)
echo "All images found: $images"
echo ""
echo ""
for image in $images; do
    # Run trivy scan and save JSON output
    trivy image --format json --output /tmp/result.json --severity HIGH,CRITICAL "$image" >/dev/null 2>&1
    # Extract binary targets that have vulnerabilities
    binaries=$(jq -r '.Results[] | select(.Vulnerabilities != null) | .Target' /tmp/result.json)
    if [ -n "$binaries" ]; then
        echo "- **Image:** $image"
        while IFS= read -r binary; do
            echo "  - **Binary:** $binary"
            jq -r --arg target "$binary" '
                .Results[] | select(.Target == $target) | .Vulnerabilities[] | 
                "    - **\(.Title)** (\(.Severity)): Affecting `\(.PkgName)` fixed in version `\(.FixedVersion)` (current version is `\(.InstalledVersion)`)."
            ' /tmp/result.json
        done <<< "$binaries"
        echo ""
        echo ""
        echo ""
    fi
done
```

### Scan Helm charts

```bash
#!/bin/bash
# scan-helm-charts.sh
# This script lists all Helm releases, renders their manifests,
# and then scans each manifest with Trivy for configuration issues.

# Check that jq is installed
if ! command -v jq &>/dev/null; then
  echo "jq is required but not installed. Please install jq and rerun."
  exit 1
fi

# List all helm releases and extract namespace and release name
echo "Listing Helm releases..."
helm list --all-namespaces -o json | jq -r '.[] | "\(.namespace) \(.name)"' > helm_releases.txt

# Check if any releases were found
if [ ! -s helm_releases.txt ]; then
  echo "No Helm releases found."
  exit 0
fi

# Loop through each Helm release and scan its rendered manifest
while IFS=" " read -r namespace release; do
  echo "---------------------------------------------"
  echo "Scanning Helm release '$release' in namespace '$namespace'..."
  # Render the Helm chart manifest
  manifest_file="${release}-manifest.yaml"
  helm get manifest "$release" -n "$namespace" > "$manifest_file"
  if [ $? -ne 0 ]; then
    echo "Failed to get manifest for $release in $namespace. Skipping."
    continue
  fi
  # Scan the manifest with Trivy (configuration scan)
  echo "Running Trivy config scan on $manifest_file..."
  trivy config --severity MEDIUM,HIGH,CRITICAL "$manifest_file"
  echo "Completed scan for $release."
done < helm_releases.txt

echo "---------------------------------------------"
echo "Helm chart scanning complete."
```



## Tips

### Kubernetes PodSecurityContext and SecurityContext

You can configure the **security context of the Pods** (with _PodSecurityContext_) and of the **containers** that are going to be run (with _SecurityContext_). For more information read:

{{#ref}}
kubernetes-securitycontext-s.md
{{#endref}}

### Kubernetes API Hardening

It's very important to **protect the access to the Kubernetes Api Server** as a malicious actor with enough privileges could be able to abuse it and damage in a lot of way the environment.\
It's important to secure both the **access** (**whitelist** origins to access the API Server and deny any other connection) and the [**authentication**](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet-authentication-authorization/) (following the principle of **least** **privilege**). And definitely **never** **allow** **anonymous** **requests**.

**Common Request process:**\
User or K8s ServiceAccount –> Authentication –> Authorization –> Admission Control.

**Tips**:

- Close ports.
- Avoid Anonymous access.
- NodeRestriction; No access from specific nodes to the API.
  - [https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#noderestriction](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#noderestriction)
  - Basically prevents kubelets from adding/removing/updating labels with a node-restriction.kubernetes.io/ prefix. This label prefix is reserved for administrators to label their Node objects for workload isolation purposes, and kubelets will not be allowed to modify labels with that prefix.
  - And also, allows kubelets to add/remove/update these labels and label prefixes.
- Ensure with labels the secure workload isolation.
- Avoid specific pods from API access.
- Avoid ApiServer exposure to the internet.
- Avoid unauthorized access RBAC.
- ApiServer port with firewall and IP whitelisting.

### SecurityContext Hardening

By default root user will be used when a Pod is started if no other user is specified. You can run your application inside a more secure context using a template similar to the following one:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: security-context-demo
spec:
  securityContext:
    runAsUser: 1000
    runAsGroup: 3000
    fsGroup: 2000
  volumes:
  - name: sec-ctx-vol
    emptyDir: {}
  containers:
  - name: sec-ctx-demo
    image: busybox
    command: [ "sh", "-c", "sleep 1h" ]
   securityContext:
    runAsNonRoot: true
    volumeMounts:
    - name: sec-ctx-vol
      mountPath: /data/demo
    securityContext:
      allowPrivilegeEscalation: true
```

- [https://kubernetes.io/docs/tasks/configure-pod-container/security-context/](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/)
- [https://kubernetes.io/docs/concepts/policy/pod-security-policy/](https://kubernetes.io/docs/concepts/policy/pod-security-policy/)

### General Hardening

You should update your Kubernetes environment as frequently as necessary to have:

- Dependencies up to date.
- Bug and security patches.

[**Release cycles**](https://kubernetes.io/docs/setup/release/version-skew-policy/): Each 3 months there is a new minor release -- 1.20.3 = 1(Major).20(Minor).3(patch)

**The best way to update a Kubernetes Cluster is (from** [**here**](https://kubernetes.io/docs/tasks/administer-cluster/cluster-upgrade/)**):**

- Upgrade the Master Node components following this sequence:
  - etcd (all instances).
  - kube-apiserver (all control plane hosts).
  - kube-controller-manager.
  - kube-scheduler.
  - cloud controller manager, if you use one.
- Upgrade the Worker Node components such as kube-proxy, kubelet.

## Kubernetes monitoring & security:

- Kyverno Policy Engine
- Cilium Tetragon - eBPF-based Security Observability and Runtime Enforcement
- Network Security Policies
- Falco - Runtime security monitoring & detection

{{#include ../../../banners/hacktricks-training.md}}



