gitea-mirror/hacktricks-cloud
1
0
Fork 0
mirror of https://github.com/HackTricks-wiki/hacktricks-cloud.git synced 2025-12-29 22:20:33 -08:00
Code Issues Packages Projects Releases Wiki Activity

Migrate to using mdbook

Browse Source
This commit is contained in:
Congon4tor
2024-12-31 17:04:35 +01:00
parent b9a9fed802
commit cd27cf5a2e
1373 changed files with 26143 additions and 34152 deletions
Download Patch File Download Diff File Expand all files Collapse all files

214
src/pentesting-cloud/kubernetes-security/pentesting-kubernetes-services/README.md Normal file
View File

@@ -0,0 +1,214 @@
# Pentesting Kubernetes Services
{{#include ../../../banners/hacktricks-training.md}}
Kubernetes uses several **specific network services** that you might find **exposed to the Internet** or in an **internal network once you have compromised one pod**.
## Finding exposed pods with OSINT
One way could be searching for `Identity LIKE "k8s.%.com"` in [crt.sh](https://crt.sh) to find subdomains related to kubernetes. Another way might be to search `"k8s.%.com"` in github and search for **YAML files** containing the string.
## How Kubernetes Exposes Services
It might be useful for you to understand how Kubernetes can **expose services publicly** in order to find them:
{{#ref}}
../exposing-services-in-kubernetes.md
{{#endref}}
## Finding Exposed pods via port scanning
The following ports might be open in a Kubernetes cluster:
| Port | Process | Description |
| --------------- | -------------- | ---------------------------------------------------------------------- |
| 443/TCP | kube-apiserver | Kubernetes API port |
| 2379/TCP | etcd | |
| 6666/TCP | etcd | etcd |
| 4194/TCP | cAdvisor | Container metrics |
| 6443/TCP | kube-apiserver | Kubernetes API port |
| 8443/TCP | kube-apiserver | Minikube API port |
| 8080/TCP | kube-apiserver | Insecure API port |
| 10250/TCP | kubelet | HTTPS API which allows full mode access |
| 10255/TCP | kubelet | Unauthenticated read-only HTTP port: pods, running pods and node state |
| 10256/TCP | kube-proxy | Kube Proxy health check server |
| 9099/TCP | calico-felix | Health check server for Calico |
| 6782-4/TCP | weave | Metrics and endpoints |
| 30000-32767/TCP | NodePort | Proxy to the services |
| 44134/TCP | Tiller | Helm service listening |
### Nmap
```bash
nmap -n -T4 -p 443,2379,6666,4194,6443,8443,8080,10250,10255,10256,9099,6782-6784,30000-32767,44134 <pod_ipaddress>/16
```
### Kube-apiserver
This is the **API Kubernetes service** the administrators talks with usually using the tool **`kubectl`**.
**Common ports: 6443 and 443**, but also 8443 in minikube and 8080 as insecure.
```bash
curl -k https://<IP Address>:(8|6)443/swaggerapi
curl -k https://<IP Address>:(8|6)443/healthz
curl -k https://<IP Address>:(8|6)443/api/v1
```
**Check the following page to learn how to obtain sensitive data and perform sensitive actions talking to this service:**
{{#ref}}
../kubernetes-enumeration.md
{{#endref}}
### Kubelet API
This service **run in every node of the cluster**. It's the service that will **control** the pods inside the **node**. It talks with the **kube-apiserver**.
If you find this service exposed you might have found an **unauthenticated RCE**.
#### Kubelet API
```bash
curl -k https://<IP address>:10250/metrics
curl -k https://<IP address>:10250/pods
```
If the response is `Unauthorized` then it requires authentication.
If you can list nodes you can get a list of kubelets endpoints with:
```bash
kubectl get nodes -o custom-columns='IP:.status.addresses[0].address,KUBELET_PORT:.status.daemonEndpoints.kubeletEndpoint.Port' | grep -v KUBELET_PORT | while IFS='' read -r node; do
ip=$(echo $node | awk '{print $1}')
port=$(echo $node | awk '{print $2}')
echo "curl -k --max-time 30 https://$ip:$port/pods"
echo "curl -k --max-time 30 https://$ip:2379/version" #Check also for etcd
done
```
#### kubelet (Read only)
```bash
curl -k https://<IP Address>:10255
http://<external-IP>:10255/pods
```
### etcd API
```bash
curl -k https://<IP address>:2379
curl -k https://<IP address>:2379/version
etcdctl --endpoints=http://<MASTER-IP>:2379 get / --prefix --keys-only
```
### Tiller
```bash
helm --host tiller-deploy.kube-system:44134 version
```
You could abuse this service to escalate privileges inside Kubernetes:
### cAdvisor
Service useful to gather metrics.
```bash
curl -k https://<IP Address>:4194
```
### NodePort
When a port is exposed in all the nodes via a **NodePort**, the same port is opened in all the nodes proxifying the traffic into the declared **Service**. By default this port will be in in the **range 30000-32767**. So new unchecked services might be accessible through those ports.
```bash
sudo nmap -sS -p 30000-32767 <IP>
```
## Vulnerable Misconfigurations
### Kube-apiserver Anonymous Access
Anonymous access to **kube-apiserver API endpoints is not allowed**. But you could check some endpoints:
![](https://www.cyberark.com/wp-content/uploads/2019/09/Kube-Pen-2-fig-5.png)
### **Checking for ETCD Anonymous Access**
The ETCD stores the cluster secrets, configuration files and more **sensitive data**. By **default**, the ETCD **cannot** be accessed **anonymously**, but it always good to check.
If the ETCD can be accessed anonymously, you may need to **use the** [**etcdctl**](https://github.com/etcd-io/etcd/blob/master/etcdctl/READMEv2.md) **tool**. The following command will get all the keys stored:
```bash
etcdctl --endpoints=http://<MASTER-IP>:2379 get / --prefix --keys-only
```
### **Kubelet RCE**
The [**Kubelet documentation**](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/) explains that by **default anonymous acce**ss to the service is **allowed:**
> Enables anonymous requests to the Kubelet server. Requests that are not rejected by another authentication method are treated as anonymous requests. Anonymous requests have a username of `system:anonymous`, and a group name of `system:unauthenticated`
To understand better how the **authentication and authorization of the Kubelet API works** check this page:
{{#ref}}
kubelet-authentication-and-authorization.md
{{#endref}}
The **Kubelet** service **API is not documented**, but the source code can be found here and finding the exposed endpoints is as easy as **running**:
```bash
curl -s https://raw.githubusercontent.com/kubernetes/kubernetes/master/pkg/kubelet/server/server.go | grep 'Path("/'
Path("/pods").
Path("/run")
Path("/exec")
Path("/attach")
Path("/portForward")
Path("/containerLogs")
Path("/runningpods/").
```
All of them sound interesting.
You can use the [**Kubeletctl**](https://github.com/cyberark/kubeletctl) tool to interact with Kubelets and their endpoints.
#### /pods
This endpoint list pods and their containers:
```bash
kubeletctl pods
```
#### /exec
This endpoint allows to execute code inside any container very easily:
```bash
kubeletctl exec [command]
```
> [!NOTE]
> To avoid this attack the _**kubelet**_ service should be run with `--anonymous-auth false` and the service should be segregated at the network level.
### **Checking Kubelet (Read Only Port) Information Exposure**
When a **kubelet read-only port** is exposed, it becomes possible for information to be retrieved from the API by unauthorized parties. The exposure of this port may lead to the disclosure of various **cluster configuration elements**. Although the information, including **pod names, locations of internal files, and other configurations**, may not be critical, its exposure still poses a security risk and should be avoided.
An example of how this vulnerability can be exploited involves a remote attacker accessing a specific URL. By navigating to `http://<external-IP>:10255/pods`, the attacker can potentially retrieve sensitive information from the kubelet:
![https://www.cyberark.com/wp-content/uploads/2019/09/KUbe-Pen-2-fig-6.png](https://www.cyberark.com/wp-content/uploads/2019/09/KUbe-Pen-2-fig-6.png)
## References
{{#ref}}
https://www.cyberark.com/resources/threat-research-blog/kubernetes-pentest-methodology-part-2
{{#endref}}
{{#ref}}
https://labs.f-secure.com/blog/attacking-kubernetes-through-kubelet
{{#endref}}
{{#include ../../../banners/hacktricks-training.md}}

109
src/pentesting-cloud/kubernetes-security/pentesting-kubernetes-services/kubelet-authentication-and-authorization.md Normal file
View File

@@ -0,0 +1,109 @@
# Kubelet Authentication & Authorization
{{#include ../../../banners/hacktricks-training.md}}
## Kubelet Authentication <a href="#kubelet-authentication" id="kubelet-authentication"></a>
[**From the docss:**](https://kubernetes.io/docs/reference/access-authn-authz/kubelet-authn-authz/)
By default, requests to the kubelet's HTTPS endpoint that are not rejected by other configured authentication methods are treated as anonymous requests, and given a **username of `system:anonymous`** and a **group of `system:unauthenticated`**.
The **3** authentication **methods** are:
- **Anonymous** (default): Use set setting the param **`--anonymous-auth=true` or the config:**
```json
"authentication": {
"anonymous": {
"enabled": true
},
```
- **Webhook**: This will **enable** the kubectl **API bearer tokens** as authorization (any valid token will be valid). Allow it with:
- ensure the `authentication.k8s.io/v1beta1` API group is enabled in the API server
- start the kubelet with the **`--authentication-token-webhook`** and **`--kubeconfig`** flags or use the following setting:
```json
"authentication": {
"webhook": {
"cacheTTL": "2m0s",
"enabled": true
},
```
> [!NOTE]
> The kubelet calls the **`TokenReview` API** on the configured API server to **determine user information** from bearer tokens
- **X509 client certificates:** Allow to authenticate via X509 client certs
- see the [apiserver authentication documentation](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs) for more details
- start the kubelet with the `--client-ca-file` flag, providing a CA bundle to verify client certificates with. Or with the config:
```json
"authentication": {
"x509": {
"clientCAFile": "/etc/kubernetes/pki/ca.crt"
}
}
```
## Kubelet Authorization <a href="#kubelet-authentication" id="kubelet-authentication"></a>
Any request that is successfully authenticated (including an anonymous request) **is then authorized**. The **default** authorization mode is **`AlwaysAllow`**, which **allows all requests**.
However, the other possible value is **`webhook`** (which is what you will be **mostly finding out there**). This mode will **check the permissions of the authenticated user** to allow or disallow an action.
> [!WARNING]
> Note that even if the **anonymous authentication is enabled** the **anonymous access** might **not have any permissions** to perform any action.
The authorization via webhook can be configured using the **param `--authorization-mode=Webhook`** or via the config file with:
```json
"authorization": {
"mode": "Webhook",
"webhook": {
"cacheAuthorizedTTL": "5m0s",
"cacheUnauthorizedTTL": "30s"
}
},
```
The kubelet calls the **`SubjectAccessReview`** API on the configured API server to **determine** whether each request is **authorized.**
The kubelet authorizes API requests using the same [request attributes](https://kubernetes.io/docs/reference/access-authn-authz/authorization/#review-your-request-attributes) approach as the apiserver:
- **Action**
| HTTP verb | request verb |
| --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| POST | create |
| GET, HEAD | get (for individual resources), list (for collections, including full object content), watch (for watching an individual resource or collection of resources) |
| PUT | update |
| PATCH | patch |
| DELETE | delete (for individual resources), deletecollection (for collections) |
- The **resource** talking to the Kubelet api is **always** **nodes** and **subresource** is **determined** from the incoming request's path:
| Kubelet API | resource | subresource |
| ------------ | -------- | ----------- |
| /stats/\* | nodes | stats |
| /metrics/\* | nodes | metrics |
| /logs/\* | nodes | log |
| /spec/\* | nodes | spec |
| _all others_ | nodes | proxy |
For example, the following request tried to access the pods info of kubelet without permission:
```bash
curl -k --header "Authorization: Bearer ${TOKEN}" 'https://172.31.28.172:10250/pods'
Forbidden (user=system:node:ip-172-31-28-172.ec2.internal, verb=get, resource=nodes, subresource=proxy)
```
- We got a **Forbidden**, so the request **passed the Authentication check**. If not, we would have got just an `Unauthorised` message.
- We can see the **username** (in this case from the token)
- Check how the **resource** was **nodes** and the **subresource** **proxy** (which makes sense with the previous information)
## References
- [https://kubernetes.io/docs/reference/access-authn-authz/kubelet-authn-authz/](https://kubernetes.io/docs/reference/access-authn-authz/kubelet-authn-authz/)
{{#include ../../../banners/hacktricks-training.md}}