THM AOC2024 DAY 22: Orchestrating Security in Kubernetes

Every December, TryHackMe's Advent of Cyber delivers 24 free daily cybersecurity challenges, offering hands-on scenarios that simulate real-world attacks and defenses. Designed for beginners and professionals alike, it's an exciting, gamified way to explore topics like threat hunting, penetration testing, cryptography, and more. This event is perfect for building skills, gaining practical experience, and spreading some cybersecurity cheer during the festive season!

---

Learning Objectives

1. Learn about Kubernetes, its purpose, and why it's used.
2. Understand the basics of DFIR and its challenges in ephemeral environments.
3. Explore how DFIR is conducted in Kubernetes environments through log analysis.

---

Tools Overview

• Kubernetes: A container orchestration platform that automates deploying, scaling, and managing applications in containers. It ensures high availability and scalability for microservices.
• DFIR (Digital Forensics and Incident Response): A two-fold investigative approach that collects digital evidence and implements responsive measures to contain and recover from security incidents.
• Audit Logging: Essential for Kubernetes DFIR, these logs capture actions within the environment, including API requests, providing invaluable insight into activities.
• Log Analysis Tools: Utilities like kubectl and Docker logs enable forensic review of ephemeral data in containerized environments.

---

Task Walkthrough

Overview

In this challenge, you investigate a Kubernetes environment targeted by Mayor Malware. By piecing together logs and using forensic techniques, you uncover the attack chain and pinpoint how the malicious actor gained access.

Steps

Task 1: Inspect Pod Logs

1. Start the Kubernetes cluster with minikube start.
2. Verify pods are running using kubectl get pods -n wareville.
3. Access the target pod (naughty-or-nice) using:bashCopy codekubectl exec -n wareville naughty-or-nice -it -- /bin/bash
4. Analyze the Apache logs (/var/log/apache2/access.log) and note the activity involving the webshell shelly.php.

Task 2: Analyze Backups

1. Review log backups stored at /home/ubuntu/dfir_artefacts/.
2. Identify suspicious activity, such as the db.php file accessed via the webshell.

Task 3: Investigate Docker Registry

1. Use docker ps to list running containers and find the registry container.
2. Examine Docker logs with docker logs CONTAINER_ID to find unusual connections from IP 10.10.130.253.
3. Confirm that a malicious image was pushed to the registry.

Task 4: Trace Attack Path

1. Analyze Kubernetes audit logs for Mayor Malware's actions:
   • Rolebinding and role inspection.
   • Use of pods/exec to shell into a privileged pod (morality-checker).
2. Verify the secrets pulled using the compromised service account (job-runner-sa).

---

Questions and Solutions

1. What is the name of the webshell that was used by Mayor Malware?
   • Answer: shelly.php
2. What file did Mayor Malware read from the pod?
   • Answer: db.php
3. What tool did Mayor Malware search for that could be used to create a remote connection from the pod?
   • Answer: nc
4. What IP connected to the docker registry that was unexpected?
   • Answer: 10.10.130.253
5. At what time is the first connection made from this IP to the docker registry?
   • Answer: 29/Oct/2024:10:06:33 +0000
6. At what time is the updated malicious image pushed to the registry?
   • Answer: 29/Oct/2024:12:34:28 +0000
7. What is the value stored in the "pull-creds" secret?
   • Answer:jsonCopy code{"auths":{"http://docker-registry.nicetown.loc:5000":{"username":"mr.nice","password":"Mr.N4ughty","auth":"bXIubmljZTpNci5ONHVnaHR5"}}}

---

Recap of Learning Objectives

1. Learn about Kubernetes, its purpose, and why it's used.

Kubernetes simplifies managing containerized applications, providing scalability and high availability by orchestrating pods, nodes, and clusters.

2. Understand the basics of DFIR and its challenges in ephemeral environments.

In dynamic systems like Kubernetes, logs and evidence can disappear quickly. Tools like audit logging and container runtime logs are critical for maintaining forensic visibility.

3. Explore how DFIR is conducted in Kubernetes environments through log analysis.

By analyzing logs, audit trails, and Kubernetes configurations, DFIR experts can trace attacker activity, reconstruct attack paths, and identify misconfigurations enabling breaches.