Links

Getting Started: Stub and Harden a Container Image with Docker and Kubernetes

Use the RapidFort Command Line Interface (CLI) tools to stub and harden a container image with Docker and Kubernetes

Prerequisites

You will need the following:
  • RapidFort server (SaaS or On-Premises) and account
  • Client System with the RapidFort Command Line Interface Tools installed
  • Kubernetes
    • Make sure that your Kubernetes environment has access to the RapidFort server
  • Container Registry (for example, Amazon Elastic Container Registry, Docker Hub, Microsoft Azure Container Registry, and so forth)

Stub, Test, and Harden the MySQL Docker Image

In this tutorial, we will stub, test, and harden the MySQL Docker image.
Original Image: The MySQL Docker image available directly from Docker Hub. The original image has not yet been optimized and secured by RapidFort and may have known vulnerabilities.
Stub Image: The original image with additional dependencies necessary for RapidFort to trace the runtime behavior.
Hardened Image: The optimized and secured image generated by RapidFort.

Part 1: Get and Deploy the Original Image

Step 1.1: Pull the Original Image

First, pull the latest MySQL Docker image from Docker Hub:
docker pull mysql:latest

Step 1.2: Push the Original Image to Your Container Registry

Tag the MySQL Docker image and push the image to your container registry so that we can deploy this with Kubernetes.
docker tag mysql:latest registry.example.com/mysql:latest
docker push registry.example.com/mysql:latest

Step 1.3: Deploy the Original Image

Download the sample Kubernetes manifest file for deploying the original image to your client system.
Update the image: field to point to your container registry.
mysql.yaml
apiVersion: v1
kind: Pod
metadata:
name: mysql
labels:
name: mysql
spec:
containers:
- name: mysql
image: registry.example.com/mysql:latest
env:
- name: "MYSQL_USER"
value: "mysql"
- name: "MYSQL_PASSWORD"
value: "mysql"
- name: "MYSQL_DATABASE"
value: "example"
- name: "MYSQL_ROOT_PASSWORD"
value: "secret"
ports:
- containerPort: 3306
Make a note of the MySQL password (secret).
To deploy the original MySQL container, run the following command:
kubectl apply -f mysql.yaml

Step 1.4: Test the Original MySQL Container

Test the original MySQL container.
kubectl exec -it mysql -- bash
Run the following command. You will be prompted for the MySQL password (secret).
kubectl exec -it mysql -- mysql -u root -p -e 'show databases'

Step 1.5: Remove the Original MySQL Pod

After testing the original MySQL container, run the following command to remove the pod:
kubectl delete -f mysql.yaml

Part 2: Generate and Deploy the Stub Image

If you have not already done so, please install the RapidFort CLI tools on your client system.

Step 2.1: Log into RapidFort

Log into RapidFort. Enter your password if prompted.
rflogin <email_address>

Step 2.2: Generate a Stub Image

Run rfstub to generate a stub image.
rfstub registry.example.com/mysql:latest
By default, this will generate a stub image called registry.example.com/mysql:latest-rfstub.

Step 2.3: Push the Stub Image to the Container Registry

Push the stub image to the container registry so that we can deploy this with Kubernetes.
docker push registry.example.com/mysql:latest-rfstub

Step 2.4: Deploy the Stub Image

Download the sample Kubernetes manifest file for deploying the stub image to your client system.
Update the image: field to point to your container registry.
mysql-stub.yaml
apiVersion: v1
kind: Pod
metadata:
name: mysql
labels:
name: mysql
spec:
containers:
- name: mysql
image: registry.example.com/mysql:latest-rfstub
securityContext:
capabilities:
add: ["SYS_PTRACE"]
allowPrivilegeEscalation: true
readOnlyRootFilesystem: false
env:
- name: "MYSQL_USER"
value: "mysql"
- name: "MYSQL_PASSWORD"
value: "mysql"
- name: "MYSQL_DATABASE"
value: "example"
- name: "MYSQL_ROOT_PASSWORD"
value: "secret"
ports:
- containerPort: 3306
Please note that the Kubernetes manifest file for running the stub image has the following differences from the manifest file for running the original image:
  • The stub manifest file image: field points to the stub image.
  • The stub manifest file contains an additional securityContext section with updates necessary for RapidFort to trace the runtime behavior.
    • The SYS_PTRACE capability must be added
    • Privilege escalation must be allowed (allowPrivilegeEscalation: true)
    • The root filesystem must allow read/write access (readOnlyRootFilesystem: false)
securityContext:
capabilities:
add: ["SYS_PTRACE"]
allowPrivilegeEscalation: true
readOnlyRootFilesystem: false
Please note that you may run the stub image as a non-root user provided that the necessary updates are made to the security context.
Stub images have an additional runtime overhead of approximately 20%. When deploying stub images, please plan to increase CPU and memory resources by 20%.
You may also need to update the livenessProbe and readinessProbe settings.
To deploy the stub MySQL container, run the following command:
kubectl apply -f mysql-stub.yaml

Step 2.5: Test the Stub MySQL Container

Test the stub MySQL container.
kubectl exec -it mysql -- bash
Run the following command. You will be prompted for the MySQL password (secret).
kubectl exec -it mysql -- mysql -u root -p -e 'show databases'

Step 2.7: Verify that Tracing Information is Propagated to the RapidFort Server

Log into the RapidFort dashboard. You should see an image list.
Select the mysql:latest image.
In the left column, select Logs. Verify that the Logs table contains tracing information. You should see file accesses, system calls, and so forth. The Logs table contents are updated in real time.
If the Logs table is empty, please verify the following:
  • The stub image (mysql:latest-rfstub) is currently deployed and running
  • The Kubernetes manifest file for the stub image contains a securityContext section that adds the SYS_PTRACE capability, allows privilege escalation, and allows read/write access to the root filesystem
  • The Kubernetes environment has access to the RapidFort server
When you generate a stub image, RapidFort also scans the original image for packages and known vulnerabilities. You may optionally view the vulnerabilities and packages that were found in the original image from the RapidFort dashboard.

Step 2.8: Remove the Stub MySQL Pod

After testing the stub MySQL container and verifying that profiling information is being propagated to the RapidFort server, run the following command to remove the pod:
kubectl delete -f mysql-stub.yaml

Part 3: Generate and Deploy the Hardened Image

Step 3.1: Harden the Stub Image

Run rfharden to generate a hardened image.
rfharden registry.example.com/mysql:test-rfstub
By default, this will generate a hardened image called registry.example.com/mysql:latest-rfhardened.

Step 3.2: Push the Hardened Image to the Container Registry

Push the hardened image to the container registry so that we can deploy this with Kubernetes.
docker push registry.example.com/mysql:latest-rfhardened

Step 3.3: Deploy the Hardened Image

Download the sample Kubernetes manifest file for deploying the hardened image to your client system.
Update the image: field to point to your container registry.
mysql-hardened.yaml
apiVersion: v1
kind: Pod
metadata:
name: mysql
labels:
name: mysql
spec:
containers:
- name: mysql
image: registry.example.com/mysql:latest-rfhardened
env:
- name: "MYSQL_USER"
value: "mysql"
- name: "MYSQL_PASSWORD"
value: "mysql"
- name: "MYSQL_DATABASE"
value: "example"
- name: "MYSQL_ROOT_PASSWORD"
value: "secret"
ports:
- containerPort: 3306
Please note that the Kubernetes manifest file for running the hardened image has the following differences from the manifest file for running the original image:
The hardened manifest file image: field points to the hardened image.
To deploy the hardened MySQL container, run the following command:
kubectl apply -f mysql-hardened.yaml

Step 3.4: Test the Hardened MySQL Container

Test the hardened MySQL container.
kubectl exec -it mysql -- bash
Run the following command. You will be prompted for the MySQL password (secret).
kubectl exec -it mysql -- mysql -u root -p -e 'show databases'

Step 3.5: Remove the Hardened MySQL Pod

After testing the hardened MySQL container, run the following command to remove the pod:
kubectl delete -f mysql-hardened.yaml

Step 3.6: Visit the RapidFort Dashboard (Optional)

When you generate a hardened image, RapidFort scans the hardened image for packages and known vulnerabilities. Visit the RapidFort dashboard to view known vulnerabilities that were detected in the hardened image and compare the results with the original image.

Review

  • In Part 1, we deployed and tested the original MySQL image.
  • In Part 2, we generated a stub image and deployed and tested the stub MySQL image.
    • When we deployed the stub image, we updated the security context to enable RapidFort to trace the runtime behavior:
      • We added the SYS_PTRACE Linux kernel capability
      • We allowed privilege escalation (allowPrivilegeEscalation: true)
      • We allowed read/write access for the root filesystem (readOnlyRootFilesystem: false)
    • We verified that the tracing information for the stub image was propagated to the RapidFort server.
  • In Part 3, we generated a hardened image and deployed and tested the hardened MySQL image.