Probing the Docker Remote API Server

The sequence of events in this planned attack campaign begins with a ping to the Docker Remote API server, which is the pivotal starting point for the subsequent chain of actions.

Container Creation with an ubuntu:mantic-20240405 Image

A container named “kube-edagent” was created from the ubuntu:mantic-20240405 base image, with “sleep 9955″ specified as the command to be executed once the container is up and running. The “sleep” command is used to identify the container process when running the “ps” command.

In the request mentioned above, the attackers create a container named “kube-edagent,” deliberately giving it a name similar to a legitimate container. They configure the container to operate in privileged mode by setting “Privileged” to “true.” They also utilize “pid mode: host,” allowing the container to share the Process ID (PID) namespace of the host system. This means the processes running inside the container will share the same PID namespace as the processes on the host. As a result, the container’s processes will be able to see and interact with all the processes running on the host system in the same way as all running processes, as if they were running directly on the host.

The image below is pulled from Docker Hub, and another attempt is made to create the container if the ubuntu image is not present in the victim machine.

Payload Execution

Payloads are executed using a Docker Exec API. The payload consists of two parts. The first part tries to escape the container by using the “nsenter” command to enter the specific namespaces of the target process with PID 1 (“target 1”), which is the process ID of “init.” This command runs as root and includes flags such as “–mount, –uts, –ipc, –net, –pid,” indicating that it should enter the target’s mount, UTS, IPC, network, and PID namespaces, effectively granting it similar capabilities as if it were running in the host system. The second part contains a Base64 encoded shell script.

The decoded Base64 payload executes the following commands:

Step 1: Check and Prevent Duplicate Processes

It checks for multiple running processes matching the pattern “nsenter.*bash.*base64” to avoid running similar processes simultaneously.

Step 2: Create a Bash Script

It creates a bash script called “kubeupd” in the “/tmp” directory. This script sets the environment variable “VEI” to “dck_<public IP of docker API>.” It is assumed that a customized payload has been generated based on the IP address identified in earlier stages. The script also includes another Base64 encoded bash payload.

Step 3: Unpack the Base64 Payload

The Base64 payload located in the “/tmp/kubeupd” file is designed to perform the following actions:

1. It contains a custom “__curl” function that can be used when curl or wget is not present in the system.
2. The script will terminate if the architecture is not “x86_64.” It initially checks for the presence of the “/tmp” directory and creates it if it does not exist. Then, it sets the “/tmp” mount as executable using the “exec” option. Subsequently, it creates “.perfc” and “xdiag” directories within the “/tmp” directory.

3. It sets the AAZHDE environment variable to localhost if it’s not already set.

4. The environment variable $VEI, which contains “dck_<public_ip>,” is included in “/tmp/.xdiag/vei.”

5. It confirms the presence of a malicious process by checking “/tmp/.xdiag/p,” which should contain the PID of the malicious process. It verifies the existence of the same process by checking the “/proc/” directory. The secondary check looks for active TCP connections using ports 44870 or 63582.

6. If it confirms that the process is not running, then it will download the malicious binary, which is disguised as a PHP extension, to avoid file extension-based detection. Downloading the binary file uses the custom __curl function if curl and wget are not in the system. It downloads the file in the “/tmp” directory with the “httpd” name to look like a legitimate name. We were unable to determine the specific payload downloaded in this incident.

7. If the downloaded file exists and its size matches a specific value (e.g., 9301499), it triggers more actions. These actions include killing processes (perfctl), setting permissions, updating the PATH environment variable, and executing a command (KRI=kr httpd) in the background.

8. It then deletes the “/tmp/.install.pid33” marker file.

Step 4: Engage a Persistence Strategy

To remain active, the malware uses a persistence strategy. If systemd runs as non-offline, it creates a systemd service using “multi-user.target.” Otherwise, it resorts to creating a cron job, which is challenging to eradicate.

Base64 Decoded Functions

In addition to the fallback mechanism, the Base64 payload, once decoded, contains four distinct functions:

• Kill_container: This command kills a process with the “sleep 9955” argument.

• Wait_run: It waits for the “/tmp/k8s.run42” file to exist or until the maximum wait time is reached.

• Restore_sh: It replaces the “/bin/sh” file with “/bin/kkbush” to bypass the detection (T1036.005).

• Fallback: It sets up a persistent backdoor by replacing the original “/bin/sh” shell with a modified version that enables privilege escalation and execution of any command. It then duplicates the original “sh” file as the “kkbush” and “kbush” binaries and later restores it during the cleanup process. In addition, it sets up a background process (kubeupd) for further interaction and maintains the backdoor using the wait_run function.

It uses the Tor network to reroute traffic, as confirmed by a Tor relay node (192.121.108.237) in the network packet capture.

Recommendations

• We recommend the following steps to enhance the security of Docker Remote API servers and mitigate the risks associated with potential exploitation for malicious activities:
• Secure Docker Remote API servers by implementing strong access controls and authentication mechanisms to prevent unauthorized access.
• Regularly monitor Docker Remote API servers for any unusual or unauthorized activities, and promptly investigate and address any suspicious behavior.
• Implement container security best practices, such as avoiding the use of “Privileged” mode and carefully reviewing container images and configurations before deployment.
• Educate and train personnel responsible for managing Docker Remote API servers about security best practices and potential attack vectors.
• Stay informed about security updates and patches for Docker and related software to address any known vulnerabilities that could be exploited by threat actors.
• Regularly review and update security policies and procedures related to Docker Remote API server management to align with the latest security best practices and recommendations.

Conclusion

Exploiting exposed Docker remote API servers has now reached a critical level where the attention of an organization and its security professionals is seriously required. The first step to avoiding such incidents is understanding the attack sequence a threat actor can use. It is essential that every organization’s Docker Remote API server is secured, monitored regularly for unauthorized access and suspicious activities to reduce the risk of attacks, and has security patches up to date. Carrying out regular security audits will significantly improve the general security posture against such types of exploits.