Aqua Nautilus has unveiled a new campaign targeting Apache big-data stack, specifically Apache Hadoop and Apache Druid. Upon investigation, it was discovered that the attacker exploits existing misconfigurations and vulnerabilities within our Apache cloud honeypots to execute the attacks.
The campaign employs a new variant of a well-known DDoS botnet that focuses on vulnerable Linux systems, transforming them into Monero cryptomining bots known as Lucifer malware. In this blog, we will delve into how the attacker exploits the vulnerability and misconfiguration to launch the campaign, illustrate the various phases of the campaign, and highlight the differences observed throughout its operation.
Apache big-data stack
Apache is recognized as an open-source software foundation supporting numerous open-source software projects. The official Apache website states that it hosts more than 320 active projects with over 8,000 contributors who contribute to various endeavors. The expansive range of activities provides opportunities for attackers to initiate attacks against various Apache tools. Our team actively monitors this trend by utilizing our cloud honeypots to track new patterns and behaviors. Over the past year, we observed a consistent rise in attacks on Apache big-data stack. In the last month alone, our monitoring detected over 3,000 distinct attacks targeting these solutions (Apache Hadoop, Apache Druid and Apache Flink).
The ongoing campaign discussed in this blog centers around two significant Apache big-data solutions —Hadoop and Druid:
- Apache Hadoop is an open-source framework designed for batch processing of large-scale data, allowing the storage and retrieval of massive amounts of data in a reliable and distributed way across clusters of computers using simple programming models. One of its modules is Apache YARN (Yet Another Resource Negotiator), a framework for job scheduling and cluster resource management. It allows for more flexible and dynamic resource management in Hadoop clusters, enabling a broader range of applications and workloads to run on the Hadoop infrastructure.
- Apache Druid is an open-source, high-performance database designed for real-time analytics on large volumes of data. It provides quick and interactive analysis of large datasets which is essential for gaining insights and making data-driven decisions.
These projects, highly contributed by the community and marked by approximately 13,000 users as favorite projects on GitHub.
Attackers exploits misconfiguration and vulnerabilities
In this campaign the attacker deploys Lucifer malware by exploiting the following vulnerabilities and misconfigurations:
Apache Druid
In 2021, it was verified that Apache Druid is vulnerable to remote code execution, and it was assigned CVE-2021-25646. According to the details, this vulnerability allows a remote unauthenticated user to execute arbitrary JavaScript code with the privileges of Druid server processes. It was detected that in some versions of Druid (0.20.0 and earlier), execution of JavaScript code is not disabled by default, making it possible to send a specially crafted request that forces Druid to run user-provided JavaScript code for that request (A detailed explanation can be found in ‘Lucifer Third Evolution’).
Apache Hadoop YARN
YARN is a unified resource management platform in Hadoop systems. Its primary goal is to achieve unified management and scheduling of cluster resources. A user is allowed to directly perform related application creation, task submission, and other operations through the APIs provided by YARN. In the case that these APIs are not configured properly, they may be exposed on the public network, leading to unauthorized access. This vulnerability can allow an attacker to exploit it for remote code execution (RCE).
Another example of this misconfiguration and how attackers exploited it can be found in one of our recent blogs.
The Lucifer campaign
We recorded the Lucifer malware campaign over a period of 6 months, and our analysis revealed a slight evolution during this time. In general, the campaign consists of three distinct steps:
- Exploitation of a vulnerability or misconfiguration.
- Downloading and executing the Lucifer malware (or two). Specifically, we observe that the malware possesses dropper capabilities.
- Downloading and executing the main payload – an XMRig cryptominer.
In the first stage of evolution, only one ELF binary is downloaded and serves as a dropper.
In the second stage, two ELF binaries are downloaded. One serves as a dropper (100% similar to the first stage), while the connection or purpose of the second binary is unclear.
In the third and final evolution, two droppers are involved. After the first one is dropped and executed, it downloads the main payload (a cryptominer), and then the second ELF file executes the cryptominer.
We speculate that these may be tests conducted by the attacker, possibly aimed at evaluating defense evasion techniques to bypass the detection of dropped and executed files. However, we are not fully certain about the purpose of this behavior.
Further capabilities of the Lucifer malware are discussed in a 2020 blog post by Unit42, “Lucifer is quite powerful in its capabilities. Not only is it capable of dropping XMRig for cryptojacking Monero, it’s also capable of command and control (C2) operation and self-propagation through the exploitation of multiple vulnerabilities and credential brute-forcing. Additionally, it drops and runs EternalBlue, EternalRomance, and DoublePulsar backdoor against vulnerable targets for intranet infections.”
In the analysis below we discuss each phase and illuminate the similarities and differences between them.
Lucifer initial evolution stage
In this section, we thoroughly discuss the Lucifer campaign. In the following sections, we will highlight the similarities and differences.
Attack Flow
In this stage the service targeted by the attacker is Apache Hadoop YARN. The attacks were performed from servers hosted by Tencent cloud in the CIDR 81.68.0.0/14. Then the Lucifer malware was downloaded and executed to launch the cryptomining activity.
Initial Access
After scanning the internet and searching for misconfigured Apache Hadoop, the attacker exploited an existing misconfiguration in our Apache Hadoop YARN, which allows an attacker to execute remote code execution (RCE) on our vulnerable instance.
As illustrated in Figure 3, the attacker exploits the misconfiguration to execute a malicious command, enabling the download of malware onto our compromised machine:
- Downloading the malicious binary
llli
from the IP address 103.255.177.55. - Granting the binary read, write, and execute permissions for all users.
- Executing the binary.
- Deleting the binary.
Execution
The attacker downloads and executes the malicious binary llli
. The binary is saved to the local directory structure used by Apache Hadoop YARN on a NodeManager (a crucial component of Hadoop YARN) and executed from there.
As can be seen in Figure 4, the binary is identified as malicious by 40 vendors, according to VirusTotal, and classified as Lucifer DDoS botnet malware.
The execution of the binary llli
triggers the following steps:
- Performs a DNS request (domain nishabii[.]xyz) and connects to the resolved IP address (110.42.1.53) using port 7895. The IP address and domain were identified as malicious, and according to the pcap file, it was found related to the pool and mining activity.
- Downloads the file
gcshfk
from the IP address 103.255.177.55 using port 6895 and saves it under /tmp with the nameQdefgh
. - Provide the binary
Qdefgh
with read, write and execute permissions for all users.
The Qdefgh
binary file was detected by 33 vendors, according to VirusTotal, and classified as Coinminer.
Persistence
As illustrated in Figure 6 below, the attacker writes a command to crontab, which will execute every 1 minute the binary llli
(detected in VirusTotal as Lucifer malware). This allows the attacker to gain persistence on our machine and ensures the malicious activity will be executed on a scheduled basis.
Defense Evasion
As seen in the attack flow, the attacker downloads the Lucifer malware llli
binary file and then deletes it from the path where it was initially saved. However, the binary still exists on the machine and is executed on a scheduled basis. The deletion helps the attacker minimize their footprint during the intrusion.
In addition, the miner is downloaded as a packed binary to evade detection by some signature-based security solutions.
Furthermore, the attacker also truncates log files to a specified length. This is a clever move, effectively reducing the evidence of intrusion without triggering alarms for log file deletion.
Discovery
To support the mining operation, the attacker executed a command that gathers information about the CPU clock speed from the /proc/cpuinfo file, providing numeric values representing the clock speed in megahertz..
Impact
Our honeypots are limited, thus we’ve only seen resource hijacking indications. However, the Lucifer malware is also capable of initiating DDoS attack.
Lucifer Second Evolution Phase
The attacker continues to target Hadoop servers via misconfiguration by utilizing servers on the Tencent cloud in the CIDR 82.156.0.0/15 and infects the misconfigured server with the Lucifer malware. This time, upon deploying the malware, two files are dropped (instead of one), but only one is executed.
In the initial stage, the Lucifer malware file was named ‘llli’, and it dropped and executed the cryptominer (Qdefgh)
. In this phase, as illustrated in Figure 7, we observe two dropped malware files (skadaf
and skadaf8
), but only the skadaf
binary file was executed. The purpose of the second file was initially unclear, but this was soon discovered in the third phase.
Figure 8 below highlights the command executed while exploiting the existing misconfiguration, showing the downloading of two binary files – skadaf
and skadaf8
.
As illustrated in Figures 9 and 10, the files that were downloaded were both detected by VirusTotal as malicious and classified by vendors as related to the Lucifer DDoS botnet malware.
As mentioned earlier, the skadaf
Lucifer malware initiates the download of the cryptominer. The malware performs a DNS request to the domain hfs.t1linux[.]com and connects to the resolved IP address 45.141.68.25 using port 7845. This connection triggers the download and execution of the cryptominer named sprxyabc
(This is the same miner that was downloaded in the initial phase).
The techniques detected in the initial phase – Persistence, Defense evasion, Discovery, and Impact –remain the same in this phase.
Lucifer Third Evolution Stage
In this stage, the attacker is looking for an Apache Druid vulnerability. Once a vulnerable host is detected, the exploit is utilized to gain initial access and download the two malicious binary files, as depicted in Figure 11 below.
It is important to note that similar attacks were observed on Apache Hadoop YARN after the exploitation of the existing misconfiguration.
Initial access
Our version of the Apache Druid service is vulnerable to CVE-2021-25646. After the attacker scanned the network for open services like ours, it became possible to exploit the vulnerability. As illustrated in Figure 12, Apache Druid provides an HTTP API that allows users to access the Apache Druid console and interact with it. It also enables authenticated users to execute user-defined JavaScript code for specific filtering and transform operations. However, the attacker exploits our vulnerable version for command injection, creating a crafted request with embedded JavaScript code that is executed, even if this feature is disabled.
In Figure 13, we can see the highlighted JavaScript code executed by the attacker., It is similar to what we observed in the second phase: the attacker downloads two binaries, in this phase are named shx
and shx8
, providing them with read, write and execute permissions for all users, executing them and then deleting them.
Execution
As observed in the second phase, the binaries downloaded were both classified as Lucifer DDoS botnet malware according to VirusTotal, as depicted in Figures 14 and 15.
Similar to the second phase, the binary shx
initiated the download of the cryptominer 1HSDXklmno
, which is save to /tmp directory, while, in this phase, the binary shx8
is responsible to execute the cryptominer.
Similar to what we observed in the previous phases, a DNS request was made towards the domain nishabii[.]xyz. According to the pcap file, we were able to connect to it, revealing it as the pool related to the mining activity, as illustrated in Figure 16 below.
Persistence
Similar to the initial phase, only that in this phase the attacker scheduled the execution of the two binary files – shx
and shx8
as illustrated in figure 17.
Defense Evasion
In this phase, the attacker introduced another defense evasion technique, enabling to bypass detection of dropped and executed files by distributing the downloading and execution of the cryptominer between two Lucifer malware binary files.
The techniques detected in the initial phase – Persistence, Defense evasion, Discovery, and Impact – remain the same in this phase.
Summary of Lucifer Phases
The table below provides a summary of the similarities and differences between the stages of evolution.
|
First Stage | Second Stage | Third Stage |
Campaign Date | July-August, 2023 | October, 2023 | Ongoing campaign, first observed in January 2024 |
Apache big-data stack | Apache Hadoop | Apache Hadoop | Apache Hadoop and Apache Druid |
Initial access | Exploiting misconfiguration | Exploiting misconfiguration | Exploiting misconfiguration/ vulnerability. |
Execution | 1 dropper
1 cryptominer |
1 dropper (1 dropper not running)
1 cryptominer |
2 droppers
1 cryptominer |
Persistence | Schedule job to execute the attack | Schedule job to execute the attack | Schedule job to execute the attack |
Defense Evasion | Binary deletion, Packed binary, Truncating logs | Binary deletion, Packed binary, Truncating logs | Binary deletion, Packed binary, Truncating logs |
Discovery | System information discovery | System information discovery | System information discovery |
Impact | Resource hijacking | Resource hijacking | Resource hijacking |
In the strings of one of the Lucifer malware samples we made an interesting discovery. A text appealing to our feeling to spare the attacker and donate some money. The wallets are different, and we found it peculiar that a Chinese speaking threat actor is targeting our APAC deployed honeypots is choosing to communicate his message in English rather than in Chinese.
MITRE Framework
Dynamic Detection of Compromised Workloads
Aqua’s Cloud Native Application Platform (CNAPP) offers comprehensive protection for your cloud-based projects. Aqua Trivy, diligently scans your code, container images, and cloud workloads to detect various threats, including known vulnerabilities like Apache Druid CVE-2021-25646, concealed malware, hidden secrets, configuration errors, and open-source license issues. With our premium version, you get access to advanced features for enhanced security. Our CI/CD pipeline integration allows for seamless configuration of acceptance gates, minimizing the risk of attacks by permitting only authorized images to proceed to deployment. Once in production, our intelligence-driven runtime controls ensure continuous protection against exploit attempts, such as container escape, safeguarding your cloud workloads in real-time.
Below are screenshots illustrating how Aqua Trivy’s premium version effectively identifies the Apache Druid CVE-2021-25646 vulnerability in an EC2 instance.
The details of the exploit to the vulnerability on Github. (and in figure 20 below).
In addition, you can see below our runtime protection capabilities that create defense in depth capabilities to detect such attacks, in case of a zero-day, misconfiguration, human error or any other reason.
Guidelines for Protecting Your Environments
In this blog, we reviewed the Lucifer DDoS botnet malware and demonstrated how it targets Apache big-data solutions – Apache Hadoop and Apache Druid. The attacker implements the attack by exploiting existing misconfigurations and vulnerabilities in those services. After gaining a foothold on the service, the attacker was able to execute malicious activities and impact our resources, as we observed, using our resources for cryptomining activity.
As we’ve seen, Apache open-source solutions are widely used by many users and contributors. Attackers may view this extensive use as an opportunity to have inexhaustible resources for implementing their attacks on them.
How can we use different services and still protect our environment?
First, attackers need to find a breach to gain a foothold in your environment. It is important to prevent those attempts and secure your environment. Updates are crucial and recommended, as they contain fixes for security issues handled by distributors. Moreover, it is important to configure your environment properly and follow guidelines to avoid misconfigurations.
Second, this campaign provides an example of how attackers improve their methods and techniques, creating sophisticated attacks that are harder to detect. Unknown threats can be identified by scanning your environments with runtime detection and response solutions, which can detect exceptional behavior and alert about it.
Third, it is important to be cautious and aware of existing threats while using open-source libraries. Every library and code should be downloaded from a verified distributor. Educating employees and securing your supply chain can reduce possible threats.
wallet
Indications of Compromise (IOCs)
Type | Value | Comment |
---|---|---|
IP Addresses | ||
IP Address | 81.68.214.122 | Attacker IP |
IP Address | 81.68.197.3 | Attacker IP |
IP Address | 82.156.146.62 | Attacker IP |
IP Address | 106.52.127.12 | Attacker IP |
IP Address | 47.88.49.239 | Attacker IP |
IP Address | 20.205.116.139 | Attacker IP |
IP Address | 111.92.242.47 | C2 server |
IP Address | 103.255.177.55 | C2 server |
IP Address | 45.141.68.25 | C2 server |
IP Address | 110.45.1.53 | Mining pool |
Domain | ||
Domain | nishabii.xyz | Mining activity |
Domain | hfs.t1linux.com | Mining activity |
Domain | auto.c3pool.org | Mining pool |
Files | ||
Binary file | Sha256: 808f0f85aee6be3d3f3dd4bb827f556401c4d69a642ba4b1cb3645368954622e | Lucifer malware |
Binary file | Sha256: b1c19e717494b33fa269b5adaf7e591d46f1ab4c1f571f22df254055349ec22c | Lucifer malware |
Binary file | Sha256: 4e4b120f5ae23a2d3a32e9dc09cc1b9ead3e8cd947555f83f84a369a4feff081 | Lucifer malware |
Binary file | Sha256: 08caa2415f19565aa1fac40ea4a9e3e2eb9c6e382507e3e93677c506e4b42f9c | Lucifer malware |
Binary file | Sha256: b87346b930120e2be9394177c530843187f7d5393738b7a935583f19cc6937b5 | Lucifer malware |
Binary file | Sha256: 42590da283f271cb55efcea7c89866d6dc3358933996166302237f040141fa12 | Lucifer malware |
Binary file | Sha256: b4135ca942f8ab7e98259dd4666d9e84ba0c6a4a7326bab4b4abab5b009551be | Lucifer malware |
Binary file | Sha256: 7fb22f3b6632ab0df493b2e80a66b6a08a3173ccf5f8cdf2fc4956afd63bff23 | Lucifer malware |
Binary file | Sha256: 6a43077384c2f348908dee0b5a5bbe119f82092cef87586af0b3eefe6d9d05c8 | Lucifer malware |
Binary file | Sha256: 4839d344d0251f9122c11222021511702ae8dfd3185c2e300cd21c0c7a574d5f | Lucifer malware |
Binary file | Sha256: 7c6f0bae1e588821bd5d66cd98f52b7005e054279748c2c851647097fa2ae2df | Miner |
Wallet | ||
Wallet | 42CJPfp1jJ6PXv4cbjXbBRMhp9YUZsXH6V5kEvp7XzNGKLnuTNZQVU9bhxsqBEMstvDwymNSysietQ5VubezYfoq4fT4Ptc |
Source: https://www.aquasec.com/blog/lucifer-ddos-botnet-malware-is-targeting-apache-big-data-stack/