Introduction
Infoblox analyzes over 70 billion DNS records each day, along with millions of domain-related records from other sources, to identify suspicious and malicious domains throughout the internet. Our algorithms work in series, making near-real time decisions on some domains using our Threat Insight infrastructure, while other decisions are made over time, leveraging a longitudinal profile of the domain. This allows us to identify domains at various points in their lifecycle when they may be a threat to our customers. The domains we describe are all related to a single tool kit we call Decoy Dog, which is identified using a DNS fingerprint matching 0.0000027% of domains currently active in the world. In writing this paper, we have found that the mysteries surrounding Decoy Dog and its presence in our networks are complex and unresolved. We expect to release further reporting as we are able to explain the activity.
As threat actors have matured over the last decade, they have learnt how to evade standard industry detection methods and they constantly adapt. It is known that they will often register a domain, but not use it for some time: a technique called strategic aging. They may also dribble DNS queries onto the internet, ensuring that the domain gets observed by security vendors, before leveraging it in malicious activities. This results in domains that have low risk when activated. We often see strategic aging of domains for between 14 and 120 days, sometimes longer. In other cases, but less frequently now, domains will be registered and used immediately. Our algorithms revisit domains multiple times in an effort to isolate those that pose an active threat in our customer networks.
Figure 1. Timeline of suspicious domain reviews by Infoblox algorithms.
The DNS community has established that much of the DNS traffic observed on the internet is a replay of previous traffic.1 Specifically, for reasons that vary and are not entirely known, legitimate DNS queries are captured and then replayed, sometimes for years. Some DNS operators report cases of receiving consistent queries for domains at authoritative name servers that have not existed for years. Because of this phenomenon, the Infoblox Threat Intelligence Group considers global DNS traffic to be useful, but not reliable as a source of truth. In contrast, queries that originate from our own networks, whether in the cloud or on-prem, can be associated with a specific device. The global enterprise and large organizational nature of our customer base provides us with strong confidence in our DNS data that allows us to make better decisions about the queries we observe.
Some of our analytics look over long periods of time and identify anomalous domains that may be involved in some form of data exfiltration, beaconing, or leakage via DNS. We identify several hundred such domains each day, and automatically add these to our suspicious domains blocklists. Most often those domains are associated with questionable browser plugins, such as free Virtual Private Network (VPN) software, or short-lived DNS tunnels. We also employ human-in-the-loop methodologies in specific cases. The discovery of Decoy Dog, and most importantly, the fact that several seemingly unrelated domains were using the same rare toolkit was a result of this combination of automatic and human processes. Because the situation is complex and we have been focused on the DNS aspects of the discovery, we expect more details to come from the industry, in addition to ourselves, in the future.
Discovery
In early April, Infoblox discovered the domains in Table 1 below, acting as command and control (C2) beacons for a then-undetermined malware. During our initial analysis, we found that the name servers for these C2 domains were returning IP addresses across a broad range of ASNs. These include unresolvable IP addresses from the Internet Assigned Numbers Authority’s (IANA) list of reserved IP addresses, for entities such as the U.S. Department of Defense, and IPs used for malicious activities such as phishing. This unusual behavior led us to investigate further and conclude that these C2 domains were related to each other but distinct from other activity on the internet, not simply suspicious DNS activity generated by legitimate appliances or security devices.
| cbox4[.]ignorelist[.]com | claudfront[.]net | hsdps[.]cc |
| ads-tm-glb[.]click | atlas-upd[.]com | allowlisted[.]net |
| Table 1. Domains flagged in early April 2023 for anomalous DNS beaconing activity. Others reported wmssh[.]com but we determined it was not one of the C2 domains. | ||
Thus far, we have found the identical DNS query patterns arising from enterprise networks, which could not be tied to consumer devices. Our asset management information confirmed that the queries originated from network appliances in a very limited number of customer networks. Within our customer set, we observed this activity in under 3% of all networks, approximately the same percentage we observed following the December 2020 disclosure of the SolarWinds supply chain compromise.2 Consulting with other security vendors, they too observed this traffic only in limited enterprise networks originating from network devices.
When we analyzed the queries in external global DNS data, the C2 communication originated almost exclusively from hosts in Russia. Because global DNS traffic is polluted with retransmitted queries from multiple sources, and because at least one of the C2 servers was located in Russia, we cannot assume that this was authentic communication from a compromised host. In other external sources, we observed traffic patterns consistent with our own networks, giving us high confidence that the activity was uncommon and unique.
An OSINT report3 from 9 April 2023 on Telegram regarding some of the domains we had been researching suggested that the DNS tunnels on these domains may be attributable to the Pupy remote access trojan (RAT)4, which we were able to confirm. Upon analyzing files and reviewing DNS logs, we determined that one of the domains listed in the OSINT report – wmssh[.]com – is unrelated to Decoy Dog and unlikely to be associated with Pupy RAT in general. See Appendix A for information about how we determined this domain was not related to the others.
| Generally speaking, Pupy is a dangerous and powerful RAT due to its fileless nature and slow, encrypted C2 communications. It is hard to detect by EDR solutions, and can stay hidden for a long time in a given network. Interestingly, Pupy is one of the few RATs that offers broad multi-platform capabilities, uses an old version of Python, and therefore is able to infect a majority of Linux and mobile devices. |
While the Pupy RAT is available as open source on GitHub, it has been connected to nation state activities in the past. The RAT is a complex, multi-module trojan that provides no instruction to the user on how to establish the DNS name server in order to carry out C2 communications. As a result, it is not easily accessible to the common cybercriminal.
Once we established that queries generated by a RAT were originating from network appliances in our networks, we analyzed the associated domains and their timeline in-depth. We were able to identify a DNS signature, independent of the Pupy components, for which only 0.0000027% of registered domains share as of early April 2023. Reviewing these candidate domains, they either:
- contained only Pupy-related query traffic, or
- they appeared to be in the aging process; there was no activity on them, but they shared some infrastructure and DNS configurations with our validated domains.
This multiple-part signature gave us strong confidence that the domains were not only using Pupy, but they were all part of Decoy Dog – a large, single toolkit that deployed Pupy in a very specific manner on enterprise or large organizational, non-consumer, devices.
Based on the timeline we have established via DNS, this toolkit became active in early April 2022. As we will show later, the original activity was limited to a single C2 domain, cbox4[.]ignorelist[.]com. This domain leverages free, dynamic DNS allowing for obfuscation of both the domain holder and their activities. However, within six weeks, an additional domain with a controller in Russian IP space was visible in our networks, and it carried the exact same DNS signature. This domain was a lookalike for the Amazon service CloudFront, and the DNS activity was consistent with what is seen from enterprise network devices.5 Later, additional domains were registered and aged for various lengths of time before being observed either in our networks or global DNS. While there are three distinct infrastructure configurations in the original set of domains disclosed by Infoblox on 12 April 2023, all of these domains use a single toolkit: Decoy Dog. It is possible that there are multiple actors using the toolkit, and that it was either sold by, or stolen from, the original cbox4 domain holder.
The Infoblox Threat Intelligence Group investigated these domains in all major security vendors and found that almost none of them had been flagged as suspicious or malicious. In some cases, the vendors indicated that the domains were low risk or reputable. We found no previous intelligence that tied the domains together or established the connection they had with limited enterprise environments beyond the Telegram posting mentioned earlier in this paper. Below we detail the connections between these Decoy Dog domains as well as the technical details of this sophisticated toolkit.
Technical Analysis
What’s in Your Toolkit?
Decoy Dog is heavily reliant on Pupy, but it is not simply a fancy name for standard deployments of the RAT. Decoy Dog is a cohesive toolkit with a number of highly unusual characteristics that make it uniquely identifiable, particularly when examining its domains on a DNS level. The characteristics of Decoy Dog’s toolkit include:
- Pupy RAT: Pupy is the primary malware component of the Decoy Dog toolkit. While the malware is open source, deploying it as a DNS C2 is not trivial, and its wide array of capabilities have led to its use by nation-state threat actors such as Earth Berberoka.6
- Unique DNS Signature: Decoy Dog exhibits a unique DNS signature that matches less than 0.0000027% of the 370 million active domains on the internet. This signature is not a feature of standard Pupy installations, which strongly indicates that the domains are being deployed by the same toolkit, if not the same threat actor.
- DNS Beaconing / Outlier Behavior: Decoy Dog domains exhibit a pattern of periodic, but infrequent, DNS requests that makes them difficult to detect without a preventative DNS solution. This is not an inherent quality of Pupy, so the fact that all of the domains exhibit this behavior indicates they were configured by the same source. Infoblox’s outlier detection algorithms ranked Decoy Dog’s domains at the top of our list of suspicious activity due to their markedly unusual behavior.
- Shared Hosting / Registration Similarities: Decoy Dog domains can be grouped together based on their shared registrars, name servers, IPs, and dynamic DNS providers. Given the other commonalities between Decoy Dog domains, this is indicative of either one threat actor gradually evolving their tactics, or multiple threat actors deploying the same toolkit on different infrastructure.
- Enterprise Focus: Decoy Dog domains have only been observed on enterprise networks, and there is zero evidence of the domains being active on consumer devices.
Establishing C2 Channels
The creation and first seen date sequencing for each of the C2 domains is illustrated in Figure 2 below. In some cases, Decoy Dog establishes C2 communication channels relatively soon after domain creation. For example, it established a communication channel with claudfront[.]net only a day after the domain was created. However, in others, the domains are aged; allowlisted[.]net was activated two months after registration, and we are aware of several other domains that are not yet active.
Before Decoy Dog communicates with the C2 via encrypted DNS packets, it typically sends a DNS request to one of the C2 ping subdomains (which we describe further in the “Common Domain Characteristics” section later in the paper). This is then followed by a continuous series of DNS queries to subdomains with encrypted and high entropy labels. Such queries can persist in the network for a long time. At the time of this writing, we continue to observe encrypted DNS communications with claudfront[.]net; activity that has persisted for nearly a year. As expected from C2 communications, the fully qualified domain name (FQDN) changes and is largely unique.
Figure 2. The timelines of Decoy Dog domain activity. Not all domains are activated as of this writing.
Anomalous DNS Activity
While every C2 domain’s activity is unique, in this case we were able to find commonalities. The DNS queries had unusual characteristics at both the second-level domain (SLD) and FQDN level. This detail allowed us to flag domains we believed to belong to Decoy Dog. When reviewing queries it may initially appear obvious that it is a communication of some kind but what appears obvious is not always accurate. For example, the obfuscated domain name
- tzzjim3fv6dsgplfphfq9999[.]ouazdkj356dbfqeel11q9[.]cbox4[.]ignorelist[.]com
contains a great deal of entropy and unusually long labels. Furthermore, it was resolved only once in the environment. All of these details point to possible C2 communication. However, this activity is fairly common in DNS. The number of domains that are only seen once, and that contain a lot of entropy, is quite large. Most of these are not DNS C2 communications, but instead are trackers, content delivery networks, or anti-virus software. Here are some examples of domains that may appear to be DNS C2 communications, but are not:
- xg7d6df13a5efb41a09a7377211664afd6[.]dlszywz[.]com
- 14ya3cdc491acb5f5a94a0f6ff6acc943e9e[.]8×8[.]com
- aeejvmirozhlmkwz7z5xdixzh45i3ks7hxt4oeil6ituur222r2vq6zevz7jyyt[.]dw76tbqkkcgtbhd2fbcf3lgwgntszq7bp5mmttxgbt3zlmlnzctclcvwf6okq[.]t[.]e5[.]sk
- 3e4bae3b6d0addd08553fcbd8a2e2d24.mapdatamsnsdn[.]info
The domains in the list below are suspicious and show unique query behavior, but do not appear to be a C2 communication channel:
- secure-access-7205bhevc287do2b9.gate40[.]xyz
- jmaa55pmyg6xoce5ikr5fc6q8.conniejmoore[.]com
For Decoy Dog, the C2 domains prove to be extreme outliers in several of our analytics. They resolve intermittently and at very low volumes. They also show resolution IP addresses in an unusually high number of ASNs, a behavior typical of encoded IP addresses. They return an abnormal number of results and do so in a predictable manner. They have distinct characteristics in their subdomain labels, which we cover in more detail later in the paper. Most importantly, they appear in a limited number of enterprise networks with a semi-regular resolution pattern, and are not associated with consumer devices. The fact that the customer networks we protect with DNS provide critical goods and services to the world economy, and are not open internet points, alarmed us.
The resolution pattern of select Decoy Dog C2s from the Infoblox perspective is shown in the two figures below, Figure 3 and Figure 4. Each figure shows the pattern of activity as DNS queries from a single network. The figures demonstrate the timeline of the Decoy Dog C2s both relative to each other within a single network as well as how the domains are present in different networks at different times. Seen in aggregate, across all of our networks, this creates a similar pattern to that of SolarWinds. Because of our vantage point, we are unable to identify the cause of these queries, only to recognize that they show the signs of a persistent unknown low-profile communication. In particular the activity is consistent with DNS beacons.
Figure 3. The pattern of resolved queries over time by domains within a single network from April 2022 to April 2023. Each dot represents a DNS resolution on a given day. The domains on the graph, from top to bottom, are: cbox4[.]ignorelist[.]com, claudfront[.]net, allowlisted[.]net, and hsdps[.]cc.
Figure 4. The pattern of resolved queries over time by domains within a different single network. Each dot represents a DNS resolution on a given day. The domains on the graph, from top to bottom, are: claudfront[.]net, cbox4[.]ignorelist[.]com, and allowlisted[.]net.
There are also surprising characteristics in a fully qualified domain name that is queried. Typically with DNS tunneling activity, or exfiltration, we see a query once, with the exception of known or expected duplicates in DNS. With Decoy Dog, however, a domain might appear repeatedly over a few days and create a range of responses from the name server. In the majority of instances, a particular FQDN was only observed for a short period of time. If a DNS request for a particular FQDN failed, the FQDN was re-queried after a few days. When an IPv4 (A) record request was successful, either 3 or 15 IP addresses were returned. At some point, DNS requests for the FQDN began to fail. After a few days of failed attempts, requests for the FQDN ceased. This would be consistent with retransmitting DNS queries to a C2 server. However, the behavior of transitioning from a SERVFAIL to SUCCESS to SERVFAIL response is unusual, and it was repeated in the data.
While a single FQDN was generally observed within a given network at a time, there are instances of requests for multiple FQDNs overlapping in time. Figure 5 below illustrates the timelines of DNS requests for multiple FQDNs within a single network over a period of two months. We can see a progression from one FQDN to another, with a few instances of overlap between their usage. The slow, methodical process of cycling through the FQDNs with the low volume of requests is very difficult to detect. For the FQDNs shown, the maximum number of successful requests per day was three and the maximum number of failed requests per day was 11.
Figure 5. Timelines of the number of DNS requests over a two month period in a single sample network. Each line represents a query for a single FQDN with green denoting successful queries (maximum: 3/day) and black unsuccessful queries (maximum: 11/day).
Now that we have covered some of the outlier behavior from a DNS timeline perspective, and the manner in which these domains appear as beacons, we’ll look at the domains themselves.
Common Domain Characteristics
As we mentioned earlier, the C2 domains share a unique combination of DNS attributes, infrastructure and behavior that are very unique in the DNS landscape, giving us high confidence that they are related to the single Decoy Dog toolkit and could be managed by the same actor since April 2022.
Domain and Name Server Infrastructure
The information we have supports the use of Decoy Dog by one to three distinct actors. It is clear that cbox4[.]ignorelist[.]com was the original C2 domain. This domain is hosted on FreeDNS, a long established free dynamic DNS company that is often abused.7 FreeDNS offers a variety of services, one of which is the ability to establish a subdomain under one of their domains for your own use. The domain ignorelist[.]com is a long-established FreeDNS domain. While it is likely abused by many actors, it also hosts legitimate content. The specific subdomain, cbox4, which would be controlled by a single actor, was first observed on 4 April 2022.
Following cbox4[.]ignorelist[.]com, we see the registration and activation of claudfront[.]net. This domain is registered through NameCheap, but has a name server, or C2, in Russian IP space. It temporarily used claudfront[.]ml, a Freenom domain, as a name server in Spring 2022. This name server has consistently resolved to IP addresses in the MivoCloud ASN. While claudfront[.]net was activated almost immediately after registration, the second domain with a Russian IP name server was aged significantly. The domain allowlisted[.]net was registered in July 2022, but we see no evidence of C2 communication in our own, or in other DNS data, until early September 2022. This domain was also registered with NameCheap and shares the same name server IP address as claudfront[.]net. The seemingly unactivated domain atlas-upd[.]com also uses this IP address. Besides these domains, several other Decoy Dog domains, some unpublished, use name servers in MivoCloud ASNs.
A third set of infrastructure was adopted for Decoy Dog deployments in December 2022. This time the domains leverage dynamic DNS available from ChangeIP. Like FreeDNS, ChangeIP is a long-established, often abused provider of free and dynamic DNS.8 The domain hsdps[.]cc, is one of these. It was registered on 28 December 2022, and we observed C2 traffic in our DNS on 18 January 2023. Other domains, some unpublished, also use ChangeIP.
Because we see three distinct infrastructure choices, it is possible that there are three distinct actors using Decoy Dog, or even more. However, the overall usage of Decoy Dog is very rare; as mentioned earlier, we identified the domains through a combination of Pupy activity and a DNS fingerprint, and only 0.0000027% of registered domains share these characteristics. It is also clear that the Russian name server, using the same toolkit as Decoy Dog, first appeared about six weeks after the original cbox4[.]ignorelist[.]com. If these are different actors, we feel confident that the toolkit was stolen or sold by the cbox4[.]ignorelist[.]com domain holder.
Ping
Another common DNS attribute was a ping subdomain. The function of this resource is to return one or more reserved local addresses depending on the exact variation of the subdomain. Querying for {ping}.domain will always return 15 private loopback IPs via the DNS A and AAAA records. Alternatively, appending a number after {ping} will return that many IPs. For example, a query to ping5[.]atlas-upd[.]com always returned 5 private loopback IP addresses. See Figure 6.
Figure 6. Dynamic loopback response to ping requests.
This pinging behavior is explicitly defined by the Pupy DNS C2 server code in Pupy/network/lib/picocmd/server.py, specifically the _q_page_decoder() function. It ultimately will produce a sequential list of loopback addresses, either 15 addresses by default or a number of addresses equal to the number at the end of the ping label. See Appendix B for a detailed description of the ping functionality of Pupy.
X.509 Certificates
On analyzing the Pupy source code, we can see that it generates self-signed certificates with the country value listed as “ZZ” and the issuer as “Pupy CA.” It then also runs a TLS proxy on port 9876, using this certificate for authentication.
This matches up with what we are seeing on public internet scanning data, where several of these IPs using “Pupy CA” certificates also host some of the Decoy Dog domains that we have talked about previously. This further supports that Pupy is in use.
FQDN Labels
Based on our observations in passive DNS, we discovered a common labeling pattern for FQDNs containing at least two labels. These patterns correspond exactly with the known padding character of Pupy. The second label is nearly always 32 characters long and ends with the suffix “9999”, with a small number of outliers being 24 characters long and ending with the suffix “9”. The first label is more dynamic and is 8, 16, 24, 32, or even 56 characters long with various suffixes such as “9”, “9999”, and “999999”. The most common kind of first label is 24 characters long and ends with the character “9”, which we know from Pupy source code to be padding. The DNS logs also showed single-character or very short subdomain labels (e.g. m[.]hsdps[.]cc or ping4[.]hsdps[.]cc). Figure 7 below lists several examples. To ensure privacy, we have redacted these domains.
| tzzjim[redacted]plfphfq9999[.]ouaz[redacted]fqeel11q9[.]cbox4[.]ignorelist[.]com
msdu[redacted]elrjxq9999[.]ebfc[redacted]tvrhsa9[.]claudfront[.]net lyy4cc[redacted]esa9999[.]nuc[redacted]jkq9[.]allowlisted[.]net qzmwn[redacted]rv1a9999[.]123ry[redacted]ymi9[.]ads-tm-glb[.]click cme[redacted]js2ckza9999[.]111[redacted]1ykaka9[.]hsdps[.]cc |
| Figure 7. Examples of first and second subdomain labels, sections of which have been redacted. |
This behavior visible in the FQDNs is explained by Pupy’s DNS C2 server behavior, specifically the_q_page_decoder()function (described in Appendix B). Pupy FQDNs with at least two labels are encoded using Base32, and according to the Base32 encoding standards in RFC 46489 the length of a Base32-encoded string must either be a multiple of 8 or be padded by “=” characters to make it a multiple of 8. This explains why the FQDN labels we observed were all multiples of 8, and also why the labels were suffixed with varying amounts of “9” characters. The reason that the padding character observed in the labels is “9” rather than “=” is that the latter isn’t a valid character for a DNS hostname, so Pupy uses a translation dictionary to convert Base32-compatible strings to DNS-compatible strings and vice versa (see Table 2).
| Base32-compatible character | DNS-compatible character |
|---|---|
| A-Z (uppercase) | a-z (lowercase) |
| 0 | – |
| 1 | 0 |
| 2 | 1 |
| 3 | 2 |
| 4 | 3 |
| 5 | 4 |
| 6 | 5 |
| 7 | 6 |
| 8 | 7 |
| 9 | 8 |
| = | 9 |
| Table 2. Pupy Base32-DNS translation dictionary. | |
The Pupy translation is fairly straightforward for alphabetical characters, which are simply converted from uppercase to lowercase. The translation for numerical characters and symbols is slightly trickier because valid DNS hostnames cannot end in a “-” character, but the Base32 standard requires padding characters to appear at the end of a string. To account for this, Pupy’s translation dictionary converts a “0” character in Base32 to a “-” character for DNS hostnames, and all subsequent numerical characters to one less than their Base32 equivalent, e.g. a “1” in Base32 is converted to “0” for the DNS hostname. The Base32 “=” character is then converted to “9” in DNS, which allows the resulting label to achieve the proper amount of padding for Base32 encoding while still producing a valid DNS hostname.
The prevalence of two to three-label FQDNs is also an artifact of Pupy’s DNS C2 behavior. For details on how the decoder works, see Appendix C.
Pupy’s DNS C2 server conveys encrypted information back to the client via encoded A / AAAA record responses according to the DNS qtype of the incoming request. The client decodes and decrypts these responses, processes their contents, then generates another FQDN query to communicate back to the server. Pupy uses DNS specifically for C2 communications. It contains several other modules to perform other RAT functions using different protocols. Some samples of Pupy can be found in public repositories including VirusTotal; see Appendix D for specific references.
IPv4 Analysis
Given that part of our initial discovery was due to name servers for these domains returning A records across a broad range of ASNs, we wanted to do analysis on the IPv4 resolutions of these domains. We gathered every distinct query, and the associated rdata, of each Decoy Dog-related domain from global DNS data for the first five days of April 2023, prior to our disclosure of the activity on LinkedIn and Mastodon.10,11 The analysis shows that Decoy Dog resolutions have three large clusters of /8 CIDR ranges:
- 64[.]0[.]0[.]0/8 – 107[.]0[.]0[.]0/8,
- 128[.]0[.]0[.]0/8 – 171[.]0[.]0[.]0/8
- 192[.]0[.]0[.]0/8 – 235[.]0[.]0[.]0/8
| As a refresher, a CIDR /8 range means that only the first 8 bits of the IP address are fixed. For example, a CIDR /8 range of 1[.]0[.]0[.]0/8 includes all IP addresses that have 1 as the first octet. This means the remaining three octets can take any value between 0 and 255 (a CIDR /8 range has 224 possible IP addresses). Examples include 1[.]1[.]1[.]1, 1[.]234[.]23[.]111, and 1[.]254[.]254[.]254. |
We generated an IPv4 map of all the resolutions. We did this by utilizing a Hilbert curve, which maps one-dimensional data onto a two-dimensional space. This curve, and its application to IPv4 maps, is perhaps best described by the famed comic series XKCD.12 We can see in Figure 8 the composite result over this 5 day period. The resolution IP addresses for the Decoy Dog domains are not evenly distributed and do occasionally collide with reserved IP spaces, denoted as light grey cells in Figure 8.
Figure 8. A Hilbert graph of all global DNS resolutions across all known Decoy Dog C2 domains. This demonstrates that the resolution IP addresses across all domains is limited to only a certain portion of the internet IPv4 space.
The key idea behind the Hilbert curve is that it groups nearby points in a one dimensional sequence (nearby IP addresses in this case) into nearby cells in a two-dimensional space. Each grid in Figure 8 represents a CIDR /8 range, represented by the number in the top-left corner of each cell. In Figure 8, red denotes where we saw a unique, resolving IPv4 address; white denotes where we did not see a resolving IPv4 address. The darker red a cell is, the more clustered the resolving IP addresses are. In Figure 9 below, cell 74, for example, is entirely filled with red, indicating we saw a tremendous number of IP addresses in that CIDR /8 range as an A record. A different way to visualize Figure 8 is via a bar chart (see Figure 10). A strange artifact of mapping the Hilbert graph not noticeable in the bar chart is that there is one large, main pattern repeated 3 times across the entire IPV4 space. Figure 9 shows a larger view of the repeated 8×8 grid and the subpatterns that constitute the main pattern.
Figure 9. Zoomed in version of Figure 8, to show the repeating 8×8 pattern of Decoy Dog IPv4 resolution.
The sporadic, blocky filling pattern identifiable in cell 64 is also seen in cells 128 and 192 (the beginning range of each IPv4 cluster – identifiable in Figure 8). Cells 82 and 83, 146 and 147, and 210 and 211 are where we find the medians of each of the three large clusters, and appear as the cells showing the pattern of white dots on the red background. The cells with the white background and red dots form a “backwards S” pattern and represent sporadic resolutions in each respective CIDR /8 range (identifiable in cells 94 to 107 in Figure 9). Figure 10 provides a different visualization of the same information as Figure 8.
Figure 10. The distribution of Decoy Dog IPv4 resolutions in /8 CIDR format. The X-Axis represents only the left-most bit (1, 2, …, 255), as that’s what distinguishes a /8 CIDR range.
Using a bar graph, it’s clearer to see the three large clusters of CIDR /8 ranges that Decoy Dog gives as resolution addresses to their qnames. Interestingly, at the start of each cluster, there is a small gap. For example, we see resolution addresses of 64[.]0[.]0[.]0/8 and 66[.]0[.]0[.]0/8, but not of 65[.]0[.]0[.]0/8. The same is true for the other two clusters.
Now let’s compare the Hilbert map for resolution IP addresses for two distinct Decoy Dog domains. In Figure 11 below, we see the resolutions for cbox4[.]ignorelist[.]com and claudfront[.]net. Notice that individually they match the same exact pattern as the aggregate distribution and each other.
 |
 |
Figure 11. A comparison of the Hilbert mapping of IPv4 resolution IP addresses for cbox4[.]ignorelist[.]com (left) and claudfront[.]net (right). The former domain is the original C2 and the latter domain is the first C2 in Russian IP space.
We expected to see IPv4 addresses flatly distributed across the routable IPv4 space, but what we actually found was far different from that. The analysis indicates a strong pattern in the distribution of the C2 queries’ resolution addresses, at least in our collection perspective, but due to time constraints, a deeper analysis of the distribution cannot be conducted for this report. What is clear is that this distribution is very similar across all the Decoy Dog domains.
Mitigation and Infoblox Solutions
This threat creates a footprint in DNS that is extremely hard to detect in isolation, but demonstrates strong outlier behavior when analyzed in a global cloud-based protective DNS system like BloxOne Threat Defense. As a specialized DNS-based security vendor, Infoblox tracks adversary infrastructure and can see suspicious activity early in the threat lifecycle, when there is “intent to compromise” and before the actual attack starts.
We believe that global security industry collaboration is necessary to understand the full end-to-end story of Decoy Dog and the C2 activity. Over the last week, a number of theories have been made to explain the anomalous behavior in our networks; however, we have been unable to validate those hypotheses fully. We are releasing this initial report to detail the DNS elements of Decoy Dog in order to help the community identify the underlying cause and to protect enterprises from Decoy Dog. We will continue to analyze the DNS patterns and collaborate with other members of the security industry so that we can fully explain how this DNS C2 appeared in our networks. Some of our findings are being held back in order to limit copycat behavior. Security vendors and network providers with visibility into enterprise networks may contact us for further information.
As with any indicators that are deemed suspicious, we included the domains listed in this advisory into Infoblox’s Suspicious domains feed in BloxOne Threat Defense (Advanced) to enable organizations to pre-emptively protect themselves from new and emerging threats. In addition to the Suspicious Domains feed, these domains and appropriate IPs have now been added to Infoblox’s anti-malware feed.
Organizations with protective DNS are able to block these domains immediately, mitigating their risk while they continue to investigate further. We recommend that organizations that are not customers of BloxOne Threat Defense add the indicators of compromise (IOCs) from this advisory to their blocklists manually or via our GitHub repository infobloxopen:threat-intelligence.13 We derived the IOCs below from our analysis of the campaigns mentioned in this report.
| Indicator of Compromise | Description |
|---|---|
| claudfront[.]net | Pupy C2 |
| allowlisted[.]net | Pupy C2 |
| atlas-upd[.]com | Pupy C2 |
| ads-tm-glb[.]click | Pupy C2 |
| cbox4[.]ignorelist[.]com | Pupy C2 |
| hsdps[.]cc | Pupy C2 |
| claudfront[.]ml | Temporary name server domain for claudfront[.]net |
| e47db5ef2a23a156856b5ea3b156a32fc8b26fb1a5c496f62e74c8ca8bf4b924 | SHA-256 Fingerprint for Pupy SSL Certificate on 213[.]183[.]48[.]75 (claudfront[.]ml and claudfront[.]net) |
| 84a2ed4270aaee360019f8136e464fbddb83d20ade79b43b712c711a632dfa14 | SHA-256 Fingerprint for Pupy SSL Certificate on 83[.]166[.]240[.]52 (allowlisted[.]net) |
| fa075deeb0af84792a08f6be728ea15f1cf6183443cc5ee8a0632c7b4209675f | SHA-256 Fingerprint for Pupy SSL Certificate on 5[.]252[.]176[.]22 (allowlisted[.]net) |
| 4996180b2fa1045aab5d36f46983e91dadeebfd4f765d69fa50eba4edf310acf | SHA-256 of a malware sample communicating with the domains |
| 0375f4b3fe011b35e6575133539441009d015ebecbee78b578c3ed04e0f22568 | SHA-256 of a malware sample communicating with the domains |
| 5[.]252[.]176[.]63 | IPv4 address for ns1[.]claudfront[.]net & ns2[.]claudfront[.]net |
| 5[.]252[.]176[.]63 | IPv4 address for ns1[.]atlas-upd[.]com
& ns2[.]atlas-upd[.]com |
| 5[.]199[.]173[.]4
5[.]252[.]176[.]22 83[.]166[.]240[.]52 |
IPv4 addresses for ns1[.]allowlisted[.]net
& ns2[.]allowlisted[.]net |
| 213[.]183[.]48[.]75 | IPv4 addresses for ns1[.]claudfront[.]ml & ns2[.]claudfront[.]ml |
Appendix A: wmssh[.]com and Freegate VPN
We assessed that the wmssh[.]com domain that was shared on Telegram is not related to the Decoy Dog beacon domains. Based on evidence we gathered, it is highly likely to be affiliated with the Freegate client, a free VPN tool for Windows users that was initially developed by Dynamic Internet Technology, Inc. to bypass internet censorship in China and Iran.
DNS queries to wmssh[.]com show FQDN patterns that are strikingly different from the other beacon domains. It typically contains subdomain labels that are over 50 characters in length, and none of the queries contain the “9”, “9999”, or “999999” paddings we commonly saw across the other beacon domains. This indicates use of a different DNS tunneling technology than the one used by Decoy Dog’s beacons. Additionally, the DNS configuration and setup of wmssh[.]com suggests a different operator. It is registered with Google and uses Amazon’s Route 53 DNS service instead of its own name servers. Furthermore, the domain is relatively old and created back in March 2013.
We identified two Freegate Windows executable clients that communicated with wmssh[.]com in the past. In one instance, Freegate version 7.40 (SHA256: a5c76a85c67fd7d81c9945432c4cace45014f68703bdcdd688f7cd3943db326a, released in 2013) made DNS queries to the domain and initial communication included calls to wua[.]40xhtgh[.]wmssh[.]com. We verified that the SHA256 matches the hash of the real Freegate 7.40 client.14 A more recent version, Freegate 7.83 Restricted (SHA256: 85cdad039eaded7cabd7264e5f2a61dc57cadeead7019f476792d844d7c2934e, released in 2020) showed initial DNS queries to qq[.]74lmth4[.]wmssh.com and rgk[.]74lmth4[.]wmssh[.]com. We also verified this hash by directly downloading the specific client from hXXp://dongtaiwang[.]com/loc/software/fg/783/fg783r.exe and comparing the checksums. This site is a primary distributor of the proxy tool in China and is tailored to Chinese-speaking users and promotes the VPN, which allows access to websites blocked by the Chinese government. However, the software does have potentially unwanted features, including browser hijacking, potentially unwanted application (PUA) distribution, and automatic promotion of the dongtaiwang[.]com news website (see Figure 11).
Figure 11. Windows Freegate user interface.
Appendix B: Ping Function Details
This function takes the incoming DNS FQDN minus the base hostname of the Pupy C2 server as its data argument. For example, a DNS query for ping5[.]atlas-upd[.]net would have ping5 as its data argument. It then splits the data string into a list separated by dot characters which it stores in the parts variable. If parts contains exactly one string that starts with “ping” it will produce a sequential list of loopback addresses, either 15 addresses by default or a number of addresses equal to the number at the end of the ping label.
Figure 12. Pupy _q_page_decoder() function
Appendix C: Pupy FQDN Label Structure
The _q_page_decoder() explicitly looks for FQDNs that are exactly 2-3 labels long and parses them according to the format {nonce}.{payload} for 2-label FQDNs, or {spi}.{nonce}.{payload} for 3-label FQDNs (see Figure 13 below). The functional meaning of each type of label is as follows:
{spi} is the Security Parameter Index, which is a unique session identifier that allows Pupy to distinguish between traffic streams and determine which encryption protocols and keys to use when communicating with a particular client. This label is only present in 3-label FQDNs, and due to its uniqueness it can be used to identify ongoing communications between the Pupy server and a particular infected client.
{nonce_blob} is a cryptographic nonce, which is an arbitrary value that can only be used once in a given cryptographic communication. Pupy’s DNS C2 checks the nonce value and produces an empty response if the nonce from the client doesn’t match the last nonce produced for the session, which the server sends to the client as part of its A / AAAA response so that the client knows the correct nonce value to use for its next query. This ensures that communications between the Pupy server and client are protected from replay attacks, threat researchers, and other third parties who may be interested in repeating queries for known Pupy FQDNs. If the length of the {nonce_blob} is greater than 4 characters after decoding from Base32, the remaining characters are interpreted as unique identifiers that the server uses to distinguish between clients.
{payload} is the encrypted data that the infected client is sending to the server. After decrypting the payload, the server will process it and take various commands based on its contents. These actions include logging information from the infected client, exchanging cryptographic key information between the server and client, providing commands for the client to execute, and more.
Figure 13. Pupy 2-3 label FQDN parsing from _q_page_decoder() function.
Appendix D: Pupy Executable Samples
Two Linux executable files related to Pupy variants communicated with the C2 domains claudfront[.]net and cbox4[.]ignorelist[.]com are available in VirusTotal. The DNS traffic generated by these files is similar to the behavior described in this report. Users geolocated in Russia were the first to submit the files to VirusTotal. The file names and hash values are described in Table 3.
| SHA256 | Submission Timestamp | Submitter’s Country |
|---|---|---|
| 4996180b2fa1045aab5d36f46983e91dadeebfd4f765d69fa50eba4edf310acf | 27 September 2022 10:36:43 AM GMT | Russia |
| 4996180b2fa1045aab5d36f46983e91dadeebfd4f765d69fa50eba4edf310acf | 27 September 2022 11:24:41 PM GMT | United States |
| 0375f4b3fe011b35e6575133539441009d015ebecbee78b578c3ed04e0f22568 | 13 February 2023 7:39:55 AM GMT | Russia |
| Table 3. Pupy executable and linkable format (ELF) files available on VirusTotal. | ||
*“Our networks” refers to the aggregate collection of all networks that resolve DNS through BloxOne Threat Defense in the cloud.
References
- https://blog.apnic.net/2016/04/04/dns-zombies/
- SolarWinds suffered a supply chain attack in March 2020 and selected networks were communicating with a controller via DNS for over 8 months undetected.
- https://t.me/cybersquattingchannel/2969/
- https://github.com/n1nj4sec/Pupy
- https://aws.amazon.com/cloudfront/
- https://documents.trendmicro.com/assets/white_papers/wp-operation-earth-berberoka.pdf
- https://freedns.afraid.org/
- http://www.changeip.com/
- https://www.rfc-editor.org/rfc/rfc4648
- https://tinyurl.com/ddoglinkedin
- https://infosec.exchange/@knitcode/110192910797988943
- https://xkcd.com/195/
- https://github.com/infobloxopen/threat-intelligence
- https://www.herdprotect.com/fg740p.exe-57edc8b94c7595c41b24fc0c9b44a13f4ce945e4.aspx