[Cyware] How Polyglot Files Enable Cyber Attack Chains and Methods for Detection & Disarmament

The survey, performed between November 2022 and January 2023, focused on identifying the role of a polyglot file within a threat actor’s cyber-attack chain.
We used publicly available independent sources, general search engines and threat intelligence feeds (e.g., ORKL, X) to gather a wide range of information security reports and articles. Those sources were searched using the following terms: polyglot, combined, and contained. We found that the term polyglot is not always utilized in reports. We therefore had to manually distinguish between reports of true polyglots (two or more valid formats in one file) and other forms of digital steganography. A number of reports described malware that contained a valid format along with an oft-encrypted set of malicious instructions. We do not consider these files as polyglots because the malicious instruction can only be correctly interpreted when passed as input to another component of the malware rather than a parser conforming to a published standard.

For each true polyglot found, we used our knowledge of threat operations to determine the role the polyglot played in the cyber-attack chain. Lastly, the online malware databases, VirusTotal and MalwareBazaar, were used to obtain the actual polyglot samples whenever hashes of the polyglot were provided in a report. The file hashes and sources from our survey of open-source intelligence can be found in the appendix in Tables 3 and  4, respectively.

The survey discovered fifteen examples of a threat actor using a polyglot file in their cyber-attack chain, along with 30 distinct polyglot files.
According to MITRE’s Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK) framework, polyglots are primarily utilized for Defense Evasion (MITRE ATT&CK TA0005). Polyglot files also fall under the Obfuscated Files or Information (MITRE ATT&CK T1027) heading since these files conceal hidden functionality by appearing to conform to only one file format. We obtained 30 polyglot samples from VirusTotal and MalwareBazaar using the file hashes specified in the reports.

For the purpose of establishing a formal taxonomy for polyglot files, we refer to polyglots as having an overt format and a covert format. The overt format is the format the file presents as (e.g., matches the extension) while the covert format is not apparent without analysis. In most cases, a polyglot consists of a malicious file combined with a benign one; however, in some cases we found that both file formats play a role in advancing the malicious attack chain, as in the HTA+CHM polyglot utilized by IcedID in Section 3.2.1. Therefore, we instead refer to polyglots as combining an overt format with a covert format. A summary of the found-in-the-wild samples is provided in Table 1. In Appendix 9.2 we discuss the capabilities of interest that each file format provides to the malware author (camouflage, non-standard execution path, etc.) to understand why these combinations exist in the wild and how they fill a desired role in attack chains.

We selected two cyber attack chains to demonstrate how well-known APTs utilize polyglots to reach the next step in their cyber attack chains. A third attack chain can be found in Appendix 9.1. CVE numbers and MITRE ATT&CK references are provided where applicable.

The Fazah framework is a modular tool written in Python that can currently generate 46464646 format combinations using 8888 covert formats. The combination method—stack and a variety of parasites—is derived from reports of malicious use in our survey and varies between covert format. As discussed in the survey, malicious actors use polyglots either to disguise malicious content using a less suspicious format (images) or add hidden functionality (scripts). Since image formats typically use comment markers, parasites are commonly used by malicious actors. Stacks, meanwhile, are the simplest and easiest method for malicious actors to implement, working well with script and archive formats. Files with distinct comment markers (necessary for zippers) are quite rare. Of the common (but by no means exhaustive) set of formats we tested, only DCM combined with either PDF/GIF/ISO could result in a zipper. Similarly, we found that only ISO paired with PE/PNG/GIF yielded cavities. This does not preclude their use in malicious campaigns, but places them beyond scope for our goal of emulating known attack chains. Table 1 provides the format pairings that Fazah can turn into polyglots.
Given the possibility for malicious abuse of the framework, Fazah will not be published publicly at this time.

Our first objective was to determine which machine learning architecture and feature set were most effective at detecting polyglots. Toward this end, we created a small (∼70,000similar-toabsent70000sim 70,000∼ 70 , 000 files) initial data set using the mitra tool (described in Section 2.2) prior to the development of our Fazah tool. On this preliminary data set, we tested a Support Vector Machine, Random Forest, GradBoost, CatBoost, LightGBM, and MalConv. With the exception of MalConv [Raff et al.(2018)], these models used the byte histogram as their only feature. The byte histogram is a vector of length 256 where the value stored at each index corresponds to the number of times that byte value occurs in the input file. This feature vector is agnostic with respect to file formats since all digitally stored files are a string of bytes. We found that, on this preliminary data set, MalConv and CatBoost were the top performers.

We focused further development on MalConv and CatBoost, labeling our improved versions PolyConv and PolyCat, respectively. At this point, we trained and tested both models on our survey-informed Wild Polyglots data set; all results and figures reported in this paper refer to the Wild Polyglots data set. We found that, for PolyCat but not PolyConv, adding the mime-type output of the file utility improved results. Although file was not competitive at detecting polyglots (see Section 6.1) or at identifying both formats contained within, it was extremely accurate at identifying the first format contained in the file. Therefore, we augmented PolyCat’s feature space with a 1111-hot encoding of the mime-type output from file. We found further improvement by adding the 8000800080008000 most common bigrams and trigrams extracted from each file using an overlapping window. Thus, the final feature space for PolyCat consisted of the byte histogram, the 1-hot encoding of the mime-type from file, and the most common bigrams and trigrams.

MalConv is an oft-cited deep learning classifier designed to detect malware [Raff et al.(2018)]. We trained the model from scratch to identify polyglots rather than to identify malware. None of the polyglots in our data set were malicious in order to guarantee that the model learned to detect multiple formats rather than malicious content. Since the model is trained on raw bytes rather than format-specific features (e.g., the EMBER feature set for PE files [Anderson and Roth(2018)]), MalConv’s architecture is well-suited to the polyglot detection problem which requires a format-agnostic approach. In lieu of a fixed feature-extraction routine, the model takes in raw bytes and learns an encoding (first layer) as well as a set of filters (the convolution layers) to recognize significant byte patterns. MalConv also features an attention and gating mechanism intended to filter out extraneous information in the raw bytes.

We experimented with changes to the architecture in order to make it more effective at our novel task, yielding the PolyConv model mentioned above. The original architecture of MalConv is presented in Figure 7 while PolyConv’s architecture is presented in Figure 8.

We established a baseline for performance by testing existing file-format identification tools on the Wild Polyglots data set: file [Darwin et al.(2019)], binwalk [ReFirmLabs(2021)], TrID [Pontello(2020)], and polydet [pol(2018)]. We also evaluated polyfile [of Bits(2022)], a DARPA-funded tool developed by Trail of Bits for detecting unusual files.
Of the aforementioned tools, file and TrID are well-established signature-based utilities for file-format identification. VirusTotal, a widely used anti-virus aggregator (www.virustotal.com), utilizes TrID when reporting detected formats. Binwalk is a file-carving tool that has been used by analysts to find and extract hidden files. We selected these tool to establish a baseline because of their wide-spread use (file), cybersecurity application (binwalk,TrID), and polyglot-awareness (polyfile, polydet). We leave as future work a comparison to FiFTy [Mittal et al.(2020)] and Sceadan [Beebe et al.(2013)], as these detectors do not appear to be polyglot-aware, but might be re-trained in order to properly label polyglot files. Since file outputs labels and not probabilities, the precision-recall curve is not an appropriate metric when comparing our deep learning model to existing tools. Instead, we used the F1 score that balances recall and precision. For any cybersecurity system deployable in the real-world, the ability to detect malware/polyglots (recall) must be tempered by a low probability of false positives (precision) to prevent red-flag fatigue. Therefore, we use F1 to provide a balanced evaluation.

Content disarmament and reconstruction (CDR) tools present an alternative approach to the pre-processing filtering approach for which we have provided solutions. CDR tools allow an end user to strip all but the most trustworthy content from certain formats. Where highly flexible formats, like PDF, have proliferated, these tools have emerged to provide secure use of files that abuse the format flexibility.

Although we have not exhaustively examined this approach, we have developed an image sanitization tool to demonstrate the potential of CDR in disarming polyglots.
Our tool, ImSan, disarms image-based polyglots by stripping away all file contents that are not required to display the image. The process is quite straightforward:

1. 1.

   The image file is loaded into Pillow, a fork of the Python Imaging Library

2. 2.

   The image contents are then written to a new file with the option to strip all metadata activated

3. 3.

   The new image file has no extraneous content before/after the image contents (stack/cavity polyglot) or inserted into comment areas (parasite/zipper polyglot)

ImSan can disarm any of the formats that are fully supported (read/write) by Pillow: BLP, BMP, DDS, DIB, EPS, GIF, ICNS, ICO, IM, JPEG, JPEG 2000, MSP, PCX, PNG, PPM, SGI,SPIDER, TGA, TIFF, WebP, XBM.
Note, ImSan should be run in an isolated environment to ensure that no vulnerability in Pillow (2 CVE’s reported in 2022) could allow a malicious image to gain execution when the image is parsed.

ImSan disarmed 100% of the image polyglots in a subset (n=392) of image polyglots drawn randomly from the benign Wild Polyglots data set.
A small subset was chosen so we could manually verify disarmament through visual inspection of the image’s code rather than relying on one of our detectors.
An evaluation of commercial CDR tools against polyglots (including those that are not image based) and the potential methods of circumventing CDR solutions, while out of scope for this work, would be a valuable direction for future work to explore.