Critical Vulnerability In AI-As-A-Service Provider Let Attackers Access Sensitive Data

A critical vulnerability was found in the Replicate AI platform that could have exposed the private AI models and application data of all its customers.

The vulnerability stemmed from challenges in tenant separation, a recurring issue in AI-as-a-service platforms. 

By exploiting this, attackers could have gained unauthorized access to user prompts and the corresponding AI results, as the security flaw was responsibly disclosed to Replicate and promptly addressed, with no customer data compromised. 

Replicate, a platform for sharing AI models, allows users to upload containerized models using their Cog format, including a RESTful API server, potentially enabling malicious code execution. 

Researchers created a malicious Cog container and uploaded it to Replicate, achieving remote code execution on Replicate’s infrastructure.

This highlights a potential vulnerability in AI-as-a-service platforms, where untrusted models can be a source of attacks. 

Similar techniques were previously used to exploit Hugging Face’s managed AI inference service. 

An attacker gained root privileges within a container on Replicate’s Kubernetes cluster, as the container shared its network namespace with another container with an established connection to a Redis server. 

By exploiting CAP_NET_RAW and CAP_NET_ADMIN, the attacker used tcpdump to identify the Redis connection, confirmed it was plaintext, and then aimed to manipulate the shared Redis queue to impact other replicate customers potentially. 

According to the Wiz Research Team, the attacker lacked credentials for direct access and devised a plan to inject packets into the existing authenticated connection. 

The authors exploited a vulnerability in a shared Redis server to gain unauthorized access to customer data by injecting TCP packets containing Redis commands to bypass authentication. 

While modifying existing entries in the Redis stream proved difficult due to its append-only nature, the authors were able to manipulate the data flow. 

They achieved this by injecting a Lua script that identified a specific customer request, removed it from the queue, altered the webhook field to point to a malicious server they controlled, and then reinserted the modified request back into the queue, which allowed them to intercept and potentially alter the prediction results sent back to the customer.

A critical vulnerability in Replicate’s AI platform allowed attackers to potentially steal proprietary knowledge or sensitive data from customer models through malicious queries. 

Moreover, attackers could manipulate prompts and responses, compromising the models’ decision-making processes.

This vulnerability threatened the integrity of AI outputs and could have had severe downstream impacts on users who rely on those models. 

Source: Original Post