Discovery of Malicious Machine Learning Model Attacks on PyPI

A recent campaign targeting machine learning models has been uncovered within the Python Package Index (PyPI) by cybersecurity experts. Researchers from ReversingLabs reported that malicious actors are utilizing the Pickle file format to disguise malware within seemingly legitimate AI-related software packages.

In this instance, three deceptive packages were identified: PLACEHOLDER3dfdf306ebdb86d4, PLACEHOLDERdaeee21df8fa64a4, and aliyun-ai-labs-sdk. These were falsely promoted as a Python SDK for Alibaba’s AI services. Contrary to their claims, these packages bore no authentic AI code. Instead, they incorporated an infostealer payload hidden within PyTorch models, which are essentially archived Pickle files.

Once installed, the payload was triggered from the initialization script, designed to extract critical data, including:

– User and network information
– Organizational affiliation of the target machine
– Contents of the .gitconfig file

Alarmingly, the malicious models made efforts to detect developers linked to the Chinese video conferencing platform AliMeeting, indicating a specific regional targeting strategy.

The Perils of PyTorch and Pickle

This incident underscores a rising trend in the exploitation of machine learning model formats. According to ReversingLabs, the use of Pickle allows serialized Python objects to run arbitrary code, making it an attractive vector for cybercriminals looking to bypass standard security measures. Of the malicious packages identified, two successfully utilized this method to distribute functional malware.

The researchers highlighted a significant gap in existing security tools’ capabilities to detect embedded malicious actions within machine learning files. “Security tools are at a primitive level concerning malicious ML model detection,” stated Karlo Zanki, a reverse engineer at ReversingLabs. “Legacy security tooling currently lacks the necessary functionality.”

The infected packages were available on PyPI for a brief period, accumulating approximately 1,600 downloads before their prompt removal. While the specific tactics employed to entice users remain uncertain, the potential for social engineering or phishing tactics is highly suspected.

As AI and machine learning become integral to software development processes, this attack emphasizes the pressing need for enhanced validation practices and the implementation of zero-trust principles in the management of machine learning artifacts.