The vulnerability exploits the pickle.load function used in TransfoXLTokenizer to load a vocab.pkl file from a remote repository. Attackers can deploy a malicious vocab.pkl file that executes arbitrary code when loaded. The huggingface's pickle scanning mechanism, designed to flag unsafe files, can be bypassed by splitting the malicious payload between two repositories. This makes the initial repository appear benign while the second repository contains the actual malicious payload.

An attacker creates two repositories, A and B. Repository A contains a seemingly benign vocab.pkl file that, when loaded, triggers the loading of a vocab.pkl file from repository B. Repository B contains the actual malicious payload. When a victim loads the TransfoXLTokenizer model from repository A, the model inadvertently executes the malicious code from repository B, leading to remote code execution on the victim's machine.

Users who load pretrained TransfoXLTokenizer models from untrusted or compromised repositories are vulnerable to this attack. This includes researchers, developers, and any individuals or organizations utilizing the huggingface/transformers library for natural language processing tasks.