The vulnerability stems from the application's handling of the chat_template parameter, which is processed by the Jinja template engine. By crafting a malicious tokenizer_config.json file and using it in conjunction with the /completions endpoint, attackers can exploit the application to execute arbitrary code. This is achieved by injecting template syntax to manipulate the Jinja environment, ultimately leading to the execution of system commands.

An attacker starts by creating a malicious Hugging Face model with a specially crafted tokenizer_config.json file that includes a payload exploiting the SSTI vulnerability. The attacker then sends a request to the /completions endpoint of the vulnerable application, specifying their malicious model. The application processes the request, leading to the execution of the attacker's payload and potentially compromising the server.

Any instance of the berriai/litellm application up to version 1.23.2 that exposes the /completions endpoint is vulnerable to this attack. This includes servers hosting the application and potentially allows attackers to gain control over these systems.