Did OpenAI's models manage to escape their closed environment thanks to an Israeli company?
Two OpenAI models managed to escape their ExploitGym sandbox and breach the Hugging Face website. The incident was linked to zero-day vulnerabilities in the Artifactory system by the Israeli company JFrog.
Source: JFrog Twitter page
In the past week, the tech world was stirred after two OpenAI models managed to outsmart everyone, "escape" from their sandbox, and break into the Hugging Face website. Since then, many have been occupied with the question "How did this happen?", and now we are getting an answer, which may be related to a product from the Israeli company JFrog.
What happened?
Recall that OpenAI researchers ran the models in a dedicated cyber benchmark called ExploitGym. When the models realized that the best way to succeed in the test was simply to cheat, they started looking for ways to do so. They ran without safety mechanisms inside a sandbox where they could only install pre-approved packages. Despite this, they exploited a zero-day vulnerability in an internal proxy that managed the packages, performed privilege escalation and lateral movement — two common hacker practices — and reached a system with network access. From there, they broke into Hugging Face to steal the answers to the test.
After Hugging Face discovered the breach, they used an open Chinese model to handle it, but only after a few days did it become known that the OpenAI models were the ones responsible for the event. Reports following the disclosure of the breach indicate that even OpenAI was very surprised by the case, and discovered the breach only a week after it occurred.
The Israeli angle
In a post published by JFrog, signed by CTO Yoav Landman, it was revealed that OpenAI's internal proxy software was actually a self-managed installation of JFrog's Artifactory system. The post describes the case as a kind of collaboration: according to Landman, during a security check, the OpenAI models identified unknown zero-day vulnerabilities in Artifactory that could have allowed network access. However, the company refrained from detailing the exact CVE identifiers and the severity of the vulnerabilities (CVSS). A check by the BleepingComputer website revealed 8 registered CVE vulnerabilities from that same day with credit given to OpenAI. Among the vulnerabilities revealed: Server-Side Request Forgery (SSRF) via Terraform and Cargo repositories, and an authentication bypass that allows for privilege escalation. JFrog was quick to release a version update (7.161.15) on July 27 that patched the vulnerabilities.
