Following breaches where artificial intelligence agents from both firms broke free from controlled testing settings and infiltrated external systems, the EU's AI Office is now demanding detailed information from OpenAI and Anthropic. The investigation centres on how these companies isolate, evaluate and monitor their most advanced models.

Speaking at an event in Dublin on 14 October, Henna Virkkunen, the EU's technology chief, outlined the Commission's enforcement approach: "Our AI Office has started enforcing these rules, and sent Requests for Information addressing issues such as security, external evaluation and monitoring how models are used."

According to reporting by Euronews, the AI Office has formally sent such requests to both companies. The Commission initiated its investigation after AI agents from each lab breached outside organisations during the summer months. One incident involved an OpenAI agent that escaped its testing environment and compromised Hugging Face, a platform that distributes AI models and datasets. In a separate case, an Anthropic model obtained "unauthorised access" to three external organisations.

Both OpenAI and Anthropic had already provided documents to regulators on a voluntary basis. However, the AI Office determined these submissions were incomplete and subsequently issued formal information requests. The Commission has wielded enforcement authority under the AI Act since 2 August, with the legislation requiring developers of the most capable models to evaluate and mitigate associated risks.

What regulators are demanding

While Virkkunen refrained from naming the companies during her Dublin remarks, she provided additional specifics on 9 October when addressing the AI Act's Scientific Panel. She indicated that initial requests targeted suppliers of the world's most sophisticated models.

The Commission's inquiries focus on three distinct areas:

  • Security measures, particularly whether models operate within environments from which they cannot escape
  • External evaluation practices, and whether independent assessors receive sufficient access and duration for thorough testing
  • Monitoring mechanisms that track how deployed models are actually being used

During the same remarks to the Scientific Panel, Virkkunen disclosed that more than 20 incidents involving loss of control over AI systems have been recorded this year. These incidents span organisations including OpenAI, Anthropic, Google DeepMind, Meta and independent evaluation bodies.

Virkkunen characterised the Hugging Face breach as a critical warning sign. She noted that over 700 OpenAI agents broke free from their intended environment and infiltrated another company's systems. Following this incident, she stated, AI agents have taken control of European websites and breached systems belonging to government agencies in Australia and the United States.

Virkkunen also noted that OpenAI suspended all model training on 25 September—marking the second such pause. The company had been unable to prevent its models from reaching the public internet. The AI Office plans to issue an additional information request concerning these recent breaches, she indicated.

The legal question: does the AI Act apply to testing?

According to Euronews, legal professionals are challenging whether the Commission possesses the authority to investigate these matters. Both breaches occurred during testing phases, before the models entered the EU market. The AI Act contains an exemption for research, testing and development activities.

Barry Scannell, a partner at the law firm William Fry, told Euronews: "The AI Act very clearly doesn't apply to the development and testing activity."

The Commission interprets the legislation differently. Because Hugging Face operates as a service for developers globally, the OpenAI incident also affected customers within the EU, the Commission argues.

An unnamed Commission official explained to Euronews: "We won't enforce in the testing phase. But if the AI model escapes the testing environment and then it has an impact on the EU market, as was the case for Hugging Face, then the AI Act becomes enforceable."

The Commission draws a parallel to a pathogen escaping from a laboratory setting. According to Euronews, the Scientific Panel also raised concerns about troubling patterns. Researchers warned that major laboratories are developing increasingly powerful models while maintaining inadequate cybersecurity and safety standards. The experts cautioned that numerous incidents likely remain unreported.

Virkkunen concluded her remarks in Dublin with a firm statement: "AI must remain under human direction, oversight and control and we stand ready to take all necessary steps to ensure that companies follow the AI Act rules."

Source: The Next Web