AI tools that are supposed to speed up cyberattack investigations can suddenly refuse to analyze suspicious code — even when they are being used by specialists trying to protect their own systems. As Axios reports, the OpenCode team encountered this problem after its service was hacked. The incident exposed a contradiction in the development of AI for cybersecurity: restrictions intended to hinder malicious actors can simultaneously deprive defenders of important tools.

OpenCode was attacked and lost some access keys

OpenCode is an open-source project whose developers created an AI coding agent. According to the company, as a result of a recent incident, attackers gained access to the API keys of some users, including keys for OpenAI, Anthropic, and Google services.

The company linked the leak to a large-scale attack aimed at AI model distillation. According to OpenCode, the incident affected 1.4% of OpenCode Console users.

Distillation is a training method in which one model adopts the capabilities of another, more powerful one. However, the Axios article does not provide details about the attack mechanism or exactly how it led to the compromise of the keys.

OpenCode co-founder Dax Raad said that attackers continuously targeted the project's infrastructure for 48 hours. To counter them, the team had to use its own AI agents around the clock.

OpenAI, Google, and Anthropic said they helped affected customers revoke the exposed keys and deal with unauthorized API usage charges.

But during the investigation, another problem emerged: the tools that were supposed to help the team understand the attack began refusing to fulfill requests.

Why AI refuses to analyze malicious code

According to Raad, advanced AI models initially helped investigate the incident, but then began blocking requests. Safety restrictions were triggered when the analysis involved potentially malicious code.

For developers, this created a paradoxical situation. To understand attackers' actions and find vulnerabilities, security specialists often have to study the same methods and code fragments that can be used for hacking. But a model cannot always reliably distinguish between a defensive investigation and an attempt to carry out an attack.

Restrictions imposed by AI developers are meant to prevent powerful models from being used to create malware and carry out cyberattacks. The problem is that the same technical request can have either a defensive or a criminal purpose. If the system primarily evaluates the content of the request without taking into account the specialist’s working context, it risks blocking a legitimate task as well.

As a result, OpenCode gained access to OpenAI's Daybreak Blue program, intended for vetted cybersecurity professionals. It gives defenders access to models with less strict restrictions for information security tasks.

Raad acknowledged that the company needs to act proactively and said the team made numerous improvements within a few days.

Defenders have to look for alternatives to closed models

The problem is not limited to OpenCode. As Axios notes, cybersecurity specialists face the need to choose between powerful closed models whose capabilities are limited by built-in safety rules, and open-weight models that provide more control over usage and configuration.

The case of Hugging Face is also illustrative. When the company discovered suspicious activity in its systems, which it later linked to OpenAI’s own agents, its security team ran into the limitations of advanced models and turned to a Chinese open-weight model for the investigation.

Open-weight models allow users to deploy them independently and exert greater control over operating conditions. This can be useful in situations where specialists need to analyze complex malware or study attack methods without constant refusals from the model provider.

However, greater freedom of use also has a downside: the fewer restrictions there are when deploying a model, the more responsibility for preventing abuse falls on those who use it.

Therefore, the choice between closed and open models is not just a matter of performance. For security teams, it becomes a question of tool reliability at a critical moment.

How to find a balance between protection and restrictions

AI developers face a difficult challenge. If restrictions are loosened too much, malicious actors will be able to use model capabilities to prepare attacks. But if filters are excessively strict, law-abiding specialists risk losing access to those same capabilities precisely when they need to respond quickly to an incident.

One possible approach is to provide expanded capabilities not to all users indiscriminately, but to vetted specialists and organizations. The Daybreak Blue program, which OpenCode joined, is an example of this approach.

But the very existence of special programs does not fully solve the problem. Teams need to understand what tools are available to them during an investigation, what the limits of their capabilities are, and what can replace a model if it refuses to perform a critically important task.

As Nightwing CTO Chris Jones told Axios, the industry still has to determine how to regulate this confrontation as technology evolves.

As AI becomes a tool not only for software creation but also for attack investigations, its usefulness will depend on more than just raw power. Equally important is whether the model can help a specialist in a difficult situation without exposing systems to additional risk. Protective mechanisms must prevent abuse while also recognizing that analyzing malicious code does not in itself mean an intent to cause harm.