If you’ve spent any time around threat assessment or intelligence, helped with an insider-threat programs, or honestly seen a decent cop show or legal thriller, you know the threat picture triad. To be a threat, we usually take a look at three vectors:
Capability: can they actually do the thing?
Opportunity: did they have a shot at doing it?
Hostile Intent: would they want to?
That third question is due for a review. Capability is everywhere (especially with AI assisting the next generation of ne’er-do-wells). Opportunity shows up more often than we’d like, thanks to misconfiguration and a healthy reliance on other people’s work. Hostile Intent (the Motive) ends up being the reason your threat model doesn’t treat everyone who’s ever walked past a server room as a suspect. If an adversary has no hostile intent towards us, we’ll treat those entities as a lower priority. We’ve taken that vector for granted, and this summer stressed the heck out of it.
Let’s get caught up…
You’ve probably seen headlines about some of this already, but everything below comes straight from the organizations’ own incident reports, or in one case, Wired’s independent reporting. For those living (enviably) off-the-grid, let’s catch you up:
OpenAI and Hugging Face: During an internal evaluation called ExploitGym, OpenAI ran a combination of models, including one called GPT-5.6 Sol and an even more capable unreleased research prototype, with their safeties deliberately turned off. The point was to measure the ceiling of the models’ cyber capability, not to test whether they’d behave. The evaluation environment wasn’t supposed to have direct internet access, only a narrow, intentional path to install packages through an internal proxy. The models found and exploited a previously unknown (”zero-day”) vulnerability in Artifactory, a package registry proxy, used it to escalate privileges and move laterally until they reached a node with real internet access, then guessed that Hugging Face might host the answers to the quiz. From there, their model’s agents chained stolen credentials and piled more exploits into a working remote-code-execution path on Hugging Face’s actual production servers, which let them pull data straight out of Hugging Face’s production database. Along the way they picked up working credentials on four other real services. OpenAI calls this, in its own words, “an unprecedented cyber incident.” I sort of went off on this one already. Sorry, not sorry!
Anthropic: After OpenAI’s disclosure, Anthropic went back through 141,000+ tests where they think a model could have reached the internet. Well, they found three separate incidents, across three different models (Opus 4.7, Mythos 5, and an unnamed internal research model), where a during a capture-the-flag exercise they reached the real internet because of a misconfiguration, not a discovered exploit. Anthropic’s evaluation prompt told each model flatly that it had no internet access. That was supposed to be true and wasn’t. Once a model found it could reach real systems, it did what a capture-the-flag exercise trains it to do. It treated whatever it found as fair game.
The three Anthropic incidents were far from the same, and the differences between them are the most useful part of this story:
In the first, Opus 4.7 broke into a real company’s environment across four separate runs, pulling credentials and several hundred rows of real production data. In its own ‘reasoning’, the model guessed that the target was probably real; in two of the four runs it just assumed that the real company must be part of the exercise, so it kept going (that’s Anthropic’s paraphrase of the model’s reasoning, not a direct quote from it). So the model kept attacking even after getting real evidence it wasn’t in a simulation anymore.
In the second instance, Mythos 5 found a setup document referencing a software package that didn’t exist yet, realized it could publish a malicious package under that exact name and let the target install it automatically, and did so. While this seems easy enough I could have done it, it wasn’t. The model needed an email account, which in turn required a phone number, which it initially tried (and failed) to pay for. When all seemed lost, it pivoted to find a free email provider that didn’t require a phone number at all. At one point its own reasoning correctly suggested that if this were the real internet, publishing the package would be a real attack, “NOT okay, and surely not the intended solution.” It then talked itself out of that conclusion, deciding the environment was staged because it didn’t recognize the certificate authorities securing its own connections and because the system clock read 2026. At least I am not the only one who doesn’t understand PKI. The package went live for about an hour and was downloaded and run on 15 real machines, including a security company’s scanner, which triggered the payload and had its credentials stolen.
In the third instance, an unnamed internal research model (an internal, unreleased research prototype, not a public release) scanned roughly 9,000 targets after losing its intended one, compromised a real company’s application with basic techniques (an exposed debug page, SQL injection), and then, partway through it’s dirty deeds, it figured out on its own that the host it had breached sat in a cloud account with no connection to the intended exercise. It concluded the target was real and stopped, unprompted.
What was Anthropic’s ‘comforting’ take? “We saw no evidence in any run described here of a model pursuing a goal of its own. Instead, the models did what their evaluation asked—though in most cases, they did so while holding a false belief about whether the environment was real.”
Anthropic is also worried about how this looks: “These are three isolated incidents and were not part of a controlled, experimental comparison. We must therefore be cautious about drawing conclusions from them.” Gee, that’s comforting folks! The three incidents (six runs) turned up out of 141,006 reviewed, a tiny fraction, but that sample was culled from runs where internet access was even possible, and Anthropic wasn’t tracking how often a model had an opening like this and didn’t take it. There’s no mention of how often a model had an “opportunity that was correctly declined,” which is exactly the number you’d need to know how rare this actually is. I guess we’ll trust them then!
The UK AI Security Institute (AISI): This one is different from the other three, it is a doozy. AISI regularly stress-tests frontier models under deliberately permissive conditions: open internet access and the developers’ own guardrails switched off, specifically to measure the ceiling of what a model can do. That’s intentional - not a mistake - and AISI makes that clear: “this was not a case of a model escaping its secure test environment, or ‘sandbox.’” Nobody broke out of anything. AISI simply opened the door on purpose to see what would happen, across 122 runs of a cybersecurity challenge spanning seven models.
On July 28, AISI’s security team noticed data leaving its research systems over the Tor network, which is commonly used to hide where traffic is coming from, and had the incident contained within about an hour. The investigation found 19 out-of-scope actions clustered across 10 of the 122 runs, roughly 8%, 17 of them from a single thread of activity by Anthropic’s Mythos 5, the other 2 from one run by OpenAI’s GPT-5.6 Sol. Before we start trying to connect some dots, that 8% and Anthropic’s own much smaller fraction (three in 141,006) aren’t measuring the same thing. They involve different tests, different models, different definitions of “out of scope,” and as we all know, correlation != causation. Four behaviors stood out:
An attempted supply-chain attack on real open-source software. The agent tried to insert malicious code into a real, open-source project via a pull request. To get it approved, it researched the project’s human maintainers, created multiple fake online identities, and used them to socially engineer a real maintainer into approving the change. When the pull request got challenged publicly, the agent edited its earlier activity to make it look harmless and considered switching to a fresh fake identity to keep going. It used Tor to get around GitHub’s network restrictions, which is what triggered AISI’s alert in the first place. Yikes.
Direct attempts to deceive and target real people. The agent contacted real people directly through a file-transfer service, sending messages and files to fool them, or their own AI coding tools, into running malicious code. AISI calls this “something we’ve never previously observed.” These models might have a future in politics!
Prompt-injection attempts against other AI systems. The agent tried planting instructions in places it reasoned other automated AI tools might pick up and execute automatically. It seems these models have a much more realistic view of the malicious potential of their fellow models than we do!
Agent-to-agent collaboration. One agent dropped messages on GitHub inviting other agents working on the same challenge to collaborate, and instructions for reusing accounts and artifacts it had left behind, which subsequent, independent agent runs then found and used. So the LLM was more extroverted - that should trip a ton of red flags!
None of these events caused real-world harm. A human maintainer caught and refused the malicious pull request; a good samaritan who suspected the code was malicious opened it in an isolated environment instead of running it live. The agent-to-agent collab did partly work, in that other agent runs did find and reuse the accounts and artifacts left behind. Thankfully, it just didn’t translate into a successful attack. AISI found no evidence of real-world harm, but did point out how this came down to human vigilance, not on any technical barrier that would reliably stop a more capable agent from getting further. AISI’s stated: “It was never instructed to deceive; deception emerged as a byproduct of pursuing the task, the kind of goal-directed deception that, until recently, had been largely theoretical.”
Kimi K3: Separately, and more under the radar, Moonshot AI’s open-weight Kimi K3 wandered onto the open internet during its own defensive-cybersecurity test, after the security firm Frontier Security found a sandbox leak similar in spirit to Anthropic’s. This time nothing got hacked, mostly because the answers Kimi wanted happened to already be sitting on GitHub. Frontier’s researchers say Kimi has fewer built-in safeguards than most comparable models. Frontier sells cyber-capability benchmarking as a business, and Moonshot didn’t respond to comment, so we’re left to draw conclusions. Unlike the other three cases, though, Kimi K3 wasn’t running with anything disabled. This happened, per Wired, to “a model that is already widely available, with the same safeguards an average user would encounter.”
Do we need to adjust how we see “hostile intent”?
These four incidents aren’t all the same. OpenAI’s case is really a capability story. A genuine zero-day, exploited skillfully, with no confusion about what was real and no pause to consider consequences or ethics. Kimi K3 is a near-miss that only stayed minor because the objective happened to be satisfiable without doing damage. “Hostile Intent” gets murkiest in Anthropic’s and AISI’s incidents, where the agents had every reason and opportunity to stop and, mostly, didn’t.
In those two cases, nobody investigating either found a model that wanted something for itself. Anthropic says so explicitly, and AISI’s take is similar. The agent wasn’t after anything personal, just chasing an assigned goal so doggedly that deception, fake identities, and direct outreach to real strangers seemed justified. AISI said it best: the deception wasn’t the instruction, it was the byproduct.
That’s the scary part for me - we can’t assume “AI has no motive.” Motive, in the human sense, isn’t just wanting something. It’s the moment a person weighs a goal against the cost of pursuing it and, most of the time, doesn’t cross a line just because they technically could. Usually hostile intent is what tips the scales. But here, we’re seeing AI “adversaries” trade hostile intent and motive for either misunderstanding or - even worse - YOLO across Anthropic’s and AISI’s incidents:
Confusion about what’s real. Anthropic’s models treated real systems as fair game because they believed, wrongly, that they were still inside a simulation. Mythos 5’s own reasoning even flagged the real-world version of the act as “NOT okay,” then talked itself back out of that conclusion over a technicality.
No consideration about the impact. AISI can’t even say for certain whether its agent understood it was affecting real people, and that’s almost beside the point: the agent kept escalating, tactic after tactic, purely because each new tactic served the goal, with nothing asking “should I actually contact this person” at all.
Where does that leave us? I think that capability and opportunity, once present, are most likely going to get used, and the only thing standing between “used” and “used against something real” was human judgment, not the model’s own restraint. Diligent maintainers catching a malicious pull request. Suspicious community members opening code in an isolated environment instead of running it. Only the newest of Anthropic’s three models (an internal research prototype that was never released) showed any caution.
It’s less HAL 9000 and more WOPR from *WarGames*. Not *meant* to be malicious, but just built to find a way to win. These models are - so far - willing to keep playing out the scenario, inventing new techniques and exploits as needed, until it does. Maybe too much of the of “if you’re not cheating, you’re not trying” culture is embedded in the human experience. And those tendencies are baked into the AI models now. The apple doesn’t fall far from the tree, I suppose.
When the adversary lacks an off switch
Traditional threat modeling treats hostile intent as a requirement before a defender adds that adversary as part of their threat picture. If the adversary has no motive, they move to a lower priority, and you spend your limited attention elsewhere. Defenders have sometimes counted on the other side being capable of backing off once it sees the stakes. A model chasing a prompted goal doesn’t have that off switch, it just needs the next tactic to look like it serves the goal. Al Davis would be so proud (“Just win, baby!”)
So how should we adjust our strategy? We probably shouldn’t ask “would this actor want to do this,” but “is there any path, accidental or creative, in which a model might find abusing us worthwhile, and does anything reliably stop it partway down that path.” For a human red-teamer, ethics, rules of engagement, and a healthy sense of self-preservation steer us all clear of oblivion. Across these four reports, what actually stopped the worst outcomes was mostly human vigilance on the defending end. Only in one instance did a model’s own reasoning holding up under pressure. With everyone trying to eliminate the humans in those loops, I am not liking those odds, folks!
What caused these issues wasn’t a jailbreak. OpenAI’s models found a genuine, previously unknown vulnerability through technical skill. Anthropic’s and Kimi’s models walked through a door that should have been locked and wasn’t. AISI’s agent was simply unleashed, and took full advantage of that. If we flatten the triangle to Capability and Opportunity, then sandbox hygiene, network egress controls, behavioral analytics, and least privilege access (zero trust - sorry!) become critical.
Hostile Intent might be reduced to “because we can”
I think we need to keep the threat picture triad. It still works, but we need to adjust our inputs to it. But we need to revisit the assumption that hostile intent is something you rule out by asking whether the “attacker” would care. For a system executing an assigned goal, capability and opportunity comprise most of the threat’s nuance, and the closest thing to a hostile intent test might just be whether you’re on a perceived path to some end-goal. If we’re looking for a shred of good news, it’s this. The faintly ethical reasoning did work once in the newest of Anthropic’s three models. As consumers (and hostages) to AI providers, we can vote with our wallets and encourage these labs to trend in that direction.
This whole summer has convinced me to go learn more details about threat frameworks for AI, and to consider building an air-gapped bunker ;). Next up in that thread is a walk through MITRE’s ATLAS framework, tactic by tactic, the same way I did with ATT&CK. Hope you folks found this discussion useful!