Thanks for the blog article, it was an interesting reading Sam 
This has been all over the internet, but is the security threat (“our models are too dangerous”) the real or main reason for not releasing it?
Some people claim it leans more toward a PR stunt, though not completely erasing the potential strength of the models. One example: On Anthropic's Mythos Preview and Project Glasswing - Schneier on Security
I certainly don’t know anything about all those complex topics, but I’m cautious when I read articles that spread lightning fast on all news sites and online communities. I assume there are some caveats on what is claimed. That there’s probably some truth and some other information that needs clarification, or is overhyped.
I don’t have any doubt over the fact that models are incredibly fast to find and probably exploit vulnerabilities, and you even highlighted this with the Discourse code example.
About the article itself, just pointing out something I felt weird reading:
Closed source has always been a weaker defense for SaaS than people want to admit. A web application is not something you ship once and keep hidden. Large parts of it are delivered straight into the user’s browser on every request: JavaScript, API contracts, client-side flows, validation logic, and feature behavior. Attackers can inspect all of that already, and AI makes that inspection dramatically cheaper. Closing the repository may hide some server-side implementation detail, but it does not make the system invisible. What it mostly does is reduce how many defenders can inspect the full picture.
Then, later:
Closed source can buy some obscurity, but obscurity is brittle. Code gets leaked, binaries get reverse engineered, APIs get mapped, and attackers learn a lot just by interrogating the running system. The real defense is not keeping the code hidden forever. It is building software and operational practices that hold up when scrutiny arrives.
When I read the 2nd paragraph, I had the feeling I already read that.
I scrolled up, and I noticed that the two paragraphs are very, very similar. They both state the same things, but using different phrasing.
I understand the need to summarize, but in this case, I really had the feeling I had read basically the same things a few paragraphs earlier.