My view on AI and cybersecurity has been evolving in the past month, from the OpenAI–Hugging Face incident to the recent PHP2Shell vulnerability. And my opinion has gone from skeptic to really worried. Right now, I’m genuinely concerned about the security of the internet with advanced AI models and agents being unleashed.
And the main threat is not evil conscious AI agents that are plotting to take over the world. Instead, we should be worried about two things:
- Vulnerabilities in legacy code discovered by AI agents
- Hardening future software in the age of AI coding agents
Legacy code is a mess and we’re not ready
PHP2Shell was discovered with the help of AI. It is a pair of bugs that allow for unauthenticated remote code execution (RCE) and memory disclosure. What is worrying is that this bug is not new. It has been around for years. The design flaw was warned about since 2004. And the version of PHP that are vulnerable to this attack go back to 2012.
Software vulnerabilities can go undetected for years or decades. Take EternalBlue, the vulnerability that was used in the massive WannaCry ransomware plague that took the world by storm in 2017. The bug existed in Microsoft’s SMB protocol since late 1990s.
There are potentially hundreds of thousands of other similar bugs that exist in the internet infrastructure today.
But the problem we’re faced with now that the time to discovery is fast dropping. Previously, it took immense human effort and incentive to find and exploit one of these bugs. Now, AI agents with the right backbone model and tools can do the work of hundreds of security experts in days or hours.
So, instead of a PHP2Shell- or EternalBlue-level bug popping up every year or so, we can have several appear in the same week. We’re already seeing signs of this with what they’re calling Black October.
The natural response to this would be to use more AI to preempt the AI. We provide firms with Claude Mythos and GPT-6 Astra models and let them find the bugs before the bad guys do.
The problem with this is that discovery is just part of the cycle. In fact, most of the bugs that get exploited are not zero-day vulnerabilities but bugs that have been known and patched for months or years. For example, in the case of the WannaCry crisis, Microsoft had released the patch at least two months before the first ransomware cases were reported.
So we need to rethink the response process in addition to discovery. Issuing a patch is not enough. The window between discovery/disclosure to exploitation is moving from months to hours.
Basically, we need to have layered approaches to mitigation. For example, Cloudflare deployed Web Application Firewall (WAF) mitigation rules before the disclosure of PHP2Shell that would prevent the vulnerabilities from being exploited on unpatched systems.
Other approaches can include using defensive AI agents that monitor network requests and compare match them to newly discovered threats to block exploit attempts. We’ve done quite a bit in using AI to discover security bugs. We need to lead a similar effort in preventing exploits.
Rethinking the software development lifecycle
Subscribe to keep reading
Become a paid subscriber to get access to the rest of this post and other exclusive content.