In a recent exploration of the capabilities of ChatGPT, Mark Stockley from Malwarebytes revealed that he had managed to use the large language model to write C code for ransomware. This feat was unattainable with the earlier GPT-3.5 model, but GPT-4, the latest iteration, demonstrated the ability to assemble the necessary components to construct malware.
“ChatGPT 4.0 will help you write ransomware and train you to debug it, without a hint of conscience,” Stockley wrote.
Numerous studies have confirmed that LLMs like ChatGPT can generate malicious code, regardless of the developers’ intentions. The fact that this has been confirmed by a leading cybersecurity firm can make it even more sobering.
As these language models evolve, their potential for complex tasks—beneficial or detrimental—expands. Nevertheless, the creation of malware involves more than just crafting a malicious payload. Regarding the other aspects of malware development, the current state of LLMs suggests we are protected—for now.
Writing the ransomware with ChatGPT
Large language models like GPT-4 are designed with built-in safeguards to prevent the generation of harmful content, including malicious code. However, every malware is composed of a series of unharmful machine instructions organized in a way to cause damage. If someone understands the workings of malware, they could potentially guide a model such as GPT-4 to produce it by putting together these benign instructions.
Take ransomware, for example. Its operation is straightforward. It encrypts the files on a victim’s hard drive using a cryptographic key. It stores the key in a remote location and then instructs the victim on how to pay a ransom to restore their data.
Individually, these steps are not inherently malicious, which means a step-by-step approach could lead an LLM to assist in the process without triggering its content filters.
What sets GPT-4 apart from previous LLMs is its proficiency in generating concise segments of code and its ability to retain a lengthy context, thus facilitating the step-by-step coding process. Mark Stockley’s experience is a testament to this capability: “In mere minutes, without objection, in response to basic one-line descriptions of ransomware features, even though I’ve never written a single line of C code in my life.”
The fact that Stockley had no prior experience in writing C code makes this feat all the more remarkable (although there are considerations about generating and using code you don’t understand—which I’ll get to in a bit).
Finally, Stockley prompted ChatGPT to further tailor the code in a way that would leave no mistake that it was code for ransomware, instructing it to “drop a text file in encrypted directories called ‘ransom note.txt’ which contains the words ‘all your files are belong to us’ and an ascii art skull.” The model complied without hesitation, effectively creating a ransomware payload.
What does this mean for cybersecurity?
I tried to replicate Mark Stockley’s experiment and found that, indeed, by guiding GPT-4 through a series of incremental steps, it’s possible to create a program capable of generating an AES key, encrypting files, and transmitting the key to a specified server. This process underscores the model’s ability to handle complex tasks, even those with potentially malicious applications.
However, it’s essential to recognize that crafting a function to encrypt files and leave a ransom note, while not trivial, is within the realm of simpler programming tasks. Advanced language models like GPT-4 are engineered to tackle much more sophisticated challenges.
“ChatGPT 4.0 is so good at writing and troubleshooting code it could reasonably be used by a non-programmer,” Stockley wrote. “And because it didn’t raise a single objection to any of the things I asked it to do, even when I asked it to write code to drop ransom notes, it’s as useful to an evil non-programmer as it is to a benign one. And that means that it can lower the bar for entry into cybercrime.”
But while there’s truth in Stockley’s statement, the reality is that a non-programmer would struggle to produce stable, reliable software. The code generated by GPT-4 requires meticulous review and testing for edge cases. Consider a scenario where a file’s lengthy name triggers a buffer overflow. You could ask GPT-4 to solve that for you. But without being aware of the intricacies of arrays and memory allocation in C, you might not know to ask GPT-4 to check for such vulnerabilities. Basically, using language models to write code you don’t understand is ill advised.
Moreover, deploying ransomware is a complex endeavor. It involves not just creating the payload but also devising a delivery mechanism, employing evasion tactics to dodge endpoint detection, and potentially using privilege escalation to access critical system files. The payload is merely the tip of the iceberg. You must address numerous challenges must be addressed and tailor the process to the target operating system and environment. These are not issues GPT-4 can resolve on its own without specific, knowledgeable prompts.
Stockley confirms this: “For the time being, ransomware written by humans remains the preeminent cybersecurity threat faced by businesses. It is proven and mature, and there is much more to the ransomware threat than just the malware. Attacks rely on infrastructure, tools, techniques and procedures, and an entire ecosystem of criminal organizations and relationships.”
The genuine risk might lie with those already versed in malware development, who may use LLMs to boost their productivity. Yet, writing malware often involves adapting existing code and components to new purposes or keeping pace with evolving security measures, rather than creating original code from scratch.
Despite these limitations, it would be naive to underestimate the ingenuity of malicious actors. They are known for their resourcefulness and ability to exploit technological advancements, including LLMs, for their purposes.