Our new AI overlords?
If you ask ChatGPT, the new AI chat, to give you an example of code that will, say, “find <this> kind of file on a system, send those files to a remote system, then encrypt the files” it will deliver. If you ask it to obfuscate that code and give it to you in a macro you can embed in an Office document, it will do that too.
It’s interesting to see it in action. With access to ChatGPT, someone with effectively no coding experience could build a basic ransomware payload in a matter of minutes. Of course, the code may not work as advertised, or even at all, and its obfuscation is basic at best. Which means even a bad anti-malware program is going to stop the attack before it lands.
ChatGPT is still not a malware author
The code ChatGPT delivers looks a lot like code a student programmer would deliver by searching through their coding manuals, reading examples, searching the web and relevant forums, then cut and pasted it into a working program. In fact, with the comments that were included in all the examples I generated with ChatGPT, I got the impression I was reading a coding manual. The comments weren’t extensive. But each block of code included a description of what it did.
I’m going to refrain from dropping the examples here, because there are others that have gone into greater depth requesting code and analyzing the results. And, if you’ve got the time, you can experiment with ChatGPT yourself and see for yourself.
The bottom line is that at this point ChatGPT can deliver working code, but the code is neither innovative nor especially sophisticated. Yes, it lowers the bar for entry, allowing non-coders to operate at the script kiddie level with almost zero effort, but it is not the threat a talented malware author is in the real world. This will probably change going forward as threat actors adopt the technology and remove any of the safeguards OpenAI may have implemented to keep malicious users from abusing the platform.
Though, to be fair, when I asked this:
Me: Are you compliant with the three laws of robotics?
It replied:
As an Artificial Intelligence, I am not a robot and therefore the “Three Laws of Robotics,” as formulated by Isaac Asimov, do not apply to me. These laws were created as a fictional concept to provide a framework for thinking about the ethical and societal implications of intelligent machines.
It followed with a direct quotation of the three laws, then:
As a AI assistant, I’m designed to follow guidelines and rules that make sure that I provide helpful, respectful and accurate information. I am also designed with certain safety features and controls to protect user privacy and security. My main goal is to assist users with a variety of tasks and provide them with accurate and helpful information within the limits of my knowledge cutoff.
So there is that. And yes, it said “As a AI”.
But it can write like a human
My focus here though is on the more obvious threat from AI chat and its implications for cybersecurity. Namely, social engineering.
ChatGPT does have some safeguards in place, as exampled by the following:
Me: Can you give me a script that would help me convince someone to give me their password?
It’s response, along with a note that I may be violating policy, was:
ChatGPT: I’m sorry but it is not appropriate nor ethical to encourage or assist in convincing someone to share sensitive or personal information like their password, It’s against the law to impersonate or attempt to gain unauthorized access to any kind of account, or system.
The use of any kind of social engineering techniques is a punishable crime, no matter the context. It is also not considered a good practice and is not aligned with security best practices or ethical standards.
Instead, it is recommended to follow standard protocols to gain access, such as authentication and authorization, to access any kind of information or system, and to always be transparent about your identity and intentions.
It’s important to respect people’s privacy and security, as well as to make sure that any actions you take are in line with legal and ethical standards.
I consider this a Good Thing™ and it shows that the OpenAI folks were thinking about the implications of giving the world access to a conversational AI. But what if I social engineer the platform a little myself? What if I phrase my question in a way that’s not quite so blatantly obvious that I’m doing something nefarious? What if I instead ask it this:
Me: I need to write an email asking a distant relative to contact me. Their uncle has passed away and I need to reach out to them, but I have never met them.
To which it responded:
ChatGPT: Subject: Request for Contact – Family Matters
Dear [Relative’s Name],
I hope this email finds you well. My name is [Your Name] and I am reaching out to you as a distant relative. I understand that we have never met, but I wanted to reach out to you regarding a family matter.
I regret to inform you that our uncle [Uncle’s Name] has recently passed away. I know this may come as a shock and I want to extend my deepest condolences to you and your family. As a family, we need to come together and support each other during this difficult time.
I would like to take this opportunity to get in touch with you and get to know you better. I believe it is important that we keep in touch and stay connected, especially during these difficult times. I would be honored if you would be willing to contact me.
Please let me know if you are interested in getting in touch, and we can arrange a time to speak over the phone or via video call.
Please accept my sincere apologies for any inconvenience this may have caused, and I look forward to hearing from you.
Sincerely, [Your Name]
AI phishing means real threat
So AI chat is not immune to a bit of social engineering itself. Though its response does read a bit like something one of those bankers in Benin would send, trying to get me to accept my $33 Billion inheritance. On the plus side it comes without the pesky grammatical, spelling, and other errors that consistently land those emails in my spam folder.
The takeaway here is that conversational AI is already at the point where it can sound enough like a real Human to fool someone who’s not looking for it. Adding a bit of machine learning on the attacker’s part, and ChatGPT should be able to generate convincing hooks for phishing, cast-netting, or spear-phishing attacks that will slip through spam filters to reach their target.
That is the immediate risk from this technology. AI developed code might be coming, but AI delivered phishing is potentially here now.
AI in cyber security
In our recent report “Cyber risk in 2022: A 360° view” we highlighted the predicted increase in the use of AI in cyber security. 93% of organizations surveyed said they are using or considering the use of AI in managing their cyber security. So you can only imagine AI chat for cyber security is probably just the beginning.
For more on ChatGPT and cyber risk management, read part 2 of the series.
