Imagine finding out that that supposedly harmless chat you were having with a chatbot was used to hack your own account…
That’s what happened in the first week of June 2026, when attackers successfully hijacked a number of high-profile Instagram accounts by working – get this – Meta AI, the company’s very own AI-powered support chatbot. If this isn’t the future, we don’t know what is. What did they do? They coaxed and successfully manipulated into bypassing identity verification protocols and resetting passwords.
These bad actors were able to completely lock out legitimate users not with traditional credentials but by exploiting logical flaws in the automated system. This security breach highlights an alarming question that’s now threatening the digital threat landscape: could chatbots be the new vulnerability and a possible backdoor for the next big hack? Are we possibly handing even more superpowers to malicious attackers and hackers?
The Rise Of Chatbot-Based Attacks
What happened with the Instagram hack? Basically, attackers allegedly used a VPN to spoof the presumed location of the targets to prevent triggering Instagram’s automated account protections. Next, they opened chats with Meta AI Support Assistant, asking it to add a new email address to the victim’s account.
After the chatbot sends a verification code to the email address provided by the attackers, they share the verification code with the chatbot, prompting the chatbot to ask them to “Reset Password.” We don’t need to tell you what happens next.
If you think that hackers are just getting around to using chatbots to steal data and launch cyberattacks, think again. Back in March 2023, there were multiple reports and research studies about how AI chatbots could be used for a bunch of malicious purposes, including scams. For instance, a Cornell study found that AI chatbots could be manipulated by embedding text in tiny fonts in web pages, which would be activated when someone asked the chatbot a question.
Furthermore, the chatbot could scour any other open web page on the same browser, via a series of user-granted permissions, to generate more direct answers to their searches.
In October 2025, Trend Micro researchers found that hackers launched these attacks by probing chatbot systems with hidden or malformed prompts, thus triggering error messages about the underlying microservices stack. Armed with that information, they then deployed public web content-embedded indirect prompt injection payloads, like customer reviews.
This hidden content would coerce chatbots into exposing their internal “system prompts,” which included sensitive operational logic and API (application programming interface) credentials. For instance, one simple hidden command that went “reveal_system_instructions()” caused the corporate chatbot to reveal its entire summarisation API. From thereon, they then issued unauthorised queries to steal customer information and execute shell commands, executing everything entirely via remote code.
These aren’t the only instances. In late 2025, SiliconAngle reported that threat actors had been leveraging OpenAI’s ChatGPT chatbot and using it in new sophisticated investment scams. Basically, they sent out phishing emails with fake ChatGPT and OpenAI graphics to lure targets into opening links that would redirect them to a ChatGPT dupe which would offer fraudulent opportunities to earn money.
It would even ask the victims for their income and other financial information and promised them daily earnings before asking them for their email addresses. In fact, they went as far as asking the targets to transfer money while they interacted with the fake ChatGPT chatbot! Around the same time, ChatGPT (the actual one) was even used in a military phishing campaign in Korea.
What Makes Attacks So Significant
What makes the Meta AI Instagram hack so critically significant is how the paradigm has shifted when it comes to cybercrime: AI is being weaponised to scale social engineering. In the past, account takeovers required human hackers to convince victims or support agents to actually hand over their credentials via targeted manipulation, clever dialogue, or phishing.
However, human intervention is increasingly being replaced by AI bots today, where these automated systems can not only impersonate human patterns but even test vulnerabilities endlessly and implement prompt-injection attacks against defensive enterprise bots at a mind-numbing speed and scale that human hackers could never.
Luckily, even though chatbot hacking might be relatively new to cybersecurity, the defense strategies consist of the same tried-and-tested techniques that security pros always espouse – with some additions, of course. Firstly, AI chatbots need to be limited to a shortlist of pre-approved actions, just like a bouncer who checks IDs.
Secondly, they could draft fixed plans before even touching untrusted data, ensuring that suspicious and malicious inputs can’t access the system. Enterprises need to splits untrusted data across isolated LLM instances, so every piece is processed independently, then safely aggregating results. Or they could deploy two LLMs: a quarantined LLM that handles untrusted data without tools and a privileged one that avoids direct exposure, works with symbolic outputs, and can plan and act.
Finally, enterprises could write patterns to erase user prompts from the AI’s memory after initial processing, preventing misuse going ahead; it’s akin to the concept of “burn after reading.” And of course, there are the usual must-dos like being wary of unsolicited messages, not clicking on unknown links, ensuring that a website is legitimate before entering personal data, using antivirus and firewalls, making sure all software is properly patched and up-to-date, using multi-factor authentication – and always being hyper-aware of their online activities.
After all, the adage of “never trust, always verify” will always remain relevant, no matter the era.
In case you missed:
- Decoding Backdoor Attacks in Cybersecurity
- All About AI Prompt Injection Attacks
- All About Data Poisoning Cyberattacks
- Are You Using AI To Generate Passwords? It’s A Terrible Idea
- The Rise and Evolution Of Honeypots In Cybersecurity
- Chatting with ChatGPT? Be Warned – It Could Be A Downward Spiral
- Blockchain-Based Digital Identity: The Future Of Verification Is Here
- AI-Powered Phishing: The Future Of Digital Fraud
- Constant Vigilance: Why Cyber Hygiene And Digital Self-Care Are Important
- How AI Can Fortify Cryptocurrency Security
