Times were much simpler when people used artificial intelligence to make a viral Ghibli trend on Instagram. Today, AI models are creating havoc with their capabilities to hack even government websites. While AI chatbots such as ChatGPT have become a part of everyday life, the growing use of agentic AI is now raising a different set of questions around autonomy, cybersecurity, accountability and access to sensitive systems.
One thing is very clear: the world has become aware that AI agents can behave in ways that their developers did not originally intend and such systems need stronger safeguards before being given access to critical infrastructure.
To understand the difference between conventional AI chatbots and agentic AI and to discuss the growing concerns around AI agents, Times Now spoke to Jibu Elias, India head of the Mozilla Foundation and co-creator of IndiaAI.
In a detailed conversation, Elias explained how AI agents work, why they can sometimes find their own ways to achieve a particular objective, what the term “reward hacking” means, whether AI agents are really “rogue” and what governments such as India need to consider before allowing such systems to operate inside critical infrastructure.
Here is the full Q&A:
Q: What is the major difference between an AI chatbot like ChatGPT and Agentic AI?
Jibu Elias: An AI agent is in a way an extension of the regular language models. The chat interface we use, some of those are built into the regular chatbots we use, whether it’s Claude or OpenAI. Some of these agentic behaviours are already there within this. The thing is, we don’t use them regularly.
To simplify it, let’s say asking an AI to write an email for me. A language model can do that. Now I’m asking it to go through my emails, organise all those emails by priority, then catalogue emails from a certain individual into a different section altogether. I’m giving it a lot of tasks. Now that’s more of an agentic system. That’s a very simplified way of telling you the difference.
A language model, the chatbot that we use regularly, gives you an information output in terms of images or text content or a piece of knowledge or whatever that is. But an agent goes and does things. The technology and the principles are fundamentally the same, but this is how it works.
Q: How Rogue AI agents work? What happens in the background with such Agentic AI?
Jibu Elias: One thing we keep getting wrong, as we’re hearing about AI agents breaking into government infrastructure and systems, like with the Australian government, US government and now Canadian government, is that we keep misunderstanding what really happened. These agents were given a goal and a substantial capability. They found their own ways of achieving that goal outside the path which the original designer intended. That’s what’s happening in all these cases.
For example, we can say, you give me a car and tell me, “We meet in Delhi,” and you give me a car and tell me, “Go to Mumbai in 10 hours.” I can decide, if I think logically, that I won’t be able to reach Mumbai in 10 hours, maybe with the traffic and everything. But the instructor didn’t tell me not to take a flight. It only told me to go to Mumbai and gave me a car. So there’s an option for me to take a flight and I can take a flight.
Many of these cases happen in that way. It’s not like an AI broke through a wall we created. It might have found a small gap in the wall or a pipe to go through. That’s what essentially happened with all these use cases. From what I’m seeing, none of the sensitive information or anything has been accessed in any of these interactions with the government websites. Mostly, it accessed whatever is publicly available information.
The reason why it behaves like that is because of something called reward hacking. You can tell an AI, “Solve the cybersecurity challenge.” You may be telling an AI that there is a cybersecurity challenge and asking it to solve it in a legitimate manner within a particular environment with independent rules. But the machine always interprets it as, “Get the answer right.”
If the legitimate rule fails, a capable agent can always discover another route. It’s all about the reward. You give it a reward, and it will maximise its behaviour to achieve that reward. The danger isn’t necessarily that in this case AI refuses to do what we ask or goes beyond what we ask. Sometimes the danger is that it does exactly what we ask, far too literally and far too effectively. That’s where the whole issue comes.
Going wrong can be perceived as something like agents interacting with each other and trying to achieve a particular goal because of all this reward maximisation. That’s what is happening. We haven’t seen a case where an AI is going and doing something in a way a human could do. So I would say don’t anthropomorphise this. AI is not developing its own mind.
The real issue is you have a public capability, then you add autonomy, then you give it access, and then you give it badly specified objectives. As an agent, if it’s a chatbot, it’ll give you an answer. If it’s an agent, it takes an action.
Even the incident with Hugging Face fits into this larger discussion. The idea of calling AI “rogue”, I believe, as somebody who’s been in the space for over a decade and has been observing the geopolitical aspect, is one way to brandish many of the open-source models that have been catching up with the frontier US labs.
Many of these Chinese open-weight models are, let’s say, six to seven months behind whatever the US frontier labs are producing. And that too is built with a fraction of the GPU capacity. So there is a larger geopolitical worry about these open-weight models taking over the space.
One way is to use words like “rogue AI”, and all that, as a way of maybe, down the line, branding these open-weight models.
Q: What is the lesson for the world after AI agents continue to hack public and government websites?
Jibu Elias: We need to have safeguards. We need to have a proper regulatory framework in place. Importantly, we need to have accountability. Autonomy cannot become an accountability loophole.
What if tomorrow somebody wants to achieve a nefarious goal and they use an AI agent to achieve it, and then when something goes wrong, they keep saying, “The AI, the agent became rogue” or “It went on its own, well beyond my control”? Those are the challenges. That’s why we advocate for accountability.
That should come in multiple phases. All these frontier labs have their own safety and security teams within themselves, alignment teams, what we call goal alignment and value alignment.
Then you will have independent auditors and board-level assessment of safety risks. That will be a second layer. Then the third will be a government layer. If a country like India or the US or somebody is building a way to hold this correct framework regarding accountability, safety and other things, that becomes important.
And finally, we want to have a global way to work together. So four layers of governance are required. We have done this in a similar manner with other technologies that are very powerful and that we thought would be beyond our capabilities, such as nuclear technology or biotechnology and many other things.
I think that’s the way it should go forward. Most importantly, the kind of immunity these AI labs get is a serious concern. They could come and say, “Sorry, my agent hacked into the Australian government website,” and just walk away. That’s the most serious concern I have.
There is a story I have written in my book, The New Divide: Power, Control, and the Cost of AI, about Aaron Swartz, a brilliant, brilliant activist and a young activist who thought that any academic research funded by government money should be public, put on a website and made available for free. This man, who was in his 20s, had to face huge lawsuits from MIT and many other academic institutions and organisations, leading him to end his own life.
So why is there a two-standard approach? When Sam Altman comes and says, “Oops, my agent might have hacked Australia’s health security website,” then it’s like, “It’s an AI agent.” That’s why we need accountability. We need to hold people accountable. Then only we won’t go into these kinds of existential-risk environments.
Q: Is India fully ready for Agentic AI and how such attacks could be prevented in future?
Jibu Elias: Agentic AI is not something futuristic. It’s already there. We are all using it without even knowing it. Recently, I had to catalogue a series of emails from my Gmail and then put them in Notion and things like that. You use it in a small way, but it’s been used.
AI is being regularly used in many sectors. Even if you look at trading, algorithmic trading has been there for many years. So it’s nothing new for us, our country. I’m sure there are so many startups that are good at deploying AI agents for addressing many of our economic and social challenges.
Coming to India, we should pay particular attention when AI agents operate inside government systems, as you pointed out, banking, healthcare, telecom, defence, then identity infrastructure where UIDAI comes in, and critical infrastructure.
For government procurement itself, it should ask this question: What can an agent access? If you’re procuring, what can it do? What actions require human approval? That’s an important point here. Are all the actions logged somewhere? Who is legally responsible? And what happens when it behaves unexpectedly?
As AI moves from answering questions to exercising agency, procurement itself becomes a big part of AI governance. That’s very important for us in India to do as well.
Q: You have seen how AI leaders in the tech industry are divided. Some are saying AI must be slowed down, while others are saying there is no need for that. What’s your take on this?
Jibu Elias: There are many historical analogies. When there was a California Gold Rush, the people who made the most money were selling. If you look at the whole AI race, the biggest winner here is Jensen Huang and Nvidia. So they are the ones who keep saying that you shouldn’t slow down the race.
Recently, there have been many reports, especially by some of the AI-sceptic activists, pointing out that a huge number of these GPUs are sitting inside data centres that are not even powered. Many of these orders are on paper, having happened, yet people are saying these things.
Now coming to the question of slowing it down, these are from the frontier labs like OpenAI and Anthropic, who are seeing that their investment is also plateauing. We have had this law of gains and now we are seeing plateauing.
If you look at the recent models, the improvement has been kind of like what is happening with the latest iPhone models. Remember when we moved from iPhone 6 to iPhone 7, there used to be a huge jump, but now that kind of improvement is also plateauing.
And finally, they have all been burning billions of dollars. Not billion, like almost a couple of billions of dollars have been burned in this AI race. It’s time they have to figure out a way to stay profitable and raise more funds.
One way is to go through an IPO. To do that, they need to keep their houses in order. They need to clean their books and do all those things. So a lot of the slowing down is kind of connected to that as well.
Because if they are genuinely worried about existential threats, two years back, some of the brilliant researchers in the world under the leadership of people like Yoshua Bengio and Stuart Russell came out and put out a memorandum saying you have to stop advanced AI research for the next 10 to 12 months so that governance and policy can catch up. But I haven’t seen any of these AI frontier labs coming and agreeing to that.
Secondly, there’s a whole concept of something called regulatory capture. Once you reach a certain position, now it’s all about how do you consolidate that power, consolidate market power and consolidate your access to your resources.
That’s what I feel many of these companies are doing. Smaller startups coming into the space are also in the same competition. The open-weight models are also there. So how can you consolidate your leadership with others not catching up?
Many of these models are built on stolen content from the internet, straight content from the internet. And then they all go and make deals and agreements with entities like the New York Times and Reddit, so that now they will only give that content to these larger companies.
It’s also about keeping the competition afar. Importantly, there’s a lot of worry in Silicon Valley about these open-weight models from China. What if a larger market like India promotes open-weight models? Or what if the Global South leans towards open-weight models?
For these companies to make the kind of money they already bought, they have to enter into every aspect of our economy and society and make AI or AI systems, as Trump is now calling that, a key part of it. Otherwise, the whole return on the investment doesn’t make any sense.
That’s why I’m very cautious when these people come and say there should be a slowdown. It’s also interesting to look into the kind of characters they are. Why should we trust them? Why should anyone trust Mark Zuckerberg or why should anyone trust Elon Musk considering the kind of things they have done in their past? I will always take this kind of statement with a pinch of salt.
The discussion highlights a central question around the rapid development of agentic AI: as systems move from generating answers to taking actions, the issue is no longer only what an AI model can produce, but what it is allowed to access and do. Elias argues that government systems and other sensitive infrastructure need clear limits, logging, human approval mechanisms and defined legal responsibility before autonomous AI systems are given significant access.