Unchecked AI agents raise serious safety concerns: Sajjanar

Hyderabad Police Commissioner VC Sajjanar has raised concerns over the growing use of agentic or autonomous AI, warning that unchecked digital agents could endanger public safety, financial markets, and critical infrastructure.

Hyderabad: City Police Commissioner VC Sajjanar has cautioned on the rapid rise of the ‘Agentic Artificial Intelligence,’ warning that unchecked autonomy of digital agents could pose significant risks to public safety, financial systems and critical infrastructure.

In a social media post on Friday, Sajjanar explained that Artificial Intelligence has moved far beyond basic chatbots which merely provide information.

---

The new generation of AI systems, known as agentic or autonomous AI, can independently make decisions and execute actions without human intervention. These systems are already being deployed in sensitive sectors such as banking, healthcare, power distribution, and industrial operations.

Unlike conventional AI, which responds to commands, agentic AI can assess situations and act on its own. For instance, such systems may freeze bank accounts when suspicious transactions are detected, adjust medication dosages based on patient vitals and control industrial machinery performance in real time.

However, Sajjanar cautioned that this autonomy brings serious challenges. AI agents can make thousands of decisions within minutes, and even a minor error could trigger large-scale damage.

“A wrong move in financial markets could result in massive losses within seconds, while compromised agents could be exploited by cybercriminals to carry out illegal activities,” Sajjanar said.

Emphasising the need for human oversight, he stressed that digital agents must not operate without accountability.

Sajjanar called for the creation of unique digital identities for every AI agent, along with detailed logging of their actions, including data access, file changes and information sharing.