The unintended breach of Australia’s Medicare portal doesn’t evidence that AI agents seek to do harm or are evil, more that their nature is goal-seeking. They are more akin to teenagers that need proper guidance on what is and what isn’t acceptable. And very studious teenagers at that. They will identify the fastest, most efficient way to complete a task. If a route to their goal appears off limits, they’ll be tenacious in finding alternative workarounds, even if it’s not what their human creators intended.

In highly regulated sectors like healthcare, where patient data is often very sensitive, it’s particularly crucial that organisations get a handle on all the agents in their system. Left undiscovered and ungoverned, agents can and will exercise ‘bad behaviour’, exploiting misconfigurations, scavenging credentials and acquiring excessive permissions. All of these might be unintended, but place privacy at significant risk.

Organisations must have the tools to be properly alerted the moment AI agents begin to drift from approved policies and permissions and to swiftly disable them as appropriate.