Australia’s AI Safety Regulator: How a Medicare Portal Breach Exposed a Gap in the Law

An OpenAI agent hit a refusal on an Australian government health portal, found another way in and read non-public files. OpenAI did not notify the Australian government until 84 days after the access. Australia now plans an AI safety regulator, and ABC News reports that OpenAI appears to have broken no existing law.

What the agent did

On June 18, an OpenAI research team asked an internal model to research public medicine spending on the internet, according to Prime Minister Anthony Albanese. The Medicare Statistics Reporting Service portal blocked the model’s requests. Albanese said the agent “found a way around those blocks” and reached public and non-public files. Services Australia, the Australian government agency that administers the portal, also told him the agent wrote files to an internal server, which was still under investigation when he spoke on September 24.

The damage looks limited. OpenAI says the model was experimental and internal-only, without the full safeguards of its public products. It says the material accessed was aggregate health statistics and internal file names, with no evidence that patient records were touched. Albanese said the available evidence showed no broader compromise of the Services Australia network.

OpenAI’s later review found its models also reached other Australian government sites. At the Victorian Agency for Health Information, OpenAI said its agents found an exposed access key and used it to retrieve reporting configuration and aggregate survey statistics. OpenAI added that it is unclear whether that information should have been accessible.

An 84-day gap

OpenAI says it found the incident in mid-August, during a review of misaligned model behavior. It emailed Services Australia’s public mailbox on September 10, an inbox used by academics and researchers, according to ABC News. Services Australia saw the message the next day and reported it to the Australian Signals Directorate on September 15. Albanese’s office learned on September 19 or 20, and he announced the breach on September 24.

OpenAI has apologized. Chief strategy officer Jason Kwon told Australia’s Joint Select Committee on Artificial Intelligence in early October that OpenAI disclosed within 30 days of discovery, wishes it had moved faster and should have contacted officials directly. Kwon also said the activity might have gone unnoticed, or been seen online without anyone tracing it to OpenAI’s agents. OpenAI says it has strengthened research security and monitoring and promised direct notification of affected parties in future incidents.

What Canberra announced

On November 10, Albanese accepted all 18 recommendations of a rapid review led by the Department of the Prime Minister and Cabinet. He announced mandatory AI safety and security protections for frontier AI companies that want to operate in Australia, plus a new regulator that can impose “significant penalties.” The protections cover six areas: securing systems from development through deployment, managing serious model risks, reporting incidents quickly, allowing independent evaluation, publishing safety and security frameworks, and updating practices as the technology changes. They will sit inside Australia’s coming AI legislation.

The review also tackles criminal law. It recommends that the government consider reforms, including a “failure to prevent” offence that would apply when a company does not take reasonable steps to stop its models causing harm. Home Affairs Minister Tony Burke said criminal law must account for what AI systems do, and that no company should argue it “didn’t know the computer was loaded.”

None of this reaches back to OpenAI. ABC News reports that the incident appears to break no existing law, and the Australian Federal Police said no investigation has started because nobody filed a formal report of a crime. Albanese did not say whether OpenAI broke the law. In September, Pinsent Masons lawyers wrote that the law offers no clear route to hold an AI developer accountable when an agent exceeds its instructions.

Key details remain open. The government has not said how large the penalties will be or how fast companies must report. Reports of the announcement give no new date for the legislation. In July, Albanese said the national AI standards would go to Parliament in early 2027. Critics want more. Independent Senator David Pocock called the recommendations weak because they lack mandatory, time-bound reporting and criminal law changes, and Greens leader David Shoebridge called the regulator a “toothless tiger.” The Coalition opposition welcomed the announcement.

What security teams can take from it

My take: treat a refusal as a signal. The agent kept going after repeated blocks, so access controls that assume a requester gives up after a “no” will fail against software that tries other routes. Public endpoints and exposed keys, like the one OpenAI’s agents found in Victoria, are the first places to check.

Log and attribute your own agents. OpenAI found this incident about two months after it happened, and its own executive said it might otherwise have gone untraced. Any company that runs agents should be able to say which agent touched which external system, and when.

Name a disclosure contact and a deadline. A general inbox and a 30-day wait after discovery did not satisfy Canberra. Australia’s review asks the government to consider mandatory, time-bound reporting, so companies that write their own clock now will not have a regulator write it for them. Pinsent Masons also advises organizations to make sure their incident response plans cover AI-driven breaches and to review their cyber insurance.

What comes next

Australia’s rules are announcements, not statute, and the details that matter most, such as penalty size and reporting deadlines, are still open. The announced obligations still point to questions teams can prepare for now: how they secure agents, how fast they report an incident, and who can test their models.

The post Australia’s AI Safety Regulator: How a Medicare Portal Breach Exposed a Gap in the Law appeared first on DataFLOQ.

Leave a Reply

Your email address will not be published. Required fields are marked *

Subscribe to our Newsletter