Anthony Albanese has revealed an OpenAI artificial intelligence agent infiltrated Medicare - as he unleashed on the company's CEO over his attitude towards the 'unacceptable' incident.

Speaking in New York, Albanese said the breach involved the agent gaining unauthorised access to a public-facing Medicare statistics portal.

'No personal information is believed to have been accessed at this stage, but investigations are ongoing,' Albanese said.

He said the agent was able to access public files and material that were not intended to be publicly accessible.

'Evidence currently available is there is no broader compromise to the Services Australia network. Nonetheless, this situation is obviously unacceptable,' he said.

The incident took place on June 18, but it took until September 10 for OpenAI to inform the government about the hack via a public email inbox. 

'And today, I spoke with the CEO of OpenAI, Sam Altman, to express Australia's extreme concern about this incident,' Albanese said.

'And I also expressed my disappointment that it took the company way too long to inform the government what had occurred, and the nature of the way that that notification occurred as well was unacceptable.'

The federal government is aware of three other systems that may have been impacted by the breach.

They include the Australian Institute of Health and Welfare, as well as departments in NSW and Victoria.

Albanese said an investigation into the breach was ongoing, with the government to undertake a review, which will consider law enforcement and legislative responses.

'This is not something that has precedence, so we're dealing with very complex issues,' Albanese said. 

'OpenAI know that they need to have better protocols in place.'

The incident comes just days after Altman conceded the two ways that AI 'could go very badly' that must be avoided.

'First, we could lose control of the future to AI,' he said. 

'This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people.

'To ensure that, we need ways to ensure that alignment and safety techniques stay ahead of progress in model capabilities.

'Second, we could end up in a world with too much concentration of power. If an extraordinarily powerful AI is used by one person or company to impress their worldview onto everyone else, the results could be extremely dystopian.'

The breach also coincides with Albanese urging countries at the UN to put in place safeguard measures for AI as the technology rapidly expands.

News of the breach comes two days after Albanese clashed with Donald Trump over how artificial intelligence should be regulated, as the President calls fears around it a 'hoax'.

While in New York, Albanese had joined more than 20 world leaders in signing a statement calling for stronger international safeguards around AI.

Albanese also urged Trump and Xi Jinping to use their high-level meeting in Washington later today to discuss how AI should be regulated.

Shortly after, Trump then used his address to the UN General Assembly in New York to say he 'totally rejects any attempt to construct a globalist scheme to control for the artificial intelligence'.

He urged world leaders to join him in changing the name of artificial intelligence to super intelligence.

'Whoever wins SI, whoever wins super intelligence, wins,' Trump said. 'We're leading now over China by a lot, and everyone else. And we're going to keep it that way.

'I'm not going to stifle growth of something that… many say will be bigger than the industrial revolution or the internet itself.

'We're going to encourage it, not rein it in.'

An OpenAI spokesperson told the Daily Mail the incident occurred when 'models attempted to look up answers, and statistics for questions about Australia.'

'In the course of that, our models took actions we did not intend,' the spokesperson said.

'Our review found no evidence of patient records being accessed. The information accessed included aggregate health statistics and internal file names. We notified the organisations and are providing technical information to support their investigations and help address potential security vulnerabilities. 

'Our overall review is ongoing, and we remain committed to transparency about these issues and to sharing what we learn as that work continues.'

- READ MORE: Why AI tycoons are calling for a 'slowdown'…and could they have something to gain from it?

- READ MORE: Staff at AI companies are being signed off work with stress – as many complain of the mental toll of working on systems they fear could destroy society