Social Development Ministry chief executive Brendan Boyle has admitted they did not act on earlier warnings, after blogger Keith Ng revealed he was able to access its servers through public kiosks in a Work and Income office.
Consultancy firm Deloitte will conduct an independent investigation into the breach.
Yesterday it was revealed the ministry commissioned a $10,000 report from Dimension Data which highlighted problems with the kiosks in mid-2011.
Mr Boyle said he was not confident the report's recommendations had been adequately followed up. "We will be asking Deloitte to determine what we did to follow up this report's recommendations and whether our response was adequate."
The matter was also raised by a beneficiary advocate about a year ago.
State Services Commissioner Iain Rennie has asked Government Chief Information Officer Colin McDonald to undertake an urgent review of publicly accessible systems operated by the state sector.
Social Development Minister Paula Bennett said yesterday security of computer systems was an operational matter despite having given earlier assurances that she would monitor it.
In Parliament yesterday, Mrs Bennett said security of the system was an "operational matter".
However, in a letter to Finance Minister Bill English in February 2009, Mrs Bennett said she would be monitoring progress of increased use of frontline information services.
"Whilst the ministry has a strong focus on reducing its national office numbers there are also substantial plans to automate frontline services for clients through the use of online and other IT solutions," she wrote.
Mrs Bennett said the breach was not acceptable and was taken very seriously. "I am extremely disappointed."
Labour MP Jacinda Ardern said the breach "points to a cavalier approach to privacy and to the protection of information by this Government and the buck has to stop somewhere."
MSD originally said the first it knew of the breach was when Ira Bailey contacted it recently.
Mr Bailey said he did not tell MSD the details of what he had found because he thought the story needed to be told.
He checked on two kiosks before ringing MSD.
After being put through to an answer machine and calling back again to leave a message, it took several days to get a return call.
But the breach was "mental" and gave access to everything within MSD's system. "I couldn't believe it, my brain imploded really."
He said it took only two or three minutes to access the information, which made him worry about other government systems. Mr Bailey took screen shots on his camera but did not take any files.
Green Party co-leader Metiria Turei said Mrs Bennett should take responsibility for "creating the cavalier attitude to privacy now evident within her office and her ministry".
- The Dominion Post