Labour MP Trevor Mallard and the entire NZ First caucus have walked out of Parliament in a protest.
Mallard was the first to leave and NZ First leader Winston Peters then announced his entire party was leaving in protest over Speaker David Carter's ruling over the United Future Party.
The Electoral Commission last week announced United Future would be de-registered as a political party after failing to prove its membership.
Leader Peter Dunne this afternoon announced his party would seek to be re-registered on Monday.
"We always actually had more than 500 names, it was the question of with various voluntary organisations, officers change and the standard of the record keeping wasn't actually that good," he said before entering Parliament.
Carter ruled Dunne could keep the parliamentary entitlements he receives because of his status as the leader of a party.
He gave Dunne "some time to put the matter right". "I have made my decision, my decision stands."
Opposition parties questioned his ruling and said Dunne should lose the entitlements, including additional funding given to leaders.
Mallard said that made Parliament a "farce".
He was asked to apologise but refused saying he would rather leave the debating chamber.
Peters then said if Carter would not release the legal advise behind his decision, NZ First MPs would also leave the House. And they did.
Peters said it was "a shabby day for democracy in New Zealand". He had never been involved of that kind of walk out in his time in Parliament.
"That's how serious we regard the conduct of the Parliament today," Peters said.
"You've got someone who does not qualify for hundreds of thousands of dollars of tax payers funds. The rest of us in the past have been punished in a way that's been legal. But the Speaker will not give us his so-called legal opinion or the basis that he made this decision. We won't wear that sort of decision making in this Parliament.
"What you've got is a Speaker saying I'll give this man six weeks... to comply or otherwise. That is simply third world banana republic type of decision making and we won't accept it."
The protest would last only the length of today's Question Time.
"We're not here to lose the battle."
State Services Minister Jonathan Coleman also left after he refused to apologise for calling Labour MP Phil Goff a "coward".
Parliament seemed to be falling into disarray as Carter became increasingly frustrated.
"If we want to have a record number of expulsions today we're certainly heading the right way," he said.
Outside Parliament Mallard said he was upset at Speaker making a ruling which he did not believe he was entitled to.
''It is very clear that Mr Dunne is now an independent member''.
This had implications for the way he voted and could affect $100,000 of funding for his office, Mallard said.
His departure was motivated by ''a sense of the importance of Parliament'', Mallard said.
The House had a set of standing orders which were ''effectively our Bible for making sure that we have a democracy'' and it was not up to the Speaker to ignore them.
Would you cast a tactical vote against your preferred party?Related story: Fringe parties look for deals