The Sevens All Blacks went unbeaten through pool play and will face defending champions Wales iin the Sevens World Cup quarterfinals tonight.
David Raikuna scored after time was up and Gillies Kaka converted as New Zealand defeated the USA 26-19 in their final Pool D match, coming back from 19-5 down early in the second half at Luzhniki Stadium.
Gordon Tietjens' men were staring at an unlikely loss, before a penalty try, then a Tim Mikkelson five-pointer 68 seconds later saw them draw level, before Raikuna's match-winner.
"We made a few mistakes and they pounced on them, and they made us pay for it," captain Tim Mikkelson said.
"They are a good team and we showed a lot of fight to come back."
Earlier, having comfortably accounted for Canada 31-12 in their tournament opener on day one, New Zealand romped away from Georgia 26-7.
Wales, winners of the last World Cup in 2009, topped Pool E, beating Uruguay 33-5, Tonga 28-19 and Fiji 19-14 in pool play.
The New Zealand-Wales quarterfinal will be played at 9.22pm tonight (NZ time).
In the other quarterfinals, South Africa face Fiji, England and Australia battle and Kenya plays France.
It was not a convincing victory for New Zealand over Georgia, despite the four-tries-to-one margin.
There were some sloppy moments and New Zealand allowed Georgia plenty of possession, relying on their strong defence to eventually provide the turnovers for tries.
Tomasi Cama and Lote Raikabula touched down to give New Zealand a 12-nil lead at halftime. Then Kurt Baker and Sherwin Stowers scored second half tries either side of Merab Kvirikashvili's well-constructed try for Georgia.
New Zealand have won only one World Cup in 2001 despite their annual dominance of the World Sevens Series where they have won that crown 11 times in 13 years.
South Africa did not concede a point in beating Scotland, Japan and Russia and will play Fiji, who along with France were one of the two best-placed group runners-up to progress to the quarter-finals.
France will take on Kenya, who beat Samoa 17-12 thanks to a late try to Michael Wanjala in their final pool match to decide who would top their group.
The final quarter-final will be between England and Australia.
England topped their group with wins over Portugal, Hong Kong and Argentina, while Australia also finished in first place despite drawing their final pool match with France 17-17, having held a 17-0 lead at one stage.
Terry Bouhraoua kicked a penalty drop-goal with less than two minutes remaining to give his side a share of the spoils.
"A bit disappointed to have the draw, we switched off a little bit in the second half," Australia's Luke Morahan said.
"We played pretty well in the first two games but just let ourselves down a little bit at the end. It is good to get through."
Arguably the biggest upsets were in Pool F, where Portugal beat Argentina 17-10 but then went down 17-14 to Hong Kong, having led 14-5 with two minutes remaining.
NEW ZEALAND RESULTS
New Zealand 26 (Penalty try, Sherwin Stowers, Tim Mikkelson, David Raikuna tries; Gillies Kaka 3 con) USA 19 (Nick Edwards 2, Matt Hawkins tries; Shalom Suniula 2 con). HT: 5-12
New Zealand 26 (Tomasi Cama, Lote Raikabula, Kurt Baker, Sherwin Stowers tries; Cama, Baker, Stowers cons) Georgia 7 (Merab Kvirikashvili try, con) HT: 12-0
New Zealand 31 (Sherwin Stowers 2, Gillies Kaka, Kurt Baker, Pita Ahki tries; Kaka 3 con) Canada 12 (Nanyak Dala, Harry Jones tries; Nathan Hirayama con). HT: 19-0
What effect will a potential ban on booze at Rugby Sevens 2015 have on you?Related story: Booze ban hovers over sevens