I am not sure if any official export documents required for a US citizen to take a rifle out of the US. There is a US Customs Form that makes it easier to return to the US with the same rifle.
No questions are asked leaving NZ without the rifle you bought in and no questions are asked if you don't bring it back to the US - even if you bring the empty rifle case with you.
Bookmarks