Assuming you we're shooting off solid front and rear rests, and allowed the barrel to not get too hot, I'd say look at the action screws first, if not that the bedding.
As an interim solution, you can try to put some tinfoil between the action and stock and see if that improved matters.
Bookmarks