I think the shaft that the planet gears run on will be in the way if you try and push it from the other side.
Personally, for the sake of a dozen nuts, I'd take the diff out, check for any swarf / fragments give it some new oil and be able to sleep at night ;-)