Make sure the vehicle brakes are off, after ensuring it wont roll anywhere, you dont have the diff lock in do you?
Also try pushing the end of the shaft down, it may be holding somewhere
Its probably just holding onto the diff, push it back and yank it a bit harder