Prince Andrew 'honourably withdrew' from joining the Royal Family for the annual gathering in Sandringham last Christmas to avoid being a distraction.