High court chancellor Sir Andrew Morritt said he had 'considerable sympathy' for Monro, but that, as the law stood, he had no hope of reclaiming the money, because the time limit to revise payments was one year.
Monro said: 'I am sure that, if this were the other way around, they would have been all over me like a rash. We all have to pay tax, but I have paid enough.'
The dispute dates back to 1998, when Monro's accountants miscalculated the tax due on his share options in Matalan. He exercised an option to acquire more than 1.3 million Matalan shares - then valued at£2.35.
However, accountants didn't realise that Monro had overpaid until a Court of Appeal declared that the method of computation was wrong, but by then it was too late, as the year in which he could claim had already elapsed.
Monro has also been ordered to pay more than£10,000 in legal costs.
Morritt refused permission to take Monro's case to the Court of Appeal, because there was 'no hope of success'.