If it doesn't affect his judgement, I don't see why he should be fired or anything.
He'd sort of be a hypocrite if he was a judge that broke the law...while in session, unlike a certain Judge located in Bel Air that emulates the honor that this position holds.