I contemplated it, but finally decided against it. But I got a haircut this week that cut it pretty short, and since I'm blond it's not much noticeable in the hat's shadow. That's gotta be good enough :-)
But I may decide to keep the goatee for a little while longer. I can honestly say that took weeks of preparation.