In windows NT and 2000 you can set the buffer size of your DOS window (I use 4000 lines). This gives you a scroll bar that you can use to scroll back up to see your messages.
Right click the upper left DOS window icon and select "properties". Then select the Layout tab. In the buffer "height" window put 4000.
If you're in Windows 95/98/xp/me/whatever then you're outta luck and have to redirect your output every time.
You've just put your finger on exactly the reason why I won't develop Java code in Windows 98.

I only use Win 2k and Linux for development. 98 is my game OS. This is one of the reasons why.