If you have a very simple program as you describe then its perfectly fine to put the main method and the other method in the same class. There is no point over-complicating things.
* Edit: as others have pointed out if you plan to reuse the code in other applications then its best to move the main method to its own class. By simple program I mean cases where you might be just
testing something out etc.
However as your applications get more complex it does make sense to move the main method into its own class. The application I'm currently working on has the main method in a class on its own. The main method verifies the right number of command line parameters are present, then constructs an instance of another class and calls a method on it to start the application.
There are no hard and fast rules though.