In Swing, I think the easiest way is to use JScrollPane (see the Scroll Pane Java Tutorial). Add all your components to a JPanel, put the panel in the JScrollPane, and then add that to your form. Everything in the JScrollPane becomes scrollable if necessary. The link contains examples.
As you suggest, though, a better solution might be to get your form to resize nicely in the first place. How easy that is depends on your layout, but the tools of choice here are Layout Managers. There are a range that can provide different types of re-sizing behaviour. Again, the Java Tutorials are a good place to start.