First of all, get the latest stable release of Netbeans, i.e., version 5.5. If you are already having this version then follow the steps below:
1. Create a new project in netbeans.
2. Go to menu File > New File ( or alternately press Ctrl - N )
3. In the dialog box that appears, notice that in the left hand side you have a list of categories, select Java GUI Forms in the category list.
4. Notice that the right hand list now changes to show you the available form types. You must have a type called JDialog form available in the right hand side list. Select this and press next!
5. Fill in the details in the next page and Voila! The GUI editor of Netbeans called Matisse now opens up for you with the JDialog that you want to create.