Check out the API for GridBagConstraints weighty and fill attributes. Essentially you need to add an invisible "filler" component after you have added all your buttons and tell the GridBagConstraints that it will occupy all of the remainder space so it "pushes" the buttons up. I usually use an empty JLabel as a filler.
Try out your code with these additions just before the call to set visible.
Some pointers.
1) Instead of frame.setLayout and frame.add,
you should be performing these operations on the content pane. Check out JFrame#getContentPane()
2) I personally prefer nested panels. getContentPane defaults to BorderLayout which I let remain. I would use a JPanel, set it to GridBagLayout, add the buttons and then add the panel to the content pane. This gives me the flexibility to add something else to the content pane North, South, East, West in the future if required. This might sound a few more lines of coding, but I find it worth it, based on my experience. But then this is a personal preference.
3) Instead of explicitly defining c.gridx=1, try to use the incremental operator. Notice c.gridy++ in my code. 1 defines an absolute value whereas ++ defines an incremental value. The advantage is that you do not need to touch any other code in case you insert/delete some other component in the future.
PS. It is nice to see someone "brave" enough to take on the GridBagLayout
. If you google, you will find why most developers shy away from it