If you add components to (or remove from) a JComponent after it's already visible, you need to revalidate() and repaint() the JComponent. Another easy fix is to not display the user interface until it's completely finished - move the call to frame.setVisible(true) to the end.
It only shows when I remove the buttons and panel and display it on the frame
When you add the Rectangle directly to the frame it get added to the content pane which uses a BorderLayout by default. The component get added to the center which means it automatically get resized to fill the entire area of the frame.
By default a JPanel uses a FlowLayout, which displays components at their preferred size.
As Darryl suggested you did not override the getPreferredSize() method of your component so its size is 0 and it doesn't get painted. You need to implement the getPreferredSize() method to return an appropriate value.