This week's book giveaway is in the OO, Patterns, UML and Refactoring forum. We're giving away four copies of Refactoring for Software Design Smells: Managing Technical Debt and have Girish Suryanarayana, Ganesh Samarthyam & Tushar Sharma on-line! See this thread for details.
Yes, you can remove a component from the containment hierarchy. Yes, you can add a different button to the container as well. Whether or not that new button is in the same place as the old button is going to depend entirely on your layout and whether or not any state or constraints that are used by the layout are the same.
The two buttons happen to have a very similar size because the text is roughly the same size. If the buttons had vastly different sizes then obviously that would cause a problem since BorderLayout honors them.