For AWT buttons (for example),
Java asks the operating system to draw a button. So, for example, a Windows control is created. The AWT component then has to keep track of a pointer to this native Windows button, and remember to dispose of it at some point. Making all these calls to the OS is what's considered "heavyweight".
For Swing buttons, the button is drawn on the screen in Java code. There are no native controls, no remembering to dispose of anything -- hence they are independent, free, "lightweight."