First, JavaBeans really don't have anything to do with GUIs. When tutorials write about JavaBeans they often use GUI components as examples because the are easy to visualize. Additionally, there are some tools which will make new JavaBeans-compliant classes by drag/dropping GUI components to a screen. But JavaBeans are not limited or specially associated with GUIs.
Now, what is a JavaBean?
This FAQ will answer a lot of questions. But very briefly: JavaBeans is a 'standard' for accessing class data more efficiently. Start with a normal Java class (often called 'Plain Old Java Object' or 'POJO') and define the concept of 'Properties' which is basically data being held by the class/object. The JavaBeans standard defines means of accessing those properties. A Property can be accessed via a 'get' method and changed via a 'set' method. The JavaBeans standard defines how those methods should be named. There is a bit more to it (you need a public, no-arg constructor, and there are property listeners so one class can listen in to when another class's properties are changed) but basically this is it - a JavaBean is a Class with Properties whose Accessors and Mutators follow a specific naming convention.
Why use JavaBeans? What makes them special? Because the Properties accessors and mutators have a specific/defined naming convention tools can use a process called Introspection to find out what Properties a particular bean has, which ones can be written to, or read from, and what their data types are. Then the appropriate methods can be called in a consistent manner without hardcoding anything about them in the 'Client' code (the code which actually uses the Beans). Specifically, it allows code generation tools to easily access the classes and learn a lot about them in an automated fashion. It also provides a nice generally naming convention for any code you write. There are some cases where programming to the JavaBeans standard is more important - like in web applications using JSPs because there is simplified access to JavaBeans using special
JSP syntax - as one example.
Coincidentally (well really by design but not by rule), a lot of the classes in the Swing API implement the JavaBeans standard which makes them easy to use with visual GUI builders you can get with most IDEs.
A lot has been written in the comparison between Swing and AWT. As a beginner, you might look at
this IBM article which compares the feature set of AWT, Swing, and a third part GUI library called SWT.