I only use frame level listeners for this relevent to the frame itself, like WindowListener or MouseListener/MouseMotionListener on Containers.
Cannot agree more.
I seem to give instructions about how to write an ActionListener at regular intervals, most recently
here. Are you using a Listener which applies specifically to the JFrame or other Component? If you are, then make the Component implement MouseMotionListener and use its addMouseMotionListener(this) method.
Are you using the same Listener more than once, either to do the same thing in different places (eg an exitButton and an exitMenuItem, both of which have the same listener), or similar things (eg the examples beginners are taught where the redButton turns something red and the blueButton turns something blue)? In that case you need a concrete named class to do it. Call it ExitListener, ColourListener, or whatever. You can either use a public class (in which case you can reuse it for other applications) or a private inner class for the Listener.
For colours, pass the Color object to the Listener constructor.
For a Listener you are using once and once only, use an anonymous inner class. No two ways about it.
As Ilja Preuss says, see how inelegant it is to have a method in your JFrame which reads like this:
Even though you will see examples like that in books
CR