Welcome to the Ranch
That looks like code from somebody who has dived in and written code without getting any design. You need to work out how a calculator will work, and start by implementing the part which does the arithmetic. You will find it a lot easier if you confine yourself to 1 + 2 and forget about complicated arithmetic like 1 + 2 × 3. Work out how you will do arithmetic like 123 + 456. How are you going to make the numbers up from individual digits? How are you going to return the result? Don’t try a GUI until you have got that all working.
I would suggest you delete the calculator listener completely and start again. What you want is new object‑oriented code. You want listeners for operators and listeners for operands.
What you are doing is adding a listener which takes the 2 from the button and uses that to set up its own values. It would be a lot easier if you had the 10 number buttons in an array, then you can use
But I think you will take all of today getting the logic for the arithmetic working, so that is something for tomorrow!