The value of factory is when you don't want or need to know exactly the class that's being returned.
Widget a = new Widget();
Widget a = WidgetFactory.getWidget();
In the first example, I know the object I get is of class Widget. This is fairly strong coupling that may make things less flexible in the future. In the second example I know the object I get implements Widget or extends Widget or maybe is Widget.
Now the design takes advantage of
polymorphism. The client code can work with Widget or any subclass. The factory can return different types on different days of the week or based on configuration or whatever logic it likes.
I wouldn't use factories and interfaces for absolutely everything (though some might make interesting arguments for that) but over time you'll start to see things that have potential to change over time and reduce your coupling through abstractions, factories, dependency injection and a number of other cool OO idioms.
That kind of question is right on target for this forum. Keep asking!