It is an inherent characteristic of the Factory design
pattern that
you should be able to construct a Factory once and re-use it thereafter. Otherwise, why bother with a Factory instead of just using a constructor?
Instances created
by a factory are a different matter. The other characteristic of a Factory is to be able to produce a product rapidly, which it may do by employing an internal cache or pool. So the overhead for factory products is typically very low. And the properties of the product in question may either have a limited lifetime (for example, network connections that time out), need cleanup (files and connections to close), or may become so dirty in the course of use that it's less trouble to simple obtain a new one.
So as a general rule, I'd cache the factory and use it to create clients on-demand.
It doesn't hurt to read the instructions, though, just in case there are special considerations.
The secret of how to be miserable is to constantly expect things are going to happen the way that they are "supposed" to happen.
You can have faith, which carries the understanding that you may be disappointed. Then there's being a willfully-blind idiot, which virtually guarantees it.