Nope, that will not do what you want. What you need is a way to check for a second attempt to initialize the reference and prohibit it. This is described by a design pattern known as a Singleton. It works something like this:
To answer the original question - yes, static initialization is guaranteed to occur only once for a given class. However your program does not guarantee that no other code will be able to instantiate a second instance, and that is the problem which Michael's code addresses.
If ConnectionFactory is a class over whose instantiation you have complete control, then using the Singleton pattern may be unnecessary. Appropriate documentation may be sufficient and is cheaper (in terms of byte code and execution time). This would certainly be the case if ConnectionFactory were a private inner class of ConnectionSource. You might argue that it was still the case if ConnectionFactory were package-private in the same package as ConnectionSource; this depends on how your code is distributed and maintained. If you want ConnectionFactory to be a public class and to allow code outside its package to instantiate it, but you want to prevent more than one instance being created, then Singleton is definitely the way to go.
Betty Rubble? Well, I would go with Betty... but I'd be thinking of Wilma.