A new persistence context begins when the container-managed entity manager is invoked in the scope of an active JTA transaction, and there is no current persistence context already associated with the JTA transaction. The persistence context is created and then associated with the JTA transaction.
and, in the same section:
If the entity manager is invoked outside the scope of a transaction, any entities loaded from the database will immediately become detached at the end of the method call.
In the second case, the persistence context is created too, isn't it ? So the good answer to "When is a Persistent Context created while using container managed EntityManager?" is
look at footnote 36 at the bottom of page 121 of the JSR 220 persistence spec:
"Specifically, when one of the methods of the EntityManager interface is invoked."
As far as container-managed transaction-scoped entity managers are concerned, "every time an operation is invoked on the entity manager, it checks to see if a persistence context is associated with the transaction. If it finds one, the entity manager will use this persistence context. If it doesn't find one, then it creates a new persistence context and associates it with the transaction. When the transaction ends, the persistence context goes away."
source: pg. 113 of Pro EJB3 JPA by Mike Keith
As far as container-managed extended-scoped entity managers are concerned, "extended entity managers create a persistence context when a stateful session bean instance is created and lasts until the bean is removed."
I highly recommend you read this entire book, Keith was a co-spec lead on JSR220.