Good question. The intention of the spec at least (reading between the lines in section 5.6), is that the entity manager is safe to use when injected into a stateless session bean. This is because there is no persistence context associated with a transaction-scoped entity manager. Instead, the persistence context is acquired dynamically, either from the transaction or created temporarily when required outside of a transaction.
In OC4J, the entity manager you inject is really a lightweight and thread-safe proxy that does the right thing when a method on the entity manager is invoked.
The relationship between persistence units, persistence contexts and the things that actually get injected into your application can be complex. We devoted an entire chapter in our book on this topic for just this reason.