This week's book giveaway is in the OO, Patterns, UML and Refactoring forum. We're giving away four copies of Refactoring for Software Design Smells: Managing Technical Debt and have Girish Suryanarayana, Ganesh Samarthyam & Tushar Sharma on-line! See this thread for details.
I imagine this is easiest to achieve using a ResourceBundle.Control subclass that overrides the getTimeToLive method by either returning zero (validate* the resource every time it is retrieved), TTL_NO_CACHE (don't put the bundle in a cache - effectively reload bundle every time the factory method is called), or has some count-down that sets a reasonable amount of time between validations.
*Validate means it will use the needsReload method to check if the data in the ResourceBundle needs to be reloaded. Typically for .properties files that would mean checking if the modified time of the file is after a timestamp the resource bundle holds internally.