In my opinion, moving the EL information out of the JSP Specification, and into its own mixed up with the JSF stuff was a big mistake. It's completely confusing for anyone not interested in the JSF aspects of the EL.
Acutally, what they did was get rid of the idosyncratic per-framework expression languages and merge them all into a Unified Expression Language (UEL). Makes the rules more consistent and voids the need for separate parsing, interpreting and support facilities.
The primary difference between the "$" and "#" indicators is that "#" indicates a reference and "$" indicates a value. Specifically, since "#" is a reference, it can be used in situations where an lvalue expression is required.
The JSTL/jsp tags retrieved only values, so they use ""$". However, JSF often updates the references, as for example in the "inputText" controls.
I think in JSF pre-historic times, the use of one or the other might have been briefly mandatory. These days, the "#" notation works for most everything JSF. JSP items, on the other hand, aren't prepared to receive input on their EL properties, so they continue to use the (read-only) "$" notation.
Bjoke: A "Bully Joke". A Statement or action made with malicious intent - unless challenged. At which point it magically transforms into "I was just funnin'" or "What's the matter, can't take a joke?"