You can typically do some simple graphics quite early. I started learning Java in a college course and we were creating simple GUIs by midterms, if I recall correctly. I think you should make sure you understand the language syntax thoroughly. You should also understand classes and interface. Also be sure you know how to navigate the Java API docs.
That's just my 2 cents, so take it for what it's worth.
One of the benefits of Java is that it actually makes to quite easy to get started with GUIs. And all kinds of graphics can be debugged visually to a degree (i.e., by simply looking at them), which makes it easier to spot bugs than if you were implementing, say, numerical algorithms. So, I agree with Layne, you can get started pretty early.