*nix systems tend to use plain text files for configuration, so at first blush I'd say yes, Java may work. However, depending on your Linux distribution, Java support may be spotty (this should get better now that Sun has released Java on an open-source license).
It all depends on what "system administration" means. Certainly if you are set up to run Tomcat, JBoss or the like as core system services, you're already using Java for system administration.
If you mean for system control and maintenance, certainly - I could, for example, write a crontab maintenance utility in Java either as a standalone app or as a webapp.
The only real restrictions are portability and overhead. You can't reasonably use Java in an environment where a JVM has to be fired up everytime it's needed but the task requires that startup delays be kept minimal (or storage is at a premium). Of course, if you by into the type of stuff that BEA (among others) is playing with, Java is essentially part of the OS, so that overhead mostly goes away.
Portability wouldn't be an issue execpt for the half-baked noncompliant gcj implementations, but thanks to Sun's opening the doors, I expect that incomplete JVMs will join J++ in the dustbin of history before much longer.
An IDE is no substitute for an Intelligent Developer.
Some problems are better suited for scripting languages and some are better suited for object-oriented programming languages. Use the correct tool for the job. Picking the correct tool may take some trial and error until you get a sense of what the strengths and weaknesses of each are.
While you technically can do anything with Java that you can do with any other language, I think this is a serious mistake. If you are the sysadmin, and will always be the sysadmin, then by all means, do what you like. But if there is any chance, however small, that the system will grow and be put in production, and managed by professional sysadims, then you should do it using the "when in Rome, do it the Roman way" approach.
For Linux, the Linux way is perl or bash scripts.
Plus, perl is a useful language to know. You can do many sysadmin things in five lines of perl that would take many times as much in Java.
p.s. you are showing your Windoze biases talking about "registry" as there is no such think in Linux, BSD, etc. All that stuff is kept in ascii text files.