Given a suitable definition of "operating system", I suppose. Could you write something using only Java which would boot a generic x86 computer? Nope. Could you write an OS in Java, given a JVM and "glue" code to interact with the hardware? Sure.
Sun once had (I don't know if they still do) a microprocessor that executes Java bytecode in hardware. It is/was called picoJava. Java would be the logical choice if you'd want to write an operating system for a computer that contains such a processor.
Yup, given hardware that has built-in support to execute Java code to the extent of revealing its entire instruction set through Java commands, you could indeed do it (and it has been done). Without that support, you can't as you don't have access to the machine instructions you need.