The access specifiers are only related to the identifiers whose access they specify. Making a class public doesn't automatically make all its members public. So if making a class protected has no bearing on its members, then what use is this specifier? Making a top level class protected simply does not make sense on a conceptual level. The same is true for private.
Having said that,
you should avoid the protected keyword as much as possible. If you don't know exactly whether its use is correct, it probably isn't. Preferably, you should make most members private or package private.