Imagine that you have two files, A.java and B.java containing classes A and B. Furthermore, imagine that class A mentions class B. Now, you type "javac A.java". At some point, the compiler is going to look for B.class, and since it doesn't exist yet, what should the compiler do? Of course, what it does is look for "B.java" and expect class B to be defined in it. If B.java instead contained class C, and C.java contained class B, then the compiler would fail to compile anything.
So even without the rule about one public class per file, you can see another common sense rule: if a class is referenced outside of the source file in which it is defined, the source file should be named after the class. Many Java compilers will warn about violations of this even for non-public classes.
Now, the only reason for a class to be public is for it to be used outside of its own source file, right? So it makes sense to make the rule stronger in this case. The common-sense rule, the one that people follow consciously or not, is that a source file should contain at most one class that is ever mentioned by name outside of the file, and the file should be named after that one class.