This public class acts as the initial class from where the JVM instance for the java program is begun.So when you provide more than one public class in a program the compiler itself stops you by throwing an error. This is because later you can't confuse the JVM as to which class to be its initial class because only one public class with the public static void main(String args) is the initial class for JVM.
It's just a choice that the inventors of Java made. There's no technical reason that explains why there must be only one public class per source file.
You can debate about why exactly they made that choice, but frankly, that isn't a very interesting discussion - there's not really a final, definitive answer that explains it. Maybe they made the choice to keep things simple and organized - if you're looking for the source code of a public class SomeClass, you know you're going to find it in a file named SomeClass.java. So, this rule makes it easier to find.