No, every machine does not have to have a CLASSPATH variable set.
Which operating system are you using? Setting classpath to "," (comma) is almost certainly wrong.
If you do not set the CLASSPATH variable at all, Java will use the current directory as the classpath by default (which means, it will only search in the current directory for packages and class files).
Sometimes you need to have Java look in more places than just the current directory. In that case, it's best to specify the classpath on the command line when you're starting your Java program, like this (example):
Note the ";.". This includes the current directory, which is denoted by "." (dot, not comma!).
Doing it like this is better than setting the global CLASSPATH variable, because the CLASSPATH variable influences every Java program running on your computer, and not just the program that the settings are relevant for.