Can't help you with the Java/MySQL specific stuff, but the logic of your SQL should check that the input user/password values match the ones in the database i.e. something like SELECT 1 FROM users WHERE user_name = :input_user AND password = :input_password. You also need to allow for encrypted passwords in the database e.g. MD5. If the SQL returns a row, then you know the details match. If not, then you know the user/password combination is wrong.
You should not just fetch all the users into your Java code and compare them. As Bear says, this is inefficient, and it is also insecure: you should not be looking at any other users here. If you code your SQL to check the username and password within the WHERE clause, then you do not need to fetch the stored password at all.