[HELP] How to login to a website and get XML data of that site

Hello,

I'm trying to login to a website, named Maxithlon.com, to get xml data from that site. So I can make a program for the users. The only problem is, I can't get the connection.

This is the login string query: http://www.maxithlon.com/maxi-xml/login.php?user=Gynnad&scode=1q2w3e4r
Where Gynnad is the username and 1q2w3e4r is the securitycode.

First I thought it was a POST-method but it isn't, so this code isn't right, but is what i had:

/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */

package Connection;

/**
 *
 * @author Danny
 */
import java.net.*;
import java.io.*;

public class ConnectToURL {

// Variables to hold the URL object and its connection to that URL.
        private static URL URLObj;
        private static URLConnection connect;

public static void main(String[] args) {
        try {
            // Establish a URL and open a connection to it. Set it to output mode.
            //login link: http://www.maxithlon.com/maxi-xml/login.php?user=Gynnad&scode=1q2w3e4r
            URLObj = new URL("http://www.maxithlon.com/maxi-xml/login.php");
            connect = URLObj.openConnection();
            connect.setDoOutput(true);
       }
        catch (MalformedURLException ex) {
            System.out.println("The URL specified was unable to be parsed or uses an invalid protocol. Please try again.");
            System.exit(1);
        }
        catch (Exception ex) {
            System.out.println("An exception occurred. " + ex.getMessage());
            System.exit(1);
        }

try {
            // Create a buffered writer to the URLConnection's output stream and write our forms parameters.
            BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(connect.getOutputStream()));
            writer.write("user=Gynnad&scode=1q2w3e4r");
            //username=MyUsername&pass=MyPassword&submit=Login (Origineel van de tutorial)
            //user=Gynnad&scode=1q2w3e4r (dit is wat de weburl zelf gebruikt)
            writer.close();

// Now establish a buffered reader to read the URLConnection's input stream.
            BufferedReader reader = new BufferedReader(new InputStreamReader(connect.getInputStream()));

String lineRead = "";

// Read all available lines of data from the URL and print them to screen.
            while ((lineRead = reader.readLine()) != null) {
                System.out.println(lineRead);
            }

reader.close();
        }
            catch (Exception ex) {
                System.out.println("There was an error reading or writing to the URL: " + ex.getMessage());
            }
        }
    }

and this was the result of that code:

Can someone help me with a explanation/tutorial or something?

Thanks!

If the login works via GET, why are you trying to use POST? Do you have any indication that that should work?

The page apparently doesn't accept POST requests; it does accept GET requests. Just put the parameters in the URL:

Usually I'd suggest to not post read passwords, but in this case we wouldn't be able to test any solutions without it. I do suggest you to change it as soon as possible after finding a proper solution though.

Rob Spoor wrote:The page apparently doesn't accept POST requests; it does accept GET requests. Just put the parameters in the URL: new URL("http://www.maxithlon.com/maxi-xml/login.php?user=Gynnad&scode=1q2w3e4r");

Usually I'd suggest to not post read passwords, but in this case we wouldn't be able to test any solutions without it. I do suggest you to change it as soon as possible after finding a proper solution though.

/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */

package Connection;

/**
 *
 * @author Danny
 */
import java.net.*;
import java.io.*;

public class ConnectToURL {

// Variables to hold the URL object and its connection to that URL.
	    private static URL URLObj;
	    private static URLConnection connect;

public static void main(String[] args) {
        try {
            // Establish a URL and open a connection to it. Set it to output mode.
            //login link: http://www.maxithlon.com/maxi-xml/login.php?user=Gynnad&scode=1q2w3e4r
            URLObj = new URL("http://www.maxithlon.com/maxi-xml/login.php?user=Gynnad&scode=1q2w3e4r");
            connect = URLObj.openConnection();
            connect.setDoOutput(true);
       }
        catch (MalformedURLException ex) {
            System.out.println("The URL specified was unable to be parsed or uses an invalid protocol. Please try again.");
            System.exit(1);
        }
        catch (Exception ex) {
            System.out.println("An exception occurred. " + ex.getMessage());
            System.exit(1);
        }

try {
//            // Create a buffered writer to the URLConnection's output stream and write our forms parameters.
//            BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(connect.getOutputStream()));
//            writer.write("user=Gynnad&scode=1q2w3e4r");
//            //username=MyUsername&pass=MyPassword&submit=Login (Origineel van de tutorial)
//            //user=Gynnad&scode=1q2w3e4r (dit is wat de weburl zelf gebruikt)
//            writer.close();

// Now establish a buffered reader to read the URLConnection's input stream.
            BufferedReader reader = new BufferedReader(new InputStreamReader(connect.getInputStream()));

String lineRead = "";

// Read all available lines of data from the URL and print them to screen.
            while ((lineRead = reader.readLine()) != null) {
                System.out.println(lineRead);
            }

reader.close();
        }
	        catch (Exception ex) {
	            System.out.println("There was an error reading or writing to the URL: " + ex.getMessage());
	        }
	    }
	}

Now it works to login, now i'm going to set some cookies so I can go to another website and get the content of that page. I will post that also here. If you have some comments please give them so I can learn about it!

The password is not the real password of the page, it is a securitycode and you can only access the pages with the xml, not the game.

Welcome to the Ranch

That question looks too difficult for us here on "beginning"; shall move it. Not sure whether to "servlets" in view of POST and GET, maybe I'll try "in general" and let Rob move it again if he thinks appropriate.

Nah, it's good here. But you could also have considered Sockets and Internet Protocols

public static void signIn() throws MalformedURLException {
        try {
            URL myUrl = new URL("http://www.maxithlon.com/maxi-xml/login.php?user=Gynnad&scode=1q2w3e4r");
            URLConnection urlConn = myUrl.openConnection();
            urlConn.connect();

String headerName = null;
            for (int i = 1; (headerName = urlConn.getHeaderFieldKey(i)) != null; i++) {
                if (headerName.equals("Set-Cookie")) {
                    String cookie = urlConn.getHeaderField(i);
                    cookie = cookie.substring(0, cookie.indexOf(";"));
                    System.out.println("cookie=" +cookie);
                    String cookieName = cookie.substring(0, cookie.indexOf("="));
                    String cookieValue = cookie.substring(cookie.indexOf("=") + 1, cookie.length());
                    System.out.println("cookieName=" +cookieName);
                    System.out.println("cookieValue=" +cookieValue);                    
                }

}

} catch (Exception e) {
            e.printStackTrace();
        }

}

Now I have this, with this output:

So I understand this page has 2 cookies, but witch one should I use and how can I go now to another webpage?

public static void signIn() throws MalformedURLException {
        try {
            URL myUrl = new URL("http://www.maxithlon.com/maxi-xml/login.php?user=Gynnad&scode=1q2w3e4r");
            URLConnection urlConn = myUrl.openConnection();
            urlConn.connect();

String headerName = null;
            for (int i = 1; (headerName = urlConn.getHeaderFieldKey(i)) != null; i++) {
                if (headerName.equals("Set-Cookie")) {
                    String cookie = urlConn.getHeaderField(i);
                    cookie = cookie.substring(0, cookie.indexOf(";"));
                    System.out.println("cookie=" + cookie);
                    String cookieName = cookie.substring(0, cookie.indexOf("="));
                    String cookieValue = cookie.substring(cookie.indexOf("=") + 1, cookie.length());
                    System.out.println("cookieName=" + cookieName);
                    System.out.println("cookieValue=" + cookieValue);

URL anotherUrl = new URL("http://www.maxithlon.com/maxi-xml/world.php");
                    URLConnection connection = anotherUrl.openConnection();
                    connection.setRequestProperty("Cookie", cookie);
                    BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));

String lineRead = "";

// Read all available lines of data from the URL and print them to screen.
                    while ((lineRead = reader.readLine()) != null) {
                        System.out.println(lineRead);
                    }

reader.close();
                }

}

} catch (Exception e) {
            e.printStackTrace();
        }

}

Ok, with the second cookie the connection works! So now I will make the code a bit clearer and only use the 2th cookie. Someone has any tips for me?

How can I now see if the user is sign in? Because the website returns always a Cookie. So I can't check if the cookie is null.