aspose file tools*
The moose likes Sockets and Internet Protocols and the fly likes Login to ASP page Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of EJB 3 in Action this week in the EJB and other Java EE Technologies forum!
JavaRanch » Java Forums » Java » Sockets and Internet Protocols
Bookmark "Login to ASP page" Watch "Login to ASP page" New topic
Author

Login to ASP page

Sayed Ibrahim Hashimi
Ranch Hand

Joined: May 17, 2001
Posts: 148
I'm trying to automate a process of downloading a file from a website. The thing is that I have to fill in the username and password then click a button on this website. This webpage is using ASP. I have checked with the company maintaining the website and they don't have another protocol for downloading that file. I know this can be done in VB.NET, can this be done in Java?
Thanks.


SCJP 1.4<br /><a href="http://www.cise.ufl.edu/~sih" target="_blank" rel="nofollow">www.cise.ufl.edu/~sih</a>
Peter den Haan
author
Ranch Hand

Joined: Apr 20, 2000
Posts: 3252
I've got difficulties squaring "ASP" with "Java". What exactly are you trying to do?
- Peter
James O'Dell
Greenhorn

Joined: Apr 29, 2002
Posts: 16
I'm really interested in this too! I believe what Ibrahim is getting at is the following: Can one automate a remote ASP form using Java.
I have a customer that wants our company to upload files to their secure web server, however they believe that most businesses will have a person manually authenticate to their site, click a "browse" button to find a file on their local hard drive and then click "submit" to send the local file to the remote web server.
I was given the task of automating this. I believe the answer lies in using the HttpURLConnection class and formulating the proper query string, but I have not had success. Anyone done this before?
Thanks,
Jim
Sayed Ibrahim Hashimi
Ranch Hand

Joined: May 17, 2001
Posts: 148
We'll here is how it is. Every week I have to go to a website then login. Then I have to mark a few checkboxes. Then I click the "Submit" button. This prompts the server to generate a file based on a database that they have. Then when that file has been generated, a URL to that file is returned then I download it. Now I'm trying to automate this.
I tried creating a URL with the parameters like this
www.thewebsite\login.asp?username=myusername&password=mypassword&check=on
But it doesn't seem to recognize those parameters. As a side note i took that same URL and placed it in a Browser and it doesn't make those settings. Maybe I'm incorrectly formatting it.
On top of that I have to "click" that submit button, how could I do this in Java?
Thanks in advance.
Sayed Ibrahim Hashimi
Ranch Hand

Joined: May 17, 2001
Posts: 148
Originally posted by Ibrahim Hashimi:
We'll here is how it is. Every week I have to go to a website then login. Then I have to mark a few checkboxes. Then I click the "Submit" button. This prompts the server to generate a file based on a database that they have. Then when that file has been generated, a URL to that file is returned then I download it. Now I'm trying to automate this.
I tried creating a URL with the parameters like this
www.thewebsite.com\login.asp?username=myusername&password=mypassword&check=on
But it doesn't seem to recognize those parameters. As a side note i took that same URL and placed it in a Browser and it doesn't make those settings. Maybe I'm incorrectly formatting it.
On top of that I have to "click" that submit button, how could I do this in Java?
Thanks in advance.

As a side note, I have some VB.NET code that will do this but I can't use it on our system. Here is that code for those who know VB.NET (I don't ) This code will actually give fill in some textboxes for a date range whereas I am going to check a checkbox for all. Some of this info has been changed for security purposes.
Peter den Haan
author
Ranch Hand

Joined: Apr 20, 2000
Posts: 3252
Clicking the submit button does no more than generating the request. The first problem is that the ASP server expects a POST rather than a GET request; the java.net.URL class will use GET by default. Try something likeThis is merely a skeleton for your code -- in real life, you'd want to check the response code, put the close() calls in finally clauses, use buffered streams, etc.
We're not there yet: from the VB.NET (ugh yuck splut) code it's clear that the site uses HTTP authentication. The code above remains unchanged except that you first install an Authenticator for the URL class to use:As you can see, a POST is a rather cumbersone operation, especially when authentication is involved. Many use helper classes, like Jason Hunter's excellent com.oreilly.servlet package. In this case, com.oreilly.servlet.HttpMessage would be suitable. It supports POST messages and BASIC authentication in a much more simple mannerI think you can see now why the O'Reilly helper classes are so popular
Hope this helps
- Peter
Sayed Ibrahim Hashimi
Ranch Hand

Joined: May 17, 2001
Posts: 148
Peter
Thanks for those hints. I was unable to use the com.oreily.servlet package because I couldn't figure out how to authenticate to one URL then send the request to another. So I just decided to write the code from the "skeleton" that you gave me. I am able to do the POST and to get the results but when I try to decode the results with URLDecoder it always throws an IllegalArgumentException:

java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape
(%) pattern - "
at java.net.URLDecoder.decode(URLDecoder.java:167)
at java.net.URLDecoder.decode(URLDecoder.java:82)
at Export.test(Export.java:167)
at Export.main(Export.java:121)

Do you have any idea how I can resolve this issue? Thanks for all your help, it means a lot to me.
James O'Dell
Greenhorn

Joined: Apr 29, 2002
Posts: 16
That was a big help!
Thanks Peter
Peter den Haan
author
Ranch Hand

Joined: Apr 20, 2000
Posts: 3252
Originally posted by Ibrahim Hashimi:
[...] when I try to decode the results with URLDecoder it always throws an IllegalArgumentException:
Is the response encoded in the first place? Can you post the response text and/or locate where the decode function breaks exactly?
- Peter
Sayed Ibrahim Hashimi
Ranch Hand

Joined: May 17, 2001
Posts: 148
Hi,
It seem like it is not decoded. If you haven't figured out I'm new to this whole network programming lol. I am able to get a response from the server, but it is not the response that I'm looking for. It just returns the same webpage I was viewing. I was thinking that I wasn't waiting for the server to crete the document so I tried using a BufferedReader and wait until reader.ready() is true but it will just return the same webpage. Then I wait after that again, and nothing happens. I even added connection.setFollowRedirects(true); to my code thinking that would solve my problem but it was to no avail. Any ideas? Thanks a lot.
Heres my code if you want to have a look

[ August 31, 2002: Message edited by: Ibrahim Hashimi ]
Raghav Mathur
Ranch Hand

Joined: Jan 12, 2001
Posts: 641
hi
probably whatever you are trying to write to the server is not being sent to the server. That can be the only reason why it shows you the same page when you try to login . I,am stuck with the same prob . I,.am trying to capture the request from the client browser , make a Url connection for that request and read the contents .

Originally posted by Ibrahim Hashimi:
[QB]Hi,
I am able to get a response from the server, but it is not the response that I'm looking for. It just returns the same webpage I was viewing.
[ September 21, 2002: Message edited by: raghav mathur ]

Raghav.
Raghav Mathur
Ranch Hand

Joined: Jan 12, 2001
Posts: 641
IN the above examples , the URl have been hardcoded . Suppose it is not then how does one judge whether to write a URL or read from it ?
Thanks in advance
Peter den Haan
author
Ranch Hand

Joined: Apr 20, 2000
Posts: 3252
Originally posted by raghav mathur:
IN the above examples , the URl have been hardcoded . Suppose it is not then how does one judge whether to write a URL or read from it ?
Depends on the HTTP method you invoke. A typical HTTP request opens a port to the HTTP server, sends the request header, a newline, and optionally a request body, then reads the response which will again consist of a header and an optional body.
The most often used methods are GET and POST; with GET, there is no request body and any parameters are encoded in the URL. With POST, the request body contains the parameter values submitted to the server. You can also upload files. For both GET and POST the server will typically respond with a response code and other header fields, followed by an HTML page.
- Peter
Raghav Mathur
Ranch Hand

Joined: Jan 12, 2001
Posts: 641
Peter
so that means by guessing from the setRequestMethod()i can retrieve the whether pirtuclar request is using GET or POST .
(1)If the request is using GET ( without a query string) that means that the data has to be read from the URL.
(2)If the request is using GET "WITH" a query string that means data has to be writin to a URL .
(3)As far post is concerned , Data has to be writin in any case .
Is that so ?
Now suppose i have an application which acts as a proxy server between the client browser and the server . Whatever request is made by he client has to go through that proxy server application,The client makes a request , which connect to the proxy server aplication which then makes URL connection and get the request for the client .The proxy server appliation doesn't know what the clients request is but because the default is GET so that it is no prob for the proxy server application to make a URL connection and read the contents and deliver it to the client. Now if the client has to login to an ASP page using "POST" then the proxy application knows that the data has to be writin to the URL. If the client is using POST that means there would be no query string attched to it , then how would the proxy application know what it has to write to the URL? . how can be that info retrieved by the proxy application?
I hope i,am clear with my query .
Thanks in advance
Peter den Haan
author
Ranch Hand

Joined: Apr 20, 2000
Posts: 3252
Originally posted by raghav mathur:
(1)If the request is using GET (without a query string) that means that the data has to be read from the URL.
(2)If the request is using GET "WITH" a query string that means data has to be writin to a URL .
No. There is no difference between having a query string or not -- as I said, the method is significant. The GET method does not expect a request body, i.e. it does not expect you to write.
Now suppose i have an application which acts as a proxy server [...] Now if the client [uses] "POST" then the proxy application knows that the data has to be writin to the URL. If the client is using POST that means there would be no query string attched to it , then how would the proxy application know what it has to write to the URL?
The proxy would expect the client to write a request body with the POST, so it would capture whatever the client is writing and write that, in turn, to the web server. In the real world, rather than doing this in two discrete steps (read everything from the client, then write it to the server) a proxy would probably stream the write straight through to the remote server. In Java terms, it would write the bytes read from the client InputStream straight into the server OutputStream.
- Peter
Raghav Mathur
Ranch Hand

Joined: Jan 12, 2001
Posts: 641
PETER


No. There is no difference between having a query string or not -- as I said, the method is significant. The GET method does not expect a request body, i.e. it does not expect you to write.

So then if an ASP page uses a GET method to actually write data on to the web server . The how will I judge whether to use GET or POST ? as you said that GET doen’t expect you to write.

In Java terms, it would write the bytes read from the client InputStream straight into the server OutputStream.

Aren’t these still 2 steps . The proxy in any case has to read from the client InputStream and then write it to the proxy servers OutputStream.
Anyways the biggest issue right now is how to judge whether to use GET ot POST.
Thanks in advance
Raghav Mathur
Ranch Hand

Joined: Jan 12, 2001
Posts: 641
By reading the request header one can judge what request method to use. Now if i want to give my proxy server the functionality to handle all major protocols like socks,smtp,pop then what will be step i have to take .In case of post i know i have to write , in case of get i know i have to read , i will also need to use PUT ....This is about http . What if i have to handle Socks , ftp , smtp,pop etc .

Thanks in advance
[ October 16, 2002: Message edited by: raghav mathur ]
Raghav Mathur
Ranch Hand

Joined: Jan 12, 2001
Posts: 641
Peter
I,am stuck with another prob now . I,am not able to extract the content length value from the header so that i can send it via post method to a specific URL( write to the URL) . What i was thinking of is that rather that extracting the content-length value( in case of post ) i should send the whole header ( which would include the content -length value) . What do you say ?
thanks in advance .
[ October 28, 2002: Message edited by: raghav mathur ]
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Login to ASP page
 
Similar Threads
File Chooser
change print settings through javascript
Remote hardware information
Tennis score keeping program
downloading files from database