This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes Sockets and Internet Protocols and the fly likes login and downloading files from website using java code Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Java » Sockets and Internet Protocols
Bookmark "login and downloading files from website using java code" Watch "login and downloading files from website using java code" New topic
Author

login and downloading files from website using java code

Karan Jain
Ranch Hand

Joined: May 30, 2007
Posts: 82
Hi,

I have access to the client website where i can login and can download some file by selecting my specific criteria. I can then download 1 file at a time by selecting the file.

What I want is write a java program which will login me on the website and then enable me to download the files 1 after the other. I have login URL and the file URLs (as I can modify the IDs in the URL to point to next file).

I tried to use Commons HTTPClient but I am not successful in doing so. I am unable to set the cookies and then point to file URLs.

Client's website is in .Net.

Do I need to ask the client how they are setting the cookie so that i can do the same in my java program?

Please guide me to achieve this.

Thanks in advance...

Regards,
Karan
Norm Radder
Ranch Hand

Joined: Aug 10, 2005
Posts: 685
I'm not familiar with the Commons HTTPClient code so I can't help you with that. The class must have methods to set the various fields in the HTTP headers.
Cookies are set by the server and passed to the client to save and return the next time the client connects to that server. Cookies are passed as part of the HTTPHeader.

The code that the server is written in should NOT have any effect on the HTTP protocol.

I understand your project to be a program to run on a client that is to connect to a server at a URL. Not sure what you mean by "log me in on the website". Are there user-ids and passwords involved? If so, the server will have to have a way to indicate that you have logged in successfully, perhaps via a cookie. After that you would issue HTTP GETs for the desired files, passing the cookie each call.
For projects like I just described, I use a proxy server I wrote that allows me to see the HTTP files going and coming between a browser and the server.
That allows me to write a program that does what the browser does and hopefully to satisfy the server.
Karan Jain
Ranch Hand

Joined: May 30, 2007
Posts: 82
Thanks Norm.

Yes the website is setting the cookies.

I know the login URL and I am able to get the cookies in the response which I am able to print.

But how to set the cookies in the system so that next time when I am trying to access the website it should recognise me and should allow me to download the required files.

Thanks,
Karan
Norm Radder
Ranch Hand

Joined: Aug 10, 2005
Posts: 685
how to set the cookies in the system

Where/what is 'the system'?

the cookie is initially set by the server. It passes a SetCookie field to a browser that connects to that server. The browser saves the cookie.
The next time the browser connects to that server, it passes a Cookie field in the request header with the value of the cookie it received from the server.

For your program to get a cookie:
1)have your program connect to the server and be assigned a cookie that it saves, just like a browser does
2)somehow get the current cookie value that is in your browser and code that into your program.
[ June 20, 2008: Message edited by: Norm Radder ]
Karan Jain
Ranch Hand

Joined: May 30, 2007
Posts: 82
Thanks Norm,

I was able to make it work using Commons HttpClient.

I did the following:
1. Send the request using the logon URL with user name and password as NV pairs.
2. The URL send the response with cookies.
3. Send the next request setting the cookies in the header and it gives the response as the file.

Thanks a lot for pointing me to right direction.
reagrds,
Karan
t horo
Greenhorn

Joined: Jul 21, 2008
Posts: 1
hi
can you put a code snippet?
I'm trying to do the same with no luck
thanks
[ July 21, 2008: Message edited by: Ulf Dittmer ]
 
wood burning stoves
 
subject: login and downloading files from website using java code
 
Similar Threads
problem retrieving a zip file from FTP server using org.apache.commons.net.ftp.
A Software for Internet Cafe Management
Passed SCEA/OCMJEA PART I.
how run this server over internet
J2EE & client applications. HELP PLEASE