wood burning stoves*
The moose likes Sockets and Internet Protocols and the fly likes Proxy Server in Java Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Soft Skills this week in the Jobs Discussion forum!
JavaRanch » Java Forums » Java » Sockets and Internet Protocols
Bookmark "Proxy Server in Java" Watch "Proxy Server in Java" New topic
Author

Proxy Server in Java

omkar patkar
Ranch Hand

Joined: Aug 25, 2005
Posts: 231
Hello Friends,

After a long time making a post here. Well, i am making a proxy server kind of application, so that every request my browser makes go through the server and the server makes a log of HTTP request and Response headers. So that user can get to see what are the parameters that are passed, what is the method used - GET/POST, what is the protocol 1.1 OR 1.0, Browser type, content type....etc.,

I am pasting the code i wrote but it is not giving me desired ! Here is the code for my proxy Server :-

-------------------------------------------------------
CODE
-------------------------------------------------------


-------------------------------------------------------

Now i configure the settings in my mozilla browser to Local host that is 127.0.0.1 and port as 5555 ...since that is the port to which my server is listening...

.... Now i get output something like this :-

-------------------------------------------------------
OUTPUT
-------------------------------------------------------
Starting the MyProxyServer ...
Inside while loop
Connection to MyProxyServer is true
Client has asked to ....
GET http://en-us.start2.mozilla.com/firefox?client=firefox-a&rls=org.mozilla:en-US fficial HTTP/1.1
Inside while loop



-------------------------------------------------------


.........Thats it !.... i am not able to see the complete HTTP headers ...what could be the problem ?


Thanks and Regards
Omkar Patkar (SCJP 1.4)[url]http://javacollectionsnotes.blogspot.com[/url] | [url]http://omkar-myscjpexp.blogspot.com[/url]
Francois Dang Ngoc
Greenhorn

Joined: Sep 02, 2007
Posts: 6
Hi Omkar,

The other meta-data are passed as single lines (key: value)
after the GET/POST request. For intance:

GET http://www.google.com/ HTTP/1.1
Host: www.google.com
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070810 Ubuntu/7.10 (gutsy) Firefox/2.0.0.6
...

That means that the program should iterate over all the lines
following the GET/POST request.

You can do something like that:

while(true) {
System.out.println("Inside while loop ");
Socket clientSocket = serverSocket.accept();
System.out.println("Connection to MyProxyServer is "+clientSocket.isConnected());

InputStreamReader inputStreamReader = new InputStreamReader(clientSocket.getInputStream());

BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
String command = bufferedReader.readLine();

System.out.println("Client has asked to ....\n"+command);

if(command.equals("Cancel")){
System.out.println("Shutting down the server ...");
break;

}

// iterate over all the lines following the GET/POST request
String meta = null;
while((meta = bufferedReader.readLine()).length() > 0){
System.out.println(meta);
}

bufferedReader.close();
}

Hope it helps,

Cheers,

Francois
omkar patkar
Ranch Hand

Joined: Aug 25, 2005
Posts: 231
Hi Francois,

Thanks Man !!! it worked ! ...great man! But one thing i want to ask, in the snippet that you had provided,....i replaced

---------------------------------
System.out.println(meta);
---------------------------------

with

---------------------------------
System.out.print(meta);
---------------------------------

the output that i got was in the second case was :-

===================================================
GET http://en-us.start2.mozilla.com/firefox?client=firefox-a&rls=org.mozilla:en-US fficial HTTP/1.1Host: en-us.start2.mozilla.comUser-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5Accept-Language: en-us,en;q=0.5X-McProxyFilter: ************Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7Keep-Alive: 300Proxy-Connection: keep-aliveCookie: __utma=183859642.485522640.1184809039.1185154927.1186198596.4; __utmz=183859642.1184809039.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none)Inside while loop
===================================================

So does that mean that, the meta information are not passed as single lines ? This is because, after intercepting this information i want to extract and hit the URL actually by doing something like this :-

---------------------------------------

URL url = new URL("http://www.google.com");

URLConnection connection = url.openConnection();
...
...
...
---------------------------------------

So i am confused if i am supposed to split the incoming strings on "\n" or " ".

Please let me know. By the way your tip worked and thanks again pal !

Thanks and Regards
mkar Patkar
omkar patkar
Ranch Hand

Joined: Aug 25, 2005
Posts: 231
Hi Francois,

One more query...i cannot see the post data ! Is the post data part of meta information ? what will i have to do inorder to get the post data ?

For e.g. i tried to post this reply .... and following is the meta information that i got :-
-----------------------------------------------------------------------
POST http://www.javaranch.com HTTP/1.1
Host: saloon.javaranch.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
X-McProxyFilter: ************
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Proxy-Connection: keep-alive
Referer: http://www.javaranch.com
Cookie: login2452372.103=09-04-2007%2008%3A18%20AM&2454348.0818; __utmz=212670614.1188915528.6.4.utmccn=(referral)|utmcsr=javaranch.com|utmcct=/|utmcmd=referral; __utma=212670614.1076877803.1188649572.1188915528.1188920989.7;__utma=254571667.1590012939.1188649525.1188649525.1188915509.2; __utmz=254571667.1188649525.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none); saloon2452372.103=00105516&3ee6d45d82575af784aed6b3d9025669; __utmc=254571667; session2452372.103=09-01-2007%2006%3A25%20AM&2454345.0625; __utmc=212670614; __utmb=212670614
Content-Type: application/x-www-form-urlencoded
Content-Length: 106
Inside while loop

-----------------------------------------------------------------------

any idea ?

Thanks and Regards
Omkar Patkar
Francois Dang Ngoc
Greenhorn

Joined: Sep 02, 2007
Posts: 6
Hi Omkar,

Yes, by single lines, I mean that there is one meta (Host, User Agent)
per line. One line for the GET followed by another line for Host, another line for User-Agent,...

So when you do:
while((meta = bufferedReader.readLine()).length() > 0){
System.out.println(meta);
}

it reads one line at a time but readLine() returns the line without the EOL (end of line) character, that is why when you print the whole thing using System.out.print, the EOLs are removed and everything is printed in one line.

The way POST works is different from GET in the sense that the
parameters are not passed in the URL as in GET
(e.g., GET http://www.google.com/search?q=test where q is
a parameter).

With POST, parameters are passed
after the meta information (e.g., User-Agent, Host, ...).
The format is as follows:
POST http://www.google.com/ HTTP/1.1
User-Agent: "Web Browser"
...
Content-Type: application/x-www-form-urlencoded
Content-Length: 11
*** HERE WE HAVE AN EMPTY LINE ***
q=foo&x=bar

Where q and x are two inputs entered using POST. The parameters are in the form of key=value separated by &.

In the meta, there is the line "Content-Length: 11" which
specifies the size of all the parameters (here, q=foo&x=bar which is 11 characters).

A small modification to the program should do to extract the string
q=foo&x=bar.

First, we need to get the content length so
we can change a bit the way we read the meta as follows:
String meta = null;
int contentLength = 0;
while((meta = bufferedReader.readLine()).length() > 0){
System.out.println(meta);
if (meta.startsWith("Content-Length:")) {
// get the number
contentLength = Integer.parseInt(meta.substring(meta.indexOf(':') + 2));
}
}

Note that this loop iterates over all the meta lines until it find an empty line. So after the
empty line, we just need to read the parameters (of size contentLength) issued by the post command as follows:
// this comes after the while((meta = ...

if (command.startsWith("POST")) {
// display params
char[] postData = new char[contentLength];
bufferedReader.read(postData);
System.out.println("Params in the POST request: [" + new String(postData) + "]");
}
bufferedReader.close();

The variable postData needs now to be parsed to extract the couples (key= value).

You can take a look at this page: http://www.jmarshall.com/easy/http/#http1.1c1
which concisely describes the HTTP protocol.

Hope it helps,

Cheers,

Francois

[ September 04, 2007: Message edited by: Francois Dang Ngoc ]
[ September 04, 2007: Message edited by: Francois Dang Ngoc ]
omkar patkar
Ranch Hand

Joined: Aug 25, 2005
Posts: 231
Thanks Man ! ... i will try this out now and get back to you !
omkar patkar
Ranch Hand

Joined: Aug 25, 2005
Posts: 231
Hi Francois, Sorry for the late reply. Good news, your tip worked MAN ! ... THANKS Francois ! Thanks a lot!

I am getting closer at making the proxy server. My intention of making the application was to track the request- response made through browser by the user, so that user can make use of this meta information for developing his/her own web-application. But i am again stuck with some issues !

Let us consider this scenario in following order :-

1) User FIRST hits a url in the browser. (HTTP Request 1)
2) He gets a form page of that site lets say. (HTTP Response 1)
3) He fills the form and clicks on submit (HTTP Request 2)
4) He gets a corresponding result page. (HTTP Response 2)

As you can see in the above, COMPLETE scenario, there are 2 pairs of Request and Response involved. With the current level of the proxy server i have made ... i can only catch the HTTP Request 1, thats it ! Now using the internet connection i want to FORWARD the HTTP Request 1 to the appropriate host and get the response (HTTP Response 1) and forward it back to the browser who made the HTTP Request 1.

Now the issues are :-
1) How do i forward the actual request and response ?
------> I have thought of using the
------>

where in the "<url _string>" will come from the meta information we have just intercepted. But what about the request parameters and other meta information that might be required by the HOST server ? how do i send that to the desired HOST server ?

2) After getting the response in the inputStream, how do i forward this response back to the browser who made the request ?

3) Can the HttpRequest and HttpResponse and Session objects available in java be of any help to us ?

I would be very greatful, if you can guide me in this regard, Francois !


Thanks and Regards
mkar Patkar
Francois Dang Ngoc
Greenhorn

Joined: Sep 02, 2007
Posts: 6
Hi Omkar,

> 1) How do i forward the actual request and response ?

First you need to extract the hostname to connect to. In the HTTP
request, you have something like:
GET /something HTTP/1.1
Host: www.yahoo.com
...

So we need to parse the line containing Host and extract www.yahoo.com.
and then open a connection and forward everything to the host www.yahoo.com.

2) After getting the response in the inputStream, how do i forward this response back to the browser who made the request ?

you can do a loop like this:

// forward the response from the server to the browser
InputStream sis = hostSocket.getInputStream();
OutputStream bos = clientSocket.getOutputStream();

System.out.println("Forwarding request from server");

do {
n = sis.read(buffer);
if (n > 0) {
bos.write(buffer, 0, n);
}
} while (n > 0);


3) Can the HttpRequest and HttpResponse and Session objects available in java be of any help to us ?

Are you referring to those used in servlets? If you write your program in a servlet, that may be useful.

Below, you find the code of a simple proxy.

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.ServerSocket;
import java.net.Socket;

public class SimpleProxyServer {

public static final int portNumber = 5555;

public static void main(String[] args){
SimpleProxyServer proxyServer = new SimpleProxyServer();
proxyServer.start();

}
public void start(){
System.out.println("Starting the SimpleProxyServer ...");
try {

ServerSocket serverSocket = new ServerSocket(MyProxyServer.portNumber,1);
byte[] buffer = new byte[10000];

while(true){
Socket clientSocket = serverSocket.accept();

InputStream bis = clientSocket.getInputStream();

// reading the request and put it into buffer
int n = bis.read(buffer);
String browserRequest = new String(buffer, 0, n);
System.out.println(browserRequest);

// extract the host to connect to
int start = browserRequest.indexOf("Host: ") + 6;
int end = browserRequest.indexOf('\n', start);
String host = browserRequest.substring(start, end - 1);
System.out.println("Connecting to host " + host);

// forward the response from the proxy to the server
Socket hostSocket = new Socket(host, 80);
OutputStream sos = hostSocket.getOutputStream();
System.out.println("Forwarding request to server");
sos.write(buffer, 0, n);
sos.flush();

// forward the response from the server to the browser
InputStream sis = hostSocket.getInputStream();
OutputStream bos = clientSocket.getOutputStream();

System.out.println("Forwarding request from server");

do {
n = sis.read(buffer);
System.out.println("Receiving " + n + " bytes");
if (n > 0) {
bos.write(buffer, 0, n);
}
} while (n > 0);

bos.flush();
hostSocket.close();
clientSocket.close();
System.out.println("End of communication");

}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}

Hope it helps,

Cheers,

Francois
Tejas Parlikar
Greenhorn

Joined: Nov 19, 2010
Posts: 1
Francois Dang Ngoc wrote:Hi Omkar,

> 1) How do i forward the actual request and response ?

First you need to extract the hostname to connect to. In the HTTP
request, you have something like:
GET /something HTTP/1.1
Host: www.yahoo.com
...

So we need to parse the line containing Host and extract www.yahoo.com.
and then open a connection and forward everything to the host www.yahoo.com.

2) After getting the response in the inputStream, how do i forward this response back to the browser who made the request ?

you can do a loop like this:

// forward the response from the server to the browser
InputStream sis = hostSocket.getInputStream();
OutputStream bos = clientSocket.getOutputStream();

System.out.println("Forwarding request from server");

do {
n = sis.read(buffer);
if (n > 0) {
bos.write(buffer, 0, n);
}
} while (n > 0);


3) Can the HttpRequest and HttpResponse and Session objects available in java be of any help to us ?

Are you referring to those used in servlets? If you write your program in a servlet, that may be useful.

Below, you find the code of a simple proxy.

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.ServerSocket;
import java.net.Socket;

public class SimpleProxyServer {

public static final int portNumber = 5555;

public static void main(String[] args){
SimpleProxyServer proxyServer = new SimpleProxyServer();
proxyServer.start();

}
public void start(){
System.out.println("Starting the SimpleProxyServer ...");
try {

ServerSocket serverSocket = new ServerSocket(MyProxyServer.portNumber,1);
byte[] buffer = new byte[10000];

while(true){
Socket clientSocket = serverSocket.accept();

InputStream bis = clientSocket.getInputStream();

// reading the request and put it into buffer
int n = bis.read(buffer);
String browserRequest = new String(buffer, 0, n);
System.out.println(browserRequest);

// extract the host to connect to
int start = browserRequest.indexOf("Host: ") + 6;
int end = browserRequest.indexOf('\n', start);
String host = browserRequest.substring(start, end - 1);
System.out.println("Connecting to host " + host);

// forward the response from the proxy to the server
Socket hostSocket = new Socket(host, 80);
OutputStream sos = hostSocket.getOutputStream();
System.out.println("Forwarding request to server");
sos.write(buffer, 0, n);
sos.flush();

// forward the response from the server to the browser
InputStream sis = hostSocket.getInputStream();
OutputStream bos = clientSocket.getOutputStream();

System.out.println("Forwarding request from server");

do {
n = sis.read(buffer);
System.out.println("Receiving " + n + " bytes");
if (n > 0) {
bos.write(buffer, 0, n);
}
} while (n > 0);

bos.flush();
hostSocket.close();
clientSocket.close();
System.out.println("End of communication");

}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}

Hope it helps,

Cheers,

Francois



Hi,
I am also trying to do same Proxy Server prog.
your information was very much helpful to me. Pleasure with that.
Can you please provide me some tips, I am in trouble.
I got problem while running the code, I am able to see the request for Google and Yahoo only.
The other web sites are not working with this code. I have used above code.

Please, if possible provide me tips and solution, It would definetely be very much helpful to me.
Thank you.

Tejas Parlikar.

Alexander Hakim
Greenhorn

Joined: Oct 04, 2014
Posts: 11
Hello every one;

Can we modify the above web proxy server to Block requests for undesirable URLs, using HTTP redirection to display an error page. How can we do that?

Regards/
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Proxy Server in Java