Search...
FAQs Subscribe
Pie
FAQs
Recent topics Flagged topics Hot topics Best topics
Search...
Search within Web Services Search Coderanch
Advance search Google search
Register / Login
polymorphism advertise on coderanch book review grid thread boost granny
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
advertise on coderanch thread boost building better world book book review grid meaningless drivel
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • Jeanne Boyarsky
  • Tim Cooke
Sheriffs:
  • Liutauras Vilda
  • paul wheaton
  • Henry Wong
Saloon Keepers:
  • Tim Moores
  • Tim Holloway
  • Stephan van Hulst
  • Carey Brown
  • Frits Walraven
Bartenders:
  • Piet Souris
  • Himai Minh

How to implement certificate/security into web service client

 
Kjeld Sigtermans
Ranch Hand
Posts: 127
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Hello,

I am looking for a code example which shows how to access a secure web service over SSL using a PFX certificate.
I have the certificate and its password and I started by creating a KeyStore instance

I am not sure what needs to go between (...) and where do I go from here?

Cheers,
Kjeld


Kjeld Sigtermans - SCJP 1.4 - SCWCD 1.4

 
R Srini
Ranch Hand
Posts: 215
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Hi. As you know, the parameter to Keystore.getInstance() is the keystore type.

Please see this tutorial for a complete example of SSL client and server programs. In this tutorial, look for the command "keytool -list -keystore. That should tell you the type of your keystore that you need to use as the parameter.

But before that ... you may need to convert your PFX to a JKS file. Hope that helps.
 
Kjeld Sigtermans
Ranch Hand
Posts: 127
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Hi,

Thanks for this information.
I need to access the remote web service of a business partner. Meanwhile a JKS certificate was generated which I can use now. I have tried to connect to the web services using SoapUI and I can succesfully access them when configuring SoapUI to use this JKS certificate.

But I cannot access the web services using my java code.
I generated client code using JAX-WS wsimport and I created a java class which uses the generated classes to access the remote service.
Because the remote service is secured, I now need to add the part that uses the certificate, so the web services can succesfully be accessed.

I have tried these lines of code much like your example:

This code is executed ok but when sending a request to a web service I get an exception:


Where do I go from here?

Thanks,
Kjeld

Kjeld Sigtermans - SCJP 1.4 - SCWCD 1.4

 
Kjeld Sigtermans
Ranch Hand
Posts: 127
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
P.S. when changing the code to:

The exception changes to:

Kjeld Sigtermans - SCJP 1.4 - SCWCD 1.4

 
R Srini
Ranch Hand
Posts: 215
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Hi. So that is good that you can connect from SOAP UI. Some thoughts/observations/questions:

1. I'd suggest moving the certificateFileInputStream.close() to the end. Maybe it does a lazy load and so keyStore.load hasn't really loaded it yet?
2. We are seeing a InvalidAlgorithmParameterException, which is in the declaration for this method:
init(ManagerFactoryParameters spec) throws InvalidAlgorithmParameterException
However, the tutorial is using this:
init(KeyStore ks, char[] password) throws KeyStoreException, NoSuchAlgorithmException, UnrecoverableKeyException
Which one are you calling? Method declarations: http://java.sun.com/javase/6/docs/api/javax/net/ssl/KeyManagerFactory.html#init(java.security.KeyStore, char[])
3. Search the Oracle Forums for "trustAnchors" and lots of solutions seem to be there.
4. Also searh the same forum for "ValidatorException: No trusted certificate found"
5. If you still have trouble, then you could get the tutorial to work first, separately, and independent of your program. That should clarify a lot of things.
 
Kjeld Sigtermans
Ranch Hand
Posts: 127
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Hi thanks.

I tried a lot of things yesterday, a.o. adding the remote certificate to a trust store which I hadn't before.
I think we're a lot further in the process of getting this to work, but now another exception takes place which I like to focus this topic on: java.net.SocketException: Unexpected end of file from server
This exception seems to occur during the handshake process. We set our JVM to debug level for SSL type activities (-Djavax.net.debug=ssl), I'll post the stacktrace here from the point where things go wrong:


However, when sending a bigger request (around 50k instead of 3k) the exception changes into org.jboss.ws.WSException: Invalid HTTP server response [413] - Request Entity Too Large

I asked our business partner who has the web service running /act as the server side, and they did not notice any weird stuff in their logs around that time.
Hope someone has an idea of what seems to be going on here.
Note that everything goes smoothly when testing the WS with SoapUI and using the same certificate.

Cheers,
Kjeld

Kjeld Sigtermans - SCJP 1.4 - SCWCD 1.4

 
R Srini
Ranch Hand
Posts: 215
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Hi Kjeld. Yes, progress is good.

So the client receives an EOFException immediately after sending the security data "WRITE: TLSv1 Application Data". A few of things:

1. Can you confirm that the SSL port number you are using is correct? Just making sure.
2. Look here for possible reasons for unexpected end of file. Maybe one of these is applicable.
3. Can you post the main 20 to 25 lines of code (including the cert file open, the handshake, etc.) so I can try it out myself with my certs? I am a little busy today, so I will try if I can get some time.
4. In case you haven't already ... Use a soap monitor like tcpmon to view the SOAP request being sent by the Java client. And then compare the request with the one from SOAP UI (which should already have the soap monitor, but you have to configure it. Simple to do.)

Thanks

Srini
 
R Srini
Ranch Hand
Posts: 215
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Also, if possible, you can request a server-side SSL trace. That should provide some information as well.

Another interesting link mentions that reading from the socket before writing may be a problem. Not sure if this applies to you.
 
Kjeld Sigtermans
Ranch Hand
Posts: 127
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Hi thanks for taking the time to help, I really appreciate it.
@1 looking into that
@2 looked into that already (right now having other people looking into it)
@3 not really sure what you mean here: you need the full stacktrace? what is it you want to try with your certs?
@4 Actually did this today, I was able to extract the generated request and used it in SoapUI: was able to send it and receive response with no problem!
@reading from the socket before writing issue: also passed this one to guys here who are more experienced with this stuff. Hope to get a response from them tomorrow.

What puzzles me the most is how SoapUI is able to get it done. I even tried looking into the SoapUI source code, but that's a needle-in-the-haystack-type of thing to do.
Hope to talk to the serverside people (again) tomorrow and see if they will give me a SSL trace.

Cheers Kjeld

Kjeld Sigtermans - SCJP 1.4 - SCWCD 1.4

 
R Srini
Ranch Hand
Posts: 215
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
@3: I don't mean the stacktrace. I mean your actual code that is resulting in the EOFException/unexpected end of file. If you can post the actual code that you are using for the handshake, then I can try the exact same code with my certificates locally. Maybe I can duplicate the error? But if you have multiple persons working on it, I have no doubt that you will find the problem
 
Kjeld Sigtermans
Ranch Hand
Posts: 127
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Well I hope you want to have a go at it. I am trying two things:
First only setting these system properties:
And secondly: both the above + this code:
Either option will result in end of file (when request is +/- 3kB) or Http error 413 request too big (when request is +/- 51kB), see the stacktraces I posted hereabove.
Note that I have not yet tried to move the close() instruction to the end of the code block.
Also note that the above code is in the constructor of my class which executes without problems. The exceptions occur only when actually doing the call to the webservice.

The variables trustStoreFile and certificateFile are both of type java.io.File and contain absolute paths to the files.
We created a truststore file called trustedCertificates using:
Hope to hear from you.
Kjeld

Kjeld Sigtermans - SCJP 1.4 - SCWCD 1.4

 
R Srini
Ranch Hand
Posts: 215
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Hi. I tried out your code, and simply put, it worked fine for me. Here are my observations:

1. The "certificateFileInputStream.close();" statement is ok where it is. My sample ran fine with your code.
2. I see that you have a trustStoreFile and a certificateFile. And both trustManagerFactory.init and keyManagerFactory.init are using the same file (certificateFile); however, from the names, it sounds like they should be using different files. Are you using the trustStoreFile at all?
3. Your posted code did not include the most important part: the socket creation and the handshake. Maybe this is what you are currently playing with. I was curious about exactly which method call results in the exception for you. The following code from that tutorial worked fine for me (resulted in a successful handshake):
Of course the host and port are most important, but if you captured the request from your client and fed it through SOAP UI and that worked, then the port should be fine. Are you using code different than the above? If yes, you could post it and I will check it out. Even better, when you have a resolution, please post that as well.

Anyway, all the best.
 
Kjeld Sigtermans
Ranch Hand
Posts: 127
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Hi,

At some point for some reason I must have removed the part that creates a socket and handshake but I changed it back this morning. This part is (still) executed with no problem, but still I get the EOF exception when actually sending the request.
I am starting to get pretty desperate here. ;-)

I tried to dig deeper into the tutorial and noticed the KeyStore object is based on the trustStore file, not the client side certificate file... I changed this so my truststore file is in the KeyStore object instead.
Still I use the System properties to set both keystore and truststore parameters.

In all cases I end up with the End Of File exception.

This is my code now:
The trustStoreFile/trustStorePassword refer to a truststore I have created on the client machine using:
The certificateFile/certificatePassword refer to a JKS certificate file on my client machine.
In the remoteHost I specified the domain from the endpoint address without protocol and paths.

This is the stacktrace.
It is especially annoying that I can access the WS just fine with SoapUI (without having to specify a trust store!) and other parties are able to access the WS with no problem as well.

Thanks,
Kjeld

Kjeld Sigtermans - SCJP 1.4 - SCWCD 1.4

 
R Srini
Ranch Hand
Posts: 215
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Ok. A few more thoughts:

1. If it worked from SOAP UI without a truststore, then I would try the client in the same way, i.e. without the trust store.
2. Lets say you have your XML request in a variable "String request;". Based on this, can you also post the code that actually sends the request by writing to an OutputStream (after the handshake is successful)?
3. What are the actual bytes you are sending? I suspect the problem might lie there. While I don't know exactly what you are sending, it may not be enough to send just the XML. Maybe you also need to send the HTTP request information, including header, number of bytes etc.? I am not sure of this though, but I feel that the server, in addition to the actual request itself, is also expecting additional bytes before and after the XML request. I wish someone with more knowledge on this would reply. Oh well.
4. Since you have a WSDL, is it an option for you to use WSDL2Java or similar tool that simply generates the needed code and build.xml? I feel that would be much quicker than writing the actual code.
5. Maybe this is applicable to you?

Once I have your code (from #2 above) that sends the request, I will try it out this weekend against an actual web service (instead of using the tutorial code that I have used so far). Haven't had the time so far.
 
R Srini
Ranch Hand
Posts: 215
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
I think you already know this but wanted to mention it anyway ... regarding #3 and #2 about the actual bytes being sent, I have been reading a bit about it, and you could use HttpClient to send the http request. Here is the HttpClient SSL Guide with some examples.
 
Kjeld Sigtermans
Ranch Hand
Posts: 127
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Hi 'R',
Just a thought: as you know, the business partner hosts the web service and they gave us a certificate which we use client side. Based on this certificate, my colleague created a jks certificate file, which I am using.
I am just wondering: doesn't the business partner need to add our generated client jks to their/server-side truststore? I am reading an article (see link herebelow) that this is necessary for mutual authentication. The 'tutorial' you posted in this topic says it 'might' be necessary also.

Just a little more information: my classes are JAX-WS generated (I used the wsimport command to create them) based on the WSDL supplied. The created packages contain a factory that delivers me a service port which methods I then access. A request is then generated under the hood and sent to the ws host.

I am now reading this: Using JAX-WS-Based Web Services with SSL
Will post more later.

Kjeld Sigtermans - SCJP 1.4 - SCWCD 1.4

 
R Srini
Ranch Hand
Posts: 215
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Hi Kjeld. Yes, the server must also import the client's certificate so that when the client presents it, the server will recognize it. Based on my understanding, I think it is necessary for mutual (2-way) authentication. Also, the JAX-WS article is good. I think once you have it working (not hard at all), the generated class should take care of the ssl connection and handshaking and you will be good to go.

- Srini
 
Kjeld Sigtermans
Ranch Hand
Posts: 127
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Could this be something, it's the exact same problem I have:
http://stackoverflow.com/questions/2596022/getting-eofexception-while-trying-to-read-from-sslsocket
The guy says:,,The problem was with the packet I was sending. The server was getting the packet, checking that it was badly formatted and dropping the connection. Fixing the packet format fixed the problem.''
What is meant by a 'packet' in this context, and how can I fix it?

Kjeld Sigtermans - SCJP 1.4 - SCWCD 1.4

 
R Srini
Ranch Hand
Posts: 215
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Yes, I think this is the problem you are facing, and its what I tried to indicate in #3 above when I was asking about the actual bytes being sent, but I wasn't sure about the solution. In that case, the guy may have been doing low-level SSL programming because the problem was finally solved by fixing the packet format. Anyway, I think the solution in your case is to use HttpClient/SSL (per the link to the guide from the earlier post).

Having said that ... I think the better approach to this whole thing is to develop your web service (per the tutorial link you have for JAX-WS), focusing on the business and technical logic and design. And then "enable" SSL communication ("<transport-guarantee>CONFIDENTIAL</transport-guarantee>") by:

1. adding an entry in web.xml (or the ejb xml file for JBoss if you are using an EJB endpoint).
2. Make sure your truststore has the server certificate
3. Make sure the server truststore has the client certificate

I think this approach is simpler than having to worry about the details of SSL communication and handshakes, etc. Simply use the generated code.
 
Kjeld Sigtermans
Ranch Hand
Posts: 127
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Hi Srini, I got it working.
...although it is not a 100% satisfying solution!

Setting the system property sun.security.ssl.allowUnsafeRenegotiation to 'true' did the trick for me.
In fact, I can remove all the code except for this part:
So, no KeyStore, SSLContext classes or handshake methods etc. unless you want the handshake to be finished at 'startup' time, before the first WS method is called.

Apparently setting sun.security.ssl.allowUnsafeRenegotiation to 'true' creates a security vulnerability, which is in its turn highly questionable.
http://java.sun.com/javase/javaseforbusiness/docs/TLSReadme.html
Although I have to discuss it with my security guy.
What do you think?

Eventually the entire initialization code (in my constructor) looks like this:

Cheers,
Kjeld


Kjeld Sigtermans - SCJP 1.4 - SCWCD 1.4

 
R Srini
Ranch Hand
Posts: 215
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Hi Kjeld. Sorry I couldn't check it out over the weekend ... too many football matches came in the way Had to watch them all.

Anyway, its good that you have something working. However, as you indicated, please do check with your security person. From the sound of it, I don't know if its a good idea to allow renegotiation, maybe under very limited circumstances inside a trusted intranet that is not exposed to the internet. But if your security guy is ok with it, then you're good to go.

Also, and maybe I should have mentioned this earlier, I have been assuming from the start that a) you want to manually write the SSL connection code, and 2) that you need mutual 2-way authentication. The only suggestion I would have is that you might consider using Netbeans/Glassfish to build a simple client that connects to the secure service. The SSL connection can be taken care of by the infrastructure without any need for special coding on your part, and you can focus on the business logic instead.

Best of luck!
 
Kjeld Sigtermans
Ranch Hand
Posts: 127
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Hi Srini,

I have been watching the games with one eye, but I was very curious about actually being able to send the business data to the web service, so I kept coding.
Today there's good news and bad news. Good news is, we won from the Danish team (I am Dutch...) and bad news is I now get this Exception:
It happens at sslSocket.startHandshake();
Could this be the client side JKS file, or the client side truststore having the server certificate? It is definately not te server certificate itself. They checked...

I think you're right about first testing a simple example. I'll look into that.

Cheers,
Kjeld



Kjeld Sigtermans - SCJP 1.4 - SCWCD 1.4

 
R Srini
Ranch Hand
Posts: 215
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Well, regarding expired certificates, this documentation says that certificates generated with keytool are valid (search for -validity option) for 90 days by default. But if someone else on the server side gave you the certificate file, maybe they need to regenerate and give you another one or somehow extend the validity of the existing one.
 
Kjeld Sigtermans
Ranch Hand
Posts: 127
posted 11 years ago
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Well I can now access the remote secure Web Service, using our Java client and our JKS file. To conclude this topic for others:

1) Make sure server-side everything is set up correctly. Our business partner is still developing and configuring the WS, which I assumed was not the case so I kept asking myself what could be the problem.

2) I have the connection working with SoapUI. Make sure to set SoapUI preferences correctly, such as Http version and Proxy settings. Configure SoapUI project with client certificate file (JKS, PFX, ...), make sure pw is correct. Also make sure to specify the certificate in the SSL section of the request properties.

3) Create a truststore file containing the server-side certificate using keytool:

Optionally when you have a pfx file instead of jks, you can convert the pfx to jks using:


4) I used METRO/JAX-WS command wsimport to create WS src files like so:

5) I used this code to first establish a handshake, where
* certificateFile is of type java.io.File and is the full absolute path+filename to our client JKS cert,
* truststoreFile is of type java.io.File and is the full absolute path+filename to our generated truststore file,
* certificatePassword and trustStorePassword are both strings containing the appropriate passwords,
* socketHost is the hostname of the service, typically the domain part of the soap address location in the WSDL, without protocol, e.g.: ws.domain.com,
* socketPort is the port of the service, in our case 443:

6) I use code like this to call the web service (pseudo code):

This concludes just about all I have done to get it working. I would like to thank you, Srini, for all your information and taking the time to help!


Kind regards,
Kjeld Sigtermans

Kjeld Sigtermans - SCJP 1.4 - SCWCD 1.4

 
R Srini
Ranch Hand
Posts: 215
posted 11 years ago
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
    Send
  • Quote
  • Report post to moderator
Amazing! Thank you for posting the final working solution too, Kjeld.

Best regards.
- Srini
 
No holds barred. And no bars holed. Except this tiny ad:
Free, earth friendly heat - from the CodeRanch trailboss
https://www.kickstarter.com/projects/paulwheaton/free-heat
reply
reply
    Bookmark Topic Watch Topic
  • New Topic
Boost this thread!
Similar Threads
More...