TLS certificate validation

[Matt Wong]

Ok, so as I'm currently on to my road to implement Monopoly - I already thought about its network-layer and came up with something unusual - but straight forward: TLS and it's feature for client authentication. So, instead of username and password a user generates an RSA keypair - generates an CSR - receives an client-certificate and uses this as auth-token (all done in the background with a ton of bouncycastle-magic). As Monopoly isn't the typical game to cheat - any network-connection is - at least in terms of sniffing and modifying it.

What? Yea - imagine this: someone wants to somehow take advantage by manipulation of the data exchanged between client an server (would make any sense as I plan to check every action happens on server-side). But if someone really tries to find some bug - I'm sure one will find one. So I looked for an easy way to block of such "users" - TLS client certificate and a list wich prevents those to get a new certificate.

To make this happen - a correct validation is needed - and here it starts where Java itself has at least something built-in - but BC lacks all of it and one has to do anything itself. There's just implementations to check if a presented server-certificate is signed by a trusted root-certificate - but there's no validation (neither CRL nor OCSP) - and if you set up a server with client auth - well, there's simply no checks at all - not even if the certificate replied by the client matches with any root-certificate requested from the server.

So I dived deep into all those magic crypto stuff - and tried to build some validation code - but as nobody ever did this on the net - there're no resources google could deliver. Here's a small stub:

import java.security.KeyStore;
import java.io.FileInputStream;
import javax.net.ssl.*;
import java.security.SecureRandom;
import java.security.cert.*;
import javax.security.auth.x500.X500Principal;
public final class Test
{
	public final static void main(final String... args) throws Exception
	{
		String caFile=System.getProperty("java.home")+"/lib/security/cacerts";
		KeyStore caStore=KeyStore.getInstance("jks");
		caStore.load(new FileInputStream(caFile), "changeit".toCharArray());
		SSLContext sslContext=SSLContext.getInstance("TLSv1.2");
		TrustManagerFactory trustManagerFactory=TrustManagerFactory.getInstance("X509");
		trustManagerFactory.init(caStore);
		TrustManager[] trustManagers=trustManagerFactory.getTrustManagers();
		SecureRandom secureRandom=SecureRandom.getInstance("SHA1PRNG");
		sslContext.init(null, trustManagers, secureRandom);
		SSLSocketFactory sslSocketFactory=sslContext.getSocketFactory();
		SSLSocket sslSocket=(SSLSocket)sslSocketFactory.createSocket("cryptearth.de", 443);
		sslSocket.startHandshake();
		SSLSession sslSession=sslSocket.getSession();
		Certificate[] certChain=sslSession.getPeerCertificates();
		X509Certificate chain=(X509Certificate)certChain[certChain.length-1];
		X500Principal issuerPrincipal=chain.getIssuerX500Principal();
		X509CertSelector x509CertSelector=new X509CertSelector();
		x509CertSelector.setSubject(issuerPrincipal);
		PKIXBuilderParameters pkixBuilderParameters=new PKIXBuilderParameters(caStore, x509CertSelector);
		pkixBuilderParameters.setRevocationEnabled(true);
		CertPathBuilder certPathBuilder=CertPathBuilder.getInstance("PKIX");
		PKIXRevocationChecker pkixRevocationChecker=(PKIXRevocationChecker)certPathBuilder.getRevocationChecker();
		pkixBuilderParameters.addCertPathChecker(pkixRevocationChecker);
		CertPathBuilderResult certPathBuilderResult=certPathBuilder.build(pkixBuilderParameters);
		PKIXCertPathBuilderResult pkixCertPathBuilderResult=(PKIXCertPathBuilderResult)certPathBuilderResult;
		
		System.out.println(pkixCertPathBuilderResult);
		
		/*CertPath certPath=pkixCertPathBuilderResult.getCertPath();
		
		System.out.println(certPath);
		
		/*CertPathValidator certPathValidator=CertPathValidator.getInstance("PKIX");
		PKIXParameters pkixParameters=new PKIXParameters(caStore);
		pkixParameters.setRevocationEnabled(true);
		pkixRevocationChecker=(PKIXRevocationChecker)certPathValidator.getRevocationChecker();
		pkixParameters.addCertPathChecker(pkixRevocationChecker);
		CertPathValidatorResult certPathValidatorResult=certPathValidator.validate(certPath, pkixParameters);
		
		System.out.println(certPathValidatorResult);*/
	}
}

//java.security.InvalidAlgorithmParameterException: Could not determine unique target subject

The last sysout for the BuilderResult shows somewhere the let's encrypt root-cert gets correctly identified - but the resulting CertPath is empty:

D:\data\java>java Test
PKIXCertPathBuilderResult: [
  Certification Path:
X.509 Cert Path: length = 0.
[

]
  Trust Anchor: [
  Trusted CA cert: [
[
  Version: V3
  Subject: CN=DST Root CA X3, O=Digital Signature Trust Co.
  Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5

  Key:  Sun RSA public key, 2048 bits
  modulus: 282378876770260322031517776571295615815220730604012338518941879525956
40780665579499663841407267510759260214748789212535957135845654219821366017427323
98535210017221162896155164717876527846524504061999428631663085221092818434609096
19063671380967157660331712611073134327722994678199366786341097089673788290134186
49505942485529500580167736159568208924601034682852941882633722952597854385181938
55768286513954563628268986245989702763251191607242145921038098795454972453662349
40643939730521864489775709894939986854040144737156887966075431399146693072344409
05936555495044671225489918726010863829142065064843131427399159251549
  public exponent: 65537
  Validity: [From: Sat Sep 30 23:12:19 CEST 2000,
               To: Thu Sep 30 16:01:15 CEST 2021]
  Issuer: CN=DST Root CA X3, O=Digital Signature Trust Co.
  SerialNumber: [    44afb080 d6a327ba 89303986 2ef8406b]

Certificate Extensions: 3
[1]: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
  CA:true
  PathLen:2147483647
]

[2]: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  Key_CertSign
  Crl_Sign
]

[3]: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: C4 A7 B1 A4 7B 2C 71 FA   DB E1 4B 90 75 FF C4 15  .....,q...K.u...
0010: 60 85 89 10                                        `...
]
]

]
  Algorithm: [SHA1withRSA]
  Signature:
0000: A3 1A 2C 9B 17 00 5C A9   1E EE 28 66 37 3A BF 83  ..,...\...(f7:..
0010: C7 3F 4B C3 09 A0 95 20   5D E3 D9 59 44 D2 3E 0D  .?K.... ]..YD.>.
0020: 3E BD 8A 4B A0 74 1F CE   10 82 9C 74 1A 1D 7E 98  >..K.t.....t....
0030: 1A DD CB 13 4B B3 20 44   E4 91 E9 CC FC 7D A5 DB  ....K. D........
0040: 6A E5 FE E6 FD E0 4E DD   B7 00 3A B5 70 49 AF F2  j.....N...:.pI..
0050: E5 EB 02 F1 D1 02 8B 19   CB 94 3A 5E 48 C4 18 1E  ..........:^H...
0060: 58 19 5F 1E 02 5A F0 0C   F1 B1 AD A9 DC 59 86 8B  X._..Z.......Y..
0070: 6E E9 91 F5 86 CA FA B9   66 33 AA 59 5B CE E2 A7  n.......f3.Y[...
0080: 16 73 47 CB 2B CC 99 B0   37 48 CF E3 56 4B F5 CF  .sG.+...7H..VK..
0090: 0F 0C 72 32 87 C6 F0 44   BB 53 72 6D 43 F5 26 48  ..r2...D.SrmC.&H
00A0: 9A 52 67 B7 58 AB FE 67   76 71 78 DB 0D A2 56 14  .Rg.X..gvqx...V.
00B0: 13 39 24 31 85 A2 A8 02   5A 30 47 E1 DD 50 07 BC  .9$1....Z0G..P..
00C0: 02 09 90 00 EB 64 63 60   9B 16 BC 88 C9 12 E6 D2  .....dc`........
00D0: 7D 91 8B F9 3D 32 8D 65   B4 E9 7C B1 57 76 EA C5  ....=2.e....Wv..
00E0: B6 28 39 BF 15 65 1C C8   F6 77 96 6A 0A 8D 77 0B  .(9..e...w.j..w.
00F0: D8 91 0B 04 8E 07 DB 29   B6 0A EE 9D 82 35 35 10  .......).....55.

]

  Policy Tree: anyPolicy  ROOT

  Subject Public Key: Sun RSA public key, 2048 bits
  modulus: 282378876770260322031517776571295615815220730604012338518941879525956
40780665579499663841407267510759260214748789212535957135845654219821366017427323
98535210017221162896155164717876527846524504061999428631663085221092818434609096
19063671380967157660331712611073134327722994678199366786341097089673788290134186
49505942485529500580167736159568208924601034682852941882633722952597854385181938
55768286513954563628268986245989702763251191607242145921038098795454972453662349
40643939730521864489775709894939986854040144737156887966075431399146693072344409
05936555495044671225489918726010863829142065064843131427399159251549
  public exponent: 65537
]

So, if you try to run the validation part - it fails cause the CertPath is empty and therefore no validation can happen. And if this doesn't work on the client side for server certs - how ever should it work on the server side for client-certs?
Using BC and write all validators myself may be an option (as BC is really not useable at all when not using the BC-provider but the lightweight-api) - but this could be as error-prone as the few lines I was able to put together with standard SE api.

Is it really that un-common to use Java for such things? If so - it has to be done in other languages as I guess client-auth is used in many applications. Can't imagine to be to that hard to port it to Java. Or is Java itself just not capable of such tasks and one has to write its own stub with libs like BC (let alone implement all this crypto-stuff by hand - wich WILL lead to security holes)?

[Matt Wong]

Ok, problem solved - it all starts way earlier than I thought - right at init of SSLContext:

		String caFile=System.getProperty("java.home")+"/lib/security/cacerts";
		KeyStore caStore=KeyStore.getInstance("jks");
		caStore.load(new FileInputStream(caFile), "changeit".toCharArray());
		SSLContext sslContext=SSLContext.getInstance("TLSv1.2");
		TrustManagerFactory trustManagerFactory=TrustManagerFactory.getInstance("X509");
		CertPathBuilder certPathBuilder=CertPathBuilder.getInstance("PKIX");
		PKIXRevocationChecker pkixRevocationChecker=(PKIXRevocationChecker)certPathBuilder.getRevocationChecker();
		pkixRevocationChecker.setOptions(EnumSet.of(PKIXRevocationChecker.Option.PREFER_CRLS,
								PKIXRevocationChecker.Option.NO_FALLBACK));
		PKIXBuilderParameters pkixBuilderParameters=new PKIXBuilderParameters(caStore, new X509CertSelector());
		pkixBuilderParameters.setRevocationEnabled(false);
		pkixBuilderParameters.addCertPathChecker(pkixRevocationChecker);
		CertPathTrustManagerParameters certPathTrustManagerParameters=new CertPathTrustManagerParameters(pkixBuilderParameters);
		trustManagerFactory.init(certPathTrustManagerParameters);
		TrustManager[] trustManagers=trustManagerFactory.getTrustManagers();
		SecureRandom secureRandom=SecureRandom.getInstance("SHA1PRNG");
		sslContext.init(null, trustManagers, secureRandom);
		SSLSocketFactory sslSocketFactory=sslContext.getSocketFactory();
		SSLSocket sslSocket=(SSLSocket)sslSocketFactory.createSocket("cryptearth.de", 443);
		sslSocket.startHandshake();

So the main difference is the initialization of the TrustManagerFactory: instead of using init(KeyStore) one uses init(ManagerFactoryParameters) to simply wrap a PKIXBuilderParameters, wich, as a subclass of PKIXParameters, one can add a PKIXCertPathChecker.
Also: the setRevocationEnabled(boolean) method is only for provider-internal checker - and is ommited when a custom checker is set/added, and, as noted in the docs, should be set to false when using a custom checker to prevent issues with provider-internal checker.
To disable revocation checking set the flag to false and don't add a custom checker. To use default OCSP one doesn't need to do at all as it's enabled by default in JSSE. To use CRL use code above.

Strangly: as I tested this with my server I could verify with wireshark that java was loading the CRL - but somehow couldn't verify it. When I tested with google, it worked fine. So either something's wrong with Let's Encrypt - or my server. Will test it with custom CA.

[Matt Wong]

Ok, so it took me a week to get a full CA up with some bouncycastle-magic - but it works now.

On top of my CA there's my root.key, a self-signed root.crt, and a root.crl to revoke the following intermediate certs.
On 2nd level are the intermediate certs with intermediate keys and crls, namely server-intermediate.key/.crt/.crl and client-intermediate.key/.crt/.crl.
On the lowest level are on one side the server cert with its key - and on the other side are the client certs with thier keys.

So now what happens on connection? Apart from some client-cert-stuff not much else than normal server-side-only TLS connections. The special key here is that the server is set to require the client to awnser with a certificate signed by the root-ca. As certificates are in a chain, the server only sends the root-ca wich it trusts - in my case only my own root. Then the client itself sends either a certificate directly signed by the root-cert - or a chain of certs wich ends up on the root-ca.
Both sides are setup to validate with CRL. So when a client connects and receives server cert chain (root is loaded as a trusted ca-cert on client) it follows the chain from top down - so firstly the server-intermediate cert is checked agains root-crl, if this passes then the server cert is checked agains server-intermediate-crl. Then the client receives the client-cert-requirement from server and responds with a chain itself. The server then does the same - and only if all certs and crls are valid by the root-cert - the connection is established.
So, to exclude someone to connect - I can decide to either no issue a client-cert in the first place (wich will be an automated process during registering) - or, if a certificate is already issued - I add its serial-number to the client-intermediate.crl - and the connection is terminated during handshake.

In the end - here's the code - if someone wants to know how to setup a CA with BC in Java - just let me know.

import java.security.interfaces.RSAPrivateCrtKey;
import java.security.KeyFactory;
import java.security.spec.PKCS8EncodedKeySpec;
import org.bouncycastle.util.io.pem.PemReader;
import java.io.FileReader;
import java.security.cert.X509Certificate;
import java.security.cert.CertificateFactory;
import java.io.FileInputStream;
import java.security.KeyStore;
import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.KeyManager;
import javax.net.ssl.TrustManagerFactory;
import javax.net.ssl.TrustManager;
import java.security.SecureRandom;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLServerSocketFactory;
import javax.net.ssl.SSLServerSocket;
import javax.net.ssl.SSLSocket;
import java.security.cert.CertPathBuilder;
import java.security.cert.PKIXRevocationChecker;
import java.util.EnumSet;
import java.security.cert.PKIXBuilderParameters;
import java.security.cert.X509CertSelector;
import javax.net.ssl.CertPathTrustManagerParameters;
public final class TestServer
{
	public final static void main(final String... args) throws Exception
	{
		RSAPrivateCrtKey rsaPrivateCrtKey=(RSAPrivateCrtKey)KeyFactory.getInstance("RSA").generatePrivate(new PKCS8EncodedKeySpec((new PemReader(new FileReader("server.key"))).readPemObject().getContent()));
		char[] password="secret".toCharArray();
		X509Certificate serverCertificate=(X509Certificate)CertificateFactory.getInstance("X509").generateCertificate(new FileInputStream("server.crt"));
		X509Certificate intermediateCertificate=(X509Certificate)CertificateFactory.getInstance("X509").generateCertificate(new FileInputStream("serverinter.crt"));
		X509Certificate[] certificateChain=new X509Certificate[]{ serverCertificate, intermediateCertificate };
		KeyStore keyStore=KeyStore.getInstance("JKS");
		keyStore.load(null, null);
		keyStore.setKeyEntry("server", rsaPrivateCrtKey, password, certificateChain);
		KeyManagerFactory keyManagerFactory=KeyManagerFactory.getInstance("PKIX");
		keyManagerFactory.init(keyStore, password);
		KeyManager[] keyManagers=keyManagerFactory.getKeyManagers();
		X509Certificate rootCertificate=(X509Certificate)CertificateFactory.getInstance("X509").generateCertificate(new FileInputStream("root.crt"));
		KeyStore trustStore=KeyStore.getInstance("JKS");
		trustStore.load(null, null);
		trustStore.setCertificateEntry("root", rootCertificate);
		TrustManagerFactory trustManagerFactory=TrustManagerFactory.getInstance("X509");
		CertPathBuilder certPathBuilder=CertPathBuilder.getInstance("PKIX");
		PKIXRevocationChecker pkixRevocationChecker=(PKIXRevocationChecker)certPathBuilder.getRevocationChecker();
		pkixRevocationChecker.setOptions(EnumSet.of(PKIXRevocationChecker.Option.PREFER_CRLS, PKIXRevocationChecker.Option.NO_FALLBACK));
		PKIXBuilderParameters pkixBuilderParameters=new PKIXBuilderParameters(trustStore, new X509CertSelector());
		pkixBuilderParameters.setRevocationEnabled(false);
		pkixBuilderParameters.addCertPathChecker(pkixRevocationChecker);
		CertPathTrustManagerParameters certPathTrustManagerParameters=new CertPathTrustManagerParameters(pkixBuilderParameters);
		trustManagerFactory.init(certPathTrustManagerParameters);
		TrustManager[] trustManagers=trustManagerFactory.getTrustManagers();
		SecureRandom secureRandom=SecureRandom.getInstance("SHA1PRNG");
		SSLContext sslContext=SSLContext.getInstance("TLSv1.2");
		sslContext.init(keyManagers, trustManagers, secureRandom);
		SSLServerSocketFactory sslServerSocketFactory=sslContext.getServerSocketFactory();
		SSLServerSocket sslServerSocket=(SSLServerSocket)sslServerSocketFactory.createServerSocket(2016);
		sslServerSocket.setNeedClientAuth(true);
		SSLSocket sslSocket=(SSLSocket)sslServerSocket.accept();
		sslSocket.startHandshake();
	}
}

import java.security.KeyStore;
import java.security.cert.CertificateFactory;
import java.io.FileInputStream;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManagerFactory;
import java.security.cert.CertPathBuilder;
import java.security.cert.PKIXRevocationChecker;
import java.util.EnumSet;
import java.security.cert.PKIXBuilderParameters;
import java.security.cert.X509CertSelector;
import javax.net.ssl.CertPathTrustManagerParameters;
import javax.net.ssl.TrustManager;
import java.security.SecureRandom;
import javax.net.ssl.SSLSocketFactory;
import javax.net.ssl.SSLSocket;
import java.security.interfaces.RSAPrivateCrtKey;
import java.security.spec.PKCS8EncodedKeySpec;
import java.security.KeyFactory;
import java.security.cert.X509Certificate;
import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.KeyManager;
import org.bouncycastle.util.io.pem.PemReader;
import java.io.FileReader;
public final class ConTest
{
	public final static void main(final String... args) throws Exception
	{
		KeyStore keyStore=KeyStore.getInstance("JKS");
		keyStore.load(null, null);
		RSAPrivateCrtKey rsaPrivateCrtKey=(RSAPrivateCrtKey)KeyFactory.getInstance("RSA").generatePrivate(new PKCS8EncodedKeySpec((new PemReader(new FileReader("client.key"))).readPemObject().getContent()));
		X509Certificate clientCertificate=(X509Certificate)CertificateFactory.getInstance("X509").generateCertificate(new FileInputStream("client.crt"));
		X509Certificate intermediateCertificate=(X509Certificate)CertificateFactory.getInstance("X509").generateCertificate(new FileInputStream("clientinter.crt"));
		X509Certificate[] clientChain=new X509Certificate[] { clientCertificate, intermediateCertificate };
		char[] password="password".toCharArray();
		keyStore.setKeyEntry("client", rsaPrivateCrtKey, password, clientChain);
		KeyManagerFactory keyManagerFactory=KeyManagerFactory.getInstance("PKIX");
		keyManagerFactory.init(keyStore, password);
		KeyManager[] keyManagers=keyManagerFactory.getKeyManagers();
		KeyStore caStore=KeyStore.getInstance("JKS");
		caStore.load(null, null);
		caStore.setCertificateEntry("root", CertificateFactory.getInstance("X509").generateCertificate(new FileInputStream("root.crt")));
		SSLContext sslContext=SSLContext.getInstance("TLSv1.2");
		TrustManagerFactory trustManagerFactory=TrustManagerFactory.getInstance("X509");
		CertPathBuilder certPathBuilder=CertPathBuilder.getInstance("PKIX");
		PKIXRevocationChecker pkixRevocationChecker=(PKIXRevocationChecker)certPathBuilder.getRevocationChecker();
		pkixRevocationChecker.setOptions(EnumSet.of(PKIXRevocationChecker.Option.PREFER_CRLS, PKIXRevocationChecker.Option.NO_FALLBACK));
		PKIXBuilderParameters pkixBuilderParameters=new PKIXBuilderParameters(caStore, new X509CertSelector());
		pkixBuilderParameters.setRevocationEnabled(false);
		pkixBuilderParameters.addCertPathChecker(pkixRevocationChecker);
		CertPathTrustManagerParameters certPathTrustManagerParameters=new CertPathTrustManagerParameters(pkixBuilderParameters);
		trustManagerFactory.init(certPathTrustManagerParameters);
		TrustManager[] trustManagers=trustManagerFactory.getTrustManagers();
		SecureRandom secureRandom=SecureRandom.getInstance("SHA1PRNG");
		sslContext.init(keyManagers, trustManagers, secureRandom);
		SSLSocketFactory sslSocketFactory=sslContext.getSocketFactory();
		SSLSocket sslSocket=(SSLSocket)sslSocketFactory.createSocket("127.0.0.1", 2016);
		sslSocket.startHandshake();
	}
}

Oh, and I also discovered why one can't validate let's encrypt certificates with CRL - cause client-certs are not issued with CRL but only OCSP, that's why CRL check fails.

[Stephan van Hulst]

This isn't done regularly because there's no need. You just pick a secure transport protocol to communicate between the client and the server. You're reinventing the wheel.

As for blocking clients, what are you gonna block them by? Do they have accounts?

Lastly, if you think a dedicated cheater will find a bug in your game, they won't find a bug in your hand-rolled security? Just focus on making it impossible to cheat on the first place. Let the server roll the dice, and perform all the game logic. All that the client has to do is decide whether they want to buy the current property, or perform other actions like buying hotels for properties they already own, or initiating trade. All of these can be verified by the server.

If you just want to get experience with TLS, it's better to use it in a project where you design your own transport protocol, not at the application layer.

[Tim Holloway]

Is this a webapp or a stand-alone server? If it's a webapp, then no application code is required to implement SSL/TLS. Everything is already there, whether you're dealing with the more common server-side certs or authenticating with a client-side cert. It's all a matter of setting the webapp server (e.g., Tomcat) security options as desired.

In fact, the stock https transport mechanisms for Java incoporate all that, including negotiation between client and server to find a mutually-agreeable encryption scheme.

Only if you are doing raw socket communications do you need to get personally involved at that level, and I suspect that in many cases, not even then.

Although considering firewall limitations on a lot of networks, you're best not using custom protocols and ports anyway.