This week's book giveaway is in the Servlets forum.
We're giving away four copies of Murach's Java Servlets and JSP and have Joel Murach on-line!
See this thread for details.
The moose likes Java in General and the fly likes How encrpyt all files on web server so decryptable by all users' keys? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Murach's Java Servlets and JSP this week in the Servlets forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "How encrpyt all files on web server so decryptable by all users Watch "How encrpyt all files on web server so decryptable by all users New topic
Author

How encrpyt all files on web server so decryptable by all users' keys?

Robert Paris
Ranch Hand

Joined: Jul 28, 2002
Posts: 585
My company wants to encrypt all the files served by our web-based CMS tool. (They want this so that if anyone breaks through the firewall and gets control of the server, they can't open any of the files) We want a few things:
1. The key will NOT be on the server. Files can only be opened by keys sent in when the user logs in and is cached in memory only
2. New users can be added and given a different identifying key than other users (to prove it's them not another user logging in with their user info) but everyone can decrpyt and open those files (assuming they have access to them)
My idea is to do this:
* All users get a copy of the same public key to decrpyt files
* Each user also gets their own private key to be their digital signature
* When the user logs in, the use their key to encrpy their login info AND the public key that each user shares
* The server checks who this is by decrpyting with a public key (based upon the username they used - it'll be stored to that)
- The server uses their public key for decrpyting and from this gets:
-- their password
-- their copy of the public key to open files
The server, after verifying them, then caches (in a session?) the public key they sent and uses it to open any files they try to download before sending it to them.
I have a few questions on this approach:
1. What sort of performance hit will this entail?
2. Is this a good approach to the problem?
3. Can anyone think of a good way to use they public keys they have (and the encrypting of the files) to be tied in to authorization rules?
4. How do I make it so people can upload files easily through a web browser unencrypted and have them encrpyted before being stored?
The main thing is I need this to be easy from the users' perspectives and it needs to be doable through a web browser. (Yikes) Any ideas?
karl koch
Ranch Hand

Joined: May 25, 2001
Posts: 388
hi
encoding everything with public/private keys will have a great impact on performance. better idea would be to encrypt it with a symetric key and only encode the symetric key with public/private key.
if all users shae the same public key then youll have higher security risk. im no mathematician...but i think if every user sends the same key it might be easier to sniff it also replacement of the key requires you to send the new key to every user...
if you go for symetric key for data encryption, then you could give every user his own key pair and store the symetric key for every user (encoded with the users public/private key)
well i hope i this does not appear encrypted to you...

k
Peter den Haan
author
Ranch Hand

Joined: Apr 20, 2000
Posts: 3252
Robert, my first problem is really that the description of what management wants is really a combination of objective and implementation. I always try to get customers/management to formulate the security objectives, and to leave it up to me to propose a security infrastructure that achieves the objectives.
For example, I recently did a project where it was stated initially that they'd need encryption. It turned out that they actually needed authentication, integrity and non-repudiation, but no confidentiality. Implementing the encryption they'd asked for would have made their life quite a bit more difficult and expensive, e.g. in reporting. But I digress.
If the concern is merely that an attacker breaking into the machine shouldn't be able to read any of the files there, then I'd like to offer the following for consideration.
  • Unless you have something better to work with than a dumb client browser, given requirements you're faced with, the servers would need to have all the necessary ingredients to decrypt the files. Given free rein on those servers an attacker will be able to decrypt the files. The best you can do is in that case make it as hard and inconvenient as possible, for example by hosting sensitive keys on a different server than the files.
  • The scheme you're proposing seems to presuppose much more intelligence on the client level than the simple browser platform provides -- unless you have sufficient control over your clients and can, for example, dictate a platform such as IE5+ with the Java 1.4 plugin.
  • Much of what your proposed scheme is doing can be achieved more simply, I think, by using secure HTTP with client-side certificates. Unfortunately although I've developed SSL and HTTPS applications I don't have experience with client-side certificates personally. Once the SSL connection has been set up, you have proof of the client identity (well, as long as their private key is safe) and all traffic will be encrypted. That doesn't address your problem of needing encrypted server-side files though.
  • There are encryption schemes that allow decryption with multiple keys, such as the clients' private keys. For example, you can encrypt a file with a secret (symmetric) session key, then create encrypted copies of this session key with each clients' public key (not necessarily the most intelligent scheme, but it's nice and easy for the purpose of this discussion). You'd send the appropriate session key copy to the client together with the encrypted file, and the client could decrypt the lot using its private key. This needs at the very least a signed Java servlet or Java WebStart application on the client. If you don't have that, you're stuck with decryption on the server and you cannot avoid the situation that your servers will between them contain all the necessary ingredients to decrypt the file.
  • Encryption would be simpler; if you keep a list of all clients' public keys, you can generate a symmetric session key, encrypt the file, the encrypt the session key with each client's public key in turn.
  • Adding a new user would require the generation of encrypted session keys for that client. This can be done by sending the encrypted keys, together with the new client's public key, to an authorised user's client-side application. That application can decrypt the session keys using the user's private key, then encrypt them using the new client's public key and send them back to the server.
  • Bottom line is, you can fully achieve your goals only if you can use client-side applications or signed applets. Failing that, the best you can do is separate your authentication/key server(s) from your file server(s) and hope that a hacker won't get into both.
    Performance should not be too bad for most purposes. As indicated by Karl, you would never encrypt a file using asymmetric encryption; rather, you'd generate and ecrypt a symmetric session key, then use the session key to encrypt the file using fast symmetric encryption.
    Authorisation rules are simple to implement. If a client doesn't have access to a file, just don't generate a session key for that client. Any modifications to authorisation will necessarily involve an authorised client, which gives you a nice audit trail.
    - Peter
    [ March 21, 2003: Message edited by: Peter den Haan ]
    Robert Paris
    Ranch Hand

    Joined: Jul 28, 2002
    Posts: 585
    my first problem is really that the description of what management wants is really a combination of objective and implementation.

    I know, but this comes from their Network Administration group and unfortunately they are listened to like gods, so it's very hard to not do what they say/ask. I can try to argue otherwise, but it's not too likely.
    Unless you have something better to work with than a dumb client browser

    Unlikely. It's almost certain (99.8%) that it'll be a web browser front-end only.
    the servers would need to have all the necessary ingredients to decrypt the files. Given free rein on those servers an attacker will be able to decrypt the files.

    I don't see how this is true. I would not store the keys to decrpyt the files on the server, they'd only be done through the key sent by the logging in user and that key is stored only in memory. So even though my code which shows the methods to use to decrypt, you'd still need the key, and I wouldn't have that. Granted if you take over the server, figure out what I'm doing and you alter the code so that the next time someone logs in, you decrypt their encryption of the key, then yes, you have the key. However, this would all take some time and I'd hope the network admin guys had figured out by that point someone was in there doing stuff. (Especially since they'd have to shut down the server to change the code that's in the jars)
    The only keys I'd need is the public ones to decrypt what the user encrypted with their private keys. And again, these could be (as you say) hosted on another server and stored only in memory. And I could use "push" technology to make sure they could not "request" these keys. In other words, someone would have to log in to the server through a special page and PASS it thoset keys to "turn on" the system. (Just an idea)
    The scheme you're proposing seems to presuppose much more intelligence on the client level than the simple browser platform provides -- unless you have sufficient control over your clients and can, for example, dictate a platform such as IE5+ with the Java 1.4 plugin.

    That's my fear. I have not yet come up with a good way around this, and there may not be one. I was thinking that I could send them (in the mail or email) a copy of the secret key encrypted with their private key, and then the login page would be a form with inputs for usernam, password and a "browse" input to find that encrypted file on their computer and send that to the server. How does this sound?
    Much of what your proposed scheme is doing can be achieved more simply, I think, by using secure HTTP with client-side certificates. Unfortunately although I've developed SSL and HTTPS applications I don't have experience with client-side certificates personally. Once the SSL connection has been set up, you have proof of the client identity (well, as long as their private key is safe) and all traffic will be encrypted. That doesn't address your problem of needing encrypted server-side files though.

    Yeah, as you point out at the end of that, it doesn't address my problem at all. I do think HTTPS/SSL is a good idea too, but that's not their concern (I think they're assuming we'll use that). Their concern is the files sitting on the server.
    you can encrypt a file with a secret (symmetric) session key, then create encrypted copies of this session key with each clients' public key (not necessarily the most intelligent scheme, but it's nice and easy for the purpose of this discussion).

    If it's not the most intelligent, could you go a bit into what a more intelligent way of doing this would entail?
    Encryption would be simpler; if you keep a list of all clients' public keys, you can generate a symmetric session key, encrypt the file, the encrypt the session key with each client's public key in turn. Adding a new user would require the generation of encrypted session keys for that client. This can be done by sending the encrypted keys, together with the new client's public key, to an authorised user's client-side application. That application can decrypt the session keys using the user's private key, then encrypt them using the new client's public key and send them back to the server.
    Bottom line is, you can fully achieve your goals only if you can use client-side applications or signed applets. Failing that, the best you can do is separate your authentication/key server(s) from your file server(s) and hope that a hacker won't get into both.

    Do you still think this with my idea for the login above?
    Performance should not be too bad for most purposes. As indicated by Karl, you would never encrypt a file using asymmetric encryption; rather, you'd generate and ecrypt a symmetric session key, then use the session key to encrypt the file using fast symmetric encryption.

    Can you go into this a bit more? I'm nott 100% certain I follow all of this. You say "encrypt a symmetric session key, then use the session key to encrypt" - from this it sounds like I'd encrypt the key then use the key to encrypt the files, but it's confusing as to which key is which. The way I understand it:
    1. Create symmetric key which we'll call "Key A"
    2. Create assymetric public/private key pairs to give to each user
    3. Encrypt "Key A" separately with each user's private key
    4. Use the unencrypted "Key A" to encrypt the files
    5. Give the users the encrypted "key A"
    Authorisation rules are simple to implement. If a client doesn't have access to a file, just don't generate a session key for that client. Any modifications to authorisation will necessarily involve an authorised client, which gives you a nice audit trail.

    Wait, but how does this work if they have access to files 1,2 and 4 but not file 3? I'm not going to make a key for every file, that's nuts. I'd have one key for all the files collectively, so if they could potentially decrypt all the files. This is why I'm thinking that this does not have anything to do with authorization. I'd like it to, because it seems like a good extension and enforcement of authroization (i.e. they couldn't decrypt it if they didn't have access) but I don't know how to implement that.
    Peter den Haan
    author
    Ranch Hand

    Joined: Apr 20, 2000
    Posts: 3252
    Originally posted by Robert Paris:
    I don't see how this is true. I would not store the keys to decrpyt the files on the server, they'd only be done through the key sent by the logging in user and that key is stored only in memory.
    Only if users send their private key to the server in addition to their public key -- possible, if less than ideal.
    Where are these keys coming from anyway? Do you plan to store them on the client machine -- possibly (shudder) a cookie -- or have the clients merely authenticate and the keys served up by a separate key server? (then, inevitably, the system is only as hack-proof as the key server).
    I was thinking that I could send them (in the mail or email) a copy of the secret key encrypted with their private key, and then the login page would be a form with inputs for usernam, password and a "browse" input to find that encrypted file on their computer and send that to the server. How does this sound?
    It's not clear to me how exactly you'd send this file (except through a clumsy browse box) or what this would achieve over a client certificate. You need https anyway.
    The way I understand it: [...]
    Correct. "Key A" is the session key, which should really be different every time, for every file. The "nuts" option
    So you'd be looking at an application server A which knows about all public keys for every user, and which gets the private keys from a key server B when the user authenticates. This private key is stored in memory only. That'd work, provided that all communication between client and A and between A and B is encrypted. An attacker who manages to break into both A and B has your files. An attacker who gets at your running application server (using a debugger, code modification, or by intercepting traffic using a man-in-the-middle attack) can get your files as well. All files share the same secret key so if any one file is broken, all are broken.
    Frankly, if this is sufficient then there might a far simpler way to achieve exactly the same level of security. Forget about per-user keys. Have just a server-wide secret key, which only the key/authentication server B knows about. I haven't thought this through in full detail but superficially it seems just as (in)secure.
    - Peter
    Robert Paris
    Ranch Hand

    Joined: Jul 28, 2002
    Posts: 585
    Thanks again for the reply!
    Only if users send their private key to the server in addition to their public key -- possible, if less than ideal.

    Wait, I'm a bit confused here - why would they have to send their private AND public key? What I'm suggesting is this:
    1. Symmetric key created to encode all files
    2. Each user given private/public key pair
    3. Symmetric key encrypted with user's private key
    4. Encrypted symmetric key is sent to server
    So all the server would need is the public keys to decrypt the user-encrypted symmetric key. Why would the private key need to be sent as well?

    Where are these keys coming from anyway? Do you plan to store them on the client machine -- possibly (shudder) a cookie -- or have the clients merely authenticate and the keys served up by a separate key server? (then, inevitably, the system is only as hack-proof as the key server).

    I'm a bit lost here. First why does storing the encrypted symmetric key in a cookie make you shudder? The user will NOT be storing their private key in a cookie, nor a public key, nor the un-encrypted symmetric key. Instead, we'll (outside this system) first encrypt the symmetric key (which was used to encrypt files) with their private key. Then the first time they log in (only the first time), they'll upload that encrypted key which from then on is stored in a cookie on their computer. The server will of course have the public keys to all the users' keys (in order to decrypt the symmetric key), but only in memory (when the system is started up, a user has to "turn on" the system by uploading the public keys in to memory through an HTTPS web browser). And yes, it'll use HTTPS.

    It's not clear to me how exactly you'd send this file (except through a clumsy browse box) or what this would achieve over a client certificate.


    First: why is a browse input clumsy? It'd only happen the first time they log in, and is then cached in their cookie. They'd still have to log in with username/pass but no longer have to explicitly send the encrypted key.
    Second: How would a certificate help? Can I put a symmetric key to decrypt files in a certificate?
    Am I missing something (probably - I wouldn't be surprised if it's just not dawning on me, so please keep hitting away at my idea)? Or am I not describing this very well?
    Thanks again for responding!
    [ March 24, 2003: Message edited by: Robert Paris ]
    Peter den Haan
    author
    Ranch Hand

    Joined: Apr 20, 2000
    Posts: 3252
    Originally posted by Robert Paris:
    Why would the private key need to be sent as well?
    You don't... iff you use one single symmetric key to encrypt absolutely everything, as opposed to a separate session key for every file.
    The server will of course have the public keys to all the users' keys (in order to decrypt the symmetric key), but only in memory (when the system is started up, a user has to "turn on" the system by uploading the public keys in to memory through an HTTPS web browser).
    Hmmm. The urge to keep public keys secret is a bit of a code smell... never mind. I'm now finally getting my head around what you want to do exactly.
    Cookies are not the most secure things around, but it probably doesn't matter much in this case. Everything depends on the security of the client machines (in addition to the server) anyway.
    First: why is a browse input clumsy? It'd only happen the first time they log in, and is then cached in their cookie.
    Not so bad then. Of course one is prone to lose cookies sooner or later, but then you simply prompt for the file again.
    Second: How would a certificate help? Can I put a symmetric key to decrypt files in a certificate?
    A client certificate allows strong authentication of a client as part of the SSL handshake. I agree that in your proposed scheme it wouldn't do a lot for you. You can probably embed a key in the certificate in theory but I wouldn't advocate it.
    Within the constraints you've got to work with, the scheme you've come up with actually is pretty good, provided that security needs are relatively modest. The main Achilles heels are that you implicitly trust your own users (9 out of 10 computer crimes originates within the organisation itself), a single security breach will compromise all documents, and that a determined attacker compromising a running server can bypass security completely.
    - Peter
    [ March 24, 2003: Message edited by: Peter den Haan ]
    Robert Paris
    Ranch Hand

    Joined: Jul 28, 2002
    Posts: 585
    Hmmm. The urge to keep public keys secret is a bit of a code smell... never mind.

    Can you say a bit more about what you are thinking of here? It is not nec. but I thought it does add a nice level of indirection and means that even if someone got a hold of the server, that they wouldn't be able to decrypt the sent-over encrypted keys with what's on the server, nor be able to find out where the public keys (needed for decryption) are. Is this more trouble than it's worth?

    Within the constraints you've got to work with, the scheme you've come up with actually is pretty good, provided that security needs are relatively modest.

    Can you say a bit more about what you mean by "security needs are relatively modest?" Granted, this system idea probably wouldn't be even half of what the military needs for security, but it's pretty far beyond most systems I've heard of. The communiation, authentication, authorization and even location are all secured and encrypted. As well, there is no one point for complete access to the documents. Without the file system housing the documents, the public key for a user's encrypted symmetric key and the encrypted symmetric key all together, you can't do anything with the files (unless of course you can crack the encryption, but if that's the case this is all moot anyways).
    The main Achilles heels are that you implicitly trust your own users...

    Isn't every security system in a thin-client-server setup in this same boat?
    a determined attacker compromising a running server can bypass security completely

    I don't see how this is the case. Granted they could delete all the files, but they wouldn't be able to decrypt them. No one point (client or server) has all the parts you'd need (encrypted key, public key, files). So even if they took over the server, they'd still need a user to log in, AND they'd need for the public keys to be loaded in - which doesn't happen on restart, it has to be done by a user explicitly.
    Thanks again for continuing the conversation, it's really helping me get a good handle on a proper solution!
    [ March 24, 2003: Message edited by: Robert Paris ]
    Peter den Haan
    author
    Ranch Hand

    Joined: Apr 20, 2000
    Posts: 3252
    Originally posted by Robert Paris:
    Can you say a bit more about what you are thinking of here?
    The urge to keep public keys semi-secret arises from the fact that a public key plus the encrypted symmetric key gives immediate access to all the files. You're trying to make it that little bit harder for an attacker to get the two, even if (s)he gets through to a running server.
    In a watertight PKI, the public key can be fully public.
    Within this context, it's a reasonable thing to do. It also means that your infrastructure is less than ideal, but this is not necessarily something you can help; it's dictated by your constraints.
    Isn't every security system in a thin-client-server setup in this same boat?
    To some extent. You can do a little better -- giving each file a different session key and only creating decryption keys for users that actually have access to the file will lower the amount of trust you have to invest in each user. But to do that in a thin-client context, clients have to transfer their private keys to the server (rather than the encrypted session key). This is less alarming than it sounds and no less safe, or perhaps I should say no more unsafe.
    I don't see how [a determined attacker compromising a running server can bypass security completely]
    Once on the machine, there are a number of attacks I can think of. Restarting the server in debugging mode. Modifying the web-app and forcing a reload. Fiddle with the machine's network configuration and set up a man-in-the-middle. Forcing the application server to coredump and trawling through the heap for security-sensitive information (which is why passwords and keys should really always be char[] or byte[] so you can zap them -- remember, Strings are immutable).
    Some of these require a restart and would need an authorised person to boot the application, but some social engineering and a well-crafted forged stack trace can probably take care of that.
    - Peter
    [ March 24, 2003: Message edited by: Peter den Haan ]
    Robert Paris
    Ranch Hand

    Joined: Jul 28, 2002
    Posts: 585
    I think I'm starting to get my head around all this (not the prettiest sight).

    In a watertight PKI, the public key can be fully public.

    Do you see any way this is possible in this context? Or am I approaching this in the best manner possible for this situation?

    It also means that your infrastructure is less than ideal...

    Can you go into this a bit more? In what way? Are you speaking performance-wise, security-wise, or flexibility or something else?

    You can do a little better -- giving each file a different session key and only creating decryption keys for users that actually have access to the file will lower the amount of trust you have to invest in each user.

    There are more than a few problems with this. For one, how would I give user A access to files 1,2,4, user B access to files 2,3,4 and user C access to files 1,2,5? It seems I'd have to give them each multiple keys to do that and that seems to be a bit of a mess as well as very poor performance. Also, doing a diff. key for each file access (or even file-group access) is MUCH less flexible/scalable. Any changes to access have to be re-coordinated with the client-machine (and probaly the client user) and needs additional key creation. What a mess.

    Once on the machine, there are a number of attacks I can think of...

    Yes, but all of your attacks need for the server to be restarted AND for the administrator to re-turn-on the app by explicitly uploading the public keys. If the app never goes down (yes, unlikely), no problem. If the attacker restarts it, we're going to notice and do some research since it will kick everyone out (since keys will be needed to do anything). If they do change stuff and we restart it, there's a chance we won't notice the changes. Is there a way around this? (Even with it, i think it's a tough security-wall - i.e. all of those cases - for someone to figure out and overcome.)
     
    Consider Paul's rocket mass heater.
     
    subject: How encrpyt all files on web server so decryptable by all users' keys?
     
    Similar Threads
    Java Security MOCK
    Safe Design? Dangerous Design?
    Test 252: Mock exam
    Using VSS through ant
    Browser and Server SSL communication