How do I enable Wildfly to "Manage" Certs?

[Al Koch]

Hello,

Its my understanding that Wildfly can perform some level of "management" of Lets Encrypt certs. Can someone point me to documentation and instructions for how to set that up?

In particular, can Wildfly be configured so that the process is fully automatic where it refreshes the certs before they expire?

Thank you in advance.

[Stephan van Hulst]

Al Koch wrote:Its my understanding that Wildfly can perform some level of "management" of Lets Encrypt certs. Can someone point me to documentation and instructions for how to set that up?

I don't think there's anything special about Let's Encrypt certificates. WildFly just uses a regular certificate store and doesn't care what certificates are in there.

You need to set up a server identity to use SSL: https://docs.wildfly.org/14/Admin_Guide.html#Security_Realms_Detailed_Configuration

In particular, can Wildfly be configured so that the process is fully automatic where it refreshes the certs before they expire?

No, and I also don't think it is a good idea to do this automatically. However, you could write a tool that does most of the work for you and gives you an overview of the changes it will make, and all you have to do is press a confirm button.

The tool should only need to update the file referenced by the keystore element in the server identity, and then you could use the native management API to tell WildFly to reload the keystore and reinitialize the key manager.

[Tim Holloway]

LetsEncrypt requires the installation of a mini-app (the so-called "acme" URL) to facilitate creation and updating of their SSL certs. In short, the server being certified has to be able to talk directly to LetsEncrypt and vice versa.

Because every appserver has a different way to deploy, the exact mechanism for updating varies with the server being used. This may help: https://opendevops.dev/install-lets-encrypt-certificate-on-jboss-wildfly-in-linux/

I should mention that I don't favor running JEE webapp servers directly off the Internet. HTTP requires port 80 and HTTPS requires port 443 for their default URLs and since those are "magic" ports, they can only be opened by a process that's running with admin/root privileges, which is a serious security risk. That's why 8080 is so popular as the default port for JEE webapp servers.

Non-JEE webapp servers executing as OS native code such as Apache, Nginx and IIS don't have that problem, since they launch with admin privileges, grab the ports, then downshift to secure user IDs. Java cannot do that - there's no "write once/run anywhere" standard for it to use.

So a better approach in most cases is to front your JEE webapp server with a general-purpose webserver running as a reverse proxy. You can use the mod_proxy and mod_jk plugins on Apache. Nginx can also easily proxy. The overhead for this approach is farirly light and you only have to encrypt going into your proxy server - and auto-renew is a done deal for Apache and Nginx. You get the additional security and on top of that, you can mix JEE and non-JEE apps at the same base URL if you like.

[Al Koch]

Thanks Stephan and Tim for your feedback.

Here is a helpful post that answers a couple of the questions:  https://wildfly-security.github.io/wildfly-elytron/blog/obtaining-certificates-from-lets-encrypt-using-the-wildfly-cli/

Tim, can you elaborate on your suggestion about fronting the Wildfly server with Apache or Nginx? You said "auto-renew is a done deal for Apache and Nginx". Is this really fully automatic (for Let's Encrypt), plus Stephan expressed the opinion that " I also don't think it is a good idea to do this automatically".

Thanks!

[Tim Holloway]

LetsEncrypt certs have a very short lifespan, so auto-renewing saves a lot of annoyance and time. If you want a long-lived cert from one of the big names, maybe manual renewal isn't such a big deal.

The mousetech.com webservers that I host are splattered over multiple physical machines, VMs and containers and operate using different products, including apache and Tomcat. But you can't tell that, since I have a battery of Nginx servers that operate as reverse proxies and present them all in one virtual place. And the various sites are secured by LetsEncrypt (I have multiple certs, not a "wild card"). And every day, my certbot checks for expiring certs and auto-renews those ones that need it.

[Al Koch]

Hi Tim,

Thanks for the additional info. Any chance you would share your script that does the daily check (using cron I suppose)?

I've run several attempts to get these certs set up using the Wildfly Elytron facility and I just hit "Duplicate Certificate limit of 5 per week" (per https://letsencrypt.org/docs/rate-limits/) so now I can't move forward for a week!

Thanks!

[Tim Holloway]

These days CentOS has a certbot package. I just got an update for it today in fact.

They handle cert renewal, but they don't use crontab. Instead there's a pair of systemd components that handle it. One is the systemd timer component, the other is a systemd service.

The actual renewal command as executed by the service looks like this:

/usr/bin/certbot renew --noninteractive --no-random-sleep-on-renew $PRE_HOOK $POST_HOOK $RENEW_HOOK $DEPLOY_HOOK $CERTBOT_ARGS

The environment variables are all takes from the properties file /etc/sysconfig/certbot and currently they're all just empty strings.

If you're set up properly, you can manually issue the command "certbot renew" and it should display what's on the list of renewable certs and attempt to renew any eligable ones. Add the "--dry-run" option to the certbot command to test without updating.

[Al Koch]

Hi Tim,

Thanks again for your feedback. Unfortunately, I am working with an Ubuntu server so your CentOS technique won;t work. Plus, I moved away from certbot because it requires that the server be restarted after a cert refresh. The Elytron subsystem to the Wildfly server eliminates that need to reboot. I think I am close to figuring out how to use Elytron to do this. I'll post the final results here.

Thanks again.

[Tim Holloway]

There's a certbot package for Ubuntu plus a whole raft of python-certbot packages, including one for Nginx. It's quite likely that those packages cover auto-renewal, as the systemd scripts are distro-independent for the most part.

I don't know if certbot absolutely has to restart a server or not in all cases. LetsEncrypt does need to talk to the acme app on the server to be certified in order to avoid spoofing, but it seems to me that you could update that on the fly. I do know that for some renewal methods where a server isn't already running then the certbot will spawn one just long enough to certify.

I haven't paid attention to whether certbot is cycling my Nginx server, since it cycles so fast that any downtime is essentially unnoticeable.