3 minute read

Categories:
Tags:

The Electric Frontier Foundation (EFF) has recommendations about encrypting the web; there is no reason to be running servers over unencrypted HTTP any longer. It is irresponsible to your users and unnecessary, as such there is a formal mechanism called HTTP Strict Transport Security (HSTS) that enforces HTTPS for all requests at the domain level. Taking it further, modern browsers include a set of domains which can only work over HTTPS, it started with Google TLDs such as .dev and .app, but it is growing https://hstspreload.org/.

Let’s Encrypt is the most popular way to get a free TLS certificate. (Many still refer to these incorrectly as the outdated an SSL certificate, leaving documentation pedantically incorrect and libraries such as OpenSSL unfortunately named) While cost is no longer a factor preventing HTTPS adoption, there continue to be additional configuration difficulties. Even glossing over certificate revocation, certificate expiration is a plague in most enterprises. In my experience managing TLS certificates has mostly been a manual process, especially when self-signed certs are used in non-production environments. In microservice setups, there are often hundreds (!) of services each with a separate certificate making it assured there is always one expiring at an inopportune moment.

The Let’s Encrypt Certbot tool automates renewal, and it is arguably better to use the Let’s Encrypt practice of 90 day expiry even when offered the choice for more. DevOps rust without use and a brisk 90-day repeating process essentially forces automation. Fortunately, it is a smooth process for anyone familiar with security practices, and if your organization lacks this talent you have much larger issues to worry about.

According to the Akka documentation: Akka relies on the JVM TLS implementation, which is sufficient for many use cases, but is planned to be replaced with conscrypt or netty-tcnative. I think this means instead of using the public/private PEM key pair generated by Certbot directly, such is done with other software such as Nginx, there are additional steps necessary to create a PKCS12 file. PKCS12 files are common with JVM software and while this is an extra step (and another password to manage), it is hardly extra work within an automated process.

Creating a PKCS12 file

Setting up a running certbot is outside the scope of this article; it really depends on your domain and environment setup. In the end, it will create the key files: cert.pem, chain.pem, fullchain.pem and privkey.pem in the /etc/letsencrypt/live/[domain] directory for your [domain].

Side note: While working through commands to generate the PKCS12 file, there is a minor issue as noted at https://community.letsencrypt.org/t/cannot-verify-domain-with-openssl/11545, that the cert.pem chain is incomplete. This makes it necessary to pass the fullchain.pem to -in below, some documentation may incorrectly suggest cert.pem be used. Basically consumers can’t traverse from the child cert.pem to a root Certificate Authority (CA) certificate in the normal way, so it is necessary to provide an explicit path/chain.

openssl pkcs12 -export \
	 -in /etc/letsencrypt/live/[domain]/fullchain.pem \
	 -inkey /etc/letsencrypt/live/[domain]/privkey.pem \
	 -out /etc/letsencrypt/live/[domain]/cert.pkcs12 \
	 -name [your-url] \
	 -CAfile /etc/letsencrypt/live/[domain]/fullchain.pem \
	 -caname "Let's Encrypt Authority X3" \
	 -password pass:[password]

This cert.pkcs12 is now the only file needed to enable HTTPS server-side encryption - Akka won’t be using any of the PEM files directly. Copy the PKCS12 file to where your Akka application can securely read it and remember your [password]!

Akka HTTP / gRPC

Enabling HTTPS on a gRPC or HTTP server in Akka is done by calling enableHttps during the Http builder to specify a httpsConnectionContext.

Http(actorSystem)
  .newServerAt(interface, port)
  .enableHttps(httpsConnectionContext)
  .bind(routes)

The httpsConnectionContext is generated using a JVM Keystore, which contains private keys to TLS certificates and should not be confused with a Truststore such as cacerts which contains only public keys.

To create an HttpsConnectionContext generate a new Keystore, load the cert.pkcs12 file into it, and use it to create a SSLContext:

def serverHttpsContext(pkcs12File: File, pkcs12Password: String): Try[HttpsConnectionContext] = {
  Using(new FileInputStream(pkcs12File)) {
    contentStream =>
      val ks = KeyStore.getInstance("PKCS12")
      ks.load(contentStream, pkcs12Password.toCharArray)
      ks
  }.map {
    keyStore =>
      val keyManagerFactory = KeyManagerFactory.getInstance("SunX509")
      keyManagerFactory.init(keyStore, pkcs12Password.toCharArray)
      val context = SSLContext.getInstance("TLS")
      context.init(keyManagerFactory.getKeyManagers, null, new SecureRandom)
      ConnectionContext.httpsServer(context)
  }
}

If Akka starts without error it still doesn’t ensure full functionality. As mentioned above it is possible to have incomplete chains due to self-signed certificates or misconfiguration, it is still important to ensure full functionality with a cryptographic tool such as OpenSSL:

openssl s_client -connect <hostname>:<port>

Certificate Renewal

Certbot certificates expire in 90 days, so repeating this process in an automated way is key. Once the certbot is initially run, certbot renew will create another set of PEM certificates without user input. Createing the PCKS12 file and more importantly restarting the Akka application will test your zero-downtime engineering.

Categories:
Tags:
Updated:

You May Also Enjoy

Job Queue Execution Management using ZIO Scopes

8 minute read

, ,

Job Queues are critical parts of Enterprise workloads. Complex queues use distributed nodes, state machines, and complex scheduling to trigger and track running jobs. But when simplicity allows the best approach is to create small idempotent jobs. The smaller the unit of work the easier progress can be tracked, jobs can be restarted or rerun with minimal waste, composability and reuse are increased, and logic is easier to reason about. These are the same arguments for Functional Programming and their Effect Systems, such as ZIO. Effect systems are congruent to the enterprise job queue, with ZIO fibers performing work and ZIO Resource Management forming the scheduling and supervision backbone. An efficient job queue can be written using ZIO constructs using surprisingly minimal amount of code. Table of Contents ZIO Resources and Scope Queue Features Class Interface ZIO Concurrency uses Fibers, Not Threads ...

Realtime Client Database using gRPC Bi-Directional Streams and ZIO Hub

1 minute read

, ,

Realtime pushed-based databases such as Google Firebase are a convenient way to ensure clients have the most recent data locally. Data updates are automatically streamed to clients immediately as they happen, or in the case of a client disconnect, immediately after reconnecting. gRPC server streaming and ZIO Hub allow this functionality to be easily replicated and customized beyond what expensive paid-for services such as Firebase can do.

Http4s Streams and Multipart Form-Data File Uploads

10 minute read

, ,

Streaming is the primary mechanism to reduce memory requirements for processing large datasets. The approach is to view only a small window of data at a time, allowing data to stream through in manageable amounts matching the data window size to the amount of RAM available. A practical example is a file-upload, where multi-GBs file streams can be handled by MBs of server RAM. However, enforcing streaming in software code is prone to errors, and misuse or incompatible method implementations will lead to breaking stream semantics, and ultimately to OOM exceptions. This article focuses on streams within the context of file uploads, using the Http4s library for examples.

Compiling Scala Native in a GitHub Action; Alternatives to GraalVM

11 minute read

, ,

Scala Native is a compiler and JDK written in Scala with the goal of removing Scala’s dependency on the JVM. This isn’t meant to achieve a higher performance such as with JDKs, and it is targeting a specialized use-case not considered to be today’s typical Scala development. Its competitors are Rust and Go, not GraalVM, Java or Kotlin. This article goes through common steps and challenges encountered when compiling Scala Native for linux with a GitHub Action.