Examples Solutioning

GitHub

Job Queue using ZIO Resource Scope

Queue using ZIO Scope to manage queue removal after external execution.

Job Queue Execution Management using ZIO Scopes

11 minute read

, ,

Job Queues are critical parts of Enterprise workloads. Complex queues use distributed nodes, state machines, and complex scheduling to trigger and track running jobs. But when simplicity allows the best approach is to create small idempotent jobs. The smaller the unit of work the easier progress can be tracked, jobs can be restarted or rerun with minimal waste, composability and reuse are increased, and logic is easier to reason about. These are the same arguments for Functional Programming and their Effect Systems, such as ZIO. Effect systems are congruent to the enterprise job queue, with ZIO fibers performing work and ZIO Resource Management forming the scheduling and supervision backbone. An efficient job queue can be written using ZIO constructs using surprisingly minimal amount of code.

Back to Top ↑
GitHub

Realtime Push Database

Realtime Database using ZIO Hub and Bi-Directional gRPC Streams.

Realtime Client Database using gRPC Bi-Directional Streams and ZIO Hub

1 minute read

, ,

Realtime pushed-based databases such as Google Firebase are a convenient way to ensure clients have the most recent data locally. Data updates are automatically streamed to clients immediately as they happen, or in the case of a client disconnect, immediately after reconnecting. gRPC server streaming and ZIO Hub allow this functionality to be easily replicated and customized beyond what expensive paid-for services such as Firebase can do.

Back to Top ↑
GitHub

HTTP Maven Receiver

HTTP server that receives artifact uploads and verifies MD5 against Maven.

Http4s Streams and Multipart Form-Data File Uploads

10 minute read

, ,

Streaming is the primary mechanism to reduce memory requirements for processing large datasets. The approach is to view only a small window of data at a time, allowing data to stream through in manageable amounts matching the data window size to the amount of RAM available. A practical example is a file-upload, where multi-GBs file streams can be handled by MBs of server RAM. However, enforcing streaming in software code is prone to errors, and misuse or incompatible method implementations will lead to breaking stream semantics, and ultimately to OOM exceptions. This article focuses on streams within the context of file uploads, using the Http4s library for examples.

Compiling Scala Native in a GitHub Action; Alternatives to GraalVM

11 minute read

, ,

Scala Native is a compiler and JDK written in Scala with the goal of removing Scala’s dependency on the JVM. This isn’t meant to achieve a higher performance such as with JDKs, and it is targeting a specialized use-case not considered to be today’s typical Scala development. Its competitors are Rust and Go, not GraalVM, Java or Kotlin. This article goes through common steps and challenges encountered when compiling Scala Native for linux with a GitHub Action.

Data Transfers and Egress within a GitHub Action

2 minute read

, ,

The free tier of GitHub Packages has limited bandwidth to download private artifacts; which can make it unsuitable for use in a CI/CD pipeline for projects on a budget. In an effort to increase GitHub Packages’ usability, this article develops an alternative approach minimizing the dependency on GitHub Packages as hot storage, but preserving it as a viable cold storage, durable storage solution.

Back to Top ↑
GitHub

List Lookup ZIO Cache

ZIO Cache expanded to efficiently support Seq[Key] calls.

List Lookup Cache with Scala ZIO

5 minute read

, ,

In-memory caches mapping Key => Value are a simple and versatile tool to reduce number of calls to an origin datasource. There are many use-cases requiring multiple cache calls, preferring Seq[Key] => Seq[Value]. Can a standard cache implementation be expanded to efficiently handle this scenario?

Back to Top ↑
GitHub

AWS Lambda DynamoDB importer

Scala3 and Python lambda functions to insert into a DynamoDB.

JVM versus Python for AWS Lambda Functions

3 minute read

, ,

The suitability of programming languages across different domains is a contested topic. AWS Lambda Functions are a serverless solution that can be used for a wide range of problems from tiny to large tasks. For lightweight tasks how does the JVM stack up?

Scala 3 and AWS Lambda Functions

5 minute read

, , , ,

AWS Lambda offer the ability to run code functions without a server. Basically standalone functions that receive JSON as a parameter and have up to 15 minutes to do anything. The source of the JSON event can be anything, AWS has configured most of their AWS products to emit events; for example uploading a file to S3 creates JSON that contains information about the file. Lambdas are meant to be simple and short-lived code snippets, so each Lambda can only listen to 1 source for events (although you can proxy multiple types of events through a single source). The most generic source for events is to listen to HTTP requests on a public URL, and we’ll cover how that can be done in this article.

Back to Top ↑
GitHub

Ordered-Columns Web Component

Polymer Web Component that builds vertically ordered columns (timelines) from arbitrary DOM content.

DOM Manipulation using Web Components

3 minute read

HTML elements are free to change the style, size, and placement of their children, and even their order. A lot of advanced use cases define rendering based on both the properties of element as well as the properties of their children; one particularly interesting case is the 2 column timeline. This is similar to a standard 2 column flow, except instead of first filling one column and overflowing to the second, columns are filled simultaneously – inserting elements into whichever has the least content. The net effect is elements occurring earlier in the HTML markup are placed vertically higher in the page than elements occurring later. The page reads top to bottom as a chronological timeline, which while being a simple enough concept cannot be done using standard HTML. In fact, the exact ordering of elements are different based on widths of the columns. Column placements are determined by previous element’s heights, and heights are a function of widths, so setting colum...

Back to Top ↑
GitHub

Sortable-Table Web Component

Polymer Web Component that generates a sortable <table> from inlined or AJAX JSON, JSON5, and arrays.

Advanced Uses of Polymer Templates

4 minute read

Most sortable HTML table generators (such as AngularJS’s ng-grid) allow cells to be customized and formatted according to templates, however all templates are specified as parsed strings of HTML content. The Web Components specification allows for HTML Templates, meaning actual HTML fragments can be used instead of pieced together strings. Two benefits are better readability, CSS encapsulation by way of Shadow DOM, and soon to be native support by modern browsers.

Back to Top ↑