When You Run Out of Types...

by cgrand (🦋 🦣 𝕏)

There is a modeling technique I’ve used in the past, and the more I use it, the more it feels like the right default in a certain class of problems.

The situation is simple enough: you model with data, happily, until one day plain values are no longer enough.

Not because you need more structure.

Because you need more distinctions, more equivalence classes.

You have values representable by the same collection type but they should not be confused. At this point we usually reach for one of three things:

• maps with a :type-like key (or worse: keyset-sniffing!),
• metadata,
• defrecord or deftype.

They all work... to some extent.

They all fail in the same way: code that looks sensible do the wrong thing, because the nuances, the invariants of our fifty shades of maps gets ignored.

Let's review them!

Maps with a :type

The classic just add a :type key, one can't go wrong with classics, right? Right?

{:type ::user-id
 :value 42}

Good enough for a while but the cost is that you are still working with a map.

Sooner or later, someone writes or runs map code over it as if it were a plain map.

It's not that one shouldn't be able to use generic functions on them, just that one shouldn't be able to use generic functions on them without being reminded they are no plain maps.

Metadata

Metadata is attractive because it does not pollute the value itself. Unfortunately that is also why it is such a poor fit for modeling: metadata is not part of equality.

Plus it's not printed by default, preserving metadata across transformations is a constant cognitive overload.

defrecord and deftype

Okay, deftype can do the job, at the cost of a lot of boilerplate to give it value semantics.

Wait! Isn't defrecord essentially deftype with value semantics? Yes, it ticks all the boxes: value semantics with its own equivalence class and prints clearly. The catch is that map? returns true on records.

Is that really a problem? Yes because one can't guard every map? with a record? (especially when using third-party code).

Imagine the mess if (every? fn? [:a 'a [] {} #{} #'*out*]) was true. That's why we have fn? and ifn?.

user=> (every? fn? [:a 'a [] {} #{} #'*out*])
false
user=> (every? ifn? [:a 'a [] {} #{} #'*out*])
true

Sadly we have no imap?.

Plus you have to go through protocols or instance? checks to tell them apart. Nothing as easy (or simple? 🤔) than :type. (Yes, there's type but then you can't have types in a case...)

Last you have the hysteresis issues caused by records silently downgrading to maps when a field key is dissoc-ed.

The silver bullet: Tagged Values

All hope is not lost, I've been increasingly trodding a fourth path: tagged values.

The idea is to (ab)use the tagged-literal function to create values which can't be construed for others.

user=> (tagged-literal `customer {:name "Wile E. Coyote"})
; prints clearly by default
#user/customer {:name "Wile E. Coyote"}

user=> (= (tagged-literal `supplier {:name "Wile E. Coyote"}) (tagged-literal `customer {:name "Wile E. Coyote"}))
; each tag is in its own equivalence class
false

user=> (= {:name "Wile E. Coyote"} (tagged-literal `customer {:name "Wile E. Coyote"}))
; since they have their own equivalence class, they are not equal to maps
false

user=> (map? (tagged-literal `customer {:name "Wile E. Coyote"}))
; they are no maps
false

user=> (coll? (tagged-literal `customer {:name "Wile E. Coyote"}))
; not even collections
false

user=> (:tag (tagged-literal `customer {:name "Wile E. 
Coyote"}))
; still, accessing the tag is easy
user/customer

user=> (:form (tagged-literal `customer {:name "Wile E. Coyote"}))
; as well as accessing the payload.
{:name "Wile E. Coyote"}

It is a wrapper with meaning, with no ceremony.

The important part is not the printed literal syntax. In fact the reader is beside the point here. The important part is that you can create a distinct semantic value for free!

So tagged value buys you something very simple and valuable: safe modeling space! (Fresh equivalence classes.)

If a plain 42 and a "user id 42" should not be interchangeable, then they should not be equal, not be confused, and not accidentally flow through the same code paths. This is what tagged values give you: not more structure, but stronger distinction to prevent unknowingly sending specific data through generic paths and its counterpoint avoiding to make specific pipelines accidentally generic.

Closing

Clojure makes it blissfully easy to model with plain data. That is one of its strengths.

When you run out of types, you don't need more shapes, you need more separation and that's what tagged values brings to the table at almost no cost.

Once you start seeing some modeling problems in terms of equivalence classes rather than representation, they make more and more sense.

Give them a try and let the community know!

Permalink

Build and Deploy Web Apps With Clojure and FLy.io

This post walks through a small web development project using Clojure, covering everything from building the app to packaging and deploying it. It’s a collection of insights and tips I’ve learned from building my Clojure side projects, but presented in a more structured format.

As the title suggests, we’ll be deploying the app to Fly.io. It’s a service that allows you to deploy apps packaged as Docker images on lightweight virtual machines. ^{[1] [1]} My experience with it has been good; it’s easy to use and quick to set up. One downside of Fly is that it doesn’t have a free tier, but if you don’t plan on leaving the app deployed, it barely costs anything.

This isn’t a tutorial on Clojure, so I’ll assume you already have some familiarity with the language as well as some of its libraries. ^{[2] [2]}

Project Setup

In this post, we’ll be building a barebones bookmarks manager for the demo app. Users can log in using basic authentication, view all bookmarks, and create a new bookmark. It’ll be a traditional multi-page web app and the data will be stored in a SQLite database.

Here’s an overview of the project’s starting directory structure:

.
├── dev
│   └── user.clj
├── resources
│   └── config.edn
├── src
│   └── acme
│       └── main.clj
└── deps.edn

And the libraries we’re going to use. If you have some Clojure experience or have used Kit, you’re probably already familiar with all the libraries listed below. ^{[3] [3]}

{:paths ["src" "resources"]
 :deps {org.clojure/clojure               {:mvn/version "1.12.0"}
        aero/aero                         {:mvn/version "1.1.6"}
        integrant/integrant               {:mvn/version "0.11.0"}
        ring/ring-jetty-adapter           {:mvn/version "1.12.2"}
        metosin/reitit-ring               {:mvn/version "0.7.2"}
        com.github.seancorfield/next.jdbc {:mvn/version "1.3.939"}
        org.xerial/sqlite-jdbc            {:mvn/version "3.46.1.0"}
        hiccup/hiccup                     {:mvn/version "2.0.0-RC3"}}
 :aliases
 {:dev {:extra-paths ["dev"]
        :extra-deps  {nrepl/nrepl    {:mvn/version "1.3.0"}
                      integrant/repl {:mvn/version "0.3.3"}}
        :main-opts   ["-m" "nrepl.cmdline" "--interactive" "--color"]}}}

I use Aero and Integrant for my system configuration (more on this in the next section), Ring with the Jetty adaptor for the web server, Reitit for routing, next.jdbc for database interaction, and Hiccup for rendering HTML. From what I’ve seen, this is a popular “library combination” for building web apps in Clojure. ^{[4] [4]}

The user namespace in dev/user.clj contains helper functions from Integrant-repl to start, stop, and restart the Integrant system.

(ns user
  (:require
   [acme.main :as main]
   [clojure.tools.namespace.repl :as repl]
   [integrant.core :as ig]
   [integrant.repl :refer [set-prep! go halt reset reset-all]]))

(set-prep!
 (fn []
   (ig/expand (main/read-config)))) ;; we'll implement this soon

(repl/set-refresh-dirs "src" "resources")

(comment
  (go)
  (halt)
  (reset)
  (reset-all))

Systems and Configuration

If you’re new to Integrant or other dependency injection libraries like Component, I’d suggest reading “How to Structure a Clojure Web”. It’s a great explanation of the reasoning behind these libraries. Like most Clojure apps that use Aero and Integrant, my system configuration lives in a .edn file. I usually name mine as resources/config.edn. Here’s what it looks like:

{:server
 {:port #long #or [#env PORT 8080]
  :host #or [#env HOST "0.0.0.0"]
  :auth {:username #or [#env AUTH_USER "john.doe@email.com"]
         :password #or [#env AUTH_PASSWORD "password"]}}

 :database
 {:dbtype "sqlite"
  :dbname #or [#env DB_DATABASE "database.db"]}}

In production, most of these values will be set using environment variables. During local development, the app will use the hard-coded default values. We don’t have any sensitive values in our config (e.g., API keys), so it’s fine to commit this file to version control. If there are such values, I usually put them in another file that’s not tracked by version control and include them in the config file using Aero’s #include reader tag.

This config file is then “expanded” into the Integrant system map using the expand-key method:

(ns acme.main
  (:require
   [aero.core :as aero]
   [clojure.java.io :as io]
   [integrant.core :as ig]))

(defn read-config
  []
  {:system/config (aero/read-config (io/resource "config.edn"))})

(defmethod ig/expand-key :system/config
  [_ opts]
  (let [{:keys [server database]} opts]
    {:server/jetty (assoc server :handler (ig/ref :handler/ring))
     :handler/ring {:database (ig/ref :database/sql)
                    :auth     (:auth server)}
     :database/sql database}))

The system map is created in code instead of being in the configuration file. This makes refactoring your system simpler as you only need to change this method while leaving the config file (mostly) untouched. ^{[5] [5]}

My current approach to Integrant + Aero config files is mostly inspired by the blog post “Rethinking Config with Aero & Integrant” and Laravel’s configuration. The config file follows a similar structure to Laravel’s config files and contains the app configurations without describing the structure of the system. Previously, I had a key for each Integrant component, which led to the config file being littered with #ig/ref and more difficult to refactor.

Also, if you haven’t already, start a REPL and connect to it from your editor. Run clj -M:dev if your editor doesn’t automatically start a REPL. Next, we’ll implement the init-key and halt-key! methods for each of the components:

;; src/acme/main.clj
(ns acme.main
  (:require
   ;; ...
   [acme.handler :as handler]
   [acme.util :as util])
   [next.jdbc :as jdbc]
   [ring.adapter.jetty :as jetty]))
;; ...

(defmethod ig/init-key :server/jetty
  [_ opts]
  (let [{:keys [handler port]} opts
        jetty-opts (-> opts (dissoc :handler :auth) (assoc :join? false))
        server     (jetty/run-jetty handler jetty-opts)]
    (println "Server started on port " port)
    server))

(defmethod ig/halt-key! :server/jetty
  [_ server]
  (.stop server))

(defmethod ig/init-key :handler/ring
  [_ opts]
  (handler/handler opts))

(defmethod ig/init-key :database/sql
  [_ opts]
  (let [datasource (jdbc/get-datasource opts)]
    (util/setup-db datasource)
    datasource))

The setup-db function creates the required tables in the database if they don’t exist yet. This works fine for database migrations in small projects like this demo app, but for larger projects, consider using libraries such as Migratus (my preferred library) or Ragtime.

(ns acme.util 
  (:require
   [next.jdbc :as jdbc]))

(defn setup-db
  [db]
  (jdbc/execute-one!
   db
   ["create table if not exists bookmarks (
       bookmark_id text primary key not null,
       url text not null,
       created_at datetime default (unixepoch()) not null
     )"]))

For the server handler, let’s start with a simple function that returns a “hi world” string.

(ns acme.handler
  (:require
   [ring.util.response :as res]))

(defn handler
  [_opts]
  (fn [req]
    (res/response "hi world")))

Now all the components are implemented. We can check if the system is working properly by evaluating (reset) in the user namespace. This will reload your files and restart the system. You should see this message printed in your REPL:

:reloading (acme.util acme.handler acme.main)
Server started on port  8080
:resumed

If we send a request to http://localhost:8080/, we should get “hi world” as the response:

$ curl localhost:8080/
# hi world

Nice! The system is working correctly. In the next section, we’ll implement routing and our business logic handlers.

Routing, Middleware, and Route Handlers

First, let’s set up a ring handler and router using Reitit. We only have one route, the index / route that’ll handle both GET and POST requests.

(ns acme.handler
  (:require
   [reitit.ring :as ring]))

(def routes
  [["/" {:get  index-page
         :post index-action}]])

(defn handler
  [opts]
  (ring/ring-handler
   (ring/router routes)
   (ring/routes
    (ring/redirect-trailing-slash-handler)
    (ring/create-resource-handler {:path "/"})
    (ring/create-default-handler))))

We’re including some useful middleware:

• redirect-trailing-slash-handler to resolve routes with trailing slashes,
• create-resource-handler to serve static files, and
• create-default-handler to handle common 40x responses.

Implementing the Middlewares

If you remember the :handler/ring from earlier, you’ll notice that it has two dependencies, database and auth. Currently, they’re inaccessible to our route handlers. To fix this, we can inject these components into the Ring request map using a middleware function.

;; ...

(defn components-middleware
  [components]
  (let [{:keys [database auth]} components]
    (fn [handler]
      (fn [req]
        (handler (assoc req
                        :db database
                        :auth auth))))))
;; ...

The components-middleware function takes in a map of components and creates a middleware function that “assocs” each component into the request map. ^{[6] [6]} If you have more components such as a Redis cache or a mail service, you can add them here.

We’ll also need a middleware to handle HTTP basic authentication. ^{[7] [7]} This middleware will check if the username and password from the request map match the values in the auth map injected by components-middleware. If they match, then the request is authenticated and the user can view the site.

(ns acme.handler
  (:require
   ;; ...
   [acme.util :as util]
   [ring.util.response :as res]))
;; ...

(defn wrap-basic-auth
  [handler]
  (fn [req]
    (let [{:keys [headers auth]} req
          {:keys [username password]} auth
          authorization (get headers "authorization")
          correct-creds (str "Basic " (util/base64-encode
                                       (format "%s:%s" username password)))]
      (if (and authorization (= correct-creds authorization))
        (handler req)
        (-> (res/response "Access Denied")
            (res/status 401)
            (res/header "WWW-Authenticate" "Basic realm=protected"))))))
;; ...

A nice feature of Clojure is that interop with the host language is easy. The base64-encode function is just a thin wrapper over Java’s Base64.Encoder:

(ns acme.util
   ;; ...
  (:import java.util.Base64))

(defn base64-encode
  [s]
  (.encodeToString (Base64/getEncoder) (.getBytes s)))

Finally, we need to add them to the router. Since we’ll be handling form requests later, we’ll also bring in Ring’s wrap-params middleware.

(ns acme.handler
  (:require
   ;; ...
   [ring.middleware.params :refer [wrap-params]]))
;; ...

(defn handler
  [opts]
  (ring/ring-handler
   ;; ...
   {:middleware [(components-middleware opts)
                 wrap-basic-auth
                 wrap-params]}))

Implementing the Route Handlers

We now have everything we need to implement the route handlers or the business logic of the app. First, we’ll implement the index-page function, which renders a page that:

1. Shows all of the user’s bookmarks in the database, and
2. Shows a form that allows the user to insert new bookmarks into the database

(ns acme.handler
  (:require
   ;; ...
   [next.jdbc :as jdbc]
   [next.jdbc.sql :as sql]))
;; ...

(defn template
  [bookmarks]
  [:html
   [:head
    [:meta {:charset "utf-8"
            :name    "viewport"
            :content "width=device-width, initial-scale=1.0"}]]
   [:body
    [:h1 "bookmarks"]
    [:form {:method "POST"}
     [:div
      [:label {:for "url"} "url "]
      [:input#url {:name "url"
                   :type "url"
                   :required true
                   :placeholer "https://en.wikipedia.org/"}]]
     [:button "submit"]]
    [:p "your bookmarks:"]
    [:ul
     (if (empty? bookmarks)
       [:li "you don't have any bookmarks"]
       (map
        (fn [{:keys [url]}]
          [:li
           [:a {:href url} url]])
        bookmarks))]]])

(defn index-page
  [req]
  (try
    (let [bookmarks (sql/query (:db req)
                               ["select * from bookmarks"]
                               jdbc/unqualified-snake-kebab-opts)]
      (util/render (template bookmarks)))
    (catch Exception e
      (util/server-error e))))
;; ...

Database queries can sometimes throw exceptions, so it’s good to wrap them in a try-catch block. I’ll also introduce some helper functions:

(ns acme.util
  (:require
   ;; ...
   [hiccup2.core :as h]
   [ring.util.response :as res])
  (:import java.util.Base64))
;; ...

(defn preprend-doctype
  [s]
  (str "<!doctype html>" s))

(defn render
  [hiccup]
  (-> hiccup h/html str preprend-doctype res/response (res/content-type "text/html")))

(defn server-error
  [e]
  (println "Caught exception: " e)
  (-> (res/response "Internal server error")
      (res/status 500)))

render takes a hiccup form and turns it into a ring response, while server-error takes an exception, logs it, and returns a 500 response.

Next, we’ll implement the index-action function:

;; ...

(defn index-action
  [req]
  (try
    (let [{:keys [db form-params]} req
          value (get form-params "url")]
      (sql/insert! db :bookmarks {:bookmark_id (random-uuid) :url value})
      (res/redirect "/" 303))
    (catch Exception e
      (util/server-error e))))
;; ...

This is an implementation of a typical post/redirect/get pattern. We get the value from the URL form field, insert a new row in the database with that value, and redirect back to the index page. Again, we’re using a try-catch block to handle possible exceptions from the database query.

That should be all of the code for the controllers. If you reload your REPL and go to http://localhost:8080, you should see something that looks like this after logging in:

The last thing we need to do is to update the main function to start the system:

;; ...

(defn -main [& _]
  (-> (read-config) ig/expand ig/init))

Now, you should be able to run the app using clj -M -m acme.main. That’s all the code needed for the app. In the next section, we’ll package the app into a Docker image to deploy to Fly.

Packaging the App

While there are many ways to package a Clojure app, Fly.io specifically requires a Docker image. There are two approaches to doing this:

1. Build an uberjar and run it using Java in the container, or
2. Load the source code and run it using Clojure in the container

Both are valid approaches. I prefer the first since its only dependency is the JVM. We’ll use the tools.build library to build the uberjar. Check out the official guide for more information on building Clojure programs. Since it’s a library, to use it, we can add it to our deps.edn file with an alias:

{;; ...
 :aliases
 {;; ...
  :build {:extra-deps {io.github.clojure/tools.build 
                       {:git/tag "v0.10.5" :git/sha "2a21b7a"}}
          :ns-default build}}}

Tools.build expects a build.clj file in the root of the project directory, so we’ll need to create that file. This file contains the instructions to build artefacts, which in our case is a single uberjar. There are many great examples of build.clj files on the web, including from the official documentation. For now, you can copy+paste this file into your project.

(ns build
  (:require
   [clojure.tools.build.api :as b]))

(def basis (delay (b/create-basis {:project "deps.edn"})))
(def src-dirs ["src" "resources"])
(def class-dir "target/classes")

(defn uber
  [_]
  (println "Cleaning build directory...")
  (b/delete {:path "target"})

  (println "Copying files...")
  (b/copy-dir {:src-dirs   src-dirs
               :target-dir class-dir})

  (println "Compiling Clojure...")
  (b/compile-clj {:basis      @basis
                  :ns-compile '[acme.main]
                  :class-dir  class-dir})

  (println "Building Uberjar...")
  (b/uber {:basis     @basis
           :class-dir class-dir
           :uber-file "target/standalone.jar"
           :main      'acme.main}))

To build the project, run clj -T:build uber. This will create the uberjar standalone.jar in the target directory. The uber in clj -T:build uber refers to the uber function from build.clj. Since the build system is a Clojure program, you can customise it however you like. If we try to run the uberjar now, we’ll get an error:

# build the uberjar
$ clj -T:build uber
# Cleaning build directory...
# Copying files...
# Compiling Clojure...
# Building Uberjar...

# run the uberjar
$ java -jar target/standalone.jar
# Error: Could not find or load main class acme.main
# Caused by: java.lang.ClassNotFoundException: acme.main

This error occurred because the Main class that is required by Java isn’t built. To fix this, we need to add the :gen-class directive in our main namespace. This will instruct Clojure to create the Main class from the -main function.

(ns acme.main
  ;; ...
  (:gen-class))
;; ...

If you rebuild the project and run java -jar target/standalone.jar again, it should work perfectly. Now that we have a working build script, we can write the Dockerfile:

# install additional dependencies here in the base layer
# separate base from build layer so any additional deps installed are cached
FROM clojure:temurin-21-tools-deps-bookworm-slim AS base

FROM base as build
WORKDIR /opt
COPY . .
RUN clj -T:build uber

FROM eclipse-temurin:21-alpine AS prod
COPY --from=build /opt/target/standalone.jar /
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "standalone.jar"]

It’s a multi-stage Dockerfile. We use the official Clojure Docker image as the layer to build the uberjar. Once it’s built, we copy it to a smaller Docker image that only contains the Java runtime. ^{[8] [8]} By doing this, we get a smaller container image as well as a faster Docker build time because the layers are better cached.

That should be all for packaging the app. We can move on to the deployment now.

Deploying with Fly.io

First things first, you’ll need to install flyctl, Fly’s CLI tool for interacting with their platform. Create a Fly.io account if you haven’t already. Then run fly auth login to authenticate flyctl with your account.

Next, we’ll need to create a new Fly App:

$ fly app create
# ? Choose an app name (leave blank to generate one): 
# automatically selected personal organization: Ryan Martin
# New app created: blue-water-6489

Another way to do this is with the fly launch command, which automates a lot of the app configuration for you. We have some steps to do that are not done by fly launch, so we’ll be configuring the app manually. I also already have a fly.toml file ready that you can straight away copy to your project.

# replace these with your app and region name
# run `fly platform regions` to get a list of regions
app = 'blue-water-6489' 
primary_region = 'sin'

[env]
  DB_DATABASE = "/data/database.db"

[http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = "stop"
  auto_start_machines = true
  min_machines_running = 0

[mounts]
  source = "data"
  destination = "/data"
  initial_sie = 1

[[vm]]
  size = "shared-cpu-1x"
  memory = "512mb"
  cpus = 1
  cpu_kind = "shared"

These are mostly the default configuration values with some additions. Under the [env] section, we’re setting the SQLite database location to /data/database.db. The database.db file itself will be stored in a persistent Fly Volume mounted on the /data directory. This is specified under the [mounts] section. Fly Volumes are similar to regular Docker volumes but are designed for Fly’s micro VMs.

We’ll need to set the AUTH_USER and AUTH_PASSWORD environment variables too, but not through the fly.toml file as these are sensitive values. To securely set these credentials with Fly, we can set them as app secrets. They’re stored encrypted and will be automatically injected into the app at boot time.

$ fly secrets set AUTH_USER=hi@ryanmartin.me AUTH_PASSWORD=not-so-secure-password
# Secrets are staged for the first deployment

With this, the configuration is done and we can deploy the app using fly deploy:

$ fly deploy
# ...
# Checking DNS configuration for blue-water-6489.fly.dev
# Visit your newly deployed app at https://blue-water-6489.fly.dev/

The first deployment will take longer since it’s building the Docker image for the first time. Subsequent deployments should be faster due to the cached image layers. You can click on the link to view the deployed app, or you can also run fly open, which will do the same thing. Here’s the app in action:

If you made additional changes to the app or fly.toml, you can redeploy the app using the same command, fly deploy. The app is configured to auto stop/start, which helps to cut costs when there’s not a lot of traffic to the site. If you want to take down the deployment, you’ll need to delete the app itself using fly app destroy <your app name>.

Adding a Production REPL

This is an interesting topic in the Clojure community, with varying opinions on whether or not it’s a good idea. Personally, I find having a REPL connected to the live app helpful, and I often use it for debugging and running queries on the live database. ^{[9] [9]} Since we’re using SQLite, we don’t have a database server we can directly connect to, unlike Postgres or MySQL.

If you’re brave, you can even restart the app directly without redeploying from the REPL. You can easily go wrong with it, which is why some prefer not to use it.

For this project, we’re gonna add a socket REPL. It’s very simple to add (you just need to add a JVM option) and it doesn’t require additional dependencies like nREPL. Let’s update the Dockerfile:

# ...
EXPOSE 7888
ENTRYPOINT ["java", "-Dclojure.server.repl={:port 7888 :accept clojure.core.server/repl}", "-jar", "standalone.jar"]

The socket REPL will be listening on port 7888. If we redeploy the app now, the REPL will be started, but we won’t be able to connect to it. That’s because we haven’t exposed the service through Fly proxy. We can do this by adding the socket REPL as a service in the [services] section in fly.toml.

However, doing this will also expose the REPL port to the public. This means that anyone can connect to your REPL and possibly mess with your app. Instead, what we want to do is to configure the socket REPL as a private service.

By default, all Fly apps in your organisation live in the same private network. This private network, called 6PN, connects the apps in your organisation through WireGuard tunnels (a VPN) using IPv6. Fly private services aren’t exposed to the public internet but can be reached from this private network. We can then use Wireguard to connect to this private network to reach our socket REPL.

Fly VMs are also configured with the hostname fly-local-6pn, which maps to its 6PN address. This is analogous to localhost, which points to your loopback address 127.0.0.1. To expose a service to 6PN, all we have to do is bind or serve it to fly-local-6pn instead of the usual 0.0.0.0. We have to update the socket REPL options to:

# ...
ENTRYPOINT ["java", "-Dclojure.server.repl={:port 7888,:address \"fly-local-6pn\",:accept clojure.core.server/repl}", "-jar", "standalone.jar"]

After redeploying, we can use the fly proxy command to forward the port from the remote server to our local machine. ^{[10] [10]}

$ fly proxy 7888:7888
# Proxying local port 7888 to remote [blue-water-6489.internal]:7888

In another shell, run:

$ rlwrap nc localhost 7888
# user=>

Now we have a REPL connected to the production app! rlwrap is used for readline functionality, e.g. up/down arrow keys, vi bindings. Of course, you can also connect to it from your editor.

Deploy with GitHub Actions

If you’re using GitHub, we can also set up automatic deployments on pushes/PRs with GitHub Actions. All you need is to create the workflow file:

name: Fly Deploy
on:
  push:
    branches:
      - main
  workflow_dispatch:

jobs:
  deploy:
    name: Deploy app
    runs-on: ubuntu-latest
    concurrency: deploy-group
    steps:
      - uses: actions/checkout@v4
      - uses: superfly/flyctl-actions/setup-flyctl@master
      - run: flyctl deploy --remote-only
        env:
          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}

To get this to work, you’ll need to create a deploy token from your app’s dashboard. Then, in your GitHub repo, create a new repository secret called FLY_API_TOKEN with the value of your deploy token. Now, whenever you push to the main branch, this workflow will automatically run and deploy your app. You can also manually run the workflow from GitHub because of the workflow_dispatch option.

End

As always, all the code is available on GitHub. Originally, this post was just about deploying to Fly.io, but along the way, I kept adding on more stuff until it essentially became my version of the user manager example app. Anyway, hope this post provided a good view into web development with Clojure. As a bonus, here are some additional resources on deploying Clojure apps:

• Deploying a Full-Stack Clojure App With Kamal on a Single Server
• JVM Deployment Options
• Deploying Clojure Like a Seasoned Hobbyist
• Brave Clojure’s Deploy Quest

Permalink

Advent of Code 2024 in Zig

This post is about six seven months late, but here are my takeaways from Advent of Code 2024. It was my second time participating, and this time I actually managed to complete it. ^{[1] [1]} My goal was to learn a new language, Zig, and to improve my DSA and problem-solving skills.

If you’re not familiar, Advent of Code is an annual programming challenge that runs every December. A new puzzle is released each day from December 1st to the 25th. There’s also a global leaderboard where people (and AI) race to get the fastest solves, but I personally don’t compete in it, mostly because I want to do it at my own pace.

I went with Zig because I have been curious about it for a while, mainly because of its promise of being a better C and because TigerBeetle (one of the coolest databases now) is written in it. Learning Zig felt like a good way to get back into systems programming, something I’ve been wanting to do after a couple of chaotic years of web development.

This post is mostly about my setup, results, and the things I learned from solving the puzzles. If you’re more interested in my solutions, I’ve also uploaded my code and solution write-ups to my GitHub repository.

Project Setup

There were several Advent of Code templates in Zig that I looked at as a reference for my development setup, but none of them really clicked with me. I ended up just running my solutions directly using zig run for the whole event. It wasn’t until after the event ended that I properly learned Zig’s build system and reorganised my project.

Here’s what the project structure looks like now:

.
├── src
│   ├── days
│   │   ├── data
│   │   │   ├── day01.txt
│   │   │   ├── day02.txt
│   │   │   └── ...
│   │   ├── day01.zig
│   │   ├── day02.zig
│   │   └── ...
│   ├── bench.zig
│   └── run.zig
└── build.zig

The project is powered by build.zig, which defines several commands:

1. Build
   • zig build - Builds all of the binaries for all optimisation modes.
2. Run
   • zig build run - Runs all solutions sequentially.
   • zig build run -Day=XX - Runs the solution of the specified day only.
3. Benchmark
   • zig build bench - Runs all benchmarks sequentially.
   • zig build bench -Day=XX - Runs the benchmark of the specified day only.
4. Test
   • zig build test - Runs all tests sequentially.
   • zig build test -Day=XX - Runs the tests of the specified day only.

You can also pass the optimisation mode that you want to any of the commands above with the -Doptimize flag.

Under the hood, build.zig compiles src/run.zig when you call zig build run, and src/bench.zig when you call zig build bench. These files are templates that import the solution for a specific day from src/days/dayXX.zig. For example, here’s what src/run.zig looks like:

const std = @import("std");
const puzzle = @import("day"); // Injected by build.zig

pub fn main() !void {
    var arena = std.heap.ArenaAllocator.init(std.heap.page_allocator);
    defer arena.deinit();
    const allocator = arena.allocator();

    std.debug.print("{s}\n", .{puzzle.title});
    _ = try puzzle.run(allocator, true);
    std.debug.print("\n", .{});
}

The day module imported is an anonymous import dynamically injected by build.zig during compilation. This allows a single run.zig or bench.zig to be reused for all solutions. This avoids repeating boilerplate code in the solution files. Here’s a simplified version of my build.zig file that shows how this works:

const std = @import("std");

pub fn build(b: *std.Build) void {
    const target = b.standardTargetOptions(.{});
    const optimize = b.standardOptimizeOption(.{});

    const run_all = b.step("run", "Run all days");
    const day_option = b.option(usize, "ay", ""); // The `-Day` option

    // Generate build targets for all 25 days.
    for (1..26) |day| {
        const day_zig_file = b.path(b.fmt("src/days/day{d:0>2}.zig", .{day}));

        // Create an executable for running this specific day.
        const run_exe = b.addExecutable(.{
            .name = b.fmt("run-day{d:0>2}", .{day}),
            .root_source_file = b.path("src/run.zig"),
            .target = target,
            .optimize = optimize,
        });

        // Inject the day-specific solution file as the anonymous module `day`.
        run_exe.root_module.addAnonymousImport("day", .{ .root_source_file = day_zig_file });

        // Install the executable so it can be run.
        b.installArtifact(run_exe);

        // ...
    }
}

My actual build.zig has some extra code that builds the binaries for all optimisation modes.

This setup is pretty barebones. I’ve seen other templates do cool things like scaffold files, download puzzle inputs, and even submit answers automatically. Since I wrote my build.zig after the event ended, I didn’t get to use it while solving the puzzles. I might add these features to it if I decided to do Advent of Code again this year with Zig.

Self-Imposed Constraints

While there are no rules to Advent of Code itself, to make things a little more interesting, I set a few constraints and rules for myself:

1. The code must be readable. By “readable”, I mean the code should be straightforward and easy to follow. No unnecessary abstractions. I should be able to come back to the code months later and still understand (most of) it.
2. Solutions must be a single file. No external dependencies. No shared utilities module. Everything needed to solve the puzzle should be visible in that one solution file.
3. The total runtime must be under one second. ^{[2] [2]} All solutions, when run sequentially, should finish in under one second. I want to improve my performance engineering skills.
4. Parts should be solved separately. This means: (1) no solving both parts simultaneously, and (2) no doing extra work in part one that makes part two faster. The aim of this is to get a clear idea of how long each part takes on its own.
5. No concurrency or parallelism. Solutions must run sequentially on a single thread. This keeps the focus on the efficiency of the algorithm. I can’t speed up slow solutions by using multiple CPU cores.
6. No ChatGPT. No Claude. No AI help. I want to train myself, not the LLM. I can look at other people’s solutions, but only after I have given my best effort at solving the problem.
7. Follow the constraints of the input file. The solution doesn’t have to work for all possible scenarios, but it should work for all valid inputs. If the input file only contains 8-bit unsigned integers, the solution doesn’t have to handle larger integer types.
8. Hardcoding is allowed. For example: size of the input, number of rows and columns, etc. Since the input is known at compile-time, we can skip runtime parsing and just embed it into the program using Zig’s @embedFile.

Most of these constraints are designed to push me to write clearer, more performant code. I also wanted my code to look like it was taken straight from TigerBeetle’s codebase (minus the assertions). ^{[3] [3]} Lastly, I just thought it would make the experience more fun.

Favourite Puzzles

From all of the puzzles, here are my top 3 favourites:

1. Day 6: Guard Gallivant - This is my slowest day (in benchmarks), but also the one I learned the most from. Some of these learnings include: using vectors to represent directions, padding 2D grids, metadata packing, system endianness, etc.
2. Day 17: Chronospatial Computer - I love reverse engineering puzzles. I used to do a lot of these in CTFs during my university days. The best thing I learned from this day is the realisation that we can use different integer bases to optimise data representation. This helped improve my runtimes in the later days 22 and 23.
3. Day 21: Keypad Conundrum - This one was fun. My gut told me that it can be solved greedily by always choosing the best move. It was right. Though I did have to scroll Reddit for a bit to figure out the step I was missing, which was that you have to visit the farthest keypads first. This is also my longest solution file (almost 400 lines) because I hardcoded the best-moves table.

Honourable mention:

1. Day 24: Crossed Wires - Another reverse engineering puzzle. Confession: I didn’t solve this myself during the event. After 23 brutal days, my brain was too tired, so I copied a random Python solution from Reddit. When I retried it later, it turned out to be pretty fun. I still couldn’t find a solution I was satisfied with though.

Programming Patterns and Zig Tricks

During the event, I learned a lot about Zig and performance, and also developed some personal coding conventions. Some of these are Zig-specific, but most are universal and can be applied across languages. This section covers general programming and Zig patterns I found useful. The next section will focus on performance-related tips.

Comptime

Zig’s flagship feature, comptime, is surprisingly useful. I knew Zig uses it for generics and that people do clever metaprogramming with it, but I didn’t expect to be using it so often myself.

My main use for comptime was to generate puzzle-specific types. All my solution files follow the same structure, with a DayXX function that takes some parameters (usually the input length) and returns a puzzle-specific type, e.g.:

fn Day01(comptime length: usize) type {
    return struct {
        const Self = @This();
        
        left: [length]u32 = undefined,
        right: [length]u32 = undefined,

        fn init(input: []const u8) !Self {}

        // ...
    };
}

This lets me instantiate the type with a size that matches my input:

// Here, `Day01` is called with the size of my actual input.
pub fn run(_: std.mem.Allocator, is_run: bool) ![3]u64 {
    // ...
    const input = @embedFile("./data/day01.txt");
    var puzzle = try Day01(1000).init(input);
    // ...
}

// Here, `Day01` is called with the size of my test input.
test "day 01 part 1 sample 1" {
    var puzzle = try Day01(6).init(sample_input);
    // ...
}

This allows me to reuse logic across different inputs while still hardcoding the array sizes. Without comptime, I have to either create a separate function for all my different inputs or dynamically allocate memory because I can’t hardcode the array size.

I also used comptime to shift some computation to compile-time to reduce runtime overhead. For example, on day 4, I needed a function to check whether a string matches either "XMAS" or its reverse, "SAMX". A pretty simple function that you can write as a one-liner in Python:

def matches(pattern, target):
    return target == pattern or target == pattern[::-1]

Typically, a function like this requires some dynamic allocation to create the reversed string, since the length of the string is only known at runtime. ^{[4] [4]} For this puzzle, since the words to reverse are known at compile-time, we can do something like this:

fn matches(comptime word: []const u8, slice: []const u8) bool {
    var reversed: [word.len]u8 = undefined;
    @memcpy(&reversed, word);
    std.mem.reverse(u8, &reversed);
    return std.mem.eql(u8, word, slice) or std.mem.eql(u8, &reversed, slice);
}

This creates a separate function for each word I want to reverse. ^{[5] [5]} Each function has an array with the same size as the word to reverse. This removes the need for dynamic allocation and makes the code run faster. As a bonus, Zig also warns you when this word isn’t compile-time known, so you get an immediate error if you pass in a runtime value.

Optional Types

A common pattern in C is to return special sentinel values to denote missing values or errors, e.g. -1, 0, or NULL. In fact, I did this on day 13 of the challenge:

// We won't ever get 0 as a result, so we use it as a sentinel error value.
fn count_tokens(a: [2]u8, b: [2]u8, p: [2]i64) u64 {
    const numerator = @abs(p[0] * b[1] - p[1] * b[0]);
    const denumerator = @abs(@as(i32, a[0]) * b[1] - @as(i32, a[1]) * b[0]);
    return if (numerator % denumerator != 0) 0 else numerator / denumerator;
}

// Then in the caller, skip if the return value is 0.
if (count_tokens(a, b, p) == 0) continue;

This works, but it’s easy to forget to check for those values, or worse, to accidentally treat them as valid results. Zig improves on this with optional types. If a function might not return a value, you can return ?T instead of T. This also forces the caller to handle the null case. Unlike C, null isn’t a pointer but a more general concept. Zig treats null as the absence of a value for any type, just like Rust’s Option<T>.

The count_tokens function can be refactored to:

// Return null instead if there's no valid result.
fn count_tokens(a: [2]u8, b: [2]u8, p: [2]i64) ?u64 {
    const numerator = @abs(p[0] * b[1] - p[1] * b[0]);
    const denumerator = @abs(@as(i32, a[0]) * b[1] - @as(i32, a[1]) * b[0]);
    return if (numerator % denumerator != 0) null else numerator / denumerator;
}

// The caller is now forced to handle the null case.
if (count_tokens(a, b, p)) |n_tokens| {
    // logic only runs when n_tokens is not null.
}

Zig also has a concept of error unions, where a function can return either a value or an error. In Rust, this is Result<T>. You could also use error unions instead of optionals for count_tokens; Zig doesn’t force a single approach. I come from Clojure, where returning nil for an error or missing value is common.

Grid Padding

This year has a lot of 2D grid puzzles (arguably too many). A common feature of grid-based algorithms is the out-of-bounds check. Here’s what it usually looks like:

fn dfs(map: [][]u8, position: [2]i8) u32 {
    const x, const y = position;
    
    // Bounds check here.
    if (x < 0 or y < 0 or x >= map.len or y >= map[0].len) return 0;

    if (map[x][y] == .visited) return 0;
    map[x][y] = .visited;

    var result: u32 = 1;
    for (directions) | direction| {
        result += dfs(map, position + direction);
    }
    return result;
}

This is a typical recursive DFS function. After doing a lot of this, I discovered a nice trick that not only improves code readability, but also its performance. The trick here is to pad the grid with sentinel characters that mark out-of-bounds areas, i.e. add a border to the grid.

Here’s an example from day 6:

Original map:               With borders added:
                            ************
....#.....                  *....#.....*
.........#                  *.........#*
..........                  *..........*
..#.......                  *..#.......*
.......#..        ->        *.......#..*
..........                  *..........*
.#..^.....                  *.#..^.....*
........#.                  *........#.*
#.........                  *#.........*
......#...                  *......#...*
                            ************

You can use any value for the border, as long as it doesn’t conflict with valid values in the grid. With the border in place, the bounds check becomes a simple equality comparison:

const border = '*';

fn dfs(map: [][]u8, position: [2]i8) u32 {
    const x, const y = position;
    if (map[x][y] == border) { // We are out of bounds
        return 0;
    }
    // ...
}

This is much more readable than the previous code. Plus, it’s also faster since we’re only doing one equality check instead of four range checks.

That said, this isn’t a one-size-fits-all solution. This only works for algorithms that traverse the grid one step at a time. If your logic jumps multiple tiles, it can still go out of bounds (except if you increase the width of the border to account for this). This approach also uses a bit more memory than the regular approach as you have to store more characters.

SIMD Vectors

This could also go in the performance section, but I’m including it here because the biggest benefit I get from using SIMD in Zig is the improved code readability. Because Zig has first-class support for vector types, you can write elegant and readable code that also happens to be faster.

If you’re not familiar with vectors, they are a special collection type used for Single instruction, multiple data (SIMD) operations. SIMD allows you to perform computation on multiple values in parallel using only a single CPU instruction, which often leads to some performance boosts. ^{[6] [6]}

I mostly use vectors to represent positions and directions, e.g. for traversing a grid. Instead of writing code like this:

next_position = .{ position[0] + direction[0], position[1] + direction[1] };

You can represent position and direction as 2-element vectors and write code like this:

next_position = position + direction;

This is much nicer than the previous version!

Day 25 is another good example of a problem that can be solved elegantly using vectors:

var result: u64 = 0;
for (self.locks.items) |lock| { // lock is a vector
    for (self.keys.items) |key| { // key is also a vector
        const fitted = lock + key > @as(@Vector(5, u8), @splat(5));
        const is_overlap = @reduce(.Or, fitted);
        result += @intFromBool(!is_overlap);
    }
}

Expressing the logic as vector operations makes the code cleaner since you don’t have to write loops and conditionals as you typically would in a traditional approach.

Performance Tips

The tips below are general performance techniques that often help, but like most things in software engineering, “it depends”. These might work 80% of the time, but performance is often highly context-specific. You should benchmark your code instead of blindly following what other people say.

This section would’ve been more fun with concrete examples, step-by-step optimisations, and benchmarks, but that would’ve made the post way too long. Hopefully, I’ll get to write something like that in the future. ^{[7] [7]}

Minimise Allocations

Whenever possible, prefer static allocation. Static allocation is cheaper since it just involves moving the stack pointer vs dynamic allocation which has more overhead from the allocator machinery. That said, it’s not always the right choice since it has some limitations, e.g. stack size is limited, memory size must be compile-time known, its lifetime is tied to the current stack frame, etc.

If you need to do dynamic allocations, try to reduce the number of times you call the allocator. The number of allocations you do matters more than the amount of memory you allocate. More allocations mean more bookkeeping, synchronisation, and sometimes syscalls.

A simple but effective way to reduce allocations is to reuse buffers, whether they’re statically or dynamically allocated. Here’s an example from day 10. For each trail head, we want to create a set of trail ends reachable from it. The naive approach is to allocate a new set every iteration:

for (self.trail_heads.items) |trail_head| {
    var trail_ends = std.AutoHashMap([2]u8, void).init(self.allocator);
    defer trail_ends.deinit();
    
    // Set building logic...
}

What you can do instead is to allocate the set once before the loop. Then, each iteration, you reuse the set by emptying it without freeing the memory. For Zig’s std.AutoHashMap, this can be done using the clearRetainingCapacity method:

var trail_ends = std.AutoHashMap([2]u8, void).init(self.allocator);
defer trail_ends.deinit();

for (self.trail_heads.items) |trail_head| {
    trail_ends.clearRetainingCapacity();
    
    // Set building logic...
}

If you use static arrays, you can also just overwrite existing data instead of clearing it.

A step up from this is to reuse multiple buffers. The simplest form of this is to reuse two buffers, i.e. double buffering. Here’s an example from day 11:

// Initialise two hash maps that we'll alternate between.
var frequencies: [2]std.AutoHashMap(u64, u64) = undefined;
for (0..2) |i| frequencies[i] = std.AutoHashMap(u64, u64).init(self.allocator);
defer for (0..2) |i| frequencies[i].deinit();

var id: usize = 0;
for (self.stones) |stone| try frequencies[id].put(stone, 1);

for (0..n_blinks) |_| {
    var old_frequencies = &frequencies[id % 2];
    var new_frequencies = &frequencies[(id + 1) % 2];
    id += 1;

    defer old_frequencies.clearRetainingCapacity();

    // Do stuff with both maps...
}

Here we have two maps to count the frequencies of stones across iterations. Each iteration will build up new_frequencies with the values from old_frequencies. Doing this reduces the number of allocations to just 2 (the number of buffers). The tradeoff here is that it makes the code slightly more complex.

Make Your Data Smaller

A performance tip people say is to have “mechanical sympathy”. Understand how your code is processed by your computer. An example of this is to structure your data so it works better with your CPU. For example, keep related data close in memory to take advantage of cache locality.

Reducing the size of your data helps with this. Smaller data means more of it can fit in cache. One way to shrink your data is through bit packing. This depends heavily on your specific data, so you’ll need to use your judgement to tell whether this would work for you. I’ll just share some examples that worked for me.

The first example is in day 6 part two, where you have to detect a loop, which happens when you revisit a tile from the same direction as before. To track this, you could use a map or a set to store the tiles and visited directions. A more efficient option is to store this direction metadata in the tile itself.

There are only four tile types, which means you only need two bits to represent the tile types as an enum. If the enum size is one byte, here’s what the tiles look like in memory:

.obstacle -> 00000000
.path     -> 00000001
.visited  -> 00000010
.path     -> 00000011

As you can see, the upper six bits are unused. We can store the direction metadata in the upper four bits. One bit for each direction. If a bit is set, it means that we’ve already visited the tile in this direction. Here’s an illustration of the memory layout:

        direction metadata   tile type
           ┌─────┴─────┐   ┌─────┴─────┐
┌────────┬─┴─┬───┬───┬─┴─┬─┴─┬───┬───┬─┴─┐
│ Tile:  │ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │ 1 │ 0 │
└────────┴─┬─┴─┬─┴─┬─┴─┬─┴───┴───┴───┴───┘
   up bit ─┘   │   │   └─ left bit
    right bit ─┘ down bit

If your language supports struct packing, you can express this layout directly: ^{[8] [8]}

const Tile = packed struct(u8) {
    const TileType = enum(u4) { obstacle, path, visited, exit };

    up: u1 = 0,
    right: u1 = 0,
    down: u1 = 0,
    left: u1 = 0,
    tile: TileType,

    // ...
}

Doing this avoids extra allocations and improves cache locality. Since the directions metadata is colocated with the tile type, all of them can fit together in cache. Accessing the directions just requires some bitwise operations instead of having to fetch them from another region of memory.

Another way to do this is to represent your data using alternate number bases. Here’s an example from day 23. Computers are represented as two-character strings made up of only lowercase letters, e.g. "bc", "xy", etc. Instead of storing this as a [2]u8 array, you can convert it into a base-26 number and store it as a u16. ^{[9] [9]}

Here’s the idea: map 'a' to 0, 'b' to 1, up to 'z' as 25. Each character in the string becomes a digit in the base-26 number. For example, "bc" ( [2]u8{ 'b', 'c' }) becomes the base-10 number 28 ($1\times 26+2=28$). If we represent this using the base-64 character set, it becomes 12 ('b' = 1, 'c' = 2).

While they take the same amount of space (2 bytes), a u16 has some benefits over a [2]u8:

1. It fits in a single register, whereas you need two for the array.
2. Comparison is faster as there is only a single value to compare.

Reduce Branching

I won’t explain branchless programming here; Algorithmica explains it way better than I can. While modern compilers are often smart enough to compile away branches, they don’t catch everything. I still recommend writing branchless code whenever it makes sense. It also has the added benefit of reducing the number of codepaths in your program.

Again, since performance is very context-dependent, I’ll just show you some patterns I use. Here’s one that comes up often:

if (is_valid_report(report)) {
    result += 1;
}

Instead of the branch, cast the bool into an integer directly:

result += @intFromBool(is_valid_report(report))

Another example is from day 6 (again!). Recall that to know if a tile has been visited from a certain direction, we have to check its direction bit. Here’s one way to do it:

fn has_visited(tile: Tile, direction: Direction) bool {
    switch (direction) {
        .up => return self.up == 1,
        .right => return self.right == 1,
        .down => return self.down == 1,
        .left => return self.left == 1,
    }
}

This works, but it introduces a few branches. We can make it branchless using bitwise operations:

fn has_visited(tile: Tile, direction: Direction) bool {
    const int_tile = std.mem.nativeToBig(u8, @bitCast(tile));
    const mask = direction.mask();
    const bits = int_tile & 0xff; // Get only the direction bits
    return bits & mask == mask;
}

While this is arguably cryptic and less readable, it does perform better than the switch version.

Avoid Recursion

The final performance tip is to prefer iterative code over recursion. Recursive functions bring the overhead of allocating stack frames. While recursive code is more elegant, it’s also often slower unless your language’s compiler can optimise it away, e.g. via tail-call optimisation. As far as I know, Zig doesn’t have this, though I might be wrong.

Recursion also has the risk of causing a stack overflow if the execution isn’t bounded. This is why code that is mission- or safety-critical avoids recursion entirely. It’s in TigerBeetle’s TIGERSTYLE and also NASA’s Power of Ten.

Iterative code can be harder to write in some cases, e.g. DFS maps naturally to recursion, but most of the time it is significantly faster, more predictable, and safer than the recursive alternative.

Benchmarks

I ran benchmarks for all 25 solutions in each of Zig’s optimisation modes. You can find the full results and the benchmark script in my GitHub repository. All benchmarks were done on an Apple M3 Pro.

As expected, ReleaseFast produced the best result with a total runtime of 85.1 ms. I’m quite happy with this, considering the two constraints that limited the number of optimisations I can do to the code:

• Parts should be solved separately - Some days can be solved in a single go, e.g. day 10 and day 13, which could’ve saved a few milliseconds.
• No concurrency or parallelism - My slowest days are the compute-heavy days that are very easily parallelisable, e.g. day 6, day 19, and day 22. Without this constraint, I can probably reach sub-20 milliseconds total(?), but that’s for another time.

You can see the full benchmarks for ReleaseFast in the table below:

A weird thing I found when benchmarking is that for day 6 part two, ReleaseSafe actually ran faster than ReleaseFast (13,189.0 µs vs 24,370.7 µs). Their outputs are the same, but for some reason, ReleaseSafe is faster even with the safety checks still intact.

The Zig compiler is still very much a moving target, so I don’t want to dig too deep into this, as I’m guessing this might be a bug in the compiler. This weird behaviour might just disappear after a few compiler version updates.

Reflections

Looking back, I’m really glad I decided to do Advent of Code and followed through to the end. I learned a lot of things. Some are useful in my professional work, some are more like random bits of trivia. Going with Zig was a good choice too. The language is small, simple, and gets out of your way. I learned more about algorithms and concepts than the language itself.

Besides what I’ve already mentioned earlier, here are some examples of the things I learned:

• The concept of Manhattan distance.
• Cliques and the Bron-Kerbosch algorithm.
• The ripple-carry adder circuit.
• LaTeX math syntax with MathJax, which I used in my write-ups.
• In Python, you can use complex numbers to traverse 2D grids!

Some of my self-imposed constraints and rules ended up being helpful. I can still (mostly) understand the code I wrote a few months ago. Putting all of the code in a single file made it easier to read since I don’t have to context switch to other files all the time.

However, some of them did backfire a bit, e.g. the two constraints that limit how I can optimise my code. Another one is the “hardcoding allowed” rule. I used a lot of magic numbers, which helped to improve performance, but I didn’t document them, so after a while, I don’t even remember how I got them. I’ve since gone back and added explanations in my write-ups, but next time I’ll remember to at least leave comments.

One constraint I’ll probably remove next time is the no concurrency rule. It’s the biggest contributor to the total runtime of my solutions. I don’t do a lot of concurrent programming, even though my main language at work is Go, so next time it might be a good idea to use Advent of Code to level up my concurrency skills.

I also spent way more time on these puzzles than I originally expected. I optimised and rewrote my code multiple times. I also rewrote my write-ups a few times to make them easier to read. This is by far my longest side project yet. It’s a lot of fun, but it also takes a lot of time and effort. I almost gave up on the write-ups (and this blog post) because I don’t want to explain my awful day 15 and day 16 code. I ended up taking a break for a few months before finishing it, which is why this post is published in August lol.

Just for fun, here’s a photo of some of my notebook sketches that helped me visualise my solutions. See if you can guess which days these are from:

What’s Next?

So… would I do it again? Probably, though I’m not making any promises. If I do join this year, I’ll probably stick with Zig. I had my eyes on Zig since the start of 2024, so Advent of Code was the perfect excuse to learn it. This year, there aren’t any languages in particular that caught my eye, so I’ll just keep using Zig, especially since I have a proper setup ready.

If you haven’t tried Advent of Code, I highly recommend checking it out this year. It’s a great excuse to learn a new language, improve your problem-solving skills, or just learn something new. If you’re eager, you can also do the previous years’ puzzles as they’re still available.

One of the best aspects of Advent of Code is the community. The Advent of Code subreddit is a great place for discussion. You can ask questions and also see other people’s solutions. Some people also post really cool visualisations like this one. They also have memes!

Permalink

Learn Ring - 3. Prerequisite

Notes

• Prerequisite
  • git
    • https://git-scm.com/
  • IDE
    • VSCode
      • https://code.visualstudio.com/
    • Antigravity
      • https://antigravity.google/
  • Clojure
    • https://clojure-book.gitlab.io/
  • Web Tech
    • HTML
      • https://www.w3schools.com/html/
    • CSS
      • https://www.w3schools.com/css/
    • JavaScript
      • https://www.w3schools.com/js/
    • SQL
      • https://www.w3schools.com/sql/

Permalink

Clojure Documentary Trailer (Mar 26, 2026)

Permalink

Memory That Collaborates

def catalog := d/connect({:store {:backend :s3, :bucket "team-a"}})
def inventory := d/connect({:store {:backend :s3, :bucket "team-b"}})

d/q('[:find ?name ?price ?stock
       :in $cat $inv
       :where [$cat ?p :product/sku   ?sku]
              [$cat ?p :product/name  ?name]
              [$cat ?p :product/price ?price]
              [$inv ?i :stock/sku     ?sku]
              [$inv ?i :stock/count   ?stock]
              [(> ?stock 0)]], @catalog, @inventory)

(def catalog   (d/connect {:store {:backend :s3 :bucket "team-a"}}))
(def inventory (d/connect {:store {:backend :s3 :bucket "team-b"}}))

(d/q '[:find ?name ?price ?stock
       :in $cat $inv
       :where [$cat ?p :product/sku   ?sku]
              [$cat ?p :product/name  ?name]
              [$cat ?p :product/price ?price]
              [$inv ?i :stock/sku     ?sku]
              [$inv ?i :stock/count   ?stock]
              [(> ?stock 0)]]
  @catalog @inventory)

;; Last quarter's catalog crossed with current inventory
def old-catalog := d/as-of(@catalog, #inst "2025-11-01")

d/q('[:find ?name ?stock
       :in $cat $inv
       :where [$cat ?p :product/sku  ?sku]
              [$cat ?p :product/name ?name]
              [$inv ?i :stock/sku    ?sku]
              [$inv ?i :stock/count  ?stock]], old-catalog, @inventory)

;; Last quarter's catalog crossed with current inventory
(def old-catalog (d/as-of @catalog #inst "2025-11-01"))

(d/q '[:find ?name ?stock
       :in $cat $inv
       :where [$cat ?p :product/sku  ?sku]
              [$cat ?p :product/name ?name]
              [$inv ?i :stock/sku    ?sku]
              [$inv ?i :stock/count  ?stock]]
  old-catalog @inventory)

require('[datahike.api :as d])

;; Two independent databases
def catalog-cfg := {:store {:backend :memory, :id java.util.UUID/randomUUID()},
                    :schema-flexibility :read}

def inventory-cfg := {:store {:backend :memory, :id java.util.UUID/randomUUID()},
                      :schema-flexibility :read}

d/create-database(catalog-cfg)
d/create-database(inventory-cfg)

def catalog := d/connect(catalog-cfg)
def inventory := d/connect(inventory-cfg)

;; Team A: products
d/transact(catalog,
           [{:product/sku "W001", :product/name "Widget", :product/price 9.99},
            {:product/sku "G002", :product/name "Gadget", :product/price 24.5},
            {:product/sku "T003",
             :product/name "Thingamajig",
             :product/price 3.75}])

;; Team B: stock levels
d/transact(inventory,
           [{:stock/sku "W001", :stock/count 140},
            {:stock/sku "G002", :stock/count 0},
            {:stock/sku "T003", :stock/count 58}])

;; Join: in-stock products with price
d/q('[:find ?name ?price ?stock
       :in $cat $inv
       :where [$cat ?p :product/sku   ?sku]
              [$cat ?p :product/name  ?name]
              [$cat ?p :product/price ?price]
              [$inv ?i :stock/sku     ?sku]
              [$inv ?i :stock/count   ?stock]
              [(> ?stock 0)]], @catalog, @inventory)
;; => #{["Widget" 9.99 140] ["Thingamajig" 3.75 58]}

(require '[datahike.api :as d])

;; Two independent databases
(def catalog-cfg  {:store {:backend :memory
                           :id (java.util.UUID/randomUUID)}
                   :schema-flexibility :read})
(def inventory-cfg {:store {:backend :memory
                            :id (java.util.UUID/randomUUID)}
                    :schema-flexibility :read})

(d/create-database catalog-cfg)
(d/create-database inventory-cfg)

(def catalog  (d/connect catalog-cfg))
(def inventory (d/connect inventory-cfg))

;; Team A: products
(d/transact catalog
  [{:product/sku "W001" :product/name "Widget"      :product/price 9.99}
   {:product/sku "G002" :product/name "Gadget"      :product/price 24.50}
   {:product/sku "T003" :product/name "Thingamajig" :product/price 3.75}])

;; Team B: stock levels
(d/transact inventory
  [{:stock/sku "W001" :stock/count 140}
   {:stock/sku "G002" :stock/count 0}
   {:stock/sku "T003" :stock/count 58}])

;; Join: in-stock products with price
(d/q '[:find ?name ?price ?stock
       :in $cat $inv
       :where [$cat ?p :product/sku   ?sku]
              [$cat ?p :product/name  ?name]
              [$cat ?p :product/price ?price]
              [$inv ?i :stock/sku     ?sku]
              [$inv ?i :stock/count   ?stock]
              [(> ?stock 0)]]
  @catalog @inventory)
;; => #{["Widget" 9.99 140] ["Thingamajig" 3.75 58]}

Permalink

LLM Integration for Internal Tools & SaaS Products (2026 Strategy Guide)

By 2026, AI software development with a native LLM layer is not an extra feature anymore- it is the standard requirement. In fact, LLM integration for SaaS has become the standard for modern platforms. If business software can not learn, adapt, or automate on its own, it is already outdated. Whether teams are automating tedious tasks within the organization or turning SaaS into something that thinks for itself depends on how closely the AI is linked to data and how the team works. 

Honestly, the pace of AI software development has been unpredictable. What was experimental just a few years back is now completely normal. All organizations, from scrappy startups to large enterprises, are integrating LLMs right into their SaaS application development pipelines. And it is not just about adding a chatbot on top. The real shift? AI is becoming embedded in the core of products, reshaping how work gets done.

What’s pushing this change? Three big things:

• People want scalable software solutions that respond instantly to users’ actions.
• AI‑powered business intelligence (BI) is not just about dashboards anymore- it is about getting real answers, in plain language, from the data.
• Companies care more than ever about privacy‑first AI software development and compliance, whether it is GDPR, SOC 2, or the new AI-related rules.

By 2026, skipping LLM integration is a sure way to fall behind. Competitors are already building with AI in mind from the very beginning. The strategy guide has really got better, too. Now businesses have everything, ranging from machine learning to smart ways to keep SaaS data separate for different customers. It is not guesswork anymore- it is a repeatable, scalable framework. If a business doesn’t adapt, it risks being left behind.

Internal Tools vs. SaaS Products- Different Goals, Different Architectures

By 2026, companies won’t be debating whether to use AI anymore. The real question is how much of their systems should rely on it.

🔗 Gartner actually predicts that over 80% of enterprises will have generative AI running in production by then. That’s a massive jump from less than 5% just a few years ago. 

It is a big shift, and it highlights that building internal AI tools is a totally different game from SaaS application development.

Comparison Table

AI-Powered Internal Tools for Smarter Workflows

Internal tools are all about making work smoother and faster. With AI, that usually means assistants that summarize meetings, draft documents, or help engineers find information without having to look all over. The goal is to focus on ROI and efficiency, not market dominance.

SaaS Application Development with Embedded AI Layers

SaaS platforms have a different mission. They need to build scalable software solutions and keep users coming back. Here, AI gets right into the workflow- LLMs offer smart suggestions, guide new users, and AI‑powered business intelligence (BI) features that actually make sense of data. This is where SaaS application development no longer just integrates chatbots but starts to feel truly AI-native.

Compliance & Privacy‑First AI Software Development

Compliance matters everywhere. Internal teams worry about leaks and passing SOC2 audits. SaaS providers deal with even tougher requirements- GDPR, privacy across lots of customers, the works. The answer? Develop privacy‑first AI software. Anonymize sensitive data before it reaches an external model. That builds trust and keeps everything on the right side of the rules.

Transforming Internal Workflows with AI Agents

The Death of Search, The Rise of Retrieval

Search is going out of use. Retrieval is taking over. Instead of forcing employees to scroll through endless wikis, Slack threads, or Jira tickets, AI steps in with Retrieval‑Augmented Generation (RAG). These days, individuals only need to ask a query, and the AI will find the appropriate information and provide a concise response.

✅️ Example: A developer asks, “What’s the latest update on the payment API?” No digging through Jira. The AI finds the right entries and gives a clear update. It seems small, but over time it saves hours.

Automating the Boring Stuff

AI agents shine when it comes to routine tasks. They can:

• Summarize meetings and automatically send out notes.
• Turn chat discussions into Jira tickets.
• Generate code documentation automatically.

✅️ Example: The AI generates Jira tickets, assigns tasks, and gives a summary after planning the sprint. Engineers skip the admin work and get back to actual engineering.

Engineering Productivity Measurement

Teams are not just guessing about the impact of AI- they track it:

• Discovery time drops. Developers find what they need faster.
• Developer satisfaction goes up. AI tools smooth out daily work.
• Routine tasks get done way faster.

✅️ Example: After rolling out RAG-based tools, a company saw developers spend 40% less time searching for documentation.

🔗 According to a McKinsey study, generative AI can boost the global economy by $2.6 to $4.4 trillion every year, just by making business functions more productive.

AI‑Native SaaS Application Development: Beyond the Chatbox

Most SaaS platforms started with simple chatbots or basic support features. But AI‑native SaaS changes the approach. Instead of adding AI later, it is built into the product’s core. Workflows shift in real time. Insights emerge before even asking. Personalization just happens- without having to do a thing.

Embedded Intelligence for Scalable Software Solutions

Forget sitting around waiting for users to type into a help chat. Now, AI takes the lead. In a project management tool, it might spot a stuck task and remind the user of the next steps. A CRM identifies leads that are being overlooked. 

• From sidebar chat → Proactive workflow suggestions.
• Intelligence is not simply added like a secondary consideration- it is built in from the start.
• And because of that, these tools scale easily to thousands of users. No fuss, no endless setup.

AI‑Powered Business Intelligence (BI) in Saas Platforms

BI dashboards are not just about flashy graphs anymore. AI steps in and explains what those trends actually mean, points out unusual spikes, and even recommends the next move- entirely in simple terms.

• Instead of complicated visuals, teams get clear reports.
• Insights feel personal, tailored to each person’s role. 
• Best of all, teams make faster decisions without waiting for a data analyst to translate the numbers.

Hyper‑Personalization through Privacy‑First AI Software Development

Personalization used to mean just showing the right product. Now, AI-native SaaS is shaped by what each user really wants, all while keeping privacy front and center. 

• Onboarding paths change instantly as users explore.
• Recommendations feel beneficial rather than enforced.
• With privacy‑first AI, teams keep trust and compliance.

Why It Matters

AI-native SaaS is not about eye-catching new features. It is about building real intelligence right into the product, so people waste less time clicking around and get more value from the start. When it is done right, it scales up, protects privacy, and turns software into something that feels less like a tool and more like a true partner.

Technical Implementation- Machine Learning with Clojure(The Flexiana Approach)

Why Clojure Works So Well for LLM Orchestration

At Flexiana, Clojure is the backbone of our AI systems. Its functional style and immutability keep code stable and predictable, even as systems grow. That is a big deal when companies are trying to keep orchestration layers simple to maintain and scalable. 

• Immutability keeps data consistent across pipelines. In practice, that means fewer weird side effects and reliable results.
  
• The REPL-driven workflow is a lifesaver, too. Developers tweak prompts and models instantly. No waiting- just fast feedback and quick fixes.
  
• Flexiana relies on Clojure’s strengths for LLM orchestration. We build clean, functional pipelines to handle model calls, manage responses, and plug into other systems. No extra complexity. 

Clojure code for LLM orchestration.

(ns orchestration
  (:require [clj-http.client :as http]
            [cheshire.core :as json]
            [clojure.spec.alpha :as s]))

(def openai-api-key "YOUR_API_KEY")

(s/def ::summary (s/and vector? #(= 3 (count %)) (s/every string?)))
(s/def ::response (s/keys :req-un [::summary]))

(defn call-llm [prompt]
  (let [response (http/post "https://api.openai.com/v1/chat/completions"
                           {:headers {"Authorization" (str "Bearer " openai-api-key)
                                      "Content-Type" "application/json"}
                            :body (json/generate-string
                                   {:model "gpt-4o-mini"
                                    :temperature 0
                                    :messages [{:role "user"
                                                :content prompt}]})})]
    (-> response :body (json/parse-string true))))

(defn build-prompt [user-input]
  (str
   "Summarize the following text into exactly 3 bullet points.\n\n"
   "Return ONLY valid JSON:\n"
   "{\"summary\": [\"point1\", \"point2\", \"point3\"]}\n\n"
   user-input))

(defn safe-parse-json [s]
  (try
    (json/parse-string s true)
    (catch Exception _
      nil)))

(defn validate-response [parsed]
  (if (s/valid? ::response parsed)
    parsed
    (throw (ex-info "Invalid LLM structure"
                    {:errors (s/explain-data ::response parsed)
                     :data parsed}))))

(defn extract-structured-answer [llm-response]
  (let [content (get-in llm-response [:choices 0 :message :content])
        parsed (safe-parse-json content)]
    (if parsed
      (validate-response parsed)
      (throw (ex-info "Failed to parse JSON"
                      {:raw content})))))

(defn orchestrate-llm-workflow [user-input]
  (let [prompt (build-prompt user-input)
        raw-response (call-llm prompt)
        validated (extract-structured-answer raw-response)]
    (:summary validated)))

(orchestrate-llm-workflow
 "Large language models are transforming how businesses automate workflows.")

What this orchestration does

1. It takes the input and builds the prompt.
2. It calls the LLM API.
3. It processes the response to pull out what matters.
4. It wraps all of that into one neat orchestration function.

Model Selection Strategy for AI‑Powered Business Intelligence (BI)

Flexiana’s model selection is not about chasing the latest and greatest. We keep it practical- balancing expenses, efficiency, and the specific job requirements.

• For heavyweight analysis or deeper reasoning, we use cutting-edge frontier models like GPT-5.3, or Opus 4.6 (as of March 24, 2026). These models dig deep and extract more valuable insights, but they do cost more.
  
• For daily BI work- routine questions, dashboards, lightweight reports- we go with smaller models, especially Sonnet 4.6 . These run faster and are affordable.
  
• Most of the time, Flexiana mixes both. Frontier models handle the big, high-value analysis. Everyday tasks are handled by smaller models, allowing solutions to grow without wasting money.

Cost vs. performance table comparing frontier vs. small models vs. Hybrid strategy

Why Flexiana’s Approach Stands Out

Flexiana actually cares about building systems that work- real solutions for real problems. We use Clojure and smart model selection to build BI tools that not only work on day one but also keep up as the business grows. Companies get valuable insights, efficient use of their resources, and a configuration that works well.

Cracking the Multi‑Tenant AI Puzzle in SaaS Application Development

Let’s be real: integrating AI with a SaaS platform is no simple task. Multi-tenant systems need to balance many customers at once, all while maintaining high performance, strong privacy, and unbreakable security. Flexiana focuses on what truly matters. 

❶ Data Isolation

When teams have numerous tenants, they can not mess around with data separation. Every customer’s info has to stay private – no exceptions, no accidental crossovers. 

Flexiana draws clear lines from the database all the way up to the AI layer. Strong tenant boundaries, workflows that keep data in place, and pipelines that scale without losing trust. Customers are assured that their data remains secure even as the system expands.

❷ Prompt Injection Defense

Large language models are powerful, but not flawless. Malicious users sometimes trick models into breaking rules or revealing hidden info.

Flexiana blocks them at the checkpoint, with built-in filters that detect suspicious input, validation layers that enforce safe responses, and monitoring that detects emerging tactics. With these protections, users do not have to worry about AI misuse.

❸ Privacy‑First AI Software Development for Multi‑Tenant SaaS

Flexiana does not add privacy as an afterthought- we integrate it from the very beginning. Every feature, every layer, follows strict privacy standards and keeps tenant data confidential. We stick to EU GDPR guidelines and give customers real control over their info, keeping everything transparent. This way, the AI is not just smart; it is responsible.

Why It All Matters

Trust is everything in multi-tenant SaaS. Flexiana’s focus-tight data isolation, strong defenses, and a privacy-first mindset-means our AI systems stay secure, scale up easily, and follow the guidelines. That is how we build something customers can actually rely on.

Measuring the ROI of LLM Integration in AI Software Development

Bringing large language models (LLMs) into business software is neither inexpensive nor fast. Businesses want to know if it is actually worth the effort. ROI is not just about saving money. It is about moving faster, getting people on board, and making things run smoother. At Flexiana, we break it down into three main areas.

Internal Tools

LLMs can take a lot of the pain out of daily work. Companies see the benefits when teams solve problems faster and feel like they actually have the right tools.

• Time to Resolution: Track how long it takes to fix issues, before and after adding AI. If tickets used to run for two hours and now get wrapped up in thirty minutes, that is real progress.
• Employee Satisfaction: Just ask the teams. Are these tools helping? Simple surveys or regular feedback can help to identify if AI really makes their work easier.

These figures demonstrate whether AI is indeed simplifying tasks rather than adding more processes.

SaaS Products

For customer‑facing platforms, ROI comes from how much people use the new features and how much less support they need.

• Feature Adoption: Check how often customers use the AI features. If people love them and use them a lot, the company knows they are useful and easy to figure out.
• Support Ticket Reduction: Monitor support ticket volume. If customers need less help because AI guides them correctly, everyone wins. Less support means lower costs and happier users.

This helps companies see whether AI is actually improving their products and removing obstacles to progress.

Cost Optimization                                                                                                                      

Behind the scenes, businesses have to make smart choices, since running LLMs is not free. There is a clear difference between using external APIs and running smaller models in‑house. 

An API may seem low-cost at a few cents per request, but costs rise quickly. If demand rises, switching to a local quantized model saves money over time. It is all about finding that right balance between staying flexible and saving in the long run. An ROI calculator helps with that.

An ROI calculator 

Why This Matters

ROI isn’t just a box to tick to prove AI is worth it. It is about making better decisions as your business grows. When companies track things like internal efficiency, how customers are using the product, and what it costs to keep everything running, they actually see where LLMs make a difference- and where they need to make changes.

❓What People Often Ask (FAQs) 

Q1: Will integrating an LLM make my SaaS too expensive?

Not always. APIs are easy to set up, but costs rise as usage grows. Running smaller models yourself takes more work at first, but you end up saving money in the long run.

Q2: How does privacy‑first AI software development prevent hallucinations?

It limits how much data the AI sees and puts safety checks in place. That reduces mistakes, keeps data safe, and supports compliance. Plus, it builds trust.

Q3: Do I need a dedicated AI software development team?

If you want to move fast and handle growth, a team helps a lot. When you are just getting started, you can stick with APIs or managed services- they get the job done. Once your SaaS starts to expand quickly, having real experts on board makes everything run more smoothly and improves what you deliver. 

Q4: What does AI‑powered business intelligence (BI) do for SaaS?

It analyzes consumer data and identifies patterns. Then it gives guidance on shaping your product.BI takes all that raw information and turns it into something you can actually use, making your platform smarter and more useful.

Q5: How do scalable software solutions support AI integration?

They let you handle more users and data without slowing down. When you add more AI features, your system stays fast, and costs remain controlled.

Q6: Can I use machine learning with Clojure for SaaS AI?

Definitely. Clojure’s concurrency capabilities and design make it a good option for machine learning pipelines. It helps you add AI features that are reliable and easy to maintain.

Here’s The Bottom Line 

If companies are building SaaS applications, LLM integration is not just a nice-to-have anymore- it is expected. Teams have two main paths. They can plug in external APIs for a faster launch, or can run smaller models in-house if they want more control. It really depends on what they want to invest in, how big they want to grow, and how closely they need to monitor things.

Sticking to privacy‑first design and building software that scales- this is what keeps the business platform solid. When teams follow smart AI development practices, customers can actually trust what they see. AI-powered business intelligence is not just a set of buzzwords, either. It gives teams a clear view of customer behavior, helps them spot trends before everyone else, and guides product decisions with real data. And if companies are working on something more advanced, tools like machine learning with Clojure make it possible to build pipelines that don’t break down and are pretty straightforward to maintain.

At the end of the day, integrating AI is not about chasing trends. It is about making SaaS tools that actually work and scale with business goals.

Want to see what that looks like for your business? Book a consultation with Flexiana, and let’s figure out how LLMs can shape your SaaS strategy.

The post LLM Integration for Internal Tools & SaaS Products (2026 Strategy Guide) appeared first on Flexiana.

Permalink

From Functions to Data - Evolving a Pull-Pattern API

Context

We built our pull-pattern API on lasagna-pull, a library designed by Robert Luo that lets clients send EDN patterns to describe what data they want. The core pattern-matching engine is solid. But as we added more resources, roles, and mutation types, we wanted a different model for how patterns interact with data sources and side effects. This article is about the design decisions behind lasagna-pattern, the successor stack that replaces the function-calling handler layer while building on the same pull-pattern ideas.

For context on what the new architecture looks like, see Building a Pure Data API with Lasagna Pattern. For the monorepo structure that hosts the libraries, see Clojure Monorepo with Babashka.

The old model: patterns call functions

In lasagna-pull, the core mechanism was :with. Patterns contained function calls: (list :key :with [args]) told the engine to look up :key in a data map, call the function stored there, and pass it args. Functions returned {:response ... :effects ...}.

Here is what a few common operations looked like.

List all dashboards (read):

{:dashboards
 {(list :role/user :with [])
  {(list :self :with []) [{:title '? :id '?}]}}}

The outer :with checked authorization. The inner :with called a function to list all entries. The vector with map shape [{:title '? :id '?}] described which fields to return.

Read by ID:

{:dashboards
 {(list :role/user :with [])
  {(list :dashboard :with [{:id 123} :read])
   {:title '? :content '?}}}}

The :read action dispatched inside the function via case.

Create:

{:dashboards
 {(list :role/user :with [])
  {(list :dashboard :with [{:title "New" :content "..."} :save])
   {:id '? :title '?}}}}

Same function, different action. The function returned {:response data :effects {:rama {...}}}, and a separate executor ran the side effects after the pattern resolved.

On the server, the data map was a nested structure of functions:

(defn pullable-data [session]
  {:dashboards
   {:role/user (with-role session :user
     (fn []
       {:dashboard (fn [data action]
                     (case action
                       :read   {:response (get-dash (:id data))}
                       :save   {:response data
                                :effects  {:rama {...}}}
                       :delete {:response true
                                :effects  {:rama {...}}}))}))}})

Authorization was a function wrapper: with-role took a session, a role keyword, and a thunk that returned the data map. If the role was missing, the thunk never ran.

The saturn handler: pure by design

This architecture had a name: the "saturn handler" pattern, designed by Robert Luo. The idea was to split request handling into three stages:

1. Injectors provided dependencies (DB snapshot) to the request
2. Saturn handler (purely functional) ran the pull pattern, accumulated {:response, :effects-desc, :session} with zero side effects
3. Executors took the effects descriptions and actually ran them (DB transactions, cache invalidation)

The context-of mechanism coordinated accumulation during pattern resolution. A modifier function extracted :response, :effects, and :session from each operation result. A finalizer attached the accumulated effects and session updates to the final result. The handler itself never touched the database for writes.

;; Saturn handler: purely functional, no side effects
(defn saturn-handler [{:keys [db session] :as req}]
  (let [pattern (extract-pattern req)
        data    (pullable-data db session)
        result  (pull/with-data-schema schema (mk-query pattern data))]
    {:response     ('&? result)
     :effects-desc (:context/effects result)
     :session      (merge session (:context/sessions result))}))

This was a clean separation. The saturn handler was fully testable with no mocks. Effects were pure data descriptions. The executor was the only impure component, and it was small. The original implementation is documented in the archived flybot.sg repository.

What pushed us to redesign

The saturn handler separation was elegant, but as the system grew, specific limitations emerged.

Response before effects. The saturn handler computed :response before the executor ran :effects. This worked when the response data was already known (e.g., returning the input entity on create). But when you needed something produced by the side effect itself (a DB-generated ID, a timestamp set by the storage layer, a merged entity after a partial update), you were stuck. The f-merge escape hatch existed: a closing function in the effects description that could amend the response after execution. But using f-merge essentially reintroduced in-place mutation, defeating the purpose of the pure/impure split.

Verb-oriented patterns. Every pattern was a set of function calls. Reading all items called a function. Reading one item called a different function with a :read action. Creating called the same function with a :save action. The case dispatch inside each :with function grew as operations multiplied. The pattern language was supposed to describe data, but it was describing procedure calls.

Authorization at two granularities. with-role gated access to the entire data map (coarse). But ownership enforcement (can this user edit this specific item?) had to live inside the :with function's case dispatch (fine). These were two different authorization mechanisms in two different places, with no intermediate layer for "can mutate, but only own entities."

Indirection through context-of. The modifier/finalizer mechanism in context-of was well-designed for what it did: accumulate effects and session updates during pattern resolution without side effects. But it was a layer you had to understand to trace a request end-to-end. Each operation returned {:response :effects :session :error}, the modifier unpacked those, and the finalizer attached the accumulations. The mechanics were sound, but the indirection meant debugging required following the data through several stages.

The saturn handler pattern achieved something valuable: a fully testable, purely functional request handler. The redesign was not about fixing a broken system. It was about recognizing that once collections replaced functions as the API's building blocks, the pure/impure split could happen at a different boundary (inside DataSource methods), and the accumulation machinery was no longer needed.

The new model: patterns match data

The rewrite inverted the relationship. Instead of patterns calling functions, patterns match against data structures. Collections implement ILookup (Clojure's get protocol) for reads and a Mutable protocol for writes. The pattern engine does not know about functions. It just walks a data structure.

Here are the same operations in the new model.

List all dashboards:

'{:user {:dashboards ?all}}

:user is a top-level key in the API map. If the session has the user role, it resolves to a map containing :dashboards. If not, it resolves to nil. ?all is a variable that binds to (seq dashboards), triggering list-all on the DataSource.

Read by ID:

'{:user {:dashboards {{:id $id} ?dash}}}
;; client sends: {:pattern ... :params {:id 123}}

{:id $id} is a lookup key. $id gets replaced with 123 before the pattern compiles. The collection's ILookup implementation receives {:id 123} and delegates to the DataSource's fetch method.

Create:

{:user {:dashboards {nil {:title "New" :content "..."}}}}

nil as a key means "create". The collection's Mutable implementation calls create! on the DataSource. The response is the full created entity.

No :with, no action keywords, no case dispatch. The pattern syntax itself encodes the operation: ?var means read, nil key means create, nil value means delete, key + value means update.

On the server, the data map is a structure of collections, not functions:

(defn make-api [{:keys [storage cache]}]
  (let [dashboards (coll/collection (->DashboardSource storage cache)
                                    {:id-key :id
                                     :indexes #{#{:id}}})]
    (fn [ring-request]
      (let [session (:session ring-request)]
        {:data   {:user  (when (:user session)
                           {:dashboards dashboards})
                  :owner (when (:owner session)
                           {:users users-collection
                            :roles roles-collection})}
         :schema {:user  {:dashboards [:vector Dashboard]}
                  :owner {:users [:vector User]}}
         :errors {:detect :error
                  :codes  {:forbidden 403 :not-found 404}}}))))

Side by side

The contrast is clearest when you see old and new patterns next to each other.

Simple read (list all)

;; OLD: two nested function calls
{:dashboards
 {(list :role/user :with [])
  {(list :self :with []) [{:title '? :id '?}]}}}

;; NEW: structural traversal
'{:user {:dashboards ?all}}

The old pattern needed two :with calls just to list everything: one for role checking, one for the listing function. The new pattern walks a data structure. If :user exists in the API map, :dashboards is a collection, and ?all binds to its contents.

Read by ID with parameters

;; OLD: function call with arguments
{:dashboards
 {(list :role/user :with [])
  {(list :dashboard :with [{:id 123} :read])
   {:title '? :content '?}}}}

;; NEW: indexed lookup with $params
'{:user {:dashboards {{:id $id} ?dash}}}
;; params: {:id 123}

:with [{:id 123} :read] called a function and passed it two arguments. {:id $id} is text substitution: $id becomes 123, then {:id 123} is used as a lookup key on the collection. The difference is that $params happens before pattern compilation. There is no function call in the pattern at all.

Create

;; OLD: function call with :save action
{:dashboards
 {(list :role/user :with [])
  {(list :dashboard :with [{:title "New" :content "..."} :save])
   {:id '? :title '?}}}}

;; NEW: nil key = create
{:user {:dashboards {nil {:title "New" :content "..."}}}}

The old model used the same function for reads and writes, distinguished by an action keyword (:read, :save, :delete). The new model uses structural conventions: nil as the key means create. The collection's Mutable protocol handles it.

Delete

;; OLD: function call with :delete action
{:dashboards
 {(list :role/user :with [])
  {(list :dashboard :with [{:id 123} :delete])
   {:id '?}}}}

;; NEW: query key + nil value = delete
{:user {:dashboards {{:id 123} nil}}}

nil as the value means delete. No action keywords, no function dispatch.

Analytics query (complex parameters)

;; OLD: arbitrary query object via :with
{:analytics
 {(list :raw :with [{:data-source [:module-1 :stats]
                      :select :col-name
                      :time-range {:from "2026-01-01" :to "2026-02-01"}}])
  '?}}

;; NEW: query object as lookup key via $params
'{:user {:analytics {$query ?result}}}
;; params: {:query {:data-source [:module-1 :stats]
;;                  :select :col-name
;;                  :time-range {:from "2026-01-01" :to "2026-02-01"}}}

The query object is the same in both cases. The difference is where it lives: inside a function call (old) versus as a lookup key (new). The DataSource's fetch method receives the full query map and routes internally.

What we gained

Authorization is structural, not functional

Old: (with-role session :user (fn [] ...)) wraps a thunk. Authorization is a function that gates access to other functions.

New: top-level keys in the API map are nil when the session lacks the role. The pattern simply gets nil for unauthorized paths. No function call, no wrapper.

;; Session has :user but not :owner
{:data {:user  {:dashboards dashboards}   ;; present
        :owner nil}}                       ;; nil: patterns against :owner return nothing

For finer-grained checks (ownership enforcement on mutations), wrap-mutable intercepts write operations:

(coll/wrap-mutable dashboards
  (fn [inner query value]
    (if (owns? session query)
      (coll/mutate! inner query value)
      {:error {:type :forbidden}})))

This is still structural: a decorator around a collection, not conditional logic inside a handler.

$params replaces :with

:with called a function with arguments at pattern-resolution time. $params does text substitution before the pattern is even compiled.

;; $params: symbol replacement before compilation
'{:users {{:id $uid} ?user}}
;; + params {:uid 123}
;; becomes: {:users {{:id 123} ?user}}

The pattern engine never sees $uid. By the time it runs, the pattern is pure data. This means patterns are always static structures from the engine's perspective, which simplifies the implementation and makes patterns easier to reason about.

No context-of, modifier, or finalizer

The old context-of mechanism was well-engineered: modifier functions extracted :response/:effects/:session from each operation, accumulated them in transient collections, and the finalizer attached them to the result. The saturn handler stayed pure throughout. It was a clean solution to the problem of accumulating side-effect descriptions during pattern resolution.

The new system does not need any of it:

• Side effects happen inside DataSource methods (not returned as data)
• Sessions are managed by Ring middleware (not returned from patterns)
• The pattern result is the final response (no post-processing)

The tradeoff: the saturn handler's strict pure/impure boundary is gone. DataSource methods perform side effects directly, which means the handler is no longer purely functional. In practice, this turned out to be acceptable because DataSource implementations are small, focused, and testable in isolation. The purity moved from the handler level to the collection wrapper level (decorators like wrap-mutable and read-only are pure transformations).

Side effects live inside DataSource

Old: functions returned {:response ... :effects {:rama {...}}}. The saturn handler accumulated these descriptions. A separate executor ran them afterward. The handler was purely functional.

New: create!, update!, and delete! in DataSource perform the side effects directly. The return value is the entity itself, not a description of work to be done.

(defrecord DashboardSource [storage cache]
  coll/DataSource
  (create! [_ data]
    (storage-append! storage [data :save])
    (assoc data :id (generate-id) :created-at (now)))
  (delete! [_ query]
    (storage-append! storage [query :delete])
    true))

This solves the "response before effects" problem directly: create! performs the write and returns the full entity with DB-generated fields. No f-merge, no two-phase response construction.

The tradeoff is that the handler is no longer purely functional. If you need the old effects-description pattern for testing, you can wrap the DataSource to capture effects without executing them. But the default path is direct execution, which is simpler to trace.

Error handling as data

Collections return errors as plain maps:

{:error {:type :forbidden :message "You don't own this resource"}}
{:error {:type :not-found}}

The remote layer maps error types to HTTP status codes via a declarative config:

{:detect :error
 :codes  {:forbidden 403 :not-found 404 :invalid-mutation 422}}

This keeps collections pure (they return data describing what happened) while the transport layer decides how to represent it. The design is heading toward GraphQL-style partial responses, where one branch failing does not fail the whole pattern. A request for {:user ?data :admin ?admin-stuff} should return :user data even if :admin is forbidden, with errors collected in a top-level array alongside the data.

Conclusion

The old saturn handler architecture was a genuinely clean design: a purely functional handler, effects as data descriptions, executors as the only impure component. It achieved testability and separation of concerns that many web frameworks do not even attempt.

The redesign was not about fixing something broken. It was about moving the purity boundary. The saturn handler kept the entire request pipeline pure by deferring effects. The new model keeps collections and their wrappers pure by pushing side effects into DataSource methods. The accumulation machinery (context-of, modifier, finalizer) disappears because there is nothing to accumulate. The response-before-effects limitation disappears because create! returns the entity directly.

The deeper lesson is about API identity. When your API is a set of handler functions, cross-cutting concerns (authorization, transport, error handling) become imperative code woven through those handlers. When your API is a data structure, those same concerns become structural: the shape of the map enforces authorization, the protocols enforce CRUD semantics, and the transport layer works generically over any ILookup-compatible structure.

Verbs become nouns, and the nouns compose.

Permalink

Clojure Protocols and the Decorator Pattern

Context

Clojure's built-in functions work on built-in types because those types implement specific Java interfaces. get works on maps because maps implement ILookup. seq works on vectors because vectors implement Seqable. count works on both because they implement Counted.

The interesting part: your custom types can implement the same interfaces. Once they do, Clojure's standard library treats them as first-class citizens. get, seq, map, filter, count all work transparently, no special dispatch, no wrapper functions.

The lasagna-pattern collection library (Clojars) does exactly this. It defines a Collection type backed by a database, then implements ILookup and Seqable so that (get coll {:post/id 3}) triggers a database query while looking like a plain map lookup to the caller. The companion article, Building a Pure Data API with Lasagna Pattern, covers the full architecture. This article focuses on the Clojure constructs that make it work.

The four constructs

Clojure provides four ways to define types that implement protocols and interfaces. Each serves a different purpose.

defprotocol: the contract

Defines method signatures with no implementation. Conceptually similar to a Java interface.

(defprotocol DataSource
  (fetch [this query])
  (list-all [this])
  (create! [this data])
  (update! [this query data])
  (delete! [this query]))

This says: "any storage backend must support these 5 operations." It does not say how. The implementation is left to the types that satisfy the protocol.

defrecord: named, map-like type

A concrete implementation of a protocol. Has named fields and behaves like a Clojure map (you can assoc, dissoc, and destructure it).

(defrecord PostsDataSource [conn]
  DataSource
  (fetch [_ query]    (d/q ... @conn))
  (list-all [_]       (d/q ... @conn))
  (create! [_ data]   (d/transact conn [data]))
  (update! [_ q data] (d/transact conn [(merge ...)]))
  (delete! [_ query]  (d/transact conn [[:db/retractEntity ...]])))

Use defrecord for persistent, reusable implementations with named fields: storage backends, services, configuration holders.

deftype: named, not map-like

Like defrecord but without map behavior. Used for structural wrappers that implement platform interfaces rather than domain protocols.

(deftype Collection [data-source id-key indexes]
  clojure.lang.ILookup
  (valAt [this q] (.valAt this q nil))
  (valAt [_ q nf] (or (fetch data-source q) nf))

  clojure.lang.Seqable
  (seq [_] (seq (list-all data-source))))

Use deftype when you need to override built-in Clojure verbs (get, seq, count). The type itself is opaque. Callers interact with it through standard Clojure functions, not through field access.

reify: anonymous, inline type

Same capability as deftype but anonymous and created inline. Closes over local variables.

(defn profile-lookup [session]
  (reify clojure.lang.ILookup
    (valAt [this k] (.valAt this k nil))
    (valAt [_ k nf]
      (case k
        :name  (:user-name session)
        :email (:user-email session)
        nf))))

Use for one-off objects, per-request wrappers, or cases where a named type would be overkill. The session value is captured from the enclosing scope.

Summary table

Overriding built-in verbs

Each Clojure interface corresponds to a built-in verb. Implementing an interface teaches Clojure how your custom type responds to that verb.

ILookup: powers get

When you call (get thing key), Clojure calls (.valAt thing key nil) under the hood. Maps implement this by default. Custom types do not.

;; Without ILookup
(deftype Box [x])
(get (->Box 42) :x)  ;; => nil (Box doesn't implement ILookup)

;; With ILookup
(deftype SmartBox [x y]
  clojure.lang.ILookup
  (valAt [this k] (.valAt this k nil))
  (valAt [_ k nf]
    (case k :x x :y y nf)))

(get (->SmartBox 1 2) :x)  ;; => 1

In the collection library, ILookup is what makes (get coll {:post/id 3}) trigger a database query. The caller writes standard Clojure. The collection translates the get call into a fetch on the underlying DataSource.

Seqable: powers seq (and map, filter, etc.)

clojure.lang.Seqable
(seq [_] (seq (list-all data-source)))

Once a type implements Seqable, all sequence functions work: (seq coll), (map f coll), (filter pred coll). The collection becomes iterable by delegating to its DataSource's list-all.

Counted: powers count directly

clojure.lang.Counted
(count [_] (count (list-all data-source)))

Without Counted, calling count on a custom Seqable type throws UnsupportedOperationException. Clojure's RT.count() does not fall back to seq. It only works on types that implement Counted, IPersistentCollection, java.util.Collection, or a few other JDK interfaces. If your custom type needs to support count, implement Counted explicitly. This also lets you provide an optimized path (e.g., a SELECT COUNT(*) instead of fetching all rows).

Custom protocols

The interfaces above override Clojure's built-in verbs. But some operations have no built-in verb. The collection library defines two custom protocols for these cases.

mutate! unifies create, update, and delete into a single function. The operation is determined by the combination of arguments: nil query means create, nil value means delete, both present means update.

Wireable is conceptually similar to clojure.core.protocols/Datafiable (datafy). Both turn opaque types into plain Clojure data. The difference is intent: datafy is for introspection and navigation, ->wire is specifically for HTTP serialization.

The decorator pattern

Here is the key design insight: one DataSource, one Collection, multiple wrappers per role.

Without wrappers (bad)

;; 3 records duplicating the same Datahike queries
(defrecord GuestPostsDataSource [conn] ...)
(defrecord MemberPostsDataSource [conn] ...)
(defrecord AdminPostsDataSource [conn] ...)

Each record contains a full copy of the same fetch, list-all, create!, update!, and delete! logic. Domain logic changes must be applied to all three.

With wrappers (good)

(def posts (db/posts conn))               ;; one DataSource, one Collection

(public-posts posts)                       ;; reify: override get/seq to strip email
(member-posts posts user-id email)         ;; wrap-mutable: override mutate! for ownership
posts                                      ;; admin: no wrapper needed

The DataSource is created once. Each role gets a thin wrapper that overrides only the behavior it needs. Reads, storage queries, and domain logic live in one place.

Wrapper functions

Example: building views per role

(let [posts (db/posts conn)]    ;; one record, created once

  {:guest  {:posts (public-posts posts)}            ;; reify over read-only, strips :user/email
   :member {:posts (member-posts posts uid email)}  ;; wrap-mutable, ownership checks
   :admin  {:posts posts}                           ;; raw collection, full access
   :owner  {:users (coll/read-only (db/users conn))}})

Guests see a read-only view with PII stripped. Members see a mutable view that enforces ownership. Admins see the raw collection. Each wrapper does one thing.

The public-posts wrapper demonstrates how reify serves as the escape hatch when the built-in wrappers are not enough:

(defn- public-posts [posts]
  (let [inner (coll/read-only posts)]
    (reify
      clojure.lang.ILookup
      (valAt [_ query]
        (when-let [post (.valAt inner query)]
          (strip-author-email post)))

      clojure.lang.Seqable
      (seq [_]
        (map strip-author-email (seq inner))))))

The library provides read-only (restricts writes) and wrap-mutable (intercepts writes), but no built-in way to transform read results. For that, you implement ILookup and Seqable directly via reify.

Three layers of authorization

Authorization in this pattern is distributed structurally rather than imperatively. Instead of a single middleware that checks permissions, three layers each handle a different granularity.

Coarse: with-role (API map structure)

Binary gate: you have the role or you don't. The entire subtree of collections is present or replaced with an error map.

(defn- with-role [session role data]
  (if (contains? (:roles session) role)
    data
    {:error {:type :forbidden :message (str "Role " role " required")}}))

;; In make-api:
:owner (with-role session :owner
         {:users users, :users/roles roles})

A non-owner sending '{:owner {:users ?all}} hits the error map, not the collection. The remote/ layer detects errors along variable paths, so the error flows through as inline data and prevents any mutation from being attempted.

A planned improvement is an error-gate function that replaces the plain map with a reify implementing ILookup (returns self for any key, so deeply nested pattern traversal keeps working), Mutable (returns the error for mutations), and Wireable (serializes as the error map). This would be a good example of composing three protocols into a single anonymous sentinel object.

Medium: wrap-mutable (per-entity mutation rules)

Controls who can create, update, or delete specific entities:

(coll/wrap-mutable posts
  (fn [posts query value]
    (if (owns-post? posts user-email query)
      (coll/mutate! posts query value)
      {:error {:type :forbidden}})))

Reads pass through untouched. Only mutations are intercepted. The check is per-entity: does this user own this specific post?

Fine: reify decorator (field-level read transformation)

Controls which fields are visible:

(public-posts posts)  ;; strips :user/email from author on every read

Every get and seq call on this wrapper runs through a transformation function that removes sensitive fields before the data reaches the caller.

Authorization summary

The DataSource stays "dumb" about authorization. It only knows about storage. This keeps it reusable across all roles without conditional logic.

When to skip the full stack

Not everything needs defrecord + DataSource + Collection. If a resource is read-only, non-enumerable, and has a single query shape, a raw reify implementing ILookup + Wireable is enough.

Example: a post history lookup that takes a post ID and returns the revision history:

(defn post-history-lookup [conn]
  (reify
    clojure.lang.ILookup
    (valAt [_ query]
      (when-let [post-id (:post/id query)]
        (post-history @conn post-id)))
    (valAt [this query not-found]
      (or (.valAt this query) not-found))

    coll/Wireable
    (->wire [_] nil)))  ;; can't enumerate all history

The pattern engine still calls get on it, so it works identically from the caller's perspective. The full DataSource/Collection stack would add index validation, Seqable, Mutable, none of which history needs.

Decision guide

Note: coll/lookup only supports keyword keys (:email, :name). For map keys like {:post/id 3}, use a raw reify.

Permalink

Annually-Funded Developers' Update: January & February 2026

Hello Fellow Clojurists!

This is the first of six reports from the developers who are receiving annual funding for 2026. We’ve also added in the final FastMath report from Thomas Clark (Q3 2025 project). There is a lot of great work here - so have fun exploring!

Bozhidar Batsov: nREPL, Clojure Mode, info-Clojure, CIDER, drawbridge
Clojure Camp: Badges, Parson’s Problems, Mobs
Eric Dallo: eca, clojure-lsp, metrepl
Jeaye Wilkerson: Jank
Michiel Borkent: SCI, babashka, Cream, clj-kondo, squint, and more

Thomas Clark:FastMath

Bozhidar Batsov

2026 Annual Funding Report 1. Published March 3, 2026.

The period was extremely productive and solid progress was made on almost every front. CIDER nad nREPL saw important releases, but so did also:

• clojure-mode
• inf-clojure
• drawbridge (first new release in years)

I’ve also did some work on updating REPLy to use jline 3 and tools.reader (instead of the abandoned sjacket)
Below are some highlights from the releases.

nREPL

Release: 1.6.0 (Feb 26)

New Features

• ^:optional middleware metadata — middleware can now be marked optional, so missing middleware is handled gracefully instead of blowing up (#415).
• ^:concat config merging — already shipped in an earlier cycle, but part of this release (#402).

Bug Fixes

• EDN transport with Unix domain sockets — long-standing issue #351 finally resolved (#421).
• load-file nil message for empty files — loading empty or comment-only files no longer sends a nil message (#422).
• err-exit throwing a string instead of a proper Exception.
• Middleware deduplication — duplicate middleware entries are now cleaned up when constructing a handler (#410).

Internal Improvements (January)

• Major refactoring wave: handler construction & middleware application (#409), stdin middleware (#408), nrepl.core functions, and conditional loading of nrepl.ack.
• Deprecated automatic inclusion of middleware via :requires/:expects — still works but will error in the future (#412).
• Replaced Thread.sleep with Thread.join in cmdline, removed unnecessary sleep in threading.
• Atomic update for running-repl atom, replaced deprecated #^{} metadata syntax.

Documentation Overhaul (February)
A massive documentation push covering:

• New troubleshooting guide, expanded FAQ with practical Q&As.
• Session lifecycle docs for client developers.
• Configuration keys reference table for server docs.
• Documented forward-system-output op, respond-to helper, .nrepl-port file/server discovery, dynamic var defaults, and ^:optional/^:concat metadata.
• Cleaned up stale labels (“experimental”, “work in progress”), removed dead links and abandoned projects from listings.
• Documented the release process itself.

Housekeeping

• Bumped dependencies and copyright years.
• Removed Fastlane from code and docs.
• Fixed typos/grammar across source and documentation.
• Renamed lein maint to lein docs.

clojure-mode 5.21.0 (Feb 18) + some ongoing work for the next release

I did the biggest updates of clojure-mode in years in the past month. This might sound odd, given that clojure-ts-mode has been in a great shape for a while now, but I felt that in the spirit of Clojure’s stability we shouldn’t force people to change their workflows if they are happy with them. And I understand that no everyone likes the complexity of using TreeSitter (or can run a new Emacs). So, I’ve decided to tackle the issues that seemed most important and I think the final result was pretty good. Some of the changes below are unreleased, but will be released soon, as everything’s merged already and I haven’t received any reports of regressions.

New Features

• edn-mode — a new lightweight major mode derived from clojure-mode with data-appropriate indentation for .edn files (#650).
• Shebang recognition — interpreter-mode-alist entries for clojure, clj, planck, joker, and jank so scripts auto-activate the right mode (#439).
• clojure-discard-face for #_ reader discard forms, separately stylable from comments (#688).
• clojure-preferred-build-tool — controls project root detection when multiple build files exist, with .git as tiebreaker (#687).
• ClojureCLR project root detection (deps-clr.edn).

Bug Fixes (many long-standing)

• clojure-sort-ns mangling :gen-class (#527).
• clojure-thread-last-all breaking forms with line comments (#619).
• clojure-add-arity severing arglist metadata (#649).
• Nested indent specs rejected by validation (#600).
• clojure-find-def failing on special characters and comments before the symbol name (#637, #686).
• clojure-mode-version returning nil (#658).
• Font-lock protocol method docstrings (#402) and letfn function names (#365).

Housekeeping

• Bumped minimum Emacs to 27.1, added Emacs 30 to CI.
• Major performance pass: cached regexps, iterative loops instead of recursion, eliminated duplicate syntax-ppss/thing-at-point calls.
• Updated clojure-mode-extra-font-locking for Clojure 1.10-1.12 (new functions, removed non-existent entries, fixed misplacements).
• Rebound clojure-align to C-c C-a (old C-c SPC deprecated).
• Comprehensive README and CHANGELOG cleanup.

inf-clojure 3.4.0 (Feb 27)

inf-clojure rarely gets much love from me, but the project has been in a good shape for a while now anyways. Still, I felt that an annual cleanup and bug-fixing session was in order and I hope you’ll appreciate it. I’ve also tried to restructure the docs to be easier to follow and I finally added some comparison with CIDER.

New Features

• inf-clojure-show-var-meta (C-c C-S-m) — display a var’s full metadata (#57).
• Namespace-aware eval via inf-clojure-eval-ns-aware (#205).
• clojure-clr REPL type for clr.tools.deps.
• Arglists now shown in the REPL buffer instead of the minibuffer (#143).

Bug Fixes

• inf-clojure-connected-p now checks for a live process.
• Preoutput filter handles chunked comint output correctly, fixing spurious newlines (#136/#153).
• project-root error when no project is detected (#219).
• Truncated autoload cookie on inf-clojure-connect.
• prefix-arg vs current-prefix-arg in REPL startup.
• Emacs 28-29 compatibility fix for derived-mode-p.
• Confirmation message after inf-clojure-set-ns (#149).

Housekeeping

• Deduplicated REPL feature definitions across Clojure-family REPLs (shared base alist).
• Unified reload/reload-all forms into inf-clojure-repl-features.
• Switched to modern eldoc-documentation-functions hook.
• Dropped lein-clr support (abandonware).
• Expanded test coverage, removed dead code, fixed docstrings and lint warnings.
• Comprehensive README improvements: inf-clojure vs CIDER comparison (#74), nREPL TTY transport docs (#155), load/reload/set-ns docs (#192), completion docs (#111).

CIDER 1.21.0 “Gracia” (Feb 7)

Last, but never least… :D I did’t do that much with CIDER for this release, as I wanted to focus on nREPL, clojure-mode and inf-clojure, but I think it’s still turned out pretty well.

• Moved elisp source under lisp/ to improve repo structure (#3477).
• Dropped Emacs 27 support (prompted by upstream deps).
• cider-ns-browser items are now clickable buttons (evil-compatible RET).
• Removed deprecated cider-*-global-opts and remaining Boot references.
• Fixed mangled printed representations (caused by an nREPL bug fixed in 1.5.2).

After the release I’ve introduced the concept of default session — cider-set-default-session / cider-clear-default-session to bypass sesman’s project-based dispatch (#3865). This was in the back of my mind for years

I’ve also spend some time cleaning up internals, improving the CI and the docs. A lot more CIDER improvements are currently brewing. :-)
As you can imagine - I have many ideas on what to tackle next, so I hope the next couple of months will be just as exciting and productive.

Thanks to everyone for your support! You rock!

Clojure Camp

2026 Annual Funding Report 1. Published March 7, 2026.

• Some of our CT funds are being set aside to support three efforts this year:

  • sponsoring conference attendance for new Clojurians,
  • supporting Clojure and non-Clojure meetups with a “pizza fund”,
  • hosting an experimental nano-conj (in-person multi-day open-ended hack-on-clojure “retreat”)

• On the “dev side”..:

  • Most of the time this period has been spent on a badge system (unreleased) to reorganize our learning roadmap ( https://map.clojure.camp/ )
    • The functionality is mostly complete, and some UI polish and content organizing remains
    • The goal is to have three levels of “badges” for topics, with clear expectations on what to do prove knowledge for a badge (and help direct learners)
    • Learners will be able to self-grant, but community members will also be able to “verified grant”
  • Mobs continue to happen regularly; our community site now lets people indicate their ability which we’ve started to use to schedule mobs more dynamically
  • Released Parson’s Problems (alpha) on our Exercise Site
    • Parson’s Problems are like multiple-choice mad-libs
    • These should be more accessible to new devs compared to word problems
    • You can check it out at https://exercises.clojure.camp/exercises/count-if (switch the “Word Problem” select input to “Fill in the Blanks”)
  • Some early experiments with a “one-file Clojure starter environment installer”

• By next checkpoint, hopefully:

  • Continue mobs, get back to weekly mob cadence, promote more regularly in more places
  • Publish backlog of exercises (based on past year of mobs)
  • Release the Badges
  • Engage a community member to help with logistical efforts (pizza fund, nano-conj, mentor program)
    

Eric Dallo

2026 Annual Funding Report 1. Published March 8, 2026.

Starting 2026 with so much energy! In these first 2 months I’ve been working hard on multiple projects, with most of the focus on ECA which is growing really fast with more and more people using it and contributing, we reached 0.109.0 with lots of features and improvements! Besides that, I worked on clojure-lsp, clojure-mcp, brepl and more, really happy with the progress and thankful for ClojuristsTogether sponsorship! :heart:

eca

ECA keeps growing with lots of new features, bug fixes, and improvements in stability and compatibility. In these 2 months we had lots of releases with some really exciting features, the changelog is huge but here are the highlights:

• Subagents: ECA now supports foreground subagents, allowing the main agent to spawn focused sub-agents for specific tasks like code exploration, review, and more. This is a huge improvement for complex workflows.
• Skills support: Following the agentskills.io spec, ECA now supports skills that can be loaded dynamically by the LLM, with a new /skill-create command to create skills from prompts.
• Auto compact: Automatic context compaction based on a configurable percentage of context window usage, avoiding context overflow and keeping conversations efficient.
• Context overflow recovery: When context limit is hit, ECA now recovers gracefully instead of failing.
• Variants: A new feature to configure model variants (low, medium, high, max) for different providers, allowing fine-grained control over model behavior.
• MCP OAuth support: Support for HTTP MCP servers that require OAuth authentication.
• Agents (formerly behaviors): Renamed behaviors to agents with improved configuration and prompt customization.
• New config API for prompts: Complete overhaul of prompt configuration, allowing override of any tool description, system prompts, and more.
• Network configuration: Support for custom CA certificates and mTLS client certificates.
• More providers and models: Added Codex login, moonshot models, kimi-k2.5, claude-sonnet-4-6, claude-opus-4-6, gpt-5.3-codex and many more.

That’s really a lot of things done, showing how users are excited with the project and asking for new features, ECA is in a really good shape after 6 months, closer to Claude code, Cursor and other tools, but free and more extensible!

Also, we have a new webpage for eca.dev!

ECA editor plugins

All ECA editor plugins received significant updates to keep up with the new ECA server features:

• eca-emacs: Support for subagents UI, variants, git worktree, improved tables UI, TRAMP fixes, light theme improvements, queue messages support, and many bug fixes.
• eca-vscode (0.34.1 - 0.40.0): Support for subagents, variants, diagnostics, prompt queuing, cycle history, export chat, image pasting, chat rename and timeline.
• eca-intellij (0.17.1 - 0.21.0): Support for subagents, variants, diagnostics, export chat to markdown, querying files via # in prompt.
• eca-nvim: Tool call improvements, implicit manual approval flow, better auto scroll behavior, stream queue improvements.

clojure-lsp

We had a big release with lots of dependency bumps that were long overdue, new code actions, and important bug fixes. The Extract Function code action got much more powerful with selection and threading support! I have plans for next months to have custom code actions, memory management, performance and classpath scan improvements!

2026.02.20-16.08.58

• New code actions: refactor cond to if, and refactor if to cond.
• Allow finding references of multiple definitions at cursor. #2176
• Fix CLI format :style/indent support via project type flag.
• Fix format :style/indent conflicts with core macros. #2197
• Add selection to Extract Function code action. #2118
• Add threading to Extract Function code action. #2175
• Fix replace-refer-all mangling unrelated symbols. #2150
• Fix unused-public-var :exclude-when-contains-meta not being considered for .lsp/config.edn.
• Fix unused-public-var not working with :config-in-ns in kondo config.
• Wrap hovered function definitions in parentheses.
• Include reader functions in special cases for finding the last element in a container.
• Lots of bumps/fixes: clj-kondo 2026.01.19, rewrite-clj 1.2.51, Clojure 1.12.4, cljfmt 0.16.0, lsp4clj 2.0.1, babashka/sci 0.12.51, opentelemetry 1.59.0 and more.

metrepl

0.5.1 - 0.5.2

• Fix middlewares metric to check response instead of payload.
• Add java.classpath missing dependency.
  

Jeaye Wilkerson

2026 Annual Funding Report 1. Published March 6, 2026.

Hey folks! We’re two months into the year and I’d like to cover all of the progress that’s been made on jank so far. Before I do that, I want to say thank you to all of my Github sponsors, as well as Clojurists Together for sponsoring this whole year of jank’s development!

jank book

To kick things off, let me introduce the jank book. This will be the recommended and official place for people to learn jank and its related tooling. It’s currently targeted at existing Clojure devs, but that will start to shift as jank matures and I begin to target existing native devs as well. The jank book is written by me, not an LLM. If you spot any issues, or have any feedback, please do create a Github Discussion.

My goals for this book include:

1. Introduce jank’s syntax and semantics
2. Introduce jank’s tooling
3. Walk through some small projects, start to finish
4. Demonstrate common use cases, such as importing native libs, shipping AOT artifacts, etc.
5. Show how to troubleshoot jank and its programs, as well as where to get help
6. Provide a reference for error messages

As the name and technology choice implies, the jank book is heavily inspired by the Rust book.

Alpha status

jank’s switch to alpha in January was quiet. There were a few announcements made by others, who saw the commits come through, or who found the jank book before I started sharing it. However, I didn’t make a big announcement myself since I wanted to check off a few more boxes before getting the spotlight again. In particular, I spent about six weeks, at the end of 2025 and into January, fixing pre-mature garbage collection issues. These weeks will be seared into my memory for all of my days, but the great news is that all of the issues have now been fixed. jank is more and more stable every day, as each new issue improves our test suite.

LLVM 22

On the tail of the garbage collection marathon, the eagerly awaited LLVM 22 release happened. We had been waiting for LLVM 22 to ship for several months, since it would be the first LLVM version which would have all of jank’s required changes upstreamed. The goal was that this would allow us to stop vendoring our own Clang/LLVM with jank and instead rely on getting it from package managers. This would make jank easier to distribute and, crucially, make jank-compiled AOT programs easier to distribute. You can likely tell from my wording that this isn’t how things went. LLVM 22 arrived with a couple of issues.

Firstly, some data which we use for very important things like loading object files, adding LLVM IR modules to the JIT runtime, interning symbols, etc was changed to be private. This can happen because the C++ API for Clang/LLVM is not considered a public API and thus is not given any stability guarantees. I have been in discussions with both Clang and LLVM devs to address these issues. They are aware of jank and want to support our use cases, but we will need to codify some of our expectations in upstreamed Clang/LLVM tests so that they are less likely to be broken in the future.

Secondly, upon upgrading to LLVM 22, I found two different performance regressions which basically rendered debug builds of jank unusable on Linux (here and here). Our startup time for jank debug builds went from 1 second to 1 minute and 16 seconds. The way jank works is quite unique. This is what allows us to achieve unprecedented C++ interop, but it also stresses Clang/LLVM in ways which are not always well supported. I have been working with the relevant devs to get these issues fixed, but the sad truth is that the fixes won’t make it into LLVM 22. That means we’ll need to wait several more months for LLVM 23 before we can rely on distro packages which don’t have this issue.

That’s a tough pill to swallow, so I took a week or so to rework the way we do codegen and JIT compilation. I’ve not only optimized our approach, but I’ve also specifically crafted our codegen to avoid these slower parts of LLVM. This not only brings us back to previous speeds, it makes jank faster than it was before. Once LLVM 23 lands, the fixes for those issues will optimize things further.

So, if you’ve been wondering why I’ve been quiet these past few months, I likely had my head buried deep into one of these problems. However, with these issues out of the way, let’s cover all of the other cool stuff that’s been implemented.

nREPL server

jank has an nREPL server now! You can read about it in the relevant jank book chapter. One of the coolest parts of the nREPL server is that it’s written in jank and yet also baked into jank, thanks to our two-phase build process. The nREPL server has been tested with both NeoVim/Conjure and Emacs/CIDER. There’s a lot we can do to improve it, going forward, but it works.

As Clojure devs know, REPL-based development is revolutionary. To see jank’s seamless C++ interop combined with the tight iteration loop of nREPL is beautiful. Here’s a quote from an early jank nREPL adopter, Matthew Perry:

The new nREPL is crazy fun to play around with. Works seamlessly with my editor (NeoVim + Conjure). It’s hard to describe the experience of compiling C++ code interactively - I’m so used to long edit-compile-run loops and debuggers that it feels disorienting (in a good way!)

A huge shout out to Kyle Cesare, who originally wrote jank’s nREPL server back in August 2025. Thank you for your pioneering! If you’re interested in helping out in this space, there’s still so much to explore, so jump on in.

C++ interop improvements

Most of my other work on jank has been related to improving C++ interop.

Referred globals

jank now allows for C/C++ includes to be a part of the ns macro. It also follows ClojureScript’s design for :refer-global, to bring native symbols into the current namespace. Without this, the symbols can still be accessed via the special cpp/ namespace.

(ns foo
  (:include "gl/gl.h") ; Multiple strings are supported here.
  (:refer-global :only [glClear GL_COLOR_BUFFER_BIT])) ; Also supports :rename.

(defn clear! []
  (glClear GL_COLOR_BUFFER_BIT))

Native loop bindings

jank now supports native loop bindings. This allows for loop bindings to be unboxed, arbitrary native values. jank will ensure that the native value is copyable and supports operator=. This is great for looping with C++ iterators, for example.

(loop [i #cpp 0]
  (if (cpp/== #cpp 3 i)
    (cpp/++ i)
    (recur (cpp/++ i))))]

There’s more work to be done to automatically use unboxed values and use native operators, when possible. For now it’s opt-in only.

Unsafe casting

jank had the equivalent of C++’s static_cast, in the form of cpp/cast. However, for some C/C++ APIs, unsafe casting is necessary. To accomplish this, jank now has cpp/unsafe-cast, which does the equivalent of a C-style cast.

(let [vga-memory (cpp/unsafe-cast uint16_t* #cpp 0xB8000)]
  )

Type/value DSL

This one is working, but not yet in main. jank now supports encoding C++ types via a custom DSL. With this DSL, we can support any C++ type, regardless of how complex. That includes templates, non-type template parameters, references, pointers, const, volatile, signed, unsigned, long, short, pointers to members, pointers to functions, and so on. The jank book will have a dedicated chapter on this once merged, but here’s a quick glimpse.

std::map<std::string, int*>

(std.map std.string (ptr int))

std::array<char, 64>::value_type

(:member (std.array char 64) value_type)

unsigned char[1024]

(:array (:unsigned char) 1024)

unsigned char(&)[]

(:& (:array (:unsigned char)))

int (*)(std::string const &)

(:* (:fn int [(:& (:const std.string))]))

int (Foo::*)(std::string const &)

(:member* Foo (:fn int [(:& (:const std.string))]))

void (*Foo::*)()

(:member* Foo (:* (:fn void [])))

This type DSL will be enabled automatically in type position for cpp/new, cpp/cast, cpp/unsafe-cast, cpp/unbox, and so on. It can also be explicitly introduced via cpp/type, in case you want to use it in value position to construct a type or access a nested value. For example, to dynamically allocate a std::map<int, float>, you could do:

(let [heap-allocated (cpp/new (std.map int float))
      stack-allocated ((cpp/type (std.map int float)))]
  )

Other improvements

jank will now defer JIT compilation of functions, when possible. In some scenarios, such as during AOT compilation, this can cut compile times in half. We do this by generating a stub object which will JIT compile the relevant code when it’s first called. It understands vars, too, so it will replace itself in its containing var when called so that subsequent calls through the var just go to the JIT compiled function. JVM folks happily don’t need to worry about these sorts of things, but we can have nice things, too.

Also, jank’s object model has been opened up. I previously documented my research into an efficient object model. Over the past couple of years of hammock time, I have found an approach which allows for JIT-defined objects while still avoiding the costs of C++’s runtime type information (RTTI). This is worthy of its own post entirely, which I will likely do once the transition is complete. For now, we have most of our code still using the old model while some of it is using the new model. This is great, though, since it allows us to port piece by piece while keeping everything in main. The main outcome of opening up the object model is that jank users can define their own jank objects which integrate well into the system, can be stored within jank data structures, and used with jank functions.

Finally, to better support nREPL, jank added support for clojure.core/future. This required an audit of all synchronization across the jank compiler and runtime. Now, we should be in a good place from which to build multi-threaded jank applications. Tools like Clang’s thread sanitizer will help ensure we stay there.

What’s next

In March, I am wrapping up work on the type DSL and getting that merged. I also need to investigate why the Arch binary package for jank is broken. Beyond that, I will be starting into some deep performance research for jank. That will mean first collecting a series of benchmarks for jank versus Clojure and then profiling and optimizing those benchmarks as needed. I would really like to get some continuous benchmarking set up, so we can track performance over time, tied to particular commits. The current plan is to spend all of Q2 on performance, but there’s a lot to do, so I won’t be able to tackle everything. Benchmark optimization posts are often quite fun, so stay tuned for the next one!

Michiel Borkent

2026 Annual Funding Report 1. Published March 6, 2026.

In this post I’ll give updates about open source I worked on during January and February 2026. To see previous OSS updates, go here.

Sponsors

I’d like to thank all the sponsors and contributors that make this work possible. Without you, the below projects would not be as mature or wouldn’t exist or be maintained at all! So a sincere thank you to everyone who contributes to the sustainability of these projects.

Current top tier sponsors:

• Clojurists Together
• Roam Research
• Nextjournal
• Nubank

Open the details section for more info about sponsoring.

Babashka conf and Dutch Clojure Days 2026

Babashka Conf 2026 is happening on May 8th in the OBA Oosterdok library in Amsterdam! David Nolen, primary maintainer of ClojureScript, will be our keynote speaker! We’re excited to have Nubank, Exoscale, Bob and Itonomi as sponsors. Wendy Randolph will be our event host / MC / speaker liaison :-). The CfP is now closed. More information here. Get your ticket via Meetup.com (there is a waiting list, but more places may become available). The day after babashka conf, Dutch Clojure Days 2026 will be happening, so you can enjoy a whole weekend of Clojure in Amsterdam. Hope to see many of you there!

Projects

I spent a lot of time making SCI’s deftype, case, and macroexpand-1 match JVM Clojure more closely. As a result, libraries like riddley, cloverage, specter, editscript, and compliment now work in babashka.

After seeing charm.clj, a terminal UI library, I decided to incorporate JLine3 into babashka so people can build terminal UIs. Since I had JLine anyway, I also gave babashka’s console REPL a major upgrade with multi-line editing, tab completion, ghost text, and persistent history. A next goal is to run rebel-readline + nREPL from source in babashka, but that’s still work in progress (e.g. the compliment PR is still pending).

I’ve been working on async/await support for ClojureScript (CLJS-3470), inspired by how squint handles it. I also implemented it in SCI (scittle, nbb etc. use SCI as a library), though the approach there is different since SCI is an interpreter.

Last but not least, I started cream, an experimental native binary that runs full JVM Clojure with fast startup using GraalVM’s Crema. Unlike babashka, it supports runtime bytecode generation (definterface, deftype, gen-class). It currently depends on a fork of Clojure and GraalVM EA, so it’s not production-ready yet.

Here are updates about the projects/libraries I’ve worked on in the last two months in detail.

• NEW: cream: Clojure + GraalVM Crema native binary

  • A native binary that runs full JVM Clojure with fast startup, using GraalVM’s Crema (RuntimeClassLoading) to enable runtime eval, require, and library loading
  • Unlike babashka, supports definterface, deftype, gen-class, and other constructs that generate JVM bytecode at runtime
  • Can run .java source files directly, as a fast alternative to JBang
  • Cross-platform: Linux, macOS, Windows

• babashka: native, fast starting Clojure interpreter for scripting.

  • Released 1.12.214 and 1.12.215
  • #1909: add JLine3 for TUI support
  • Console REPL (bb repl) improvements: multi-line editing, tab completion, ghost text, eldoc, doc-at-point (C-x C-d), persistent history
  • Support deftype with map interfaces (e.g. IPersistentMap, ILookup, Associative). Libraries like core.cache and linked now work in babashka.
  • Compatibility with riddley, cloverage, editscript, charm.clj
  • #1299: add new babashka.terminal namespace that exposes tty?
  • Add keyword completions to nREPL and console REPL
  • deftype supports Object + hashCode
  • #1923: support reify with java.time.temporal.TemporalQuery
  • Fix reify with methods returning int/short/byte/float
  • Full changelog

• SCI: Configurable Clojure/Script interpreter suitable for scripting

  • Released 0.12.51
  • deftype now macroexpands to deftype*, matching JVM Clojure, enabling code walkers like riddley
  • case now macroexpands to JVM-compatible case* format, enabling tools like riddley and cloverage
  • Support async/await in ClojureScript. See docs.
  • Support functional interface adaptation for instance targets
  • Infer type tags from let binding values to binding names
  • defrecord now expands to deftype* (like Clojure), with factory fns emitted directly in the macro expansion
  • macroexpand-1 now accepts an optional env map as first argument
  • Add proxy-super, proxy-call-with-super, update-proxy and proxy-mappings
  • Support #564: this-as in ClojureScript
  • Store current analysis context during macro invocation, enabling tools like riddley to access outer locals
  • Full changelog

• clj-kondo: static analyzer and linter for Clojure code that sparks joy.
  @jramosg, @tomdl89 and @hugod have been on fire with contributions this period. Six new linters!

  • Released 2026.01.12 and 2026.01.19
  • #2735: NEW linter: :duplicate-refer which warns on duplicate entries in :refer of :require (@jramosg)
  • #2734: NEW linter: :aliased-referred-var, which warns when a var is both referred and accessed via an alias in the same namespace (@jramosg)
  • #2745: NEW linter: :is-message-not-string which warns when clojure.test/is receives a non-string message argument (@jramosg)
  • #2712: NEW linter: :redundant-format to warn when format strings contain no format specifiers (@jramosg)
  • #2709: NEW linter: :redundant-primitive-coercion to warn when primitive coercion functions are applied to expressions already of that type (@hugod)
  • Add new types array, class, inst and type checking support for related functions (@jramosg)
  • Add type checking support for clojure.test functions and macros (@jramosg)
  • #2340: Extend :condition-always-true linter to check first argument of clojure.test/is (@jramosg)
  • #2729: Check for arity mismatch for bound vectors, sets & maps, not just literals (@tomdl89)
  • #2768: NEW linter: :redundant-declare which warns when declare is used after a var is already defined (@jramosg)
  • Add type support for pmap and future-related functions (@jramosg)
  • Upgrade to GraalVM 25
  • Full changelog

• squint: CLJS syntax to JS compiler @tonsky and @willcohen contributed several improvements this period.

  • Add squint.math, also available as clojure.math namespace
  • #779: Added compare-and-swap!, swap-vals! and reset-vals! (@tonsky)
  • #788: Fixed compilation of dotimes with _ binding (@tonsky)
  • #790: Fixed shuffle not working on lazy sequences (@tonsky)
  • Multiple :require-macros with :refer now accumulate instead of overwriting (@willcohen)
  • Fix emitting negative zero value (-0.0)
  • Fix #792: prn js/undefined as nil
  • Fix #793: fix yield* IIFE
  • Full changelog

• scittle: Execute Clojure(Script) directly from browser script tags via SCI

  • Support async/await. See docs.
  • Implement js/import not using eval
  • Support this-as
  • nREPL: print #<Promise value> when a promise is evaluated

• nbb: Scripting in Clojure on Node.js using SCI

  • Support async/await. See docs for syntax.
  • Print promise result value in REPL/nREPL: (js/Promise.resolve 1) ;;=> #<Promise 1>

• fs - File system utility library for Clojure

  • Released 0.5.31
  • #212: Introduce :keep true option in with-temp-dir
  • #188 copy-tree now throws if src or dest is a symbolic link when not following links (@lread)
  • #201 gzip now accepts source-file Path (@lread)
  • #207 review and update glob and match docstrings (@lread)

• clerk: Moldable Live Programming for Clojure

  • Fix browse when using random port by passing 0, fixes #801
  • bb now supports editscript

• neil: A CLI to add common aliases and features to deps.edn-based projects.
  

  • #258: neil test now exits with non-zero exit code when tests fail

• cherry: Experimental ClojureScript to ES6 module compiler

  • Multiple :require-macros clauses with :refer now properly accumulate instead of overwriting each other

Contributions to third party projects:

• ClojureScript:
  • Working on async/await support (CLJS-3470). I also implemented this in SCI, scittle, and nbb.
  • CLJS-3471: fix printing of negative zero
  • CLJS-3472: str on var that is set! returns empty string
• editscript: Added babashka support, deps.edn for git dep usage, fixed CLJS tests
• riddley: Added babashka compatibility, clj-kondo config
• cloverage: Added babashka compatibility, migrated tools.cli from deprecated cli/cli to cli/parse-opts, bumped riddley
• specter: Added babashka compatibility
• compliment: Added babashka compatibility (PR #131)
• rebel-readline: Removed JLine impl class dependencies for babashka compatibility, released 0.1.7
• Selmer: Namespaced script tag context keys to avoid collisions, removed runtime require of clojure.tools.logging
• charm.clj: Contributed JLine integration, FFM native terminal interface, babashka and native-image compatibility

Other projects

These are (some of the) other projects I’m involved with but little to no activity happened in the past month.

FastMath: Thomas Clark

Q3 2025 Funding Report 2. Published Feb. 6, 2026.

Table of Contents

1. Overview
2. New Design
3. New API (and implementations)
4. Outlook

Overview

In the second term of this project, I refined and finalised the overall protocol structure for mathematical, representational and predicate operations, particularly confirming the split between what is ’necessary’ and what is ’extra’ when identifying and manipulating mathematical objects. Using this, I created several more protocols and implemented more rigorous testing namespaces - involving multiple tiers - and leaning on another Clojurists Together project for generating references: Wolframite. By introducing flexible constructor namespaces for both (real and complex) matrices and numbers, as well as multimethod-based type definitions, I solidified the overall design into a generalised API. Such an API now facilitates different layers of use: allowing for operator overloading, domain promotion and variadic arguments across objects as standard, but leaving the concrete implementations separate and available for when speed really matters.

Overall then, this work completes the funding period, successfully implementing the fastmath matrix protocol for complex matrices and extending this to a generalised API. There is however, still much to do, some of which really has to be done soon for these extensions to be practically useful. Below, I expand on what was done during this period: with a discussion of the overall design, some illustrative examples of the API and a reflection on the overall status and the necessary and hopeful next steps.

New Design

Although an enthusiastic consumer of parts of fastmath previously, it only became apparent to me, in the first part of the project, just how vast the library is. This, combined with the notion that a complex matrix API necessarily requires interaction with complex numbers (and that complex matrix implementation requires a consistent real matrix implementation), naturally led to the need to consider wider interaction problems and so wider design concerns. As highlighted in the midterm report, it became clear that, for the library to continue to expand organically, the implementation details would need to be abstracted, so that we can swap alternative backends in and out more easily in the future. This created contradictions however.

The contradictions arose as the demands for a user-friendly (and consistent) API, orients the solution towards having all of the compatible methods in the same namespace - who wants to import 20 nses to work an a single linear algebra problem? A decomplected (non-hierarchical) protocol structure however, mirrors mathematical implementations well, allows for maximal code reuse and seems to fit with Clojure’s design philosophy. Furthermore, some hierarchies are good and necessary and yet these cannot be done using Clojure protocols. My initial attempt in the first term approximated this with aliasing ’lower’ (level of abstraction) protocols within higher ones, but trying to balance this with generalised (non implementation dependent) constructors led to cyclical dependencies.

I settled in the end, therefore, for a complete separation of concerns, with protocols expanding modularly in the same way (there are now about 30), the introduction of separate, generalised constructors for numbers and matrices - that worked regardless of domain - and type implementations that were explicitly coupled to underlying libraries.

The ’user friendliness’ was then implemented entirely separately using a new, generalised, library-level, version-controlled API that exposes the object protocols and constructors in meaningful ways, but independent of the implementation. This allows the ’backends’ to be changed simply by changing the constructor dependencies. On this layer, I then explicitly implemented the linear algebra generalization, using multimethods, such that matrices and numbers can interact consistently, subject to variadic operators, with automatic domain promotion.

It is perhaps also worth mentioning, in addition to practical things like systematic naming conventions (see forthcoming documentation), that in order to integrate the new features with the existing library, I also started to abstract operations outside of the mathematical and physical needs. Fastmath is primarily used within the SciCloj/Noj ecosystem, and so ’representational’ protocols were introduced to model matrix access as a two dimensional computing structure (table), rather than a mathematical object. This helps to push the library towards integration with the wider ecosystem, with a long term goal of tighter consistency with tablecloth, tableplot and adjacent libraries.

New API (and implementations)

As a quick illustration of the APIs, consider the process of creating and working with complex numbers with pure protocols for efficiency.

(require '[fastmath.api.v2.algebra.complex :as C])
  (C/i 1.0)
;; #object[fastmath.algebra.object.number.complex.ejml.ComplexNumber 0x38306cba "1.00000+0.000000i"]

(C/add (C/i 1 2) (C/i 3 4))
;; #object[fastmath.algebra.object.number.complex.ejml.ComplexNumber 0x7454ba0 "4.00000+6.000000i"]
(C/negate (C/i 1 2))
;; #object[fastmath.algebra.object.number.complex.ejml.ComplexNumber 0x27ce1f84 "-1.00000-2.000000i"]
(C/norm (C/i 1 2))
;; 2.23606797749979

Creating matrices consistently.

  (def m--r (mat/diagonal (repeat 3 5)))
 ;; #object[fastmath.algebra.object.matrix.rectangular.real.ejml.RealDense 0x4a419cba nil]
;; :shape [3 3] :type float64
;; [[5.000 0.000 0.000]
;;  [0.000 5.000 0.000]
;;  [0.000 0.000 5.000]]
(mat/<-real ... )
;; #object[fastmath.algebra.object.matrix.rectangular.complex.ejml.ComplexDense 0x3913463b nil]
;; :shape [3 3] :type float64
;; [[5.000 0.000 0.000]
;;  [0.000 5.000 0.000]
;;  [0.000 0.000 5.000]]
(def m--c (mat/<-coll 3 3 (partition 2 (range 18))))
;; #object[fastmath.algebra.object.matrix.rectangular.complex.ejml.ComplexDense 0x9a42bb7 nil]
;; :shape [3 3] :type complex128
;; [[0.000+1.000000i   2.000+3.000000i   4.000+5.000000i  ]
;;  [6.000+7.000000i   8.000+9.000000i   10.000+11.000000i]
;;  [12.000+13.000000i 14.000+15.000000i 16.000+17.000000i]]
(mat/<-rows [[10.0 1.0  -5.0]
             [0.0 4.0  2.789]
             [-5 31 8]])
;; #object[fastmath.algebra.object.matrix.rectangular.real.ejml.RealDense 0x3c9a1815 nil]
;; :shape [3 3] :type float64
;; [[10.000 1.000  -5.000]
;;  [0.000  4.000  2.789 ]
;;  [-5.000 31.000 8.000 ]]
(mat/<-rows [[(i 78 0.0) (i 0.0 77) (i 16.13456 56)]
             [(i 9134 -341) (i 24 2341) (i 10 -56)]])
;; #object[fastmath.algebra.object.matrix.rectangular.complex.ejml.ComplexDense 0x113c5d0b nil]
;; :shape [2 3] :type complex128
;; [[78.000+0.000000i     0.000+77.000000i     16.135+56.000000i   ]
;;  [9134.000-341.000000i 24.000+2341.000000i  10.000-56.000000i   ]]

And mixing it altogether with domain promotion, general symbols and variadic arguments.

(refer-clojure :exclude [+ -])
(require '[fastmath.api.v2.algebra.general :as alg :refer [+ *]])

(+ 5 (C/i 1 2) (mat/identity 2))
;; #object[fastmath.algebra.object.matrix.rectangular.complex.ejml.ComplexDense 0x61223e14 nil]
;; :shape [2 2] :type complex128
;; [[7.000+2.000000i 6.000+2.000000i]
;;  [6.000+2.000000i 7.000+2.000000i]]

(+ m--c m--r m--r m--c)
;; #object[fastmath.algebra.object.matrix.rectangular.complex.ejml.ComplexDense 0x4f72c29b nil]
;; :shape [3 3] :type complex128
;; [[10.000+2.000000i  4.000+6.000000i   8.000+10.000000i ]
;;  [12.000+14.000000i 26.000+18.000000i 20.000+22.000000i]
;;  [24.000+26.000000i 28.000+30.000000i 42.000+34.000000i]]

Outlook

As can be seen above, it is now possible to interleave fundamental matrix and numeric operations without overt concern for domain. This is simply scratching the surface of fastmath’s potential however. At the end of the first term, it had already become clear that the original scope of the project was too large to be completed within the given timeframe, and so all work had to be done with future extension in mind.

There are some immediate next steps however, that ideally would already have been completed. These are to coordinate the integration of this branch with generateme, to finalize the optimization and clay documentation and to publicize subsequent performance metrics. I hope to complete these promptly.

Medium-term goals would then see expansion of the integration of this extension with other fastmath features. For, as discussed, fastmath is a large library and although the core matrix protocol has been implemented, there are many other functions that apply not just to matrices in particular but which could benefit from a unified numeric tower of mathematical operations. Other features within reach are also the exploitation of EJML’s parallelism for longer calculations, as well as an alternative implementaton using ojAlgo.

Following this, I would also like to develop explicit links between fastmath and other libraries. The lowest-hanging fruit would be to officially support fastmath-wolframite interop. Having already used Wolframite to provide tests and having already worked on the Wolframite library itself, this should be relatively straightforward - and would allow easier delegation from fastmath to Wolframite (and return) for currently unimplemented algorithms. Similarly, as part of my long-term goal to integrate various Clojure libraries towards physics research, I would like to make interop with emmy a smooth reality. Potentially, even using fastmath as an implementation for backend calculations. Likewise, and more easily than for emmy, I would like to see libraries like qclojure use fastmath for their quantum implementations.

Overall then, although I would have liked to get further in the time available, I am excited by this experiment in the suitability of Clojure for general mathematics. I hope that the library, even just the algebraic namespace protocols themselves, provide a basis for future work in a wide array of applications and I hope that, having read this little report, you are inspired to try it out in your own work. Please don’t hesitate to get in touch if that’s the case.

Permalink

Ultra Process 2

Another Ultra Process gig for The Printer Jam.

Venue: Folklore, Hoxton.

Tickets here.

Image credit: Evan Raskob.

Permalink

Learn Ring - 02. Who Am I?

Notes

• Freelancer
• Karthikeyan A K (Car + Thick)
  • Chennai, India
• Tech freak
• Web Developer
  • Ruby on Rails
• Data Scientist
  • SciCloj Book: Stats With Clojure
• Numerical Works
• Injee

Permalink

The tools of an Agentic Engineer

Permalink

Senior Frontend Engineer (ClojureScript) at Pitch Software GmbH

Permalink

Negative Sets as Data

Cofinite sets with one sentinel key: all set operations reduce to three map primitives.

Permalink