Build and Deploy Web Apps With Clojure and FLy.io

This post walks through a small web development project using Clojure, covering everything from building the app to packaging and deploying it. It’s a collection of insights and tips I’ve learned from building my Clojure side projects but presented in a more structured format.

As the title suggests, we’ll be deploying the app to Fly.io. It’s a service that allows you to deploy apps packaged as Docker images on lightweight virtual machines.^[1] My experience with it has been good, it’s easy to use and quick to set up. One downside of Fly is that it doesn’t have a free tier, but if you don’t plan on leaving the app deployed, it barely costs anything.

This isn’t a tutorial on Clojure, so I’ll assume you already have some familiarity with the language as well as some of its libraries.^[2]

Project Setup

In this post, we’ll be building a barebones bookmarks manager for the demo app. Users can log in using basic authentication, view all bookmarks, and create a new bookmark. It’ll be a traditional multi-page web app and the data will be stored in a SQLite database.

Here’s an overview of the project’s starting directory structure:

.
├── dev
│   └── user.clj
├── resources
│   └── config.edn
├── src
│   └── acme
│       └── main.clj
└── deps.edn

And the libraries we’re going to use. If you have some Clojure experience or have used Kit, you’re probably already familiar with all the libraries listed below.^[3]

;; deps.edn
{:paths ["src" "resources"]
 :deps {org.clojure/clojure               {:mvn/version "1.12.0"}
        aero/aero                         {:mvn/version "1.1.6"}
        integrant/integrant               {:mvn/version "0.11.0"}
        ring/ring-jetty-adapter           {:mvn/version "1.12.2"}
        metosin/reitit-ring               {:mvn/version "0.7.2"}
        com.github.seancorfield/next.jdbc {:mvn/version "1.3.939"}
        org.xerial/sqlite-jdbc            {:mvn/version "3.46.1.0"}
        hiccup/hiccup                     {:mvn/version "2.0.0-RC3"}}
 :aliases
 {:dev {:extra-paths ["dev"]
        :extra-deps  {nrepl/nrepl    {:mvn/version "1.3.0"}
                      integrant/repl {:mvn/version "0.3.3"}}
        :main-opts   ["-m" "nrepl.cmdline" "--interactive" "--color"]}}}

I use Aero and Integrant for my system configuration (more on this in the next section), Ring with the Jetty adaptor for the web server, Reitit for routing, next.jdbc for database interaction, and Hiccup for rendering HTML. From what I’ve seen, this is a popular “library combination” for building web apps in Clojure.^[4]

The user namespace in dev/user.clj contains helper functions from Integrant-repl to start, stop, and restart the Integrant system.

;; dev/user.clj
(ns user
  (:require
   [acme.main :as main]
   [clojure.tools.namespace.repl :as repl]
   [integrant.core :as ig]
   [integrant.repl :refer [set-prep! go halt reset reset-all]]))

(set-prep!
 (fn []
   (ig/expand (main/read-config)))) ;; we'll implement this soon

(repl/set-refresh-dirs "src" "resources")

(comment
  (go)
  (halt)
  (reset)
  (reset-all))

Systems and Configuration

If you’re new to Integrant or other dependency injection libraries like Component, I’d suggest reading “How to Structure a Clojure Web”. It’s a great explanation about the reasoning behind these libraries. Like most Clojure apps that use Aero and Integrant, my system configuration lives in a .edn file. I usually name mine as resources/config.edn. Here’s what it looks like:

;; resources/config.edn
{:server
 {:port #long #or [#env PORT 8080]
  :host #or [#env HOST "0.0.0.0"]
  :auth {:username #or [#env AUTH_USER "john.doe@email.com"]
         :password #or [#env AUTH_PASSWORD "password"]}}

 :database
 {:dbtype "sqlite"
  :dbname #or [#env DB_DATABASE "database.db"]}}

In production, most of these values will be set using environment variables. During local development, the app will use the hard-coded default values. We don’t have any sensitive values in our config (e.g., API keys), so it’s fine to commit this file to version control. If there are such values, I usually put them in another file that’s not tracked by version control and include them in the config file using Aero’s #include reader tag.

This config file is then “expanded” into the Integrant system map using the expand-key method:

;; src/acme/main.clj
(ns acme.main
  (:require
   [aero.core :as aero]
   [clojure.java.io :as io]
   [integrant.core :as ig]))

(defn read-config
  []
  {:system/config (aero/read-config (io/resource "config.edn"))})

(defmethod ig/expand-key :system/config
  [_ opts]
  (let [{:keys [server database]} opts]
    {:server/jetty (assoc server :handler (ig/ref :handler/ring))
     :handler/ring {:database (ig/ref :database/sql)
                    :auth     (:auth server)}
     :database/sql database}))

The system map is created in code instead of being in the configuration file. This makes refactoring your system simpler as you only need to change this method while leaving the config file (mostly) untouched.^[5]

My current approach to Integrant + Aero config files is mostly inspired by the blog post “Rethinking Config with Aero & Integrant” and Laravel’s configuration. The config file follows a similar structure to Laravel’s config files and contains the app configurations without describing the structure of the system. Previously I had a key for each Integrant component, which led to the config file being littered with #ig/ref and more difficult to refactor.

Also, if you haven’t already, start a REPL and connect to it from your editor. Run clj -M:dev if your editor doesn’t automatically start a REPL. Next, we’ll implement the init-key and halt-key! methods for each of the components:

;; src/acme/main.clj
(ns acme.main
  (:require
   ;; ...
   [acme.handler :as handler]
   [acme.util :as util])
   [next.jdbc :as jdbc]
   [ring.adapter.jetty :as jetty]))
;; ...

(defmethod ig/init-key :server/jetty
  [_ opts]
  (let [{:keys [handler port]} opts
        jetty-opts (-> opts (dissoc :handler :auth) (assoc :join? false))
        server     (jetty/run-jetty handler jetty-opts)]
    (println "Server started on port " port)
    server))

(defmethod ig/halt-key! :server/jetty
  [_ server]
  (.stop server))

(defmethod ig/init-key :handler/ring
  [_ opts]
  (handler/handler opts))

(defmethod ig/init-key :database/sql
  [_ opts]
  (let [datasource (jdbc/get-datasource opts)]
    (util/setup-db datasource)
    datasource))

The setup-db function creates the required tables in the database if they don’t exist yet. This works fine for database migrations in small projects like this demo app, but for larger projects, consider using libraries such as Migratus (my preferred library) or Ragtime.

;; src/acme/util.clj
(ns acme.util 
  (:require
   [next.jdbc :as jdbc]))

(defn setup-db
  [db]
  (jdbc/execute-one!
   db
   ["create table if not exists bookmarks (
       bookmark_id text primary key not null,
       url text not null,
       created_at datetime default (unixepoch()) not null
     )"]))

For the server handler, let’s start with a simple function that returns a “hi world” string.

;; src/acme/handler.clj
(ns acme.handler
  (:require
   [ring.util.response :as res]))

(defn handler
  [_opts]
  (fn [req]
    (res/response "hi world")))

Now all the components are implemented. We can check if the system is working properly by evaluating (reset) in the user namespace. This will reload your files and restart the system. You should see this message printed in your REPL:

:reloading (acme.util acme.handler acme.main)
Server started on port  8080
:resumed

If we send a request to http://localhost:8080/, we should get “hi world” as the response:

$ curl localhost:8080/
hi world

Nice! The system is working correctly. In the next section, we’ll implement routing and our business logic handlers.

Routing, Middleware, and Route Handlers

First, let’s set up a ring handler and router using Reitit. We only have one route, the index / route that’ll handle both GET and POST requests.

;; src/acme/handler.clj
(ns acme.handler
  (:require
   [reitit.ring :as ring]))

(def routes
  [["/" {:get  index-page
         :post index-action}]])

(defn handler
  [opts]
  (ring/ring-handler
   (ring/router routes)
   (ring/routes
    (ring/redirect-trailing-slash-handler)
    (ring/create-resource-handler {:path "/"})
    (ring/create-default-handler))))

We’re including some useful middleware:

• redirect-trailing-slash-handler to resolve routes with trailing slashes,
• create-resource-handler to serve static files, and
• create-default-handler to handle common 40x responses.

Implementing the Middlewares

If you remember the :handler/ring from earlier, you’ll notice that it has two dependencies, database and auth. Currently, they’re inaccessible to our route handlers. To fix this, we can inject these components into the Ring request map using a middleware function.

;; src/acme/handler.clj
;; ...

(defn components-middleware
  [components]
  (let [{:keys [database auth]} components]
    (fn [handler]
      (fn [req]
        (handler (assoc req
                        :db database
                        :auth auth))))))
;; ...

The components-middleware function takes in a map of components and creates a middleware function that “assocs” each component into the request map.^[6] If you have more components such as a Redis cache or a mail service, you can add them here.

We’ll also need a middleware to handle HTTP basic authentication.^[7] This middleware will check if the username and password from the request map matche the values in the auth map injected by components-middleware. If they match, then the request is authenticated and the user can view the site.

;; src/acme/handler.clj
(ns acme.handler
  (:require
   ;; ...
   [acme.util :as util]
   [ring.util.response :as res]))
;; ...

(defn wrap-basic-auth
  [handler]
  (fn [req]
    (let [{:keys [headers auth]} req
          {:keys [username password]} auth
          authorization (get headers "authorization")
          correct-creds (str "Basic " (util/base64-encode
                                       (format "%s:%s" username password)))]
      (if (and authorization (= correct-creds authorization))
        (handler req)
        (-> (res/response "Access Denied")
            (res/status 401)
            (res/header "WWW-Authenticate" "Basic realm=protected"))))))
;; ...

A nice feature of Clojure is that interop with the host language is easy. The base64-encode function is just a thin wrapper over Java’s Base64.Encoder:

;; src/acme/util.clj
(ns acme.util
   ;; ...
  (:import java.util.Base64))

(defn base64-encode
  [s]
  (.encodeToString (Base64/getEncoder) (.getBytes s)))

Finally, we need to add them to the router. Since we’ll be handling form requests later, we’ll also bring in Ring’s wrap-params middleware.

;; src/acme/handler.clj
(ns acme.handler
  (:require
   ;; ...
   [ring.middleware.params :refer [wrap-params]]))
;; ...

(defn handler
  [opts]
  (ring/ring-handler
   ;; ...
   {:middleware [(components-middleware opts)
                 wrap-basic-auth
                 wrap-params]}))

Implementing the Route Handlers

We now have everything we need to implement the route handlers or the business logic of the app. First, we’ll implement the index-page function which renders a page that:

1. Shows all of the user’s bookmarks in the database, and
2. Shows a form that allows the user to insert new bookmarks into the database

;; src/acme/handler.clj
(ns acme.handler
  (:require
   ;; ...
   [next.jdbc :as jdbc]
   [next.jdbc.sql :as sql]))
;; ...

(defn template
  [bookmarks]
  [:html
   [:head
    [:meta {:charset "utf-8"
            :name    "viewport"
            :content "width=device-width, initial-scale=1.0"}]]
   [:body
    [:h1 "bookmarks"]
    [:form {:method "POST"}
     [:div
      [:label {:for "url"} "url "]
      [:input#url {:name "url"
                   :type "url"
                   :required true
                   :placeholer "https://en.wikipedia.org/"}]]
     [:button "submit"]]
    [:p "your bookmarks:"]
    [:ul
     (if (empty? bookmarks)
       [:li "you don't have any bookmarks"]
       (map
        (fn [{:keys [url]}]
          [:li
           [:a {:href url} url]])
        bookmarks))]]])

(defn index-page
  [req]
  (try
    (let [bookmarks (sql/query (:db req)
                               ["select * from bookmarks"]
                               jdbc/unqualified-snake-kebab-opts)]
      (util/render (template bookmarks)))
    (catch Exception e
      (util/server-error e))))
;; ...

Database queries can sometimes throw exceptions, so it’s good to wrap them in a try-catch block. I’ll also introduce some helper functions:

;; src/acme/util.clj
(ns acme.util
  (:require
   ;; ...
   [hiccup2.core :as h]
   [ring.util.response :as res])
  (:import java.util.Base64))
;; ...

(defn preprend-doctype
  [s]
  (str "<!doctype html>" s))

(defn render
  [hiccup]
  (-> hiccup h/html str preprend-doctype res/response (res/content-type "text/html")))

(defn server-error
  [e]
  (println "Caught exception: " e)
  (-> (res/response "Internal server error")
      (res/status 500)))

render takes a hiccup form and turns it into a ring response, while server-error takes an exception, logs it, and returns a 500 response.

Next, we’ll implement the index-action function:

;; src/acme/handler.clj
;; ...

(defn index-action
  [req]
  (try
    (let [{:keys [db form-params]} req
          value (get form-params "url")]
      (sql/insert! db :bookmarks {:bookmark_id (random-uuid) :url value})
      (res/redirect "/" 303))
    (catch Exception e
      (util/server-error e))))
;; ...

This is an implementation of a typical post/redirect/get pattern. We get the value from the URL form field, insert a new row in the database with that value, and redirect back to the index page. Again, we’re using a try-catch block to handle possible exceptions from the database query.

That should be all of the code for the controllers. If you reload your REPL and go to http://localhost:8080, you should see something that looks like this after logging in:

The last thing we need to do is to update the main function to start the system:

;; src/acme/main.clj
;; ...

(defn -main [& _]
  (-> (read-config) ig/expand ig/init))

Now, you should be able to run the app using clj -M -m acme.main. That’s all the code needed for the app. In the next section, we’ll package the app into a Docker image to deploy to Fly.

Packaging the App

While there are many ways to package a Clojure app, Fly.io specifically requires a Docker image. There are two approaches to doing this:

1. Build an uberjar and run it using Java in the container, or
2. Load the source code and run it using Clojure in the container

Both are valid approaches. I prefer the first since its only dependency is the JVM. We’ll use the tools.build library to build the uberjar. Check out the official guide for more information on building Clojure programs. Since it’s a library, to use it we can add it to our deps.edn file with an alias:

;; deps.edn
{;; ...
 :aliases
 {;; ...
  :build {:extra-deps {io.github.clojure/tools.build 
                       {:git/tag "v0.10.5" :git/sha "2a21b7a"}}
          :ns-default build}}}

Tools.build expects a build.clj file in the root of the project directory, so we’ll need to create that file. This file contains the instructions to build artefacts, which in our case is a single uberjar. There are many great examples of build.clj files on the web, including from the official documentation. For now, you can copy+paste this file into your project.

;; build.clj
(ns build
  (:require
   [clojure.tools.build.api :as b]))

(def basis (delay (b/create-basis {:project "deps.edn"})))
(def src-dirs ["src" "resources"])
(def class-dir "target/classes")

(defn uber
  [_]
  (println "Cleaning build directory...")
  (b/delete {:path "target"})

  (println "Copying files...")
  (b/copy-dir {:src-dirs   src-dirs
               :target-dir class-dir})

  (println "Compiling Clojure...")
  (b/compile-clj {:basis      @basis
                  :ns-compile '[acme.main]
                  :class-dir  class-dir})

  (println "Building Uberjar...")
  (b/uber {:basis     @basis
           :class-dir class-dir
           :uber-file "target/standalone.jar"
           :main      'acme.main}))

To build the project, run clj -T:build uber. This will create the uberjar standalone.jar in the target directory. The uber in clj -T:build uber refers to the uber function from build.clj. Since the build system is a Clojure program, you can customise it however you like. If we try to run the uberjar now, we’ll get an error:

# build the uberjar
$ clj -T:build uber
Cleaning build directory...
Copying files...
Compiling Clojure...
Building Uberjar...

# run the uberjar
$ java -jar target/standalone.jar
Error: Could not find or load main class acme.main
Caused by: java.lang.ClassNotFoundException: acme.main

This error occurred because the Main class that is required by Java isn’t built. To fix this, we need to add the :gen-class directive in our main namespace. This will instruct Clojure to create the Main class from the -main function.

;; src/acme/main.clj
(ns acme.main
  ;; ...
  (:gen-class))
;; ...

If you rebuild the project and run java -jar target/standalone.jar again, it should work perfectly. Now that we have a working build script, we can write the Dockerfile:

# Dockerfile
# install additional dependencies here in the base layer
# separate base from build layer so any additional deps installed are cached
FROM clojure:temurin-21-tools-deps-bookworm-slim AS base

FROM base as build
WORKDIR /opt
COPY . .
RUN clj -T:build uber

FROM eclipse-temurin:21-alpine AS prod
COPY --from=build /opt/target/standalone.jar /
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "standalone.jar"]

It’s a multi-stage Dockerfile. We use the official Clojure Docker image as the layer to build the uberjar. Once it’s built, we copy it to a smaller Docker image that only contains the Java runtime.^[8] By doing this, we get a smaller container image as well as a faster Docker build time because the layers are better cached.

That should be all for packaging the app. We can move on to the deployment now.

Deploying with Fly.io

First things first, you’ll need to install flyctl, Fly’s CLI tool for interacting with their platform. Create a Fly.io account if you haven’t already. Then run fly auth login to authenticate flyctl with your account.

Next, we’ll need to create a new Fly App:

$ fly app create
? Choose an app name (leave blank to generate one): 
automatically selected personal organization: Ryan Martin
New app created: blue-water-6489

Another way to do this is with the fly launch command, which automates a lot of the app configuration for you. We have some steps to do that are not done by fly launch, so we’ll be configuring the app manually. I also already have a fly.toml file ready that you can straight away copy to your project.

# fly.toml
# replace these with your app and region name
# run `fly platform regions` to get a list of regions
app = 'blue-water-6489' 
primary_region = 'sin'

[env]
  DB_DATABASE = "/data/database.db"

[http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = "stop"
  auto_start_machines = true
  min_machines_running = 0

[mounts]
  source = "data"
  destination = "/data"
  initial_sie = 1

[[vm]]
  size = "shared-cpu-1x"
  memory = "512mb"
  cpus = 1
  cpu_kind = "shared"

These are mostly the default configuration values with some additions. Under the [env] section, we’re setting the SQLite database location to /data/database.db. The database.db file itself will be stored in a persistent Fly Volume mounted on the /data directory. This is specified under the [mounts] section. Fly Volumes are similar to regular Docker volumes but are designed for Fly’s micro VMs.

We’ll need to set the AUTH_USER and AUTH_PASSWORD environment variables too, but not through the fly.toml file as these are sensitive values. To securely set these credentials with Fly, we can set them as app secrets. They’re stored encrypted and will be automatically injected into the app at boot time.

$ fly secrets set AUTH_USER=hi@ryanmartin.me AUTH_PASSWORD=not-so-secure-password
Secrets are staged for the first deployment

With this, the configuration is done and we can deploy the app using fly deploy:

$ fly deploy
# ...
Checking DNS configuration for blue-water-6489.fly.dev

Visit your newly deployed app at https://blue-water-6489.fly.dev/

The first deployment will take longer since it’s building the Docker image for the first time. Subsequent deployments should be faster due to the cached image layers. You can click on the link to view the deployed app, or you can also run fly open which will do the same thing. Here’s the app in action:

If you made additional changes to the app or fly.toml, you can redeploy the app using the same command, fly deploy. The app is configured to auto stop/start, which helps to cut costs when there’s not a lot of traffic to the site. If you want to take down the deployment, you’ll need to delete the app itself using fly app destroy <your app name>.

Adding a Production REPL

This is an interesting topic in the Clojure community, with varying opinions on whether or not it’s a good idea. Personally I find having a REPL connected to the live app helpful, and I often use it for debugging and running queries on the live database.^[9] Since we’re using SQLite, we don’t have a database server we can directly connect to, unlike Postgres or MySQL.

If you’re brave, you can even restart the app directly without redeploying from the REPL. You can easily go wrong with it, which is why some prefer to not use it.

For this project, we’re gonna add a socket REPL. It’s very simple to add (you just need to add a JVM option) and it doesn’t require additional dependencies like nREPL. Let’s update the Dockerfile:

# Dockerfile
# ...
EXPOSE 7888
ENTRYPOINT ["java", "-Dclojure.server.repl={:port 7888 :accept clojure.core.server/repl}", "-jar", "standalone.jar"]

The socket REPL will be listening on port 7888. If we redeploy the app now, the REPL will be started but we won’t be able to connect to it. That’s because we haven’t exposed the service through Fly proxy. We can do this by adding the socket REPL as a service in the [services] section in fly.toml.

However, doing this will also expose the REPL port to the public. This means that anyone can connect to your REPL and possibly mess with your app. Instead, what we want to do is to configure the socket REPL as a private service.

By default, all Fly apps in your organisation live in the same private network. This private network, called 6PN, connects the apps in your organisation through Wireguard tunnels (a VPN) using IPv6. Fly private services aren’t exposed to the public internet but can be reached from this private network. We can then use Wireguard to connect to this private network to reach our socket REPL.

Fly VMs are also configured with the hostname fly-local-6pn, which maps to its 6PN address. This is analogous to localhost, which points to your loopback address 127.0.0.1. To expose a service to 6PN, all we have to do is bind or serve it to fly-local-6pn instead of the usual 0.0.0.0. We have to update the socket REPL options to:

# Dockerfile
# ...
ENTRYPOINT ["java", "-Dclojure.server.repl={:port 7888,:address \"fly-local-6pn\",:accept clojure.core.server/repl}", "-jar", "standalone.jar"]

After redeploying, we can use the fly proxy command to forward the port from the remote server to our local machine.^[10]

$ fly proxy 7888:7888
Proxying local port 7888 to remote [blue-water-6489.internal]:7888

In another shell, run:

$ rlwrap nc localhost 7888
user=>

Now we have a REPL connected to the production app! rlwrap is used for readline functionality, e.g. up/down arrow keys, vi bindings. Of course you can also connect to it from your editor.

Deploy with GitHub Actions

If you’re using GitHub, we can also set up automatic deployments on pushes/PRs with GitHub Actions. All you need is to create the workflow file:

# .github/workflows/fly.yaml
name: Fly Deploy
on:
  push:
    branches:
      - main
  workflow_dispatch:

jobs:
  deploy:
    name: Deploy app
    runs-on: ubuntu-latest
    concurrency: deploy-group
    steps:
      - uses: actions/checkout@v4
      - uses: superfly/flyctl-actions/setup-flyctl@master
      - run: flyctl deploy --remote-only
        env:
          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}

To get this to work, you’ll need to create a deploy token from your app’s dashboard. Then, in your GitHub repo, create a new repository secret called FLY_API_TOKEN with the value of your deploy token. Now, whenever you push to the main branch, this workflow will automatically run and deploy your app. You can also manually run the workflow from GitHub because of the workflow_dispatch option.

End

As always, all the code is available on GitHub. Originally, this post was just about deploying to Fly.io, but along the way I kept adding on more stuff until it essentially became my version of the user manager example app. Anyway, hope this article provided a good view into web development with Clojure. As a bonus, here are some additional resources on deploying Clojure apps:

• Deploying a Full-Stack Clojure App With Kamal on a Single Server
• JVM Deployment Options
• Deploying Clojure Like a Seasoned Hobbyist
• Brave Clojure’s Deploy Quest

Permalink

Apple silicon support in Clojure's Neanderthal Fast Matrix Library

I've applied for Clojurists Together funding to explore and hopefully implement a Neanderthal Apple silicon backend. If you are a Clojurists Together member, and you think that this functionality is important, please have that in mind when voting. Here's the proposal that I've sent. Of course, any suggestions are welcome and highly appreciated!

Permalink

Loose Ends

by cgrand (🦋 🦣 𝕏)

Today, just some quick CLJD news and some things I found interesting over the last week:

• The CVM algorithm,
• Least Squares Circle Fit,
• Fractional Brownian Motion applied to Signed Distance Fields.

To echo the title: at last, a video that made me understand why the direction in which you tie shoelaces matters:

Some ClojureDart news

What's New: Broader Testing Support

We've expanded testing support in ClojureDart beyond unit testing, now integrating seamlessly with Dart's widget and integration testing tools. Read more here or ask on Clojurians' Slack #Clojuredart channel.

Open-Source Support Plans for Companies

For companies seeking accountant-friendly options beyond GitHub Sponsors, we've introduced Open-Source Support Plans with Stripe payments and proper invoicing.

The CVM algorithm

(via Null Bitmap)

The CVM algorithm elegantly computes an unbiased approximation of the number of distinct values in a stream with an upper bound on memory usage.

Here is an annotated Clojure version I wrote to understand the algorithm:

(defn approximate-count-distinct
  "Returns an approximation of the number of distinct values in xs.
  If this number is less than than threshold, the returned count is exact.
  Threshold also acts as a bound on memory consumption.
  Implement the CVM algorithm https://en.wikipedia.org/wiki/Count-distinct_problem#CVM_Algorithm"
  [threshold xs]
  (loop [multiplier 1 values #{} xs xs]
    (if (<= (count values) threshold)
      (if-some [[x & xs] (seq xs)]
        (let [f (if (zero? (rand-int multiplier)) conj disj)]
          ; it looks like randomly conj or disj but it can
          ; be seen differently:
          ; an unconditional disj followed by a potential conj  
          (recur multiplier (f values x) xs))
        (* multiplier (count values)))
      ; the sample is too big, we need to probabilistically halve it
      ; each time we halve the sample we double the multiplier
      ; ⚠️ you have to go through the (<= (count values) threshold) check after halving
      ; because if you are very unlucky you may not have shrunk the sample!
      (recur (* 2 multiplier) (into #{} (filter (fn [_] (zero? (rand-int 2)))) values) xs))))

If you compare to the Knuth's version of the algorithm described in the Wikipedia's article, you may wonder why a map of values to random numbers is used while in the above Clojure code a plain set of values is used.

That's because we only conj in the set when the random integer is zero so we would only store 0. Quite useless.

The random values of the maps are only used when shrinking the samples. Here we do a coin flip ((rand-int 2)) which can be seen as lazily computing further digits of the number we didn't store.

Thus it doesn't change the correctness of the implementation.

The algorithm works surprisingly well even for ridiculously small sample sizes: see below, with a sample size of 4, averaging over 10 runs to make up for the small sample.

=> (/ (reduce + (repeatedly 10 #(approximate-count-distinct 4 (range 2e6)))) 10.0)
1939865.6
=> (/ (reduce + (repeatedly 10 #(approximate-count-distinct 4 (concat (range 1e6) (range 1e6))))) 10.0)
1127219.2

Least Squares Circle Fit

A closed formula to fit a circle to a cloud of points.

Fractional Brownian Motion applied to Signed Distance Fields

Inigo Quiliez constantly blows my mind, this time generating a fractal landscape as a single SDF, a real SDF, one whose gradient has length 1.0.

Stay Tuned!

This installment was a lighter update, but the Worst Datalog Ever series will be back soon with the next chapter—constraints!

Permalink

Ruby: One of the top 5 highest-paying technologies, according to Stack overflow

Introduction

In the world of software development, Ruby has a unique charm that’s kept it relevant and in demand for years. Known for its focus on simplicity and developer happiness, Ruby powers some of the world’s most popular websites and applications, making it a language that doesn’t just get the job done but makes coding genuinely enjoyable.

Year after year, Ruby continues to rank among the highest-paying technologies globally, and it’s not hard to see why. Beyond its elegant syntax and the powerful Ruby on Rails framework, Ruby appeals to a wide range of businesses looking for efficient, well-crafted code—and they’re willing to pay top dollar for it.

The Competitive Landscape for Tech Salaries

In the fast-paced world of technology, programming languages rise and fall in popularity as new ones emerge, specialized for unique tasks or optimized for performance. In recent years, languages like Clojure, Perl, and Zig have consistently ranked at the top in terms of salary, alongside established languages like Ruby. These high salaries are often a direct reflection of each language’s demand, the scarcity of skilled developers, and the specific value they offer to organizations.

For example, Clojure and Zig may not have the same mainstream appeal as JavaScript or Python, but they’re highly valued for specialized roles that require advanced performance or functional programming capabilities. This demand creates a smaller, competitive job market, where companies pay a premium for expertise that can help solve complex problems or deliver cutting-edge solutions.

Salaries in the tech industry can tell us a lot about the market forces at play. High compensation often signals that a technology requires deep expertise, is less commonly known, or has a significant impact on an organization’s efficiency or product.

In this competitive landscape, Ruby’s consistently high salaries suggest it remains indispensable to many organizations, particularly in fields like web development and startups. The continued willingness to invest in Ruby developers, even as newer languages emerge, speaks to Ruby’s staying power and the value it brings to projects that prioritize developer productivity and rapid iteration.

Ruby salary data

According to the Developer Survey conducted annually by the most respected programming resource, stackoverflow.

Stack Overflow’s annual Developer Survey is the largest and most comprehensive survey of people who code around the world.

• 2019

• 2020

• 2021

• 2022

• 2023

Take a look at the references section if you want to double check the data.

Ruby Salary Trends: 2019-2023

• 2019: 6th highest salary globally, at $75k, with a $15k gap from the highest (Clojure).
• 2020: 5th place with $71k, narrowing the gap to $5k from Perl, the highest-paid language.
• 2021: Returns to 6th at $80k, with a $15k gap from the top (Clojure).
• 2022: Climbs to 5th at $93k, with a $13k gap from the highest (Clojure).
• 2023: Reaches 4th with nearly $99k, narrowing the gap to $4k from the highest (Zig).

Data breakdown

Consistent High Ranking

Ruby’s steady position among the top 10 highest-paying technologies from 2019 to 2023 highlights its enduring value in the tech world.

Despite a rapidly changing landscape filled with emerging languages and tools, Ruby has consistently held strong, maintaining a rank between 4th and 6th place globally. This high ranking isn’t just a matter of coincidence—it reflects the steady demand for Ruby expertise across a variety of industries.

The fact that Ruby has remained highly paid over several years suggests a level of trust and reliance from companies, especially in sectors like web development, e-commerce, and tech startups, where Ruby on Rails has become a go-to framework for building robust applications quickly.

Businesses that rely on Ruby are often focused on developer productivity, maintainability, and the ability to bring applications to market smoothly.

This consistent ranking demonstrates that Ruby isn’t just a temporary trend; it’s a mature and reliable choice for companies who value efficiency, scalability, and developer satisfaction.

Steady Salary Increase

Ruby has seen a noticeable rise in average salaries over the past few years, reflecting its sustained demand and relevance in the tech industry. From 2019 to 2023, Ruby developer salaries steadily climbed, starting at $75k and reaching nearly $99k.

This consistent trajectory is a strong indicator that companies recognize and are willing to pay for Ruby’s unique benefits, especially as experienced Ruby developers remain highly valued across many sectors.

These salary increases aren’t just about inflation—they reflect the strategic importance of Ruby in areas like web application development, where speed and efficiency are necessary. Ruby’s ability to help teams build high-quality, scalable applications quickly makes it a favorite in industries where time-to-market and agility are essential.

As more companies see the value of Ruby’s productivity-focused design, demand for skilled developers continues to rise, and salaries follow suit.

For developers, this steady salary growth offers an encouraging outlook. Whether you’re new to Ruby or a seasoned expert, the market’s consistent investment in Ruby skills suggests that learning and mastering the language can lead to increasingly rewarding career opportunities. It’s a clear signal that the language remains valuable, even as new technologies emerge.

Narrowing Salary Gap with Top Technologies

One of the most striking trends in recent years is Ruby’s steady approach to the top of the tech salary charts. In 2019, the gap between Ruby developers and those working with the highest-paying technology was around $15,000.

Fast-forward to 2023, and that gap has shrunk to just about $4,000. This narrowing gap tells an interesting story about Ruby’s place in the tech landscape: it’s a language that continues to grow in competitiveness and value, even as new, highly specialized languages enter the market.

For developers, this trend is promising. It indicates that Ruby, despite being a more established language, remains as financially rewarding as many newer alternatives. This speaks to its adaptability, robust ecosystem, and the strong developer community that continues to advance the language.

Companies clearly recognize that hiring skilled Ruby developers brings long-term value, and as they compete to attract top talent, salaries have risen accordingly.

Ultimately, this narrowing gap highlights that Ruby is a strategic choice for developers seeking both high compensation and diverse opportunities, affirming that it’s not only relevant but also competitive at the highest levels in today’s job market.

Why Ruby?

Developer Productivity and Happiness

One of Ruby’s core philosophies is to make coding a pleasant and productive experience. The language was designed with simplicity and readability in mind, allowing developers to write clear, concise code that’s easy to understand and maintain.

This focus on developer happiness has made Ruby especially popular in environments that prioritize rapid iteration, such as tech startups and agile development teams.

By reducing the cognitive load on developers and speeding up the coding process, Ruby allows teams to focus on building impactful features rather than getting stuck in syntax or overly complex structures.

Loyal community

A key factor behind Ruby’s success is its loyal community and the extensive resources this network has cultivated over the years. Ruby has been around long enough to develop a vast ecosystem filled with powerful libraries, versatile plugins, and thorough documentation.

This community-driven support system provides developers with everything they need to succeed—from robust tools and educational content to networking opportunities and collaborative projects, such as Meetups and Tropical Rails in Brazil.

One of Ruby’s greatest assets is the continued popularity of Ruby on Rails, which has been widely adopted for building reliable, scalable applications. The Ruby community actively contributes to Rails and the language itself, consistently introducing updates, best practices, and new features.

This collaborative environment attracts both new talent and experienced developers, while offering companies confidence in the language’s long-term viability and stability.

For businesses, hiring Ruby developers means accessing not only technical expertise but also a community that has made Ruby development faster, easier, and more reliable.

References

2023 Top paying technologies

2022 Top paying technologies

2021 Top paying technologies

2020 Top paying technologies

2019 Top paying technologies

Done

Let's network

• Github
• LinkedIn
• Dev.to

Permalink

Sept. and Oct. 2024 Long-Term Project Updates

Check out their latest project updates from our 2024 long-term developers! These reports just in for September and October. Thanks to all!

Long-Term Project Updates

Bozhidar Batsov: CIDER
Michiel Borkent: squint, babashka, neil, cherry, clj-kondo, and more
Toby Crawley: clojars-web
Thomas Heller: shadow-cljs, shadow-grove
Kira McLean: Scicloj Libraries. tcutils, Clojure Data Cookbook, and more
Nikita Prokopov: Humble UI, Datascript, AlleKinos, Clj-reload, and more
Tommi Reiman: Reitit 7.0. Malli, jsonista, and more
Peter Taoussanis: Carmine, Nippy, Telemere, and more

Bozhidar Batsov

The big news for the past couple of months was the release of CIDER 1.16 (“Kherson”).
It was a fairly small release, mainly focused on adding support for nREPL 1.3 (which brought a lot of internal improvements). The other interesting bit about CIDER 1.16 was switching to an “in-house” tracing back-end that resulted in slightly nicer tracing output. Check the release notes for more details.

Other interesting things that happened in the realm of CIDER & friends were:

• Orchard 0.28 brought improvements to the inspector and simplified the Java parser codebase
• We’ve fixed the font-locking of CIDER results in the minibuffer
• We’ve revived the cider completion style (details here)
• You can now display the available log frameworks with cider-log-show-frameworks
• A few bugs were fixed in clojure-ts-mode and we’ve started to port clojure-mode’s tests over there

In other news - I turned 40 in October, which means I’ve spent 30% of my life working on CIDER & co! :D As usual - huge thanks for supporting my work!

Michiel Borkent

Updates In this post I’ll give updates about open source I worked on during September and October 2024. To see previous OSS updates, go here.

Sponsors

I’d like to thank all the sponsors and contributors that make this work possible. Without you, the below projects would not be as mature or wouldn’t exist or be maintained at all.

Current top tier sponsors:

• Clojurists Together
• Roam Research
• Nextjournal
• Nubank

Sponsor info

If you want to ensure that the projects I work on are sustainably maintained, you can sponsor this work in the following ways. Thank you!

• Github Sponsors
• The Babaska or Clj-kondo OpenCollective
• Ko-fi
• Patreon
• Clojurists Together

If you’re used to sponsoring through some other means which aren’t listed above, please get in touch. On to the projects that I’ve been working on!

Updates

In September I visited Heart of Clojure where Christian, Teodor and I did a workshop on babashka. The first workshop was soon fully booked so we even did a second one and had a lot of fun doing so. It was so good to see familiar Clojure faces in real life again. Thanks Arne and Gaiwan team for organizing this amazing conference.

Although I didn’t make it to the USA for the Clojure conj in October, Alex Miller did invite me to appear towards the end of his closing talk when he mentioned that 90% of survey respondents used babashka.

If you are interested in a full stack web framework with babashka and squint, check out borkweb.

Here are updates about the projects/libraries I’ve worked on in the last two months.

• clj-kondo: static analyzer and linter for Clojure code that sparks joy. \

  • Unreleased
  • #1784: detect :redundant-do in catch
  • #2410: add --report-level flag
  • 2024.09.27
  • #2404: fix regression with metadata on node in hook caused by :redundant-ignore linter
  • 2024.09.26
  • #2366: new linter: :redundant-ignore. See docs
  • #2386: fix regression introduced in #2364 in letfn
  • #2389: add new hooks-api/callstack function
  • #2392: don’t skip jars that were analyzed with --skip-lint
  • #2395: enum constant call warnings
  • #2400: deftype and defrecord constructors can be used with Type/new
  • #2394: add :sort option to :unsorted-required-namespaces linter to enable case-sensitive sort to match other tools
  • #2384: recognize gen/fmap var in cljs.spec.gen.alpha

• babashka: native, fast starting Clojure interpreter for scripting.

  • #1752: include java.lang.SecurityException for java.net.http.HttpClient support
  • #1748: add clojure.core/ensure
  • Upgrade to taoensso/timbre v6.6.0
  • Upgrade to GraalVM 23
  • #1743: fix new fully qualified instance method in call position with GraalVM 23
  • Clojure 1.12 interop: method thunks, FI coercion, array notation (see below)
  • Upgrade SCI reflector based on clojure 1.12 and remove specific workaround for Thread/sleep interop
  • Add tools.reader.edn/read
  • Fix #1741: (taoensso.timbre/spy) now relies on macros from taoensso.encore previously not available in bb
  • Upgrade Clojure to 1.12.0
  • #1722: add new clojure 1.12 vars
  • #1720: include new clojure 1.12’s clojure.java.process
  • #1719: add new clojure 1.12 clojure.repl.deps namespace. Only calls with explicit versions are supported.
  • #1598: use Rosetta on CircleCI to build x64 images
  • #1716: expose babashka.http-client.interceptors namespace
  • #1707: support aset on primitive array
  • #1676: restore compatibility with newest at-at version (1.3.58)
  • Bump SCI
  • Bump fs
  • Bump process
  • Bump deps.clj
  • Bump http-client
  • Bump clj-yaml
  • Bump edamame
  • Bump rewrite-clj
  • Add java.io.LineNumberReader

• SCI: Configurable Clojure/Script interpreter suitable for scripting and Clojure DSLs

  • Fix #942: improve error location of invalid destructuring
  • Fix #917: support new Clojure 1.12 Java interop: String/new, String/.length and Integer/parseInt as fns
  • Fix #925: support new Clojure 1.12 array notation: String/1, byte/2
  • Fix #926: Support add-watch on vars in CLJS
  • Support aset on primitive array using reflection
  • Fix #928: record constructor supports optional meta + ext map
  • Fix #934: :allow may contain namespaced symbols
  • Fix #937: throw when copying non-existent namespace
  • Update sci.impl.Reflector (used for implementing JVM interop) to match Clojure 1.12

• squint: CLJS syntax to JS compiler

  • Fix watcher and compiler not overriding squint.edn configurations with command line options.
  • Allow passing --extension and --paths via CLI
  • Fix #563: prioritize refer over core built-in
  • Update chokidar to v4 which reduces the number of dependencies
  • BREAKING: Dynamic CSS in #html must now be explicitly passed as map literal: (let [m {:color :green}] #html [:div {:style {:& m}}]). Fixes issue when using lit-html in combination with classMap. See demo
  • #556: fix referring to var in other namespace via global object in REPL mode
  • Pass --repl opts to watch subcommand in CLI
  • #552: fix REPL output with hyphen in ns name
  • Ongoing work on browser REPL. Stay tuned.

• cherry: Experimental ClojureScript to ES6 module compiler

  • Fix referring to vars in other namespaces globally
  • Allow defclass to be referenced through other macros, e.g. as cherry.core/defclass
  • Fix emitting keyword in HTML
  • #138: Support #html literals, ported from squint

• http-client: babashka’s http-client \

  • #68 Fix accidental URI path decoding in uri-with-query (@hxtmdev)
  • #71: Link back to sources in release artifact (@lread)
  • #73: Allow implicit ports when specifying the URL as a map (@lvh)

• http-server: serve static assets

  • #16: support range requests (jmglov)
  • #13: add an ending slash to the dir link, and don’t encode the slashes (@KDr2)
  • #12: Add headers to index page (rather than just file responses)

• bbin: Install any Babashka script or project with one command \

  • Fix #88: bbin ls with 0-length files doesn’t crash

• scittle: Execute Clojure(Script) directly from browser script tags via SCI

  • Add cljs.pprint/code-dispatch and cljs.pprint/with-pprint-dispatch

• clojurescript

  • Throw when calling ana-api/ns-publics on non-existing ns

• neil: A CLI to add common aliases and features to deps.edn-based projects. \

  • #241: ignore missing deps file (instead of throwing) in neil new (@bobisageek)

• sci.configs: A collection of ready to be used SCI configs.

  • Added a configuration for cljs.spec.alpha and related namespaces

• nbb: Scripting in Clojure on Node.js using SCI

  • Include cljs.spec.alpha, cljs.spec.gen.alpha, cljs.spec.test.alpha

• qualify-methods

  • Initial release of experimental tool to rewrite instance calls to use fully qualified methods (Clojure 1.12 only0

• clerk: Moldable Live Programming for Clojure

  • Add support for :require-cljs which allows you to use .cljs files for render functions
  • Add support for nREPL for developing render functions

• deps.clj: A faithful port of the clojure CLI bash script to Clojure

  • Upgrade/sync with clojure CLI v1.12.0.1479

• process: Clojure library for shelling out / spawning sub-processes

  • Work has started to support prepending output (in support for babashka parallel tasks). Stay tuned.

Other projects

There are many other projects I’m involved with but that had little to no activity in the past month. Check out the Other Projects section (more details) of my blog here to see a full list.
Published: 2024-11-01

Toby Crawley

October 2024

Commit Logs: clojars-web, infrastructure

• Increased max upload size from 20MB to 30MB This is to allow larger artifacts to be uploaded, now that we see native builds more often.
• Addressed CVEs

September 2024

Commit Logs: clojars-web, infrastructure

• Upgraded to clojure 1.12.0
• Upgraded to java 21
• Added hcaptcha to signup form
• Fixed status-message sending on validation failure
  

Thomas Heller

shadow-cljs Update

Time was mostly spent on doing maintenance work and some bugfixes. As well as helping people out via the typical channels (eg. Clojurians Slack).

Current shadow-cljs version: 2.28.18 Changelog

Notable Updates

• Added support for package.json "imports"

Blog

Wrote 2 new blog posts describing my personal REPL-based Workflow using shadow-cljs

• Fullstack Workflow with shadow-cljs
• Supercharging the REPL Workflow
  

Kira McLean

This is a summary of the open source work I spent my time on throughout September and October 2024. This was a very busy period in my personal life and I didn’t make much progress on my projects, but I did have more time than usual to think about things, which prompted many further thoughts. Keep reading for details :)

Sponsors

I always start these posts with a sincere thank you to the generous ongoing support of my sponsors that make this work possible. I can’t say how much I appreciate all of the support the community has given to my work and would like to give a special thanks to Clojurists Together and Nubank for providing incredibly generous grants that allowed me reduce my client work significantly and afford to spend more time on projects for the Clojure ecosystem for nearly a year.

If you find my work valuable, please share it with others and consider supporting it financially. There are details about how to do that on my GitHub sponsors page. On to the updates!

Personal update

I’ll save the long version for the end but there is one important personal update that’s worth mentioning up front: I go by Kira Howe now. I used be known as Kira McLean, and all of my talks, writing, and commits up to this point use Kira McLean, but I’m still the same person! Just with a new name. I even updated my GitHub handle, which went remarkably smoothly.

Conj 2024

The main Clojure-related thing I did during this period was attend the Conj. It’s always cool to meet people in person who you’ve only ever worked with online, and I finally got to meet so many of the wonderful people from Clojure Camp and Scicloj who I’ve had the pleasure of working with virtually. I also had the chance to meet some of my new co-workers, which was great. There were tons of amazing talks and as always insightful and inspiring conversations. I always leave conferences with tons of energy and ideas. Then get back to reality and realize there’s no time to implement them all :) But still, below are some of the main ideas I’m working through after a wonderful conference.

SVGs for visualizing graphics

Tim Pratley and Chris Houser gave a fun talk about SVGs, among other things, that made me realize using SVGs might be the perfect way to implement the “graphics” side of a grammar of graphics.

Some of you may be following the development of tableplot (formerly hanamicloth), in which Daniel Slutsky has been implementing an elegant, layered, grammar-of-graphics-inspired way to describe graphics in Clojure. This library takes this description of a graphic and translates it into a specification for one of the supported underlying Javascript visualization libraries (currently vega-lite or plotly, via hanami). Another way to think about it is as the “grammar” part of a grammar of graphics; a way to declaratively transform an arbitrary dataset into a standardized set of instructions that a generic visualization library can turn into a graphic. This is the first half of what we need for a pure Clojure implementation of a grammar of graphics.

The second key piece we need is a Clojure implementation of the actual graphics rendering. Whether we adopt a similar underlying representation for the data as vega-lite, plotly, or whatever else is less consequential at this stage. Currently we just “translate” our Clojure code into vega-lite or plotly specs and call it a day. What I want to implement is a Clojure library that can take some data and turn it into a visualization. There are many ways to implement such a thing, all with different trade-offs, but Tim and Chouser’s talk made me realize SVGs might be a great tool for the job. They’re fast, efficient, simple to style and edit, plus they offer potentially the most promising avenues toward making graphics accessible and interactive since they’re really just XML, which is semantic, supports ARIA labels, and is easy to work with in JS.

Humble UI also came up in a few conversations, which is a totally tangential concern, but it was interesting to start thinking about how all of this could come together into a really elegant, fully Clojure-based data visualization tool for people who don’t write code.

A Clojurey way of working with data

I also had a super interesting conversation on my last night in Alexandria about Clojure’s position in the broader data science ecosystem. It’s fair to say that we have more or less achieved feature parity now for all the essential things a person working with data would need to do. Work is ongoing organizing these tools into a coherent and accessible stack (see noj), but the pieces are all there.

The main insight I left with, though, was that we shouldn’t be aiming for mere feature parity. It’s important, but if you’re a working data scientist doing everything you already do just with Clojure is only a very marginal improvement and presents a very high switching cost for potentially not enough payoff. In short, it’s a tough sell to someone who’s doesn’t already have some prior reason to prefer Clojure.

What we should do is leverage Clojure’s strengths to build tools that could leapfrog the existing solutions, rather than just providing better implementations of them. I.e. think about new ways to solve the fundamental problems in data science, rather than just offering better tools to work within the current dominant paradigm.

For example, a fundamental problem in science is reproducibility. The current ways data is prepared and managed in most data (and regular) science workflows is madness, and versioning is virtually non-existent. If you pick up any given scientific paper that does some sort of data analysis, the chances that you will be able to reproduce the results are near zero, let alone using the same tools the author used. If you do manage to, you will have had to use a different implementation than the authors, re-inventing wheels and reverse-engineering their thought process. The problem isn’t that scientists are bad at working with data, it’s the fundamental chaos of the underlying ecosystem that’s impossible to fight.

If you’ve ever worked with Python code, you know that dependency management is a nightmare, never mind state management within a single program. Stateful objects are just a bad mental model for computing because they require us to hold more information in our heads in order to reason about a system than our brains can handle. And when your mental model for a small amount of local data is a stateful, mutable thing, the natural inclination is to scale that mental model to your entire system. Tracking data provenance, versions, and lineage at scale is impossible when you’re thinking about your problem as one giant, mutable, interdependent pile of unorganized information.

Clojure allows for some really interesting ways of thinking about data that could offer novel solutions to problems like these, because we think of data as immutable and have the tools to make working with such data efficient. None of this is new. Somehow at this Conj between some really interesting talks focused on ways of working with immutable data and subsequent conversations it clicked for me, though. If we apply the same ways we think about data in the small, like in a given program, more broadly to an entire system or workflow, I think the benefits could be huge. It’s basically implementing the ideas from Rich Hickey’s “Value of values” talk over 10 years ago to a modern data science workflow.

Other problems that Clojure is well-placed to support are:

• Scalability – Current dominant data science tools are slow and inefficient. People try to work around it by implementing libraries in C, Rust, Java, etc. and using them from e.g. Python, but this can only get you so far and adds even more brittleness and dependency management problems to the mix.
• Tracking data and model drift – This problem has a very similar underlying cause as the reproducibility issue, also fundamentally caused by a faulty mental model of data models as mutation machines.
• Testing and validation – Software engineering practices have not really permeated the data science community and as such most pipelines are fragile. Bringing a values-first and data-driven paradigm to pipeline development could make them much more robust and reliable.

Anyway I’m not exactly sure what any of this will look like as software yet, but I know it will be written in Clojure and I know it will be super cool. It’s what I’m thinking about and experimenting with now. And I think the key point that thinking about higher-level problems and how Clojure can be applied to them is the right path toward introducing Clojure into the broader data science ecosystem.

Software engineers as designers

Alex Miller’s keynote was all about designing software and how they applied a process similar to the one described in Rich Hickey’s keynote from last year’s conj to Clojure 1.12 (among other things). The main thing I took away from it was that the best use of an experienced software engineer’s time is not programming. I’ve had the good fortune of working with a lot of really productive teams over the years, and this talk made me realize that one thing the best ones all had in common is that at least a couple of people with a lot of experience were not in the weeds writing code all the time. Conversely a common thread between all of the worst teams I’ve been a part of is that team leads and managers were way too in the weeds, worrying too much about implementation details and not enough about what was being implemented.

I’ve come to believe that it’s not possible to reason about systems at both levels simultaneously. My brain at least just can’t handle both the intense attention to detail and very concrete, specific steps required to write software that actually works and the abstract, general conceptual type of thinking that’s required to build systems that work. The same person can do both things at different times, but not at the same time, and the cost of switching between both contexts is high.

Following the process described by Rich and then Alex is a really great way to add structure and coherence to what can otherwise come across as just “thinking”, but it requires that we admit that writing code is not always the best use of our time, which is a hard sell. I think if we let experienced software engineers spend more time thinking and less time coding we’d end up with much better software, but this requires the industry to find better ways to measure productivity.

Long version of personal updates

As most of you know or will have inferred by now, I got married in September! It was the best day ever and the subsequent vacation was wonderful, but it did more or less cancel my productivity for over a month. If you’re into weddings or just want a glimpse into my personal life, we had a reel made of our wedding day that’s available here on instagram via our wedding coordinator.

Immediately after I got back from my honeymoon I also started a new job at BroadPeak, which is going great so far, but also means I have far less time than I used for open source and community work. I’m back to strictly evening and weekend availability, and sadly (or happily, depending how you see it) I’m at a stage of my life where not all of that is free time I can spend programming anymore.

I appreciate everyone’s patience and understanding as I took these last couple of months to focus on life priorities outside of this work. I’m working on figuring out what my involvement in the community will look like going forward, but there are definitely tons of interesting things I want to work on. I’m looking forward to rounding out this year with some progress on at least some of them, but no doubt the end of December will come before I know it and there will be an infinite list of things left to do.

Thanks for reading all of this. As always, feel free to reach out anytime, and hope to see you around the Clojureverse :)

Nikita Prokopov

Hi, this is Niki Tonsky and past two month we made a huge progress on Humble UI, culminating in Heart of Clojure workshop. Clojure Sublimed also saw some love. Enjoy!

Humble UI, Clojure Desktop UI framework:

• Documentation is now part of Humble UI jar, can be opened from any app as a second window
• Lots of components were documented
• Declarative paints
• Support oklch color model
• More options for containers (column/row/grid)
• hsplit component
• Button styles
• Added user-facing ui/measure that only needs a component
• Tuned measurement/sizing caching, still a bit buggy
• Add bring-to-front to ui/window, cmd/ctrl+w to close docs, signal to ui namespace
• Convenience: default focusable on top level
• Convenience: allow ^{:stretch true} as a most common use case
• Fixed: with-bounds, draggable, vscrollable after resize,
• Fixed: crash on window close
• Demo: humble-file-picker demo as a separate application, with step-by-step progression
• Demo: new Sand sandbox
• Demo: new Mirrors demo, mirrored from Philippa Markovics

Clojure Sublimed, Clojure support for Sublime Text 4:

• Support Clojure 1.12 array type annotations
• Pretty print selection #123
• Toggle Comment command that uses #_ instead of ;;
• Execute code from inside top-level ; ... and #_... #124
• Handle eval of #_ forms in nREPL JVM
• Better handle socket close
• Highlight namespace name as entity.name, same as defs
• Simplified formatting rules: if list’s first form is a symbol, indent 2, in other cases, indent to opening paren
• Better handle selection after formatting with cljfmt
• Tests for indentation, cljfmt default config
• cljfmt correctly indents forms with custom rules
• Tuned color scheme a bit

Sublime Executor, executable runner for Sublime Text:

• Fixed result_file_regex/result_line_regex, clear them on clear_output

Clerk, Moldable Live Programming for Clojure:

• Better drop-in implementation of clojure.walk (should make it into a library)
  

Tommi Reiman

My planned Open Source Time got postponed due to small family crisis, so no big releases.

Had some time to help people, do PR reviews and small updates to viesti, to be released later this year or early next.

Reserved OS time for December, to (╯°□°)╯︵ LIBS out!

Something Else

Peter Taoussanis

A big thanks to Clojurists Together, Nubank, and other sponsors of my open source work! I realise that it’s a tough time for a lot of folks and businesses lately, and that sponsorships aren’t always easy 🙏

- Peter Taoussanis

Recent work

Hi folks! 👋 I’m a bit crunched for time atm, so will keep this update short for a change :-) Hope everyone’s well!

Telemere

Telemere is a modern rewrite of Timbre that offers an improved API to cover traditional logging, structured logging, tracing, basic performance measurement, and more.

v1.0.0-RC1 is now available! 🎉

It’s been a lot of work getting here, but I’m happy with the results. Big thanks to everyone that’s been testing the (many) betas and giving valuable feedback! 🙏

Have also recorded a new lightning intro video that gives a 7-min tour of what Telemere is and roughly what it can do. If you’ve been curious but short on time, this might be a good way to get started.

Timbre

Timbre is a pure Clojure/Script logging library.

v6.6.0 was released, and v6.6.1 shortly after.

These mark the stable final release of the previous release candidate. Main highlight is Timbre’s new out-the-box SLF4Jv2 support.

Nippy

Nippy is fast serialization library for Clojure.

v3.5.0-RC1 is out now, which updates dependencies and adds read support for some new primitive array types that’ll follow shortly in a v3.6.

Sente

Sente is a realtime web comms library for Clojure/Script

v1.20.0-RC1 is out now, and includes an experimental new Jetty adapter and a number of internal improvements.

Upcoming work

Can’t believe it’s almost the end of the year! It’s been a productive one, and I’d like to focus on wrapping up a few dangling ends before the year’s out.

Plans include:

• Telemere v1 final before year’s end.
• Tempel v1 final before year’s end.
• Tufte v3 pre-release (hopefully) before year’s end. Have actually already invested quite a bit of time in this - but it’s turned out to be a bigger job than expected, and I want to get it right.

Also hoping to find some time for some http-kit maintenance, and I have a couple talks I’d like to record at some point - though those’ll probably need to wait until next year.

On a background thread, I’m continuing to make progress on Carmine v4. Just recently deployed my first production code using v4’s alpha branch, so that’ll be a good opportunity for testing.

Thanks everyone! Cheers 👋

Permalink

Software Engineer at Scarlet

Permalink

Ring JDK Adapter

Ring JDK Adapter is a small wrapper on top of a built-in HTTP server available in Java. It’s like Jetty but has no dependencies. It’s almost as fast as Jetty, too (see benchmars below).

Why

Sometimes you want a local HTTP server in Clojure, e.g. for testing or mocking purposes. There is a number of adapters for Ring but all of them rely on third party servers like Jetty, Undertow, etc. Running them means to fetch plenty of dependencies. This is tolerable to some extent, yet sometimes you really want something quick and simple.

Since version 9 or 11 (I don’t remember for sure), Java ships its own HTTP server. The package name is com.sun.net.httpserver and the module name is jdk.httpserver. The library provides an adapter to serve Ring handlers. It’s completely free from any dependencies.

Ring JDK Adapter is a great choice for local HTTP stubs or mock services that mimic HTTP services. Despite some people think it’s for development purposes only, the server is pretty fast! One can use it even in production.

Availability

It’s worth mentioning that some Java installations may miss the jdk.httpserver module. Please ensure the JVM you’re using in production supports it first. Check out the following links:

• StackOverflow: Is package com.sun.net.httpserver standard?
• Java® Platform, Standard Edition & Java Development Kit Version 21 API Specification

Installation

;; lein
[com.github.igrishaev/ring-jdk-adapter "0.1.0"]

;; deps
com.github.igrishaev/ring-jdk-adapter {:mvn/version "0.1.0"}

Requires Java version at least 16, Clojure at least 1.8.0.

Quick Demo

Import the namespace, declare a Ring handler as usual:

(ns demo
  (:require
   [ring.adapter.jdk :as jdk]))

(defn handler [request]
  {:status 200
   :headers {"Content-Type" "text/plain"}
   :body "Hello world!"})

Pass it into the server function and check the http://127.0.0.1:8082 page in your browser:

(def server
  (jdk/server handler {:port 8082}))

The server function returns an instance of the Server class. To stop it, pass the result into the jdk/stop or jdk/close functions:

(jdk/stop server)

Since the Server class implements AutoCloseable interface, it’s compatible with the with-open macro:

(with-open [server (jdk/server handler opt?)]
  ...)

The server gets closed once you’ve exited the macro. Here is a similar with-server macro which acts the same:

(jdk/with-server [handler opt?]
  ...)

Parameters

The server function and the with-server macro accept the second optional map of the parameters:

Example:

(def server
  (jdk/server handler
              {:host "0.0.0.0" ;; listen all addresses
               :port 8800      ;; a custom port
               :threads 8      ;; use custom fixed trhead executor
               :root-path "/my/app"}))

When run, the handler above is be available by the address http://127.0.0.1:8800/my/app in the browser.

Body Type

JDK adapter supports the following response :body types:

• java.lang.String
• java.io.InputStream
• java.io.File
• java.lang.Iterable<?> (see below)
• null (nothing gets sent)

When the body is Iterable (might be a lazy seq as well), every item is sent as a string in UTF-8 encoding. Null values are skipped.

Middleware

To gain all the power of Ring (parsed parameters, JSON, sessions, etc), wrap your handler with the standard middleware:

(ns demo
  (:require
    [ring.middleware.params :refer [wrap-params]]
    [ring.middleware.keyword-params :refer [wrap-keyword-params]]
    [ring.middleware.multipart-params :refer [wrap-multipart-params]]))

(let [handler (-> handler
                  wrap-keyword-params
                  wrap-params
                  wrap-multipart-params)]
  (jdk/server handler {:port 8082}))

The wrapped handler will receive a request map with parsed :query-params, :form-params, and :params fields. These middleware come from the ring-core library which you need to add into your dependencies. The same applies to handling JSON and the ring-json library.

Exception Handling

If something gets wrong while handling a request, you’ll get a plain text page with a short message and a stack trace:

(defn handler [request]
  (/ 0 0) ;; !
  {:status 200
   :headers {"Content-Type" "text/plain"}
   :body "hello"})

This is what you’ll get in the browser:

failed to execute ring handler
java.lang.ArithmeticException: Divide by zero
	at clojure.lang.Numbers.divide(Numbers.java:190)
	at clojure.lang.Numbers.divide(Numbers.java:3911)
	at bench$handler.invokeStatic(form-init14855917186251843338.clj:8)
	at bench$handler.invoke(form-init14855917186251843338.clj:7)
	at ring.adapter.jdk.Handler.handle(Handler.java:112)
	at jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:98)
	at jdk.httpserver/sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:82)
	at jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:101)
	at jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:873)
	at jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:98)
	at jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:849)
	at jdk.httpserver/sun.net.httpserver.ServerImpl$DefaultExecutor.execute(ServerImpl.java:204)
	at jdk.httpserver/sun.net.httpserver.ServerImpl$Dispatcher.handle(ServerImpl.java:567)
	at jdk.httpserver/sun.net.httpserver.ServerImpl$Dispatcher.run(ServerImpl.java:532)
	at java.base/java.lang.Thread.run(Thread.java:1575)

To prevent this data from being leaked to the client, use your own wrap-exception middleware, something like this:

(defn wrap-exception [handler]
  (fn [request]
    (try
      (handler request)
      (catch Exception e
        (log/errorf e ...)
        {:status 500
         :headers {...}
         :body "No cigar! Roll again!"}))))

Benchmarks

As mentioned above, the JDK server although though is for dev purposes only, is not so bad! The chart below proves it’s almost as fast as Jetty. There are five attempts of ab -l -n 1000 -c 50 ... made against both Jetty and JDK servers (1000 requests in total, 50 parallel). The levels of RPS are pretty equal: about 12-13K requests per second.

Measured on Macbook M3 Pro 32Gb, default settings, the same REPL.

Permalink

Clojure Deref (Nov 15, 2024)

Permalink

Scheming About Clojure

Clojure is a LISP for the Java Virtual Machine (JVM). As a schemer, I wondered if I should give Clojure a go professionally. After all, I enjoy Rich Hickey's talks and even Uncle Bob is a Clojure fan. So I considered strength and weaknesses from my point of view:

Pros

• S-Expressions
• Makes functional programming easy
• Schemy naming with ? and ! suffixes
• Integrated testing framework
• Platform independence due to JVM
• Simple Java interoperability
• Clojure map type corresponds to JSON
• Web-server abstraction with extensions (Ring)
• Dedicated Ubuntu-based Docker image

Cons

• Too many core functions
• Too many concurrency concepts
• Having collection functions and the sequence API is confusing
• Keywords feels unnecessary, given symbols
• Unwieldy default project structure
• Leiningen feels forced upon you
• Clojure is not just a single jar (anymore)
• No integrated JSON parser

Insight

Clojure seems good enough. It is not flawless and somewhat overloaded, but far, far ahead of Javascript, Python, Go, or Rust. Of course, I would always prefer CHICKEN Scheme for any passion project. But in an environment that already runs databases written in Java, the JVM has street cred, and a large community hints at sustainability, Clojure presents itself as well balanced in novelty and stability. All in all, Clojure seems to be the enterprise Lisp.

References

• https://clojure.org/reference/lisps
• https://www.more-magic.net/posts/thoughts-on-clojure.html
• The Clojure Workshop (Book)

Permalink

Project Euler Problem 9

Code

;; euler_9.clj

(def possibilities
  (for [a (range 1 1001) b (range (+ a 1) 1001)]
    [a b]))

(defn satisfies-condition [possibility]
  (let [a (first possibility)
        b (second possibility)
        c (- 1000 (+  a b))]
    (== (+ (* a a) (* b b)) (* c c))))

;; (filter satisfies-condition possibilities)

(let
 [[a b] (first (filter satisfies-condition possibilities))
  c (- 1000 (+ a b))]
  (* a b c))

Notes

• Project Euler Problem 9

Permalink

What the Reagent Component?!

Did you know that when you write a form-1, form-2 or form-3 Reagent component they all default to becoming React class components?

For example, if you were to write this form-1 Reagent component:

(defn welcome []
  [:h1 "Hello, friend"])

By the time Reagent passes it to React it would be the equivalent of you writing this:

class Welcome extends React.Component {
  render() {
    return <h1>Hello, friend</h1>
  }
}

Okay, so, Reagent components become React Class Components. Why do we care? This depth of understanding is valuable because it means we can better understand:

• JavaScript, ES6 classes and the idea behind "syntax sugar"
• React's strategy for distinguishing class and function components
• How ClojureScript interacts with JavaScript

The result of all of this "fundamental" learning is we can more effectively harness JavaScript from within ClojureScript.

A Pseudoclassical Pattern

The reason all of your Reagent components become class components is because all of the code you pass to Reagent is run through an internal Reagent function called create-class.

create-class is interesting because of how it uses JavaScript to transform a Reagent component into something that is recognized as a React class component. Before we look into what create-class is doing, it's helpful to review how "classes" work in JavaScript.

Prior to ES6, JavaScript did not have classes. and this made some JS developers sad because classes are a common pattern used to structure code and provide support for:

• instantiation
• inheritance
• polymorphism

But as I said, prior to ES6, JavaScript didn't have a formal syntax for "classes". To compensate for the lack of classes, the JavaScript community got creative and developed a series of instantiation patterns to help simulate classes.

Of all of these patterns, the pseudoclassical instantiation pattern became one of the most popular ways to simulate a class in JavaScript. This is evidenced by the fact that many of the "first generation" JavaScript libraries and frameworks, like google closure library and backbone, are written in this style.

The reason we are going over this history is because the thing about a programming language is there are "patterns" and "syntax". The challenge with "patterns" is:

• They're disseminated culturally (tribal knowledge)
• They're difficult to identify
• They're often difficult to search
• They often require a deeper knowledge to understand how and why to use a pattern.

The last point in praticular is relevant to our conversation because patterns live in a context and assume prior knowledge. Knowledge like how well we know the context of a problem, the alternative approaches to addressing a problem, advancements in a language and so on.

The end result is that a pattern can just become a thing we do. We can forget or never know why it started in the first place or what the world could look like if we chose a different path.

For example, the most common way of writing a React class component is to use ES6 class syntax. But did you know that ES6 class syntax is little more than syntactic sugar around the pseudoclassical instantiation pattern?

For example, you can write a valid React class component using the pseudoclassical instantiation pattern like this:

// 1. define a function (component) called `Welcome`
function Welcome(props, context, updater) {
  React.Component.call(this, props, context, updater)

  return this
}

// 2. connect `Welcome` to the `React.Component` prototype
Welcome.prototype = Object.create(React.Component.prototype)

// 3. re-define the `constructor`
Object.defineProperty(Welcome.prototype, 'constructor', {
  enumerable: false,
  writable: true,
  configurable: true,
  value: Welcome,
})

// 4. define your React components `render` method
Welcome.prototype.render = function render() {
  return <h2>Hello, Reagent</h2>
}

While the above is a valid React Class Component, it's also verbose and error prone. For these reasons JavaScript introduced ES6 classes to the language:

class Welcome extends React.Component {
  render() {
    return <h1>Hello, Reagent</h1>
  }
}

For those looking for further evidence, we can support our claim that ES6 Classes result in same thing as what the pseudoclassical instantiation pattern produces by using JavaScript's built-in introspection tools to compare the pseudoclassical instantiation pattern to the ES6 class syntax.

pseudoclassical instantiation pattern:

function Welcome(props, context, updater) {
  React.Component.call(this, props, context, updater)

  return this
}

// ...repeat steps 2 - 4 from above before completing the rest

var welcome = new Welcome()

Welcome.prototype instanceof React.Component
// => true

Object.getPrototypeOf(Welcome.prototype) === React.Component.prototype
// => true

welcome instanceof React.Component
// => true

welcome instanceof Welcome
// => true

Object.getPrototypeOf(welcome) === Welcome.prototype
// => true

React.Component.prototype.isPrototypeOf(welcome)
// => true

Welcome.prototype.isPrototypeOf(welcome)
// => true

ES6 class

class Welcome extends React.Component {
  render() {
    console.log('ES6 Inheritance')
  }
}

var welcome = new Welcome()

Welcome.prototype instanceof React.Component
// => true

Object.getPrototypeOf(Welcome.prototype) === React.Component.prototype
// => true

welcome instanceof React.Component
// => true

welcome instanceof Welcome
// => true

Object.getPrototypeOf(welcome) === Welcome.prototype
// => true

React.Component.prototype.isPrototypeOf(welcome)
// => true

Welcome.prototype.isPrototypeOf(welcome)
// => true

What does all of this mean? As far as JavaScript and React are concerned, both definions of the Welcome component are valid React Class Components.

With this in mind, lets look at Reagent's create-class function and see what it does.

What Reagent Does

The history lesson from the above section is important because create-class uses a modified version of the pseudoclassical instantiation pattern. Let's take a look at what we mean.

The following code sample is a simplified version of Reagent's create-class function:

function cmp(props, context, updater) {
  React.Component.call(this, props, context, updater)

  return this
}

goog.extend(cmp.prototype, React.Component.prototype, classMethods)

goog.extend(cmp, React.Component, staticMethods)

cmp.prototype.constructor = cmp

What we have above is Reagents take on the pseudoclassical instantiation pattern with a few minor tweaks:

// 1. we copy to properties + methods of React.Component
goog.extend(cmp.prototype, React.Component.prototype, classMethods)

goog.extend(cmp, React.Component, staticMethods)

// 2. the constructor is not as "thorough"
cmp.prototype.constructor = cmp

Exploring point 1 we see that Reagent has opted to copy the properties and methods of React.Component directly to the Reagent compnents we write. That is what's happening here:

goog.extend(cmp.prototype, React.Component.prototype, classMethods)

If we were using the the traditional pseudoclassical approach we would instead do this:

cmp.prototype = Object.create(React.Component.prototype)

Thus, the difference is that Reagent's approach copies all the methods and properties from React.Component to the cmp prototype where as the second approach is going to link the cmp prototype to React.component prototype. The benefit of linking is that each time you instantiate a Welcome component, the Welcome component does not need to re-create all of the React.components methods and properties.

Exploring the second point, Reagent is doing this:

cmp.prototype.constructor = cmp

whereas with the traditional pseudoclassical approach we would instead do this:

Object.defineProperty(Welcome.prototype, 'constructor', {
  enumerable: false,
  writable: true,
  configurable: true,
  value: Welcome,
})

The difference in the above approaches is that if we just use = as we are doing in the Reagent version we create an enumerable constructor. This can have an implication depending on who consumes our classes, but in our case we know that only React is going to be consuming our class components, so we can do this with relative confidence.

What is one of the more interesting results of the above two Reagent modifications? First, if React depended on JavaScript introspection to tell whether or not a component is a child of React.Component we would not be happy campers:

Welcome.prototype instanceof React.Component
// => false...Welcome is not a child of React.Component

Object.getPrototypeOf(Welcome.prototype) === React.Component.prototype
// => false...React.component is not part of Welcomes prototype chain

welcome instanceof React.Component
// => false...Welcome is not an instance of React.Component

welcome instanceof Welcome
// => true...welcome is a child of Welcome

Object.getPrototypeOf(welcome) === Welcome.prototype
// => true...welcome is linked to Welcome prototype

console.log(React.Component.prototype.isPrototypeOf(welcome))
// => false...React.Component not linked to the prototype of React.Component

console.log(Welcome.prototype.isPrototypeOf(welcome))
// is Welcome is the ancestory?

What the above shows is that Welcome is not a child of React.component even though it has all the properties and methods that React.Component has. This is why were lucky that React is smart about detecting class vs. function components.

Second, by copying rather than linking prototypes we could inccur a performance cost. How much of a performance hit? In our case this cost is likely negligible.

Conclusion

In my experience, digging into the weeds and going on these detours has been an important part of my growth as a developer. The weeds have allowed me to be a better programmer because I'm honing my ability to understand challenging topics and find answers. The result is a strange feeling of calm and comfort.

This calm and comfort shouldn't be overlooked. So much of our day-to-day is left unquestioned and unanalyzed. We let knowledge become "cultural" or "tribal". This is scary. It's scary because it leads to bad decisions because no one around us knows the whys or wherefores. Ultimately, it's a bad habit. A bad habit which is seen by some as a virtue because it would simply take too much time for to learn things ourselves. That's until you actually start doing this kind of work and spend time learning and observing and seeing that these "new things" we're seeing all the time aren't really new, but just another example of that old thing back.

Permalink

What are the Clojure Tools?

The Clojure Tools are a group of convenience tools which currently consist of:

• Clojure CLI
• tools.build

The Clojure Tools. were designed to answer some of the following questions:

• How do I install Clojure? (Clojure CLI)
• How do I run a Clojure program? (Clojure CLI)
• How do I manage Clojure packages (dependencies)? (Clojure CLI)
• How do I configure a Clojure project? (Clojure CLI)
• How do I build Clojure for production? (tools.build)

The rest of this post will dig into each of these tools.

Clojure CLI

The Clojure CLI is a CLI program. Here is what it looks like to use the Clojure CLI and some of the things it can do:

Run a Clojure repl

clj

Run a Clojure program

clj -M -m your-clojure-program

manage Clojure dependencies

clj -Sdeps '{:deps {bidi/bidi {:mvn/version "2.1.6"}}}'

Like all Clojure programs, the Clojure CLI is built on a few libraries:

• clj/clojure - a bash script
• tools.deps a clojure library (commonly referred to as deps)
• deps.edn - an edn file with a specific structure

The following sections will provide overviews of each of the above tools.

The Clojure CLI is invoked by calling either clj or clojure shell commands:

# clj
clj -M -m your-clojure-program

# clojure
clojure -M -m your-clojure-program

Under the hood, clj actually calls clojure. The difference is that clj wraps the clojure command with a tool called rlwrap. rlwrap improves the developer experience by making it easier for you, a human, to type in the terminal while you're running your Clojure REPL. However, even though it's easier for you to type, rlwrap can make it hard to compose the clj command with other tools. As a result, it's a common practice to use clojure in production/ci environments . Additionally, not all environments have access to rlwrap so it's another dependency you have to install.

Okay, so they do the same thing. What do they do? clj/clojure has one job: run Clojure programs against a classpath.

The next sections will outline the tools that make up the Clojure CLI tool.

clj/clojure

If you dig into the clj/clojure is just a bash script which ultimatley calls a command like this:

java [java-opt*] -cp classpath clojure.main [init-opt*] [main-opt] [arg*]

Thus, the Clojure CLI tool makes it easier to run Clojure programs. It saves you having to type out a gnarly Java command and make it work on different environments (windows, linux, mac etc). However, it orchestrates the building of the classpath by calling out to tools.deps.

tools.deps

tools.deps is a Clojure libary responsible for managing your dependencies. It does the following things:

• reads in dependencies from a deps.edn file
• resolves the dependencies and their transitive dependencies
• builds a classpath

What's interesting about this program is that it's just a Clojure library. This means that you can use it outside of the Clojure CLI.

The other thing that makes tools.deps great is that it's a small and focused library. Why this is great is that if something goes wrong it's easy to read and learn the library in a short period of time.

deps.edn

deps.edn is just an edn file where you configure your project and specify project dependencies. You can think of it like Clojure's version of package.json. The deps.edn file is a Clojure map with a specific structure. Here's an example of some of the properties of a deps.edn file:

{:deps    {...}
 :paths   [...]
 :aliases {...}}

As you can see, we use the keywords :deps, :paths and :aliases and more to start to describe your project and the dependencies it requires.

As we noted above, deps.edn is read in when you run clj/clojure and tells clj/clojure which dependencies are requires to run your project.

Tools.Build

tools.build is a Clojure library with functions for building clojure projects. For example, build a jar or uberjar.

The way you would use tools.build is by writing a separate program inside your app which knows how to build your app. The convention is to create a build.clj file in the root of your project. Import tools.build and use the functions provides by tools.build to build your program.

The 3 main types of Clojure programs one might build into 3 sub categories:

• A tool
• A library
• An app

When you run your build.clj file, you will use Clojure CLI's -T switch. The -T switch is meant to run general clojure programs via the Clojure CLI and since build.clj is a separate program, distinct form the app you are writing, you would run it via the -T switch.

You would use -T for Clojure programs that you want to run as a "tool". For example, deps-new is a Clojure library which creates new Clojure projects based on a template you provide. This is a great example of a Clojure project which is built to be a "tool".

I don't want to go into more detail about -T now because that means we would have to dive into other Clojure CLI switches like -X and -M. That's for another post. On to the Installer!

Installer

The "Clojure CLI Installer" is a fancy way of referring to the brew tap used to install Clojure on mac and linux machines. As of February 2020, Clojure started maintaining their own brew tap. Thus, if you installed the Clojure CLI via

brew install clojure

you will likely want to uninstall clojure and install the following:

brew install clojure/tools/clojure

In all likelihood, you would probably be fine with brew install clojure as it will recieve updates. However, while brew install clojure will still see some love, it won't be as active as the clojure/tools/clojure tap.

clj v lein v boot

This section will provide a quick comparison of clj, lein and boot.

Firstly, all of the above tools are more or less addressing the same problems in their own way. Your job is to choose the one you like best.

If you're curious which to choose, my answer is the Clojure CLI. The reason I like the Clojure CLI is because the tool is simple. You can read through clj and tools.deps in an afternoon and understand what they are doing. The same (subjectively of course) cannot be said for lein or boot. I will note that Clojure CLI's API is not straightforward and can be confusing.

Secondly, the Clojure Tools promote libraries over frameworks. This is important when working with a language like Clojure because it really does reward you for breaking down your thinking.

Finally, the Clojure community is really leaning into building tools for Clojure CLI. For example, where lein used to have significantly more functionality, the community has built a ton of incredible tools that will cover many of your essential requirements.

Permalink

Project Euler Problem 8

Code

(def data "73167176531330624919225119674426574742355349194934
96983520312774506326239578318016984801869478851843
85861560789112949495459501737958331952853208805511
12540698747158523863050715693290963295227443043557
66896648950445244523161731856403098711121722383113
62229893423380308135336276614282806444486645238749
30358907296290491560440772390713810515859307960866
70172427121883998797908792274921901699720888093776
65727333001053367881220235421809751254540594752243
52584907711670556013604839586446706324415722155397
53697817977846174064955149290862569321978468622482
83972241375657056057490261407972968652414535100474
82166370484403199890008895243450658541227588666881
16427171479924442928230863465674813919123162824586
17866458359124566529476545682848912883142607690042
24219022671055626321111109370544217506941658960408
07198403850962455444362981230987879927244284909188
84580156166097919133875499200524063689912560717606
05886116467109405077541002256983155200055935729725
71636269561882670428252483600823257530420752963450")

(def num-str (clojure.string/replace data #"\n" ""))

(def digit-chars (partition 13 1 num-str))

(defn product [chars]
  (reduce * (map #(read-string (str %)) chars)))

(defn product-and-chars [chars]
  [(product chars) chars])

(last (sort-by first (map product-and-chars digit-chars)))

Notes

Project Euler Problem 8

Permalink

Clojure/conj 2024

After last year's regular posts about my Clojurists Together-funded work on clojure-doc.org and other projects, and the end of my monorepo/polylith series, I've mostly taken a break from blogging -- and from my open source work, to be honest. I've been focusing on my day job and on some personal stuff.I attended Clojure/conj 2024 last month and wanted to write about the event and the talks I attended. It's been eleven years since Conj was last at the George Washington Masonic Memorial in Alexandria, and I'd forgotten what a climb it is, up that hill! My watch says I got over 9,000 steps each day going back and forth from the hotel to the venue.

Permalink

Scicloj scientific papers - initial planning meeting

Now that the Clojure stack for scientific computing is maturing rapidly, the Scicloj group is looking at ways of reaching other communities. One avenue would be to consider publishing academic papers about the ecosystem: regarding individual libraries or outlining the ecosystem as a whole. Another avenue, is that the Scicloj group would like to encourage and help out with research projects themselves. In the second half of November, we will have an initial planning meeting to discuss possible challenges and opportunities.

Permalink