Planet Clojure

Massively scalable collaborative text editor backend with Rama in 120 LOC

Operational transformations

There’s a few more intricacies to how operational transformations work, but this gets the gist of the idea across. Since this post is really about using Rama to handle the storage and computation for a real-time collaborative editor backend, I’ll keep it simple and only handle adds and removes. The code can easily be extended to handle other kinds of document edits such as formatting changes.

The Rama code will make use of two functions that encapsulate the “operational transformation” logic, one to apply a series of edits to a document and the other to transform an edit against a particular version against all the edits that happened until the latest version. These functions are as follows:

Backend storage

This PState is a map from a 64-bit document ID to the string contents of the document. The name of a PState always begins with $$ , and this is equivalent to a key/value database.

Here’s the PState tracking the history of all edits to a document:

Rama concepts

Implementing the backend

This declares a Rama module called “CollaborativeDocumentEditorModule” with a depot called *edit-depot which will receive all new edit information. Objects appended to a depot can be any type. The second argument of declaring the depot is called the “depot partitioner” – more on that later.

To keep the example simple, the data appended to the depot will be defrecord objects for the Clojure version and HashMap objects for the Java version. To have a tighter schema on depot records you could instead use Thrift, Protocol Buffers, or a language-native tool for defining the types. Here are the functions that will be used to create depot data:

This subscribes the topology to the depot *edit-depot and starts a reactive computation on it. Operations in dataflow do not return values. Instead, they emit values that are bound to new variables. In the Clojure API, the input and outputs to an operation are separated by the :> keyword. In the Java API, output variables are bound with the .out method.

Whenever data is appended to that depot, the data is emitted into the topology. The Java version binds the emit into the variable *edit and then gets the fields “id” and “version” from the map into the variables *id and *version , while the Clojure version captures the emit as the variable *edit and also destructures its fields into the variables *id and *version . All variables in Rama code begin with a * . The subsequent code runs for every single emit.

Remember that last argument to the depot declaration called the “depot partitioner”? That’s relevant here. Here’s that image of the physical layout of a module again:

The $$edits PState contains every edit applied to the document, and the latest version of a document is simply the size of that list. The PState is queried with the “local select” operation with a “path” specifying what to fetch. When a PState is referenced in dataflow code, it always references the partition of the PState that’s located on the task on which the event is currently running.

Paths are a deep topic, and the full documentation for them can be found here. A path is a sequence of “navigators” that specify how to hop through a data structure to target values of interest. A path can target any number of values, and they’re used for both transforms and queries. In this case, the path navigates by the key *id to the list of edits for the document. The next navigator view runs a function on that list to get its size. The Clojure version uses Clojure’s count function, and the Java version uses the Ops.SIZE function. The output of the query is bound to the variable *latest-version . This is a fast sub-millisecond query no matter how large the list of edits.

The next few lines run the “operational transformation” algorithm if necessary to produce the edits to be applied to the current version of the document:

First, an “if” is run to check if the version of the edit is the same as the latest version on the backend. If so, the list of edits to be applied to the document is just the edit unchanged. As mentioned before, the operational transformation algorithm can result in multiple edits being produced from a single edit. The Clojure version produces the single-element list by calling vector , and the Java version does so with the Ops.TUPLE function. The list of edits is bound to the variable *final-edits .

The “else” branch of the “if” handles the case where the edit must be transformed against all edits up to the latest version. The “local select” on $$edits fetches all edits from the input edit’s version up to the latest version. The navigator to select the sublist, srange in Clojure and sublist in Java, takes in as arguments a start offset (inclusive) and end offset (exclusive) and navigates to the sublist of all elements between those offsets. This sublist is bound to the variable *missed-edits .

The function shown before implementing operational transformations, transform-edit in Clojure and CollaborativeDocumentEditorModule::transformEdit in Java, is then run on the edit and all the missed edits to produce the new list of edits and bind them to the variable *final-edits .

Any variables bound in both the “then” and “else” branches of an “if” conditional in Rama will be in scope after the conditional. In this case, *final-edits is available after the conditional. *missed-edits is not available since it is not bound in the “then” branch. This behavior comes from Rama implicitly “unifying” the “then” and “else” branches.

The next bit of code gets the latest document and applies the transformed edits to it:

The “local select” fetches the latest version of the document from the $$docs PState. The second navigator in the path, nil->val in Clojure and nullToVal in Java, handles the case where this is the first ever edit on the document. In that case the document ID does not exist in this PState. In that case the key navigation by *id would navigate to null , so the next navigator instead navigates to the empty string in that case.

The next line runs the previously defined “apply edits” function to apply the transformed edits to produce the new version of the document into the variable *new-doc .

The next two lines finish this topology:

The two PStates are updated with the “local transform” operation. Like “local select”, a “local transform” takes in as input a PState and a “path”. Paths for “local transform” navigate to the values to change and then use special “term” navigators to update them.

The first “local transform” navigates into $$docs by the document ID and uses the “term val” navigator to set the value there to *new-doc . This is exactly the same as doing a “put” into a hash map.

The second “local transform” appends the transformed edits to the end of the list of edits for this document. It navigates to the list by the key *id and then navigates to the “end” of the list. More specifically, the “end” navigator navigates to the empty list right after the overall list. Setting that empty list to a new value appends those elements to the overall list, which is what the final “term val” navigator does.

This entire topology definition executes atomically – all the PState queries, operational transformational logic, and PState writes all happen together and nothing else can run on the task in between. This is a result of Rama colocating computation and storage, which will be explored more in the next section.

The power of colocation

Query topology to fetch document and version

The module needs one more small thing to complete the functionality necessary for a real-time collaborative editor backend. When the frontend is loaded, it needs to load the latest document contents along with its version. The contents are stored in the $$docs PStates, and the version is the size of the list of edits in the $$edits PState. So we need to read from both those PStates atomically in one event.

If you were to try to do this with direct PState clients, you would be issuing one query to the client for the $$edits PState and one query to the client for the $$docs PState. Those queries would run as separate events, and the PStates could be updated in between the queries. This would result in the frontend having incorrect state.

This declares a query topology named “doc+version” that takes in one argument *id as input. It declares the return variable *ret , which will be bound by the end of the topology execution.

The next line gets the query to the task of the module containing the data for that ID:

The line does a “hash partition” by the value of *id . Partitioners relocate subsequent code to potentially a new task, and a hash partitioner works exactly like the aforementioned depot partitioner. The details of relocating computation, like serializing and deserializing any variables referenced after the partitioner, are handled automatically. The code is linear without any callback functions even though partitioners could be jumping around to different tasks on different nodes.

Interacting with the module

This uses the previously defined helper functions to create an add edit for document ID 123 at version 3, adding “to the ” to offset 5.

Here’s an example of querying the $$edits PState for a range of edits:

Workflow with frontend

Summary

Permalink

State of CIDER 2024 Survey Results

Back in 2019, I shared the first community survey results for CIDER, the beloved Clojure interactive development environment for Emacs. Five years later, we’re back to take the pulse of the ecosystem and see how things have evolved.

In this post, we’ll explore the results of the 2024 State of CIDER Survey, compare them to the 2019 edition, and reflect on the progress and the road ahead.

Who Are CIDER Users Today?

Experience With CIDER

In 2019, most users had between 1-5 years of experience with CIDER. Fast forward to 2024, and we now see a significant portion with 5+ years under their belts — a great sign of CIDER’s stability and staying power. Newcomers are still joining, but the user base has clearly matured.

I guess also has to do with the fact that Clojure is not growing as much as before and few new users are joining the Clojure community. For the record:

• 550 took part in CIDER’s survey in 2019
• 330 took part in 2024

I’ve observed drops in the rate of participation for State of Clojure surveys as well.

Prior Emacs Usage

The majority of respondents were Emacs users before trying CIDER in both surveys. A fun new entry in 2024: “It’s complicated” — we see you, Vim-converts and editor-hoppers!

Tools, Setups, and Installs

Emacs & CIDER Versions

Users have generally moved in sync with CIDER and Emacs development. CIDER 1.16 is now the dominant version, and Emacs 29/30 are common. This reflects good community alignment with upstream tooling.

Probably by the time I wrote this article most people are using CIDER 1.17 (and 1.18 is right around the corner). Those results embolden us to be slightly more aggressive with adopting newer Emacs features.

Installation Methods

Package.el remains the most popular method of installation, although straight.el has carved out a notable niche among the more config-savvy crowd. Nothing really shocking here.

Usage Patterns

Professional Use

Just like in 2019, around half of the respondents use CIDER professionally. The remaining half are hobbyists or open-source tinkerers — which is awesome.

Upgrade Habits

There’s a visible shift from “install once and forget” toward upgrading with major releases or as part of regular package updates. CIDER’s release cadence seems to be encouraging healthier upgrade practices.

Used Features

This was the biggest addition to the survey in 2024 and the most interesting to me. It confirmed my long-held suspicions that most people use only the most basic CIDER functionality. I can’t be sure why is that, but I have a couple of theories:

• Information discovery issues
• Some features are somewhat exotic and few people would benefit from them

I have plans to address both points with better docs, video tutorials and gradual removal of some features that add more complexity than value. CIDER 1.17 and 1.18 both make steps in this direction.

Community & Documentation

Documentation Satisfaction

Documentation continues to score well. Most users rate it 4 or 5 stars, though there’s always room for growth, especially in areas like onboarding and advanced features.

From my perspective the documentation can be improved a lot (e.g. it’s not very consistent, the structure is also suboptimal here and there), but that’s probably not a big issue for most people.

Learning Curve

The majority of users rate CIDER’s learning curve as moderate (3-4 out of 5), consistent with the complexity of Emacs itself. Emacs veterans may find it smoother, but newcomers still face a bit of a climb.

I keep advising people not to try to learn Emacs and Clojure at the same time!

Supporting CIDER

While more users and companies are aware of ways to support CIDER (like OpenCollective or GitHub Sponsors), actual support remains low. As always, a small donation or contribution can go a long way to sustaining projects like this. As a matter of fact the donations for CIDER and friends have dropped a lot of since 2022, which is quite disappointing given all the efforts me and other contributors have put into the project.

Conclusion

CIDER in 2024 is a mature, well-loved tool with a seasoned user base. Most users are professionals or long-time hobbyists, and satisfaction remains high. If you’re reading this and you’re new to CIDER — welcome! And if you’re a long-timer, thank you for helping build something great.

Thanks to everyone who participated in the 2024 survey. As always, feedback and contributions are welcome — and here’s to many more years of productive, joyful hacking with Emacs and CIDER.

Keep hacking!

Permalink

Clojure Is Awesome!!! [PART 17]

Adapting the DTO Pattern to Functional Bliss

Welcome back to Clojure Is Awesome! In Part 17, we’re crossing the bridge from the object-oriented world to functional territory by adapting the DTO (Data Transfer Object) pattern—a staple in Java/Spring Boot projects. DTOs are all about moving data between layers or systems, and while they’re typically tied to mutable classes in OOP, we’ll reimagine them in Clojure with immutability, specs, and pure functions.

What is the DTO Pattern, and Why Does It Matter?

In Java/Spring Boot, a Data Transfer Object (DTO) is a simple class designed to carry data between layers (e.g., from a database to a REST API) or across system boundaries. It’s not about business logic—just a lightweight container for structured data. A typical Java DTO might look like this:

public class UserDTO {
    private String id;
    private String name;
    private boolean active;

    // Getters, setters, constructors...
    public UserDTO(String id, String name, boolean active) {
        this.id = id;
        this.name = name;
        this.active = active;
    }
}

Why DTOs exist:

• Decoupling: They separate internal models (e.g., database entities) from external representations (e.g., API responses).
• Efficiency: They expose only what’s needed, trimming unnecessary fields.
• Safety: They enforce a clear contract for data exchange.

In Clojure, we don’t have classes or mutable state—so how do we adapt this? The answer lies in immutable maps, clojure.spec for validation, and pure functions for transformation. Let’s build a functional DTO that keeps the spirit alive!

Crafting a Functional DTO in Clojure

In Clojure, a DTO can be a plain map with a well-defined shape, validated by specs, and paired with transformation functions. We’ll create a UserDTO equivalent, along with utilities to convert between internal data and DTOs—mimicking a real-world scenario like a REST API response.

Here’s the implementation:

(ns clojure-is-awesome.dto
  (:require [clojure.spec.alpha :as s]
            [clojure.pprint :as pp]))

(s/def ::id string?)
(s/def ::name string?)
(s/def ::active boolean?)
(s/def ::user-dto (s/keys :req-un [::id ::name ::active]))

(s/def ::created-at inst?)
(s/def ::user-entity (s/keys :req-un [::id ::name ::active ::created-at]))

(defn entity->dto
  "Converts an internal user entity to a DTO.
   Args:
     entity - A map representing the internal user entity.
   Returns:
     A map conforming to ::user-dto, or throws an exception if invalid."
  [entity]
  (if (s/valid? ::user-entity entity)
    (select-keys entity [:id :name :active])
    (throw (ex-info "Invalid user entity" {:problems (s/explain-data ::user-entity entity)}))))

(defn dto->entity
  "Converts a DTO to an internal user entity, adding default fields.
   Args:
     dto - A map conforming to ::user-dto.
   Returns:
     A map conforming to ::user-entity, or throws an exception if invalid."
  [dto]
  (if (s/valid? ::user-dto dto)
    (assoc dto :created-at (java.util.Date.))
    (throw (ex-info "Invalid DTO" {:problems (s/explain-data ::user-dto dto)}))))

(defn valid-dto?
  "Checks if a map is a valid user DTO.
   Args:
     dto - A map to validate.
   Returns:
     Boolean indicating validity."
  [dto]
  (s/valid? ::user-dto dto))

Testing with Pretty-Printed Output

Let’s simulate a round-trip from entity to DTO and back, with pprint for clarity:

(def user-entity {:id "u123"
                  :name "Borba"
                  :active true
                  :created-at (java.util.Date.)})

(def user-dto (entity->dto user-entity))
(println "User DTO:")
(pp/pprint user-dto)

(println "Valid DTO?" (valid-dto? user-dto))

(def new-entity (dto->entity user-dto))
(println "New Entity:")
(pp/pprint new-entity)

(try
  (dto->entity {:id "u124" :name "Borba"})
  (catch Exception e
    (println "Error:")
    (pp/pprint (ex-data e))))

Practical Use Case: REST API Response

Imagine a web service returning user data. We’ll convert an internal entity to a DTO for the API response:

(defn fetch-user-dto
  "Simulates fetching a user and returning its DTO representation.
   Args:
     user-id - The ID of the user to fetch.
   Returns:
     A user DTO map."
  [user-id]
  (let [mock-db {:u123 {:id "u123"
                        :name "Borba"
                        :active true
                        :created-at (java.util.Date.)}}]
    (entity->dto (get mock-db user-id))))

(def api-response (fetch-user-dto :u123))
(println "API Response:")
(pp/pprint api-response)

(def incoming-dto {:id "u124" :name "Bob" :active false})
(def stored-entity (dto->entity incoming-dto))
(println "Stored Entity:")
(pp/pprint stored-entity)

Why This Matters in Clojure

Our functional DTO brings some serious advantages over its Java cousin:

• Immutability: No setters, no surprises—data stays consistent.
• Validation: Specs enforce structure upfront, catching errors early.
• Simplicity: Maps and functions replace boilerplate classes and frameworks.
• Flexibility: Transformation functions can evolve without breaking the contract.

Unlike Java DTOs, we don’t need Lombok or Jackson for serialization—clojure’s data structures are naturally serializable (e.g., to JSON). It’s leaner, yet just as powerful.

Permalink

Experience with Claude Code

I spent one week with Claude Code, vibe coding two apps in Clojure. Hours and $134.79 in. Let me tell you what I got out of that.

Claude Code is a new tool from Anthropic. You cd to the folder, run claude and a CLI app opens.

You tell it what to do and it starts working. Every time it runs a command, it lets you decide whether to do the task, or do something else. You can tell it to always be allowed to do certain tasks which is useful for things like moving in the directory, changing files, running tests.

Task 1: Rewrite app from Python to Clojure

We have a small standalone app here implemented in Python. It’s an app that serves as a hub for many ETL other jobs. Every job is run, the result is collected, packed as JSON and sent into the API for further processing.

So I copied:

• the app that should be converted from Python to Clj (basically job runner, data collector)
• source code of the API that receives data
• source code of workers that process received data

I explained exactly what I want to achieve and started iterating.

Also, I created a CLAUDE.md file which explained the purpose of the project, how to use Leiningen (yes, I still use Leiningen for new projects; I used Prismatic Schema in this project too). It also contained many additional instructions like adding schema checking, asserts, tests.

An excerpt from CLAUDE.md file.

## Generating data

Double check correctness of files you generate. I found you inserting 0 instead of } in JSON and “jobs:” string to beginning of lines in yml files, making them invalid.

The app is processing many Git repositories and most of the time is spent waiting for jobs to finish.

One innovation Claude Code did itself was creating 4 working copies for each repo and running 8 repositories in parallel. This means, the app was processing all work 32* times faster. I had to double check if it is really running my jobs because of how fast the result app was.

There were minor problems, like generating invalid JSON files for unit tests, or having problems processing outputs from commands and wrapping them in JSON.

I was about 6 hours in and 90% finished.

There were still some corner cases, where jobs got stuck. Or when jobs took much longer than they should. Or when some commits in repositories weren’t processed.

I instructed Claude Code to write more tests, testing scripts and iterate to resolve this. I wanted to make sure this will be a real drop in replacement for existing Python app that will take the same input config, will call API compatible, and will be just 30* faster.

This is where I ran into a problem. Claude Code worked for hours creating and improving scripts, adding more tests. For an app that’s maybe 1000 lines of code, I ended up with 2000 lines of tests and 2000 lines of various testing scripts.

At that time, I also ran into problems with bash. Claude Code generated code that required associative arrays.

I started digging, why does this app even need associative arrays in bash. To my big disappointment, I found that over 2 days, Claude Code copied more and more logic from Clojure to bash.

It got to the point where all logic was transported from Clojure to bash.

My app wasn’t running Clojure code anymore! Everything was done in bash. That’s why it all ended up with 2000 lines of shell code.

That was the first time in my life, when I told LLM I was disappointed by it. It quickly apologized and started moving back to Clojure.

Unfortunately, after a few more hours, I wasn’t able to really get all the tests passing and my test tasks running at the same time. I abandoned my effort for a while.

Task 2: CLI-based DAW in Clojure

When I was much younger, I used to make and DJ music. It all started with trackers in DOS. 

Most of the world was using FastTracker II, but my favourite one was Impulse Tracker.

Nowadays, I don’t have time to do as much music as in the past. But I got an idea. How hard might it be to build my own simple music making software.

Criteria I gave to Claude Code:

• Build it in Clojure
• Do not rewrite it in another language (guess how I got to this criteria?)
• Use Overtone as an underlying library
• Use CLI interface
• Do not use just 80×25 or any small subset. Use the whole allocated screen space.
• The control should be based on Spacemacs. So for example loading a sample would be [SPC]-[S]-[L].

Spacemacs [SPC]- based approach is awesome. Our HR software Frankie uses a Spacemacs-like approach too (adding new candidates with [SPC]-[C]-[A]). So it seems to me, it’s only natural to extend this approach to other areas.

Imagine, a music making software that is inspired by Spacemacs!

I called it spaceplay and let Claude Code cook.

After a while, I had a simple interface, session mode, samples, preferences, etc.

So now, I got to the point where I wanted to add tests and get the sound working.

One option was to hook directly on the interface. Just like every [SPC] command sequence dispatches an event, I could just simulate those presses and test, if the interface reacts.

The issue is, when you have a DAW, you have many things happening at the same time. I didn’t want to test only things that are related to the user doing things.

And at the same time, I wanted my DAW to be a live set environment, where the producer can make music. It is an environment with many moving parts. So it made sense to me to make a testing API. I exposed the internal state via API and let tests manipulate it.

This was a bit of the problem for Claude.

However, Claude Code is extremely focused on text input and output. There’s no easy way to explain how to attach to sound output and test it. Even overtone test folder doesn’t spend a lot of time doing this overtone/test/overtone at master · overtone/overtone.

I wanted to test Claude Code in vibe coding mode. I didn’t want to do a lot of code review, or fixing code for it, like I do with Cursor.

So we ended up in a cycle where I wanted it to get sound working & test it and it was failing at this task.

Conclusion

After 6000–7000 lines of code (most of it Clojure, some of it bash) generated, $134.79 spent, and 12 hours invested, I came to a conclusion.

Cursor is a much better workflow for me. AI without overview is still not there to deliver maintainable apps. All of those people who are vibecoding are going to throw those apps away later, or are going to regret it.

AI is absolutely perfect in laying out in the first 90%. Unfortunately, the second 90% it takes to finish the thing.

Apps, I have successfully finished with LLMs, were code reviewed by humans constantly, they were architected by human (me), they were deployed to production early and big parts were implemented, or refactored by hand.

I will be happy to try Claude Code again in 6 months from now. Until then, I got back to Cursor and Calva.

The post Experience with Claude Code appeared first on Flexiana.

Permalink

(Clojure Managing-User-Permissions)

Clojure practice task:

Managing User Permissions with Sets

You are developing a role-based access control system for a web application. Each user has a set of permissions (e.g., :read, :write, :delete). Your goal is to implement a permission management system using Clojure sets.

User {:user-name :email :permission}

The set of all available permissions in the system is :read :write :delete.

Tasks

Create a function to assign a set of permissions to a user.

(defn valid-user-permission? [permission]
  (subset? permission #{:read :write :delete}))

(defn valid-user? [user]
  (and (some? (:user-name user))
       (some? (:email user))))

(defn assign-user-permissions [user permissions]
  (when (and (valid-user? user) (valid-user-permission? permissions))
    (assoc user :permissions permissions)))

Write a function to check if a user has a specific permission.

(defn has-user-permission? [user permission]
  (when (and (valid-user? user) (valid-user-permission? #{permission}))
    (contains? (:permissions user) permission)))

Write a function to revoke a specific permission from a user.

(defn revoke-user-permissions [user permission]
  (when (and (valid-user? user) (valid-user-permission? #{permission}))
    (let [current-p (:permissions user)
          remove-p (disj current-p permission)]
      (assoc user :permissions remove-p)
      )))

Find users who have at least one common permission from a given permission set.

(defn find-users-common-permissions [users permissions]
  (filter #(subset? permissions (:permissions %)) users))

Find users who have all required permissions from a given permission set.

(defn find-users-all-permissions [users permissions]
  (filter #(= permissions (:permissions %)) users))

Compute the difference between two users permissions.

(defn diff-users-permissions [user-1 user-2]
  (difference (:permissions user-1) (:permissions user-2)))

Generate a report of all users and their permissions.
Example Alice -> #{:read :write}

(defn users-permissions-report [users]
  (map #(let [user (:user-name %)
              permissions (:permissions %) ]
          (cond
            (some? permissions) (str user " -> " permissions)
            :else (str user)
            ))
       users))

Permalink

Clojure macros continue to surprise me

Clojure macros have two modes: avoid them at all costs/do very basic stuff, or go absolutely crazy.

Here’s the problem: I’m working on Humble UI’s component library, and I wanted to document it. While at it, I figured it could serve as an integration test as well—since I showcase every possible option, why not test it at the same time?

This is what I came up with: I write component code, and in the application, I show a table with the running code on the left and the source on the right:

It was important that code that I show is exactly the same code that I run (otherwise it wouldn’t be a very good test). Like a quine: hey program! Show us your source code!

Simple with Clojure macros, right? Indeed:

(defmacro table [& examples]
  (list 'ui/grid {:cols 2}
    (for [[_ code] (partition 2 examples)]
      (list 'list
        code (pr-str code)))))

This macro accepts code AST and emits a pair of AST (basically a no-op) back and a string that we serialize that AST to.

This is what I consider to be a “normal” macro usage. Nothing fancy, just another day at the office.

Unfortunately, this approach reformats code: while in the macro, all we have is an already parsed AST (data structures only, no whitespaces) and we have to pretty-print it from scratch, adding indents and newlines.

I tried a couple of existing formatters (clojure.pprint, zprint, cljfmt) but wasn’t happy with any of them. The problem is tricky—sometimes a vector is just a vector, but sometimes it’s a UI component and shows the structure of the UI.

And then I realized that I was thinking inside the box all the time. We already have the perfect formatting—it’s in the source file!

So what if... No, no, it’s too brittle. We shouldn’t even think about it... But what if...

What if our macro read the source file?

Like, actually went to the file system, opened a file, and read its content? We already have the file name conveniently stored in *file*, and luckily Clojure keeps sources around.

So this is what I ended up with:

(defn slurp-source [file key]
  (let [content      (slurp (io/resource file))
        key-str      (pr-str key)
        idx          (str/index-of content key)
        content-tail (subs content (+ idx (count key-str)))
        reader       (clojure.lang.LineNumberingPushbackReader.
                       (java.io.StringReader.
                         content-tail))
        indent       (re-find #"\s+" content-tail)
        [_ form-str] (read+string reader)]
    (->> form-str
      str/split-lines
      (map #(if (str/starts-with? % indent)
              (subs % (count indent))
              %)))))

Go to a file. Find the string we are interested in. Read the first form after it as a string. Remove common indentation. Render. As a string.

Voilà!

I know it’s bad. I know you shouldn’t do it. I know. I know.

But still. Clojure is the most fun I have ever had with any language. It lets you play with code like never before. Do the craziest, stupidest things. Read the source file of the code you are evaluating? Fetch code from the internet and splice it into the currently running program?

In any other language, this would’ve been a project. You’d need a parser, a build step... Here—just ten lines of code, on vanilla language, no tooling or setup required.

Sometimes, a crazy thing is exactly what you need.

Permalink

Data analyis with Clojure - workshop, May 10th - initial survey

Following the maturing of the Noj toolkit for Clojure data science, we are planning a workshop for people who are curious to learn the Clojure language for data analysis. Please share this page broadly with your friends and groups who may be curious to learn Clojure at this occasion. The SciNoj Light conference schedule is emerging these days, with a fantastic set of talks. We want a broader audience to feel comfortable joining, and thus we wish to run a prep workshop one week earlier.

Permalink

Data analyis with Clojure - free workshop, May 10th - initial survey

Following the maturing of the Noj toolkit for Clojure data science, we are planning a free online workshop for people who are curious to learn the Clojure language for data analysis. Please share this page broadly with your friends and groups who may be curious to learn Clojure at this occasion. The SciNoj Light conference schedule is emerging these days, with a fantastic set of talks. We want a broader audience to feel comfortable joining, and thus we wish to run a prep workshop one week earlier.

Permalink

Refactoring Ring. Keep your handlers clean.

Notes

• Commit
• Injee

Permalink

Talk: Clojure workflow with Sublime Text @ SciCloj

A deep overview of Clojure Sublimed, Socket REPL, Sublime Executor, custom color scheme, clj-reload and Clojure+.
We discuss many usability choices, implementation details, and broader observations and insights regarding Clojure editors and tooling in general.

Permalink

Can jank beat Clojure's error reporting?

Hey folks! I&aposve spent the past quarter working on jank&aposs error messages. I&aposve focused on reaching parity with Clojure&aposs error reporting and improving upon it where possible. This has been my first quarter spent working on jank full-time and I&aposve been so excited to sit at my desk every morning and get hacking. Thank you to all of my sponsors and supporters! You help make this work possible.

Permalink

Clojure Deref (Mar 28, 2025)

Permalink

Using JS in ClojureScript Projects

The pull toward JavaScript has never been stronger. While ClojureScript remains an extremely expressive language, the JavaScript ecosystem continues to explode with tools like v0, Subframe & Paper generating entire UI trees and even full websites.

I found the feedback loop of these tools extremely quick and often use v0 to prototype specific components or interactions.

To benefit from these new tools and development experiences in an existing ClojureScript codebase you have two options:

1. Rewrite all the code to CLJS
2. Somehow use it as JS

In reality what I do is a bit of both. I mostly translate components to UIx but sometimes will use JavaScript utility files as is. This post is about that second part.

(I’ll probably write about the first part soon as well!)

The shadow-cljs JS import toolchain

shadow-cljs, the de facto frontend for the ClojureScript compiler, has built-in support for importing JavaScript .js files directly into your ClojureScript codebase.

Recently this was helpful when I wanted to add a custom d3-shape implementation to a codebase. I experimented in v0 until I had the desired result, leaving me with rounded_step.js:

// rounded_step.js
function RoundedStep(context, t, radius) {
  this._context = context;
  this._t = t;           // transition point (0 to 1)
  this._radius = radius; // corner radius
}

RoundedStep.prototype = {
  // ... implementation details full of mutable state
};

const roundedStep = function (context) {
  return new RoundedStep(context, 0.5, 5);
}

export { roundedStep };

Now this code would be kind of annoying (and not very valuable) to rewrite to ClojureScript. I tried briefly but eventually settled on just requiring the JS file directly:

(ns app.molecules.charts
  (:require
   [applied-science.js-interop :as j]
   [uix.core :as uix :refer [defui $]]
   ["/app/atoms/charts/rounded_step" :refer [roundedStep]]
   ["recharts" :as rc]))

Note the path /app/atoms/charts/rounded_step - shadow-cljs understands this refers to a JavaScript file in your source tree and will look for it in on the classpath.

Assuming you have :paths “src” then the file would be at src/app/atoms/charts/rounded_step.js.

When to use JavaScript directly

While I generally will still translate components to UIx (using these instructions) using plain JS can be nice in a few cases:

1. Code relying on mutability - some library APIs may expect it and it’s usually a bit annoying and perhaps even error prone to articulate in CLJS
2. Hard to translate syntax constructs - spreading operators, async/await, etc.
3. Performance - If you want to drop down a level to squeeze out higher performance

Limitations

1. To use JSX you’ll need to set up a preprocessor, something I didn’t want to get into. And for writing components UIx is nicer anyways.
2. “Leaf nodes” only, meaning you can’t require CLJS from JS and things like that. (Fine for my use cases.)

Making it work

Generally when dealing with JS libraries, the following has been helpful for my workflow:

1. Use js-interop libraries - applied-science/js-interop and cljs-bean make working with JavaScript objects more ergonomic
2. Use literal objects - The j/lit macro makes passing complex configuration objects cleaner

The payoff

The real benefit? You get to use the best of both worlds:

• ClojureScript's expressive syntax, immutable data structures and functional approach where and when you want it
• Plug in JavaScript snippets when it makes sense
• Less friction when adopting new JavaScript tools

Some folks will be arguing for pure ClojureScript solutions to everything. But in today's landscape, embracing JavaScript interop is the pragmatic choice.

After all, sometimes the best code is the code you don't have to write.

Permalink

Next-level backends with Rama: storing and traversing graphs in 60 LOC

Rama concepts

Materializing the PState

This declares a Rama module called “FamilyTreeModule” with a depot called *people-depot which will receive all new person information. Objects appended to a depot can be any type. The second argument of declaring the depot is called the “depot partitioner” – more on that later.

To keep the example simple, the data appended to the depot will be defrecord objects for the Clojure version and HashMap objects for the Java version. To have a tighter schema on depot records you could instead use Thrift, Protocol Buffers, or a language-native tool for defining the types. Here’s the function that will be used to create depot data:

This subscribes the topology to the depot *people-depot and starts a reactive computation on it. Operations in dataflow do not return values. Instead, they emit values that are bound to new variables. In the Clojure API, the input and outputs to an operation are separated by the :> keyword. In the Java API, output variables are bound with the .out method.

Whenever data is appended to that depot, the data is emitted into the topology into the variable *person . All variables in Rama code begin with a * . The subsequent code runs for every single emit.

Remember that last argument to the depot declaration called the “depot partitioner”? That’s relevant here. Here’s that image of the physical layout of a module again:

In the Clojure version, the ID for the person was destructured from the object into the variable *id on the source> line. In the Java version, the Ops.GET function is run on the *person map to fetch all the fields into variables of the same name.

The PState is updated with the “local transform” operation. The transform takes in as input the PState $$family-tree and a “path” specifying what to change about the PState. When a PState is referenced in dataflow code, it always references the partition of the PState that’s located on the task on which the event is currently running.

Paths are a deep topic, and the full documentation for them can be found here. A path is a sequence of “navigators” that specifies how to hop through a data structure to target values of interest. A path can target any number of values, and they’re used for both transforms and queries.

In the Clojure version, the path is very basic and navigates to exactly one value. It first navigates by the key in the variable *id which hops to the inner map for that ID. The “term val” navigator then says to replace whatever’s there with the provided value, which is just the person object from the depot with the “id” field removed.

In the Java version, the fields of the inner map are set by the path individually. It first navigates to the inner map by the key *id . Then multiPath does three separate navigations from that point for each of the fields, setting them to the values from the *person map.

The next part of the topology relocates computation to each parent in order to update their “children” field:

First, the two parents are put into a single list. This is done in the Clojure version by just putting *parent1 and *parent2 into a vector, and in the Java version it’s done with the call to Ops.TUPLE .

Then the “explode” operation is called on that list, which emits one time for each element of the list. In this case it emits twice, once for each parent, and the subsequent code runs for each parent. Putting the two parents into one list and then exploding it is an easy way to share the code for updating the two parents.

What’s happening here is very different from “regular” programming, where you call a function which returns one value as the result. In dataflow programming, operations can emit any number of times. You can think of the output of an operation as being the input arguments to the rest of the topology. When an operation emits, it invokes the “rest of the topology” with those arguments (the “rest of the topology” is also known as the “continuation”). This is a powerful generalization of the concept of a function, which as you’re about to see enables very elegant code.

The final line in this code does a “hash partition” by the value of *parent . Partitioners relocate subsequent code to potentially a new task, and a hash partitioner works exactly like the aforementioned depot partitioner. The details of relocating computation, like serializing and deserializing any variables referenced after the partitioner, are handled automatically. The code is linear without any callback functions even though partitioners could be jumping around to different tasks on different nodes.

The way the code is structured also causes the children for the two parents to be updated in parallel, which lowers the latency of the topology.

The final part of the topology updates the “children” field for each parent:

This is just like the previous transform, except it adds one element to the children set instead of replacing the entire inner map. It navigates first to the inner map for the *parent key and then to the “children” field within that map. The next navigator, called NONE-ELEM in Clojure and voidSetElem in Java, navigates to the “void” element of the set. Setting that “void” element to a value causes that value to be added to that set.

That completes everything involved in materializing that PState. In just 20 lines of code we’ve implemented the equivalent of a graph database, except tailored to match our use case exactly.

Rama’s dataflow API is as expressive as a full programming language with the additional power of making it easy to distribute computation. What you’ve seen in this section is just a small taste of what it can do.

Querying the PState directly

Let’s look at a few examples of querying the $$family-tree PState using this client. This query fetches the name for the person with the UUID “aa7d7c5c-0679-4b01-b40b-cb3c20065549”:

Implementing the first graph traversal query

This declares a query topology named “ancestors” that takes in input arguments *start-id and *num-generations . It declares the return variable *ancestors , which will be bound by the end of the topology execution.

The next line starts traversal of the family tree graph by initiating a loop:

Loops in dataflow first declare “loop variables” which are in scope for the body of the loop and are set to new values each time the loop is recurred. Here the two variables *id and *generation are declared, and they are initialized to *start-id and 0 for the first iteration of the loop. Loops can be emitted from any number of times (with :> in Clojure and .emitLoop in Java), and each emit runs the code after the loop with the emitted values. The Clojure version binds the emitted value from this loop as *ancestor in the binding vector, while that’s specified after the loop body in the Java version.

Then next line of code checks whether the query has traversed too far:

This operation, filter> in Clojure and .keepTrue in Java, emits exactly one time if its input is true and otherwise doesn’t emit. Just like how the “explode” call differs from regular functions by emitting many times, this differs from regular functions by potentially not emitting at all. No values are captured for the output since this operation doesn’t emit any values when it emits, but you can still think of its emit as invoking the “rest of the topology”.

The *generation variable keeps track of how many parents have been traversed, and this code stops execution of the loop iteration if *generation is greater than the *num-generations parameter from the invoke of the query.

The next lines fetch the two parents for node:

The hash partition, just like the ETL topology, moves the computation to the task containing information for that ID in the $$family-tree PState. The local select call selects both parents and emits them. The “multi path” navigator causes this local select to emit twice, once for each parent. The filter at the end of the path causes it not to emit parents that are null, which are nodes that have no parents specified.

The next line emits the found ancestor from the loop:

As mentioned before, this runs the code following the loop with that value bound to the variable *ancestor .

The next line invokes another iteration of the loop with that node:

This runs the loop from the start, setting the value of *id to *parent and *generation to one more than it was before.

Something very different from loops in languages like Java or Clojure is happening here. The loop is being continued multiple times in one iteration, once for each parent. Along with the hash partitioner, this is causing the loop to recur an ever increasing number of times in parallel across the cluster until iterations reach the generation limit and filter themselves out. This is a very elegant way to express a parallel traversal.

The next line is declared after the loop and runs for each emit from the loop:

Every query topology invocation has a temporary, in-memory PState it can use with the name of the query topology surrounded by $$ . In this case, that PState is called $$ancestors$$ . This code uses that temporary PState to record when it traverses a node with a set on each task and to skip traversal if it’s already seen it. Using the temporary PState like this is common in graph queries.

Let’s take a look at how to invoke this query topology from outside a module, like from a web server. First, you retrieve a client for the query topology just like how we retrieved a PState client earlier:

Implementing the second graph traversal query