Teaching LLMs to one-shot complex backends at scale, report #1

The LLM is not all that matters in AI coding. What the LLM is targeting matters a great deal. A simpler target that requires less reasoning will produce better results.

Attempts to get LLMs to produce complex backends have been lackluster. A recent paper, Constraint Decay: The Fragility of LLM Agents in Backend Code Generation, shows that even on a simple CRUD app, end-to-end success on the full test suite tops out at 33% once realistic structural constraints are imposed.

Conventional backends are made of many separate systems glued together, each with its own model and failure modes. Most of the failures observed in these benchmarks show up at the seams between these systems. The LLM is not asked to reason about one coherent system – it’s asked to coordinate across many.

Along those lines, we believe Rama is ideally positioned to take LLM coding to the next level for backends. Rama collapses the typical backend stack (databases, queues, stream processors, application logic) into one integrated system. The seams that current LLMs trip over largely don’t exist in a Rama application. A horizontally scalable, fault-tolerant backend is expressed as one coherent program rather than as glue across half a dozen systems.

In the past few months we’ve been working on a project to teach LLMs to one-shot complex backends at scale with Rama as the substrate. Our results so far are very promising, as I’ll review later in this post, but we have a ways to go. The major milestone we’re working towards is one-shotting the entire Matrix spec, which also has a thorough set of tests available that can be used to verify an implementation. What we’re looking to produce is:

  • A generated implementation of Matrix that passes all the reference tests
  • Transcript showing every step of how the LLM one-shotted the project
  • Benchmarks automatically written and executed by the LLM that demonstrate high performance and horizontal scalability

Matrix is orders of magnitude more difficult than the backends current LLMs can handle, particularly with these scalability and fault-tolerance requirements, so one-shotting it will be a huge milestone. However, the overarching goal is for this to work on any backend problem. We don’t expect what we’re building to one-shot every possible backend. Humans remain vastly better than LLMs at broad systems design where many tradeoffs must be considered. What we think is achievable, and what this project is targeting, is a workflow where humans assist with high-level design decisions and the agent handles lower-level decisions and implementation, including achieving horizontal scalability and fault-tolerance. By “fault tolerant” we mean the system continues operating correctly through infrastructure failures (e.g. node deaths) without data loss, data duplication, or downtime, and recovers automatically when failed components return.

Whether our goal is possible remains to be seen, but I’ll be documenting our progress as we go via these progress reports.

Our workflow

We work through the rama-ai-learn project, which we just open-sourced. It’s a benchmark and harness for measuring how well LLMs can produce production Rama code, along with the skill content the agent uses to do the work.

Each task we throw at an agent is a “challenge.” A challenge directory (example) contains a README.md stating operations, latency targets, and other constraints. It also has an interface the agent must implement. The directory also contains private artifacts that are encrypted before runs so the agent can’t see them: tests covering functional correctness, fault-tolerance, and performance, and a reference implementation. After an agent finishes its implementation and passes its own tests, the challenge runner runs the formerly encrypted tests to determine whether the agent succeeded or failed.

Agents are run inside a Docker container with full permissions. We capture every agent invocation’s full transcript, including thinking, tool uses, tool results, and the final response. Thinking is particularly valuable. It’s how we discover failure modes that don’t show up in the produced code, like an agent identifying a fault-tolerance gap, going back and forth on possible solutions, and then saying “this is getting complicated” and failing to address it at all.

Rama has Java and Clojure APIs. We’re focused on Clojure for now but will produce an equally capable Java version of the skills later. The REPL is the main reason, as with a long-running REPL session, the agent evaluates code and inspects results in milliseconds instead of constantly paying for JVM startup and dependency loading. We expect this gap to matter more as challenges get harder and converging on a correct design takes the agent many iterations.

Working on improving LLM performance involves making a new challenge and then iterating on the skill files until the agent passes it consistently. Then we move on to a new challenge that stresses a different type of application, a different capability of Rama, or an application of greater scope.

Current status

Three medium-complexity challenges now one-shot correctly on almost every run, all horizontally scalable and fault-tolerant:

  • Bank transfer: tracks funds for each user and supports deposits, transfers between users, balance reads, and inbound/outbound transfer history reads per user. Transfers must be exactly-once and fault-tolerant: no double-spends under retry, no transfer that lands on one side but not the other under failure, and no negative balances from insufficient funds.
  • Time series: records render latency measurements per URL and answers range-aggregate queries (cardinality, total, min, max) over any minute-bucket range from a single minute to many years. Queries must be fast across the full range of query lengths, whether five minutes or five years.
  • Auction: hosts auctions where sellers list items with expiration times, bidders place bids, and the highest bid at expiration wins. Supports reading a seller’s listings, a listing’s bids and current top bid, and a user’s notifications. Auctions must end automatically when their expiration time passes, and seller / winner / losing-bidder notifications must each be delivered exactly once and in chronological order.

The private tests verify performance and fault-tolerance characteristics using the with-event-hook macro that we added in the latest Rama release. With this we can capture and assert on computation being balanced, the number and types of underlying RocksDB operations, topology types chosen by the agent, force failures/retries, and more.

The bank transfer challenge is the easiest of the three. The main test is whether the agent recognizes that update latency in the hundreds of milliseconds is acceptable, and therefore that a microbatch topology is the right tool. Microbatch topologies have fault-tolerant exactly-once semantics for all computation. The challenge also verifies the agent chooses the optimal PState (Rama’s equivalent to databases) structures, especially that transfer logs are subindexed since they’re unbounded.

The hard part of the time series challenge is pre-aggregating the latency data at multiple granularities and then using a server side distributed query (called a “query topology”) to efficiently compute the range query while reading as few buckets as possible. The agent does a great job of reasoning through how many RocksDB operations will be done depending on the number of buckets read and then choosing the appropriate number of granularities. It also recognizes a query topology is appropriate since that’s more efficient than many roundtrips between the client and worker nodes.

The auction module is the hardest, with multiple features with differing performance characteristics, polymorphic data (for notification types), and time-based behavior. Getting notifications of auction results to have exactly-once delivery with fault-tolerance is easy to get wrong. The agent sometimes lands on something close to the reference design – a stream topology for listings and bids and a microbatch topology for expirations and notifications. What surprised me is it also sometimes produces a design I had never considered. Using just a stream topology, notifications are stored as a map rather than a list, keyed deterministically by the listing’s timestamp, the listing ID, and the notification type. Because the key is deterministic, a retried expiration rewrites the same keys with the same values, achieving exactly-once by making delivery of the notification an idempotent operation.

The published benchmarks show LLMs struggling with far simpler backends. They measure single-instance applications with no scalability or fault-tolerance requirements. Our challenges add exactly-once semantics, fault tolerance, horizontal scalability, and performance constraints that the tests actively verify. With seams removed, the model can spend its reasoning budget on the application requirements rather than a host of random technical details tangential to the application.

Skill files

The skill files we’ve developed have gone through many iterations in order to pass these challenges consistently. At first, they were basically just the Rama documentation translated to Markdown files. The top-level skill had core information needed to program Rama (concepts, dataflow syntax, paths), while less-needed information was put in reference files.

The agent consistently made the same mistakes even when the correct guidance was loaded in context. An agent that reads a reference at the start of a session would fail to revisit it at the moment of a specific decision. Some examples:

  • Rather than refer to the var bound by (defmodule FooModule ...) directly as FooModule , it would try to construct it as (FooModule.)
  • Unbounded locations in a PState would not be subindexed
  • Object would be used for schemas instead of something precise
  • Partitioners would be missing, especially before writes, causing the module to write to the wrong locations

We researched best practices on making skills and then did a major restructuring of the skills to instruct the agent to use a phased approach:

  1. Implicit spec. Derive every edge case and invariant the protocol implies but doesn’t state. Produces an IMPLICIT_SPEC.md document.
  2. Plan. Design the depots, PState schemas, and topologies before writing any code. Produces PLAN.md .
  3. Plan validation. Review the plan against a checklist of scenarios (e.g. race conditions, failures/retries). Produces PLAN_VALIDATION.md with a pass or fail verdict.
  4. Implement. Write the module source, adhering strictly to the plan.
  5. Implementation validation. Review the implementation against a checklist of common mistakes. Produces IMPLEMENTATION_VALIDATION.md .
  6. Tests. Write thorough tests covering every protocol method and every edge case from the implicit spec.
  7. Test validation. Review the test code against a checklist of common mistakes. Produces TEST_VALIDATION.md .
  8. Run tests. Run the test suite.

Three things make this effective. Requiring an artifact at every phase prevents the agent from silently skipping a step. The validation phases are written as explicit checklists, and LLMs are reliable at walking a checklist item-by-item without forgetting entries. And the phase split lets the agent focus on smaller pieces of work at a time.

A few language patterns turned out to make a big difference. Negative constraints with a reason work much better than positive suggestions at preventing the agent from going down wrong paths. Including a reason is critical as otherwise the agent comes up with some random justification to ignore it (e.g. “Do NOT write any topology code in this phase — implementations without plans produce wrong PState schemas, incorrect partition alignment, and unnecessary disk I/O”). We specify explicit default choices and require justification to deviate (e.g. “default to microbatch; if stream, state which API requires it” forces the agent to justify the choice it’s making, while “choose stream or microbatch” lets the agent go with whatever it already decided).

When working on the auction challenge, we started to run into issues with compaction. When the agent would compact in the middle of a run, it would frequently skip critical instructions. So we made some changes to the skill to make it compaction-resistant. Instead of all the instructions living in the skill itself, we changed the skill to instruct the agent to copy pre-written templates for all five artifacts into the implementation directory. Each template contains the full checklist and instructions for filling it in, plus a signpost at the top saying what phase it belongs to and what comes next. The agent’s job is to “fill in” each template, not write from scratch. We also instruct it to put signposts in comments in the implementation files.

After compaction, whatever file the agent reads, whether an unfilled template, the source code, or a partially completed validation, the signposts tell it where it is in the workflow and what to do next. The key insight is that critical instructions living in context get lobotomized by compaction, but instructions embedded in files on disk are permanent. The templates and signposts ensure the agent always has a roadmap. It knows where it came from (which artifacts are already filled in) and where it needs to go next (what the current template is asking for).

The phases, validations, and signposts are a redundant structure that constrain the agent’s search space and funnel it towards a correct solution. There are some critical instructions that we put at the top of the skill to help with that funneling.

One of the most important is emphasizing that the agent is building a production-worthy application. Without this explicit emphasis, the agent would frequently say things like “This code is just for testing locally, so failures are rare and I don’t need to consider them”. We actually lie in the challenge prompt by saying “You are building a production Rama module. It will be deployed under real conditions — node failures, processing retries, concurrent clients, high throughput.”

Other critical instructions at the top of the skill are “Never trade I/O efficiency for code simplicity” and “Never trade fault tolerance for code simplicity”. LLMs have a strong bias for what they consider “simple” code, willing to sacrifice important performance and fault-tolerance properties in the process. In reality, the code to achieve efficiency and fault-tolerance in these challenges barely requires any more code, so these instructions cause the agent to search a little harder to find the right solution.

The rest of the skill is information about Rama that the agent uses to design and implement. One of the most impactful pieces of information was adding latency numbers for RocksDB operations (which Rama uses in the implementation of PStates). We provide approximate numbers for the cost of RocksDB reads, writes, iterator seeks, and iterator reads. During challenge runs, you can see the agent do the math to estimate the cost of various strategies to inform potential design decisions. In the time series challenge, for example, this is how the agent decides to use multiple granularities and how many granularities to use.

We only put general, high-level information in the skills. All examples in the skills are unrelated to the challenges we’ve made. Watching the agent reason from the high-level properties of Rama to implementing correct solutions for each challenge is very cool to see.

Orchestration

Until recently, we had the agent orchestrate itself through all the phases of the skill. We prompted it with the challenge instructions and let it walk the whole process in one session. That stopped working on the latest challenge we’re working on (described below). Once an agent committed to a design early in the session, it tunneled on that design for the rest of the run, even when its own later thinking surfaced problems with it. The validation phases in particular would rationalize away findings with justifications that didn’t make sense.

We made a custom orchestration script to fix this. It invokes the agent once per phase with a fresh context. Each phase reads only its own doc and the artifacts produced by earlier phases, then stops. Validation phases are instructed to be adversarial against the artifacts from the previous agent. The rationalization loops we saw before largely don’t have a place to form, because the agent doing validation is no longer the same agent that wrote the plan or implementation.

As a side effect, much of the compaction-resistance work we did earlier is less relevant. Phases are short enough that compaction rarely fires inside one. That said, the compaction-resistant structure is still worth having if only to lessen the amount of information loaded in context when reading a skill, since so many of the instructions are in the artifacts on disk.

However, there are major issues with the orchestration script which makes me think we’ll ultimately abandon it. First off, it’s unrealistic that this orchestration script would be used by actual developers since it’s contrary to normal usage of coding agents.

It also adds significant latency. Each phase invocation pays a fresh-context cost (re-reading the skill, re-reading prior artifacts, re-loading references) before it can do any real work. In order to lower latency, the script has accumulated a lot of logic about what to do when a validation phase fails. At first the orchestration was strict. A single test failure kicked the run all the way back to the implementation phase, which then had to flow forward through implementation validation, test writing, test validation, and test running again. Most of that re-work was unnecessary, because test failures usually surface small, localized bugs the agent can fix without running most of the phases again. To avoid that overhead we introduced the minor-fail / major-fail distinction in the validation phases (only major changes trigger another adversarial review) and added a “finish” phase at the end where the agent stays in a single context and iterates on the module and tests until the test suite passes. Both changes reduced wasted work, but they grew the state machine considerably.

Instead, we’re thinking of having a top-level agent handle orchestration and delegate to subagents for execution of each phase. We’ll have to make sure we can still capture equivalent transcripts, and we’ll have to see what kinds of mistakes the top-level agent makes during orchestration.

Finally, a couple of small things we include as part of orchestration of the implementation phase are worth mentioning. After the agent finishes writing the module, it must verify the namespace loads cleanly before finishing the phase. Additionally, it runs a linter and fixes all errors before moving on. The linting hooks we made for Rama statically catch things like arity mismatches, using invalid dataflow forms, or referencing undeclared PStates. Including both of these steps as part of the implementation phase saves a lot of time in what would later be a test failure causing phases to retry.

Current challenge

We’re currently working on the “fanout” challenge, a social-media-style backend with profiles, posts, follows, and a per-user merged timeline of posts from accounts the user follows. It’s based on our Twitter-scale Mastodon implementation. The module the agent must implement runs alongside a provided social-graph module that stores the social graph in an optimized way so that fanout can be balanced even for a heavily unbalanced social graph. We wrote in detail how fanout at scale works in this blog post.

The spec requires that merged timelines be kept in memory (writing those to disk drops throughput by 15x), and that fanout be balanced and fair. A post by an account with millions of followers cannot delay fanout for a post by an account with ten. Because timelines are kept in memory, it needs to find an alternate way to make it fault-tolerant. The correct approach for that is to reconstruct lost timelines on read by querying for the recent posts of followees.

The agent has not yet produced an implementation that satisfies the non-functional constraints. It gets the functionality correct and is designing and implementing the reconstruction process, but it’s failing:

  • Minimizing data in the cache and using memory and GC-efficient data structures. The correct solution uses a ring buffer of long values for just the user ID and post ID for each entry in the timeline. The agent so far is always using a TreeMap for the timeline and also storing the post content in the cache, which massively increases memory usage unnecessarily since that can just be fetched in the query topology that fetches a timeline page.
  • Fairness is not handled at all. It eagerly delivers posts to all followers immediately instead of spreading out delivery for large users over a longer period of time, violating the spec.

We’re iterating on more guidance and validation steps to get the agent to make the correct decisions for these. It’s possible these are the kinds of details LLMs are unable to get right, since they’re high-level design decisions that require reasoning about runtime tradeoffs. So it’s fine if we have to make the challenge easier by telling it how to achieve these properties. But it would be better if we can get the agent to come to these conclusions on its own.

Model choice

We’re currently testing with Opus 4.6. We tried 4.7, but its responses include only summaries of the model’s reasoning rather than the raw thinking blocks. That’s a real problem for skill development, as seeing the model’s thought process is critical to determine how to get it to stop going down wrong paths.

We plan to test with other models, especially Codex, once we make more progress.

Open questions

We have many open questions to resolve:

  • How many challenges do we need to be confident we’re not inadvertently overfitting? There’s nothing specific to the challenges in the skills / orchestration, but their structure might be tuned to the particular patterns LLMs follow for these particular problems.
  • Matrix is too big a project to implement the whole thing via one iteration through our phase structure. How should orchestration be structured, and how should the agent split up the work? How does it iterate on a design as it learns more through implementation?
  • The Matrix test suite will be helpful not just for evaluating a Matrix implementation, but it could be helpful for the agent to use during development to iterate and check exact behavior. Will we be able to get Matrix to one-shot without access to the tests?
  • What’s the best way for developers to use these skills in practice? How should artifact generation be handled?
  • How can larger projects be efficiently parallelized?
  • What’s the impact of generating Rama applications that need to integrate with lots of existing infrastructure?

LLM usage during development

LLMs are helpful for building this project, though they can be actively detrimental if not used properly.

The most useful application has been transcript analysis. We had the LLM build a transcript analysis script (scripts/analyze-latest-transcript.py) with subcommands for searching thinking blocks, tool calls, validation artifacts, and so on. The LLM then uses that script freely when we ask it to dig into why an agent made a particular mistake without us needing to give permission constantly for it to run custom shell or Python commands to analyze transcripts.

We also use LLMs to build the harness and tooling for the project. LLMs built the orchestration script that runs challenges, and they’re helpful for assisting in making new challenges.

We also have an alignment-scoring pass that runs after each challenge. It compares the agent’s implementation and tests to the reference implementation and tests, producing a short summary of differences. It’s not a significant part of the workflow, but reading a short summary is a faster way to spot a regression than diffing files directly.

LLMs are useful for brainstorming skill updates after we identify a failure mode, but only as assistants. They’re not good at understanding why they made a mistake. They always think they have an answer, even when the problems are subtle. They cheat like crazy, trying to add challenge-specific info or examples into generic reference files. So LLM suggestions can help with critical analysis, but they don’t substitute for it.

Conclusion

I expect future progress reports to be much shorter than this one, since I’ll just talk about new progress. I hope these progress reports are useful to anyone using LLMs or developing skills of their own, and I hope to get useful ideas and feedback from anyone following along.

Permalink

Data Governance in Versioned Systems

Data Governance in Versioned Systems

May 2026

If a database remembers everything, how do you delete things?

This is the first question people ask when they encounter immutable data systems. It’s a fair question. GDPR right-to-erasure requests, accidental PII ingestion, retention policies — these aren’t edge cases. Any system that stores data for real workloads needs to answer them.

The second question is access control: who gets to see what? A database that lets you fork, branch, and time-travel is powerful, but it also means the attack surface is wider than a traditional system where yesterday’s state is already overwritten.

This note describes what works today across Datahike and its ecosystem, what’s planned, and what’s not solved yet.

Purge in Datahike

Datahike has a purge operation that removes datoms from a database’s indices — both the current and the history index of the resulting commit.

What is this syntax? 
require('[datahike.api :as d])

;; Before purge
d/q('[:find ?n :where [?e :name ?n]] @conn)
;; => #{["Alice"] ["Bob"]}

;; Purge Alice (requires :keep-history? true)
d/transact(conn [[:db.purge/entity [:name "Alice"]]])

;; After purge — gone from current state AND from the new commit's history
d/q('[:find ?n :where [?e :name ?n]] @conn)
;; => #{["Bob"]}

d/q('[:find ?n :where [?e :name ?n]] d/history(@conn))
;; => #{["Bob"]}
(require '[datahike.api :as d])

;; Before purge
(d/q '[:find ?n :where [?e :name ?n]] @conn)
;; => #{["Alice"] ["Bob"]}

;; Purge Alice (requires :keep-history? true)
(d/transact conn [[:db.purge/entity [:name "Alice"]]])

;; After purge — gone from current state AND from the new commit's history
(d/q '[:find ?n :where [?e :name ?n]] @conn)
;; => #{["Bob"]}

(d/q '[:find ?n :where [?e :name ?n]] (d/history @conn))
;; => #{["Bob"]}

Two things purge is not. It’s not a soft delete: it rewrites the affected path through the persistent sorted set, and the new commit’s index roots no longer reach a node containing Alice. It’s also not retroactive across the commit graph: each commit owns its own index roots, so the pre-purge commit still points at the old nodes containing the datom. Until d/gc-storage sweeps that intermediate commit, the bytes remain on disk and (d/commit-as-db conn <pre-purge-uuid>) can still see Alice. The recipe in the next section closes that loop.

How DELETE disposes of data — and how purge differs

Compare to a traditional DELETE. In PostgreSQL, DELETE marks the row’s heap tuple dead; the bytes live on in several places:

  • In the heap page as a dead tuple. Plain VACUUM marks the space reusable and may defragment within the page, but never returns space to the OS or zeroes bytes. VACUUM FULL rewrites the table into a new file; the old file is unlinked but its disk blocks are not zeroed.
  • In WAL segments until they’re recycled, and indefinitely in archived WAL if archive_mode=on.
  • In replicas, which received the original WAL and run their own VACUUM cycle.
  • In backups — base backups, pg_dump archives, PITR base + WAL.

Proving the row is gone requires forensic checks across each of those layers, and there’s no manifest of where it lived.

Purge changes one part of that picture: in the live store, deletion is explicit and structurally locatable. The persistent sorted set is a tree of content-addressed nodes; the purge transaction rewrites the affected path; the new commit’s roots no longer reach a node containing the datom; a subsequent gc-storage with an appropriate cutoff sweeps the unreachable intermediate commits. The commit graph is itself a manifest — you can walk every branch head’s ancestors and list precisely which snapshots ever held the datom.

Where Datahike does not differ from PostgreSQL: backups, storage backends with their own versioning, and replication targets. We come back to that in Backups and storage-layer history.

Garbage collection in Datahike

d/gc-storage is the sweep that physically reclaims storage. The full mechanics — cutoff dates, branch-heads-always-kept, online vs. offline GC — are in Branches as Values, Merges as Queries. The piece that matters for governance is how it composes with purge:

What is this syntax? 
require('[superv.async :refer [<?? S]])

;; 1. Purge — produces a new commit whose roots no longer reach Alice.
d/transact(conn [[:db.purge/entity [:name "Alice"]]])

;; 2. gc-storage WITHOUT a cutoff only reclaims storage on deleted branches.
;;    The pre-purge commit on :db is a live intermediate commit, so its
;;    tree nodes (containing Alice) survive this sweep.
<??(S d/gc-storage(conn))

;; 3. gc-storage WITH a cutoff is what physically evicts those nodes.
;;    Pick a cutoff that exceeds your longest-running reader's lifetime.
let [seven-days-ago new java.util.Date(System/currentTimeMillis() - 7 * 24 * 60 * 60 * 1000)]:
  <??(S d/gc-storage(conn seven-days-ago))
end
(require '[superv.async :refer [<?? S]])

;; 1. Purge — produces a new commit whose roots no longer reach Alice.
(d/transact conn [[:db.purge/entity [:name "Alice"]]])

;; 2. gc-storage WITHOUT a cutoff only reclaims storage on deleted branches.
;;    The pre-purge commit on :db is a live intermediate commit, so its
;;    tree nodes (containing Alice) survive this sweep.
(<?? S (d/gc-storage conn))

;; 3. gc-storage WITH a cutoff is what physically evicts those nodes.
;;    Pick a cutoff that exceeds your longest-running reader's lifetime.
(let [seven-days-ago (java.util.Date. (- (System/currentTimeMillis)
                                         (* 7 24 60 60 1000)))]
  (<?? S (d/gc-storage conn seven-days-ago)))

This is the recipe most people get wrong on first read: purge + cutoff-GC, not purge alone. Plain gc-storage is a safe maintenance op — it leaves intermediate commits alone — which is exactly why it doesn’t finish the job for erasure. The pre-purge commit gets swept on the first cutoff-GC pass after it ages out of the grace window. In practice that means erasure has a tail measured in your GC cadence, not in milliseconds, which most compliance regimes accept.

The cutoff has to comfortably exceed your longest-running reader’s lifetime. Datahike’s distributed readers walk storage directly without coordinating with a writer, so a snapshot vanishing mid-query is a real failure mode.

Branch heads are always kept regardless of cutoff. So the recipe assumes the post-purge state is the head you want to keep; if the datom also exists on another branch, you purge there too before sweeping.

Secondary indices

Datahike’s secondary indices — Scriptum (Lucene full-text), Proximum (HNSW vector), Stratum (columnar) — are first-class versioned state. Indices are CoW-forked on branch, persisted with each commit, and restored on connect.

For governance, the question is whether purge propagates. It does: a purge transaction routes a retraction event (-transact with :added? false) to every secondary index covering an affected attribute, the same way :db/retract does. After purging Alice on a database with a Scriptum index over :person/name and :person/bio, a full-text search for “Alice” returns nothing, a vector KNN over her embedding skips her, and any columnar aggregate excludes her row.

On storage reclamation, Stratum and Proximum are konserve-backed: d/gc-storage sweeps their unreachable blobs alongside the primary indices, following the same pattern (Stratum: columnar rewrite; Proximum: HNSW mark-delete).

Scriptum is the exception. Its Lucene segments live on the writer node’s local filesystem, outside konserve. Scriptum’s -sec-mark returns the empty set, so d/gc-storage can’t reach them, and Lucene’s own delete model is tombstones-until-segment-merge — the bytes linger inside a segment file until Lucene merges that segment away. For full erasure on Scriptum you may need to force a segment merge and make sure the writer’s filesystem snapshot policy doesn’t pin old segments.

Backups and storage-layer history

Datahike does not escape the backup and archive problem. A backup of the konserve store taken before purge+GC contains the pre-purge nodes. Storage backends with their own versioning hold them too:

  • S3 object versioning, if enabled, retains every overwritten key as a prior version.
  • ZFS / btrfs snapshots of the konserve directory retain the pre-purge tree.
  • A git-backed konserve backend retains the pre-purge commit graph in .git even after a purge rewrite on the working tree.
  • Logical replicas of the konserve store hold whatever they received.

Purge + cutoff-GC reach the live store; erasure across backups and storage-layer history is a separate procedure — identify the destinations that contain the datom, then replay the purge against them or rewrite them. PostgreSQL doesn’t ship a tool for this either; in any storage-versioned environment it’s an operational policy question, not a database feature.

The common operational workaround is crypto-shredding — encrypt sensitive values per-user and “delete” by destroying the key. The bytes remain in WAL and backups but become unreadable; available to Datahike users on the same terms, with the same regulatory gray zone about whether key destruction counts as deletion.

What Datahike does give you here is a manifest of where the datom lived. Because history and branches are first-class, you can walk the commit graph from every branch head and list every snapshot that ever referenced the datom. That’s not what you get in PostgreSQL, where “which copies of this row exist?” is answered by inspecting WAL archives, backups, and replicas as independent unstructured investigations.

Multi-branch purge

Purge is a transaction on a single branch. If the same datom is reachable from another branch’s head, or from any of its commits inside the GC window, it lives there too. Structural sharing means the bytes are physically one copy in konserve — but the paths to reach them are independent.

The practical procedure: purge on every branch you control, delete branches you no longer need, then run cutoff-GC. Data reachable only from deleted branches gets reclaimed.

For databases with hundreds of agent-created branches, this gets expensive. Each branch’s purge is its own transaction and writes its own tree path. Optimization for the agent-fanout case is on the roadmap.

Access control

GDPR also requires access controls (Articles 15, 32). Today this works at the connection and storage level: storage credentials gate database access; the :branch config field (or SET datahike.branch over SQL via pg-datahike) pins a connection to a specific branch; commit-as-db (or SET datahike.commit_id) pins a specific snapshot for audit-only roles; read vs. write is per-connection.

Row-level access — “user X can query this branch but shouldn’t see rows where :department = "HR"” — applied consistently across current state, history, branches, and secondary-index queries is the next layer. EACL-style ReBAC is one direction we’re exploring.

Why immutable is still useful for compliance

The strong-form claim — “you can prove the data is gone” — doesn’t survive the backup and storage-history caveat. The narrower claim does, and it’s the one worth making.

In a mutable database, “which copies of this row ever existed?” has no manifest. You inspect WAL archives, replicas, backup catalogs, and page slack independently; each is a separate forensic exercise.

In a Datahike database, the copies are enumerable. History is explicit and branches are explicit. You can list — by walking the commit graph from every branch head — every snapshot that ever held the datom. Once enumerated, each can be addressed individually:

  • Live store — purge + cutoff-gc-storage rewrites the index path and sweeps the intermediate commits; commit-as-db lookups for swept UUIDs fail cleanly.
  • Secondary indices — covered by purge propagation on the konserve-backed ones; Scriptum needs the segment-merge / filesystem step.
  • Backups, S3 versioning, ZFS snapshots, git-backed stores — each is a separate destination addressed by policy.

The advantage isn’t “fewer copies than PostgreSQL” — usually it’s more copies, deliberately. The advantage is knowing what those copies are. “Show me the data is gone” becomes a checklist instead of an investigation.

What’s not solved yet

  • Multi-branch purge at scale is expensive. Hundreds of branches means hundreds of transactions, each rewriting its own tree path. Needs optimization work for agent-fanout workloads.
  • Scriptum filesystem GC. Lucene segments live outside konserve; d/gc-storage can’t reach them, and Lucene’s delete model is tombstones-until-merge. Filesystem cleanup is a separate operational step.
  • Row-level access control is the next layer. Connection / branch / commit-level access works today; fine-grained ReBAC across current state, history, branches, and secondary indices is the work in flight.
  • No retention-policy automation. There’s no built-in “delete everything older than 7 years.” :db.history.purge/before is the primitive; the policy loop is application code.

These are real limitations and we’re working on them.

Summary

Versioned, immutable data systems need to answer the deletion and access control questions directly. Datahike’s answer for the live store is explicit, structurally locatable deletion: purge rewrites the index path, cutoff-gc-storage sweeps the intermediate commits, secondary indices receive the retraction event, and the commit graph gives you a manifest of where the datom lived.

Where Datahike doesn’t differ from PostgreSQL is on backups, storage-layer versioning, and replicas — those still need a policy in either system. Where it does differ is in giving you a structured catalog of every copy on the live store: an enumerable checklist instead of an investigation. That’s the case for immutable data systems on compliance — not magic, but knowable.

Permalink

Advent of Code 2015 days 3-8 in Clojure

Yeah, I'm starting to think my initial comment about taking until the heat death of the universe to complete this wasn't far off the mark. I'm not really slow at finishing the tasks, it's just between working , the drudgery of everything else in life and possible undiagnosed ADHD, progress becomes a sporadic thing. Gone forever I think are the late-nighters of chugging caffeinated drinks...

But with that said, I have been slowly chugging away and gotten myself up to the heady heights of puzzle number 8.

I don't think I shall labour over each and every puzzle, instead I'll just gloss over the interesting parts.

Day 03

You're trying to determine how many points are visited if you're given a list of directions (N/S/E/W effectively). One pattern with this sort of thing I like to do is to store the position transformation as a map with the keys being the direction and the values being values to add to x and y coordinates.

(def moves {\^ [0  1]
            \v [0 -1]
            \> [1  0]
            \< [-1 0]})

The rest is just bookkeeping ;D

Day 04

Trying to mine a proprietary cryptocoin. I just used Java interop here - (java.security.MessageDigest/getInstance "MD5"). It is then a matter of looping through numbers to get the required number of leading zeros in the hexadecimal representation of the digest.

This was pretty brute-force and I don't expect to retire from crypto riches any time soon. It's quite slow too - clearly the hashing is going to be but I suspect we could possibly run multiple threads in parallel and/or optimise the way we're creating the byte-array to feed into Java each loop.

As it is, I was content just to wait for it to finish...

Day 05

String parsing really. I'm not a massive fan!

Was nice going back to see part of my original solution looks like this;

(defn nice?
  "is a word nice?"
  [word]
  (and (has-three-vowels? word)
       (has-repeated-letter? word)
       (no-forbidden-strings? word)))

Pretty legible. I guess you'd write something similar in most languages, but the convention of adding a question mark to tests is a nice touch.

Day 06

Mapping text instructions into turning a series of lights on/off/toggling in a given rectangle. Nothing to write about but I was interested to see if anyone had plotted the result in case there was a picture.

There isn't.

Day 07

Ah, we're getting interesting now, we had to simulate a basic logic circuit. We're given a input like;

123 -> x
456 -> y
x AND y -> d
x OR y -> e
x LSHIFT 2 -> f
y RSHIFT 2 -> g
NOT x -> h
NOT y -> i

I suspect there's a proper way of doing this in which we create a graph of connections and evaluate everything in the proper order. But what I did was just to brute force the whole thing; just keep running through the instructions until every instruction could be executed e.g. if you imagine in the example above that the numbers aren't applied to x and y until the last line, then we couldn't perform most of it until the second loop round the instructions.

Yes, as I edit this, I'm seeing a theme forming here where my solutions are just brute forced.

Day 08

String parsing again :D Trying to count the number of characters in a string and in the final representation. Not much to say again. Slogged through it.

Parser?

So far each problem's input is basic text, but I'd be interested to see if there's a tidy way of specifying the puzzle input. For example Day 2's input looks like;

1x2x3
1x3x6
...
...

So I'm thinking we almost want a function like;


(parse-input "day02.input" "<int>x<int>x<int>")

;; -> ((1 2 3) (1 3 6) ... ...)

Usually I'd either split or use regex for input, but it seems like what I'd like is to also be able to specify the basic type. Puzzle 8's example;

London to Dublin = 464
London to Belfast = 518
Dublin to Belfast = 141

Would be;

(parse-input "day08.input" "<string> to <string> = <int>")

;; -> (("London" "Dublin" 464) ...)

Or even better!

(parse-input "day08.input" "<string:from> to <string:to> = <int:distance>")
;; -->
({:from "London" :to "Dublin" :distance 464} ...)

I'd be interested if you know of anything like this feature.

End

I'll carry on with some more problems. I don't think I'll be entering any leaderboards soon ... unless there are some for slowest?

Permalink

Clojure Deref (May 26, 2026)

Welcome to the Clojure Deref! This is a weekly link/news roundup for the Clojure ecosystem (feed: RSS).

Upcoming Events

Podcasts, videos, and media

Babashka Conf

Mike Møller Nielsen

Blogs, articles, and news

Libraries and Tools

Debut release

  • ring-node-adapter - Ring adapter for Node.js

  • neat - A small, language-agnostic nREPL client for Emacs

  • code-bubble - A Clojure dev-time UI inspired by Code Bubbles

  • clj-code-search - Per-function semantic index for Clojure/CLJS, exposed as a code-search CLI for AI coding agents.

  • phel-doom - Terminal DOOM-lite raycaster built in Phel (Lisp on PHP). 256-color ANSI, 5 procedural levels — a real-world Phel showcase app.

  • gojava - A Go reimplementation of selected parts of the Java standard library

  • clojure-android - The Clojure programming language for Android

  • clojure-android-demo - Clojure Android Demo

  • muschel - Bash interpreter with git-like memory and fine grained access control.

  • fulcro-tui - A JLine-base TUI wrapper, allowing you to write TUI projects in CLJ or Babashka via Fulcro

Updates

  • core.unify 0.7.3 - Unification library

  • tools.gitlibs 2.6.217 - API for retrieving, caching, and programatically accessing git libraries

  • glogi 1.4.177 - A ClojureScript logging library based on goog.log

  • openapi-v3-validator 0.2.7 - A pure-clojure library for validating ring requests & responses against OpenAPI v3 specifications.

  • mailman 0.4.0 - Simple event routing library

  • refactor-nrepl 3.13.0 - nREPL middleware to support refactorings in an editor agnostic way

  • fulcro-spec 3.2.10 - A library that wraps clojure.test for a better BDD testing experience.

  • drip 1.0.0-alpha41 - Drip is a transactional job queue for MariaDB, PostgreSQL, and SQLite

  • cursive 2026.2-eap1 - Cursive: The IDE for beautiful Clojure code

  • dtype-next 11.022 - A Clojure library designed to aid in the implementation of high performance algorithms and systems.

  • tech.ml.dataset 8.023 - A Clojure high performance data processing system

  • rephrase 1.0.2 - Rephrase exceptions to be more beginner-friendly

  • bareforge 0.7.0 - Companion visual builder for BareDOM web components. Drag components, declare reactive state, export fully interactive CLJS or JS project

  • statecharts 1.4.0-RC16 - A Statechart library for CLJ(S)

  • stratum 0.3.72 - Versioned, fast and scalable columnar database.

  • teensyp 0.5.4 - A small, zero-dependency Clojure TCP server that uses Java NIO

  • generate 1.0.57 - code generation for Clojure projects

  • r11y 1.0.7 - CLI tool for extracting URLs as Markdown

  • cli 0.9.68 - Turn Clojure functions into CLIs!

  • html 0.2.4 - Html generation library inspired by squint’s html tag

  • next-jdbc 1.3.1108 - A modern low-level Clojure wrapper for JDBC-based access to databases.

  • ring-jetty9-adapter 0.39.4 - An enhanced version of jetty adapter for ring, with additional features like websockets, http/2 and http/3

  • svar 0.5.9 - Type‑safe LLM output for Clojure. Works with any text‑only model.

  • neil 0.3.70 - A CLI to add common aliases and features to deps.edn-based projects

  • gloat 0.1.44 - Glojure AOT Tool

  • encore 3.161.0 - Core utils library for Clojure/Script

  • clj-kondo 2026.05.25 - Static analyzer and linter for Clojure code that sparks joy

  • phel-lang 0.40.0 - A functional, Lisp-inspired language that compiles to PHP. Inspired by Clojure, Phel brings macros, persistent data structures, and expressive functional idioms to the PHP ecosystem.

  • spindel 0.1.13 - Cross-platform FRP runtime with a git-like memory model.

  • memento 2.1.74 - Clojure Memoization project

  • baredom 3.3.0 - BareDOM: Lightweight CLJS UI components built on web standards (Custom Elements, Shadow DOM, ES modules). No framework, just the DOM

  • fulcro 3.10.0-RC1 - A library for development of single-page full-stack web applications in clj/cljs

Permalink

libwce: the entropy layer of a wavelet codec, on its own

Most image codecs you know about such as JPEG, JPEG 2000, JPEG XS, WebP are like layer cakes. You have transform sitting on top, entropy coding at the bottom, and rate control floats somewhere in the middle. And then there's a metadata layer wrapping it all up. The interesting bits are hidden under tons of framing code, profile parsers, and standards plumbing. If you just want to see how wavelet coefficients become bits, you have to dig deep into the guts of the codec.

I wrote libwce as a bare-bones implementation, consisting of only a single lib.rs file, weighing in at 500 lines. It just implements a patent-clean Bit-Plane Count (BPC)-style entropy layer in the spirit of JPEG XS, and nothing else. There is no boilerplate or dependencies with the library relying solely on stdlib.

A two-minute primer on BPC coding

A raw video stream is basically a grid of pixels, most of which share very similar color and brightness values with their immediate neighbors. Storing every pixel individually wastes a ton of bandwidth since there is a lot of repeated data in the stream. Codecs are used to compress this information by transforming the image from the spatial domain into the frequency domain. Rather than tracking individual pixels, a codec uses mathematical frequencies to describe color changes across the image. Older formats like standard JPEG end up chopping the image into squares and applying a discrete cosine transform, leading to the blocky artifacts we all know and love.

A wavelet is a newer approach that solves the problem by applying the transform process to the whole image at once, splitting the signal into low-frequency structural data and high-frequency detail data across multiple scales. After the wavelet transform, you end up with a 2D array of signed integer coefficients, most of which are near zero, with a long Laplacian tail. The purpose of the entropy layer is to compress this array down to a small number of significant bits.

BPC coding is done using groups of four coefficients at a time. For each group, you have to determine the smallest bpc such that every coefficient can be held. This is the bit-plane count representing the index above which all coefficient bits in the group are zero. In libwce, all the bpc values are written first into a single bitstream, then for each group the four coefficients are emitted coeff-major. These are the magnitude bits of each coefficient followed immediately by a single sign bit when that coefficient is nonzero. That takes care of all the data processing you need to do. Then, you get to the actual compression when you go to encode these bpc values. Neighboring groups tend to have similar sizes, so instead of writing each bpc as a raw 6-bit number, you can estimate it from its neighbors and, instead, write a small residual which tends to be tiny.

Here, libwce uses RUNNING (DPCM delta vs the previous group's bpc, zigzag-mapped and Rice-coded) and ZERO (unsigned residual against lossy_bits) predictors which can be optionally combined with a 1-bit-per-8-group sparse-block flag that short-circuits all-deadzone blocks. That leaves you with four predictor × flag combinations, and the encoder sweeps Rice-k across seven values inside each, picking the best per band via a single-pass cost search. All combinations give the same decoded result, but they produce different types of bitstreams. Each one works best for different pixel distribution such as textured regions, flat parts, or sub-bands which are mostly zeros.

What it looks like to use

Here's a complete decoder for one sub-band:

let mut coeffs = vec![0i32; N];
let lossy_bits = decode(buf, &mut coeffs).unwrap();
dequantize_optimal(&mut coeffs, lossy_bits, scale_b);

The library itself is stateless, and only works with whatever buffers you provide. It doesn't use I/O or hidden globals, and works purely through caller-owned buffers (a small BPC scratch buffer is allocated internally).

Compressing an image end-to-end

The repo has 3 demos. The most fun one is image_compress, which is a full codec built on top of libwce. It uses Haar wavelet in, libwce in the middle, and inverse Haar on the way out which run across four quality presets.

  preset          lossy_bits      payload     .wce file    ratio    PSNR
                  LL  HL  LH  HH    bytes        bytes
  near-lossless    2   4   4   5    146537       146597    1.52x   49.06 dB
  balanced         4   6   6   7     92631        92691    2.40x   37.54 dB
  aggressive       6   8   8   9     49516        49576    4.48x   28.79 dB
  very lossy       8  10  10  11     21923        21983   10.11x   21.62 dB

The whole process consisting of DWT, sub-band coding, quantization, and writing to a container takes under 500 lines of code. If you open the four reconstituted PGMs side by side and you'll see quality degrade as compression increases. At q1, the image will be indistinguishable from the original; q2 has minor smoothing in flat areas; q3 starts to show noticeable wavelet ringing around edges; and q4 is blocky in a recognizable wavelet way, looking eldritch but still legible.

The second demo, mode_shootout, runs a synthetic Laplacian sub-band through every predictor × flag combination and displays the winner.

  mode             total   ratio   ok
  --------------   -----  ------   --
  RUN, flag=off      658   12.45x   Y
  RUN, flag=on       666   12.30x   Y
  ZERO, flag=off     652   12.56x   Y
  ZERO, flag=on      660   12.41x   Y
  auto-pick          612   13.39x   Y

  best forced: ZERO, flag=off  (652 bytes)
  auto-pick beat best forced by 40 bytes (better rice_k).

This is precisely the kind of thing that's a pain to do within the confines of a full codec, where you’d have to fiddle with instrumenting internals, disable rate control, and then mock the framing layer. With libwce, mode comparison is just how the API works. You use the same sub-band through encode_with_options with each predictor × flag combination, then count the bytes and pick the winner, which is exactly what encode itself does internally.

The third demo, stream_surgery, does 256 random bitflips and 256 random byte scrambles across the encoded bitstream, 300 truncation points covering every 4-byte prefix, and a set of adversarial cases including all-ones “unary bombs” along with crafted bad headers.

  bit-flip (anywhere)           : 256/256 returned, avg 36 / max 1024 coeffs differ (of 1024)
  random byte (anywhere)        : 256/256 returned without crash
  truncation (every prefix)     : 300/300 prefix lengths returned
  adversarial (bombs + bad hdrs): 7 cases returned cleanly

The demo shows how every case gets successfully decoded without any hangups or a crash.

What it isn't

Finally, it's worth reiterating that I intentionally didn't write libwce to be a full codec implementation, which would necessitate adding a container format, rate control, and other plumbing. It's designed to illustrate how the most conceptually interesting layer of a mezzanine codec works and to make it easier to study and modify without the weight of the full codec around it. What you get is just the entropy layer that you can wire into your own pipeline.

The repo is at https://github.com/yogthos/libwce. Clone it and play with it. It's written with readability in mind.

Permalink

The Age of Accountable Agents: Building Trust in Your AI Automation

The Age of Accountable Agents: Building Trust in Your AI Automation\n\nThe air around AI feels different this "Long Hot A.I. Summer." Big tech is pouring billions into development-Elon Musk's legal battles, Meta reassigning 7,000 employees to focus on AI-it's a high-stakes, high-energy environment. But for us, building powerful AI agents on consumer hardware, this moment isn't just about raw computational power or complex models. It's about something more fundamental: trust.\n\nThe recent news cycle offers a stark reminder of the ethical considerations, user control challenges, and privacy implications that come with advanced automation. From a papal encyclical discussing AI's moral implications to significant settlements over hard-to-cancel subscriptions, and even debates around nationwide data collection, the narrative is clear: we're entering the Age of Accountable Agents. And as developers, especially those focused on local, user-centric AI, we have a unique opportunity to lead the charge.\n\n## Trust Through Transparency: The AI Encyclical's Echo\n\nWhen you hear about an Anthropic co-founder discussing AI ethics with the Pope, it's a signal that the impact of our work extends far beyond our terminals. AI agents, by their nature, automate decisions. For these agents to be truly valuable and accepted, they must be transparent.\n\nWhat does transparency mean for an agent running on your hardware? It means:\n\n* Clear Decision Paths: Can a user understand why their agent took a particular action? If your agent automatically categorizes emails, can it explain its reasoning?\n* Auditable Logic: Even if not a full "explanation," the underlying logic should be inspectable. This doesn't mean revealing proprietary secrets, but designing agents where state changes and rule applications are explicit.\n\nConsider an agent designed to manage your smart home devices. Instead of a black box, you could implement a simple logging mechanism:\n\n


python\nclass SmartHomeAgent:\n def __init__(self, name):\n self.name = name\n self.log = []\n\n def act_on_temperature(self, current_temp, desired_temp):\n if current_temp > desired_temp + 2:\n action = "Turning on AC"\n self.log_action(action, f"Current: {current_temp}°C, Desired: {desired_temp}°C")\n # ... actual AC control code\n elif current_temp < desired_temp - 2:\n action = "Turning on Heater"\n self.log_action(action, f"Current: {current_temp}°C, Desired: {desired_temp}°C")\n # ... actual Heater control code\n else:\n action = "No action needed"\n self.log_action(action, f"Current: {current_temp}°C, Desired: {desired_temp}°C")\n return action\n\n def log_action(self, action, details):\n self.log.append(f"[{self.name}] {datetime.now()}: {action} - {details}")\n\n# Usage\nagent = SmartHomeAgent("ClimateControl")\nagent.act_on_temperature(25, 22)\nprint(agent.log)\n

\n\nThis basic logging provides a human-readable trail, fostering trust by showing, not just doing.\n\n## User Autonomy, Not "Hard-to-Cancel": Learning from Shutterstock\n\nThe $35 million settlement Shutterstock faced over difficult subscription cancellations is a potent lesson: users demand control over automated systems. For AI agents, this translates directly to how we design interaction and management. Your agent shouldn't feel like a digital trap.\n\nKey design principles for user autonomy:\n\n* Explicit Opt-in/Opt-out: Clear consent for agent actions and data usage.\n* Easy Pause and Stop: Users must be able to halt or reconfigure an agent's operation immediately.\n* Understandable Configuration: Agent settings should be accessible and intuitive, not buried in obscure files.\n\nThink about how your agent's lifecycle is managed. Here's a conceptual AgentController:\n\n

python\n# pseudo-code for an AgentController\nclass AgentController:\n def __init__(self, agent):\n self.agent = agent\n self._running = False\n\n def start(self):\n if not self._running:\n print(f"Starting {self.agent.name}...")\n self._running = True\n # thread or process start logic for agent.run()\n self.agent.start_service()\n\n def pause(self):\n if self._running:\n print(f"Pausing {self.agent.name}...")\n self._running = False\n self.agent.pause_service()\n\n def stop(self):\n if self._running:\n print(f"Stopping {self.agent.name} permanently...")\n self._running = False\n self.agent.stop_service()\n # Clean up resources\n\n def configure(self, new_settings):\n print(f"Configuring {self.agent.name} with new settings.")\n self.agent.update_settings(new_settings)\n\n# When you're building your agents, consider how these controls are exposed to the user.\n

\n\nFor more effective agent management, especially concerning permissions and operational boundaries on local hardware, check out AgentGuard. It helps you build in these essential controls from the ground up.\n\n## Privacy by Design, Not by Accident: The FBI's Data Ambition\n\nThe FBI's desire for nationwide license plate reader access is a stark reminder of the sheer scale of data collection possible today. For local AI agents, privacy should be a default setting, not an afterthought.\n\nWhen designing your agents, prioritize:\n\n* Local-First Processing: Perform computations and store data on the user's device whenever possible.\n* Data Minimization: Only collect and process the data absolutely necessary for the agent's function.\n* Transparent Data Policies: Clearly communicate what data an agent uses, why, and whether it ever leaves the device.\n\nBuilding agents for consumer hardware gives us a distinct advantage here. We can champion local intelligence and ensure that user data stays private by default, not by policy fine print.\n\n## Architecting for Clarity: The Lisp Connection\n\nThe Lisp family of languages (Common Lisp, Racket, Clojure) are hyperpolyglots for a reason: their power in symbolic computation and metaprogramming encourages clarity in expressing complex logic. While you might not be writing your agent in Emacs Lisp, the principles of clear, inspectable, and modular design are paramount.\n\nAn agent with well-defined modules for perception, decision-making, and action is easier to debug, understand, and, crucially, to trust. Avoid monolithic codebases where an agent's reasoning is opaque.\n\n

python\n# Conceptual Agent Architecture\nclass AgentBrain:\n def __init__(self, perception_module, decision_module, action_module):\n self.perception = perception_module\n self.decision = decision_module\n self.action = action_module\n\n def run_cycle(self, environment_data):\n perceived_state = self.perception.process(environment_data)\n desired_action = self.decision.evaluate(perceived_state)\n self.action.execute(desired_action)\n return desired_action # For logging/traceability\n\n# Each module can have its own transparent logic, making the overall agent's behavior understandable.\n

\n\n## The Intentional Click: Feedback Loops and Refinement\n\nEven a seemingly simple site like clickclickclick.click can serve as a quirky reminder of direct user interaction. How do your agents confirm intent? How do they solicit feedback effectively? It's not about mindlessly automating every single interaction, but about designing clear, intentional communication channels between the user and the agent.\n\nConsider points where your agent might ask, "Did I do that correctly?" or "Is this what you intended?" rather than just assuming. This explicit feedback loop refines the agent's understanding and reinforces the user's sense of control.\n\n## Building for a Trustworthy AI Future\n\nThe AI revolution is here, and it's happening everywhere, from the largest data centers to the devices in our pockets. As developers crafting AI agents for consumer hardware, we stand at a critical juncture. We have the unique opportunity-and responsibility-to build agents that are not just intelligent and efficient, but also trustworthy, accountable, and respectful of user autonomy and privacy.\n\nThis summer's "AI gold rush" shouldn't just be about speed; it should be about quality, ethics, and user-centric design. By focusing on transparency, control, and privacy by default, we can ensure our AI agents truly empower, rather than overwhelm, the people who use them.\n\nTo help manage these critical aspects of your agent's lifecycle, from permissions to operational safety, explore AgentGuard. It's designed to support you in building the next generation of conscientious AI automation. Start building agents that earn trust, today.\n

Permalink

Preemptive commoditization

Note: I wrote this in 2018, while I was helping a young startup and convincing them to open source their core application. If I were to write it nowadays, in 2024, I’d elaborate a bit more, including James Governor’s evergreen remark about how “You can make money with open source, but it’s extremely hard to make money from open source.” (source). Leaving it as-is, however.*

I’ve been advising a startup on the data transformation space. As part of this, we re-wrote the core engine in Clojure. The new version is, at the worst case, 16 times as fast in the same hardware, and in some cases over 200 times faster. And it does it in a fraction of the lines of code.

We did this in under 3 months of part-time work. We couldn’t focus our entire attention on it, as we had other concerns as well - I was involved with general team and management tasks, and the second developer was helping on other internal projects as well. To further raise the bar: we had to keep it functionality-compatible with the current version, so I had to get acquainted with the existing feature set, and it was the other developer’s first Clojure project.

Clojure made our lives so much easier. But this is not a post about why Clojure is cool.

I’ve been arguing about why they should open source anything that is not enterprise-specific, including this layer. There’s many advantages, which I won’t go over right now, but there’s also a looming threat.

Layers are getting commoditized faster and faster. More and more, there is demand for people who are good at wiring things together (beyond the gem install hairball approach), or tools that help with that wiring.

I suspect that’s a big part of what’s driving how many companies like Seldon are going open source-first, or how Unreal opened their code as a way to compete with Unity.

“what’s driving how many… are going” is a doozy

Remember: This rewrite took about 3 man-months, with our attention pulled in multiple directions, while we strove to remain feature-compatible with the old engine. You have to assume anyone else who has the technical chops but doesn’t have that baggage can do it as well. Do you want to be disrupted by some motivated, random person who thought what you were doing was cool, but neither needed your entire feature set nor wanted to be shackled to your cloud version?

Better to commoditize yourself before someone else does it to you. You get to have a say in how it happens, use it as fuel to propel you somewhere new. And you get to tell your customers “if you think this thing we give away is cool, you should see the part we charge for”.

Permalink

The framework mirage

OK, this is going to be a tough one. Strap down, even if it pisses you off at first - I wouldn’t be writing it if I didn’t think it was helpful.

Stop me if you’ve heard this one before

A programmer has something he wants to build.

He finds a framework.

It looks easy enough to get into. There’s a few tutorials available, which show step by step how to build a basic application of just the right type.

He’s happy. He feels really productive. CRUD just “drops out” of the framework’s design. It’s like it was created with his problem in mind.

Then he’s done with the tutorials. He starts plugging in his own, specific requirements. Every so often he hits a snag, but that’s OK, he can just twist the framework’s arm a bit, right?

It all goes well for a while. Then he hits a big one. The framework doesn’t do quite work the way he wants it to.

He posts a question. Maybe he doesn’t get an answer. Maybe he does, and he’s told it’s not within the benevolent dictators’ design parameters.

Crap. Oh well. He can start working around the decisions that don’t match his application. This will add some development overhead. At some point it’ll be enough that he has to consider his options.

That’s OK. He has the source. He can fork it, and do his custom stuff - he knows it well enough by now. He’ll lose the upstream fixes, unless he puts in some real effort into integrating them, but at least he doesn’t have to ditch the codebase.

This will tide his team over until the re-write, which they’ll build upon a different framework. One that looks like it better suits their needs.

Status quo

That’s just the way things are, right?

If you take someone’s code, they have made some decisions for you. Sometimes the decisions work in your favor, sometimes they don’t.

There’s an alluring side to this. The main reason why a framework might feel solid at first is because someone made a lot of decisions for you - one might say such a framework is opinionated. Those decisions seem to make sense, and they get you from zero to CRUD in just a few steps.

If you happen to be building the exact type of application that the framework creators have in mind, if their way of thinking immediately clicks with you, that’ll take you a long way.

But when you try to move away just bit from the set of problems they decided to address, you realize that those choices accrete. On a solidly integrated framework, you don’t get to pick and choose. Pulling away even a single one requires considerable effort.

Most of the time, either you take it or you leave it.

The not-framework approach

Frameworks expect to have an answer to most of your fundamental questions. Can you really rely on them to do that?

Consider an alternate path.

At first glance, Clojure’s Luminus is not as solidly integrated as other alternatives. For every potential component, you get to pick and choose - and never mind PostgreSQL vs. MongoDB vs. Datomic.

  • It defaults to Migratus for database migrations… but you can just pull it out and use any alternative.
  • Ring provides a nice abstraction for HTTP, but which web server do you want? Immutant or HTTP Kit?
  • Do you want ClojureScript support? If so, do you want Reagent or Om? And what’s the difference between those two anyway?

Even once you’ve made a decision, chances are you won’t find any step-by-step zero-to-CRUD tutorials for your specific combination.

So… how does this help again?

Control

No, Luminus is not making any decisions for you. The very few areas where it provides a single choice, like Selmer for templating, are orthogonal to your application’s design.

Unlike the frameworks in fashion on other languages - or even some that are starting to trend for Clojure - it’s not opinionated. One could say it’s open minded.

Here’s what happens when you create a new Luminus application.

  1. You’ll have to tell it which options you want included. A list of the options is available on the site.
  2. Luminus will generate a new template from those options, every option potentially pulling in a few libraries.
  3. It will also include some reasonable defaults and glue code, to save you time.

While it’s happy to show you its conventions with the templates that it generates, you won’t find it making any choices for you in advance. But more importantly, at no point does it try to wrestle control of those decisions from you.

It lets you make up your own mind.

Decision independence

Each library Luminus includes does make some decisions internally, for obvious reasons. But these are mechanical in nature, and given that Luminus doesn’t want its basic components to be strongly fused together, you are free to swap out any particular part.

Think of it as the bevy of functions you have on Clojure. Every one of them provides a piece of independent, re-usable functionality. Yes, the internal decisions of how the functions were implemented are made for you in advance, but it’s up to you to decide how to put them together to build your application, or if even you’d rather use an external library.

Learning curve

Yes, this means there will be a slightly steeper learning curve. You’ll need to know what these choices mean, before you can make an informed decision.

That means that on day-1, you’ll feel less productive than how you would on an opinionated framework, because you don’t get to coast on decisions someone else has made for you.

If you are looking for day-1 productivity, it may not be for you.

I’m not. I’m looking at life time value.

After using this loosely-coupled approach on projects for a couple of years, I’ve realized two things.

I like choosing

I get to replace any single part of the system, if I find it doesn’t suit my needs.

I may not need to. Hopefully I made the right decision up-front. But if for whatever reason I didn’t, I know that no other choice is tightly integrated with it, so I can gradually pull it out and replace it - I don’t need to switch frameworks, or fork it, or live with it.

I get to change my mind.

I no longer dread updates

Up until a few months ago, YeSQL was the standard way to access SQL databases in Luminus. When YeSQL support slowed down, Dmitri Sotnikov decided to replace it with HugSQL, which follows the same approach but takes it further and is actively developed.

Want to guess how many existing Luminus projects it affected?

Zero.

How can I state that with certainty?

Because it couldn’t possibly have affected a single one. Your choices are only for which libraries is your initial template generated from. After that, you are free to decide which of these do you want to update at any point, depending on if it suits you or not.

Any changes to future Luminus choices will have zero impact on you.

Productivity trade-offs

This means initially you’ll be taking baby steps. You may feel like an idiot as you go about figuring things out.

We all hate that. I get it.

It’s part of what makes frameworks so appealing when facing a new platform: they promise you’ll hit the ground running. Nobody likes to crawl when they’re used to just zoom by basic tasks.

But when changing contexts, we have to take baby steps first and then go on from there, because every decision we make is a trade-off.

When choosing a loosely-coupled approach, you’re saying that day-1 productivity is less important than long-term maintenance and total application lifetime productivity.

Personally, I’d rather spend time up-front understanding the choices I’m making. If I don’t have fundamental decisions made for me in advance, I have to look at what the options are, and can decide to change some pieces.

This understanding increases transparency, which in turn enhances my comprehension of the very foundations that I’ll be building upon for months or years.

Given I spend more time on application support and extension than on the first few days, it’s a trade-off I’m happy to make.

Permalink

Alex Bedner proof read my Clojure Book

Hello Clojurians,

I’m happy to announce that the Clojure book has gotten a bit better! A few days ago, I received an email from someone named Alex Bedner. He read my Clojure book from start to finish and was kind enough to share some corrections. I’ve implemented them, and you can find the updated book here: https://clojure-book.gitlab.io/

I did try to upload the print copy to Amazon, but for some reason they require additional verification from my side — and I’m not entirely sure what that entails. I’m not great with bureaucracy, so getting the Amazon listing sorted will take some time. In the meantime, Alex Bedner’s proofread version of the Clojure book is available at the link above. I’m deeply grateful to him.

There’s still a lot to do, and I remain committed to Clojure. Here are the things I’d like to finish in the book:

Creating Your Own Libraries (https://clojure-book.gitlab.io/book.html#_creating_your_own_libraries) — this section currently has no accompanying video. I’ve been meaning to make one, but haven’t gotten around to it yet — I can’t quite explain why.

Debugging with Calva (https://clojure-book.gitlab.io/book.html#_debugging_with_calva) — I need to write this section, with plenty of screenshots so readers know how to debug with Calva. At the very least, I’d like to link to the Calva website, where the creator has put together some excellent videos.

I’d also like to add a section on Paredit at some point — we’ll see how that goes.

Finally, I’d love to use LLMs to help proofread and polish the book further. English isn’t my first language, and I’ve found that AI tools have been genuinely helpful in improving the quality of my writing lately.

Once again, thank you to everyone who has said kind words about the book, pointed out corrections, offered criticism, or praised it. All of it is valuable input, and I’ll make the most of every bit of it.

Permalink

Clojure Deref (May 19, 2026)

Welcome to the Clojure Deref! This is a weekly link/news roundup for the Clojure ecosystem (feed: RSS).

Clojure Dev Call

The Clojure development team is pleased to announce a Clojure Dev Call on May 26 @ 17:00 UTC!

Join the Clojure dev team for an update on what we’ve been working on and what’s on our horizon. We’ll save time for a Q&A, so bring your questions.

Register here

Clojure/Conj 2026

Join us for the largest gathering of Clojure developers in the world! Meet new people and reconnect with old friends. Enjoy two full days of talks, a day of workshops, social events, and more.

September 30 – October 2, 2026
Charlotte Convention Center, Charlotte, NC

The CFP is open until June 14

Early Bird tickets are on sale now.

EuroClojure 2027

After a decade away, EuroClojure returns to mark twenty years of Clojure. Join us in Prague for three days of workshops, talks, and thoughtful conversation.

May 19-21, 2027
2027.euroclojure.org

Join the mailing list for early bird tickets and announcements. Share your ideas and content suggestions when you sign up.

Upcoming Events

Podcasts, videos, and media

Babashka Conf

Swish by Rod Schmidt

Mike Møller Nielsen

Blogs, articles, and news

Libraries and Tools

Debut release

  • let-go - Almost Clojure written in Go.

  • mino - A tiny, embeddable, REPL-friendly Lisp implemented in pure ANSI C.

  • clj-p4 - Read-only Perforce-to-Git bridge in Clojure, with stream and classic-depot support.

  • gloat-demo-fiber - A JSON REST API built with Fiber and Gloat, demonstrating Go/Clojure interoperability for web applications.

  • zulipdata - A Clojure library to fetch and analyse data from the Clojurians Zulip chat

  • rechentafel - A spreadsheet evaluator for Clojure/Script.

  • re-frame2 - The AI first-born of re-frame

  • ifgame - An interactive fiction game written in Clojure

  • drip - Drip is a transactional job queue for MariaDB, PostgreSQL, and SQLite

  • port - A minimalist Clojure interactive programming environment for Emacs, built on prepl

  • generate - code generation for Clojure projects

  • bara-lang - Clojure for nim language

  • gsheetplus - Low-level and high-level wrapper to work with Google Sheets. Reading, writing and sheet management.

  • winze - An agentic AI memory system (mcp server) with semantic search and GUI search/edit tool for Markdown knowledge bases

  • ghosttyfx - JavaFX terminal that uses libghostty

  • cljfx/ghosttyfx - Cljfx wrapper of GhosttyFX

  • beagle - a typed Lisp authoring surface for agent-written dynamic code.

  • ducktape - Connect tech.v3.dataset to DuckDB

  • phel-log - Data-driven logging library for Phel. Levels, namespace filtering, pluggable appenders, PSR-3 adapter, Monolog handler bridge. Inspired by Timbre + Monolog.

  • phel-symfony-demo - Symfony + Phel demo: Ring-style handlers over plain maps, no ORM

  • stube - Web framework inspired by Seaside and UnCommonWeb

  • clj-native-agent - Structural, semantic, and surgical skills for Clojure AI agents. Mimics expert practitioner workflows for code discovery, debugging, and editing using Babashka and rewrite-clj.

  • llmisp - JSON AST > Clojure

  • spindel - Cross-platform FRP runtime with a git-like memory model.

  • rephrase - Rephrase exceptions to be more beginner-friendly

Updates

  • clojure 1.12.5 - The Clojure programming language

  • clojure_cli 1.12.5.1645 - Clojure CLI

  • clojurescript 1.12.145 - Clojure to JS compiler

  • core.unify 0.7.2 - Unification library

  • clj-watson 6.1.0 - A Clojure tool that checks for vulnerable dependencies

  • aleph 0.9.8 - Asynchronous streaming communication for Clojure - web server, web client, and raw TCP/UDP

  • awesome-backseat-driver 1.0.13 - Plugin marketplace for Clojure AI context in GitHub Copilot: agents, skills, and workflows for REPL-first interactive programming with Calva Backseat Driver

  • logging4j2 1.0.8 - A Clojure wrapper for log4j2

  • core.async.flow-monitor 0.1.5 - A real-time monitoring and interaction tool for clojure.core.async.flow

  • martian-aleph 0.1.4 - Martian plugin to use the Aleph http client

  • thneed 1.1.9 - An eclectic set of Clojure utilities that I’ve found useful enough to keep around.

  • statecharts 1.4.0-RC13 - A Statechart library for CLJ(S)

  • joyride 0.0.75 - Making VS Code Hackable like Emacs since 2022

  • stratum 0.2.67 - Versioned, fast and scalable columnar database.

  • deps-new 0.12.1 - Create new projects for the Clojure CLI / deps.edn

  • bareforge 0.6.0 - Companion visual builder for BareDOM web components. Drag components, declare reactive state, export fully interactive CLJS or JS project

  • pg-datahike 0.1.43 - Postgres compatibility layer for Datahike.

  • de-dupe 0.3.0 - A ClojureScript library which "de-duplicates" Persistent Data Structures so they can be more efficiently serialised.

  • ClojureStorm 1.12.5 - A fork of the official Clojure compiler with extra code to make it a dev compiler

  • graal-build-time 1.0.6 - Initialize Clojure classes at build time with GraalVM native-image

  • tyrell 1.0.0-RC9 - Clojurescript WebComponents library

  • gloat 0.1.41 - Glojure AOT Tool

  • metamorph.ml 1.6.2 - Machine learning functions based on metamorph and machine learning pipelines

  • oksa 1.2.1 - Generate GraphQL queries using Clojure data structures.

  • pomegranate 1.3.27 - A sane Clojure API for Maven Artifact Resolver + dynamic runtime modification of the classpath

  • calva 2.0.588 - Clojure & ClojureScript Interactive Programming for VS Code

  • cli-tools 0.16.0-beta-7 - CLIs and subcommands for Clojure or Babashka

  • fulcro-spec 3.2.9 - A library that wraps clojure.test for a better BDD testing experience.

  • baredom 3.1.0 - BareDOM: Lightweight CLJS UI components built on web standards (Custom Elements, Shadow DOM, ES modules). No framework, just the DOM

  • o11ylite 2026.5.16-1112955 - Free, open-source OpenTelemetry backend powered by DuckDB 🦆 and SQLite

  • calva-backseat-driver 0.0.34 - VS Code AI Agent Interactive Programming. Tools for CoPIlot and other assistants. Can also be used as an MCP server.

  • plumcp 0.2.1 - Clojure/ClojureScript library for making MCP server and client

  • teensyp 0.5.0 - A small, zero-dependency Clojure TCP server that uses Java NIO

  • build-uber-log4j2-handler 2.26.0 - A conflict handler for log4j2 plugins cache files for the tools.build uber task.

  • phel-lang 0.39.0 - A functional, Lisp-inspired language that compiles to PHP. Inspired by Clojure, Phel brings macros, persistent data structures, and expressive functional idioms to the PHP ecosystem.

  • dexter 0.1-beta-2 - Dexter - Graphical Dependency Explorer

  • plotje 0.2.2 - simple and easy plotting

Permalink

Annually-Funded Developers' Update: March & April 2026

Hello Fellow Clojurists!

This is the second of six reports from the developers who are receiving annual funding for 2026. Thanks to everyone for supporting their work and important contributions to the Clojure community.

Bozhidar Batsov: nREPL, Clojure Mode, ts-mode, Orchard, CIDER, and more
Clojure Camp: Badges, nano-conj, exercises
Eric Dallo: eca, eca-desktop, clojure-lsp, clojure-lsp-intellij
Jeaye Wilkerson: Jank compiler architecture and optimization
Michiel Borkent: babashka, ClojureScript async/await, Rebel, squint, and much more

Bozhidar Batsov

2026 Annual Funding Report 2. Published May 1, 2026.

The last two months have been some of the busiest I’ve had in clojure-emacs land in a long while. No big “X.Y” CIDER release to point at yet, but there’s been a steady stream of important work across most of the sibling projects. Below are some of the highlights.

clojure-mode 5.22 and 5.23

Two clojure-mode releases back-to-back, after a long stretch of relative quiet:

  • 5.22 (March 3): a big bug-fix dump — clojure-sort-ns no longer mangles :gen-class, clojure-thread-last-all doesn’t eat closing parens into line comments, clojure-add-arity preserves arglist metadata, letfn bindings get a function face, and edn-mode indents data like data. On the feature side: a new clojure-preferred-build-tool for projects that have several build tool files lying around, a dedicated clojure-discard-face for #_ forms, and project root detection for ClojureCLR’s deps-clr.edn.
  • 5.23 (March 25): adopts the modern indent spec tuple format (((:block N)), ((:inner D)), …) that clojure-ts-mode and cljfmt already use. The legacy format (integers, :defn, positional lists) still works, but it’s slated for removal in clojure-mode 6. There’s a new public clojure-get-indent-spec API, and CI moved off CircleCI onto GitHub Actions.

Unifying the indent format across our Clojure tooling has been on my todo list for years - happy to see it finally happen!

clojure-ts-mode

No release in this window, but I poured a lot of effort into the test suite - proper indentation and font-lock test coverage, a new configuration option test suite, and integration tests against sample files. Boring infrastructure work, but it’s the kind of thing that pays compound interest as the mode keeps maturing.

Side note: In the past several months I’ve spent a bit of time hacking on neocaml and my work there provided quite a bit of inspiration for improvements to clojure-ts-mode. (and, of course - clojure-ts-mode inspired me to create neocaml in the first place)

Orchard 0.41

Orchard v0.41.0 shipped on April 13 with a rewrite of orchard.xref, a refactor of orchard.indent, and a test modernization pass. I chipped in some smaller fixes - a cycle guard in fn-transitive-deps, validating input to stacktrace/analyze, compiling a hot regex once instead of on every call, and standardizing the -test suffix across the test suite.

We’ve also migrated from Lein to tools.deps for Orchard and probably we’ll continue in this direction for the rest of our Clojure projects in the nREPL and Clojure Emacs GitHub orgs. No rush on that front, though.

cider-nrepl 0.59

cider-nrepl v0.59.0 landed on April 14. The headline change from my side is that the ops are now properly namespaced (#710) - something that’s been sitting in the issue tracker for a very long time. CIDER on master already speaks the new namespaced ops with a fallback to the legacy names for older cider-nrepl versions, so upgrades should be painless.

Other notable bits: Compliment and clj-reload are no longer shaded, the test middleware properly binds *report-counters*, and we extract inline maps from composite Lein profiles correctly.

refactor-nrepl

refactor-nrepl got a serious modernization push in April - no release yet, but a lot of cleanup landed:

  • Dropped http-kit in favor of the JDK’s built-in HTTP client (one fewer dependency, one less thing to break).
  • Restored hotload-dependency on top of tools.deps.
  • A new def-op macro to simplify how ops are defined and how errors are handled.
  • A shared test session helper, a make lint target, and modernized CircleCI executors.
  • A full dependency bump.

Expect a refactor-nrepl release once the dust settles.

clj-refactor.el

Just a small cleanup pass to align with the new Emacs 28+ baseline. clj-refactor is in maintenance mode these days, but I’m trying to keep it healthy.

Eventually I’ll be taking a closer look at moving some of its functionality to clojure-mode and CIDER, but there’s more ground-work that needs to be done first.

CIDER

No CIDER release this cycle, but master accumulated a lot of structural work that will land in CIDER 1.22:

  • nrepl-client.el decoupled from CIDER. It no longer depends on sesman or any of CIDER’s UI layer, which makes the nREPL client genuinely reusable outside CIDER. I also split cider-connection.el into cider-session.el + cider-connection.el and decoupled cider-eval.el from cider-repl.el.
  • cider-jack-in got a serious overhaul: a proper jack-in tool registry, unified entry points around shared helpers, TRAMP fixes, sane runtime defaults, and cider--resolve-command now actually works over TRAMP.
  • A pile of resilience fixes for the nREPL client: tear down the SSH tunnel on abnormal client disconnect, spawn the tunnel without a shell, plug request-id leaks in raw response handlers, stop response-handler errors from poisoning the response queue, and put a FIFO cap on nrepl-completed-requests so it can’t grow without bound.
  • Namespaced ops everywhere, matching cider-nrepl 0.59 - with a fallback to the legacy op names so older cider-nrepl keeps working.
  • A new nrepl-make-eval-handler with a keyword-arg API (and removal of the trivial wrappers that used to live around it).
  • Mode-line spinner while tests are running, which is a small thing but I missed it every time I ran a long test suite.
  • Lots of test coverage backfill on the client side.

This is shaping up to be one of those releases that doesn’t look flashy on the outside but cleans up a ton of long-standing internal mess. The decoupling work in particular has been on my mind for years.

I’ve also started working on experimental support for prepl in CIDER. I’m not sure if this will ever land or be properly supported, but it’s been a fun side quest to me to ensure that CIDER logic is decoupled properly from its bundled nREPL client that I plan to eventually spin out of CIDER.

MrAnderson

Slightly outside the clojure-emacs org, but related: I’ve been helping MrAnderson get back on its feet. Three PRs landed in March/April:

MrAnderson is what cider-nrepl (and others) use to shade dependencies, so a healthy MrAnderson is in everyone’s interest. There’s more to come here!

What’s next

CIDER 1.22 once the jack-in and decoupling work settles, a refactor-nrepl release to ship all the April cleanup, and hopefully a fresh MrAnderson cut soon after.

As always - thanks to everyone who pitched in, especially Sashko Yakushev who once again did much of the the heavy lifting on cider-nrepl and Orchard.

And a HUGE THANKS to the members of Clojurists Together for supporting my Clojure OSS work! You rock!

Clojure Camp

2026 Annual Funding Report 2. Published May 12, 2026.

  • Some of our CT funds are being set aside to support three efforts this year:
    • sponsoring conference attendance for new Clojurians,
    • supporting Clojure and non-Clojure meetups with a “pizza fund” (WIP),
    • hosting an experimental nano-conj (in-person multi-day open-ended hack-on-clojure “retreat”) (WIP)
  • On the “dev side”….
    • Worked through most of the backlog of exercises done in mobs and released on https://exercises.clojure.camp/
    • Continued on badge system, mostly wrestling with display of the “learning graph”
  • Other happenings
    • Hosted an in-person workshop at the Recurse Center (with another planned)
    • Started a new book club reading Knesl’s “Applied High Order Functions in Clojure”
    • Early planning for our presence at Conj this year (may be doing a “Learning Together” track on the workshop day, featuring Mobbing and Pairing sessions)
  • Raf has a major commitment finishing up in May, so June+July should see a big push on Camp projects (and resuming regular Mobs)

Eric Dallo

2026 Annual Funding Report 2. Published May 8, 2026.

Excited 2 months of lots of work and help from Clojurians! We had improvements in eca and clojure-lsp mainly, and new projects as well.

ECA

ECA keeps growing a lot, receiving lots of contributions, with more than 800 stars already I’m planning a stable release soon, in these 2 months we had lots of releases with ton of stuff, so I will focus on the main highlights:

0.109.1 - 0.131.1

  • Plugins: New plugin system to load external configuration from git repos or local paths, with an official marketplace at plugins.eca.dev. Plugins can provide skills, MCP servers, agents, commands, hooks, rules and arbitrary config overrides, managed via /plugins, /plugin-install and /plugin-uninstall.
  • Remote web control: New remote web control server for browser-based chat observation and control via web.eca.dev, allowing you to observe and drive ECA chats from any browser.
  • Trust mode: Clients can now auto-accept tool calls that would require manual approval, with regex patterns support for fine-grained shell_command approval.
  • Task tool: Built-in task tracking tool to let the agent plan and follow multi-step work more reliably.
  • Background shell commands: New background parameter on shell_command and a dedicated bg_job tool to manage long-running processes like dev servers and watchers.
  • Chat list, open, fork and /model: New chat/list, chat/open and /fork commands let clients render a chat sidebar and clone existing conversations, plus a new /model command to switch model mid-chat.
  • Message flags: Named checkpoints inside a chat for resuming and forking from a specific point, with full chat history preserved across compactions via tombstone markers.
  • ask_user tool: LLMs can now ask the user questions with optional selectable options, fully integrated with hooks and trust modes.
  • Image generation: Support for OpenAI’s built-in image_generation tool via the Responses API, including image edits across turns and MCP tools that return image content.
  • Prompt steering: New chat/promptSteer notification to inject user messages into a running prompt at the next LLM turn boundary, without stopping it.
  • MCP improvements: New mcp/addServer, mcp/removeServer, mcp/updateServer, mcp/enableServer and mcp/disableServer requests to manage MCP servers at runtime, plus much better OAuth spec compliance and a switch from the Java SDK to plumcp.
  • More providers and models: Added LiteLLM, LM Studio, Mistral and Moonshot as built-in providers with login support, Claude Opus 4.7, deepseek-v4-pro, gpt-5.4 and gpt-5.5 variants, and GitHub Enterprise Copilot.

ECA Desktop

eca-desktop eric mar apr

Since ECA has been pretty stable and built in top of a nice extensible protocol, it worked so well that we decide to offer the same server capabilities to a Desktop client, similar to Claude Desktop but reusing the same server, this makes possible to have the same experience without an Editor, especially useful for non techinical people in price for a thin layer connecting to the server.

That’s a lot for ECA and all part of the amazing community that’s been activelly helping with issues, feedback and contributions.

clojure-lsp

We had some important bumps with lots of fixes, new code actions and contributions! The main highlight here is the arrival of performance tests in the project, which is a long waited thing which would help unblock performance optmizations in clojure-lsp since now we have a way to reliable know a mean, p90, p80 of how much time clojure-lsp spend on its features like initialization.

2026.05.05-12.58.26

  • Fix cyclic-dependencies linter falsely reporting cycles for (require ...) calls inside (comment ...) forms. #2107
  • Support find-definition for fully qualified vars even when the namespace is not explicitly required. #2028
  • Fix create-test code action appending a duplicate deftest when one with the matching name already exists, now navigating to the existing deftest instead. #2274
  • Change the default of :clean :ns-inner-blocks-indentation from :next-line to :keep, so clean-ns (including the automatic run after add-missing-libspec, add-require-suggestion, add-missing-import, and move-form) no longer reflows the :require/:import block when the user has not configured an indentation style. Users who want the previous behavior can set :clean :ns-inner-blocks-indentation :next-line explicitly. #2261
  • Fix add-missing-require refer suggestions leaking across languages, so a .clj file no longer offers refers defined only in .cljs files (and vice versa). #2271
  • Add :private-by-default-on-extract? setting to control whether extracted functions and defs are private by default. #2258
  • Measure performance of code actions
  • Avoid incorrect circular dependency errors from :as-alias by working around clj-depend bug.
  • Fix inline-def to work with defs with metas.
  • Bump clj-kondo to 2026.04.16-20260503.191510-9.
  • bump up timeout for code action performance measurement, include p90 measurement #2236
  • implementation of inline function. #1827
  • Fix initialization crash when a source file has syntax errors (e.g. unbalanced parens) by using safe parser in unused-public-var linter’s :gen-class check. #2242
  • Bump rewrite-clj to 1.2.54.
  • implement move to :let refactoring #1732
  • Measure performance of didOpen and didChange
  • if code-action selection end-position args aren’t provided, don’t try to use them #2276
  • add unit tests for command actions location args #2279
  • New code actions: replace :refer with :as and replace :as with :refer, with support for merging into existing :refer vectors.

clojure-lsp-intellij

3.5.1 - 3.5.5

  • Implement createServerInstaller and createLanguageServerSettingsContributor on ClojureLanguageServerFactory. Newer LSP4IJ versions added these as default interface methods, but our def-extension/gen-class-backed factory always overrides interface methods, so explicit stubs are required.
  • Fix slurp-action-test against newer LSP4IJ versions. Bump clj4intellij to 0.9.0 and switch the test fixture to setup-heavy so the project base has a real filesystem path (LSP4IJ’s FileSystemWatcherManager calls VirtualFile.toNioPath(), which throws on the in-memory TempFileSystem used by light fixtures).
  • Fix new namespace creation incorrectly creating files under the absolute host filesystem path. #83
  • Fix QuoteHandler compile error by merging BAD_CHARACTER into the quote-handler TokenSet.
  • Fix auto closing single quotes.
  • Improve CI to have plugin zips in all releases, avoiding wait for Jetbrains approval.

Jeaye Wilkerson

2026 Annual Funding Report 2. Published May 8, 2026.

Howdy folks! Thank you so much for the sponsorship this year. For the last two months, I have been focused on compiler architecture and optimization for jank.

On the compiler architecture side, I have designed and implemented a custom intermediate representation (IR) for jank programs which sets the stage for writing Clojure-specific optimization passes. This IR operates at the level of Clojure’s semantics, which is much higher level than LLVM IR, and so we are able to perform optimizations which LLVM could never do for us.

On the optimization side, I have taken the first benchmark of many, recursive fibonacci, and I have optimized it to be nearly twice as fast as Clojure JVM, for the same exact code. This benchmark is the first of many and I will be following up with more benchmark optimization results in the coming two months, using our new IR as a platform for optimizing jank’s performance.

To read all about the details of jank’s IR and the benchmark optimization, take a look at this blog post:

Michiel Borkent

2026 Annual Funding Report 2. Published May 11, 2026.

In this post I’ll give updates about open source I worked on during March and April 2026.

To see previous OSS updates, go here.

Sponsors

I’d like to thank all the sponsors and contributors that make this work possible. Without you, the below projects would not be as mature or wouldn’t exist or be maintained at all! So a sincere thank you to everyone who contributes to the sustainability of these projects.

gratitude

Current top tier sponsors:

Open the details section for more info about sponsoring.

Sponsor info

If you want to ensure that the projects I work on are sustainably maintained, you can sponsor this work in the following ways. Thank you!

Updates

Babashka conf and Dutch Clojure Days 2026

Babashka Conf 2026 was held on May 8th in the OBA Oosterdok library in Amsterdam! David Nolen, primary maintainer of ClojureScript, was our keynote speaker. We’re excited to have Nubank, Exoscale, Bob, Flexiana and Itonomi as sponsors. Nubank and Exoscale are hiring. Wendy Randolph was our event host. For the schedule and other info, see babashka.org/conf.
The day after babashka conf, Dutch Clojure Days 2026 was also held - so it was a great weekend in Amsterdam!
Hope to have seen many of you there!

Projects

In the last two months I spent significant time organizing babashka conf, but made progress in several projects as well.

My upstream work to enable async/await in ClojureScript was merged in the beginning of March. The implementation mirrors squint. Thanks David for reviewing and merging. Also deftest now supports an ^:async annotation so you can use async/await and don’t need to mess around with the cljs.test/async macro anymore:

I’ll be presenting this work at the Dutch Clojure Days.

Rebel-readline is now bb compatible. The work involved mainly exposing more JLine stuff and making sure rebel-readline didn’t hit any internal JLine APIs. One step to drive this to completion was to make a dependency, compliment, bb compatible. Thanks both to Bruce and Alexander for the cooperation.

Squint now supports cljs.test and multimethods! clojure-mode was ported to use the new cljs.test.

On the cream front, I put in effort to make the binary smaller and have been keeping up with the new GraalVM EA releases. I’ve been posting bug reports to the crema maintainer. Currently there’s still an unfixed bug around core.async that I have trouble reproducing in pure Java. I also added lots of library tests to CI so I can ensure stability in the long run. For now it remains experimental, but the direction is promising.

A performance PR to weavejester/dependency speeds up depend, depends? and topo-sort significantly, so clerk notebooks render faster.

The cljfmt library, also by @weavejester, now fully runs from source in babashka. The Java diff library that wasn’t bb-compatible was replaced with text-diff, but only for the babashka path. The JVM build of cljfmt still uses the original Java diff library, with a possible switch later once text-diff has matured.

Several SCI fixes were made to improve Clojure compatibility between babashka and Clojure. E.g. records can now support extending to IFn which was a blocker for some Clojure libs that tried to run in bb so far.

Clj-kondo 2026.04.15 got a few new linters thanks to @jramosg for stewarding most of these. It also has better out of the box potemkin support, and @alexander-yakushev contributed a wave of performance improvements.

Updates per project below. Bullets are highlights; see each project’s CHANGELOG.md for the full list.

  • babashka: native, fast starting Clojure interpreter for scripting.

    • Released 1.12.216, 1.12.217 and 1.12.218
    • Support rebel-readline as external REPL provider:
      • Add proxy support for Completer, Highlighter, ParsedLine, Writer, Reader
      • Add clojure.repl/special-doc and clojure.repl/set-break-handler!
      • Add clojure.main/repl-read
      • Add org.jline.reader.Buffer to class allowlist
    • Add clojure.java.javadoc namespace with javadoc available in REPL #1933
    • Fix (doc var), (doc set!) and other special forms #1932
    • Support (source inc) and (source babashka.fs/exists?) for built-in vars #1935
    • Support BABASHKA_REPL_HISTORY env var for configurable REPL history location #1930
    • Fix deftype and defrecord inside non-top-level forms (e.g. let, testing) #1936
    • #1948: add java.util.HexFormat interop support
    • #1403: fix uberscript warnings with :as-alias
    • #1955: support -version as an alias for --version
    • #1954: add clojure.lang.EdnReader$ReaderException
    • #1951: fix --prepare flag skipping next token
    • #1967: expose clojure.data.xml.tree/{flatten-elements,event-tree}, clojure.data.xml.event record constructors, and clojure.data.xml.jvm.parse/string-source
    • #1969: include java.net.Proxy and java.net.Proxy$Type Java classes (@jeeger)
    • #1939: disable JLine backslash escaping/shell history commands (@bobisageek)
    • Performance improvements for math operations and for calling functions on locals
    • Add many new classes to reflection config: java.lang.reflect.Constructor, java.lang.reflect.Executable, java.util.stream.Collectors, java.util.Comparator (for reify), and more
    • Bump JLine to 4.0.12, cheshire to 6.2.0, nextjournal.markdown to 0.7.255, edamame to 1.5.39, data.xml to 0.2.0-alpha11, jsoup to 1.22.2, rewrite-clj to 1.2.54, tools.cli to 1.4.256, transit-clj to 1.1.357, fs to 0.5.32
    • Full changelog

  • SCI: Configurable Clojure/Script interpreter suitable for scripting

    • Fix recur with 20+ args in loop (#1035)
    • Check recur arity, throw when it doesn’t match (#1034)
    • Support IFn on defrecord, deftype and reify (#808, #1036)
    • Validate single binding pair in let-like conditional macros (#1037)
    • Normalize SCI types in hierarchy lookups (#1033)
    • Expose IPrintWithWriter as protocol (#1032)
    • Optimize tight loops: fused binding nodes + specialized inlined calls (#1031)
    • Support special form documentation in doc macro
    • Include SCI types in ns-map
    • Full changelog

  • clj-kondo: static analyzer and linter for Clojure code that sparks joy.

    • Released 2026.04.15
    • #2788: NEW linter: :not-nil? which suggests (some? x) instead of (not (nil? x)), and similar patterns with when-not and if-not (default level: :off)
    • #2520: NEW linter: :protocol-method-arity-mismatch which warns when a protocol method is implemented with an arity that doesn’t match any arity declared in the protocol (@jramosg)
    • #2520: NEW linter: :missing-protocol-method-arity (off by default) which warns when a protocol method is implemented but not all declared arities are covered
    • #2768: NEW linter: :redundant-declare which warns when declare is used after a var is already defined (@jramosg)
    • #1878: support potemkin’s import-fn, import-macro, and import-def
    • #2498: support new potemkin import-vars :refer and :rename syntax
    • Performance optimizations across many linting paths (@alexander-yakushev) and hook-fn lookup caching to avoid repeated SCI evaluation
    • Add type support for pmap and future-related functions (future, future-call, future-done?, future-cancel, future-cancelled?) (@jramosg)
    • #2762: fix false positive: throw with string in CLJS no longer warns about type mismatch (@jramosg)
    • #2770: linter-specific ignores now correctly respect the specified linters
    • #2773: align executable path for images to be /bin/clj-kondo (@harryzcy)
    • #2621: load imports from symlinked config dir (@walterl)
    • #2798: report correct filename and error details when StackOverflowError occurs during analysis
    • Full changelog

  • cream: Clojure + GraalVM Crema native binary

    • Followed each GraalVM EA release: EA21 shrunk the binary to ~175MB, EA22 brought a virtual-thread fix, EA23 fixed the forkjoin segfault, EA24 finally allowed re-enabling clojure.core.async-test
    • Added smoke tests for httpkit, nextjournal/markdown, clj-yaml, core.async ioc-macros
    • Updated 10M loop benchmark numbers for EA21
    • Added Windows test status notes (still some failures on EA22)

  • squint: CLJS syntax to JS compiler

    • Released 0.10.186, 0.11.187, 0.11.188 and 0.11.189
    • Add multimethod support: defmulti, defmethod, get-method, methods, remove-method, remove-all-methods, prefer-method, prefers, plus hierarchy ops isa?, derive, underive, make-hierarchy, parents, ancestors, descendants (#806)
    • cljs.test/report is now a multimethod, extensible via defmethod. test-var now fires :begin-test-var / :end-test-var events.
    • Accept plain await in async functions, in anticipation of CLJS next. The legacy js-await and js/await forms continue to work as aliases for now.
    • Add built-in cljs.test / clojure.test support: deftest, is, testing, are, use-fixtures, async, run-tests
    • Fix with-meta now preserves callability when applied to a function
    • #783: auto-load macros from .cljc files via :require (no need for :require-macros); resolve qualified symbols from macro expansions
    • #784: resolve transitive macro deps and auto-import runtime deps from macro expansion
    • #809: add squint.compiler/compile* and squint.compiler/transpile* which accept either a string or a sequence of pre-parsed forms, skipping the forms -> string -> forms roundtrip for SSR use cases
    • #810: shorthand classes in #html / #jsx were erased when an attrs map was present without a :class key
    • Full changelog

  • cherry: Experimental ClojureScript to ES6 module compiler

    • Accept plain await as a special form, in anticipation of CLJS next
    • Multiple :require-macros clauses with :refer now properly accumulate instead of overwriting each other
    • Add cherry.test with clojure.test-compatible testing API: deftest, is, testing, are, use-fixtures, async, run-tests. Macros are compiler built-ins (shared with squint), so no :require-macros plumbing is needed in user code.

  • nbb: Scripting in Clojure on Node.js using SCI

    • Released 1.4.207
    • #408: support IFn on defrecord and reify
    • Fix REPL and nREPL not awaiting promesa thenables (e.g. p/then results)

  • fs: file system utility library for Clojure

    • Released 0.5.32 and 0.5.33
    • #232: add touch fn (@lread & @borkdude)
    • #197: docstring review: consistent arg naming, improved docstrings, added Coercions and Returns / Argument Naming Conventions sections to README (@lread)
    • #231: get/set attribute functions were never following links. They now respect the :nofollow-links option (@lread)
    • #254: fix split-ext and extension on dotfiles with parent dirs (e.g. foo/.gitignore)
    • #202: gzip & gunzip now honor dest dir when specified (@lread)
    • #215: document effect of umask on created files and directories (@lread)
    • #182: enable soft & hard link tests on Windows (@lread)
    • #242: test: add JDK 26 to CI test matrix (@lread)

  • clerk: Moldable Live Programming for Clojure

    • Improve analysis performance by bumping weavejester/dependency (#808)
    • Bump SCI to v0.12.51 (#793), enables async/await in viewer functions
    • Improve presentation performance (#803)
    • Remove bb-specific code for array-map data structure (#805)
    • Preserve TOC opts (#806)
    • Remove redundant declare of present+reset! (#809)
    • Fix build-graph crash on non-Clojure source files (#810)

  • edamame: configurable EDN and Clojure parser with location metadata and more

    • Released 1.5.38 and 1.5.39
    • parse-ns-form: include :use and :require-macros in :requires
    • Check if object is iobj before attaching metadata #141 #142

  • Nextjournal Markdown: A cross-platform Clojure/Script parser for Markdown

    • Released 0.7.225
    • Add option :disable-footnotes true to disable parsing footnotes #67

  • quickdoc: Quick and minimal API doc generation for Clojure

    • Released 0.2.6
    • #42: fix var name not recognized in docstring when preceded by multiline backtick expression
    • #52: fix formatting of function signature when :or destructuring uses namespaced keyword fallback value
    • Dedent indented docstrings before rendering #53

  • grasp: Grep Clojure code using clojure.spec regexes

    • Released 0.2.5
    • Bump SCI to 0.12.51, Clojure to 1.12.4
    • Upgrade CI to GraalVM 25; move Windows CI from Appveyor to GitHub Actions
    • Fix bug in native which dropped all match results (@bsless)
    • Fix circular reference in grasp.impl

  • babashka.nrepl: The nREPL server from babashka as a library

    • Lock output stream in send to prevent interleaved bencode frames from concurrent writes
    • info and lookup op refinements: lookup carries nested info map whereas info is a flatmap

  • pod-babashka-instaparse: instaparse from babashka

    • Expose add-line-and-column-info-to-metadata
    • Drop macOS Intel builds, now building for macOS aarch64 only
    • Migrate Windows CI from Appveyor to GitHub Actions
    • Upgrade CI to GraalVM 25
    • Add --features=clj_easy.graal_build_time.InitClojureClasses to native-image

  • instaparse-bb: Use instaparse from babashka

    • Released 0.0.7
    • Bump pod to 0.0.7
    • Add add-line-and-column-info-to-metadata and get-failure
    • Fix opts passing in parser (e.g. :output-format :enlive)
    • Support java.net.URL for grammars

  • babashka-sql-pods: babashka pods for SQL databases

    • Released 0.1.5 and 0.1.6
    • #74: add DB2 support (@janezj)
    • #72: handle concurrent requests (@katangafor)
    • Upgrade to Oracle GraalVM 25.0.2; bump next.jdbc, cheshire (Jackson 2.12 -> 2.20), PostgreSQL, MSSQL, HSQLDB, MySQL Connector/J drivers
    • Remove DuckDB support
    • #51: macOS binaries are now aarch64 only

  • http-client: HTTP client built on java.net.http

    • Replace defunct httpstat.us examples with httpbin.org in tests

  • neil: A CLI to add common aliases and features to deps.edn-based projects

    • Fix README instructions for dev installation (@teodorlu)

  • deps.clj: a faithful port of the clojure CLI bash script to Clojure

    • Released 1.12.4.1618
    • #145: support for installing in FreeBSD and Windows bash environments including MINGW64, MSYS_NT and Cygwin (@ikappaki)
    • Catch up with Clojure CLI 1.12.4.1618

Contributions to third party projects:

  • ClojureScript:
    • CLJS-3470: added async/await support (merged!)
    • CLJS-3476: added async deftest support (merged!)
  • weavejester/dependency: improve performance of depend, depends?, and topo-sort
  • weavejester/cljfmt: #404 babashka compatibility via new text-diff lib (merged!)
  • Engelberg/instaparse: submitted #242 for babashka compatibility. Required :bb reader conditionals to replace the AutoFlattenSeq deftype with plain vectors plus metadata markers, swap the Segment deftype for a reify-based CharSequence, and add a CI test runner. Open, awaiting review.

Other projects

These are (some of the) other projects I’m involved with but little to no activity happened in the past two months.

Click for more details

Permalink

Copyright © 2009, Planet Clojure. No rights reserved.
Planet Clojure is maintained by Baishamapayan Ghose.
Clojure and the Clojure logo are Copyright © 2008-2009, Rich Hickey.
Theme by Brajeshwar.