Learn linear least squares with me

Permalink

Making Magic stable

Rationale

MAGIC (Morgan And Grand Iron Clojure) compiles Clojure to .NET so we can run it in Unity, including on iOS. When its creator Ramsey Nasser no longer had time to maintain it, I consolidated his six repositories into one monorepo under our Flybot org. I then improved the tooling around the compiler, which helped me fix bugs faster, and improved the integration in Unity, which was the whole point of the compiler in the first place. This article is about the decisions behind all that, not how the compiler works. The docs cover the how.

ClojureCLR vs MAGIC

The first question is always why not just use ClojureCLR, David Miller's mature Clojure-to-.NET port, which runs well on desktop. Its dynamic dispatch goes through the DLR (Dynamic Language Runtime), which builds each call site by emitting IL at runtime through System.Reflection.Emit. IL is the bytecode the .NET runtime executes, so this is a form of JIT (Just-In-Time) compilation: new executable code is produced while the program runs.

The problem is that Unity's IL2CPP backend compiles everything to C++ ahead of time, so there is no runtime left to execute IL that was generated on the fly. iOS forces IL2CPP, because Apple forbids any third-party JIT, and Android forces it too, not through a JIT ban but because Google mandates 64-bit and Unity's Mono has no ARM64 build. Consoles have the same constraints. So ClojureCLR is just not an option if you want to build your Unity app for anything other than desktop, which is pretty much everybody really. That constraint is what pushed Ramsey to create his own compiler that writes all IL at build time, so the IL2CPP transpiler has everything it needs to generate its C++.

How we use it at Flybot

At Flybot we helped port a client's old Java game libraries to Clojure. Then, because we knew MAGIC already existed, we took on the harder task of making those Clojure libraries run as .NET DLLs inside Unity. The payoff is that the same game APIs run in both the server backend and the Unity frontend, written once. I worked closely with Ramsey across two stretches, first on performance and then on stability (the earlier story), until those games shipped in production. I was doing the bug reporting, he was fixing the compiler.

The compiler worked fine for the most part, but the toolchain around it was painful. Six repositories, each with its own version and no shared release. Ramsey's time for it had become limited, so bugs could sit a while. And the internals were undocumented, with no public dev workflow, so contributing meant first reverse-engineering how the pieces fit. Our gaming platform frontend team actually came up with quite a few workarounds over the years. By the time I took over the compiler, their repos still carried patches just to get MAGIC to compile and integrate with Unity, both in the Clojure libs (ported to the CLR) and on the Unity side. I wanted to get rid of these patches by improving MAGIC directly, so everybody could benefit from it in the future.

1. Gather the six repos into one

The first step was to gather everything in one place so I could add proper project tasks, proper CI, and therefore a more convenient dev workflow.

A monorepo was the obvious choice here because these six repos always worked as one system. One version instead of six, one place to file bugs, and the freedom to land a compiler change, the runtime tweak it needs, and a stdlib fix in a single PR, instead of coordinating three separate repos. The diagram below shows the merge:

flowchart LR
    m1["magic"] --> gfr
    m2["mage"] --> gfr
    cr["Clojure.Runtime"] --> gfr
    mr["Magic.Runtime"] --> gfr
    no["nostrand"] --> gfr
    mu["Magic.Unity"] --> gfr
    gfr{{"git-filter-repo<br/>(full history kept)"}} --> mono["flybot-sg/magic<br/>one repo · one version · one CI"]

I used git-filter-repo to merge the six trees while keeping every author and commit date, going all the way back to 2009 since the runtime carries David Miller's history from its ClojureCLR fork. So the history itself credits the extensive work of Ramsey, of David Miller, and of everyone who contributed.

It also lets anyone trace a bug back to the commit that introduced it. A human or an LLM can bisect far faster when the entire history of every piece sits in one place.

2. Build tooling instead of becoming a compiler expert

I am not a compiler expert but I still had a plan. The best move was to make the compiler understandable by anyone. Everything runs as a Babashka (bb) task. I really like Babashka and it works very well for monorepos (see Clojure Monorepo with Babashka).

In order to understand a bit more what is going on when I compile something, I created two tasks:

• bb pipeline walks a form through macroexpansion, the AST, and the symbolic IL
• bb prepl-eval runs a form against a live MAGIC runtime.

Between them, that is usually enough to see where something goes wrong without reading the compiler internals.

For example, for (+ 1 2), bb pipeline shows how it compiles, walking the form through macroexpansion, the AST and the type stages down to the symbolic IL the emitter produces:

$ bb pipeline '(+ 1 2)'

================================================================
FORM   (+ 1 2)
================================================================

================================================================
MACROEXPAND
================================================================
(. clojure.lang.Numbers (add 1 2))

================================================================
AST (skeleton)
================================================================
{:args ...
 :method #object[RuntimeMethodInfo 0x6ab6fcd0 "Int64 add(Int64, Int64)"],
 :original ...
 :type System.Int64,
 :op :intrinsic,
 :il-fn #object[<magic>magic_intrinsics$add-mul-compiler__0 ...],
 :form (. clojure.lang.Numbers (add 1 2)),
 :target ...}

================================================================
TYPES (4 typed nodes)
================================================================
  :intrinsic (. clojure.lang.Numbers (add 1 2)) :: System.Int64
  :const 1 :: System.Int64
  :const 2 :: System.Int64
  :const clojure.lang.Numbers :: :class

================================================================
SYMBOLIC IL (3 instructions)
================================================================
  ldc.i8 1
  ldc.i8 2
  add.ovf

There is no Var lookup and no IFn.invoke: + inlines to the static Numbers.add call, which the compiler recognises as an intrinsic, a call it knows how to emit directly, so the whole form lowers to three CLR instructions. Very cool.

bb prepl-eval is the other half, running the same form on a live MAGIC runtime and handing back a structured reply:

$ bb prepl-eval '(+ 1 2)'

{:tag :ret, :val "3", :ns "user", :ms 2.1492, :form "(+ 1 2)"}

With these two tasks, we can see both what the compiler emits, as pure data, and what it actually does, which is most of what I need to localise a bug. It also pays off with LLMs: given these two tasks, Claude Code finds the origin of a bug way faster than by digging through the compiler code. It is truly impressive.

There are quite a few other tasks, used in CI among other places; you can find more about them in docs/development.

3. Drift check the bootstrapping

MAGIC is bootstrapped, which means it uses a previous version of itself to compile its next version (see docs/bootstrap). MAGIC commits the emitted DLLs alongside the Clojure source code and the C# runtime, so it can build the next version from these assemblies. Committing a bug fix therefore requires two things:

• the Clojure or C# source change
• the new DLLs that contain the fix

So I have a bb check-drift task that checks that nothing is stale (including that the DLLs were regenerated, among other things). I just needed to be sure the new DLLs were committed alongside the source change.

However, I could not easily know which DLLs were really affected by a source change, since they all came out different on every rebuild: the compilation was not deterministic. So I worked on making it deterministic (see the repo doc Deterministic compilation and the drift check). This allowed me to byte-diff the DLLs! This is really valuable, because it means I can see exactly which DLLs are impacted by any source change.

More details on how I made the compiler deterministic and its caveats in the article: Drift Checks for a Self-Hosting Compiler.

4. Catching IL2CPP bugs

Ramsey had told me that the IL2CPP documentation is sometimes incomplete and even wrong, so a lot of the behavior has to be inferred by testing and disassembling what it produces. The consequence: some code runs totally fine on Mono and fails only when we build with IL2CPP.

Rather than keep rediscovering those failures inside our large game projects, I built a standalone Unity project that collects a minimal repro of every IL2CPP edge case we have hit so far: nine suites, 90 checks, all green on Mono and on a Standalone Mac IL2CPP build. Every time a fix is suspected to behave differently under IL2CPP, its repro lands in the suite in the same commit.

Example of divergence: overriding ToString on an anonymous object like so:

(.ToString
 (reify System.Object
   (ToString [_] "from reify")))
;; Mono: "from reify"
;; IL2CPP: the build itself dies

MAGIC emitted the reify class with System.Object listed both as its base type and in its interface list, which is invalid metadata, since Object is a class, not an interface. Mono loads the class without a word and runs it correctly. UnityLinker, the code stripper Unity runs during every IL2CPP build, walks that interface list to mark methods, meets the class's own base type there, and recurses until it overflows its stack. The build dies before producing a player, so no test that runs on Mono can ever see this bug: only an actual IL2CPP build surfaces it.

The suite is the one piece that does not run in CI, because the IL2CPP build needs a machine with Unity installed. So I run it by hand after any suspect fix.

5. Foundation first, then the backlog

Only with the monorepo, tooling, CI, and smoke suite in place did I start on the bugs that had been open on Ramsey's repos for years. Each commit references the issue it closes, including the original nasser/* numbers, and the conventions in CONTRIBUTING.md mean a human or an LLM can file and fix without re-asking how we work.

The releases came fast once the base held:

timeline
    title MAGIC release arc (May to August 2026)
    v0.1.0 May 22 : Monorepo, bb tooling, CI, IL2CPP smoke
    v0.2.0 May 23 : Compiler and stdlib bug fixes
    v0.3.0 Jun 01 : Clojure 1.10 stdlib, magic.flags
    v0.4.0 Jun 04 : Native deps.edn in Nostrand
    v0.5.0 Jun 04 : Consumer quality-of-life
    v0.6.0 Jun 07 : Unity editor/player coexistence
    v0.7.0 Jun 09 : Dual Unity package
    v0.8.0 Jun 24 : Compiler fixes, bootstrap drift guard
    v0.9.0 Jul 08 : deps-clr.edn and magic.edn, by-ref fix
    v0.10.0 Jul 14 : Deterministic compilation, byte-diff drift
    v0.11.0 Jul 24 : Constant and integer-promotion fixes, per-test skip
    v0.12.0 Aug 18 : One Unity package, editor runtime by define

Versioning is one version.edn, and bb tag creates the tag that a CI job turns into a published release tarball on GitHub. One command, and a release builds and ships itself with nothing done by hand. That predictable, hands-off release path is what the single shared repo finally makes possible. Per-release detail is in the CHANGELOG.

I was happy to see that for the first time, I was able to use David Miller's clr.test.check as is with MAGIC! Before, I had to comment out its clojure.core require and rewrite every core/let to its fully qualified form, just to dodge a MAGIC bug. After the v0.2.0 fixes, his port compiled under MAGIC with zero source patches, sooner than I expected. Then, testing against our own libraries, I found that some workarounds were still necessary, because MAGIC had never been fully ported to Clojure 1.10. So v0.3.0 filled that gap and put every compiler option behind one magic.flags namespace.

So latent bugs were fixed and Clojure 1.10 fully ported: good progress. And yes, Claude Code clearly helped me find bug sources and suggest fixes, using the bb tasks I made it write when I took over the repo.

6. Managing dependencies

MAGIC is one of these old projects that predate deps.edn! So Ramsey made his own resolver that reads a project.edn. Nostrand is the runtime environment that loads MAGIC and executes tasks (via nos), including the deps resolver. Since MAGIC was more stable and on par with Clojure 1.10, it was the right time to modernise its dependency handling: get rid of the dedicated project.edn deps files and support deps.edn.

The obvious first task was to adopt David Miller's CLR port of tools.deps (clr.tools.deps), but it did not load as-is on MAGIC's Clojure 1.10 base: .cljr files were not recognized yet, and it calls a few stdlib functions newer than 1.10. Adopting it meant maintaining a compat fork and re-applying the patches on every upstream sync, which was not worth it.

Our need was narrow anyway: resolve git and local coordinates transitively, skip Maven, and authenticate through the developer's own git and SSH config. So I wrote my own resolver first, then aligned it with the ClojureCLR conventions.

A native deps.edn resolver (v0.4.0, v0.5.0)

I added native deps.edn resolution, one file for both JVM and CLR runtimes, with a :clr alias that swaps a JVM library for its CLR fork via :override-deps. It worked well, and I found it quite clean to have a dedicated alias carry the JVM-only / CLR-only mapping. However, that was not how the existing ClojureCLR community did it. David Miller's convention is a dedicated deps-clr.edn file that is read in place of deps.edn. It is a bit more verbose, but it is convenient for loading different paths per platform, notably a precompiled-assembly loader namespace that the CLR must load and the JVM must ignore.

deps-clr.edn, the file the CLR community already writes (v0.9.0)

David Miller's cljr, the ClojureCLR CLI, reads a deps-clr.edn in place of deps.edn when it is present, and that is where the CLR community already writes its CLR-specific dependencies. So I made nos prefer it the same way. Now both the cljr and nos CLIs resolve deps-clr.edn, so a library already ported to the CLR for ClojureCLR builds the same with nos (assuming no core functions above 1.10). This was a necessary milestone to unify the effort of porting libraries to the CLR. I recently ported robertluo/fun-map to the CLR: it carries a deps-clr.edn and its CI runs the tests with ClojureCLR, matching the existing convention. And fun-map also builds with MAGIC as is, which is really nice.

The CLR dependency flow is documented in docs/clr-dependency-files.

magic.edn, build and test config (v0.9.0)

However, for the test runner, I could not follow the ClojureCLR way. We could not use David Miller's CLR port of Cognitect's test-runner because its dependency chain bottoms out in clr.tools.reader, which reads record literals through runtime reflection (ClojureCLR's Reflector class), and MAGIC deliberately ships no runtime reflection since that is exactly what IL2CPP forbids.

The other MAGIC-only file was the dotnet.clj build helper. So nos build and nos test became built-in tasks that read an optional magic.edn, a small map where a project states only what differs from the defaults. A library that needs no tweaks omits the file; the hand-written dotnet.clj is gone.

So a lib still specifies the io.github.dmiller/test-runner port in its test deps to run tests with cljr, and adds a small magic.edn file at its root to run them with nos.

The full guide is in docs/porting-libraries-to-magic.md.

7. The right runtime per phase, in Unity

With consumers able to build and depend on CLR libraries cleanly, the last piece left was the one we actually ship into. The right arrangement was not my idea: Hong, an engineer on our client's Unity team, had arrived at it out of necessity: Run ClojureCLR in the editor, where it compiles Clojure from source in memory so hot reload works, and run MAGIC only in the player build, where its static IL is what IL2CPP needs. I wanted this setup in a UPM package that ships both runtimes with the proper defineConstraints in their .meta files, to avoid conflicts in the Unity editor.

This took some time and was actually only available in version 0.12.0. My first draft was one package that ships MAGIC only, meant to be used in both the editor and the player build. The downside of course was that it was slow, because changing a Clojure source file required a full AOT compilation and reset the scene on every change.

So then, since the Unity team was using a fork of ClojureCLR 1.11 in the editor, I generated a second package variant whose MAGIC DLLs carry a !UNITY_EDITOR constraint, so the editor never loads them and their ClojureCLR DLLs work as usual.

This was not ideal of course, so a colleague of mine looked into packaging both runtimes while letting the Unity consumer project choose which one the editor loads. The solution was a single scripting define symbol, MAGIC_RUNTIME_IN_EDITOR, whose constraint applies to the MAGIC DLLs, the ClojureCLR DLLs, and, through a reconcile pass after each domain reload, to any ported Clojure libs present under Assets/Plugins.

We actually had to fork ClojureCLR to fix a few bugs, for reasons I detail further down this article.

The consumer setup is documented in docs/unity-integration.md.

8. Test each version of MAGIC on 30+ repos

My goal was to be able to compile with MAGIC all the libs our Unity gaming platform depends on, without custom forks carrying workarounds just to make them compile. I wanted to be able to use David Miller's CLR ports right away (as long as they use no features above Clojure 1.10), and to run rich comment tests (RCT) on the CLR, since most of our recent internal libraries use them for unit tests.

rct-clr: rich comment tests on the CLR

rich-comment-tests puts a function's example calls and their expected results in a (comment ...) block and runs them as assertions, keeping the documentation and the tests as one thing. The problem is that the library relies heavily on the JVM, so it is not easy to port with just interop. So my colleague Parth had the idea to extract the assertions on the JVM and emit a plain .cljc file of ordinary deftests, which nostrand can run. That became rct-clr: the (comment ...) blocks stay the single source of truth, and the CLR runs the very same assertions as the JVM, just in a generated file of deftests instead of via the RCT runner.

magic-conformance: does it still build under MAGIC?

While working on MAGIC, I want to be sure that all the libs our gaming platform depends on compile and test OK on the latest MAGIC release. So I wanted an easy way to rebuild and re-run the tests of each of our libs in these two scenarios:

• the MAGIC version got bumped: I want to be sure there is no regression
• a lib SHA moved: I want to be sure it still compiles with MAGIC

magic-conformance is a runner that reads a manifest of libraries and, for each, clones it and runs its nos build and nos test on the ci-clj-clr image, which carries the JVM, MAGIC, and ClojureCLR toolchains. A manifest entry can also carry an inline magic.edn or deps-clr.edn, which the runner writes into the clone when the library ships none of its own.

flowchart TD
    L["manifest<br/>(the libraries)"] --> R["conformance run"]
    R -->|per library| S{"commit + MAGIC version<br/>+ config unchanged?"}
    S -->|yes| K["reuse cached result"]
    S -->|no| A["clone, inject config"] --> B["nos build + nos test<br/>+ cljr -X:test when declared"]
    B --> W[("results")]
    K --> W

The public repo ships the runner with a small green example manifest, a few public libraries such as fun-map that build under MAGIC straight from upstream. Internally, we run magic-conformance on around 30 repos to be sure they compile with both MAGIC and our fork of ClojureCLR 1.11.

It is not every open source library being recompiled, like Rust does with its Crater, but that is a good start!

Our fork of clojure-clr

To recap our setup, we actually need three compilers:

• JVM Clojure on the server
• ClojureCLR in the Unity editor
• MAGIC in the Unity player build

In one MAGIC release, I shipped a fix for a bug where datafied class names came out in their short form (String) where the JVM uses the fully qualified name (System.String). JVM and MAGIC tests were green, so a consumer lib deleted its own datafy workaround. Then a Unity dev reported that their editor broke: ClojureCLR has the same datafy bug, and ClojureCLR is what runs in the editor, so the editor hit the bug the workaround had been hiding, while player builds stayed green. That is when I realised that for this dual CLR compile workflow to work in Unity, I needed to guarantee similar behaviour from both compilers.

So I forked ClojureCLR 1.11 (the version the client was using) into flybot-sg/clojure-clr, and with my colleague we added a few things:

• the post-1.11 backports the cljr CLI needs to run
• later fixes David Miller shipped upstream, backported to 1.11
• our own fixes (including the datafy fix mentioned above), found while running ClojureCLR next to MAGIC

I also added ClojureCLR to our ci-clj-clr image, so clients can run their backend libs with cljr and my conformance check can run all our libs with cljr as well.

9. Documentation

I tried to have an LLM generate the docs for me, and it was bad. So I did almost all of it manually first, and let the LLM fix the usual typos and generate the Mermaid diagrams, because diagrams help me understand. So the doc is written for humans and understood by LLMs.

Internally, I made a Claude Code plugin with skills to port a lib to the CLR with both MAGIC and ClojureCLR. I have not made it public yet, because I am aware of the skepticism of some in the Clojure community about LLMs, and I am still polishing it anyway. The skills mainly refer to the docs of the magic repo, so it should be easy for anybody to write their own. ClojureCLR is the reference compiler and the most up to date with upstream Clojure, so when I port an open source lib to the CLR, I use cljr and not nos. For people who want to run their Clojure lib in Unity, I advise using our ci-clj-clr image and running the tests on both compilers.

What is next

The next real effort is dropping Mono for CoreCLR, which Unity is moving to and which Nostrand still predates. We also plan on porting Clojure 1.11.

Permalink

Drift Checks for a Self-Hosting Compiler

Why bother making it deterministic

MAGIC is a compiler. It turns Clojure into .NET so Clojure can run in Unity, even on iOS. To do its job, it commits some of its build outputs straight into the repo, including the compiler's own compiled files. You can read more about it here Making Magic stable. At first the compilation was not deterministic, so a small change to the compiler would generate different DLLs during bootstrap which made the byte diff impossible. Technically it is fine, but having the compiler deterministic unlocks some interesting features such as doing the drift check via a byte diff directly and being able to see at a glance which DLLs are impacted by a change.

What led to that decision

I wanted to always have a pair of commits: the source change and the bootstrap containing only the DLLs impacted. But since the compiler was not deterministic and all DLLs had moved, I was handpicking the ones I thought were impacted based on what the source change was. That works fine when it is, say, a Clojure fix in the Clojure compiler, but it is very hard to "guess" when it is a runtime change that could subtly impact a lot of DLLs.

In bc629a67 I changed how the C# runtime hashes strings and maps to match JVM Clojure. The change reached binaries I never considered, 17 stdlib ones affected indirectly through hashes baked into their compiled bytes, and since their .clj sources never moved, nothing flagged them. The code compiled, the tests passed, the review was green. Seven weeks later every clojure.spec.alpha regex op threw No matching clause (#40), and the trail led back to that hash change; b70ac965 is the patch that refreshed the binaries my selection had missed.

As I was explaining this to my colleague, he mentioned very casually "why not make the compiler deterministic then". And this is how I decided to look into it.

Why the compiler commits its own output

A compiler turns source code into something else, and MAGIC turns Clojure source into compiled .dll files (a .dll is the .NET unit of compiled code). Other compilers rebuild those files on every build, so they always match the source. MAGIC cannot do that, because the compiler is built from its own previous output. It compiles itself, which means its self-hosted .clj.dlls have to be committed and then reused to build the next version (the C# parts rebuild like any project though). It is called bootstrapping.

A committed file is a snapshot of its source at one moment in time. Edit the source without rebuilding, and that snapshot quietly becomes a lie, a drift. And catching this drift is almost trivial with determinism: just a byte diff.

Drift check attempt before determinism

Before determinism, comparing the binaries was off the table: every rebuild changed every DLL's bytes, so a diff against the committed ones would fail on every commit and prove nothing. What I had instead were two workarounds, one for picking the DLLs to commit and one for catching a forgotten rebuild. Let's look at these workarounds first.

Claude Code picked the DLLs to commit

Since every rebuild left every committed DLL modified, choosing which ones carried a real change was a judgment call, and I delegated the judgment to Claude Code: given the source change, reason about which binaries it can reach, commit only those, revert the rest. It was impressively good at this. For the :extend-via-metadata port (2af529c5), it refreshed core_deftype, clojure.core.protocols and the C# Clojure.dll in one paired commit, three artifacts for one logical change. Even the stdlib miss from earlier was very likely my fault, not its analysis: I did not suspect that the stdlib DLLs could be affected by the C# change, so my prompt never put them in scope.

But I could not trust the LLM to always be right. There was no tool that could have caught a wrong pick. Plus, it is bad software engineering practice to rely on an LLM for a check that must be deterministic. An LLM should help us write the deterministic tooling, not be the tooling. In our example, we should use Claude Code to help us find how to make the compilation deterministic, so drift is caught by a predictable CI pipeline instead of a judgment call.

A manifest of source fingerprints

The second workaround guarded against forgetting the rebuild entirely. A committed manifest maps each binary to a fingerprint of its .clj source (a SHA hash, a short string that changes whenever the file changes). Back then it was two files, stdlib-manifest.edn and bootstrap-manifest.edn; determinism later merged them into one dll-sources.edn. The entry for clojure.core looked like this:

clojure.core {:source "magic-compiler/src/stdlib/clojure/core.clj"
              :sha256 "33435bc12bcc893ebe91319b5cefa21aa6cfb31254b5269c4cc687feedebb1d2"}

I had a refresh task that rewrote the manifest, in the same run that rebuilt the DLLs. CI then re-hashed every source and compared against it. It answers only one question: does this source change come with its updated DLL? It is not much, but at least it forces the developer to run the bootstrap and commit its output.

What neither workaround could prove

Neither workaround ever looked at the committed bytes. The hasheq incident is the proof: a case over keywords bakes each key's hash into a jump table inside the DLL, so a C# change to hashing invalidates binaries whose .clj sources never moved, and no source hash can see it. The diagram below shows the blind spot:

flowchart TD
    SRC["Source .clj"] --> BIN["Compiled binary"]
    RT["Runtime hashing, in C#"] --> BIN
    SRC -->|watched| FP["Fingerprint says: in sync ✅"]
    RT -->|not watched| GAP["Blind spot ❌<br/>runtime changed, source did not"]

The honest fix is a byte diff of the binaries themselves, and that is impossible while the compiler is non-deterministic. So the real work became removing the non-determinism.

Making the build deterministic

Deterministic here means one thing: identical inputs produce identical bytes. Five things stood in the way, each found the same way, by compiling the same namespace twice (and later, on two different machines) and diffing the disassembled output. The full site-by-site inventory is in docs/deterministic-compilation.md; here is what each one was.

Class member order

Take one small form:

(reify
  System.IDisposable (Dispose [_] ...)
  Object             (ToString [_] ...))

Compile it twice in a row and diff the disassembly of the generated class:

 .class private '<magic>user$reify__0'
-  .method public Dispose  ...   ; body at IL offset 0x2050
-  .method public ToString ...   ; body at 0x20c4
+  .method public ToString ...   ; body at 0x2050
+  .method public Dispose  ...   ; body at 0x20c4

Both DLLs hold the same two methods, but the order the class members are emitted in decides where each body lands in the file, so one swapped pair shifts every byte after it and the two files diff from that point on.

What happens is that each method in a reify implements a method that already exists on an interface or base class, so the compiler pairs every body with the host method it implements, in a map keyed by that method's MethodInfo object.

A MethodInfo has no value to hash by, so its hash code is just an arbitrary number the runtime assigns to that object per process; same method, new process, new number, so the map iterates in a different order in every process. Maps keyed by Type or Var objects have the same problem, and this was not one bug but a pattern: five places in the emitter iterated collections that way. Each now sorts its entries by a key built from the content itself. For a method it hashes the full signature string, so every process emits in the same order. The doc linked above lists all five.

String sort order

The CLR's default string compare does not behave the same across OSes. String.CompareTo does not compare character codes: it asks the platform's collation library how the user's language orders these words, Windows' NLS or ICU on macOS and Linux, each machine shipping its own version of the rules. Collation is linguistic (it files "a" before "B", where a code-unit compare puts every uppercase letter first), and which rules run, in which version, depends on the machine, so the same sort could emit different bytes on macOS and Linux. Every sort in the emitter now compares ordinally, raw UTF-16 units via String.CompareOrdinal, which is arithmetic and therefore the same everywhere.

The CLR does offer the ordinal compare, but only as an explicit opt-in at each call site, and that is the trap: the bare String.CompareTo is what IComparable binds, so every generic path, a default comparer, a plain sort, Clojure's own compare, silently gets the culture-sensitive one. There is not even a parameter to forget, because generic code offers no place to pass it. Most languages put the defaults the other way around, the JVM included: Java's String.compareTo is defined as code-unit comparison, and locale-aware sorting is the explicit opt-in. So the emitter carries its own ordinal comparator, and the fix simply mirrors the JVM compare behavior.

Source paths

Every def bakes its source's :file path into metadata, and it used to be the absolute path, so the bytes depended on where the repo was cloned. It is now the load-relative path, clojure/zip.clj instead of /home/me/.../clojure/zip.clj for instance, matching JVM Clojure. Old committed DLLs could be dated by this alone: some still carried Ramsey's /home/nasser/... paths in their bytes from years ago.

Generated names

Every anonymous fn compiles to a generated type, and those types are numbered by process-global counters, gensym included. Everything nos does before compiling your file consumes them: booting compiles nostrand's own Clojure in memory, resolving dependencies runs more. So any edit to that prelude shifted every number that every later file baked.

For example, one edit to nostrand/core.clj renumbered every committed stdlib DLL, the only difference in each being __49 becoming __50. The counters now reset at each file-writing compile. So an emitted name is a function of the namespace and the toolchain alone; REPL evals keep the process-global counter and its uniqueness guarantee. The diagram below shows both regimes:

%% mermaid lays disconnected subgraphs out perpendicular to the parent
%% direction, so "after" is declared first to render "before" on the left
flowchart TD
    subgraph after["after"]
        direction TB
        a1["one fn added early"] --> a2["every file starts numbering<br/>from the same reset value"]
        a2 --> a3["only DLLs whose own<br/>source changed differ"]
    end
    subgraph before["before"]
        direction TB
        b1["one fn added early"] --> b2["the shared counter is one<br/>ahead from there on"]
        b2 --> b3["every DLL compiled after it<br/>changes: __49 becomes __50"]
    end

When I described the reset to Ramsey Nasser, he immediately said it could collide, without looking at the code. He was right: a process-global counter guarantees a gensym name is fresh for the whole process, and resetting per file lets two files mint the same one. Equal names are harmless where they are scoped, and locals and type names both are, but a namespace split across files whose macro defs a gensym-named var is not:

;; main.clj
(defmacro defslot [reader v]
  (let [g (gensym "slot")]
    `(do (def ~g ~v) (defn ~reader [] ~g))))

(load "part_a")   ;; (defslot read-a :from-a)
(load "part_b")   ;; (defslot read-b :from-b)

Each sub-file compiles as its own unit, so both reset to the same value, both mint slot10001, and part_b's def silently shadows part_a's: (read-a) returns :from-b. No stdlib or compiler macro writes that pattern, so nothing in the tree hits it, but MAGIC compiles anyone's code, and the trigger is ordinary: a plain nos build of a namespace split across (load ...) files is enough, no special driver. The possible fix would be to seed the counter per file, or to fail the build when a gensym-shaped var is redefined across files of one namespace. It's a niche case but I will address it.

Assembly ID and timestamp

Two values in the emitted file say nothing about the code:

• the build timestamp
• MVID, a GUID identifying the module that gets randomized on every run.

Reflection.Emit offers no option to control either, where the C# compiler has -deterministic, so the only way is to patch the bytes ourselves once the assembly is saved. The timestamp is easy, a fixed offset in the PE header, the container format .NET assemblies use. The MVID is not in that header at all: it sits in the metadata #GUID heap, so reaching it means walking the section table and the stream headers by hand.

Then the trick, in the arrows below: a file cannot contain the hash of itself, so both fields are cleared first, and the hash of that cleared file becomes the MVID.

flowchart LR
    A["the .clj.dll as saved"]
    B["timestamp and MVID zeroed"]
    C["the .clj.dll rewritten in place"]
    A -->|"clear both fields"| B
    B -->|"SHA256 the whole file,<br/>write it into the MVID slot"| C

Together, the bytes are a function of the source and the toolchain, nothing else. A Linux container running mono 6.12 rebuilds the 73 committed compiler and stdlib binaries byte-for-byte from a tree committed on macOS with mono 6.14.

The first real audit of the now-comparable bytes also settled how much the old checks had missed: it found seven committed DLLs that no build flow ever rebuilt, the oldest untouched since 2020, one of them from a source file that could no longer compile at all (#45). Five years of green checks, and no judgment, no review, and no hash had any way to see it. The byte-diff did, on its first run.

With that, the special case for compiled binaries disappears: the check rebuilds everything and byte-compares the lot, binaries included.

What the check does today

All of it runs from a single command, bb check-drift, on every change in GitHub CI, right after a fresh build. It regenerates the C# call sites MAGIC pre-generates for IL2CPP (committed .g.cs files, plain text, so these were always byte-comparable), recompiles and redeploys the stdlib binaries, syncs the Unity version, re-authors the Unity package's runtime-selection constraints, then asks git one simple question: did any tracked file change?

Because the build is deterministic, that question now covers the committed binaries too, byte for byte (the two runtime DLLs that embed a git-derived version stamp are restored from the commit instead, since their bytes change with every commit by design).

If the answer is yes, something was not refreshed, and the build fails, listing exactly what drifted.

flowchart TD
    Start["bb check-drift<br/>(after a fresh bb build)"] --> S1["Regenerate the C# from templates"]
    S1 --> S2["Recompile + redeploy the stdlib binaries"]
    S2 --> S3["Sync the Unity version number"]
    S3 --> S4["Re-author the Unity DLLs'<br/>runtime-selection constraints"]
    S4 --> Q{"Did any tracked<br/>file change?<br/>(binaries byte-compared)"}

The fix and its binary travel together

The convention I used from the start is that for changes in the monorepo requiring new DLLs, I push a pair of commits: one for the source change and one for the bootstrap.

* 8c9a0d7e - fix(compiler): resolve inherited interface properties via interface walk
* 5da7deff - chore(bootstrap): refresh analyze-host-forms DLL for inherited interface property fix
* 7b639c7b - fix(compiler): resolve proxy-super base type from enclosing proxy
* 8ae40888 - chore(bootstrap): refresh typed-passes DLL for proxy-super shadowed-this fix

It is just a convention, both could go in the same commit really, but some commits do not affect the DLLs at all (updating the docs, a bb task, and so on) so I find it clearer to have two separate commits when a dll moves. Sometimes a third one follows, adding the smoke test that guarantees the fix under IL2CPP.

The payoff

The result is that we can see exactly which DLLs a source change affects, whether it came from the Clojure compiler or the C# runtime.

On top of that, a byte diff is enough as a drift check in CI to catch any missing bootstrap.

And we get usable versioning on the DLLs themselves. Since only the DLLs a change really touched ever move, git log on one of them lists the commits that actually changed it. So when a DLL misbehaves, the last commit that touched it, and the source commit paired with it, is the change that caused it.

Permalink

August 2026 Short Term Project Updates

Here are August’s updates for short term projects funded in Q2 2026. You can find overviews of these projects and the two others which will be reporting on a slightly different schedule in the original funding announcement. Thanks everyone for the awesome work!

Clojure LLM: Dragan Djuric
Gloat and Glojure: Ingy dot Net
PluMCP: Shantanu Kumar

Clojure LLM: Dragan Djuric

Q2 2026 Final Report 3. Published August 1, 2026

The proposal was (in short):

The goal is to provide a high performance local LLM (large language model) AI solution, that supports mainstream open models, freely available at Hugging Face and elsewhere. Something like llama.cpp (https://llama-cpp.com/), but (hopefully!) simpler and faster, with both GPU and CPU support baked-in from the start.

I even have a catchy name for the library: iLLaManati :)

iLLaManati should:

• work :)
• be very fast,
• have a very simple API (possibly even a NO-API if you use the default configuration),
• have a fairly elegant implementation with not many lines of code, which will be a great showcase for Clojure as an enabling technology, and a good learning source for Clojurians.
• integrate into the Clojure ecosystem naturally and seamlessly,
• NOT require Clojurists to know anything about CUDA, ONNX, tensors, or linear algebra, to be able to use it (will require some of that if you want to extend it, though!),
• run on your laptop, server, or cloud; wherever Clojure runs. It’s your choice.
• be a great low-effort gateway for Clojurists to peek, as users, into high-performance and GPU computing,
• be a very attractive topic to tell the world about!

TLDR results after 2006 Q2 funding

At the very end, i am quite pleased what I achieved for this first version. The project delivered on all proposed bullets (with caveats, of course, but still :)

iLLaManati:

• does work! I implemented model-agnostic code, and tested it with both official-ish Gemma 3 variants available on Hugging Face (onnx-community and onnxruntime)
• is very fast. On my 7 year old computer I get 16 tokens/second on the CPU (Intel i7-9900X), and on a very old GPU Nvidia RTX 2080Ti, 80 + tokens/second. I had not had time to do detailed benchmarks and apples to apples comparisons with established frameworks, but this is similar to the speed that llamma.cpp achieves on my machine. So, surprisingly, not bad at all, given that it runs on ONNX Runtime, which is not very well adapted to running LLM models!
• has a very simple API. Practically, no API. There’s just a function for loading the model config, and a couple extra functions that automatically plug the LLM generator and the tokenizer into the core.async or core.async Flow!
• has a fairly elegant implementation. If you look at https://github.com/uncomplicate/iLLaManati , you’ll see it’s 100% Clojure (yes!). The whole LLM implementation for several different model architectures, and both CPU and GPU engines is less than 1000 lines of Clojure, and 3 supported tokenizers take up 800. And since there’s lots of nitty-gritty low-level code, I can even have ideas how to further squeeze some LOC-s, I just didn’t have time to do it during this funding round.
• integrates into Clojure ecosystem naturally and seamlessly. What can I tell you, other than the only functions in the public API are the a couple functions that talk through core.async channels. Everything is automatized under the hood!
• NOT require Clojurists to know anything about CUDA, ONNX, tensors, or linear algebra, to be able to use it. Please see the examples here: https://github.com/uncomplicate/iLLaManati/blob/main/illamanati-onnxrt/test/uncomplicate/illamanati/internal/onnxrt/generator_test.clj That’s haiku-level code!
• runs on your laptop, server, or cloud; wherever JVM Clojure runs. It’s your choice.
• be a great low-effort gateway for Clojurists to peek, as users, into high-performance and GPU computing. Although looking at the final result it may seem to you it was walk in the park, I had to conquer many subtle issues with diverse complex technologies. There’s lot of real-world tensor/GPU/CUDA/native interop primers here. Of course, the only reason it’s so elegant is because it leverages the power of Clojure, and many Uncomplicate libraries. Compare that to any other equivalent on the internet.
• will be a very attractive topic to tell the world about. Unfortunately, it is not yet production ready since ONNX Runtime does not support many of the real world optimizations needed for server use, but I can solve that in the future by implementing backends in whatever LLM runner that offers C++ API! So, I’m optimistic that we’ll have a production-ready systems much earlier than an average Clojurian would be able to afford the proper hardware with enough memory for this kind of software :)

Now, some details about the work during these 3 months, especially the obstacles.

The heart: LLM runner

In the first and second month, I was focused a lot on the hammock, to learn the details of LLM implementations, and also several side quests to related topics. I also accomplished plenty of implementation, especially on the tokenizers and the logits sampler. I also created a LLM runner that worked, sort of, but needed lots of polishing and bugfixing.

In the third month, I concentrated on the implementation. In the first two months, I have already hit a few serious deficiences in ONNX Runtime when it comes to LLM support, so my hopes for a super-great solution were low. Specifically, ONNX Runtime lags when it comes to supporting high-performance implementations of more recent operations used in LLMs (such as those that Gemma 3 uses), and it generally does not expose the internals of such operations even when it supports them, so creating a server production-worthy general LLM with it is practically impossible now.

Too bad, you’d think, but hold your breath: even the state of the art runners, such as Nvidia’s TensorRT is like that, and that’s why Nvidia has a special runner, optimized just for LLM models: TensorRT-LLM. So, it’s not that ONNX Runtime is a bad model runner, it’s that LLM models are so specific that they require a more specialized runners. Bummer, ha? Not at all, since most of the code that I wrote during this project (and the lessons learned) fits perfectly into how LLM model works, and provide the parts of the solution orthogonal to specialized runners such as TensorRT-LLM, or llama.cpp.

So, in the third month I hoped to assemble what could be assembled and get at least a decent implementation. There were MANY obstacles. I hoped to leverage ONNX Runtime vendor execution providers, but it turned out both OpenVINO and TensorRT EPs were riddled with unsupported LLM operations, and heavy builds. I kept helping with upstream builds and bugfixes, but 5 hour builds are the norm there. At the end, I managed to make even these work, but their performance was much worse that stock CPU and CUDA EP so I stayed with stock ONNX Runtime. Also, information was scarce. Sometimes I felt I was the only one actually trying many of these code paths in practice, and I probably was :) Nevermind, these were long hours, but I learned lots of things, and, most importantly, REPL enabled me to debug my way out of most of that technical obstacles, so in the last seconds of the funding period, I managed to put everything into a very nice package. Maybe not suitable for production, but definitely something anyone can try on their machine and learn (it works on the CPU, too!).

The hammock

Lots of reading and thinking. And again.

Tokenizer

It turned out that the tokenizer I was using in the first two months (from Hugging Face, wrapped in Java by DJL) messes up with the CUDA context. It probably took me a week or two of hair-pulling chasing that weird bug, but REPL helped me again. I literally discovered this by eliminating every other possibility.

So, in the third month, I had to integrate another tokenizer, just to be able to work with CUDA. I integrated Sentencepiece, which is Google’s tokenizer, which has been created for Gemma. It’s a lot faster than Hugging Face, but it doesn’t support nearly as many models. But it works with CUDA…

So, now, we have 2 tokenizer integrations (HF and Sentencepiece), and we have my own streaming detokenizer implementation i pure Clojure (it’s fast while still being elegant).

The original superfast token sampler

I further polished this sampler and integrated it into the LLM loop. Although I hoped to, I didn’t have time to write up a scientific article, so I didn’t publish the source of that part yet. The LLM implementation woes took all of my time, so this source will be published as soon as I write that damn article. The CPU model doesn’t have this sampler algorithm implemented yet, so I provided a default logmax for now (it’s in the public part of the Snapdragan project) so people can try the code and the examples.

Miscellaneous

During all this time, whenever LLM models required something that’s not supported in mainstream matrix/tensor backends that I use (cuDNN, cuBLAS, BLAS…), such as long integers in tensors, or float16 in matrices, I created a workaround and implemented that under the hood of Neanderthal and Deep Diamond. Several Uncopmlicate libraries got new features under the hood along the way!

I’ll spare you the details of the C++ compilations and bug hunts. Too many bad memories :) I also don’t have any more strength (just a joke, I’m fine :) to itemize many detailed paths and activities that I had to do to make this work.

But I hope that I mentioned the most important thing: iLLaManati is here, and it’s not all that bad! :)

(Also worth noting: the code is human-written!)

Gloat and Glojure: Ingy dot Net

Q2 2026 Report 2. Published July 31, 2026

Today is the final day of my Q2 2026 Clojurists Together funding cycle for Gloat and Glojure.

This was my original commitment for the grant:

Make Gloat/Glojure binaries smaller and faster. Pass more of the Clojure Compatibility Test Suite. Create tutorial docs for using Gloat in the real world.

The short version is that Glojure now passes every enabled test in the current Clojure Compatibility Test Suite, Gloat’s default AOT binaries are leaner and the Glojure compiler has made major performance gains, a new tutorial series takes you from installation through compiling a binary and using Clojure in a Go project.

Gloat can now be seriously considered as a full replacement for GraalVM native-image when compiling Clojure programs to native binaries!

The second half also took Gloat somewhere I hadn’t imagined at the halfway point: it has become a front end for multiple Clojure compilation engines, including Glojure, let-go, and GraalVM native-image.

Since the Halfway Report

At the halfway point, Gloat was at v0.1.50 and the first upstream Glojure release of this grant had just landed.

Since then Gloat has had 16 more releases and is now at v0.1.67. Glojure has had another 8 releases and is now at v0.7.3.

The most important changes were:

• A Glojure AOT runtime that Gloat that performs as well as GraalVM
• Significantly smaller native binaries (as small as 7.5MB so far)
• A large wave of compiler and runtime performance work
• Addition of let-go as a second compilation engine
• Added shared-library support for let-go (which doesn’t support it itself)
• A GraalVM native-image engine for direct comparison and use
• Better installation, formatting, coloring, paging, classpath, and REPL UX
• A new Gloat tutorial series

The Performance Story

My long-term goal for Gloat was to make a common YAML framework for all programming languages which I call YAMLStar.

TL;DR YAMLStar now builds its shared library libyamlstar.so with Gloat instead of GraalVM, and the resulting binary is about half the size and has similar runtime performance. Being back by Gloat means I can release prebuilt binaries for 15 platforms instead of the 4 platforms it was confined to with GraalVM. YAMLStar now delivers identical YAML capabilities (via a Clojure engine) to 32 (and counting…) programming languages!

Happy ending, but until this week I was not confident that Gloat would get there by today.

About 4 or 5 weeks ago Norman Nunley from the let-go project offered to help with YAMLStar by getting let-go’s native lowering to match GraalVM’s performance. The Glojure and let-go projects have been in friendly competition for a while, challenging each other and contributing to each other’s projects. Soon after Norman’s work started, I decided that Gloat could and should do all its automations over let-go as well as Glojure.

Then 2 or 3 weeks ago James Hamlin, Glojure’s original author, came out of hiding and decided to do a major optimization push on Glojure to have it reach this goal the go faster than the Speed of GraalVM!

We all worked together and the result was a major success.

Soon there will be very few reasons to use GraalVM to compile Clojure programs to binaries instead of Gloat.

Smaller Binaries

A happy result of the performance work is that the AOT Go code (and thus the native binaries) became much smaller. Glojure used to produce a 50MB hello world that was about 40x slower than GraalVM’s 28MB hello world.

Now the Glojure hello world is 19MB and both Glojure and GraalVM binaries take 0.007s to run.

The existing Gloat -Xprune extension was also updated to work with the optimized runtime. The prune extension tree-shakes out the unused parts of clojure.core and all the transitive Go dependencies they would normally pull in.

On hello world, -Xprune produces a 7.6MB binary that runs in 0.004s here.

Pretty nice!

Gloat Became a Multi-Engine Tool

Gloat now has an -E / --engine option and a gloat --engines command. The current engine list is:

glj       Glojure (default)

lgvm      let-go bytecode VM
lglvm     let-go native lowering with VM fallback
lgl       let-go native lowering (not yet implemented)

graalvm   GraalVM Native Image (binaries only)

The goal is to have let-go be a replacement engine for all of the things that Gloat does with Glojure. Compiling to binary was easy, but let-go does not yet support shared libraries and I really needed that to try it out with YAMLStar. So I made Gloat add the things let-go was missing, and now it can produce shared libraries from let-go as well. Hopefully soon we’ll get this working in let-go itself.

Gloat+Glojure can convert Clojure to Go source code, but that isn’t yet supported for let-go. It should happen eventually.

I also added a GraalVM engine to Gloat. I was doing a lot of time comparisons between engines and wanted gloat users to be able to do the same. But most people likely aren’t familiar with using GraalVM, and even if they are, they have to install it and set up their environment to use it.

If you have Gloat installed, GraalVM is ready to go with a single command:
gloat -E graalvm hello.clj # produces ./hello binary

Doesn’t get easier than that.

New Tutorials

The final grant commitment was tutorial documentation.

I am finishing a new three-part Gloat tutorial series today:

1. Introduction and Installation
2. Compiling your first Glojure binary
3. Using Clojure in a Go project

I really want to create at least a dozen tutorials because there’s so much cool stuff you can do with Gloat.

If you would like to see something covered as a tutorial, please find me on the Clojurians Slack or open an issue on the Gloat repository and let me know what you let me know what you would like to see.

Looking Back

I am very happy with the results of this grant. I have a working YAMLStar for myself, a Gloat that does away with all the shortcomings of GraalVM and a new community of dialect making friend that Gloat can help to do awesome things with Clojure.

That feels like a very good three months.

Thanks

Thank you to Clojurists Together, and to every person and company that funds it. The grant created the sustained time needed to work through compiler internals, runtime behavior, test suites, release engineering, examples, and documentation as one connected project.

Thank you to James Hamlin for the enormous Glojure optimization effort and for making upstream Glojure such an exciting place to work.

Thank you to Marcin Gasperowicz, Norman Nunley, and the let-go contributors for the ideas, benchmarks, collaboration, and friendly competition.

A special shout-out to Dmitri Sotnikov the author of the Jolt Clojure dialect, for his friendship and daily collaboration. Jolt is Gloat’s next engine target!

And thank you to everyone who tried Gloat, reported a problem, asked a sharp question, or followed these reports.

The grant ends today. The work most definitely does not.

Time to Gloat!

PluMCP: Shantanu Kumar

Q2 2026 Final Report 2. Published Aug. 15, 2026

I am happy to report my final progress on the scope of work for this sponsorship:

• MCP spec 2025-11-25 implementation
• PluMCP Usage documentation enhancement

When I shared my first update of this sponsorship cycle, about half of the work was already complete, although no release had been made yet. The second half focused on completing the remaining pieces, including Task orchestration and the OAuth overhaul, and resulted in several releases covering the overall scope.

MCP 2025-11-25 implementation

The major feature item that received most of its implementation work in the first part was Task orchestration. It was fully completed and released in the second part, alongside a substantial expansion and overhaul of OAuth support. Thanks to these OAuth changes, PluMCP is now ready to move beyond Preview and is available for beta testing.

While the entire work has been tracked under a single pull request at Pull Request #6, with much of the Task orchestration work already reflected in its WIP changelog, there have now been several releases:

• 0.3.0-alpha1
• 0.3.0-alpha2
• 0.3.0-beta1

Progress in Second part

In the second part, I completed the remaining 4 of 9 major feature changes and 5 of 10 minor feature changes. The completed work includes:

Major Changes completed

• Add support for OpenID Connect Discovery 1.0 to authorization server discovery
• Enhance authorization flows with incremental scope consent via WWW-Authenticate
• Add support for OAuth Client ID Metadata Documents as a recommended client registration mechanism
• Add support for (potentially long running) tasks to enable tracking durable requests with polling and deferred result retrieval

Minor Changes completed

• Review Security Best Practices Guidance - add required utility functions
  • Mostly inapplicable as the majority is meant for MCP proxy
  • MCP server Origin header check already implemented
• Return Input validation errors as Tool Execution Errors rather than Protocol Errors to enable model self-correction
  • Validating “Required tool params” is implemented
  • JSON-Schema validation is pending; Malli-based validation is already available
• Support polling SSE streams by allowing servers to disconnect at will
  • Already supported by PluMCP through session-level detachment of the SSE stream and server
• Support polling in GET streams, resumption always via GET regardless of stream origin
  • Polling streams and resumption via GET is already supported
• Align OAuth 2.0 Protected Resource Metadata discovery with RFC 9728, making WWW-Authenticate header optional with fallback to .well-known endpoint

Current status

Since all planned feature changes are now complete and included in the latest 0.3.0-beta1 release, PluMCP is now open for testing by users. A GA release is planned following beta testing.

Remaining work

Apart from squashing any bugs that arise during the beta testing period, the subsequent 0.3.x releases would focus on internal refactoring and code organization. It is also planned to deprecate some API in sync with changes introduced in the MCP 2026-07-27 spec version.

PluMCP Usage Documentation Enhancement

The PluMCP documentation website has been updated with the following new entries:

• MCP Server
  • Completion
  • Logging
• MCP Client
  • Roots
  • Sampling
  • Elicitation

The existing MCP Server → Tools entry has also been updated with a manual tool definition mechanism.

With these changes, the sponsored scope has been completed, and PluMCP 0.3.0 is now entering beta testing ahead of the planned GA release.

Permalink

Debates over AI consciousness are a trap

link / technologyreview.com (via) / #accountability #ai #ai-safety #ethics #gen-ai #responsibility #tech-industry

Permalink

cljrs - first steps

Notes

Installs cljrs (needs Rust to be installed first)

$ cargo install cljrs

Start repl

$ cljrs repl

Run a .cljrs file

$ cljrs run somefile.cljrs

Start nrepl

$ cljrs nrepl

Clojurust project structure

myproj/
├── cljrs.edn
└── src/
    └── myproj/
        └── core.cljrs

Start nrepl

$ cljrs nrepl

Compile project

$ cljrs compile --src-path ./src --main cljrs-project.core --out ./target/hello_cljrs_project

• cljrs
• Data Science With Rust

Code

• cljrs first steps
• Clojure Rust Project

Permalink

Serialization Considered Boring

require('[konserve.mmap :as kmm])

kmm/update-in!(store
  [:report :region "us-west" :views] inc
  {:durability :checked})

(require '[konserve.mmap :as kmm])

(kmm/update-in! store [:report :region "us-west" :views] inc
                {:durability :checked})

require('[boring.core :as boring])

boring/encode({:user/name "Ada", :scores [99 100], :tags #{:x :y}})
;; => #object["[B" ...]  58 bytes

boring/decode(*1)
;; => {:user/name "Ada", :scores [99 100], :tags #{:x :y}}

(require '[boring.core :as boring])

(boring/encode {:user/name "Ada" :scores [99 100] :tags #{:x :y}})
;; => #object["[B" ...]  58 bytes

(boring/decode *1)
;; => {:user/name "Ada", :scores [99 100], :tags #{:x :y}}

require('[konserve.mmap :as kmm] '[boring.nav :as nav])

kmm/with-mmap-value([c store "customers"]
  nav/value(get-in(c ["customer-137" "name"])))
;; walks the memory-mapped wire format to that one key;
;; the other customers are never built

(require '[konserve.mmap :as kmm]
         '[boring.nav :as nav])

(kmm/with-mmap-value [c store "customers"]
  (nav/value (get-in c ["customer-137" "name"])))
;; walks the memory-mapped wire format to that one key;
;; the other customers are never built

kmm/assoc-in!(store [:doc :section :field] 42)
kmm/update-in!(store [:doc :counter] inc)
kmm/dissoc-in!(store [:doc :section :stale])

(kmm/assoc-in!  store [:doc :section :field] 42)
(kmm/update-in! store [:doc :counter] inc)
(kmm/dissoc-in! store [:doc :section :stale])

require('[konserve-lmdb.store :as s])

s/scan(store {:from "a", :to "m"})
s/scan-keys(store {:prefix ["user"]})

(require '[konserve-lmdb.store :as s])

(s/scan store {:from "a" :to "m"})       ; ordered [key value] pairs
(s/scan-keys store {:prefix ["user"]})   ; keys only — reads no value pages

s/project(store {:prefix ["user"]} [:profile :address :city])

(s/project store {:prefix ["user"]} [:profile :address :city])

s/project-reduce(store
  {:prefix ["user"]} [[:profile :city] [:revenue]]
  fn [acc _k [city revenue]]: update(acc city fnil(+ 0) revenue) end
  {})

(s/project-reduce store {:prefix ["user"]} [[:profile :city] [:revenue]]
                  (fn [acc _k [city revenue]]
                    (update acc city (fnil + 0.0) revenue))
                  {})

[{:e 1, :a :x} {:e 2, :a :y}]
;; shaped:  tag 39649 [ [:e :a]            ← the key set, once
;;                      [[1 :x] [2 :y]] ]  ← rows: values only

[{:e 1 :a :x} {:e 2 :a :y}]
;; shaped:  tag 39649 [ [:e :a]            ← the key set, once
;;                      [[1 :x] [2 :y]] ]  ← rows: values only

org.replikativ/boring

{:mvn/version "0.1.27"}

org.replikativ/boring {:mvn/version "0.1.27"}

Permalink

Clojure Vs Rust - Waiting for clj.rs

Notes

• Clojure Vs Rust - Waitiing for https://clj.rs
  • Special case
    • The problem
      • 
  • .jar
    • loading
    • unloading
    • thousands of times
    • GNU Parallel https://www.gnu.org/software/parallel/
    • If there way to spawn lot of threads in Clojure / Java?
      • No idea
      • Maybe my ignorance
  • GraalVM
    • Oracle
      • I hate it
      • Litigation, not a tech company
        • Who like lawsuites once when one is succesful?
  • Rust
    • ups
      • No memory leaks
      • Fast
      • More compute per watt
      • Saves time
    • downs
      • Complex code
      • Not at all terse
      • Checks data for dimensions
        • So many columns in a spreadsheet
          • Maybe A.I way of understanding it
            • Humans should correct it
        • In Clojure its just a sequence of maps
      • Shipping
        • Binary for each platform
  • How was the port
    • Clojure
      • A.I understood clojure well
      • Code might be better than strings typed by human
        • Well designed code is understood by AI well

        • docs/
            requirements/
            	2026_08_20_01_do_a_clojure_vs_rust_video.md
          

    • File to file port
    • A.I took less than 30 minutes, just one shot
    • I was able to make A.I to debug stuff
      • The bug was due to bad data fed
    • Was cool, bit unbeliveable
  • clj.rs
    • -ve
      • 1 person project :(
  • Other things to explore
    • Does clj.rs have a repl that calva can work with?
    • jolt https://jolt-lang.net/
    • jank https://jank-lang.org/
  • Speed gains
    • 70X in Single file performance
    • 2.3X project perfomance
      • Project contains Python code

Permalink

A bundle of CLI tasks for tools.deps projects

I've just released biff.run, a very light clj-based task runner; and biff.tasks, a bunch of default CLI tasks that can be used in tools.deps projects.

The tasks are mostly the same as what I've already been including with Biff projects for the past several years. I've reworked them a bit, added a few new tasks, and have structured them to be hopefully useful for tools.deps projects in general rather than being coupled to "Biff projects."

The biff.tasks docs linked above gives a good overview of what tasks it includes. Some unorganized thoughts/random stuff I think is interesting/design opinions:

• biff.run is an answer to the question "what's an ergonomic way to publish a curated collection of tasks rather than a single task." Instead of one deps.edn alias per task, there's a single :run alias for all your tasks.

• A criticism of tools.deps is that it's lacked various functionality that lein has had built in (simple but not necessarily easy). biff.tasks is my take on addressing that, and I'm interested in if it could help people get going with Clojure more smoothly. However Biff users specifically are still the main audience I'm designing stuff for; time's limited.

• Startup time has been fine. I've settled on a few approaches for managing it: each task goes in a separate namespace so biff.run can load only the code for the task you're running; tasks use requiring-resolve when they have heavy dependencies that are used conditionally; some tasks shell out to pre-compiled binaries (like cljfmt and clj-kondo). In return, being able to do everything in plain Clojure and use the regular dependency tooling is nice.

• I did tinker with passing the classpath to Babashka like bb -cp $(clj -A:run -Spath) -m com.biffweb.tasks.lib -h. Seems like it'd be kinda cool to default to plain clj but then say "push this button to get instant startup times with Babashka instead." It hasn't been a huge need for me though so I haven't explored it too much.

• Speaking of pre-compiled binaries, biff.tasks uses some internal machinery that makes working with these binaries pretty seamless. biff.tasks specifies a default version for each binary, users can set a different version in their project config if desired, and whenever a task needs to use that binary, biff.tasks ensures the right version is installed. If not, biff.tasks downloads the correct version. Example.

• biff.tasks doesn't include a task for creating a new project because IMO project templates should provide a way to use them without having to install another tool first. That's perhaps the one bit of functionality that might be nice to have baked into clj; only then could I as a project template author safely assume that all my users will already have the task installed.

• biff.tasks does come with an init task that renames the current project's main namespace from com.example to a namespace that the user provides. The idea is that you use project templates by doing git clone <some template>; rm -rf .git; git init; clj -M:run init. (The project template would have biff.tasks already installed in its deps.edn). Probably with that all wrapped up into a little script so you get a nice one-liner.

• For library projects, I settled on a docs task that turns your docstrings into markdown files like this one. I'm quite fond of it.

Permalink

Deploy an app with biff.tasks

I demonstrate how to provision a new Ubuntu server on DigitalOcean and deploy a hello-world Clojure web app to it with biff.tasks.

• biff.tasks

Permalink

AI coding tools unlock small software

link / fastcompany.com (via) / #ai #ai-agents #coding-agents #software-development

Permalink

Quoting Difficulties

In my last post I discussed DSLs for database querying in Clojure. These typically take the form of data structures.

I also discussed how some query languages, like SPARQL and Datomic, use variables in their queries, and that these appear in Clojure as symbols. That post also demonstrated using quoting to embed symbols easily into a structure, and unquoting to use values inside those same structures.

Some of it got messy.

Symbol Reuse

A colleague was recently trying to build SPARQL queries using Flint. This is a library that allows SPARQL queries that look very similar to Datomic queries.

He was trying to programmatically build query fragments that could be appended to each other to form a complete query. Each fragment was generated by functions that returned a small structure that could be added into the query.

In most cases, he could use quoting to return his structure. For instance, the following fragment might be used to find the name of a person who had changed an entity:

[[entity :data/modifiedBy ?person] [?person :data/firstName ?name]]

This is not the final form though, since he wanted to both pass in a value for entity, while also quoting the symbols in his structure.

Using the techniques from the last post, this is relatively straightforward:

(defn modifier-name
 [entity]
 [[entity :data/modifiedBy '?person]
  ['?person :data/firstName '?name]])

However, there are occasions where the entity might be modified more than once, and so multiple names should be returned. This will need a configurable name variable:

(defn modifier-name
 [entity name-var]
 [[entity :data/modifiedBy '?person]
  ['?person :data/firstName name-var]])

This would let a developer call something like:

(concat (modifier-name '?entity '?name1) (modifier-name '?entity '?name2))

(note: There are overlaps between what ?name1 and ?name2 will bind to. This is just for illustration.)

Autogensym

Unfortunately, this has a bug. In both cases, the ?person variable is used, meaning that both ?name1 and ?name2 will always be bound to the same values. One way to address this is to generate a new symbol for the query.

Since we're been using quoting, then a common way to generate symbols is to use a syntactic feature called an autogensym inside a quote. This uses a symbol name with a trailing # character:

(defn modifier-name
 [entity name-var]
 `[[~entity :data/modifiedBy ?person#]
   [?person# :data/firstName ~name-var]])

This version is using syntax quoting, and embedding entity and name-var, as discussed in the previous post.

However, this version has a bug too. The appearance of ?person# in the code tells the Clojure reader to generate a new symbol.

user=> (modifier-name '?entity '?name)
[[?entity :data/modifiedBy ?person__2__auto__]
 [?person__2__auto__ :data/firstName ?name1]]

Each new use of this ?person# expression (in a new context) should result in a new symbol. However, this symbol gets reused when the function gets called again.

user=> (concat (modifier-name '?entity '?name1)
               (modifier-name '?entity '?name2))
([?entity :data/modifiedBy ?person__2__auto__]
 [?person__2__auto__ :data/firstName ?name1]
 [?entity :data/modifiedBy ?person__2__auto__]
 [?person__2__auto__ :data/firstName ?name2])

The symbol ?person__2__auto__ was returned from both calls, because the generation actually occurred when the function was read, not when it was executed.

This is the same issue that was discussed in an Ask Clojure Question. Syntax quoting and autogensyms are most often used in macros, and the scope of a generated value is typically restricted so that any generated symbols cannot interact with each other. The case discussed in that Clojure question was when a macro was recursive. In that case, the symbols generated during recursion were all the same, since they all shared scope.

Our query is not using recursion, but instead it is capturing the name of this symbol and returning it to the calling scope. This means that the scope of the generated name is extended to the calling context, allowing it to interfere with other generated names in that context. i.e. the scope "escaped".

In other words, despite autogensyms being common when quoting expressions (most commonly in macros), they are not appropriate for anything that can escape the current context.

Addressing the issue

One solution to this is to generate a symbol on each execution of the function. This can be done manually, rather than using the autogensym syntax:

(defn modifier-name
 [entity name-var]
 (let [?person (gensym "?person")]
   [[entity :data/modifiedBy ?person]
    [?person :data/firstName name-var]])

This creates a new value every time.

Recap

Using functions to generate fragments of queries can result in conflicting fragments, particularly in graph languages that have variables in their syntax. In these cases, a new variable is needed for each fragment.

Clojure has a facility for creating new variable names easily, called "autogensym". However, the new name is only generated when the code is read, meaning that any function using this construct will always return the same symbol. "Autogensym" can be helpful, but only if the context of the generated symbol can never overlap with the context of another call to the same autogensym.

Query Mess

In the first post I mentioned that macros can simplify a query DSL. The next post will demonstrate this.

Permalink

Clojure Query DSLs

Domain Specific Languages (DSLs) are a popular technique for writing database queries. There are a few reasons for this, including:

• They help ensure queries are syntactically correct
• They make programmatic construction of queries more tractable
• Fragments of queries are easier to reuse

Some languages implement DSLs in libraries, some allow the language to be extended to include the DSL, and some can use already existing syntax and data structures to implement the DSL. While most languages can implement DSLs in libraries, Clojure also has the option of extending the language via macros, as well as providing syntax for regular data structure that can also be convenient to use.

Macros

While basic macros are relatively easy to write in Clojure, they can become quite complex. In general, when it comes to Clojure macros, I think Sandra Sierra's 2010 advice holds up well: You do not write macros.

That said, some libraries may use macros to make writing queries easier for developers. This can make it easier to incorporate names and labels into expressions that would otherwise lead to errors on "unbound" values. Many Clojure DSL libraries eschew this, since keywords often work just as well. As an example, a SELECT expression for reading the name and age columns of a table would need a macro if a developer wanted to write:

(select name age)

However, the complexity of a macros can be avoided by switching to keywords instead:

(select :name :age)

Structures

Clojure code is almost always written to use the regular data structures that are built into the syntax of the language. These are:

• Maps: {key value …} Also called a "Dictionary" in languages like Python.
• Sets: #{data …}
• Vectors: [data …] Called "Lists" in Python.
• Lists: (data …) Implemented as Linked Lists.

Lists are a little different, as they are "executed" by defautl in Lisp dialect like Clojure. This is avoided by introducing the list with a ' quote character. e.g. '(1 2 3)

Since these structures are part of the language, they can be a simple way to build a DSL for querying. For instance, Sean Corfield's HoneySQL can represent an SQL query using a map and vectors:

{:select [:a :b :c]
 :from   [:foo]
 :where  [:= :foo.a "baz"]}

Datomic does something similar, using either a map or a vector for the queries:

'[:find ?title 
  :where [?e :movie/title ?title] 
         [?e :movie/release-year 1985]]

There is an issue here though. Unlike previous examples, this last query is using variables (marked with a ? prefix) as a part of the query language. This is especially common in graph languages like Datomic or SPARQL.

Quoting

Using a : to turn these variables into keywords makes it harder to distinguish variables from actual values stored in the database (since Datomic stores keywords directly, and SPARQL libraries assume keywords to be CURIEs). Instead, Clojure Symbols are used. The problem is that symbols are the mechanism that Clojure uses for associating data with a name, so using a symbol usually results in Clojure looking for that data, which will be an error if the symbol is not bound, and inserts a value where you wanted a variable if it is found. This is avoided using the ' quote character.

To illustrate this, let's look at that Datomic query again, this time without the quote. I'll show what happens at a REPL (the Clojure prompt), where the prompt includes the current namespace (user by default, though it can be something else):

user=> [:find ?title 
        :where [?e :movie/title ?title] 
               [?e :movie/release-year 1985]]
Syntax error compiling at (REPL:0:0).
Unable to resolve symbol: ?title in this context

user=> (def ?title "not a variable")
user=> (def ?e "also not a variable")
user=> [:find ?title 
        :where [?e :movie/title ?title] 
               [?e :movie/release-year 1985]]
[:find "not a variable" :where ["also not a variable" :movie/title "not a variable"] ["also not a variable" :movie/release-year 1985]]

This is read, but we can see that the use of the symbols has placed their saved values into the query, rather than a variable like we wanted.

We can avoid this problem by quoting the symbols that we want to keep as symbols:

user=> [:find '?title 
        :where ['?e :movie/title '?title] 
               ['?e :movie/release-year 1985]]
[:find '?title :where ['?e :movie/title '?title] ['?e :movie/release-year 1985]]

We can also quote entire structures that contain multiple variables:

user=> '[:find ?title 
         :where [?e :movie/title ?title] 
                [?e :movie/release-year 1985]]
[:find ?title :where [?e :movie/title ?title] [?e :movie/release-year 1985]]

Note that the result is printing the data structure, and not trying to evaluate it. Because of this, the symbols are not printed with a quote.

Mixed Symbols

Quoting entire structures makes it easy to include multiple symbols, but it also makes it hard to include values from a program. For instance, a user may be asking to get all titles from a year that they provide in a user-interface:

user=> (let [release-year (get-user-input)] ;; user input 1985
         '[:find ?title 
           :where [?e :movie/title ?title] 
                  [?e :movie/release-year release-year]])
[:find ?title :where [?e :movie/title ?title] [?e :movie/release-year release-year]]

That's put a symbol into the query when we wanted a number associated with that symbol.

There are lots of ways to address this. One is to only quote the parts we need to:

user=> (let [release-year (get-user-input)] ;; user input 1985
         [:find '?title 
          :where '[?e :movie/title ?title] 
                  ['?e :movie/release-year release-year]])
[:find ?title :where [?e :movie/title ?title] [?e :movie/release-year 1985]]

This works, but it can (and frequently does, at least in my code) lead to lots of quote characters everywhere, which can be a bit bug prone too (LLMs will usually catch it, but I'm here to discuss how WE write code, not LLMs).

Another way is to bind the names to symbols (binding to a symbol with the same name would reduce confusion):

(let [release-year (get-user-input)  ;; user input 1985
      ?e (symbol "?e")
      ?title (symbol "?title")]
   [:find ?title 
    :where [?e :movie/title ?title] 
           [?e :movie/release-year release-year]])

But now we're asking the developer to increase their code significantly. That's not really helpful.

For completeness, I'll also mention that sometimes it works to build the parts you need, and create the query structure with code:

(let [release-year (get-user-input)]  ;; user input 1985
   (conj '[:find ?title :where [?e :movie/title ?title]]
          ['?e :movie/release-year release-year]])

This is sometimes useful, but is too cumbersome for basic querying.

Unquoting

Another option is the "unquote". This tells the Clojure reader that is reading quoted data that the next item is not to be considered as quoted. This is done with the ~ character. However, it does not work quite as you might expect.

For simplicity, I will reduce the evaluation to just the last part:

user=> (def release-year 1985)
user=> '[?e :movie/release-year ~release-year]
[?e :movie/release-year (clojure.core/unquote release-year)]

Unfortunately, this has shown us what the ~ unquoting gets translated to. Quoting actually gets translated like that too. We just haven't seen it before:

user=> '['?e :movie/release-year ~release-year]
[(quote ?e) :movie/release-year (clojure.core/unquote release-year)]

Instead, we need to use a different kind of quoting: syntax quoting. This is done with a single "back-quote" or .
clojure
user=> (def release-year 1985)
user=> `[?e :movie/release-year ~release-year]
[user/?e :movie/release-year 1985]

This embedded the value ofrelease-yearas we wanted, but it has also change the?esymbol. Now it tells us that it's the the symbol?ein the namespaceuser`. That's the current namespace, so that's correct, but we want to embed the symbol without the namespace.

This is a tricky form. We want to "unquote" from the syntax-quote, but then we want to immediately "quote" again:
clojure
user=>[~'?e :movie/release-year ~release-year]
[?e :movie/release-year 1985]
`

Going back to the complete query, we can see the full form:
clojure
(let [release-year (get-user-input)] ;; user input 1985
[:find '?title
:where '[?e :movie/title ?title]
[~'?e :movie/release-year ~release-year]])
`

This is not entirely satisfactory, and it explains why macros may be attractive, but it does show an approach.

Wrap Up

This post demonstrates some of the approaches of using Domain Specific Languages (DSLs) in Clojure, focusing on data structures to represent database queries. Query languages like Datomic and SPARQL use symbols in their queries, and we looked at a few ways that these can be embedded into a query structure.

Permalink

A practical workflow for LLM-assisted development

When LLMs work, it can feel like magic, but when they fail, it feels like you are arguing with a confident bullshit artist. It took me many months of daily use to develop some intuition for where LLMs are likely to produce code that is useful and where they are likely to fail. It also took me a bit of time to figure out how to limit scope and provide enough scaffolding to ensure I get useful results reliably. Having invested the time to learn to use the tool effectively, I very much see the benefits, as I am able to build projects on a scale I would not have attempted before.

In a way, the process is the inverse of regular programming. We tend to build up programs step by step when writing code by hand as we add each function with intention. LLMs tend to produce a lot of code out of the gate and the focus shifts to whittling the code down to what you actually need.

A good way to look at the agentic loop is to view the process as a genetic algorithm. Agentic harnesses are effective because you have an evolutionary process happening. The model outputs something roughly correct before the code gets tested, and then the model gets feedback to iterate on the code. Through this process, it gradually converges on a solution that fits the parameters being tested. In that sense, it is not actually all that different from how humans write code either. You almost never solve a non-trivial problem in one shot. You write your first approximation and then iterate on it. The difference is that the LLM can do this process a lot faster.

What to Delegate

LLMs are trained on massive amounts of public code, which makes them excellent at completing typical tasks. These are things that have been done a million times before and constitute what largely amounts to boilerplate. Throwing a sample JSON response at an LLM and having it write a service endpoint or throwing a bunch of API endpoints at it and having it build a UI using them can be very effective. These are the kinds of common tasks the agent will have a lot of training on, and they can produce something reasonable in one shot. It will probably put more diligence into that task than you would by adding tests and handling all the obvious edge cases.

They are also great at doing explorative work. Identifying a particular call graph and tracing through the steps to figure out how a particular service endpoint is implemented or what parameters you have to pass it are all tasks an LLM can do easily. This can save an enormous amount of time tracing through a codebase and mapping out a particular workflow that you are interested in.

These tools are also great at handling language specific syntax. If you know conceptually what you want to do, like looping through a collection and filtering by a specific parameter, but you are working in a language you are rusty in, then LLMs are great for bridging the gap. They can easily express the logic you want using idiomatic syntax. You can describe the algorithm in pseudo code where you write out the steps and it will handle the rest.

For example, I recently had to work on a JavaScript project, and I have not touched the language in over a decade. I am not familiar with modern tooling or libraries or best practices, and I just did not have the time to get up to speed on all that.

Using DeepSeek allowed me to use JavaScript as effectively as I do Clojure, which I am well versed in. It completely removed the friction of figuring out all the incidental things like syntax or tooling. If you are an expert in a particular domain and you understand the problem you are trying to solve, then LLMs can be a huge amplifier for what you are able to do. They do not replace your skills, but they do allow you to move a lot faster and focus on the big picture of the problem you are trying to solve.

When to Take the Wheel

In my experience, the biggest place where agents trip up is dealing with context and creativity. You have to remember that the AI does not know the specific quirks of your project. For example, if you just tell it to use a Clojure dialect, it might reach for the JVM toolchain it learned Clojure on, such as clojure and lein, none of which exist in that context, or it might assume a tree walking interpreter and try to run the source directly. You need to give it the exact logic, like telling it explicitly that the runtime is pure Chez Scheme and that everything builds through make commands via a chez --script execution, while specifying that the authoritative sources are host/chez/*.ss and jolt-core/*.clj over anything JVM flavored.

Then there is also the trap of the naive implementation. Often, when you give an agent a vague goal, it will hand you something that looks correct on the surface but ends up being structurally wrong. For example, the agent might decide that string method calls should be routed through a generic dispatch table, which ends up re-deriving the receiver type on every single invocation. The proper fix here is to do a type inference pass to prove that those values are strings at compile time, which allows you to emit a direct native call and skip dispatch entirely. An agent told to make the string methods fast will almost certainly keep the generic path by reordering a few cond arms and never bother designing a proper solution. Similarly, if you ask it to implement count on a sequence, it will likely walk the whole thing allocating a fresh cell per element when the collection already knows its own length that can be called in constant time. Ask it to join strings and you will probably get repeated concatenation instead of a single walk. It is akin to an evil genie that will interpret your queries in the worst way possible, leading to the solution having a completely wrong shape. The trick is that you have to spell out the constraint, which incidentally forces you to think through the problem as well.

The key to using LLMs effectively is to make sure you already have a solid understanding of what you are aiming to build before you start. You always have to be explicit regarding what you want done at a structural level. The more scaffolding you provide up front the less room the agent has to go outside your design. A corollary to this observation is that you do have to understand the domain to make effective use of LLMs. If you are not equipped to evaluate whether the code it produced solves the problem in a correct way, then you basically end up at a casino pulling a lever on a slot machine and hoping for a decent solution to fall out. LLMs are good at filling in the gaps and doing boilerplate, but you still have to do design and architecture the same way you always did.

Here are some tricks that I found useful for keeping it on the rails.

Always start out by planning out the task. Make sure you have a clear picture of what you are aiming to do along with what algorithms you are intending to use and how the code should be structured to fit within the existing architecture. You must be able to answer these questions before you even think about delegating to the LLM.

Once you have a clear picture in your head, you can move on to the planning stage with the agent. Give it the requirements and spell out the goals before asking the model to write a phased plan in Markdown. Even better, ask it to generate a Mermaid.js diagram of the flow.

After it makes the diagram, you can visually inspect the logic. If a particular step looks wrong in the diagram, you tell it to change that specific step to do something else. Doing that is a lot easier than simply arguing with it using text prompts. Once there is a clear structure for the steps being performed, it is easy to identify parts that you do not like. Review the plan and get the model to break it up into independent tasks, each focusing on implementing a specific feature. Have the model create a branch and then make a pull request for the task. At that point you can review the code fairly easily because you know what the scope of the change is and what specific problem it solves.

It can be very helpful to have the model do research on prior work for steps where you are not sure which approach to take. It is rare that the problem being solved is entirely novel, and agents are great for looking up relevant papers you can review to get a better idea of what is more likely to work. Again, it is important to spend the time to familiarize yourself with the different paths you can take and to pick one consciously.

I would also argue that having a clean architecture with low coupling becomes extremely important when using LLMs. They tend to do best on smaller tasks that do not have dependencies because there is less context to consider. So if you can break up your project into small pieces that can be worked on in isolation, then you can give the agent a task with clear boundaries. That also makes it much easier to review its output as well.

I find that functional style maps particularly well here because it focuses on context isolation and passing state around explicitly. The same tricks that make large code bases manageable by humans also help LLMs for the same reasons. Aggressively controlling the context is a key tactic for using LLMs effectively.

It bears repeating that you never want to give the AI a blank canvas. Always do the work of laying out what the scaffolding should look like yourself. Make sure you intentionally set up the file structure and decide on the components before asking the agent to fill in the blanks.

But even with all these great functional tools, we still tend to tangle two rather different kinds of code together. We tend to mix code that cares what the data means and the code that decides how it travels from one component to another. Traditional software design structures embed the routing implicitly in the function call graph. Control logic often ends up being coupled with the internal implementation details in an ad hoc manner. Breaking things up into independent steps helps control the scope.

Routing logic should be elevated to first class citizenship in the design. State machines are the natural fit for this, since they force the separation of what to do from how to do it. The control flow logic can be largely declarative and expressed as a graph such as the Mermaid diagram I mentioned earlier, while the implementation details live at each step in the flow and become the tasks the agent works on.

Doing these steps forces the agent to work within your architecture rather than inventing its own structure, which largely avoids the problem of it going off the rails. Once you get it to build a diagram and you have reviewed it, you can create the initial project structure based on that.

Use Tests as a Contract

I find it is useful to think of tests as the ultimate requirement doc when working with LLMs. If you define your desired functionality as tests first, you can get the agent to work through them using test driven development until they pass. It will typically do a decent job running the tests and analyzing the failures and fixing its own code to meet the spec. The tests are the contract that the agent works against. Going back to the whole genetic algorithm analogy, these are the selection pressures that drive the evolution of the code.

Having tests up front gives you a solid guarantee that the code is doing what you intended functionally. It is also your best defense against regressions. Without tests, an agent adding a new feature is just as likely to silently break three old ones. Having a contract for the existing functionality avoids that problem.

The types of tests that tend to be most valuable are the ones that focus on the functionality of different components along with end to end integration tests. They do not need to be too granular because issues will get shaken out as the whole workflow gets exercised. I can also highly recommend making storybooks and creating automated testing using Playwright for web apps where the test goes through the entire workflow end to end driving the page as the user would.

Additionally, since tests do not capture performance characteristics, it is helpful to create a benchmarking suite to check performance metrics such as CPU and memory usage. Having one from the start has been very informative in guiding my development of Jolt.

Git is Your Safety Net

You can think of Git like having a quick save in a video game. Every single time the agent gets into a stable state where tests pass and the code looks like it is doing what you want, you should commit that code immediately. This gives you the freedom to let the agent try different experiments or complex refactors. If the agent makes a mess or the idea does not pan out, you do not have to untangle it manually. You just revert to the last good commit and move in a different direction.

I have noticed that if the agent does not get the solution mostly right on the first shot, it is unlikely to make it work properly later. The agent is not going to step back to understand the underlying problem when you point out a bug. Instead, it just adds kludges to fix your specific complaint and the problems tend to multiply as a result. If the original solution was not a good fit, then adding more kludges on top only makes a huge mess that will never work right. If it starts spiraling, then it is time to reframe your problem statement and start from scratch.

A related point is that LLMs make it very cheap to do exploration with your codebase. I mentioned earlier that you always want to understand the problem before you get the agent to start working on it and that is true for code you intend to keep. However, working through a problem is a great way to understand it better. So when you hit a point where you are not sure what to do or which approach might be best, that is when you can spike up different ideas and see how they pan out. Since you have version control, it is trivial to roll back to a known stable commit and try something new from there.

This sort of thing used to take a significant amount of effort, but the barrier to exploration is a lot lower. For example, when I started working on Jolt, I picked Janet as the runtime for it. My rationale was that Janet was superficially similar to Clojure and had a compact runtime while being embeddable. However, I quickly realized that the lack of generational garbage collection did not mesh well with the lots of short lived objects that persistent data structures generate. So I did a bit of research and landed on Chez Scheme instead. I was able to do the whole Janet spike in around a week, and that is something that could have easily been a months long project without LLM use. Similarly, proving out a solution on top of Chez only took a few days to get to the point where it was clear that it would work better.

The Harness Matters

There are a lot of agentic harnesses around, and they all optimize for different use cases. What I found to be important is that the harness meets the expectations of the model and provides the flexibility to customize the workflow to fit a specific project.

In the end, I ended up building my own harness, which I discussed in a previous post here. I spent some time observing how models like DeepSeek and GLM behave within the agentic loop and where they appear to get tripped up. Dirge also integrates proven tricks from existing tools like the official deepseek-harness to avoid reinventing the wheel here. Additionally, I used Janet to provide a plugin system similar to Pi. You can create a .dirge folder per project to place custom plugins there, allowing the harness to evolve alongside each project.

I also spent some time on addressing the common pitfalls that I kept seeing to make the workflow smoother. For example, one common problem is that the model will produce mismatched parens in code. If you simply send the code back to the model, then it is going to burn tokens trying to figure out where the missing paren is. Often, it ends up doing things like writing python scripts to count them. Doing the repair inside the harness solves the problem mechanically so that the model never has to be involved.

Another thing I found was that tracking things using Markdown files tends to be fragile. These files can get stale, which leads them to be misleading and the models do not do a good job keeping them up to date. My solution was to use sqlite as the datastore for the harness and to use it as project memory. I extended that to track tasks as well, modelled on the way beads works. The harness asks the model to create tasks before it starts work and then tracks the active tasks and injects the task being worked on at the top of the context. This helps keep the model focused and continue working on larger features. When the task is done, I use a separate critic role to review it by examining the diff and then provide feedback, which helps avoid cases where the model decides to ship a half baked solution.

I have also integrated some ideas from papers such as Behavior Trees Enable Structured Programming of Language Model Agents, which focus on having the language model act as a leaf node in a larger deterministic control structure instead of trusting it to make decisions end to end. The main idea is to move from treating the model as the whole agent to making it a primitive, which produces a behavior. The workflow is then composed with a small set of classical control structures. The finalization gates like the verifier and critic, along with the code reviewer, form a fixed sequence of deterministic checks the model must clear before a run is allowed to finish. The failure ladder rungs use retry fallback nodes to catch a stuck or failing model, and mechanisms like publish state guard enforce safety constraints structurally. Once you have the right structure around it, even a local model can solve fairly complex tasks competently.

Conclusion

You are still the engineer who is responsible for understanding the problem you are trying to solve and what the project is meant to be doing. Your job is to provide the high level thinking and the architecture while understanding what the correct solution should look like. The LLM is there to save you from the boring and repetitive work like typing out boilerplate and looking up syntax.

The key part to keep in mind is that an LLM is just another tool in your belt. It cannot help you solve problems that are outside your existing expertise effectively. These tools lets you work faster once you learn their sharp edges, but they do not do your thinking for you.

In fact, an LLM on its own is not able to do much of anything useful. It requires its user to have domain expertise to apply it effectively. My ability to build a Clojure compiler using these tools stems from nearly two decades of experience working with the language. I know how it works internally along with what the end state needs to look like and what pitfalls to avoid.

Trying to solve a problem I have no familiarity with would just be me throwing darts at the board. Maybe the LLM will produce the right solution and maybe it will not. I would not be equipped to evaluate that one way or the other.

Permalink

Software Engineer at Scarlet

Permalink