Planet Clojure

OSS updates July and August 2026

In this post I&aposll give updates about open source I worked on during July and August 2026.

To see previous OSS updates, go here.

Updates

In the past two months it was summertime in Europe. Due to a couple of heatwaves, it was the perfect time to spend inside and enjoy my new air conditioning, while coding ;-).

The first half of July was mostly spent on improving performance and compatibility of SCI on CLJS. SCI now JIT-compiles interpreted function bodies to JavaScript at runtime, which closes a lot of the gap with compiled ClojureScript: a tight numeric loop went from ~175ms to ~7ms, over 20 times faster than the interpreter. Implementing core protocols on custom types now also works. There&aposs hardly anything you can&apost do in SCI that you can do in compiled CLJS. I released new versions of scittle and nbb that take full advantage of this.

Also in the middle of July, clj-kondo got a pretty cool enhancement. It infers types of function arguments from how they are used. E.g. when you write (defn foo [x] (inc x)) we can infer that foo is a function that takes a number. I took this principle as far as I could while preventing false positives. Of course, clj-kondo supports the latest Clojure 1.13 destructuring changes too.

In the second half of July I spent significant time on improving babashka tasks with automatic help and completions, backed by babashka.cli. You can read all about that in this blog post: Babashka tasks with automatic help and completions

In August I had the pleasure of giving a talk about Reagami at Func Prog Sweden. In the talk I gave an interactive demo of how to use Reagami in a Squint project through a REPL. I also went into detail on the algorithm that powers the fast DOM diffing. While preparing for the talk, I added SSR to Reagami too. You can view the talk on YouTube:

The last few weeks of August I created babashka.ffi, a new namespace in babashka to call C libraries. See my previous blog post: Babashka 1.13.220 gets FFI. To validate the design I wrote four libraries with it: babashka.sqlite, babashka.duckdb, babashka.postgres and filewatcher. Each one exercised a different corner of the API, along with some examples based on raylib. PacMan is particularly cool:

pac-man running in babashka through babashka.ffi and raylib

Right now I&aposm looking forward to giving a babashka workshop at the Clojure/conj together with Rahul Dé. We&aposre still polishing the workshop material behind the scenes and I&aposm excited to see how it&aposs turning out. I&aposm sure it&aposll be a lot of fun and hope to catch many of you there.

In between all of this, I also worked on squint. It now supports the core protocols, so you can plug in your own collections and use them with core functions. E.g. you can use Immutable.js with Squint. I&aposm thinking about lightweight immutable persistent data structures for squint, but so far I haven&apost had much need for them, outside of Advent of Code puzzles.

The above was all about making existing projects better. But I also had a few new creative ideas:

Choq: Cherry hosted on QuickJS, nREPL included.
Buzz: a cross client-server framework that lets you write web-apps on the JVM or babashka without any JS tooling, while still having full JS expressivity via Squint. I wrote tube-pod and multi-snake with it.
Cljbang.el: A Clojure-like language that runs as Emacs Lisp

Here are some highlights per project. See each project&aposs CHANGELOG.md for the full list.

Babashka: native, fast-starting Clojure interpreter for scripting.
- 1.13.220: Add experimental babashka.ffi: call C functions in shared libraries straight from babashka and JVM Clojure! See the guide
- 1.13.220: On Linux, the install script installs the dynamic binary by default. It installs the static binary on musl systems and on systems with glibc older than 2.17. The --static and --dynamic options override the automatic selection
- 1.13.220: :exec-args can sit directly on a task, not only under :cli, the way (exec ...) already reads it. Before, it was ignored on an :exec-fn or :cmd task
- 1.13.220: A task&aposs :cli spec adds to the runner-level :tasks {:cli {:spec ...}} instead of replacing it. An option from the runner level keeps its coercion and default, and --help lists it under Inherited options
- 1.13.220: A task with :exec-fn runs when another task :depends on it. Before, it did nothing
- 1.13.220: Options declared by an :exec-fn task named in :depends also parse for the CLI task that runs, with their coercion and default. --help lists them under Inherited options
- 1.13.220: :cmd can be a symbol naming a var that holds the command tree, like :cli. Its namespace loads on demand
- 1.13.220: Shell completion offers inherited options (via :depends) too
- 1.13.220: SCI: call site caching for instance and static methods, constructors and fields. Interop calls are up to 5x faster
- 1.13.219: Tasks get automatic --help and shell completions, through the new :exec-fn and :cmd keys. See the blog post! These task keys should be considered experimental and may change in a future version of babashka, depending on feedback from the community
- 1.13.219: Clojure 1.13 map destructuring: :keys!, :syms!, :strs!, & inside a directive, :select, :all and :defaults. Adds req! and some-vals to clojure.core
- #1321: support implementing the clojure.core/Inst protocol on records, types and reify, and with extend-protocol and extend-type
- #2054: a proxy of java.io.Writer supports the one-argument write and append, so binding *out* to it works
- #1918: fall back to $HOME when the OS does not supply a home directory, e.g. for LDAP users in the static binary
- #1994: fix :eval and :print options of clojure.main/repl being ignored in the interactive REPL (@jeroenvandijk)
- Bump jline to 4.4.0: security hardening, a rewritten signal path for the FFM terminal, Kitty keyboard protocol
- #2021: bump http-kit to 2.9.0-beta4, which fixes four security advisories
- Class additions by @weavejester (#1985, #1986, #1987, #1988), @paintparty (#1982) and @christoph-frick (#2003)
- Full changelog
babashka.ffi: call C functions in shared libraries from Clojure. New library, also usable from JVM Clojure. See the guide and the examples. The API is experimental
babashka.sqlite: SQLite for babashka through babashka.ffi
- Uses the SQLite shared library that macOS, Linux and Windows already ship with, so there is nothing to install
- with-conn, queries, aggregates, transactions, last-insert-rowid, interrupt, and create-function! for defining a Clojure function callable from SQL
- CI green on three operating systems
babashka.duckdb: DuckDB for babashka through babashka.ffi
- Query CSV files directly with SQL, results as Clojure data
- HoneySQL support, thread safety, prepared statement cleanup
babashka.postgres: PostgreSQL for babashka through babashka.ffi and libpq
- connect, close!, with-conn, query, execute!, with-transaction, in-transaction?, cancel!, json, jsonb, version, server-version
- Vectors map to arrays in both directions, maps map to JSON. Bring your own JSON library through :read-json and :write-json
- clj-kondo export with a with-conn hook, CI on three operating systems
filewatcher: watch files and directories from babashka
- Built on babashka.ffi: FSEvents on macOS, inotify on Linux, ReadDirectoryChangesW on Windows, and polling everywhere
- The same event types on all three platforms, modeled after chokidar
- A watcher keeps the process alive until close
SCI: Configurable Clojure/Script interpreter suitable for scripting
- ClojureScript JIT compilation. SCI on CLJS compiles interpreted function bodies to JavaScript at runtime via js/Function. This is enabled by default and needs no configuration. When JIT is enabled, loops and numerical computations become much faster (and, in unrestricted contexts, JS interop too)
- When eval is unavailable (e.g. under a Content Security Policy) SCI falls back to the interpreter. Results, error messages and error locations should be identical. And of course, it works under :advanced compilation
- You can turn JIT off at runtime with js/globalThis.SCI_DISABLE_JIT = true before loading SCI, or in your Google Closure compile settings with :closure-defines {sci.core/disable-jit true}
- More CLJS JIT performance improvements. Up to 20x on arithmetic-dense code for >2 arity. Keyword lookups, instance? and js globals no longer fall back to the interpreter
- ClojureScript: native protocol support (#639). SCI code can implement CLJS protocols on deftype, defrecord and reify, and host code calling protocol methods on such instances dispatches into the sci implementations. Works under :advanced compilation
- #1063: CLJS: deftype and defrecord fields are JS accessors on the type&aposs prototype: (.-field x) works on instances, (set! (.-field x) v) mutates deftype fields
- New :unrestricted option on init and eval-string: when true, evaluated code may mutate built-in vars and CLJS instance interop skips :classes checks. The option applies only to the context it was passed to
- BREAKING: enable-unrestricted-access! now throws. Use the :unrestricted option instead. The old function set a process-global flag that leaked into nested contexts
- Support async functions by adding :async true in the attr map of defn
- Caches resolved JVM instance methods per call site for performance
- Fix babashka#2030: aset on a primitive array was reflective and 170x slower than aset-double
- Errors thrown inside a loop now report a located stack frame for the loop form instead of a frame without location (all platforms, including babashka)
- Full changelog
clj-kondo: static analyzer and linter for Clojure code that sparks joy.
- Type checker: infer the type of a function param from how it is used in the body. E.g. (defn f [s] (subs s 1)) (f 42) will warn, since the evidence (subs s 1) tells us that s should be a string.
- Type checker: infer the value type of a destructured map key from how it is used in the body. E.g. (defn f [{:keys [x]}] (inc x)) (f {:x "foo"}) will warn. A key whose use rejects nil and that has no :or default is required.
- Type checker: a destructured binding gets the value type of its key when the map&aposs type is known, including through function return maps. E.g. (defn cfg [] {:port "8080"}) (let [{:keys [port]} (cfg)] (inc port)) will warn.
- Type checker: a key missing from a map literal is provably nil, also through destructuring, keyword access chains and function return maps. E.g. (inc (:y {})) will warn.
- Type checker: narrow the type of a local in the then-branch of if or the body of when when it is guarded by a known predicate. E.g. (if (string? x) (inc x) ...) will warn.
- Built-in analysis now uses Clojure 1.13.0-alpha4. Param type inference over the core sources grows the arg type coverage of clojure.core from 23 to 150 vars. E.g. (interleave 1 [2]) and (mod "a" 2) will warn.
- #721: NEW linter: :constant-condition: warn on a condition whose truthiness is the same on every run. On by default. Replaces :condition-always-true, whose config and ignores still apply to always-true conditions, and takes over the cond catch-all warning from :unreachable-code
- Clojure 1.13 CLJ-2961: infer required keys from :keys!, :syms! and :strs! and report them at call sites
- #2874: Clojure 1.13 CLJ-2964: support :select in map destructuring. The bound map&aposs keys are known to the type checker
- Clojure 1.13 CLJ-2966: support :defaults in map destructuring, error when used without :or
- #2943: Type checker: when an :analyze-call hook rewrites a call, clj-kondo checks the arity of the original function but not its parameter types.
- #2900: :discouraged-var: new per-var :positions option (a set or vector of :call and/or :value) to limit the warning to call position or value position. A var passed to a higher-order function such as map counts as :value.
- #2851: NEW linter: :seq-rest: suggest using (next x) over (seq (rest x)). Defaults to :off (@tomdl89)
- #1882: built-in support for clojure.test.check.clojure-test/defspec
- #2877: warn when #_ before an unmatched reader conditional discards the next form. E.g. [#_#?(:cljs 1) 2] reads as [] in :clj and will warn.
- Vars defined in comment forms no longer count for :shadowed-var, :unused-private-var and :inline-def.
- Performance: use a record for var usages: 13.5% less allocation, ~5-10% faster linting. More performance work by @alexander-yakushev
- The minimum Clojure version to run clj-kondo on the JVM is now 1.11.
- Full changelog
babashka CLI: Turn Clojure functions into CLIs!
- #197: :positional spec marker: positional args get their own Arguments: help section and may not be passed as options
- #197: :restrict-args: error on positional args not consumed by :args->opts
- #219: :cmd-aliases on a table entry or tree node gives a command one or more alternative names.
- A short option that declares a non-boolean :coerce takes the rest of its token as its value, like getopt: -J-Dfoo=bar binds "-Dfoo=bar", -p80 binds 80. Flag letters may precede the valued option in a cluster: with :b a flag and :a valued, -ba x parses as -b -a x
- #216: in a cluster of flags, where no letter takes a value, an interior hyphen is an error instead of silently ending option parsing.
- Help: show the dispatch-level :spec options under Inherited options:. The parser always accepted these options, but help did not show them
- Help: format-command-help accepts :spec, the dispatch-level spec, so a standalone call shows the same options as dispatch
- dispatch: the command named on the command line wins over the :exec-args of its ancestors. A value the user typed at an ancestor level still wins over both
- Add ordered :enum values for validation, help and completion
- Support :doc and :epilog as a vector of lines, joined with newlines
- #198: :cmd may be a vector of [name command] pairs, preserving command order without :cmd-order
- #199: fix hang on variadic arguments that weren&apost "collected" (e.g. (repeat :k))
- #203: parse-opts* resolves :spec so its :coerce/:collect entries steer parsing like in parse-opts
- Completion: the fish snippet registers with --keep-order, so fish offers options in the order they are emitted, long option before its short alias, rather than sorting short options first
- zsh completion: offer a command&aposs options without typing a dash first, by opting the registered program names out of zsh&aposs prefix-needed style
- Thanks to @lread for continued documentation review and maintenance
- Full changelog
Squint: CLJS syntax to JS compiler
- Preparatory release before adding immutable + persistent collections in squint.immutable. Added a lot of protocols and made sure core functions work properly with them
- Add the ILookup, IAssociative, IMap, ICounted, IKVReduce, ICollection, IEmptyableCollection and IEquiv protocols. get, assoc, contains?, find, dissoc, count, reduce-kv, conj, empty and = dispatch to them on custom types. Plain objects and arrays keep their fast paths
- Add the IStack, IIndexed, IVector, IWriter and IPrintWithWriter protocols, write-all, and an ITransientVector -pop! slot; nth, peek, pop, pop!, subvec, vec, vector?, sequential?, set?, map?, seq, = and printing dispatch to custom collection types
- Add equiv, hash, hash-ordered-coll, hash-unordered-coll and the IHash protocol. hash follows equiv: plain mutable objects and arrays hash by reference
- Add the IMeta and IWithMeta protocols; meta and with-meta dispatch through them and the internal meta symbol property is gone
- clojure.set dispatches through the collection protocols: results keep the input&aposs type, membership tests against a protocol set are value-based, and rename-keys/map-invert no longer mutate a record
- Add defrecord, record? and the IRecord marker protocol. Records store their fields as own string-keyed properties and implement the map-facing protocols, so keyword lookup, keys, seq, assoc, conj and = work through the regular core functions. assoc keeps the record type, dissoc of a basis field gives a plain map, printing gives #TypeName{:a 1}
- Clojure 1.13 destructuring: :keys!/:syms!/:strs! for required keys, & inside them for keys required but not bound, :select, :all, :defaults, and :or by key
- Fix #975: & {:keys [...]} now destructures a map instead of the raw rest args, and a seq destructured as a map is read as kwargs
- Fix #977: recur inside try no longer emits an illegal continue
- Support :as-alias in ns :require like CLJS: no runtime import, only a compile-time alias so a namespaced keyword such as ::alias/x resolves
- Add :require-global and :refer-global to ns, binding globals loaded via a script tag to consts without emitting an import
- Add :squint/compile-time opt-in mechanism for macro/compile-time namespaces. See doc/compile-time.md
- A defmacro is compile-time only: no longer emitted to the runtime module, and :refering a macro no longer emits a runtime import for it, matching CLJS
- The CLI reports the file, line and column of a compile error and exits non-zero, instead of dumping the raw exception
- Fix #957: vite HMR: support ^:dev/after-load + ^:dev/before-load hooks similar to shadow-cljs
- .indexOf on a lazy seq now uses reference equality like a JS array, not value equality. This diverges from CLJS but keeps = out of any bundle that only builds lazy seqs, shrinking a conj bundle from 3801 to 2215 bytes
- Use Symbol.for for protocol method dispatch, so pulling in multiple copies of squint.core (e.g. via http://esm.sh/) does not break protocol dispatch
- Full changelog
Cherry: Experimental ClojureScript to ES6 module compiler
- Add cherry.test with clojure.test-compatible testing API, requirable as cljs.test or clojure.test
- cherry.test/report is a multimethod dispatching on [*current-reporter* type] like cljs.test, so reporting can be extended with defmethod
- Add a vite plugin with browser REPL over nREPL and ^:dev/after-load / ^:dev/before-load hot-reload hooks, sharing squint&aposs implementation: import cherry from &aposcherry-cljs/vite.js&apos
- Add reify, defmulti/defmethod and the vswap! macro. #&aposfoo emits foo&aposs value, like squint
- Dynamic vars compile to squint&aposs box scheme, so set! and binding work across ESM modules. cljs.core dynamic vars are exported as accessor boxes proxying the real var
- defprotocol :extend-via-metadata impls resolve under the fully qualified method symbol, so replicant&aposs mutation-log renderer works: replicant&aposs own test suite passes under cherry
- Fix deftype implementing cljs.core protocols such as Inst, IIterable and IAtom: their marker properties were Closure-renamed in the precompiled core and missing from the emitter&aposs core protocol set. The externs list and the set are now generated from cljs.core&aposs protocols (bb gen-externs) and the build fails on drift
- Fix #190: share PROTOCOL_SENTINEL with coexisting CLJS runtimes in the same JS realm
- Share the macro scan and macro lookup with squint. Namespaces flagged {:squint/compile-time true} load only their compile-time part into the macro environment, like squint
- CLI: --help/-h, argument validation and error messages via babashka.cli&aposs dispatch, like squint. Adds watch and nrepl-server commands, shell tab completion, and reads options from cherry.edn instead of squint.edn
- Fix emitted import specifiers on Windows: backslashes are normalized via the path resolution now shared with squint
- Full changelog
Choq: a ~5 MB binary running the cherry compiler on embedded QuickJS
- New project. Runs cherry inside quickjs-ng via rquickjs
- No JIT, so hot code is slower than Node.js, Bun or Deno, but the binary is small, startup is fast and memory use stays low. A Hono app serves around 30k requests per second locally, using less memory than the same app on Node.js or Bun
- An install script for macOS, Linux and Windows, and dev release binaries
- Clojure git and Maven deps, a module table covering url and util, @babashka/fs, and a test runner
- Experimental
Buzz: write a web application with the JVM or babashka only
- New project. Server state is watched and updated from client code. The UI compiles through squint and renders with Reagami, so no ClojureScript toolchain and no Node.js
- Rendering is asynchronous by default and coalesces at 20ms, and a failing render is contained to its own connection
- Examples: a whiteboard, a tap viewer, and a Datalevin browser with a CodeMirror query editor
- Highly experimental, the API will change
tube-pod: turn YouTube videos into a private podcast
- New project, written with Buzz. Add a link in the browser, tube-pod downloads the audio with yt-dlp, writes an RSS feed and serves both
- Rsyncs the audio and the feed to a remote after each change, since a laptop is asleep when you want to listen
multi-snake: snake for as many players as show up
- New project, written with Buzz. Everyone plays on one board, in one world, held in one atom on the server
- Runs at multi-snake.michielborkent.nl
Reagami: A minimal zero-deps Reagent-like for Squint and CLJS
- Add reagami.ssr to render hiccup to an HTML string on the JVM, Babashka, Squint and CLJS. See Server-side rendering
- reagami.core/render (the regular render function) now hydrates a server-rendered page. It adopts the existing DOM instead of clearing the root
- Add create-reagami-app. Run npm create reagami-app my-app to create a Vite project with hot reload and a browser nREPL
- Breaking: :on-render now takes a map: (fn [{:keys [node lifecycle state save]}]). Call save with a value to keep it for the next call, and read it back as state. In previous versions, the hook took three arguments and its return value became the state
- Move reordered nodes with moveBefore where the browser has it, so a moved subtree keeps its iframe state, animations, focus and selection (#54)
- Set value, checked, selected and disabled on a tag with a hyphen as attributes, not as JS properties. A custom element observes attributes, so a property never had any effect. Native elements still handle them as properties
- Custom events, e.g. :on-rated, now reach the element through addEventListener, because a browser only wires an on* property for standard events
- Add web component example. A <todo-list> custom element, used from Squint, from JavaScript with and without Reagami
- Fix memory leak with :on-render nodes and other :on-render improvements
- I gave a talk about Reagami at Func Prog Sweden
cljbang: a Clojure-like language that runs as Emacs Lisp
- Compiles Clojure forms to Emacs Lisp forms and evaluates them in the running Emacs. No subprocess and no transpiled text, following the same approach as squint
- Namespaces with per-namespace aliases, multiple arities in fn and defn, loop/recur with a tail position check, try/throw/ex-info, case, atoms, syntax quote including nesting, &form and &env in macros, regex and set literals, #_, edn/read-string, slurp and spit
- el! for calling Emacs Lisp names that are not valid Clojure symbols
nbb: Scripting in Clojure on Node.js using SCI
- ClojureScript JIT compilation. Nbb now bundles a SCI that compiles interpreted function bodies to JavaScript at runtime via js/Function. This is enabled by default. This makes loops, numerical computations and JS interop much faster
- Nbb now ships babashka.fs as a built-in library. The full file system API (glob, copy, move, create-dirs, delete-tree, with-temp-dir, path helpers and more) is available via (require &apos[babashka.fs :as fs]), matching Babashka
- Support implementing CLJS protocols (e.g. ILookup, etc) on deftype and defrecord
- Support editscript: CLJS deftype/defrecord field interop, set! on ^:unsynchronized-mutable fields, add cljs.core type classes like PersistentHashMap, write-all and goog.math.Long
- #416: Fix problem with prn in nREPL
- SCI now covers most CLJS capabilities, so nbb should run existing CLJS libraries unless they rely on very specific macros that require the JVM. If you have anything that does not run, please report it in #nbb!
Scittle: Execute Clojure(Script) directly from browser script tags via SCI
- ClojureScript JIT compilation. Scittle now bundles a SCI that compiles interpreted function bodies to JavaScript at runtime via js/Function. This is enabled by default
- Include helitorus demo to show improved JIT
- Bump reagent to 1.2.0, re-frame to 1.4.7, replicant to 2026.06.2 and shadow-cljs to 3.4.11
squint-inline: write squint functions and inline expressions in a ClojureScript project
- New project. Squint operates on JavaScript objects and arrays, so assoc, update-in and select-keys work on those without js->clj and clj->js
- Squint core is tree-shaken through :js-provider :import, and each function&aposs tree-shaken size is recorded
- Squint functions can call each other across namespaces, and JS module references work inside squint bodies
Edamame: configurable EDN and Clojure parser with location metadata and more
- Speed up parsing by holding parse context in record fields instead of the extmap: ~10% faster on JVM, ~4% on ClojureScript
- Respect :refer + rename in :auto-resolve-ns
- With :auto-resolve-ns, qualify syntax-quoted imported classes (e.g. `Date with (:import [java.util Date])) with the full classname
- With :auto-resolve-ns, leave method, constructor and dotted syntax-quoted symbols (`.toString, `Bar., `foo.bar) as-is, matching Clojure
- Do not resolve function literal params in a syntax quote
- ClojureDart: fix parsing zero literals and make plain readers non-indexing, matching tools.reader (#144)
fs: file system utility library for Clojure
- Released 0.5.34, which ships the Node.js support mentioned in the previous update as the @babashka/fs npm package
http-client: HTTP client for Clojure and babashka
- #80: accept a function of the request URI in :proxy to select a proxy per request (@jeeger)
http-server: serve static assets
- Range requests: inclusive Content-Range last-pos per RFC 9110, suffix ranges (bytes=-N, previously a 500), clamping last-pos beyond EOF, reading the full range, and a test suite (@slagyr)
Cream: Clojure + GraalVM Crema native binary
- Reduced the core.async virtual thread memory corruption I reported upstream to a pure Java repro. GraalVM 25.0.3-ea.04 fixes it, and the pipeline test is back on now that the compile NPE is gone too
- Enable the Ristretto JIT for runtime-loaded bytecode, and update the benchmarks for it
- Clojure code runs without a JDK present, with the boot class loader warning suppressed
- Pick up pom.xml when there is no deps.edn, and recompile Java sources when a dependency changed
graaljs-cherry: a native-image cherry REPL on GraalJS
- New prototype. Compiles cherry expressions on the JVM and evaluates the resulting JS in an embedded GraalJS context
- Two variants: a default Truffle JIT build, and a 49MB --small build without it
clj-kondo-browser: a static Clojure source browser built from clj-kondo analysis
- New prototype. Renders a codebase as a static HTML page where every symbol links to its definition and usages, scope-aware, so a local is linked only within its scope
- Runs clj-kondo as a pod and gets the classpath from deps.clj
grasp: Grep Clojure code using clojure.spec regexes
- Babashka compatibility (#34)
deps.clj: a faithful port of the Clojure CLI Bash script to Clojure
- As always, catching up with the most recent Clojure CLI versions
lein-clj-kondo and clj-kondo-bb: released alongside each clj-kondo release

Other projects

These are some other projects I&aposm involved with, but little to no activity happened in the past two months.

Click for more details

quickblog: lightweight static blog engine for Clojure and babashka
quickdoc: Quick and minimal API doc generation for Clojure
neil: A CLI to add common aliases and features to deps.edn-based projects
pod-babashka-go-sqlite3: A babashka pod for interacting with sqlite3
pod-babashka-gozxing: a babashka pod for QR code and barcode decoding/encoding
unused-deps: Find unused deps in a clojure project
pod-babashka-fswatcher: babashka filewatcher pod
sci.nrepl: nREPL server for SCI projects that run in the browser
babashka.nrepl-client
sci.configs: A collection of ready-to-use SCI configs.
rewrite-edn: Utility lib on top of rewrite-clj
rewrite-clj: Rewrite Clojure code and edn
tools-deps-native and tools.bbuild: use tools.deps directly from babashka
bbin: Install any Babashka script or project with one command
graal-build-time: initialize Clojure classes at build time for GraalVM native-image
html: HTML generation library inspired by squint&aposs HTML tag
qualify-methods: experimental tool to rewrite instance calls to use fully qualified methods (Clojure 1.12 only)
tools: a set of bbin installable scripts
babashka.json: babashka JSON library/adapter
speculative
squint-macros: a couple of macros that stand-in for applied-science/js-interop and promesa to make CLJS projects compatible with squint and/or cherry.
http-kit: Simple, high-performance event-driven HTTP client+server for Clojure.
jet: CLI to transform between JSON, EDN, YAML and Transit using Clojure
lein2deps: leiningen to deps.edn converter
cljs-showcase: Showcase CLJS libs using SCI
babashka.book: Babashka manual
pod-babashka-buddy: A pod around buddy core (Cryptographic API for Clojure).
gh-release-artifact: Upload artifacts to GitHub releases idempotently
carve - Remove unused Clojure vars
4ever-clojure - Pure CLJS version of 4clojure, meant to run forever!
pod-babashka-lanterna: Interact with clojure-lanterna from babashka
clj2el: transpile Clojure to elisp
deflet: make let-expressions REPL-friendly!
deps.add-lib: Clojure 1.12&aposs add-lib feature for leiningen and/or other environments without a specific version of the clojure CLI
process: Clojure library for shelling out / spawning subprocesses
parmezan: fixes unbalanced or unexpected parens or other delimiters in Clojure files

Permalink

Lazuli, beyond Ruby: Python, JavaScript, and Elixir

It’s been a while since I’ve been working on Lazuli – the editor plug-in for Pulsar (and probably VSCode in the near future) where I try to bring the Clojure REPL experience – that is, running code inside your editor and printing the result inside your editor, without having to copy-paste between two different tabs – to other languages.

If you didn’t read the previous post, the TL;DR; is: Lazuli is basically “Chlorine, but for more languages than Clojure”. Ruby was the first target, for two reasons – first, because I decided to go back to the Ruby/Rails stack; and second, because it’s probably one of the easiest languages to implement that on.

Since then, I’ve added support for Python, and I’ve been experimenting with JavaScript and Elixir. Each of them ended up being its own can of worms, and I think it’s worth writing down what I found – because some of the “obvious” solutions turn out not to work at all.

A quick refresher on watch points

I explained watch points in the previous post, but here’s a very short version: a watch point is a binding around a method or function – all the local variables, self/this/whatever the language calls it, class variables, and so on. When you evaluate something in the editor, the nREPL tries to match a watch point at that line, so if you evaluate self.some_method it knows where you are – the instance, the class variables you have defined – and it evaluates the code as if it was running inside your already-running process, be it a webserver, an app, etc.

I originally called them “watch points” precisely so I would not bind the name to a single language. The concept is the same everywhere – only the implementation changes. And that’s what this whole post is about – the different levels of pain each language gives you when trying to implement them.

Python – the easy one

At my job I started to work with Python, so I did the same thing I did for Ruby: implemented an nREPL server, and added support for parsing the code in duck-repled. Then, at the nREPL layer, I had to implement some kind of “watch point”, but using what Python offers me.

For Ruby, I used binding to define how the server captures the context. For Python, I used the frame object that the setprofile API gives you – it plays essentially the same role. Both languages support manually-defined watch points and automatically-defined ones, and they map surprisingly well to the same concept. Updating the watch points (and patch, and imports) are harder in Python because in Ruby, you can evaluate something passing binding as a parameter – you can’t do that in Python, and instead you need to convert the frame to locals and globals “dict” objects, pass those around, and then replace the watch point with the result (so that it’s not a frame anymore, but a dict containing the values). But it works, although it does feel like cheating (you are essentially faking an environment to the Python eval function) and sometimes can cause some weird issues – for example, let’s suppose I have this code:

class Example:
  def m1(self, a, b):
    return a + b

  def m2(self, a, b):
    return a - b

Supposing I have a watch point on m1, then I add an import re at the first line, before the class. That will “alias” an re inside m1, but not inside m2. Lazuli supports “patching” so I can “patch” m1 to use a new implementation, but if I do evaluate a code that adds a watch point on m2 later… it’ll gladly say that re doesn’t exist. The reason is that the “import” only exists for things that have watch points (it’s confusing, I know) because namespaces in Python don’t get profiled and don’t get their own watch points (so when you evaluate an import, it’ll only update existing watch points, not new ones that might appear later – and for the interpreter, that current namespace can’t be changed).

The printing part of Python was actually easier than Ruby. In Ruby, there is no one-size-fits-all for inspect – so many gems override how things are printed that I gave up trying to intercept the output, and instead I decided to parse the Ruby result and instance variables, properties, and everything else into a structure based on what Ruby actually gives me. Not perfect, but it works. In Python, not many people change the way outputs are presented, so the parser-based approach was way less painful (I decided to implement a structure similar to hiccup, but containing what most languages offer – so arrays and strings are supported, and nothing else. To say that it’ll print a “number” in the screen, you return ["number", "10"] for example).

So – Python: mostly a rerun of Ruby, with fewer surprises. Good.

Then I decided to try JavaScript.

JavaScript – the DevTools Protocol saves us (almost)

JavaScript is where things start getting complicated.

The whole idea of Lazuli, and of the nREPL I’m building, is that you’re running your normal code, and you just add a thin server that can somehow connect to whatever is running right now and then evaluate commands. You don’t change anything in your codebase – you add a require, import, or similar, and whatever you bring needs to be as unintrusive as possible – for example, it can’t depend on additional libraries, because otherwise you need them installed locally, and that’s not how some languages work anyway (the first version of the Ruby nREPL depended on a BEncode library, and that was a no-go – I would have to include the nREPL into the Gemfile for example).

So, what are the main problems in JavaScript? Well, you can’t just “connect into” a JS virtual machine – you need to open a server. But if you’re in a web app, you can’t start a local socket in your machine (or any server for that matter) your webpage needs to be connected to a server, and it needs to be a WebSocket. But the socket needs to be running in the server that is offering the JavaScript, so completely unacceptable (and even if it was, it would not work – it would need to “rewrite” the JS before sending to the browser, but a server can, and will, send multiple JS files – which ones is the “right one” to connect to the websocket?)

To solve that, Lazuli uses the DevTools Protocol. Both Node.js and the browsers support it – you can tell a browser that DevTools is available for your localhost machine, and, surprisingly, that just works (well, it opens a devtools channel for every page but you can just filter for the one you want and that will work). But we gain access to more things by using this protocol – one of them being the debugging API, which is actually how we add watch points to the JavaScript nREPL.

But, as always, there’s a problem. Well, actually, three of them.

Problem 1: source maps

JavaScript is not a compiled language, but most people use some kind of bundler. If you’re using React, for example, you’ll write JSX inside your JavaScript, and that will get transformed – changing lines, changing names, wrapping things.

So when you get an exception (or when you’re trying to figure out where a watch point should go), you have to parse that through a source map. And unfortunately, for the Lazuli project, we can’t escape this. There are ways to avoid minifying code – in fact, on Pulsar we actually do that – but it’s simply not how most JavaScript development works nowadays. The solution is to try to do a “reverse source map”. Essentially: if I evaluate something like a in my editor, that has to map to something that was already transpiled to the final JavaScript code, and then I evaluate that transpiled name over there. It works, but it’s flimsy and still not perfect – for example, supposing a const value = 10 gets transpiled to var a=10, then later the bundler found that this variable isn’t referenced anymore and decided to reuse const otherValue = 20 to a=20 – the “original” value, containing 10, will be lost forever, and there’s no way we can avoid this. Most bundlers don’t do that in development, luckily, but it’s still an issue that might happen.

Problem 2: functions that don’t exist

The second problem is way more complicated. And it was a surprise for me.

If you have three functions in a file, and only two of them are used, then the third one will never exist. Not “will not be called” – literally does not exist. For some reason, it seems that either the JavaScript engine garbage collects the function even if it’s in a top-level global namespace, or – more likely – it simply never actually compiles it to bytecode. For example, in the code below:

function unused(a, b) {
  return a + b
}

function functionOne(a, b) {
  return a / b
}

export function functionTwo(a, b) {
  return functionOne(a + b, 2)
}

Now, supposing I have a watch point on functionTwo. Changing the code of functionTwo to call functionOne twice works; inspecting a and b on both functionTwo and functionOne works too. Trying to call parseInt inside functionTwo also works, but trying to call unused won’t work – ever. Even just typing unused and evaluating it will return undefined, because the JS virtual machine simply “erases” that function.

This might pose a problem for future Lazuli features – if the whole point is to be able to interactively evaluate anything, and if half of what you wrote isn’t there anymore, that’s a hole can of works. That might be possible to mitigate with the same technique as Python, though – maybe we can “fake” it by making “evaluate top block” produce a “global-ish” identifier, then update manually each watch point so that this identifier is in scope – but it’s still kind of hard to make it work.

Problem 3: namespaces and patching

And that’s not all. JavaScript is a very different language in the context of namespaces and files – there’s some magic going on around ESM that’s different from CommonJS and other stuff, and it’s close to impossible to “patch” a function. There’s a Chrome DevTools API that will try to patch things, but it only works with some very strict constraints that are not really clear what they are (and I mean very strict – for example, adding a new line won’t work).

One thing I’m thinking about is to make a Babel transformer that injects functions with their own inspect handlers, and somehow keeps a global state of these functions so that they can never be garbage collected. This might work, but I don’t yet know if it’s a good idea – but if it does, we could have Lazuli working perfectly in a JavaScript environment, with just some changes to whichever bundler you’re using (it’s not fully unintrusive as I wanted, but if it’s the only way, that might be the path).

Elixir – the interesting one

Elixir is the last language I’m working with, because I find it very interesting, and I’m checking whether it’s possible to replicate the Ruby experience there.

Some things seem easy. Some things are hard. And some things might be impossible.

Private functions are invisible

The first problem is that private functions in Elixir are not visible to my plugin. Elixir has some ways to get the bindings and the environment around a call, and context – but because I don’t know Elixir that well yet, I don’t fully understand:

what’s the reason for using one of those over the other,
why we have to use both sometimes,
and how (or if) I can capture the whole context, including private functions.

For now, seems that private functions (defined with defp) are not “compiled” into the bindings and the env. That complicates things, because I can get the “local variables” part of my watch point, but if I try to call a public function, that works… but private ones don’t. I could, theoretically, re-create the private function as a “public” one while I am evaluating the watch point just to know what is the result, but then I get into the second problem:

The VM is immutable, kinda

Another problem with Elixir is that the BEAM is immutable – kinda. You can’t just patch a function and make every caller in the runtime use the new version. That literally isn’t possible at the VM level. You can have some of this approach with GenServer and other constructs, but I don’t fully believe that this will actually patch stuff in production the way REPL-driven development expects – because your code needs to be a “GenServer” already, so it’s essentially easier to just save the file and hot-reload it.

The issue is – a REPL-driven development is meant to be a way to test solutions without needing to save the file. I can write “fragments”, evaluate them, patch functions, evaluate them, repeat, until I’m satisfied with the output – then I will save the code and allow either the hot-reload of my environment do its magic, or reload the whole file and check if I didn’t introduce any bug. With this approach, a “evaluate top block” might not even be useful for Elixir, honestly.

Bindings only exist at call time

Elixir does have the bindings of a function – but only when the function is called. So you know which local parameters are passed in, you know which global ones are used, but you don’t actually know what are the variables that you create over time inside the function.

Python, with locals and globals, have the same problem. But we can bypass that by storing the frame object, and when I evaluate the code in the editor the first time then it’s converted to the dicts. Unfortunately, this won’t work in Elixir, and I’m not sure it’ll work at all, ever – even if we had some way to capture “just before the end of the function”, we would still capture only when nothing crashed – and capturing when something crashed is probably the biggest reason for a watch point now.

Shadowing

And here’s the one that might complicate things a lot: variables are shadowed. Suppose I have this code:

module Something do
  def example() do
    a = 10
    fun = fn -&gt; a + 1 end
    a = 20
    fun2 = fn -&gt; a + 1 end
  end
end

If we define a watch point on this example function, then start to evaluate line per line, we’ll update the watch point with a=10, fun=<function>, then we’ll update the a to be 20, and then define a fun2. The issue is – fun.(), in this case, will return 11 – because that was the value of a at the time – and fun2.() will return 21 for the same reason. This is the happy path… but if I want to understand why the value of fun was 11, I can try to select its inner body – that is, a + 1 – and then evaluate it. BUT – watch points are, unfortunately, tied to a single point in a specific file and line, and they are updated when we evaluate code – meaning that evaluating that selection would return 21, confusing things.

The solution could be to add some “metadata” for each evaluation, meaning that it’ll redefine where that variable was defined, and then evaluate could capture the scopes but only consider variables defined before or at the current line. I don’t think this is easy or simple to do (considering that changing the code inside the editor also needs to update where variables were defined and also the editor will need to be “deletion and change aware” (if we rewrite a to a1, what happens?) and there might be even more trade-offs that I didn’t think about. Considering that I don’t know Elixir that well that might cause such a huge amount of bugs that this might be too difficult for now.

Final thoughts

Lazuli is growing, and the project – in my view, at least – is quite interesting. It might be bringing some of the superpowers from Clojure REPL-driven development that people love to other languages.

I’m still not comfortable with the level of tests that I have, because I really want to avoid breaking existing REPLs when I update the plug-in, and vice versa – and I also want to keep adding languages, some of which I don’t personally use at work (I don’t use Python anymore, and I only use Elixir for personal experiments).

Another thing about Lazuli is that it sometimes feels like it’s moving in the diametrically opposite direction of what people want to do nowadays. Lazuli is a project to bring the code and the developer closer together – to the point that you’re evaluating code live while you’re typing it. But we live in a world of LLMs, where people want to distance themselves from the code. I, honestly, don’t think that’s the right position to be betting on – especially when LLMs are not perfect yet, and I don’t know if they ever will be. There is still a lot of misinformation, broken promises, and hype around AI-assisted coding – some people trust too much on the ability of the AI agents. Some are even comparing source code to assembly language and to the binary that runs on your computer, which is completely absurd – compilers are (supposedly) deterministic, and LLMs are not – by design.

So maybe, if the whole LLM boom proves to be a great mistake, and people end up with millions (or even billions) of lines of code that they don’t understand, and it simply doesn’t work… maybe Lazuli can help in the future.

Or maybe I’m completely wrong and I’ll move to a different position.

But here’s the thing: I still believe that as developers, we need to understand our creation – be it written by us, or by a machine.

Permalink

Clojure 1.12.6

Clojure 1.12.6 is now available! Find download and usage information on the Downloads page.

Changes since previous release:

CLJ-2794 - gen-class - incorrectly treats interface default methods as abstract, throwing when the var delegate is unbound
CLJ-2974 - FnLoaderThunk - de/serialization no longer supported, throw

Permalink

Announcing DatomicConf 2026

Have you been thinking you need another conference to go? Maybe in lovely Durham, North Carolina? We thought you might, so we made one. Later this year we’re hosting DatomicConf 2026.

Join us on Friday, December 11, 2026, in Durham, North Carolina, for a one-day, single-track conference dedicated to Datomic and the ideas and community around it. There will be news from the Datomic team, real-world Datomic use cases and opportunities to connect with other people who are building reliable, thoughtful systems.

Registration is free but limited, so if you’re planning on attending, register now.

Our Call for Proposals is also open and ready for your submissions.

If you’re interested in the possibility of a livestream, let us know at the livestream interest form.

All the details right now are at conf.datomic.com. We’ll soon share more information about hotels and visiting Durham. Check in on the #datomic channel at the Clojurians Slack for updates or reach out to us at conf@datomic.com with any questions.

Permalink

Introducing Plotje: Composable Plotting

An introduction to Plotje, a composable plotting library built on Membrane, using Dewey’s index of Clojure repositories as the running example

Permalink

Babashka 1.13.220 gets FFI

Today babashka 1.13.220 is released, with a new babashka.ffi namespace for calling C libraries directly from Babashka scripts. The babashka.ffi library is also available as a standalone library for JVM Clojure, so you can use it in your Clojure projects as well. Note that the API is still experimental, although no changes are currently planned. It just needs more exposure and your feedback :). Here&aposs a small demo.

Calling C

This example loads libz from your system and requests the version.

(require &apos[babashka.ffi :as ffi :refer [defcfn]])

(def zlib (ffi/load-system-library "z"))
(def zlib-version (ffi/cfn zlib "zlibVersion" [] :string))

(zlib-version)
;;=> "1.3.1"

This example loads an OS-specific library for doing math:

(ffi/load-library
 {:mac "libm.dylib"
  :linux "libm.so.6"
  :windows "ucrtbase.dll"})

(defcfn cos "cos" [:double] :double)
(defcfn pow "pow" [:double :double] :double)

(cos 0.0)      ;;=> 1.0
(pow 2.0 10.0) ;;=> 1024.0

To get a feeling for how to use it in larger, non-trivial projects, read the library guide. Some of the API decisions like defcfn are clearly inspired by coffi, so I want to thank Joshua Suskalo for leading the way with his excellent library. But babashka.ffi is not simply a copy of coffi. It does a few things differently. You can provide an explicit library (or a function or delay that resolves to one) to defcfn for example. Also it has a place concept (inspired by Specter&aposs paths) that efficiently lets you read from and write to structs and unions. Like coffi, babashka.ffi builds on java.lang.foreign and makes you manage memory explicitly through arenas. One benefit of this is that you&aposll get exceptions rather than segfaults that tear down your REPL and you can use with-open to release allocated memory.

Install

To use babashka.ffi and libraries that build on it, you have to use a dynamically linked version of babashka. On Mac and Windows this was always the default. On Linux, the static binary was preferred historically since it did not depend on your system&aposs libc and zlib. In this release we flip this default to a mostly-static binary: all the shared C libraries that babashka needs are statically linked, and glibc is the only dynamically linked part. The aarch64 binary, although it carries -static in its name, was already built this way. Babashka on Linux is built in a container that pins the glibc version to the lowest one possible so it should work on all mainstream LTS versions of Linux today. If you still prefer the fully static binary, you can use the install script with the --static flag. If you use a package manager or a GitHub Action to install babashka, it may not yet be up to date with this new policy. If that is the case, feel free to open an issue at the babashka GitHub repo and I&aposll reach out to get this fixed. Meanwhile you can install babashka using the installer script on GitHub to a temporary directory to get a second installation of babashka with FFI enabled:

$ curl -sLO https://raw.githubusercontent.com/babashka/babashka/master/install
$ bash install --dir /tmp/bb-test
$ /tmp/bb-test/bb -e "(require &apos[babashka.ffi :as ffi]) (ffi/load-system-library \"z\")"

The installer script probes your system for the supported glibc version and falls back to the fully static version when necessary.

Demos

To validate the design of babashka.ffi, I built a few shiny demos:

pacman.clj: pac-man with the classic ghost personalities (requires raylib)
doom.clj: a raycaster with textures and sprites (requires raylib)
helitorus.clj: a helix around a torus (requires raylib)
gtk4.clj: a native GTK 4 window rendering from an atom
portaudio.clj: an arpeggio through a realtime audio callback
python.clj: embedded CPython calling back into Clojure

A one-liner to try these demos:

$ bb -e &apos(load-string (slurp "https://raw.githubusercontent.com/babashka/ffi/main/examples/pacman.clj"))&apos

FFI-based libraries

To validate the design of babashka.ffi even more, a couple of new libraries were born. These libraries mostly resemble existing pods but now use FFI to fulfill similar use cases.

One cool thing you could not do with a pod before is defining a Clojure function in SQLite:

(require &apos[babashka.sqlite :as sq])

(sq/with-conn [db nil]
  (sq/create-function! db "initials"
    (fn [s] (apply str (map first (clojure.string/split s #" ")))))
  (sq/query db ["select initials(?) i" "gerald jay sussman"]))
;;=> [{:i "gjs"}]

Tasks: :exec-fn composition

This release also has some really nice task improvements: :exec-fn tasks now compose through :depends. A task can depend on another CLI task, and the dependency&aposs options parse, coerce and show up in --help and shell completion:

{:tasks
 {compile {:exec-fn build/compile-sources
           :cli {:spec {:release {:coerce :boolean}}}}
  jar     {:depends [compile]
           :exec-fn build/jar}}}

$ bb jar --help
...
Inherited options:
  --release

Also you can now directly provide :exec-args on a task:

{:tasks
 {deploy {:exec-fn deploy/run
          :exec-args {:env "staging"}}}}

A :cmd tree can now be provided through a var, whose namespace is loaded on demand:

{:tasks
 {cli {:cmd my.project.cli/commands}}}

AI disclosure

While developing FFI and while validating the design through examples and writing libraries, I have made use of LLM assistance.

Wrapping up

Hope you&aposll like these new features!

The full changelog can be found here.

Permalink

AI "features" and integrations should remove friction, not add it

AI is everywhere now and most people hate that, at least in the West. I can understand why. I have to use AI daily now for my work as a software engineer, and increasingly many things I build are effectively some version of “use an LLM to make this feel like magic”. I can appreciate the sentiment behind this in the sense that LLMs appear remarkably capable when asked simple questions about a number of topics, but it is also way too easy to just throw an LLM into the mix and actually end up making everything worse. The industry has vastly underestimated the amount of engineering discipline it takes to make LLMs actually complete tasks reliably or do anything useful from a normal person’s perspective. Which IMO is a big part of what is behind the general disapproval of AI outside the tech industry (among many other reasons).

The overarching principle I wish all product engineers would adopt in their approach to integrating “AI” is that it should always remove friction, not create it. If your “agent” is effectively just throwing up walls of text for me to review, that is not helpful. Other things that are not helpful:

making decisions that I have to figure out how to go double check or undo
doing things on my behalf that I have to correct
impersonating me

Hopefully you get the idea. If you are embedding AI into a product that people use, please ask yourself before you ship “does this make my user’s life better or worse?” If it adds friction to their experience in any way, do not ship it. Fix it first so that your users genuinely benefit from this new integrated “intelligence”, such as it is, and make sure it’s not just another thing they have to work around or figure out how to disable to use your product.

note / #ai #friction #llms #product-development #software-engineering #user-experience

Permalink

Decoupling from the Data POV: Stop-and-Start Boundaries, Independent Pointers, and Why Your Code (and AI) Need It

The Blind Spot in Modern Architecture Debates

Ask five engineers what "decoupling" means, and you will get five abstract answers about SOLID principles, hexagonal layers, microservice boundaries, or dependency inversion interfaces.

Almost nobody talks about decoupling from the point of view of the data itself.

The Fundamental Law:

If you decouple the data, the logic decouples automatically.

If you only decouple the logic while sharing mutable data, you haven't decoupled anything.

What does data actually experience as it moves through a running system? Is it continuously tethered across shared memory, or does it move across clean, discrete boundaries?

Understanding the physics of data decoupling—specifically independent memory pointers and stop-and-start boundaries—not only transforms how you structure production software, but also unlocks how we solve the two biggest bottlenecks in modern engineering: Human Snippet Tunnel Vision and AI Context Amnesia.

1. The Physics of Coupling: The Shared Pointer Trap

In a tightly coupled codebase, modules don't just depend on each other conceptually—they are physically tethered in RAM.

❌ COUPLED DATA (Continuous Live Tether / Shared Mutable Pointer):

   Pointer A (package auth) ────┐
                                ├──► [ RAM Memory Slot: 0x7FFE4A20 ]
   Pointer B (package payment) ─┘    Data: { UserID: 42, Status: "Active", Balance: 100 }

   * Danger: If auth.go mutates the status or alters the memory layout, 
     payment.go reads corrupted state or fails at runtime without warning.

When multiple packages hold pointers to the same mutable memory block:

Temporal Coupling: Package A and Package B must execute in lockstep. You cannot delay, retry, or parallelize one without coordinating locks.
Invisible Side Effects: Changes made inside auth.go propagate silently across the heap into payment.go.
The "Decoupling Illusion": Even if you wrap both packages in clean interfaces, if they are still passing shared mutable pointers underneath, they are not decoupled.

2. The Decoupled Mental Model: Stop-and-Start Boundaries

True data decoupling happens when data moves in discrete "stops and starts" across explicit boundaries.

Instead of sharing a live pointer, each system holds an independent pointer pointing to its own isolated memory allocation:

✅ DECOUPLED DATA (Independent Pointers & Stop-and-Start Handoff):

   [ Stage 1: Auth Engine ]
      Pointer A ──► [ Local Buffer 1: { UserID: 42, Status: "Active" } ]
                           │
                           ▼ (Serialization / Value Handoff)
                    [ Boundary / SQLite / Queue / Channel ]  <── "STOPS" (Data at rest)
                           ▲
                           │ (Deserialization / Local Allocation)
   [ Stage 2: Payment Engine ]
      Pointer B ──► [ Local Buffer 2: { UserID: 42, Status: "Active" } ]

Why Independent Pointers Win:

Spatial Isolation: Pointer A lives only in Auth's scope; Pointer B lives only in Payment's scope. If Pointer A is mutated or garbage collected, Pointer B remains 100% intact.
Temporal Independence: The handoff boundary acts as a temporal air gap. Auth can run at 10:00:01 AM, write to the boundary, and shutdown. Payment can wake up at 10:00:05 AM and process the payload.
Reference by Identity (IDs), Not RAM Addresses: Instead of passing raw RAM addresses (0x7FFE4A20), decoupled systems pass IDs (UserID: 42 or node_id: "ast_func_402"). Each component queries or constructs what it needs.

3. Where Does the Data "Stop"? (RAM vs. Disk)

You can place your stop-and-start boundaries in two places depending on your performance and persistence needs:

┌──────────────────────────────────────┬──────────────────────────────────────┐
│       IN-MEMORY RAM BOUNDARIES       │       PERSISTENT DISK BOUNDARIES     │
├──────────────────────────────────────┼──────────────────────────────────────┤
│ • In-memory SQLite (`:memory:`)      │ • Local SQLite files (`synapse.db`)  │
│ • In-RAM JSON / DTO string snapshots │ • File system snapshots (.json/.pb)  │
│ • In-memory Go Channels (`ch <- v`)  │ • Message queues (Kafka/RabbitMQ)    │
│ • Deep clones & move transfers       │ • Write-Ahead Logs (WAL / Append)    │
│ • Pass-by-value stack copies         │ • Embedded KV stores (RocksDB/Pebble)│
│                                      │                                      │
│ Speed: Microseconds to Nanoseconds   │ Speed: 1–2ms (via OS page cache)     │
│ Scope: Same process / local runtime  │ Scope: Cross-process / Air-gapped    │
└──────────────────────────────────────┴──────────────────────────────────────┘

4. The Double Crisis in Modern Codebases

As codebases scale past tens of thousands of lines, this data flow problem triggers two simultaneous breakdowns:

+-------------------------------------------------------------------------+
|                         YOUR ENTIRE REPOSITORY                          |
|    [auth.go]      [payment.go]      [user.go]      [db.go]   [queue.go] |
|                                                                         |
|            +---------------------------------------+                    |
|            | YOUR IDE VIEWPORT (30-50 lines)       |                    |
|            | Editing line 42 in auth.go...         |                    |
|            +---------------------------------------+                    |
|                                                                         |
|    * Blind to cross-package blast radius & contract mutations! *        |
+-------------------------------------------------------------------------+

1. The Human Problem: Snippet Tunnel Vision (The Straw Problem)

Standard IDEs (VS Code, JetBrains) show 30 to 50 lines of code at a time. Trying to comprehend complex data flows through a 50-line viewport is like peering into a skyscraper through a drinking straw. You cannot see the blast radius of your changes.

2. The AI Problem: Context Window Amnesia (The Overflow Problem)

Autonomous AI coding agents (Claude Code, Cursor, Windsurf) struggle when developers dump 50 raw source files into the prompt window:

Token Inflation: $20+/hr in API fees burning context on boilerplate.
Mental Pointer Tracking: The LLM is forced to mentally simulate live data pointers across 50 text files, leading directly to hallucinations and broken imports.

5. The Solution: Treating Code Itself as Decoupled Relational Data

When I ran into these two friction points on large projects, I realized the answer wasn't to write another static linter or dump more raw text into an LLM prompt.

The answer was to apply data decoupling principles to the codebase itself:

The Stop-and-Start Boundary: Parse the repository's Abstract Syntax Tree (AST) and LSP symbols into a local, relational SQLite database (synapse.db), and let the parser terminate.
Independent AI Pointers via MCP: Instead of forcing an AI agent to read 50 raw text files, expose the SQLite database via a local Model Context Protocol (MCP) server. The agent runs recursive SQL queries in 2 milliseconds, retrieving exact dependency graphs without swamping its context window.
Independent Spatial Canvas: Wire the relational tables into a local 2D visual canvas (http://127.0.0.1:8080). When an engineer or AI agent refactors a module, the canvas lights up the blast radius and traces data taint flows in real time.

6. How Different Languages Decouple Data

Every major language runtime has wrestled with this problem, producing some ingenious data-decoupling mechanics:

1. JavaScript: `structuredClone()` & Transferable Objects

Many developers still use JSON.parse(JSON.stringify(obj)) for deep copies, which silently strips functions, undefined, Date objects, and crashes on circular references. Modern JS includes structuredClone(), which creates a 100% isolated heap allocation and correctly clones circular graphs, Map, Set, ArrayBuffer, and Blob (though functions and DOM nodes still throw a DataCloneError). Even faster: Transferable Objects (postMessage(buffer, [buffer])) completely transfer memory ownership from the main thread to a Web Worker, instantly zeroing out the sender's pointer for zero-copy concurrency.

2. Erlang & Elixir (BEAM): The Zero-Shared-Heap Actor Model

Unlike Java, Node, or Go (where threads share a single global heap), every single Erlang/Elixir process has its own private heap and private garbage collector (with off-heap reference counting for large binaries >64 bytes). When one process sends a message to another, the BEAM VM physically copies the bytes across process heaps. There is literally no shared mutable memory in the entire VM—making deadlocks and race conditions structurally impossible.

3. Clojure: Persistent Data Structures (HAMT)

How do you update an immutable collection with 1,000,000 items without copying the entire array every time? Clojure uses Hash Array Mapped Tries (HAMT). When you "modify" an immutable map, Clojure shares 99.9% of the existing tree nodes (structural sharing) and only allocates a tiny new path of 3–4 nodes. Because of its 32-way branching factor (M=32), you get a brand-new, decoupled immutable snapshot in Olog₃₂n (effectively bounded O(1)) time with minimal memory overhead.

4. Rust: Compile-Time Move Semantics

Rust takes a different route: instead of copying memory or running a garbage collector, it enforces single ownership at compile time. When you pass data to a new function, Rust moves ownership and marks the original pointer as invalid in the compiler. If you try to read from the old pointer on the next line, the code won't even compile—giving you zero-cost pointer isolation with zero runtime overhead.

5. Go: Channels & Value Receivers

In Go, structs are value types by default (b := a copies top-level fields, though beware that inner slices, maps, or pointers still share underlying backing storage). When pairing goroutines, Go favors channels (ch <- msg) to transfer data across memory boundaries without shared locks: "Do not communicate by sharing memory; instead, share memory by communicating."

6. SQLite as the Universal Polyglot Air Gap

When you need to decouple across completely different languages (e.g. a Go compiler engine, a Python AI model, and a TypeScript web browser), in-memory pointers are impossible. Storing state in a local SQLite database acts as a universal relational boundary. It utilizes the OS page cache for sub-2ms reads, ensures ACID serialization, and lets any tool or language query the state with independent pointers and zero runtime coupling.

Conclusion: Drawing the Line in the Sand

Decoupling isn't about design patterns or complex class hierarchies—it’s about drawing a line in the sand for your data.

On one side of the line: Your memory, your pointers, and your execution scope.
On the other side of the line: Their memory, their pointers, and their execution scope.
At the line itself: Clean, stop-and-start data boundaries.

When you draw clear lines in the sand, you eliminate invisible regressions, free your systems to scale independently, and give both yourself and your AI agents the clarity to build with confidence.

I’ve been exploring these mechanics while building Go-Synapse—a local, 2D AST canvas and SQLite MCP engine. How do you handle data boundaries and pointer ownership in your own architecture? Drop your thoughts in the comments below!

Permalink

vim-slime

Permalink

Swipe Keyboard

<!DOCTYPE html> <html> <head> <title>Tarn Barford</title> <meta charset="utf-8"/> <link rel="icon" type="image/x-icon" href="/favicon.ico"> <link href="/style.css" media="screen" rel="stylesheet" type="text/css" /> <link rel="alternate" type="application/atom+xml" title="Journals of Tarn Barford" href="/atom" /> <link href="/highlight.css" media="screen" rel="stylesheet" type="text/css" /> <link href="/highlight-console.css" media="screen" rel="stylesheet" type="text/css" /> <style> #swipe-canvas { position: relative; width: 900px; height: 300px; } #swipe-results { font-size: 30px; padding-left: 50px; padding-left: 50px; } #swipe-results ul { margin: 0px; padding: 0px; } #swipe-results li { float: left; background-color: #DDDDDD; list-style-type: none; padding: 10px; margin: 5px; border-radius: 5px; } #swipe { position: relative; } #swipe-loading { position: absolute; height: 50px; width: 300px; top: 85px; left: 300px; background-color: darkgray; border-radius: 10px; text-align: center; padding-top: 20px; border: black; border-width: 5px; } </style> </head> <body> <div id="container"> <div id="header"> <div id="header"> <p>From the <a href="/journal">Journals</a> of <a href="/">Tarn Barford</a></p> <h1> Swipe Keyboard </h1> <p> Apr 06, 2014 </p> </div> </div> <div id="post_content"> <html><body><p>When I first tried a <a href="http://www.swype.com/">Swype</a> keyboard I was impressed how effective it was. Even though I don't use the feature on my phone I was interested in how it could be built, so I <a href="https://github.com/tarnacious/swipe-keyboard">implemented this otherwise useless swipe-able keyboard</a> below. It probably doesn't work on mobile devices, but works on modern browsers with mouse pointers (although I've only really tried Chrome and Firefox).</p> <div id="swipe"> <canvas height="300px" id="swipe-canvas" width="900px"></canvas> <div id="swipe-results"></div> <div style="clear: both"></div> <h2 id="swipe-loading">Loading<noscript>Javascript is Required</noscript></h2> </div> <p>I initially tried to solve this using the technique Peter Norvig famously uses in his <a href="http://norvig.com/spell-correct.html]">spell checker</a>. He takes a sequence of characters and generates a set of word candidates by adding, removing and swapping characters in the original sequence, the generated candidates are removed if they are not found a dictionary. This can work but to be effective too many combinations need to be generated.</p> <p>If the dictionary is indexed into a <a href="http://en.wikipedia.org/wiki/Trie">trie</a> the number of combinations generated can be reduced significantly by traversing the trie and only generating valid letter combinations. This is a pretty bare implementation of that, it requires: </p> <ul> <li>The first and last characters of the initial sequence are used </li> <li>Intermediate characters in the initial sequence can be repeated or discarded </li> <li>No characters are added or swapped</li> </ul> <p>Basically, if you swipe through all the characters in a word in order, then the word will be found if it is in the index regardless how many characters are swiped in between. It is surprisingly quick and effective.</p> <p>This implementation uses <a href="https://raw.github.com/first20hours/google-10000-english/master/google-10000-english.txt">these 10000 words</a>, I intended to use digital books but never got around to it as these words demonstrate the concept well enough.</p> <p>This is the first thing I've written in <a href="https://github.com/clojure/clojurescript">ClojureScript</a> or <a href="https://github.com/clojure/clojurescript">Clojure</a>, so my code my vary from non-idiomatic to shamblolic. I initially used a <a href="http://clojuredocs.org/clojure_core/clojure.zip/zipper">zipper</a> to build the trie with immutable data structures, but found the indexing took to long with my zipper implementation so I <a href="https://github.com/tarnacious/swipe-keyboard/commit/6edd7b26e78121fbe8586b3f0ef54ca8277d9e32">switched to using native Javascript maps</a>.</p> <p>I found that <a href="https://github.com/clojure/core.async">core.async</a> library is really awesome, the <a href="http://docs.closure-library.googlecode.com/git/index.html">Google closure library</a> and <a href="https://developers.google.com/closure/compiler/">compiler</a> integration with <a href="http://leiningen.org/">Leiningen</a> the <a href="https://github.com/emezeske/lein-cljsbuild">cljsbuild plug-in</a> to be impressive. My main pains were the slow JVM start-up time, the advanced closure compiler build of the web worker script fails silently when run (but the main script works fine when compiled with the advanced compiler), and at times I felt some compile time type checking would be nice.</p> <p>I would like to extend this experiment to index the word occurrence counts and proceeding word counts in original text and rank the found words as most likely. Support casing, umlauts, special characters, spelling correction and compound words in the indexing and lookup. I think a live lookup while swiping would also be possible.</p> <p>Overall this was fun, turned out OK I think, and was a great learning experience.</p></body></html> </div> <div id="comments"> </div> </div> <div id="footer"> <p> </p> <p>Questions, comments, suggestions? Email me, <a href="mailto:tarn@tarnbarford.net">tarn@tarnbarford.net</a> (<a href="/pgp.txt">public key</a>)</p> <p> </p> </div> <script src="swipe.js" type="text/javascript"></script> </body> </html>

Permalink

Smarter Form Targeting Is Not Coming to CIDER

A couple of days ago I wrote that smarter form targeting was coming to CIDER, and I ended that post by asking whether I’d got the resolution rules right and whether anything still surprised people. I got an answer. CIDER 2.1 will ship with the classic behaviour intact.

This is not a sad story, though. The detour turned up a bug that was quietly mangling people’s comments, and CIDER came out of it better than it went in.

What I was actually after

The targeting change wasn’t really about cursor positions. What I wanted was for every CIDER command that operates on a form to behave the same way, without adding yet another command to get there. CIDER has an enormous surface of evaluation commands, and every “at point” variant I could have added would have made that worse. If the existing commands simply resolved the form you meant, newcomers would have had fewer commands to learn, not more.

That was the bet: consistency by redefinition rather than by addition. In hindsight it was the wrong bet for a project this old. Fifteen years in, the existing behaviour isn’t an implementation detail I get to tidy up - it’s the contract. And as it turned out, the redefinition wasn’t nearly as transparent as I’d convinced myself it was.

The feedback

Several users, including CIDER’s co-maintainer Sashko Yakushev, voiced concerns and flagged issues I’d overlooked while playing with this initially. The most important one: inspection is just another flavour of evaluation, and people inspect bare symbols constantly. If it got the same treatment, the disruption would be real enough to run a fork over.

My instinct was that this was an edge case, so I measured it instead of arguing. Across every cursor position in a buffer, only three rules actually differ, and the flagship flow - type a form, hit C-x C-e - is identical under both. Two of those three were fine. The third was this one:

Cursor on a closing paren: the classic rules evaluate the last form inside, smart targeting evaluates the whole enclosing call

The cursor doesn’t move between those two evaluations. That’s the same position, twice, and the answers differ - "b" under the classic rules, "ab" under the new ones.

I’d filed “cursor on a closing paren” as an oddity nobody hits deliberately. But think about when you land there: you finish typing the last thing inside a form, and the cursor is now sitting on the ). For someone inspecting symbols all day, that fires constantly. And the classic answer isn’t arbitrary either - as Sashko put it, the rule of thumb is “whatever paredit-backward jumps back to”, which is a better description of the tradition than anything I’d written down.

Why the tradition exists in the first place

Here’s the part I under-weighted, and it’s worth spelling out for anyone who finds “the form before the cursor” arbitrary.

Emacs form navigation overwhelmingly leaves the cursor after a form. C-M-f (forward-sexp) moves over the next form and stops just past its closing delimiter. C-M-e (end-of-defun) leaves you after the whole top-level form. C-M-n (forward-list) does the same for the enclosing list. Paredit’s paredit-forward behaves the same way, and so does typing: finish a form and the cursor is, by definition, right after it.

So “evaluate the preceding form” isn’t a quirk - it composes with how you already move around. Navigate forward over a form, evaluate it. Type a form, evaluate it. The cursor is already in the right place, every time.

Getting onto a form instead takes deliberate effort: C-M-b (backward-sexp), C-M-a (beginning-of-defun), paredit-backward, or a jump package like avy. All perfectly good tools, but you have to reach for them - unless you’re clicking around with a mouse, in which case the cursor lands wherever you pointed and “the form before the cursor” genuinely is useless. Which, I suspect, is exactly the workflow difference behind this whole argument.

What CIDER got instead

The half of the idea that was never controversial is still there, as commands you opt into rather than a new meaning for keys you already use. Every operation now has an “at point” variant, not just eval and tap: cider-inspect-sexp-at-point, cider-pprint-eval-sexp-at-point, cider-macroexpand-1-at-point, cider-macroexpand-all-at-point, cider-format-edn-sexp-at-point, cider-insert-sexp-at-point-in-repl.

Three ways to say which form you mean, and now every command supports all of them:

The same expression evaluated three ways: the preceding form, the form at the cursor, and the enclosing top-level form

The at-point commands fall back to the preceding form when there’s nothing to point at, so they’re drop-in replacements rather than a separate mode of working. Which means anyone who wanted smart targeting can simply have it:

(with-eval-after-load 'cider-mode
  (define-key cider-mode-map (kbd "C-x C-e") #'cider-eval-sexp-at-point)
  (define-key cider-mode-map (kbd "C-c C-e") #'cider-eval-sexp-at-point))

Two lines, no hidden mode, and everyone else’s fingers keep working. This is what I should have shipped in the first place, and it’s what the manual now recommends. Yes, it’s more commands than I wanted. It’s also the version that doesn’t break anyone.

Macroexpansion is the one place a bit of cleverness survived on its own merits, because an expansion needs a call form. Stand on a bare symbol and cider-macroexpand-1-at-point widens to the call around it, since expanding a lone symbol is never what anyone meant.

The bug at the bottom of the hole

While unifying the plumbing I found this, which is much worse than anything form targeting was ever guilty of. Put the cursor at the end of a comment:

(defn foo [])
;; a comment|

CIDER answered comment. Not the (comment ...) form - the word, lifted out of your prose, because sexp motion has no notion of comments once the cursor is inside one and happily reads the words as symbols.

For evaluation that produced a puzzling error. For the in-place macroexpansion commands, which replace the region they resolved, it did this:

;; a comment          ->   ;; a EXPANDED<comment>
(+ 1 2) ; hey         ->   (+ 1 2)          ; EXPANDED<hey>

It rewrote the comment. That bug has been in CIDER for years, and I only found it because I went looking at the primitives while cleaning up after myself.

Then I checked the neighbours, and this is my favourite part of the whole episode: SLIME, SLY and Emacs Lisp itself all still do this. Both Lisp environments use a bare backward-sexp, and if you put the cursor after (+ 1 2) ; hey in any Emacs Lisp buffer and ask for the preceding sexp, you get hey. CIDER now steps out of the comment first, which as far as I can tell makes it the only one of the family that gets this right.¹

One idea worth stealing

The same survey turned up something CIDER was missing. SLY briefly flashes the region it compiled, so you see what it acted on. That’s a direct answer to the confusion this whole saga was about - “which form did it just take?” - and it changes nothing about what gets taken.

(setq cider-flash-evaluated-region t)

Off by default, because I’ve learned my lesson about switching things on for everyone. But if the targeting rules ever puzzle you, turn it on for a day.

The moral

In the original post I described cider-form-targeting, the escape hatch back to the classic rules, as “living on borrowed time” - clutter left over from a plan I’d since reversed, which I was itching to delete.

That option is the only reason the conversation stayed a conversation. Its existence made the objection “please don’t remove the fallback” rather than “I’m forking CIDER”, and I very nearly removed it before anyone had tried the change.

Every mistake is a learning experience for me, and this one was cheap: nothing had shipped, so the whole thing cost a few days and some rewriting. The testing and feedback cycle worked exactly as it should have - people tried something on master, told me plainly what was wrong with it, and the result is better than either what I proposed or what we had before. Everyone gets the behaviour they want, and CIDER lost a text-eating bug on the way.

Thanks to everyone who took the time to tell me I was wrong.

That’s all I have for you today. Keep hacking!

If you’re an Emacs maintainer reading this: elisp--preceding-sexp has the same behaviour, and I’d be happy to be told why it’s intentional. ↩

Permalink

Wrapping GTK4 in 800 lines of Clojure with Jolt

Native toolkits are not terribly ergonomic, and are a lot more painful to use compared with web dev in many ways. Building a UI with them is an imperative exercise where you construct widgets one call at a time, pack them into containers, and wire each event to its handler by hand. What's worse is that the structure of the interface often ends up living outside your language altogether forcing you to use tools like GtkBuilder XML or Xcode storyboards, where none of your usual tools for composing and refactoring code can reach. Layout is governed by box packing rules and constraint systems that are easy to describe but hard to predict. And a common task such as turning a list of items into a list of widgets that stay in sync with the data results in a ton of boilerplate that you have to repeat over and over.

The pain of working with native toolkits gave rise to things like Electron which simply package a browser engine as a frontend for the application. While that technically works, it's a clunky and inefficient hack around the problem. Every app ends up having to ship its own copy of the browser along with a JavaScript runtime, and the result never quite feels native.

However, even doing that still doesn't address the biggest frustration about having a compile cycle that breaks your development flow. And that's especially problematic when building a UI where you can't easily automate testing. You have to load up the app, click through its menus, and get it to a particular state so that you can visually inspect whether a change works as you intended and has decent UX.

Working with Reagent and other Clojure UI toolkits in ClojureScript is an enjoyable experience precisely because you can build up the UI gradually, and keep a running state as you add more components to it. You make a change, look at the app, see it instantly, and then iterate on it.

I've always been a big fan of the Reagent reactive model which I find to be intuitive. You treat your UI state as a data structure, and have UI components subscribe to paths within it. Whenever an element at a particular path changes, the UI component associated with it gets updated. And that's really all there is to it. While Reagent is built on top of React, the latter isn't actually needed in this model. React uses a VDOM that gets diffed and then rendered to the actual DOM, and the reason it needs a VDOM is due to the fact that React is agnostic regarding what triggers a component change. That's why you need the whole React lifecycle with checks for componentDidMount, componentWillUnmount, and so on. In Reagent a component collapses to a single render function, and the reactive tracking decides which components need to be re-run, so the lifecycle is derived from the state of the data. And because the reactive model already knows exactly what changed, it makes it possible to use it to drive the DOM directly without needing React as the mr-clean library illustrates.

Since this model works well with a browser DOM, then why not apply it to a native toolkit? All that's needed is to create wrappers to render native widgets by passing them values from the reactive atom, and then provide a callback for the widgets to trigger on user input. We don't need an equivalent of a VDOM because all the changes are driven by the state of the reactive atoms, and can be rendered directly to the UI. This can even be done with a batching layer to control the rate of UI updates if needed. And this is how glimmer works, providing a reactive core that can be hooked up to a particular UI toolkit. Then, there are glimmer-gtk, glimmer-uikit, and glimmer-tui to provide concrete bindings for different types of UI widgets.

The canonical counter with glimmer-gtk looks pretty much exactly like its Reagent counterpart:

(ns counter
  (:require [glimmer.ratom :as r :refer [atom]]
            [glimmer.core :as ui]
            [glimmer-gtk.core])) ; installs the GTK4 backend

(defn counter []
  (let [count (atom 0)]
    (fn []
      [:vbox {:spacing 12}
       [:label {:label (str "Count: " @count)}]
       [:hbox {:spacing 8}
        [:button {:label "- 1" :on-click #(swap! count dec)}]
        [:button {:label "+ 1" :on-click #(swap! count inc)}]
        [:button {:label "reset" :on-click #(reset! count 0)}]]])))

(defn -main [& _]
  (ui/run counter :title "counter" :width 320 :height 160))

The outer function runs once to create the local state, and the inner one re-runs whenever @count changes, patching the live widgets in place instead of rebuilding the tree. If you've used Reagent before, this should all look very familiar with the only difference being that the elements are GTK4 widgets instead of DOM nodes.

What it takes to wrap a widget

It turns out that there is surprisingly little code involved in hooking a C toolkit up to a reactive Clojure core. The entire glimmer-gtk backend sits under 800 lines of Clojure split across four namespaces, and none of it involves anything exotic.

Jolt's FFI lets you promote a C function to the Clojure layer by naming the symbol it points at along with its argument and return types. We can see a few examples of what bindings from the GTK backend look like below.

(ns glimmer-gtk.ffi
  (:require [jolt.ffi :as ffi]))

(ffi/defcfn gtk-button-new-with-label "gtk_button_new_with_label" [:string]
  :pointer)
(ffi/defcfn gtk-button-set-label "gtk_button_set_label" [:pointer :string]
  :void)
(ffi/defcfn gtk-box-new "gtk_box_new" [:int :int]
  :pointer)
(ffi/defcfn gtk-box-append "gtk_box_append" [:pointer :pointer]
  :void)

Pointers are plain machine addresses represented as numbers, strings are marshalled to and from C strings automatically, and GTK booleans are ints that are handled by a one line ->bool helper. There is no C shim to compile or bindings generator to run, and no interface DSL to learn. The shared libraries are declared in deps.edn under :jolt/native, and Jolt loads them before the namespaces are required.

One thing to note here is that the main loop binding needs a bit of special treatment.

(ffi/defcfn g-application-run "g_application_run" [:pointer :int :pointer]
  :int :blocking)

The :blocking flag tells the runtime that the call parks the thread for the lifetime of the app, so it shouldn't pin the garbage collector while GTK owns the main loop.

GTK's API is also full of enums like GTK_ALIGN_START and GTK_ORIENTATION_VERTICAL, so a common approach is to maintain a table of constants mirroring the C headers. But glimmer-gtk has no need for such tables since every GObject enum registers its members with a lowercase nick which maps to a Clojure keyword. So, three extra bindings are all it takes to resolve a nick to its integer value at runtime through the GObject type registry.

(ffi/defcfn g-type-from-name
  "g_type_from_name" [:string] 
  :size_t)
(ffi/defcfn g-type-class-ref         
  "g_type_class_ref" [:size_t] 
  :pointer)
(ffi/defcfn g-enum-get-value-by-nick 
  "g_enum_get_value_by_nick" [:pointer :string]
  :pointer)

When you write [:label {:halign :start}], the backend looks up the GtkAlign type to get its class struct, asks it for the member with the start nick, and reads the integer out of the struct it gets back. Successful lookups are then memoized, and a raw integer can still be used as an escape hatch. This trick is the reason why there isn't a single GTK_* constant needed anywhere in the library.

So that's all the boilerplate that's needed to expose the needed GTK components. With that in place, each hiccup tag can map to a widget spec in a registry. These specs are represented as small maps describing how to construct the widget, apply props to it, and what kind of container it is. Here is what the code for declaring a button looks like.

(defn- ->bool [x] (if x 1 0))

(defn- button-spec []
  {:ctor    (fn [p]
              (if (:label p)
                (g/gtk-button-new-with-label (:label p))
                (g/gtk-button-new)))
   :apply   (fn [w p]
              (when (contains? p :label)
                (g/gtk-button-set-label w (:label p)))
              (when (:tooltip p)
                (g/gtk-widget-set-tooltip-text w (:tooltip p)))
              (when (contains? p :sensitive)
                (g/gtk-widget-set-sensitive w (->bool (:sensitive p)))))
   :container :none})

The reconciler drives these through a fixed lifecycle where create! runs the constructor, applies the props, and wires up the event handlers, while apply-props! re-runs the prop application against the existing widget on each re-render, patching it in place as needed. Handlers are connected once when the widget mounts, and they're expected to close over reactive cells, so the closure captured on the first render stays correct for the life of the widget, just like it does in Reagent.

Event props are :on-* keys looked up in a signal table.

(def signals
  (atom {:on-click    "clicked"
         :on-change   "changed"
         :on-activate "activate"
         :on-toggled  "toggled"}))

Each handler gets wrapped in a foreign-callable and connected with g_signal_connect_data.

(doseq [[event handler] props]
  (when-let [signal (@signals event)]
    (let [cb (ffi/foreign-callable
              (fn [widget _data] (handler))
              [:pointer :pointer] :void :collect-safe)]
      (retain-callable! cb)
      (g/g-signal-connect-data widget signal cb ffi/null ffi/null g/CONNECT-DEFAULT))))

The :collect-safe flag is important here because GTK calls the handler from inside the blocking main loop, and the callable also has to be retained on the Clojure side, since C holds it as a raw pointer that's opaque to the garbage collector.

Another detail worth noting is that GTK emits signals synchronously from its own setters, so when a re-render programmatically sets an entry's text, GTK fires changed on the spot which triggers the handler. Since the handler causes the atom to reset, you end up in a render loop. The fix is to bracket programmatic setters with a suppression set so that their emissions are ignored.

(defn- set-entry-text! [widget text]
  (when (and (some? text) (not= text (g/gtk-editable-get-text widget)))
    (swap! suppressing conj widget)
    (g/gtk-editable-set-text widget text)
    (swap! suppressing disj widget)))

Both the widget and the signal registries are open, so adding a widget the library doesn't know about is simply a matter of writing its spec and registering it.

Finally, glimmer itself doesn't need to see any of this because a backend simply has to provide a map of eight functions handed to the reconciler at registration time.

(def backend
  {:name           :gtk4
   :create!        w/create!
   :apply-props!   w/apply-props!
   :append-child!  w/append-child!
   :remove-child!  w/remove-child!
   :replace-child! w/replace-child!
   :reorder-child! w/reorder-child!
   :schedule       post-to-gui
   :run            run!})

That's the entire contract between the reactive core and the platform, and that's why writing a backend for a different toolkit is simply a matter of wiring up the widgets into the lifecycle. It's even possible to have a stable subset of common widgets across platforms as long as you keep the naming consistent.

Bring your own toolkit

Another notable approach is seen with glitter and its AppKit renderer glitter-uikit, which are based on Replicant. While glimmer is driven by a reactive atom where you build a tree of stateful components, Replicant rejects having local state entirely, and models the entire user interface as a single pure function which turns your application data into hiccup. While Reagent couples your rendering logic with a reactive state graph to optimize updates behind the scenes, Replicant acts as a remarkably strict unidirectional renderer where data goes in and hiccup comes out.

The same counter in glitter-uikit looks like this:

(require '[glitter-uikit.app :as app]
         '[glitter-uikit.appkit :as appkit]
         '[glitter.core :as core])

(defonce state (atom {:count 0}))

(defn view [{:keys [count]}]
  [:vbox {:spacing 12}
   [:label {:label (str "Count: " count)}]
   [:hbox {:spacing 8}
    [:button {:label "+ 1" :on {:click [[:action/inc]]}}]]])

(defn execute-actions [_event actions]
  (doseq [[kind] actions]
    (case kind
      :action/inc (swap! state update :count inc)
      nil)))

(core/set-dispatch! execute-actions)

(defn -main [& _]
  (app/run (fn [window] (appkit/mount! window view state))))

Here the leaves are AppKit views instead of GTK widgets, and notice that the button no longer closes over the atom. It simply declares what it wants done as data, and a dispatch function interprets those actions against the application state. The hiccup stays the same, and the only difference is where the state lives and who is responsible for updating it.

Conclusion

Bringing web-style development to native widgets isn't a new idea, of course. React Native popularised the approach letting you write React components, and render to the platform's native widgets. The catch there is that the app still runs inside a JavaScript runtime that talks to the platform through a bridge, while the development loop revolves around bundling JavaScript and hot-reloading it into a running app. Flutter sidesteps the bridge by bringing its own rendering engine and drawing every control itself, which means you're no longer using native widgets. Another approach is what Tauri does keeping the web frontend backed by the OS webview. This approach is lighter than Electron but still renders HTML rather than native controls, and is subject to the quirks of the webview implementation on each platform. Meanwhile, the native world has been converging on the same idea from the other side, with SwiftUI and Jetpack Compose offering declarative UIs, but those put you right back in a compiled language with a rebuild cycle and no REPL. Each of these approaches ends up being a compromise involving either having a heavyweight runtime or giving up ergonomics.

With Jolt, we can finally have the best of both worlds using native widgets without having to bundle a whole browser engine just to render the UI, have a clean Hiccup based API that lets you arrange components just like you would with HTML elements in the DOM, and have an interactive development environment where you can see the application evolve as you make changes to it. Since Jolt is a Clojure dialect that compiles to native code, there's no JVM or JavaScript runtime in the way, and the app compiles into a lean binary. You get the same feedback loop that makes web development pleasant, while driving real platform widgets in a native application.

Permalink

Previewing the Model Hardware Standard

One of my summer jobs in university was working in an analytical chemistry lab. I spent countless hours pipetting liquids from tube to tube, scanning labels, manually entering data into spreadsheets, and operating fancy machines. Even way back then (please don’t ask exactly how long ago 🙈) I couldn’t help but think “we must be close to automating most of this work”. Turns out we were not, but with this announcement I think its closer than ever.

MHS enables AI agents to operate multiple lab and manufacturing instruments, such as microscopes, liquid handlers, and robotic arms, in parallel, and perform intricate tasks ranging from routine drug discovery experiments to laser calibration on a quantum computer.

This is a really cool development and one way I could imagine AI having some genuinely beneficial impact on society. I think we’re arguably living in the bad timeline right now and unless public opinion shifts pretty dramatically somehow it will be difficult to get to a place where AI does more good than harm. Part of getting there is delivering some credible public benefit for all the cost and risk the public is expected to bear as a result of its development and deployment. I think meaningful progress in drug discovery is a potential area where that could happen.

link / anthropic.com (via) / #ai-agents #ai-benefits #drug-discovery #human-computer-interaction #lab-automation #model-hardware-standard #robotics #society

Permalink

Macro Macros

This is a follow up to my recent posts on Domain Specific Languages (DSLs) for database queries in Clojure, and different quoting forms in Clojure to enable this.

I had originally thought that I would write this post in a "Tutorial" style. But then it occurred to me that I used to get the best responses from people when I just documented what I learned as I learned it. I think it's a more "open" and personable style of writing, which may be what people liked. This approach might have extra appeal in the current era of AI slop, since people ought to see that I've written it myself. Unless I usually sound like an AI 🙃

I'm also hoping that this approach will be easier to write, since I don't need to restructure my exploration as an instructional post.

On the other hand, it may be a terrible idea. But I won't know unless I try…

Graph Queries

SQL is very structured around column selection, so it does not usually need anything like variables inside a query. This allows HoneySQL to build most of its structures using functions that take keywords and values as arguments.

On the other hand, graph query languages like Datomic, SPARQL, and GQL typically use pattern matching on the graph, where parts of the pattern contain a variable that will be "bound" to associated values when a pattern matches. For instance:

?person :hasFriend ?friend

… is a pattern for matching edges in a graph where something is connected by a property labelled :hasFriend to another node. Every edge in the graph that matches this pattern, leads to a pair of nodes that the variables ?person and ?friend get bound to.

GQL would do something similar with a MATCH clause of:

(person:Person)-[:HAS_FRIEND]->(friend:Person)

Like the SPARQL form, this binds the first node (which must be of type Person) to the variable person, and the second node (which is also a Person) to the variable friend.

I'm focused on Datomic and SPARQL, which are more closely related to each other, so I won't address GQL here.

The issue with variables is that keywords are already being used in queries (such as for the :hasFriend property), so we need something else. The obvious candidate is the Symbol, and this is the route that Datomic took.

However, Clojure uses symbols to refer to values, meaning that they need to be inserted in queries without being evaluated. That was why I did that post on quoting symbols: quoting is a mechanism to create a symbol as a part of a structure without asking Clojure to instead insert the value the symbol references. i.e. if I say:

(let [?person "bad data"]
  [?person :hasName "Fred"])

;; returns=> ["bad data" :hasName "Fred"]

This isn't what I wanted. It is even worse if I don't have ?person predefined (as the let expression does), since the code can't even run.

Instead, I want the symbol in the first position:

(let [?person "bad data"]
  ['?person :hasName "Fred"])

;; returns=> [?person :hasName "Fred"]

This doesn't care if the symbol is defined or not. We get back the structure that we wanted with a symbol in it.

Flint

A colleague has been using Flint to build SPARQL queries programmatically. I like Flint, because it uses Datomic-style queries to express SPARQL. Datomic has been designed to use from Clojure, with a query language based in Clojure data structures. So Flint should make it just as easy to query SPARQL.

However, because the queries are being built programmatically, we run the risk of reusing a variable name each time a new expression is added into the query. Reusing a variable for a different purpose will generally break a query, so he was generating new variable names whenever he was generating a new query clause.

There are a couple of ways to generate new variables for a query. One way is to generate a new name, build a symbol with that name, and then insert it without quoting the name being used to carry that symbol:

(let [?container (gensym "?container")]
  [[:my-object :contains ?container]
   [?container :value "hello"]])

;; result=> [[:my-object :contains ?container180]
;;           [?container180 :value "hello"]]

Another approach is to use an auto gensym:

`[[:my-object :contains ?container#]
  [?container# :value "hello"]]

;; result=> [[:my-object :contains ?container__2__auto__]
;;           [?container__2__auto__ :value "hello"]]

Both of these seem fine, but it gets harder to read and write when new constraints are added. For instance, I may be looking for a :value that is a string containing the substring "lo":

(let [?container (gensym "?container")
      ?value (gensym "?value")]
  [[:my-object :contains ?container]
   [?container :value ?value]
   [:filter '(contains ~?value "lo")]])

;; result=> [[:my-object :contains ?container140]
;;           [?container140 :value ?value141]
;;           [:filter (contains (clojure.core/unquote ?value) "lo")]]

Which doesn't work. We need syntax quoting instead:

(let [?container (gensym "?container")
      ?value (gensym "?value")]
  [[:my-object :contains ?container]
   [?container :value ?value]
   [:filter `(contains ~?value "lo")]])

;; result=> [[:my-object :contains ?container144]
;;           [?container144 :value ?value145]
;;           [:filter (user/contains ?value145 "lo")]]

Again… the quoting didn't work:

(let [?container (gensym "?container")
      ?value (gensym "?value")]
  [[:my-object :contains ?container]
   [?container :value ?value]
   [:filter `(~'contains ~?value "lo")]])

;; result=> [[:my-object :contains ?container148]
;;           [?container148 :value ?value149]
;;           [:filter (contains ?value149 "lo")]]

To be fair, I did know how to quote that all along, but I wanted to demonstrate that it can catch people out.

Another Step

This is all very contrived. But what about if I want to construct something more complex. Say, I want a general way to select an entity and its contained value, and then I decide that I want to filter that by containing "lo". The first part would be in a function, but I will need to provide the binding ?value variable to filter on it:

(defn select-entity-value [?entity ?value]
  (let [?container (gensym "?container")]
    [[?entity :contains ?container]
     [?container :value ?value]]))

(let [?e (gensym "?e")
      ?v (gensym "?v")]
  `[~@(select-entity-value ?e ?v)
    [:filter (~'contains ~?v "lo")]])

;; result=> [[?e178 :contains ?container180]
;;           [?container180 :value ?v179]
;;           [:filter (contains ?v179 "lo")]]

Splice-unquoting like this (the ~@ syntax) isn't really necessary, though quoting everything means that the expression inside the :filter doesn't need its own quoting (since it is a list, and we have to quote lists, or else they get executed).

We could even avoid almost quotes with:

(let [?e (gensym "?e")
      ?v (gensym "?v")]
  (conj (select-entity-value ?e ?v)
        [:filter (list 'contains ?v "lo")]))

But now we have conj and list in the expression, we're still quoting contains, and we have the messy gensym declarations at the top. This DSL doesn't look all that easy to use.

Filters and Bindings

Until now, I've been rehashing what I've talked about already, and some of the things my colleague was building. It all worked, but it just looked… clunky. I found myself thinking that surely we could do better, right?

Patterns are not too complex to work with, since they are just short vectors, containing keywords, symbols, or simple values like strings and numbers. The mess really showed up when we tried to introduce a filter.

The filter we used here was the contains function. There are a lot of other functions in SPARQL, so there isn't anything particularly special about that function. Hopefully, whatever we do to address one function could be applied to all of them.

It would be nice to just insert the contains expression directly into the query. Something like:

[:filter (contains ?v "lo")]

Bindings work similarly, except instead of evaluating an expression and passing through everything that returns a true result, they evaluate an expression and save it in a variable. For instance, to save a the lower-case form of the string ?v, you can bind it:

[:bind (lcase ?v) ?lowv]

The problems here are that contains and + looks like a function. Even if we created a function for it, the ?v is not bound to anything, so the function would fail. However, Clojure macros can accept any symbol as an argument. Can we use them somehow?

Macros are Clojure code that generates Clojure data. The trick is that the macro is called during compilation, and the resulting data is inserted into the source code before it gets compiled. This lets you generate code using code.

Clojure itself uses macros everywhere. Most of the "built in" syntax is actually just macros that write out other code. At the end of the day, Clojure only contains a few "special forms" that get presented to the compiler. But don't let that list of special forms fool you: many of those (e.g. defn, fn, and let) are actually macros as well, with simpler special forms underlying them.

Many of these macros accept symbols that are not bound, safely generating code that uses those symbols. For instance, consider an expression for an identity function: (fn [x] x)

We can see what the fn macro expands to by using macroexpand:

=> (macroexpand '(fn [x] x))
(fn* ([x] x))

This is just a rewrite to use fn* instead, which is much simpler: fn* does not do argument destructuring, it does not handle pre nor post conditions, it does not attach metadata, and it always expects parentheses around a function argument list and body even when there is only a single arity.

But importantly, notice how we can provide fn with an expression that includes x and it returns a new expression that also includes x? We don't need x to exist before doing this. That may be what we need.

Let's try it out. Can we create a contains macro that accepts a symbol argument and returns a list with appropriate symbols in it?

My first attempt was laughable:

(defmacro contains [a b] `(~'contains ~a ~b))

So when I call (contains "foot" "foo") it should return a list containing the symbol contains, the string "foot" and the string "foo". That was what was returned, but then that goes to the compiler, and the symbol contains doesn't exist, so it fails. Doh.

I needed an actual symbol object in the position of contains:

(defmacro contains [a b]
 (let [c# (symbol "contains")]
  `(~c# ~a ~b)))

But I didn't need to evaluate that to know what it would do… calling (contains "foot" "foo") would generate the list (contains "foot" "foo") and that would be evaluated, which then repeats the process until the macro evaluation overflows the stack. I had forgotten that I'm not returning the data structure anymore. Now I'm returning code that evaluates to the required data structure.

Aside from quoting a list (which gets tricky, because macro expansion essentially unquotes what you've quoted, so you need to double-quote), you can create a list with the list function:

(defmacro contains [a b]
 (let [c# (symbol "contains")]
  `(list ~c# ~a ~b)))

How does this look?

=> (macroexpand '(contains "foot" "foo"))
(clojure.core/list contains "foot" "foo")

Oh. That was sort of obvious in hindsight.

Let's forget inserting the symbol and just put it in place:

(defmacro contains [a b]
 `(list (symbol "contains") ~a ~b))
=> (macroexpand '(contains "foot" "foo"))
(clojure.core/list (clojure.core/symbol "contains") "foot" "foo")
=> (contains "foot" "foo")
(contains "foot" "foo")

See why I don't usually like writing this way? Showing how long it takes to get to something that should be easy is embarrassing.

But I'm really not happy about embedding the symbol by generating one based on a string. Is there another way? What if I quote it?

(defmacro contains [a b] `(list 'contains ~a ~b))
=> (macroexpand '(contains "foot" "foo"))
(clojure.core/list (quote user/contains) "foot" "foo")

That got me part of the way there, but I forgot that I'm in a syntax quote, so contains is resolved with its full namespace. I can write out (quote ~'contains), but I'm curious… can I just chain the quote/unquote syntax here?

(defmacro contains [a b] `(list '~'contains ~a ~b))
=> (macroexpand '(contains "foot" "foo"))
(clojure.core/list (quote contains) "foot" "foo")
=> (contains "foot" "foo")
(contains "foot" "foo")

I might stick to the (quote …) form though, since cascading quote/unquote could get hard to read. It turns out that this mattered later on.

Arguments

That did work, but what about the arguments? Right now they are being passed through, and they just appear verbatim in the final form. That doesn't work for variables like ?value:

=> (contains ?value "foo")
Syntax error compiling at (REPL:1:1).
Unable to resolve symbol: ?value in this context

We need to quote the arguments too.

(defmacro contains [a b] `(list (quote ~'contains) (quote ~a) (quote ~b)))
=> (macroexpand '(contains ?v "foo"))
(clojure.core/list (quote contains) (quote ?v) (quote "foo"))
=> (contains ?v "foo")
(contains ?v "foo")

This looked great! Until my colleague pointed out that he couldn't pass non-literal values into the macro. Of course he couldn't 🤦‍♀️

In this case, I need to check if the argument is a symbol, and if it is, then look for the form ?name or $name (SPARQL allows either ? or $ to indicate a variable). I'll create a helper function that tests if an argument is a symbol that starts with one of these characters, and only quote the ones that match. However, a function like that can't just return (quote a), since that just evaluates to a and won't get inserted into the output correctly. Instead, I need to return a list of the form (quote a):

(defmacro contains [a b]
 (let [vquote (fn [s]
               (if (and (symbol? s) (#{\? \$} (first (name s))))
                 (list 'quote s)
                 s))
       a# (vquote a)
       b# (vquote b)]
   `(list (quote ~'contains) ~a# ~b#)))

user=> (macroexpand '(contains ?v "foo"))
(clojure.core/list (quote contains) (quote ?v) "foo")
user=> (macroexpand '(contains v "foo"))
(clojure.core/list (quote contains) v "foo")

OK, I'll admit it… it took me a few iterations to get that right. Quoting quote was not something I was expecting.

Macro Macros

This works for contains, but what about all the other SPARQL functions? Do I have to write this out for all of them as well? Then there are functions like regex that can take 2 or 3 arguments.

It would be nice if I could take a function name, and generate the appropriate macro.

Except, macros are just code, and macros generate code. So I could always try generating a macro with a macro, right?

Well, sort of. It turned out to be harder than I thought.

Let's try creating a macro that generates a macro for a single-argument function, like lcase. I decided to skip the argument handling to start with, just to get the basic structure down. So my first attempt was:

(defmacro sparql-fn1 [s]
 `(defmacro ~s [~'a] `(list (quote (unquote ~s)) ~'a)))

This made it clear that I wasn't really sure of how to perform the (quote ~'contains) when I didn't have a literal value to put into the quoted position.
I thought I'd try it anyway, and it complained about "No such var: user/s", which didn't really surprise me. So let's see, what did it look like? I'll reformat the output so it's not all on a single line:

=> (macroexpand '(sparql-fn1 lcase))
(do (clojure.core/defn lcase
      ([&form &env a]
       (clojure.core/seq
         (clojure.core/concat
           (clojure.core/list (quote clojure.core/list))
           (clojure.core/list
             (clojure.core/seq
               (clojure.core/concat
                  (clojure.core/list (quote quote))
                  (clojure.core/list
                    (clojure.core/seq
                      (clojure.core/concat
                        (clojure.core/list (quote clojure.core/unquote))
                        (clojure.core/list user/s)))))))
           (clojure.core/list (quote user/a))))))
    (. (var lcase) (setMacro))
    (var lcase))

Let's remove the clojure.core namespaces, the redundant seq calls, and use some quoting syntax:

(do (defn lcase
     ([&form &env a]
      (concat
        (list 'list)
        (list (concat
                (list 'quote)
                (list (concat
                        (list 'unquote)
                        (list user/s)))))
        (list 'user/a))))
    (. (var lcase) (setMacro))
    (var lcase))

This helped me figure out where I'm passing things in poorly, but there was something much more important going on here.

The returned data structure was not the code to call defmacro but instead was a do block with 3 steps:

Define a function called lcase. This takes 3 arguments, prepending &form and &env before the argument I had declared.
Defining a function creates a var for it. The second step calls setMacro on that var.
Returns the lcase var.

The arguments &form and &env are explained in the Clojure docs as special variables that are available inside macros without having to declare them. We are seeing this being set up.

Also, rather than creating a "macro", a function is being created that is then converted to a macro. I'd seen elements of this before, but I'd never really dug into it. This prompted me to look at the defmacro source (by typing (source defmacro) a the repl), and… yes, this is what it's doing.

Most surprisingly, the code of the macro is not in the function, but rather the list structure of the macro is being constructed and returned. Reading the body, it looks like the final structure evaluates to:

(list 'list (list 'quote (list 'unquote user/s)) 'user/a)

The user/s part came about due to my unfortunate quote/unquote structure, and the user/a is because of how I'm calling macroexpand, but the rest of it actually evaluates to what I was expecting, to wit:

(list (quote (unquote user/s)) user/a)

And that structure would evaluate to what gets inserted into source code.

So the function here generates a structure that evals to what the macro returns, which then evals a second time to what gets inserted into the source.

So I can manually create a macro by creating a function that needs to evaluate twice to create the final code structure. That's going to be confusing.

But then I realized that I don't have to worry about the quote/unquote mess anymore. If I want to quote something, then using quote does not hide the quoted value, because now I am creating a list where quote is the first element, and the thing to be quoted is exactly what I want. So it's more complex, but more flexible.

Let's try then. To start with, I decided on a form that just takes all of its arguments, and wraps each one in the helper function that quotes it if the argument is a symbol starting with $ or ?.

(defmacro sparql-fn
  [s]
  (let [argq# (fn [a]
                (list 'if (list 'and (list 'symbol? 'a) (list #{\? \$} (list 'first (list 'name 'a))))
                      (list 'list (list 'quote 'quote) a)
                      a))]
    (list 'do
          (list 'defn s
                '[&form &env & args]
                (list 'list
                      (list 'quote 'cons)
                      (list 'list (list 'quote 'quote) (list 'quote s))
                      (list 'concat
                            (list 'list (list 'quote 'list))
                            (list 'map
                                  (list 'fn '[a] (argq# 'a))
                                  'args))))
          (list '. (list 'var s) '(setMacro))
          (list 'var s))))

This took me a few attempts, but it worked out pretty well:

=> (sparql-fn contains)
#'user/contains
=> (sparql-fn lcase))
#'user/lcase
=> (contains ?x "lo")
(contains ?x "lo")
=> (lcase "Hello")
(lcase "Hello")
=> (lcase a)
Syntax error compiling at (REPL:1:1).
Unable to resolve symbol: a in this context

Let's see what the generated code looks like:

=> (macroexpand '(sparql-fn lcase))
(do
  (defn lcase
   [&form &env & args]
   (list (quote cons)
         (list (quote quote) (quote lcase))
         (concat
           (list (quote list))
           (map (fn [a]
                 (if (and (symbol? a) (#{\? \$} (first (name a))))
                   (list (quote quote) a)
                   a))
                args))))
  (. (var lcase) (setMacro))
  (var lcase))

How does the returned list structure eval then? Working it manually, I came up with:

(list 'cons (list 'quote 'lcase) (concat '(list) (map (fn [a] …) args))))

The function here was a bit long, so I elided it. It's looking good, but evaluating it one more time, gets us to:

(cons (quote lcase) (list (map (fn [a] …) args)))

Which, when called as (lcase "Hello") should return a list containing 'lcase followed by the function mapped over all arguments. The function returns its input for a string, so the final list would be (lcase "Hello").

Similarly, for (lcase ?value) it's the same kind of thing, but now the function will return a list of (quote ?value), so the final output will be (lcase (quote ?value)).

There is no checking on the arguments at all, but I know that Fink is already doing that kind of work, so I thought this was where I could stop.

Small Issue

At this point I decided to try it with all of the SPARQL functions, and immediately hit a wall with concat. This is a function that appears in both SPARQL and in the above macro. As soon as concat is declared as a SPARQL macro, then the next thing I try to declare as a SPARQL macro will use the new concat macro instead of clojure.core/concat. Fink also tries to map and to SPARQL's &&, so that should be addressed as well:

(defmacro sparql-fn
  [s]
  (let [argq# (fn [a]
                (list 'if (list 'clojure.core/and (list 'symbol? 'a) (list #{\? \$} (list 'first (list 'name 'a))))
                      (list 'list (list 'quote 'quote) a)
                      a))]
    (list 'do
          (list 'defn s
                '[&form &env & args]
                (list 'list
                      (list 'quote 'cons)
                      (list 'list (list 'quote 'quote) (list 'quote s))
                      (list 'clojure.core/concat
                            (list 'list (list 'quote 'list))
                            (list 'map
                                  (list 'fn '[a] (argq# 'a))
                                  'args))))
          (list '. (list 'var s) '(setMacro))
          (list 'var s))))

Finally, I can write SPARQL queries using Fink with clean filters:

['[?entity :contains ?c]
 '[?c :name ?name]
 [:filter (contains ?name "lo")]]

Wrap Up

Well, this wasn't a tutorial. It was more like a litany of my mistakes as I stumbled towards understanding. I'm sure an LLM could have told me how to do this immediately, but then I wouldn't have learned anything.

I have no idea if anyone else will ever read this, but if you did, then thanks. If not, then that's OK. Writing these things down helps my clarify and consolidate what I learned. This is something I used to do regularly a long time ago, and I should do more of it.

As per my AI policy, I'm going to leave the typos alone so you know I really did type this. But let me know if you see any sentences that make no sense, and I'll clean them up.

Permalink

Smarter Form Targeting Is Coming to CIDER

If I had a dollar for every time someone asked on the Clojurians Slack in #cider why C-x C-e evaluated “the wrong thing”, I’d probably be writing this post from a yacht. The answer was always the same: the cursor wasn’t where CIDER expected it to be. The upcoming CIDER 2.1 release changes that - the evaluation commands now figure out which form you mean from where your cursor actually is.

NOTE: Update: this didn’t survive contact with its users, and CIDER 2.1 ships with the classic behaviour after all. See /posts/2026/08/29/smarter-form-targeting-is-not-coming-to-cider.html for what the feedback was, what replaced it, and the rather nasty bug the detour turned up.

A bit of history

Emacs has a very particular tradition when it comes to evaluating code: eval-last-sexp (the venerable C-x C-e) acts on the expression before the cursor. Not the one you’re looking at, not the one you’re inside of - the one that ends exactly where your cursor stands. SLIME follows this tradition, Emacs Lisp itself follows it, and for the past 15+ years CIDER has followed it too.¹ If you grew up in Emacs, this rule is in your fingers and you’ve never once thought about it.

Here’s the thing, though - most people using CIDER didn’t grow up in Emacs. And many of them never programmed in Emacs Lisp and Common Lisp with SLIME. They came to Emacs because of CIDER (or Clojure in general), and for them the rule is invisible, arbitrary and mildly hostile. You put your cursor on a form, you press the eval key, and CIDER cheerfully evaluates… something else. Meanwhile every other modern Clojure environment - Calva, Conjure, the various vim plugins - resolves the form from the cursor position and just does what you meant.²

For a long time I resisted changing this, mostly out of respect for the Emacs tradition (and my own muscle memory). But at some point I had to admit that I was optimizing for the wrong audience. CIDER’s users are mostly casual Clojure hackers who happen to use Emacs, not Emacs experts who happen to write Clojure. The tradition was serving me, and confusing them.

What’s actually changing

The evaluation commands (and their macroexpansion, inspection and tapping siblings) now resolve “the form the cursor indicates”. Concretely, with | marking the cursor:

(map inc |(range 10))

Pressing C-c C-e here used to evaluate inc - the form before the cursor, which is almost never what you wanted. Now it evaluates (range 10) - the form your cursor is pointing at.

(str "hello" " " "world"|)

This one used to evaluate "world" (really!), because the last complete expression before a cursor sitting on the closing paren is the final string. Now it evaluates the whole (str ...) call.

Macroexpansion benefits too:

(when tru|e (launch-missiles))

C-c C-m here used to complain that true is not a macro. Now it expands the enclosing (when ...) call, because expanding a bare symbol is never what anyone means.

And my favorite one - the rich comment workflow is now consistent everywhere:

(comment
  (calculate-all-the-things|))

Every defun-level command - eval, pretty-print, inspect, debug - now treats the form inside the (comment ...) as the top-level one. Evaluating a whole comment form returns nil by definition, which has exactly zero uses, so CIDER no longer does that no matter which command you reach for.

Why you probably won’t notice

Here’s the part I’m most pleased with: the new behavior agrees with the old one at every position where “the form before the cursor” made sense. Cursor right after a form? Same result as always. Cursor in the whitespace after a form? Same. Cursor in the middle of a symbol? Same. The two behaviors only diverge where the classic answer was something nobody ever wanted - a previous sibling, a lone trailing atom.

So if your muscle memory follows the Emacs tradition, nothing changes for you. If it doesn’t - CIDER stops punishing you for it. That’s the whole change.

Reverting to the classic behavior (for now)

If you do want the traditional rules - maybe you genuinely use “evaluate the previous sibling while standing on an opening paren” - one setting restores them exactly:

(setq cider-form-targeting 'preceding)

There’s also a per-session toggle in the eval menu (C-c C-v T) that shows the active mode in the mode line while you experiment.

This option is probably living on borrowed time, though. I added it back when I planned to keep the classic behavior as the default and offer smart targeting as an opt-in. Now that the roles are reversed, an option whose only job is restoring rules almost nobody deliberately relied on doesn’t really make much sense, and lately I’ve been trying to trim that kind of clutter from CIDER, not add to it. Don’t be surprised if the option quietly disappears - possibly even before the release ships. Which is one more reason to speak up now if the classic behavior genuinely matters to you.

Farewell, “last sexp”

This change forced my hand on something I’d been putting off for years - the command names. cider-eval-last-sexp is a fine name for a command that evaluates the last sexp. It’s a lie for a command that evaluates the form your cursor indicates. So the commands got honest names:

cider-eval-last-sexp is now cider-eval-form
cider-eval-defun-at-point is now cider-eval-defun (the -at-point never carried information)
likewise for the pprint/tap/inspect/insert variants

Every old name keeps working as an alias, so your config and your M-x habits are safe. But why “form” and not “sexp”? Beyond the targeting change, there’s a Clojure-specific reason: in Clojure a form isn’t always a single sexp. ^:private x is two sexps but one form; so is #inst "2024-01-01". The commands operate on forms - the reader’s unit of evaluation - and now they say so.³ The manual’s evaluation docs got a proper glossary explaining all of this.

Closing thoughts

All of this is on master and in the MELPA snapshots today, ahead of the next stable release. I’d really love for people to play with it before the release ships - especially if you’re an Emacs veteran whose fingers disagree with my reasoning, or a newcomer for whom this was supposed to just work. Did we get the resolution rules right? Does anything still surprise you?

Share your feedback on the CIDER discussions board, in #cider on the Clojurians Slack, or just file an issue. This is exactly the kind of change that’s easy to adjust before a release and painful after - and the fate of the compatibility option depends on what I hear.

That’s all I have for you today. Keep hacking!

CIDER started its life as a SLIME “clone” for Clojure, after all - the tradition runs deep. ↩
Interestingly, Cursive is the only major non-Emacs Clojure environment that kept the classic “form before the caret” model. ↩
This also explains a subtlety Emacs veterans might appreciate: plain forward-sexp movement doesn’t know that Clojure metadata belongs to the form it annotates, which is why clojure-mode has always needed its own “logical sexp” movement functions. The new targeting is built on those, so metadata is never silently dropped from what you evaluate. ↩

Permalink

Sponsors

Updates

Other projects

A quick refresher on watch points

Python – the easy one

JavaScript – the DevTools Protocol saves us (almost)

Problem 1: source maps

Problem 2: functions that don’t exist

Problem 3: namespaces and patching

Elixir – the interesting one

Private functions are invisible

The VM is immutable, kinda

Bindings only exist at call time

Shadowing

Final thoughts

Calling C

Install

Demos

FFI-based libraries

Tasks: :exec-fn composition

AI disclosure

Wrapping up

The Blind Spot in Modern Architecture Debates

1. The Physics of Coupling: The Shared Pointer Trap

2. The Decoupled Mental Model: Stop-and-Start Boundaries

Why Independent Pointers Win:

3. Where Does the Data "Stop"? (RAM vs. Disk)

4. The Double Crisis in Modern Codebases

1. The Human Problem: Snippet Tunnel Vision (The Straw Problem)

2. The AI Problem: Context Window Amnesia (The Overflow Problem)

5. The Solution: Treating Code Itself as Decoupled Relational Data

6. How Different Languages Decouple Data

1. JavaScript: structuredClone() & Transferable Objects

2. Erlang & Elixir (BEAM): The Zero-Shared-Heap Actor Model

3. Clojure: Persistent Data Structures (HAMT)

4. Rust: Compile-Time Move Semantics

5. Go: Channels & Value Receivers

6. SQLite as the Universal Polyglot Air Gap

Conclusion: Drawing the Line in the Sand

What I was actually after

The feedback

Why the tradition exists in the first place

What CIDER got instead

The bug at the bottom of the hole

One idea worth stealing

The moral

What it takes to wrap a widget

Bring your own toolkit

Conclusion

Graph Queries

Flint

Another Step

Filters and Bindings

Arguments

Macro Macros

Small Issue

Wrap Up

A bit of history

What’s actually changing

Why you probably won’t notice

Reverting to the classic behavior (for now)

Farewell, “last sexp”

Closing thoughts

About

Subscriptions

Planetarium

Syndicate

1. JavaScript: `structuredClone()` & Transferable Objects