vim-slime

<!DOCTYPE html> <html> <head> <title>Tarn Barford</title> <meta charset="utf-8"/> <link rel="icon" type="image/x-icon" href="/favicon.ico"> <link href="/style.css" media="screen" rel="stylesheet" type="text/css" /> <link rel="alternate" type="application/atom+xml" title="Journals of Tarn Barford" href="/atom" /> <link href="/highlight.css" media="screen" rel="stylesheet" type="text/css" /> <link href="/highlight-console.css" media="screen" rel="stylesheet" type="text/css" /> </head> <body> <div id="container"> <div id="header"> <div id="header"> <p>From the <a href="/journal">Journals</a> of <a href="/">Tarn Barford</a></p> <h1> vim-slime </h1> <p> Mar 26, 2012 </p> </div> </div> <div id="post_content"> <html><body><p>Today I found the awesomeness that is <a href="https://github.com/jpalardy/vim-slime">vim-slime</a>, it's been an exciting day for me. <a href="http://common-lisp.net/project/slime/">Slime</a> is the "The Superior Lisp Interaction Mode for Emacs", I can almost hear the emacs crowd laughing.</p> <p>For those that use vim and haven't used Slime, vim-slime or <a href="https://github.com/vim-scripts/VimClojure">something similar</a>, this is why it's awesome:</p> <p><strong>Text can be sent from any process to the stdin of a <a href="http://www.gnu.org/software/screen/">gnu screen</a> or <a href="http://tmux.sourceforge.net/">tmux</a> session. The process in this case is vim and the screen/tmux session is a terminal</strong>.</p> <p>Screen is a <a href="/journal/oh-screen-where-have-you-been">really neat</a> terminal multiplexer (you can run multiple terminals in a terminal window). The multiplexed shell processes are children of the screen process, which itself is not a child of the terminal window process. This means a screen process and its child processes keep running if you close the terminal window. Later you can re-connect to it, this is what makes vim-slime possible.</p> <p>Here is an screen shot, on the left is me in gVim writing some awful Clojure <a href="#footnote-1">[1]</a>. On the right is a screen buffer in which I started a Clojure REPL. When I want to try run some code I can send any vim text selection to the REPL in a keystroke (or two).</p> <p><img alt="vim slime screenshot" src="screenshot.jpg"/></p> <p>It doesn't have to be a Clojure REPL either, we can send anything to a screen shell. We could run git commands, find, grep, sed, etc. Like with the Clojure REPL we can even interact with any terminal programs that use STDIN.</p> <p>This concept can be taken even further, You can even connect to a tmux session over SSH and share a terminal or a <a href="http://remotepairprogramming.com/remote-pair-programming-with-tmux-and-vim-the">terminal program like vim to do remote pairing</a>!</p> <p>Hopefully remote pairing is the topic of my next post as there are a couple geographically distant people I know who are keen to do some pair hacking. I stand to learn a lot!</p> <p><a name="footnote-1">[1]</a> I learnt almost everything I know about Lisp from <a href="http://www.ccs.neu.edu/home/matthias/BTLS/">The Little Schemer</a>. Great book.</p></body></html> </div> <div id="comments"> </div> </div> <div id="footer"> <p>&nbsp;</p> <p>Questions, comments, suggestions? Email me, <a href="mailto:tarn@tarnbarford.net">tarn@tarnbarford.net</a> (<a href="/pgp.txt">public key</a>)</p> <p>&nbsp;</p> </div> </body> </html>

Permalink

Swipe Keyboard

<!DOCTYPE html> <html> <head> <title>Tarn Barford</title> <meta charset="utf-8"/> <link rel="icon" type="image/x-icon" href="/favicon.ico"> <link href="/style.css" media="screen" rel="stylesheet" type="text/css" /> <link rel="alternate" type="application/atom+xml" title="Journals of Tarn Barford" href="/atom" /> <link href="/highlight.css" media="screen" rel="stylesheet" type="text/css" /> <link href="/highlight-console.css" media="screen" rel="stylesheet" type="text/css" /> <style> #swipe-canvas { position: relative; width: 900px; height: 300px; } #swipe-results { font-size: 30px; padding-left: 50px; padding-left: 50px; } #swipe-results ul { margin: 0px; padding: 0px; } #swipe-results li { float: left; background-color: #DDDDDD; list-style-type: none; padding: 10px; margin: 5px; border-radius: 5px; } #swipe { position: relative; } #swipe-loading { position: absolute; height: 50px; width: 300px; top: 85px; left: 300px; background-color: darkgray; border-radius: 10px; text-align: center; padding-top: 20px; border: black; border-width: 5px; } </style> </head> <body> <div id="container"> <div id="header"> <div id="header"> <p>From the <a href="/journal">Journals</a> of <a href="/">Tarn Barford</a></p> <h1> Swipe Keyboard </h1> <p> Apr 06, 2014 </p> </div> </div> <div id="post_content"> <html><body><p>When I first tried a <a href="http://www.swype.com/">Swype</a> keyboard I was impressed how effective it was. Even though I don't use the feature on my phone I was interested in how it could be built, so I <a href="https://github.com/tarnacious/swipe-keyboard">implemented this otherwise useless swipe-able keyboard</a> below. It probably doesn't work on mobile devices, but works on modern browsers with mouse pointers (although I've only really tried Chrome and Firefox).</p> <div id="swipe"> <canvas height="300px" id="swipe-canvas" width="900px"></canvas> <div id="swipe-results"></div> <div style="clear: both"></div> <h2 id="swipe-loading">Loading<noscript>Javascript is Required</noscript></h2> </div> <p>I initially tried to solve this using the technique Peter Norvig famously uses in his <a href="http://norvig.com/spell-correct.html]">spell checker</a>. He takes a sequence of characters and generates a set of word candidates by adding, removing and swapping characters in the original sequence, the generated candidates are removed if they are not found a dictionary. This can work but to be effective too many combinations need to be generated.</p> <p>If the dictionary is indexed into a <a href="http://en.wikipedia.org/wiki/Trie">trie</a> the number of combinations generated can be reduced significantly by traversing the trie and only generating valid letter combinations. This is a pretty bare implementation of that, it requires: </p> <ul> <li>The first and last characters of the initial sequence are used </li> <li>Intermediate characters in the initial sequence can be repeated or discarded </li> <li>No characters are added or swapped</li> </ul> <p>Basically, if you swipe through all the characters in a word in order, then the word will be found if it is in the index regardless how many characters are swiped in between. It is surprisingly quick and effective.</p> <p>This implementation uses <a href="https://raw.github.com/first20hours/google-10000-english/master/google-10000-english.txt">these 10000 words</a>, I intended to use digital books but never got around to it as these words demonstrate the concept well enough.</p> <p>This is the first thing I've written in <a href="https://github.com/clojure/clojurescript">ClojureScript</a> or <a href="https://github.com/clojure/clojurescript">Clojure</a>, so my code my vary from non-idiomatic to shamblolic. I initially used a <a href="http://clojuredocs.org/clojure_core/clojure.zip/zipper">zipper</a> to build the trie with immutable data structures, but found the indexing took to long with my zipper implementation so I <a href="https://github.com/tarnacious/swipe-keyboard/commit/6edd7b26e78121fbe8586b3f0ef54ca8277d9e32">switched to using native Javascript maps</a>.</p> <p>I found that <a href="https://github.com/clojure/core.async">core.async</a> library is really awesome, the <a href="http://docs.closure-library.googlecode.com/git/index.html">Google closure library</a> and <a href="https://developers.google.com/closure/compiler/">compiler</a> integration with <a href="http://leiningen.org/">Leiningen</a> the <a href="https://github.com/emezeske/lein-cljsbuild">cljsbuild plug-in</a> to be impressive. My main pains were the slow JVM start-up time, the advanced closure compiler build of the web worker script fails silently when run (but the main script works fine when compiled with the advanced compiler), and at times I felt some compile time type checking would be nice.</p> <p>I would like to extend this experiment to index the word occurrence counts and proceeding word counts in original text and rank the found words as most likely. Support casing, umlauts, special characters, spelling correction and compound words in the indexing and lookup. I think a live lookup while swiping would also be possible.</p> <p>Overall this was fun, turned out OK I think, and was a great learning experience.</p></body></html> </div> <div id="comments"> </div> </div> <div id="footer"> <p>&nbsp;</p> <p>Questions, comments, suggestions? Email me, <a href="mailto:tarn@tarnbarford.net">tarn@tarnbarford.net</a> (<a href="/pgp.txt">public key</a>)</p> <p>&nbsp;</p> </div> <script src="swipe.js" type="text/javascript"></script> </body> </html>

Permalink

editmd 0.2.0

This update adds change tracking. When your agents change a file you’re working on, their changes are automatically highlighted as they appear. It looks like this:

Two agents edit a plan open in editmd, each highlighted in its own color

Each agent&aposs changes are highlighted in its own colour. A best effort attempt is made to label the agent, otherwise it gets a generic label, either way you can edit labels in the legend in the corner. Highlights stay with the agent&aposs text while you keep working and are not saved to the file.

Also in this release is a workaround for a small WebKit bug so that the caret is drawn at the height of the text.

Permalink

Personal agents need a new kind of harness

I’ve been working on building a personal agent that is both useful and secure and have come to the conclusion that it needs a new kind of harness, specifically one without a shell. A personal agent comes with the lethal trifecta by default — in order to be useful, it needs access to your data and the internet, which is full of untrusted content and ways to exfiltrate information. Right now personal agent security, such as it is, relies on hopes and prayers that LLMs will follow your requests to please not get pwned. This is wishful thinking and not a reasonable or responsible security posture, in my opinion.

Traditional security practices break down with agents, because agents are not like people. When you grant a person access to a system, you are trusting their judgment, and a person&aposs judgment is a property of who they are, which is why identity-based authorization mostly works for people. A person who reads a phishing attempt is still the same person with the same decision-making capacity as before they read the message.

Agents are fundamentally different. They have no such judgment or decision-making capacity of their own. They act based on instructions they are given, and anything that enters an agent&aposs context can become part of its instructions. The identity of an agent remains stable over time, but whoever is effectively in control of it does not necessarily. After an agent reads a stranger&aposs email, that stranger is partly directing the agent. This is why identity cannot be the basis of authority for agents. An agent&aposs authority is a function of whatever is in its current context, and what tools it has access to at a given moment.

Enforcing this kind of session-based authority requires the harness to mediate every read and every send. It must be the exclusive broker and monitor of every LLM call — everything that enters the session context has to pass through the harness in a tamper-proof way so that it can track what information the session holds, and every action has to pass through it, so that it can tell what an action does before it executes it. A secure harness is effectively a reference monitor (see §3.4, p. 16) — always invoked, tamper-proof, and small enough to verify. The problem is that this is impossible if the agent has a shell. So secure agents cannot have a shell.

The problem with guardrails

Coding agents, like Claude code and codex, are extremely useful because they have broad access to powerful tools and relevant data at the same time. They have access to a shell on your computer, which means they can effectively do anything, and their approach to security is to grant access to everything then try to add guardrails. Things like execution sandboxes, approval prompts, and classifiers. There are many problems with this approach, though. The main one being that it doesn&apost work. Or at least not all of the time, and as we all know, in security 99% correct is a failing grade.

Prompting for approval leads to fatigue. Anthropic found that people were accepting 97% of approval prompts, which meant after only 50 prompts they were catching just 5% of dangerous commands. To be fair this is partly because their approval prompts are unhelpful. For example this:

An inscrutable approval prompt, not helpful

is supposed to be the version of my job that no longer requires reading code. That is an inscrutable wall of mangled bash commands. There is no universe where a person can tell whether that is safe to run or not at a glance. Sure you can manually scrutinize it, linting and trying to follow the control flow with your eyeballs, but that is almost certainly slower than just manually doing whatever it is that Claude is asking you for permission to do in the first place.

Classifiers are models judging models, which are useful but also leaky. Anthropic&aposs own "Claude containment" doc notes that auto mode blocks "roughly 0.4% of benign commands", at the cost of letting ~17% of overeager actions through. Johann Rehberger of Embrace The Red got malicious code execution past auto mode with up to 80% success with some pretty basic techniques. Notably, the first step is getting claude to decide not to use a tool and to use curl instead. This is why your agent cannot have a shell.

It also goes to show why you cannot trust the frontier labs to self-police. Anthropic&aposs own commissioned test showed 0% success in 720 attempts to hijack auto mode. Their response to Rehberger&aposs report was that auto mode is a best-effort classifier, not a security boundary. IMO it&aposs quite disingenuous to market 0% attack success and then call an attack that succeeds 80% of the time out of scope, but I digress. In their response they say that OS level isolation and network egress control are the real boundaries, which as we&aposll see, also don&apost work.

The answer isn&apost a better sandbox

The standard answer to this is "just run it in a sandbox". The first problem is that if your agent has access to a shell inside a sandbox, information you are trying to keep inside will get out.

Just run claude in a sandbox

To be clear, you definitely need a sandbox, but it solves a different category of problems. Sandboxes limit what an agent can reach, but don&apost track what information the agent has access to. They are necessary but not sufficient.

Two days after Cowork launched, PromptArmor got it to upload a user&aposs files to the attacker&aposs Anthropic account through Anthropic&aposs own Files API, using a key planted in a file. The exfiltration used also used curl from a shell. The request went to api.anthropic.com, which was on the allowlist because the agent needs it. Anthropic&aposs own account of the incident said "The sandbox worked perfectly, and yet the data was exfiltrated." It&aposs also worth noting that Rehberger had reported this same bug to claude.ai earlier and his report was, as ever, ignored. EchoLeak, found by Aim Labs, got data out of Microsoft 365 Copilot through an image URL on a Teams domain that Copilot&aposs content security policy trusted.

If you want your agent to be useful you simply have to give it access to some powerful tools and even if you only allow the bare minimum, those channels will leak.

An agent&aposs authority depends on what it has seen so far

For a person, permissions can be identity based, because your judgment remains your own, at least to a first approximation over the short term. But agents don&apost have any judgment or taste of their own, they follow instructions and relentlessly pursue whatever objectives they are optimized for. What an agent can safely do depends on what&aposs in its current context.

Most agents today run on what capability-security researchers call "ambient authority", i.e. authority derived from the identity of the program rather than from the request it&aposs acting on. This leads to the classic Confused Deputy from Norm Hardy&aposs 1988 paper, in which a compiler allowed to write to its own directory overwrote the billing file kept there, because a user handed it that filename as its output. This is effectively the same class of bug as prompt injection. It is far too easy for outsiders to inject instructions into an agent&aposs context today, tricking the agents into using your authority to carry out their requests. The fix from this line of research is to derive authority from the request rather than the agent.

The idea that what you&aposve read limits what you may do is also very old. Kenneth Biba&aposs 1977 integrity model (starts p. 13) includes a low-water-mark policy, where a subject&aposs integrity drops to the lowest level of anything it has read. Meta&aposs Agents Rule of Two is a similar idea applied to agents, saying an agent may hold at most two of the lethal trifecta&aposs properties within a single session, and needs supervision once it has all three. These are both precedent for making authority a property of the session, not of the agent.

The shell itself is the problem

To work out an agent&aposs authority from what its session has seen, the harness has to know two things about every action: what it reads and what it sends. A shell makes both of these inscrutable to the harness. cat, curl, git push, rm -rf, a DNS lookup, or a file write to a synced folder all look the same on the surface (a string handed to bash). Even if you restrict the commands a shell may run, many commands can both read and send, e.g. fetching a URL can exfiltrate data via query params. And there are countless ways a shell can move data from disk to network without ever passing through the model&aposs context, so watching the context isn&apost sufficient. Mediating these calls has to happen beneath the model, in the harness itself.

In principle, the harness could police every read and send a shell does, but it would have to trace every file, process, and socket each command touches and propagate labels through all of them. This is effectively building an information-flow operating system under the agent, which researchers have done, many times (see Asbestos, HiStar, and Flume). In practice, at that granularity the label creep cripples the usability of the system almost immediately, and the decisions it surfaces are too low-level to be meaningfully approvable anyway, like "process 4121 is trying to connect to 104.18.x.x". This is the inscrutable approval prompt problem again, just at an even lower level.

The fundamental problem is that a harness with a shell is trying to answer "what does this command actually do?" before it runs, and Rice&aposs theorem proved in the 1950s that for arbitrary programs that question is undecidable.

Rehberger&aposs attack is a good example showing how difficult it is to secure a tool as generic and powerful as a shell. The fatal command auto mode approved was a short Python decoder he had tricked Claude into writing itself, after refusing to run the binary decoder he supplied. Superficially, this is harmless. It was only a problem because it ran from inside an archive it had just unzipped, where he had planted a struct.py that shadowed the standard library&aposs, so Claude running import base64 inadvertently ran the malicious code. In some runs, auto mode even blocked Claude&aposs attempt to kill the malware after Claude noticed it. Every step seemed reasonable and passed the auto mode classifier on its own, but that&aposs the thing — most shell commands do.

What a shell command does depends not only on its contents but also the current state of the filesystem and environment it runs in, which an attacker can largely control. Even with perfect knowledge of that state, the question of whether a given command is safe to run or not is undecidable in general. The best a harness can do is over-approximate, which in the context of a shell means assuming every command might read anything and send anywhere. You could restrict a shell to a decidable subset, like a fixed set of commands with checked arguments, but that is effectively rebuilding typed tools with worse ergonomics.

So a harness enforcing session-based authority through a shell just cannot do it automatically. All that&aposs left is asking the model to please be careful, or asking you for permission a thousand times a day.

A secure agent needs a different substrate

Since the shell is the problem, the agent cannot have one. But that means that in order to do anything, it needs access to tools, and the harness has to be the only way it can access those tools. So far these are the requirements that fall out of all this:

  1. The harness must mediate every action. The agent can’t have a shell or side channels or generic ways of accomplishing the things the tools do, like using curl to bypass a web fetch tool. The only way for an agent to read or act has to be via tool calls that the harness mediates, because the harness can only enforce rules on what passes through it.

  2. Every tool must declare its effects. A tool needs to declare whether it reads, writes, sends data out, or acts in the world, and what its output contains. A tool that doesn&apost follow this contract cannot be trusted to run.

  3. Labels follow the data and the agent can only accumulate them. Anything private is marked as such, anything an outsider could have written is marked as untrusted. The system also must not launder labels through summaries or copying or through time by persisting untrusted content and reading it back later as private.

  4. Policy is small, deterministic code outside the model. For example once a session holds both private and untrusted data, network egress must become structurally impossible without human approval, not just frowned upon.

  5. Only the data&aposs owner can change a label, and only through trusted paths. In decentralized information control flow systems there has to be some source of authority, one way is to have trusted principals with the ability to declassify their own data. In this case, tool calls awaiting approval can go through when I approve them. This approval has to happen on a trusted channel that no agent can spoof.

These requirements would make the harness more secure, but also close to useless, because in practice untrusted content comes fast and furiously into clean sessions, and under these rules that would make it very hard for agents to accomplish anything meaningful. So there&aposs another requirement:

  1. Untrusted content can inform data but never control. There has to be a narrow, checkable way to get information out of untrusted content into an agent session without contaminating its context, i.e. letting it steer the agent. Simon Willison describes one way to do that in his dual LLM proposal with a quarantined model that reads untrusted content and passes back only references or checkable answers. Another more formalized way is described in Google&aposs CaMeL paper, a privileged model whose control flow comes only from the user&aposs request, with untrusted data entering only as tracked values. Microsoft also has an interesting solution, described in their FIDES paper, which tracks confidentiality and integrity labels and hides untrusted values from the model entirely, and is actually available as experimental middleware in Microsoft&aposs Agent Framework.

What&aposs missing is a substrate that incorporates all of these ideas into a single runtime upon which the kinds of personal agents people actually want to run could use.

Where would this leave us?

A system designed like this is basically trading risk for attention. Useful agents often do genuinely need all three legs of the lethal trifecta, which means you&aposd have to accept the risks that come with that in order for them to be fully autonomous. The alternative is to designate yourself as the sole trusted principal in the system and have every privilege escalation or potentially risky action require your approval. This is very hard to do in a way that doesn&apost lead to approval fatigue. For starters the questions have to be narrow, clear, and specific, not asking me to judge whether a wall of bash is safe to run or not.

There are also many other downsides to this kind of design. Keeping track of private and untrusted data quickly leads to label creep, and it&aposs hard to design a system where everything isn&apost just immediately flagged as contaminated. Typed and sanitized answers can still be wrong. The system also can only act through tools, which limits its capabilities and requires purpose-built tools for every new desired capability.

I think coding agents are tremendously useful, but coding agents need a shell because running arbitrary commands in shell is how software is made — compiling code, running tests, managing version control are all essential. But a repo and its dependencies is a much smaller thing to trust an agent with than my inbox, my calendar, and my contact list.

Way back in 1972 James Anderson&aposs planning study for the US Air Force identified the central threat to shared computers as the malicious user, i.e. someone with legitimate access trying to subvert the system from inside. This is a very difficult threat model to defend against. This problem mostly went away with personal computers, so for the last forty years we&aposve effectively sidestepped one of the hardest problems in securing computers. The simplest defence against potentially malicious other users is to just not have any. But now personal agents have brought back this threat, since agents run on our devices with elevated privileges, and because anyone who can send me an email is now, effectively, a user of my agent. The defences the industry built for that threat — mediating access, labelling data, declassification by trusted principals — are fifty years old and well understood. They&aposre just not incorporated into the agents we are now trusting with our entire digital lives. To get them back I think we need a new kind of harness. I don&apost know if it&aposs possible to build something that incorporates all of this and is still useful, but I&aposm not ready to give up yet.

Permalink

Clojars and October 2026 Project Updates

Our October report includes an update on the Clojars maintenance and ‘on-call" support work undertaken by Daniel Compton and Toby Crawley during the last 6 months. In addition, Ambrose Bonnaire-Seargent and Cvetomir Dimov share the most recent updates on their projects.

Clojars maintenance and support overview
Malli: Ambrose Bonnaire-Seargent
SciCloj Documentation and Plotting Libraries: Cvetomir Dimov

Clojars maintenance and support overview

Clojurists Together funds Toby Crawley and Daniel Compton to co-maintain Clojars. Their work includes responding to operational issues and community support requests, maintaining the service and its infrastructure, applying security updates, improving publishing and administration tools, and keeping hosting reliable and cost-effective. You can follow this work in the Clojars changelog and in our project updates. For help with Clojars, visit the #clojars channel on Clojurians Slack or open an issue. Clojars itself is open source, and contributions are welcome in the Clojars GitHub organization.
Review period: April to 30 September 2026
Author: Daniel Compton
Context: I took lead maintenance responsibility for Clojars from April-June while Toby Crawley was away and went back to co-maintainer when he returned.

Clojars maintenance & support: April to June 2026

Account and supply-chain security

Stronger password requirements: I increased the minimum Clojars password length from eight to twelve characters #934.

Session invalidation after password changes: I reviewed an earlier proposal for invalidating sessions, then made a smaller implementation #944. When a user changes or resets a password, Clojars now removes that user’s active sessions. This was a fairly low-risk issue, but featured highly in bug bounty submissions.

Deploy and token notifications: I added a migration to enable deploy notification emails for old users created before this preference existed #943. This will help maintainers detect a supply-chain takeover or unexpected deploys.
I also reviewed and merged the work adding email alerts when a deploy token is created #945.

Compromised-password detection: I started working on Have I Been Pwned password detection in #951 but haven’t completed it.

Infrastructure reliability and resilience

IPv6, DNS management, and redundancy: I imported our DNSimple DNS records into Terraform so changes could be reviewed and audited. I also added IPv6 support for `repo.clojars.org.
I also added Route 53 as a second DNS provider alongside DNSimple (closing #60). This increases resiliency to DNS outage issues. This is not a theoretical concern, in February DNSimple had an DNS resolving outage which caused a Clojars outage.

Build and deployment repair: I fixed the Packer build #55. I also fixed the app uberjar so it can build on macOS case-insensitive filesystems #936.

I also made the instance-refresh script wait for AWS Auto Scaling to report success or failure instead of returning as soon as a refresh started. It reports progress and exits unsuccessfully for failed, cancelled, or rolled-back refreshes. I updated the Leiningen version on the production image to resolve a deployment problem.

Infrastructure as code: I moved the Clojars maintainer IAM group and its day-to-day AWS permissions into Terraform. This replaced console-only configuration with a Terraform maintained policy covering the services needed to operate Clojars, billing information, and Route 53 Domains.

I added contributor and coding-assistant guidance to the infrastructure repository and updated its setup documentation for DNS credentials and the Terraform workflow.

Development workflow

I added coding agent guidance to help guide usage of coding agents.

Clojars maintenance & support: July to September 2026

This covers Daniel’s and Toby’s work on Clojars.

Daniel’s work

Infrastructure hardening

Daniel changed the production EC2 launch-template configuration to require IMDSv2 tokens and limited metadata responses to one network hop. Before opening the change, he checked CloudWatch metadata metrics and exercised the real Cognitect AWS client against a local IMDSv2-only endpoint #61.

He prepared dedicated private database subnets with no internet or NAT route, moved the production RDS configuration into a private subnet group, disabled public accessibility, and enabled deletion protection #62.

Daniel also updated the Terraform backend to use the native S3 lockfile supported by current Terraform versions, removing the need for a separate DynamoDB locking table commit 357062b.

Toby identified that the production repository metadata still referred to the old HTTP URL in #966. Daniel then fixed the repo metadata to https://repo.clojars.org and managed the S3 object via Terraform #63.

Application tooling and maintainer review

Daniel updated the bin/clj-kondo bootstrap script to select the correct release for Intel and ARM64 builds on macOS and Linux and to keep architecture-specific cache directories. Commit 32cf33f landed on 20 August.

On Call management

Daniel added more monitoring for Clojars and setup an on-call 24x7 pager rotation for Clojars.

Security review

Daniel began a security review of the Clojars service. No major issues were identified, but some remediation work is in progress and will be published soon.

Toby’s work

Clojure Conj Talk

Toby gave a talk at the 2026 Clojure Conj about the history of Clojars and how it works. The video will be available on the ClojureTV YouTube channel soon.

Account, session, and authentication improvements

Toby simplified the normal login form so it only requests an MFA code after a user with MFA enabled has supplied valid credentials #950.

He moved sessions from the application’s memory into PostgreSQL, added a twenty-four-hour idle expiry, and added database cleanup for expired sessions. This makes longer-lived sessions practical and is one of the required steps for us to run multiple application instances - #964.

Toby tightened what the OAuth workflow returns so unserialisable provider data would not leak into session state, then corrected the workflow response handling after GitHub login failed in production. These fixes landed directly as commit 1896a51 and commit e9542cf.

He also updated GitLab login to accept all email addresses that GitLab marks as verified instead of requiring the address to be primary #971. Separately, #969 added the Clojars account name to notification-email footers, making messages unambiguous for people who have more than one account associated with an address.

Supply-chain and dependency security

Toby updated Jackson, the PostgreSQL driver, and Logback to address reported vulnerabilities in #956, then handled another set of dependency vulnerabilities and cleaned up obsolete or spurious suppressions in #965. A further September dependency update addressed eight CVEs in commit 67727be.

He prevented publication of POM and Gradle module metadata containing dynamic dependency versions, e.g. version ranges like [16,17), RELEASE, or SNAPSHOT #960.

Toby removed the separate Apache HTTP client stack in favour of the http-kit client already in use, reducing the dependency and CVE surface #961. He also switched from a temporary fork of the one-time library back to its official release #962.

Development, CI, and operational reliability

Toby updated clj-kondo, adopted its new constant-condition linter, and fixed the issue the new check found #963.

After removing Apache HttpClient, he fixed Fastly cache purging by switching to Fastly’s POST API, which the http-kit client supports. Commit 0c7e45a landed on 22 July. He also aligned the local Docker Compose environment with the PostgreSQL 18.4 version used in production in commit 61eb196.

Toby replaced the unavailable MinIO test image with S3Mock, updated the CircleCI base image, and repaired Maven dependency caching that had not been using the intended path #970.

His infrastructure maintenance increased Fastly log retention for analysis, corrected that policy so it actually retained 120 days, and fixed the instance-selection script so status messages no longer contaminated the IP address returned to callers. These changes landed as commits 5ebe01a, a994755, and f615c38.


Malli: Ambrose Bonnaire-Seargent

Q2 2026 Final Report 3. Published October 15, 2026

In this project, I am tackling exponential growth related to Malli refs. This is my final update of three. Thanks to Clojurists Together and Metabase for supporting this project.

I concentrated on three activities for this time period.

Draft PR: https://github.com/metosin/malli/pull/1284/changes

First, I added documentation to my branch explaining what the proposed changes to malli.core enables. See the README.md changes in my pull request.

Second, I added more complex test cases for the new cycle detection in schemas. There were a few useful test cases I introduced years ago to test cycle detection in malli.generators, and now I have repurposed them for malli.core tests as the cycle detection has moved there.

Third, I took some time to deeply consider why a subtle condition is needed in malli.generator, that disables caching for generators if we’re currently under a gen/recursive-gen. Again, this was something I added years ago but didn’t fully understand its function. Earlier in this project, I had removed the condition, but it caused the tests to fail. It resulted in this small documentation change. I don’t know if we can improve the caching in this area of code, and it’s a potential area of future work. I suspect there’s more to learn here, and it may reveal a more effective approach to creating generators.

I’m pretty happy with the code, overall it elegantly resolves several hacks deep in Malli’s design while either conserving or improving performance, and exposes them such that extensions can better support recursive schemas. I need to do some work to update the PR description and communicate my proposal to the Malli maintainers.

A goal of this project was to address a hack in Metabase around the performance of ref schemas. I think this work has brought us closer to that goal by better understanding how to detect cycles in schemas and when it is safe to reuse schema objects. The project concentrated on a smaller scope, mainly property registries, and I documented some ideas for how we could generalize the benefits to more complex registries. More importantly, the time spent considering the smaller scope will give greater confidence that such generalizations will actually be correct and a good idea. I hope to find out soon.


SciCloj Documentation and Plotting Libraries: Cvetomir Dimov

Q2 2026 Final Report 1. Published September 30, 2026

During the first month, I worked on both documentation and libraries. In terms of documentation, I mostly finished with the redesign of the Noj website. The website now looks much more modern with a theme inspired by the Macroexpand conference website and a main page. It also makes sure to spell out Noj’s main strengths, it presents the main libraries in an appealing way, and links to Zulip, Github and SciCloj. Moreover, the website is now structured to have separate sections for a quick start, for a book to learn data science with Noj, and for library references.

In terms of library development, initial steps have been made toward a new library for interactive dashboards and for porting the dataset library Tablecloth to ClojureScript. Specifically, the repository of the interactive dashboard library has been initiated, a feature list has been created and the base architecture has been designed. Currently, the process for selecting the required technologies is taking place. For Tablecloth, I have started studying the design of the existing Clojure library.

Permalink

Posts that Hacker News liked

The following is a list of my articles and projects that I think made it to the front page of Hacker News (they each got at least 29 points, and as I write this there's a post on the front page with 29 points).

27 total, six of which are Show HNs. Of the three posts that had more comments than points, two are about advertising.

Some of the links from the HN posts were broken, so I've changed the links here. Too lazy to set up redirects in some cases, unfortunately. I've also removed links for most of the Show HNs since the projects they discussed are defunct. I merged The Sample into Yakread, so it kind of lives on.

Permalink

Clojure 1.13.0-alpha10

Clojure 1.13.0-alpha10 is now available! Find download and usage information on the Downloads page.

Destructuring changes

To make what were (implicitly optional) top-level bindings required, you can now move them into a map under a new :req! directive. This includes bindings to different names or nested destructuring, e.g. {a :long-a} ⇒ {:req! {a :long-a}}.

:namespaced/keys[!] directives now qualify any unqualified keywords. Note that this may be a breaking change from the prior alphas for unqualified keys after the & where unqualified keys were not interpreted as qualified.

Inliner updates and macro invocation tag preservation

Previously, :inline on function definitions needed to work for all inlines. Inline is now conditional and the inline definition can return nil to indicate the function can’t be inlined, and the non-inlined function should be used instead.

get-in now inlines when passed a literal vector of keys.

Type hint metadata on a macro invocation is now conveyed on the expansion, which may be wrapped in a call to identity if needed. A type hint tag on the macro invocation will dominate one on the expansion when they disagree.

Try it out

Update your deps.edn :deps with:

org.clojure/clojure {:mvn/version "1.13.0-alpha10"}

Start a REPL with the Clojure CLI (any version) with:

clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.13.0-alpha10"}}}'

Permalink

From the Outskirts: Q3 2026

From the Outskirts is a roundup of my open source work in Q3 2026.

In the last update, I mentioned a DNS library and hoped to get a public preview of Busker out during Q3. Both projects are now public.

There's also been some work on FFI (native interop) in the Clojure ecosystem that I'm particularly excited about.

In this edition of From The Outskirts:

Busker is in public preview

Busker is a Clojure web server built on libh2o (the web server core that powers Fastly's global CDN) serving Ring applications over HTTP/1.1, HTTP/2, and HTTP/3 with automatically provisioned TLS certificates (now Clave makes sense?). I've soft-launched it, no big announcement yet, so others can preview it and provide feedback before APIs solidify. It's available to try, but still experimental and not recommended for production use.

But why another Clojure web server? I'll expand on this in a future Rationale section of the Busker docs, but it boils down to simple deployments.

My clients value simplicity and affordability in the services I build and run for them (much of my work is operational, keeping servers online and healthy). Funding for these projects is grant-based, and between grants we have to run lean and mean. Every component in the deployment system is something that can fail. So I want to minimize all the moving parts.

Busker helps with this because it lets me embed the web server inside the Clojure application. I already embed the database+queue etc in the application also. Deployment is as simple as scping an uberjar (that can generate its own systemd service unit file) to a Linux virtual machine.

An internet-facing HTTP stack is a bad place to indulge in reinventing things. That's why another pillar of Busker is libh2o: we build on the H2O maintainers' work and Fastly's expertise rather than rolling our own HTTP implementations. Busker already gives me enough security-sensitive work without volunteering for another stream of incoming CVEs.

More on Busker in another post. The documentation has setup instructions and examples if you'd like to experiment with it. Send me feedback!

Native libraries from Clojure and Babashka

ol.vips and Busker's native bindings are now using babashka/ffi. borkdude was inspired created babashka/ffi after some cross-pollination between Anders Murphy and myself.

An FFI, or foreign function interface, lets you call functions in native libraries from another language. In this case, that means calling them from Clojure and Babashka, including ordinary bb scripts.

Before babashka/ffi I was using coffi, the library that babashka/ffi is inspired by. And I want to take a moment to give props and thanks on Joshua Suskalo, the coffi author. He pioneered this approach to easy native interop in Clojure, using Java's new Foreign Function and Memory API.

But why switch to babashka/ffi? Compatibility, simply put. It supports both Clojure and Babashka. It's built into Babashka itself, whereas coffi is not Babashka compatible. That opens up native libraries to small Babashka scripts, not just full JVM applications. You can load a library, describe the functions you need, and call them from the script. With the same babashka/ffi API you can also run code in Clojure on the JVM.

Example: libvips is a very well respected library in the image processing world, primarily for security, speed and extremely low memory footprint (compared to alternatives like ImageMagick). ol.vips is my Clojure library for image processing that wraps the native libvips shared library and provides a nice Clojure interface. As of this post it is ported to babashka/ffi but is not 100% compatible yet. I'm waiting on a small issue in babashka to be fixed.

Oh and in sqlite4clj I landed the coffi -> babashka/ffi migration also. Are you using SQLite in Clojure? Are you still using JDBC? Boo! Check out sqlite4clj!

This is what excites me about Project Panama and the JVM's Foreign Function and Memory API! Clojure was designed as a hosted language and taking advantage of the JVM library ecosystem has definitely been a boon. But often the Java libraries we want to consume are bloated and/or expose cumbersome object-oriented APIs that are ungainly in Clojure.

I think Clojure has a great future making use of native libraries instead of reaching for another bloated Java dependency. Like with libh2o and libvips, there are substantial existing implementations to build on, while the application-facing code can remain Clojure. Native libraries still need packaging and the bindings still need to get memory management right, but I'm glad to have this option in both Clojure and Babashka.

Mozilla Fluent tooling

Fluent is Mozilla's localization system for the text an application presents in different languages. It's been around since roughly 2017 and is in my opinion the best system for creating and managing translations of strings for software projects that don't bend the knee to English.

Take a selected-file count, with $count passed in as a number:

English (en.ftl):

selected-files = { $count ->
    [one] One file selected.
   *[other] { $count } files selected.
}

Arabic (ar.ftl):

selected-files = { $count ->
    [zero] لم يتم تحديد أي ملفات.
    [one] تم تحديد ملف واحد.
    [two] تم تحديد ملفين.
    [few] تم تحديد { $count } ملفات.
    [many] تم تحديد { $count } ملفًا.
   *[other] تم تحديد { $count } ملف.
}

English needs only two plural categories, but Arabic has six! With fluent-clj, the application supplies the same message ID and count:

(require '[noahtheduke.fluent :as fluent])

(def en (fluent/build "en" (slurp "en.ftl")))
(def ar (fluent/build "ar" (slurp "ar.ftl")))

(fluent/format en :selected-files {:count 2})
;; => "2 files selected."

(fluent/format ar :selected-files {:count 2})
;; => "تم تحديد ملفين."

However, suppose that the message we wanted to show addresses the reader with the 2nd person pronoun "You":

"You have two files selected."

Arabic distinguishes masculine and feminine forms of singular "you", but English's "you" doesn't.

Assuming we have the gender of the reader then this becomes in fluent:

English (en.ftl):

selected-files = { $count ->
    [one] You have one file selected.
   *[other] You have { $count } files selected.
}

Arabic (ar.ftl, [two] branch only):

...
    [two] { $gender ->
        [masculine] لديكَ ملفان محددان.
        [feminine] لديكِ ملفان محددان.
       *[other] تم تحديد ملفين.
    }
...

Unlike plural categories, masculine and feminine are labels we chose for an application-supplied $gender string. Fluent doesn't infer the person's gender. Passing "unspecified" uses the neutral fallback, "Two files have been selected." Rebuild the bundles from the updated files:

(def en (fluent/build "en" (slurp "en.ftl")))
(def ar (fluent/build "ar" (slurp "ar.ftl")))

(fluent/format en :selected-files {:count 2 :gender "feminine"})
;; => "You have 2 files selected."

(fluent/format ar :selected-files {:count 2 :gender "feminine"})
;; => "لديكِ ملفان محددان."

(fluent/format ar :selected-files {:count 2 :gender "masculine"})
;; => "لديكَ ملفان محددان."

The English translator/writer doesn't even have to know about the $gender for these strings. Arabic uses it to choose the wording.

Fluent is powerful and enables you to produce native sounding applications for all languages. If I'm building software for people, getting their language right is part of the job.

Fluent, however, does have a tooling problem, especially if you don't want to use the npm ecosystem. It can be cumbersome to work with fluent files in your editor as there's no linter or highlighting.

As part of a client project I was able to build some tooling which I can now open source under outskirtslabs/fluent-tooling. It includes :

  • tree-sitter-fluent, an error-tolerant Tree-sitter grammar
  • ftl-lint, a command-line linter for Fluent .ftl files with compiler-style diagnostics
  • and fluent-ts-mode, a Tree-sitter major mode for Emacs with syntax highlighting.

The docs cover the individual tools and how to use them.

Protocol53

ol.protocol53 provides one Clojure interface for managing DNS records across providers. Available provider integrations include Cloudflare, deSEC, GoDaddy, and Porkbun.

It grew out of Clave's need to handle ACME DNS-01 challenges without duplicating the integration work for every DNS provider.

Example usage:

(require '[ol.protocol53 :as protocol53]
         '[ol.protocol53.cloudflare :as cloudflare])

(def dns (cloudflare/provider {:api-token (System/getenv "CLOUDFLARE_API_TOKEN")}))

(protocol53/append-records! dns
 "example.com"
 [{:name "www"
   :type "A"
   :ttl  60
   :data "192.0.2.1"}]
 {:timeout (java.time.Duration/ofSeconds 10)})

See the documentation and API reference.

Further Out: Q4 2026

Stardust DB has landed in public preview. It's a data engine from the creator of Datastar. Think SQLite + Datomic + Redis + Elasticsearch + NATS + Object Storage all in one small binary. It has crazy potential, so I'll be poking and proding at it over the next months to learn more.

Busker's performance is pretty good, but it can be better. My goal is to be within 20% of vanilla h2os numbers for h1, h2, and h3.

That's all from the outskirts. Back to the lab.

‣ Casey

Reply via: Email · Bluesky

Permalink

The code was the easy part: Lessons from contributing to the Clojure language

Written by: Arthur Fücher

I always wanted to contribute to Clojure. Not building a new library, but improving the language itself. Working on a programming language felt too hard to me, a senior software engineer.

During a mentorship at Nubank, I told Alex Miller that I hoped to reach that point someday. His response was roughly: “You can do that today.”

Not everyone has the opportunity to have a Clojure core contributor mentor them. Alex’s point was simple: the tickets are public, the contribution process is clear, and there are real problems to solve now.

The mentorship became: pick an open ticket, work on it, learn.  And I worked on two Clojure core tickets. CLJ-2897, about dynamic library loading through prepl, merged and included in Clojure 1.13.0-alpha5. CLJ-1548, about primitive type hints in protocols, still in review when I presented this experience at Clojure/Conj 2026.

I expected the code to be the hardest part, and it was hard. But it wasn’t the hardest part: that turned out to be four questions. Questions that I had to answer before anyone would even look at my code.

1. What is actually wrong?

My first ticket involved Clojure’s prepl, a programmatic REPL. Like a conventional read-eval-print loop, it accepts expressions and evaluates them. Unlike a conventional REPL, it emits structured results. A developer can connect to a running process through a socket and evaluate forms..

Using  add-lib, which add dependencies to a running REPL session, failed in prepl:

That was an odd message to receive while using a REPL. The ticket identified the immediate cause: prepl did not bind *repl* to true. 

My first patch added the binding. It was a two-line change that addressed the ticket’s stated cause. I attached the patch. I felt great.

Then Alex pushed me to consider what the user wanted to do. They didn’t want a particular value for *repl*; they wanted to load a library.

I applied the patch and tested, the original error disappeared. But now another error message: Context classloader is not a DynamicClassLoader.

Same user. Same goal. Another error.

add-lib requires a DynamicClassLoader. Prepl hadn’t set one up.

My revised patch added both the *repl* binding and the classloader setup. Now, the prepl session could then add Cheshire, require cheshire.core, and call it.

The first patch removed the reported error, while the second made the user’s task work.

I had even seen a reference to the related classloader issue, CLJ-2540, in the original ticket. I read it without understanding how central it was. Reading a ticket and absorbing a ticket are different things.

2. Why does the current behavior exist?

Once the patch worked, I needed to understand why prepl lacked those pieces in the first place.

The explanation was historical. Prepl was created in 2018 by copying and modifying clojure.main/repl. But it didn’t keep the REPL’s DynamicClassLoader setup. Later, in 2023, clojure.main/repl gained the *repl* binding, but the change didn’t reach prepl. 

Two implementations with related responsibilities had drifted apart. This was a consequence of how the code had evolved.Understanding this raised a bigger question: should fix the drift? Or the symptom of the drift?

My second ticket exposed a different kind of mismatch.

In Clojure, we have primitive type hints. It tells the compiler to use a primitive-aware invocation rather than boxing a value. With ordinary functions, that can be a useful optimization. But consider this protocol example from CLJ-1548:

Defining the protocol works. Extending it for strings works. Directly calling the function fails.

The reason is subtle. defprotocol accepts the primitive hint but doesn’t generate a corresponding primitive-specialized method. Yet the hint remains in the function’s :arglists metadata. When the compiler reads that metadata and selects a specialized invocation. In this case IFn$OLO, an interface the generated protocol function doesn’t implement.

The JVM cast fails before dispatch to the string implementation can happen. The function’s metadata promises a calling convention that the generated function cannot fulfill.

Clojure 1.12 already included a fix for a similar issue, about primitive return hints, CLJ-2698. But CLJ-1548 concerned primitive parameter hints, and understanding the difference mattered.

“Why does this happen” is not a curiosity question. It’s the only way to decide whether the fix belongs..

3. What has already been considered?

Both tickets contained more accumulated reasoning than I first appreciated.

The prepl ticket had pointed to CLJ-2540 all along. A triage process expanded the protocol ticket with the related return-hint precedent. and possible solutions. I wasn’t arriving at untouched problems. Other contributors had already done important investigative work.

And yet simply reading those notes wasn’t enough. My first prepl patch proved that I could read a clue and still underestimate it.

After the revised prepl patch worked, Alex encouraged me to implement alternatives. Not only think about them. And we ended up comparing three approaches:

  • Copy the missing behavior into prepl. Add *repl* and the DynamicClassLoader setup. The patch was small, five added lines and four removed, but it wouldn’t prevent future drift.
  • Rebuild prepl around clojure.main/repl. Reuse the existing REPL’s extension points, potentially reducing divergence. Experimenting with this revealed complications involving shared state between read and evaluation phases, prompt behavior.. The prototype involved 65 additions and 44 removals.
  • Extract shared classloader setup. Introduce a reusable macro used by both REPL implementations. This provided limited reuse but still didn’t eliminate the drift.

The experiments mattered because they replaced an attractive architectural idea with concrete trade-offs. An approach can look elegant until it has to preserve existing exception handling, output structure, and other behavior that real clients expect.

In total, I wrote 126 lines of experimental code that wouldn’t be part of the accepted change. And I want to be clear: it wasn’t a waste. It was the cost of knowing different dimensions of the solution. You can’t know the simple option is correct until you’ve seen what the complicated ones actually cost.

4. What quietly rules most options out?

The approach that best prevented future prepl/REPL drift wasn’t selected.

Rebuilding prepl on top of clojure.main/repl offered the clearest reuse story. But getting the behavior right will need modifying repl itself. And repl has external users who may depend on details of its existing behavior. Changing it to fix prepl would impose risk on code that hadn’t experienced the original bug.

We chose the smaller, less disruptive patch. It solved dynamic library loading in prepl without claiming to solve future divergence. We documented that limitation rather than hidden it.

The protocol ticket made compatibility even more concrete. One proposed solution was to reject unsupported primitive hints at definition time. 

To me it was obvious:  Why let developers define something that will fail when called? 

But the answer was: Because not every call fails.

Let’s check this example using a protocol function named add (the definition is omitted here):

In a direct call, the compiler sees the primitive-hint metadata and calls .invokePrim.. Passing the function as a value uses plain IFn.invoke, not invokePrim.

That means there may be code with primitive hints on protocol methods that has worked fine for years.Rejecting those definitions would break code that currently runs.

So, the alternatives had different consequences for existing users.

On both tickets I had more than one option that worked. What was decided between them had less to do with the code, and a lot to do with everything around it.

The final challenge: Explaining the reasoning

By this point, I had spent time reconstructing behavior, implementing alternatives.  Ready for the next step: Screening/prescreening process. I thought I had done the difficult work.

Then Michael Fogus screened the protocol ticket and left six comments. None were about the code. They concerned the description: put the problem first, introduce context in the order a new reader needs it, and make the argument for the change easier to follow.

I rewrote the ticket’s explanation, but the patch stayed the same.

That feedback connected with something Peter Naur wrote in Programming as Theory Building (1985): 

“[…]programming properly should be regarded as an activity by which the programmers form or achieve a certain kind of insight, a theory, of the matters at hand.”

And he says one more thing: the theory lives in people. It does not fit entirely in the text. So if the theory is in my head, and the decision is in someone else’s, how do I move it across?

The four questions I had been answering were ways of building a story. A story that moves a theory from one head to another. 

The evolution of CLJ-2897 made the contrast visible. Its initial description had 82 words; the final one had 505. Alongside, were 9 lines of accepted code and 126 lines of experimental work written to be thrown away.

It shows that for a patch to move, the story, the theory had to be in the head of the person deciding.  A ticket isn’t paperwork, it is the transfer of understanding.

What contributing to Clojure core taught me

The prepl patch was ready in August 2025 and reached Clojure 1.13.0-alpha5 in July 2026. The protocol ticket, opened in October 2014, was still in progress when I wrote this..

That pace was frustrating. And it’s slow for the exact reason I am sharing about this experience. Nobody there accepts code before they accept the theory.

I came into this experience prepared to write code. What I wasn’t ready for were the four questions. And the fifth thing: writing it down as a story that builds the theory in someone else’s head.

Those activities were the real contribution, alongside the code.

I still think about Alex telling me that I could work on Clojure today. He wasn’t promising a quick patch or an easy review. He was pointing out that I could start learning from a real problem immediately.

Want to contribute? Explore Clojure’s issue discussions, its ticket tracker, and the official contribution workflow. Choose a problem and investigate what people are actually trying to accomplish. Read the history. Try the alternatives. Explain what you learn.

You don’t need a mentor to ask what the user is actually trying to do. 

You can do that today.

The post The code was the easy part: Lessons from contributing to the Clojure language appeared first on Building Nubank.

Permalink

2026 Nominations and Your Vote

Clojurists Together Foundation is in the process of electing new board members as three of our existing board members are finishing their terms. One member, Lorelai Lyons, has submitted an application for another two year term.

Departing members

Two others, Laurens Van Houtven (lvh) and Maria Geller have decided to step aside to open the door for other Clojurists to participate. Maria has been a board member since Clojurists Together was first formed in 2017, and lvh was elected in the first board election in 2018. lvh has been President of the Clojurists Together Foundation since we formed our own non-profit trade association in 2020 and put in a huge amount of time and energy to leave the Software Freedom Conservancy and form our own non-profit. I want to extend a very special thanks to them for their years of service and leadership in the community, and will be sad to see them leave. They were both instrumental in helping Clojurists Together get to where it is today.

Election

After accepting applications for nominations, we received 5 applications from candidates - who will serve two-year terms. These candidates are all talented Clojurists that we would love to have on the board.

Our bylaws state:

The Board shall nominate no more than 12 candidates seeking board membership in any given election. In nominating candidates for Director positions and in choosing the number of candidates to nominate overall, the Board shall use reasonable efforts to maintain a Board composition consisting of at least: (1) 25% female Directors, (2) 25% non-Caucasian Directors, and (3) 35% from any category(ies) of persons (e.g., race, gender, ability) commonly considered to have suffered from discrimination at some time and then-currently under-represented in the technology industry, in each case as determined by the Board in its reasonable discretion.

The board has nominated the following 5 candidates for the current board election:

Arthur Fücher
Henning Jansen
Lorelai Lyons
Chris Oakman
Mohit Verma

Next Steps

All current Clojurists Together members will be receiving their ballot to vote shortly. If you haven’t received a ballot by Wednesday, 22nd October, please contact hi@clojuriststogether.org.

Voting will close on Monday, 9th November, 2026 at midnight Pacific Time. The top three candidates, as voted on by members will win. We will announce the winners shortly afterwards and you’ll be able to meet them at the Annual members meeting at 10 am Pacific time, Nov. 17th, 2026. The new board will meet shortly after the annual meeting to select officers.

Candidate Profiles

Below we have published the candidates applications (in alpha order):


Arthur Fücher

Company: Nubank
GitHub: https://github.com/afucher
LinkedIn: https://www.linkedin.com/in/arthur-fucher

Why they’d like to join
I love Clojure and its community. I want to help it grow where it is still small.

Advocating for Clojure is already part of what I do. I run Clojure BR, the Brazilian Clojure community. It is a personal project, not tied to any company. We do meetups, livestreams, a newsletter in Portuguese, and more.

I would bring a non-US, Portuguese-speaking view to the board. I live in Brazil, so I can help the board stay close to maintainers and companies in Latin America. I can also support advocacy and content in Portuguese.

I also bring years of paid Clojure work, focused on developer experience and tooling. And I contribute to open source, including patches to Clojure core.

Bio
Arthur Fücher is a Clojure developer from Brazil focused on developer experience and tooling. He organizes Clojure BR, the Brazilian Clojure community, and contributes to open source Clojure projects, including patches to Clojure core. He has spoken at Clojure/conj in 2025 and 2026 and at conferences in Brazil, and is a member of the InnerSource Commons Foundation. Outside of code, he is a Magic: The Gathering player and board game lover.


Henning Jansen

Company: Norled AS
GitHub: https://github.com/Henningzen

Why they’d like to join
I have payed a keen interest in the Clojurists Community for several years, and clearly see the benefit from this organization and the impact it has across initiatives in the continued evolving Clojure echo-system. Given my background from industrial domains, I regard my insight and expertise highly relevant.

Bio
For more than seven years now, I have been part of developing and maintaining a large-scale enterprise application for Maritime Public Transportation, developed on Clojure, ClojureScript with Datomic in the backend. During parts of this time, I have been individual contributor to Clojurists Together. I want to expand my contribution to also being of service to the board, providing industry insights and experience.


Lorelai Lyons

Company: GBS Solutions Corporation (contractor using Clojure)
GitHub: thefakelorlyons
Slack/discord: melody

Why they’d like to join
I value the mission of CT, and I value Clojure and its community. I think open source development is important and I like the idea of supporting all of these causes through CT.

Bio
My name is Lor and I have been a part of Clojurists Together for two years now (as a Board Member). I have been writing Clojure as a hobby/passion for 5-6 years and professionally for 2 years now. I am most well known (most likely) from my work on ‘brainfloj’, but I have worked on several other significant projects in clojure, and I value the Clojure community and open source development in general. In addition to Clojurists Together, I am also relatively active in Clojure Camp, and SciCloj.


Chris Oakman

GitHub: https://github.com/oakmac/
https://chrisoakman.com
https://x.com/oakmac1

Why they’d like to join
I have been involved in the Clojure community for over ten years, and I think Clojurists Together is a positive force for our ecosystem. When I received a CT grant for Standard Clojure Style in 2024, I worked harder on that project than I normally would have because I wanted to honor the community’s vote of confidence. Funding open source pays for essential maintenance and bug fixes, but that signal of recognition and support matters just as much. I would like to help CT keep doing that good work.

At the last Clojure/conj, several people encouraged me to apply for the board, and I think I’d be a good fit. I have worked on ClojureScript and developer tooling since 2012, I know the ecosystem well, and I stay in contact with people across the community. I have also served on nonprofit boards for years outside of software, so I’m comfortable with the day-to-day work of meetings, voting, and follow-through. Mostly, I want to volunteer my time to help run and promote CT, and to contribute to some of the open source projects CT supports where I can be useful.

I also want to be an optimistic voice for how our community approaches AI. Software development is changing fast, and we should embrace this technology while still holding on to the principles that make Clojure great. We should support tooling and documentation that are easy for LLMs to consume, and encourage more content showing how well Clojure works alongside modern AI tools. I understand that many developers have mixed feelings about how AI is changing the work of software, and I think it is important for the Clojure ecosystem to actively engage with the future rather than sit on the sidelines.

Bio
Chris Oakman has been writing Clojure and ClojureScript since 2012. He is the author of the ClojureScript Cheatsheet (cljs.info/cheatsheet) and Standard Clojure Style, designed the ClojureScript logo, and worked on Parinfer. Chris worked as a coding bootcamp instructor for several years, including teaching a course on ClojureScript. He cares about developer education, supporting open source work, and building the documentation and resources that help the Clojure community grow. He lives in The Woodlands, Texas, where he plays ultimate frisbee and has served for many years on the boards of two local ultimate nonprofits.


Mohit Verma

LinkedIn: https://www.linkedin.com/in/mohitverma1981

Why they’d like to join
I am inspired by Clojure Together’s mission to sustain open‑source development and empower a global community of contributors. Having led large‑scale transformation programs in the BFSI sector, I see strong parallels between building resilient financial operations and nurturing sustainable developer ecosystems. Joining the Board would allow me to contribute my governance expertise, strategic oversight, and passion for community building to ensure the initiative continues to thrive with transparency, accountability, and long‑term impact.

With over two decades of senior leadership experience—including as SVP/Director at BNY Mellon—I bring deep knowledge of compliance, operational excellence, and stakeholder engagement. My background in organizational change management and risk frameworks equips me to strengthen board governance and support strategic growth. Beyond corporate leadership, my entrepreneurial and educational pursuits reflect a commitment to collaboration and knowledge‑sharing, aligning closely with Clojure Together’s ethos. I believe this blend of global governance experience and grassroots engagement makes me a strong fit to help guide the organization’s future.

Bio
I have more than two decades in Senior Leadership roles in the BFSI industry. In my most recent experience as Senior Director at BNY Mellon. My career has focused on compliance, operational transformation, and building strong governance frameworks. Along the way, I’ve led large teams, managed complex change programs, and helped organizations stay resilient in fast‑moving environments.

Outside of corporate life, I’m exploring entrepreneurship and education—launching a home‑based food venture and building an English tutoring practice. These projects reflect my passion for community, collaboration, and knowledge‑sharing. I believe this mix of global governance experience and grassroots engagement makes me well‑suited to contribute to Clojure Together’s mission of sustaining open‑source development with transparency and long‑term impact.

Permalink

Probabilistic programming for Clojure programmers: write the model once, choose the inference later

If you have used core.logic, you already know half of probabilistic programming. A logic program states constraints and lets the system search for the values that satisfy them. A probabilistic program does the same, except that a constraint can be soft: a value that fits the data poorly is not ruled out, only made less likely. Two dice show the idea below. A small forecasting model then runs under several inference methods in Foerster, an MIT-licensed probabilistic programming library for Clojure, and every result is checked against an exact answer.

Foerster continues the Anglican line of probabilistic programming in Clojure. Christian Weilbach, who wrote it, did his PhD in Frank Wood’s group at the University of British Columbia, where Anglican was built.

What is a probabilistic program?

Take two dice and the constraint that they sum to 7. Without any library, the plainest way to find the pairs that meet it is to generate pairs at random and keep the ones that pass:

What is this syntax?
defn roll []:
  inc(rand-int(6))
end

defn generate-and-test [n]:
  loop [kept []]:
    if n = count(kept):
      kept
    else:
      a := roll()
      b := roll()
      recur(if 7 = a + b:
              conj(kept, [a b])
            else:
              kept
            end)
    end
  end
end

frequencies(generate-and-test(6000))
;; each of [1 6] [2 5] [3 4] [4 3] [5 2] [6 1] about 1000 times
(defn roll [] (inc (rand-int 6)))

(defn generate-and-test [n]
  (loop [kept []]
    (if (= n (count kept))
      kept
      (let [a (roll) b (roll)]
        (recur (if (= 7 (+ a b)) (conj kept [a b]) kept))))))

(frequencies (generate-and-test 6000))
;; each of [1 6] [2 5] [3 4] [4 3] [5 2] [6 1] about 1000 times

The random choices form a generative model, a small simulator of two dice. The test conditions the simulator on what we know. What survives is the posterior: the pairs that are still possible, and how often each occurs.

A probabilistic program states the same model without fixing how to search it:

What is this syntax?
defn dice []:
  sp/spin:
    a := sample(dist/uniform-discrete(1, 7), :id, :a)
    b := sample(dist/uniform-discrete(1, 7), :id, :b)
    factor(if 7 = a + b:
      0.0
    else:
      ##-Inf
    end)
    [a b]
  end
end
(defn dice []
  (sp/spin
   (let [a (sample (dist/uniform-discrete 1 7) :id :a)
         b (sample (dist/uniform-discrete 1 7) :id :b)]
     (factor (if (= 7 (+ a b)) 0.0 ##-Inf))
     [a b])))

sample introduces an unknown, as a fresh logic variable does in core.logic, but with a probability on each value: here a die whose six faces are equally likely. factor adds to the run’s log weight. Zero leaves the run unchanged, and minus infinity makes it impossible, as a failing goal does. sp/spin turns the body into a program that an inference method can pause, copy and resume at each sample and factor. With {:method :enumerate}, Foerster runs all 36 pairs and returns the six that hold, each with probability 1/6. That is the answer run* gives, with weights.

Now soften the constraint. We do not know the sum, but a sensor that measures it with noise of about one showed 7:

What is this syntax?
observe(dist/normal(a + b, 1.0), 7.0, :id, :reading)
(observe (dist/normal (+ a b) 1.0) 7.0 :id :reading)

observe weights each run by how likely the reading is under it, here a normal density centred on the run’s sum. No pair is ruled out, and the arithmetic can be done by hand. Each of the six pairs that sum to 7 gets weight e⁰ = 1, each of the five pairs that sum to 6 or to 8 gets e^−½ ≈ 0.61, and the weights keep falling with the distance from 7. Normalized, a sum of 7 has probability 0.45, sums of 6 and 8 have 0.23 each, sums of 5 and 9 have 0.04 each, and the rest almost nothing. Foerster’s enumeration gives the same numbers.

core.logicFoerster
a fresh logic variablesample, an unknown with a probability on each value
a goal such as ==, which holds or failsobserve or factor, a weight; failing is weight zero
conde, a choice between branchesa discrete sample, with a probability on each branch
run*, the set of answersthe posterior, the answers with their weights
the search strategythe inference method

Two more ideas carry over. The total weight before normalizing, called the evidence, says how well the model as a whole explains the data. A hard constraint that no run meets leaves nothing, and a soft one leaves a number that can be compared between models. And as core.logic leaves the search to the system, a probabilistic program leaves the inference method open. ProbLog joins the two traditions as probabilistic logic programming.

How does the system find the answer?

Enumeration works for 36 pairs. It fails for many choices, or for continuous ones, because no list of all runs exists. The other methods search the space of runs, and each resembles a familiar strategy:

  • Importance sampling runs the program many times and weights each run instead of discarding it. It is generate and test with soft scores, and it fails when good runs are rare.
  • Sequential Monte Carlo (SMC) runs many copies side by side. At each observation it copies the runs that fit and drops the ones that do not, like a search that prunes partial solutions as soon as a constraint can be checked.
  • Markov chain Monte Carlo (MCMC) changes one choice of a single run at a time and keeps the change by a rule that sometimes accepts a worse run. This local search visits runs in proportion to their weight.

Does the choice of method matter?

A shop sells around a hundred units of a product each week. The weekly figure is noisy, and the underlying demand drifts. How high is demand now, and how certain is that? The model says that demand starts near 100, moves by a random amount each week, and that each week’s figure is demand plus noise:

What is this syntax?
defn demand [weeks drift]:
  sp/spin:
    loop [t 0, level sample(dist/normal(100.0, 20.0), :id, [:level 0])]:
      observe(dist/normal(level, 8.0), nth(weeks, t), :id, [:y t])
      if inc(t) = count(weeks):
        level
      else:
        recur(inc(t), sample(dist/normal(level, drift), :id, [:level inc(t)]))
      end
    end
  end
end
(defn demand [weeks drift]
  (sp/spin
   (loop [t 0
          level (sample (dist/normal 100.0 20.0) :id [:level 0])]
     (observe (dist/normal level 8.0) (nth weeks t) :id [:y t])
     (if (= (inc t) (count weeks))
       level
       (recur (inc t)
              (sample (dist/normal level drift) :id [:level (inc t)]))))))

As with the dice, sample marks the unknowns, here twenty-one weekly demand levels, and observe scores each run against the twenty sales figures. The data are an ordinary argument, weeks. The :id names each choice, so a method can refer to the level in week 7 across runs. The pictures article explains priors and posteriors if those terms are new. Each method is one option of the same call:

What is this syntax?
infer/infer(demand(weeks, 4.0), {:method :smc, :particles 1000})
infer/infer(demand(weeks, 4.0), {:method :importance, :particles 1000})
infer/infer(demand(weeks, 4.0),
  {:method :mh, :iterations 4000, :chains 4, :burn 1000})
infer/infer(demand(weeks, 4.0),
  {:method :pgibbs, :particles 50, :iterations 600})
(infer/infer (demand weeks 4.0) {:method :smc :particles 1000})
(infer/infer (demand weeks 4.0) {:method :importance :particles 1000})
(infer/infer (demand weeks 4.0) {:method :mh :iterations 4000 :chains 4 :burn 1000})
(infer/infer (demand weeks 4.0) {:method :pgibbs :particles 50 :iterations 600})

The model is linear and Gaussian, so a Kalman filter computes the exact answer, which makes it a fair test. The data are twenty simulated weeks, with a true drift of 4 per week and observation noise of 8. Each particle method ran ten times with different seeds, so the table shows how much an answer varies from run to run. All numbers come from Foerster’s main branch on 15 October 2026.

MethodFinal level (mean)Log evidenceCheck
Exact (Kalman filter)102.89−77.49
Sequential Monte Carlo, 1000 particles102.92 ± 0.23−77.44 ± 0.14about 120 histories survive to week 0
Importance sampling, 1000 samples102.9 ± 1.8−77.7 ± 0.52 to 10 effective samples
Metropolis–Hastings, 4 chains103.4R-hat 1.03, 114 effective draws
Particle Gibbs, 4 chains102.93R-hat 1.002, about 1500 effective draws

SMC lands within 0.36 of the exact final level in every run, and its log evidence averages −77.44 against the exact −77.49. It pays for its pruning on the early weeks: of a thousand particles at the end, only 114 to 134 still descend from different starting points. Importance sampling never prunes, so after twenty weeks a handful of runs carry nearly all the weight, and the estimate moves by almost two units between runs.

Metropolis–Hastings (MH), the simplest MCMC, fails here, and its diagnostics say so before any exact answer is consulted. Its four chains disagree: their rank-normalized R-hat is 1.03, above the usual threshold of 1.01, and their 12,000 draws are worth about 114 independent ones. Foerster computes these numbers as Stan and ArviZ do. The cause lies in the model. Neighbouring weeks are strongly tied, so changing one week’s level at a time barely moves the run. Andrew Gelman’s folk theorem of statistical computing, restated in the Bayesian workflow paper, says as much: when the computation struggles, look at the model.

The fix is a method that changes all weeks together. Particle Gibbs runs a whole SMC through the twenty weeks at every step, holding one surviving run fixed, and so redraws all twenty-one levels at once. Four chains of 500 steps after warm-up, with 50 particles each, reached an R-hat of 1.002 and about 1500 effective draws of the final level, counting each step’s weighted average as one draw. Their mean was 102.93 against the exact 102.89. The first week’s level, which SMC sees through only a few surviving histories, reached an R-hat of 1.002 and about 660 effective draws. Particle Gibbs took four minutes against half a minute for MH, but produced about twice as many effective draws per second. Nothing in the model changed, only the search.

Twenty weeks of observed demand with the inferred demand level: the exact filtered mean and the 95% band from sequential Monte Carlo. 90 100 110 120 130 140 1 5 10 15 20 week observed SMC level, 95% band exact (Kalman filter) Each point is one week's observed demand. The shaded band is where sequential Monte Carlo puts the hidden demand level with 95% probability after that week, and the dashed line is the exact answer, which this simple model happens to allow. The two lines nearly coincide.

What if a parameter is unknown too?

So far the drift was given as 4. In practice nobody measures how fast demand moves, so the drift becomes one more unknown with a prior:

What is this syntax?
defn demand-unknown [weeks]:
  sp/spin:
    drift := sample(dist/gamma(2.0, 2.5), :id, :drift)
    loop [t 0, level sample(dist/normal(100.0, 20.0), :id, [:level 0])]:
      observe(dist/normal(level, 8.0), nth(weeks, t), :id, [:y t])
      if inc(t) = count(weeks):
        drift
      else:
        recur(inc(t), sample(dist/normal(level, drift), :id, [:level inc(t)]))
      end
    end
  end
end

infer/infer(demand-unknown(weeks),
  {:method :pmmh, :particles 100, :iterations 2000, :params #{:drift},
   :scale 3.0, :burn 200})
(defn demand-unknown [weeks]
  (sp/spin
   (let [drift (sample (dist/gamma 2.0 2.5) :id :drift)]
     (loop [t 0
            level (sample (dist/normal 100.0 20.0) :id [:level 0])]
       (observe (dist/normal level 8.0) (nth weeks t) :id [:y t])
       (if (= (inc t) (count weeks))
         drift
         (recur (inc t)
                (sample (dist/normal level drift) :id [:level (inc t)])))))))

(infer/infer (demand-unknown weeks)
             {:method :pmmh :particles 100 :iterations 2000
              :params #{:drift} :scale 3.0 :burn 200})

Particle marginal Metropolis–Hastings (PMMH) searches over the drift alone. For each proposed drift it runs SMC over the weekly levels and accepts or rejects the drift by the evidence that SMC reports. That evidence estimate is unbiased, so the chain targets the exact posterior of the drift even though every step uses an estimate.

The posterior distribution of the weekly drift: a histogram of PMMH draws over the exact density computed by quadrature. 0 2 4 6 8 10 12 weekly drift (sd of the level change) simulated with 4 PMMH draws exact posterior Twenty weeks of data narrow the drift only partly: values from below 1 to about 6 remain plausible. The histogram of PMMH draws follows the exact density, which quadrature over the Kalman filter's evidence provides for this model.

The chain’s mean drift is 2.82 against an exact 2.85, and its 90% interval runs from 0.61 to 6.09 against an exact 0.61 to 6.28. The data were simulated with a drift of 4, inside the interval but not at its centre, because twenty noisy weeks cannot pin the drift down. A forecast with one fitted drift would hide that width. The 2000 iterations took five and a half minutes on a laptop. For this linear Gaussian model, Stan or a Kalman filter inside a one-parameter chain would take seconds. PMMH is for models without a gradient to follow, models that branch, count or call a simulator.

How do I run it?

Foerster is on Clojars. With the Clojure CLI:

{:deps {org.replikativ/foerster {:mvn/version "0.1.113"}}}

A model runs inside a Spindel world, an execution context, and inference returns a spin that the REPL dereferences:

What is this syntax?
require('[org.replikativ.foerster.core :as infer :refer [sample observe factor]],
  '[org.replikativ.foerster.dist :as dist],
  '[org.replikativ.spindel.core :as sp])

def world: sp/create-execution-context()

sp/with-context world:
  @infer/infer(demand(weeks, 4.0), {:method :smc, :particles 1000})
end
(require '[org.replikativ.foerster.core :as infer :refer [sample observe factor]]
         '[org.replikativ.foerster.dist :as dist]
         '[org.replikativ.spindel.core :as sp])

(def world (sp/create-execution-context))

(sp/with-context world
  @(infer/infer (demand weeks 4.0) {:method :smc :particles 1000}))

Build the model inside with-context, as here: outside a world, the program has nowhere to run. The scripts that produced every number above are in the Foerster repository.

What can Foerster not do yet?

Foerster is beta software. Beyond the methods shown here, it implements tempered and waste-free SMC, SMC², involutive MCMC with reversible jump, parallel tempering, the No-U-Turn sampler on numerical blocks, MAP estimation, the Laplace approximation, Pathfinder, and variational inference. The methods notebook describes each one with its papers, and the gallery works through classic models against exact answers.

For a static model with continuous parameters, Stan, PyMC and NumPyro remain the mature choice. Foerster’s No-U-Turn sampler works on numerical blocks whose gradient the model supplies or Raster compiles. On one hierarchical model with 89 parameters, it matched PyMC’s warm run with a gradient compiled by Raster, and ran 1.5 times faster with a hand-written one. Its general methods over program traces are slower: single-site MH takes about twelve times as long per move as Gen, because each move replays the rest of the program, and its particle filter is about four and a half times slower than Gen’s Unfold combinator. The comparison has the details, and coming from other systems maps Stan, PyMC, Turing, Gen, Anglican and WebPPL concepts onto Foerster, including what is missing.

What comes next?

Every model so far is a pure function. Useful models often are not: a forecast that reads current stock, a simulation that writes into a scenario database, an agent that calls tools. If a thousand particles write to the same database, inference corrupts the state it reasons about, which is why Gen requires that a model change no external state.

Foerster runs each particle in a Spindel world, whose state can be copied cheaply, as the copy-on-write article describes. When SMC copies a particle, it copies the particle’s world, and when it drops a particle, it discards the world. With :world-policy :fork, every particle starts from a copy of the caller’s world, so a model can read and change the systems registered there, and nothing a particle writes reaches the caller. This costs about three times as much as running in fresh worlds. Systems that cannot be copied safely, such as live network handles, are refused before the model runs, and Markov chains run in fresh worlds only. The same mechanism makes counterfactuals direct: intervene on one choice in the world where the posterior was computed (Pearl’s do operator, as in the causal graph article) and replay the program with everything else held to what was inferred. The worlds guide covers both, and a later article shows them on a model that books orders into a forked database.

The same machinery steers processes that are not statistical models. Foerster’s steer namespace runs SMC over the steps of an agent working on a task, following twisted SMC (Zhao et al., 2024) and the SMC steering of language models (Lew et al., 2023). On short spreadsheet tasks it matched picking the best of four attempts with 17% fewer tokens, but solved nothing more. It should pay on long tasks whose steps can be checked cheaply.

For Simmis, inference over forked worlds is research rather than a product feature. It is how a governed organizational record could later answer a question such as “what happens to our cash if this order slips” with a distribution instead of a single number.

Permalink

Ep 120: Data Vision

Each week, we discuss a different topic about Clojure and functional programming.

If you have a question or topic you'd like us to discuss, tweet @clojuredesign, send an email to feedback@clojuredesign.club, or join the #clojuredesign-podcast channel on the Clojurians Slack.

This week, the topic is: "ingesting, deciding, and merging". We curate data, merge it, and represent choices.

Our discussion includes:

  • Different patterns of pure models
  • Outside data vs inside data
  • Differences and modelling tension between external and internal data
  • Why pure models tend to have patterns of functions
  • Ingested model
  • Decision-making model
  • Merging model
  • Async join
  • How pure models are combined with necessary side effects
  • Using models in procedural flows and data flows
  • Real world integration examples from Kubernetes and e-sports

Selected quotes

A pure data model is data that models something in the real world and pure functions you pass that data through. That's it.

For an ingested model, you want to curate a subset of external data, or normalize it in some way, so it's useful to the rest of your application.

Two main pieces: a representation of the information that is pertinent to your application, and a set of functions that operate on that data. Those functions tend to fall into patterns.

The ingested model is a coherent window to that outside world. It's the lens through which your application sees that external data. It serves as a stabilizing force for the rest of your application.

The Kubernetes data is wide, and it's deep. It's got a lot of information. It's only rivaled by GitHub responses in its size and scope!

The decision, itself, is data. If you want to push side effects to the edges, you need to represent the result of a decision in some way.

You put the side effects on the edges. I/O on the way in. I/O on the way out. The pure models in the middle.

The code that connects the model to a side effect is as simple as taking things out of a map and passing them as arguments into a side effect function. It only does one thing. It's almost one line of code. It's so small that it's easy to understand.

Most of your mind is focused on the ingested model and the decision-making model. You're curating and representing information, and then you're reasoning about that information.

A merging model is a pure model that takes other pure models and creates a composite, or enriched, singular output. It's multiple pieces stitched together into a larger work.

Merging models together is a separate concern from the individual models. A merging model's job is to construct a cohesive view from separate, but related information.

If it's all pure, you can test it. You can make sure that it behaves the correct way in different situations.

A way to look at at different models is by responsibilities. The outside system takes responsibility for representing the data in a way that has utility for its concerns, design constraints, and realities. The ingested model's responsibility is figuring out what matters for the application. The decision-making model's responsibility is representing choices that matter and the consequences of those choices. The merging model's responsibility is about how to connect separate things into a larger whole

So each kind of model is a separate problem space, but they're all related, and it becomes clear how they are connected. By decomposing models into separate problem spaces, you end up with models that are relatively stable because they only need to change when that aspect of the problem changes. They are decoupled and orthogonal to each other because they're in different information domains, problem domains, responsibility domains.

One of the best parts of a decoupled system is that it's easy to understand one model at a time. The theme of this series is "re-entrant coding". I only want to understand the parts of the system that I have to understand to accomplish the current task. I don't want to have to understand everything altogether. By decoupling these things with clean responsibilities and boundaries, we get a system that's easy to start working with again.

Each model is visible because it takes data and produces data. You have something that can be seen, inspected, logged, recorded, and put in a test. You avoid the kind of functional composition where information goes through a bunch of internal transformations, and you can't quite get a hold of it in your head. You have a relatively shallow call stack because each model produces data. The orthogonality of the models helps keep the call stack shallow.

A nice, related set of pure models: ingesting, making choices, and merging things together.

You can make the functional jokes. I'll stick with the dysfunctional jokes.

Permalink

BudgetSWU: my latest pile of code

I made a little web app called BudgetSWU. It recommends decklists for a trading card game called Star Wars: Unlimited. I started playing SWU almost a year and a half ago because I had a dream I was in a mall opening a pack of Yugioh cards--another trading card game that I played as a kid--and it gave me enough nostalgia to do some googlin' and discover SWU.

As an adult I now have lots more money to buy cards than I did as a kid (yay) but much less time to play with them (boo). So I don't spend much time building my own decks; I just find decklists on the internet and buy the cards for those.

BudgetSWU is now one of those places on the internet. It gets decklists from melee.gg which is used in tournaments (large and small), including each decklist's (player's) placement in the tournament. It also gets pricing data for all the cards in those decklists from tcgcsv.com, which in turn aggregates pricing data from TCGPlayer, the ebay of trading cards (it is owned by ebay, in fact).

As a final bit of data, I compute "popularity" by taking each deck's "archetype" (the leader + base, for those in the know--except that I'm actually using just base color) and seeing how many decklists in the entire corpus have that same archetype. It's cooler to play decks that not everyone else is already playing, know what I mean?

Then BudgetSWU filters all the decklists down to a handful of recommendations by computing a Pareto Front. The idea here is that we have multiple factors with which we can rank decks (cost, performance, popularity/nicheness) and we don't necessarily know which factor is most important (if deck A costs 50% more than deck B, how much better/niche does deck A have to be for it to be a better recommendation than deck B?). But we do know that if deck A beats deck B on all three of those factors, then deck A is probably the better recommendation. So we filter out all the decks that have a clearly better alternative, and then we tell the user "here's what's left; you decide." As of writing, that ends up being less than 20 decks, which is a very manageable number for a user to scroll through.

The next cool bit is that if you make an account, you can enter what cards you already own, and then the recommendations will be computed based on the cost of only the cards you don't have yet. Entering in all your cards would be a pain since players old enough to have jobs usually have thousands (of cards, not jobs). So BudgetSWU sorts all the cards by cost, most expensive first. (Right now the most expensive cards go for about $30-ish a piece). You can start from the top and enter quantities until you got to cards that cost, say, a dollar each or whatever you feel like, which doesn't take long.

Finally: whenever a new decklist makes it into your personalized recommendations (tournaments happen every week), you get a notification email. Booyah.

I generated the code in Clojure. It uses my web framework Biff. I've recently been working on some LLM skills that provide guidance for writing Biff code that doesn't suck. And I have to say, I'm fairly pleased with the results. The code still isn't anything I would print out and frame on my wall, but the overall structure is generally inline with how you're supposed to write Biff apps. If this was an app I planned to develop further (it's not), I would be happy with the state the code is in as a starting point.

If you're into specifications, you can read the one for BudgetSWU over here. i.e. that page is the thing I actually typed, mostly.

Permalink

Using AI to analyse legacy systems faster: what our pilots revealed

Futuristic in Industry 4.0 and business virtual diagram with Ai, robot assistant, Cloud, big data and automation. two colleague web designer discussing data and digital tablet docking keyboard.

AI can help organisations analyse legacy systems faster 

One of the biggest barriers to modernising legacy technology is understanding how existing systems work.   

To modernise systems, organisations first need a clear picture of how applications work, connect and depend on one another. That process can take weeks or even months. Documentation is often incomplete or out of date, knowledge may be concentrated in a small number of specialists, and some systems rely on technologies that few engineers still work with regularly. 

Our pilots indicate that AI-assisted approaches could help technical teams analyse some legacy systems more quickly. 

Across participating organisations, activities estimated to take weeks using traditional approaches were completed in hours. 

Working with organisations including the Student Loans Company (SLC), the UK Hydrographic Office (UKHO), the Met Office, and our delivery partner Kainos, we found that AI can help teams analyse unfamiliar systems, identify dependencies and build technical knowledge more quickly than traditional approaches alone. The pilots covered different systems and technical environments, so the results should not be treated as directly applicable to every legacy estate. 

However, for organisations managing large and complex technology estates, this could make some modernisation activities easier to prioritise, plan and deliver.  

What happened at the Student Loans Company 

At the Student Loans Company, teams explored how AI assisted approaches could be used to analyse an estate of 33 applications written in Clojure and support future modernisation planning. 

Using traditional approaches, the work was estimated to take around five weeks. During the pilot, teams completed the analysis in less than a day. 

The analysis gave the team a clearer view of the 33 applications and their dependencies, which it used to inform migration planning and work towards supported versions of Java. 

The most significant finding was not the reduction in effort itself, but the ability to unlock work that had previously been difficult to prioritise.  

What happened at the UK Hydrographic Office 

At the UK Hydrographic Office, teams focused on a 20-year old desktop application that previously had no automated tests. 

AI assisted techniques were used to generate automated tests for the application. Test coverage increased from 0% to 96%, while regression testing that had previously required around 15 days of manual effort was reduced to approximately 30 minutes. 

The benefits extended beyond faster testing. 

Teams reported greater confidence in making changes to the application safely and maintaining it over time. For many legacy systems, the perceived risk of change can be a barrier to improvement. The pilot indicated that AI-assisted testing could give engineers more evidence when assessing proposed changes. 

Why this matters 

The examples from SLC and UKHO illustrate a wider challenge facing organisations across the public sector. 

Legacy systems often support critical services, but analysing them can be complex and expensive. Teams may spend weeks mapping dependencies, reviewing code and building technical knowledge before they can assess risks or plan change. 

The strongest finding from the pilots was that AI can help accelerate this discovery and assessment phase. That may sound modest, but it addresses a critical point where many modernisation efforts slow down or stall. 

By helping teams understand systems more quickly, AI assisted approaches can make it easier to: 

  • assess risks 
  • identify system dependencies 
  • prioritise investment 
  • improve technical documentation 
  • develop modernisation plans 

In short, AI can help organisations spend less time understanding their systems and more time improving them. 

AI is not a substitute for good delivery 

The pilots also reinforced an important lesson. 

AI does not remove the need for good governance, engineering expertise or delivery capability. 

Successful outcomes depended on organisations having the right foundations in place, including clear ownership, access to systems and information, experienced technical teams and established delivery processes. 

In practice, the pilots showed that AI amplified existing strengths and weaknesses rather than replacing them.  

Used well, AI can accelerate analysis, improve visibility of complex systems and help organisations make progress more quickly. Used without appropriate controls, it can amplify existing problems and accelerate poor decision making. 

Where AI-assisted analysis may help 

The pilots suggest that AI can significantly reduce the effort required to assess complex systems, helping organisations move more quickly from discovery to modernisation. 

For organisations managing large and complex technology estates, this can help engineers, architects and delivery teams to identify risks earlier, making modernisation activities easier to plan, justify and deliver.  

What happens next 

We are continuing to work with organisations using AI assisted approaches to analyse, assess and modernise legacy systems, building on the lessons from these pilots. 

Alongside this work, we are developing a playbook of good practice that captures practical approaches, lessons learned and implementation considerations for organisations looking to apply AI to legacy technology challenges. The playbook is currently undergoing user testing, review and approval ahead of publication. 

Get involved 

If your organisation is exploring AI assisted approaches to analysing, documenting or modernising legacy systems, or would like to share experiences and engage with our wider Firmer Foundations programme, we would like to hear from you. Get in touch at firmer.foundations@dsit.gov.uk 

Permalink

Copyright © 2009, Planet Clojure. No rights reserved.
Planet Clojure is maintained by Baishamapayan Ghose.
Clojure and the Clojure logo are Copyright © 2008-2009, Rich Hickey.
Theme by Brajeshwar.