I have a weird relationship with statistics: on one hand, I try not to look at it too often. Maybe once or twice a year. It’s because analytics is not actionable: what difference does it make if a thousand people saw my article or ten thousand?
I mean, sure, you might try to guess people’s tastes and only write about what’s popular, but that will destroy your soul pretty quickly.
On the other hand, I feel nervous when something is not accounted for, recorded, or saved for future reference. I might not need it now, but what if ten years later I change my mind?
Seeing your readers also helps to know you are not writing into the void. So I really don’t need much, something very basic: the number of readers per day/per article, maybe, would be enough.
Final piece of the puzzle: I self-host my web projects, and I use an old-fashioned web server instead of delegating that task to Nginx.
Static sites are popular and for a good reason: they are fast, lightweight, and fulfil their function. I, on the other hand, might have an unfinished gestalt or two: I want to feel the full power of the computer when serving my web pages, to be able to do fun stuff that is beyond static pages. I need that freedom that comes with a full programming language at your disposal. I want to program my own web server (in Clojure, sorry everybody else).
Existing options
All this led me on a quest for a statistics solution that would uniquely fit my needs. Google Analytics was out: bloated, not privacy-friendly, terrible UX, Google is evil, etc.
What is going on?
Some other JS solution might’ve been possible, but still questionable: SaaS? Paid? Will they be around in 10 years? Self-host? Are their cookies GDPR-compliant? How to count RSS feeds?
Nginx has access logs, so I tried server-side statistics that feed off those (namely, Goatcounter). Easy to set up, but then I needed to create domains for them, manage accounts, monitor the process, and it wasn’t even performant enough on my server/request volume!
My solution
So I ended up building my own. You are welcome to join, if your constraints are similar to mine. This is how it looks:
It’s pretty basic, but does a few things that were important to me.
Setup
Extremely easy to set up. And I mean it as a feature.
Just add our middleware to your Ring stack and get everything automatically: collecting and reporting.
(def app
(-> routes
...
(ring.middleware.params/wrap-params)
(ring.middleware.cookies/wrap-cookies)
...
(clj-simple-stats.core/wrap-stats))) ;; <-- just add this
It’s zero setup in the best sense: nothing to configure, nothing to monitor, minimal dependency. It starts to work immediately and doesn’t ask anything from you, ever.
See, you already have your web server, why not reuse all the setup you did for it anyway?
Request types
We distinguish between request types. In my case, I am only interested in live people, so I count them separately from RSS feed requests, favicon requests, redirects, wrong URLs, and bots. Bots are particularly active these days. Gotta get that AI training data from somewhere.
RSS feeds are live people in a sense, so extra work was done to count them properly. Same reader requesting feed.xml 100 times in a day will only count as one request.
Hosted RSS readers often report user count in User-Agent, like this:
My personal respect and thank you to everybody on this list. I see you.
Graphs
Visualization is important, and so is choosing the correct graph type. This is wrong:
Continuous line suggests interpolation. It reads like between 1 visit at 5am and 11 visits at 6am there were points with 2, 3, 5, 9 visits in between. Maybe 5.5 visits even! That is not the case.
This is how a semantically correct version of that graph should look:
Some attention was also paid to having reasonable labels on axes. You won’t see something like 117, 234, 10875. We always choose round numbers appropriate to the scale: 100, 200, 500, 1K etc.
Goes without saying that all graphs have the same vertical scale and syncrhonized horizontal scroll.
Insights
We don’t offer much (as I don’t need much), but you can narrow reports down by page, query, referrer, user agent, and any date slice.
Not implemented (yet)
It would be nice to have some insights into “What was this spike caused by?”
Some basic breakdown by country would be nice. I do have IP addresses (for what they are worth), but I need a way to package GeoIP into some reasonable size (under 1 Mb, preferably; some loss of resolution is okay).
Finally, one thing I am really interested in is “Who wrote about me?” I do have referrers, only question is how to separate signal from noise.
Performance. DuckDB is a sport: it compresses data and runs column queries, so storing extra columns per row doesn’t affect query performance. Still, each dashboard hit is a query across the entire database, which at this moment (~3 years of data) sits around 600 MiB. I definitely need to look into building some pre-calculated aggregates.
When we released Datalevin 1.0.0,
the message was that one database could handle application state across
relational, graph, document, and logical workloads. With
Datalevin 1.1.0,
we can make a stronger case: that breadth comes with state-of-the-art
performance on demanding benchmarks.
The results cover durable transactions, complex relational joins, social graph
queries, nested documents, and recursive rules. Datalevin competes with SQLite,
PostgreSQL, Neo4j, MongoDB, and dedicated logic engines, using the same database
and Datalog query interface.
Here are the headline observations from the benchmark artifacts in the
Datalevin repository:
Workload
Datalevin result
Comparison in the measured configuration
Durable transactions
114,739 records/s, synchronous strict WAL, batches of 1,000
3.57× SQLite's throughput
Relational queries
All 113 JOB queries in 38.073 seconds
3.37× as fast as PostgreSQL by total query time
Graph queries
Lower latency on 20 of 21 LDBC-derived read queries
8.56× as fast as Neo4j by summed time; 5.55× by geometric mean
Document reads
11,454 operations/s, one worker, workload C with document queries
3.99× MongoDB, the next fastest system
Logical queries
Lowest latency on all ten selected OpenRuleBench-derived tasks
1.07×–18.11× as fast as the fastest alternative for each task
Each comparison applies to the configuration and workload measured. Together,
they make the case for Datalevin as a serious performance choice
across data models. The details also show where other systems remain ahead.
Durable transactions
A useful database has to accept changes quickly while maintaining its indexes
and honoring its commit promises.
The write benchmark
inserts one million person records. Each contains a UUID string identity, first
name, last name, and age. SQLite maintains its primary-key index and three
explicit value indexes; Datalevin maintains its entity-attribute-value and
attribute-value-entity indexes automatically. The comparison aligns logical
records and API transaction boundaries, although the physical index work differs.
The chart uses Datalevin's strict WAL profile and SQLite's WAL with
synchronous=FULL. Both use their ordinary OS sync behavior; the benchmark's
separate macOS fullfsync condition is outside this chart. Data generation,
transaction processing, and commit completion contribute to throughput.
For synchronous writes, SQLite is slightly faster at one record per
transaction. Datalevin pulls ahead as batches grow: 1.51× at ten records,
1.96× at 100, and 3.57× at 1,000.
The asynchronous API adds another useful capability. It combines queued
requests into physical transactions, reaching 245,485 records per second
at a request batch size of 100 while retaining strict WAL acknowledgment.
This is a different submission pattern from the blocking APIs: multiple
requests remain outstanding, so request throughput should not be read as the
number of physical commits or as single-request latency.
Concurrent callers also benefit. With four synchronous callers and
1,000-record batches, Datalevin reaches 151,622 records/s, compared with
SQLite's 50,735, a 2.99× throughput advantage. In the mixed workload,
each iteration looks up a person and upserts a complete record. The blocking
strict-WAL paths deliver 5,616 pairs/s for Datalevin and 5,041 for SQLite.
Datalevin's asynchronous path reaches 16,996 pairs/s, with reads using the
latest available snapshot and no read-your-write barrier between outstanding
requests. These results come from the retained
concurrent
and mixed
artifacts.
The practical result is that Datalevin offers both a competitive blocking
transaction path and substantial throughput when an application can batch or
pipeline its work.
Relational queries: complex joins in 38 seconds
The Join Order Benchmark
asks a harder question than point-lookup benchmarks do: can an optimizer choose
good plans for complex joins over real, correlated data?
JOB contains 113 queries over an IMDB dataset comprising 21 tables. In
Datalevin, the data becomes 277,878,411 datoms. Each engine executes a
complete warmup pass followed by a complete measurement pass. PostgreSQL and
Datalevin report planning and execution time inside the database, excluding
client startup and communication overhead.
Datalevin completes the suite in 38.073 seconds, versus 128.231 seconds
for PostgreSQL. SQLite spends 281.849 seconds on the 104 queries it
finishes, and nine more queries hit the 60-second cutoff. Charging only that
cutoff for each timeout gives SQLite a lower bound of 821.849 seconds, or
21.59× Datalevin's total. The hatched part of its bar makes those timeouts
visible.
Datalevin's advantage comes from the suite as a whole: it is faster than
PostgreSQL on 65 of 113 queries. PostgreSQL wins the other 48, including
some queries where Datalevin's planning overhead is substantial. Datalevin
spends 6.689 seconds planning, about 17.6% of its total. There is still
room to make planning cheaper while preserving the execution savings.
For applications with complex relationships, the result challenges the idea
that moving away from a relational storage model requires giving up relational
query performance.
Graph queries: a general database takes on Neo4j
The graph harness
implements all 14 Interactive Complex reads and seven Interactive Short reads
from LDBC Social Network Benchmark Interactive v1. The SF1 dataset represents
a social network; the Neo4j import contains approximately 3.65 million nodes
and 20.63 million relationships.
Both engines run embedded, eliminating network transport from this comparison.
Each gets a complete warmup pass, followed by a measurement pass in a fresh
JVM. Filesystem pages can remain warm, while query parsing and planning are
included in the measured call. Final-result caching is disabled.
In the September 1 comparison,
Datalevin takes 4.480 seconds across all 21 reads, versus 38.346 seconds
for Neo4j Community Embedded 2026.06.0. Datalevin wins 20 queries; Neo4j is
about 5% faster on IC10.
The 8.56× summed-time advantage is influenced heavily by IC14. Giving each
query equal weight through the geometric mean of its latency ratio still
favors Datalevin by 5.55×. Across the seven short reads, the summed-time
advantage is 2.95×.
Indexing policy matters here. Neo4j has ID uniqueness constraints and its
automatic token lookup indexes, with no workload-specific secondary indexes.
Datalevin automatically indexes attribute values. IC6 illustrates the
difference: its selected parameter produces an empty result, and Datalevin can
use an indexed tag-name lookup. Its 161.67× ratio describes that particular
case; it should not be generalized to every graph traversal.
This is an LDBC-derived read-latency study, not an official audited LDBC
throughput result. It retains one observation for one bundled parameter per
query, and the first query can include lazy compiler initialization in the
fresh JVM. All result counts agree; 17 queries also have identical canonical
result digests across engines. Four have documented output-representation
differences. Those details define what this strong result establishes.
Documents: indexed paths and fast application operations
Datalevin's indexed document type stores nested documents and indexes their
paths. An application can query fields, numeric ranges, wildcard paths, and
array contents while keeping documents intact.
The document benchmark
compares this feature with PostgreSQL JSONB, SQLite JSON1, and MongoDB. It uses
10,000 documents and 10,000 measured operations per pass. The base mixes are
reads and updates for A, reads for C, and read-modify-write for F. Each adds
document queries with weight 30, producing roughly 23% document queries
in the actual schedules.
All systems use explicit durable acknowledgment settings: Datalevin strict
WAL, PostgreSQL synchronous_commit=on, SQLite WAL synchronous=FULL, and
MongoDB {w: 1, j: true}. PostgreSQL, SQLite, and MongoDB receive indexes for
the query mix where supported. Measurements include client-observed execution,
transfer, and complete result-ID realization; Datalevin and SQLite are
embedded, while PostgreSQL and MongoDB use local servers.
With one worker, Datalevin leads all three mixes. Its 11,454 operations/s
on C is 3.99× MongoDB's 2,868, the best alternative. On A and F, its
advantage over runner-up PostgreSQL is approximately 1.48× and 1.50×.
The four-worker results
show a more varied picture. Datalevin reaches 32,192 operations/s on C,
2.43× PostgreSQL's throughput. PostgreSQL leads A by about 10.5% and F
by about 5.3%. Datalevin's strongest advantage here is document querying;
concurrent mutation remains an area for further improvement.
The latency breakdown shows where the query advantage comes from.
Datalevin has the lowest p50 for all five query shapes. Nested equality takes
0.057 ms; an any-depth wildcard takes 0.147 ms; array matching takes
0.207 ms. SQLite is competitive on indexed scalar paths, but the two
nested-array shapes require scanning documents in this implementation.
Automatic path indexing makes a concrete difference for applications that
store evolving, nested records and later need to ask precise questions about
their contents.
Logical workloads: recursion and derived relations
Recursive rules are central to Datalog. They express reachability, dependency
analysis, and relationships derived from other relationships in a compact
form. Their execution can also generate enormous intermediate results.
The portable OpenRuleBench-derived suite
tests transitive closure (TC), same generation (SG), and trees of joins
(Join1). It compares Datalevin with SQLite, PostgreSQL, XSB, Soufflé, Clara
Rules, and O'Doyle Rules under a query-and-full-result-materialization timing
boundary. Data loading and program compilation are outside that interval.
Datalevin has the lowest measured latency in all ten selected tasks. For
cyclic transitive closure over 50,000 input facts, it materializes one million
result rows in 102.59 ms. The fastest alternative, Soufflé, takes
1,030.36 ms, a 10.04× ratio. For Join1 b1 with both arguments free,
Datalevin takes 99.14 ms, versus XSB's 1,795 ms, an 18.11× ratio.
Other leads are much smaller. Join1 b2 is 162.41 ms in Datalevin and
174.00 ms in XSB. The observed 1.07× ratio is useful to report alongside
the large wins, especially with only one retained measurement per task.
The suite uses deterministic generated relations following the paper's task
definitions; it does not recreate the lost historical input files. These ten
tasks are a representative subset, excluding the designated Join1 a
free/free stress case and the full scale/binding matrix. Clara's out-of-memory
cell and O'Doyle's 60-second timeouts occurred during warmup. Unsupported cells
remain marked N/A. Each Clojure wrapper uses an 8 GiB maximum heap; external
engines have their own resource configuration.
This is particularly encouraging for a persistent database: expressive rules
can deliver performance competitive with specialized logic systems.
What changed in 1.1.0
The release changelog
describes improvements throughout the engine: better join-cost estimates,
selective indexed lookups, parallel scans, execution in smaller work units,
specialized transitive-closure evaluation, and faster batched writes and local
identity upserts. Together, these changes target wasted intermediate work,
allocation, and transaction overhead.
The release also makes strict durability the default when enabling WAL without
an explicit profile. Python and JavaScript gain idiomatic, composable query
and transaction APIs. Performance and usability move forward together.
These cross-system results measure the builds recorded in the artifacts. They
are not a controlled 1.0-versus-1.1 experiment, so they do not assign a numerical
speedup to an individual optimization.
The larger lesson is architectural. Relational joins, graph edges, document
paths, and logical rules all benefit when the database can find relevant facts
quickly and avoid producing unnecessary intermediate results. Datalevin's
fact-based model gives those capabilities a common foundation.
Read the numbers, then try your workload
The measurements were collected on a 12-core Apple Silicon macOS host with
Java 21.0.11; the JOB and logic studies identify the machine as an M3 Pro
MacBook Pro with 36 GB of memory. They are project-run benchmarks with specific
datasets, configurations, and timing boundaries.
The write study includes database growth in one measurement pass with no
discarded warmup. JOB, graph, and document studies retain a measurement pass
after a separate-process warmup; document runs also warm the newly built
database within each pass. Logic uses a complete warmup and measurement in the
same child JVM. These protocols produce observations, not confidence intervals,
and their different metrics should not be combined into one overall score.
For reproducibility, the charts have a downloadable
data snapshot with source-file hashes.
The repository links above contain the harnesses and retained artifacts.
The graph and logic charts use newer 1.1.0 artifacts than the older tables
still present in their benchmark READMEs.
Datalevin 1.1.0 makes a strong case that one database can combine broad
expressiveness with leading performance across demanding workloads. That
opens up a useful design choice: keep application facts together, and use
relations, graphs, documents, and logic wherever each is most natural.
Get Datalevin 1.1.0,
explore the guide, and run the benchmark closest to
your application. I would love to see what you build with it.
Writing software with AI is a really different experience than writing it by hand. Before coding agents, software was expensive to produce, in the sense that it required a lot of time from a lot of highly skilled and highly compensated people. Now, generating code is very cheap comparatively speaking, and the expensive part is deploying, operating, and maintaining it. People say AI can do this too but my experience in the industry is that it can’t, which I think is mostly why software engineers still have jobs and are actually more in demand than ever. This only makes it more important to choose wisely what software is worth producing in the first place.
The thing is, if you ask AI to build you something now, it will. Even with quite a complex request, it will give you an app or library that looks like it more or less works. The problem arises if you are trying to build software that anyone other than you will use. In these cases it is inconsiderate and embarrassing to release software that is super buggy and has obvious problems, so you want to make it more robust and reliable before releasing. You might think to yourself "well, I&aposll just get the AI to do that too". The problem is that if you ask AI to find problems in your code, it absolutely will. It will invent all kinds of crazy imaginary scenarios where something could plausibly go wrong, with no consideration given to whether those scenarios are even possible, let alone likely to happen. It will go off and write thousands of lines of code, developing "production hardening plans" and conducting "security reviews", leaving you with an impressive looking readme and much more code than you started with.
The thing is, if you check, most of this code is just duplicated, tangled, intractable slop that solves superficial or non-existent problems and obscures the actual point of the thing you were trying to build in the first place.
Anyway, my point is that in order to get your agents to write software worth actually releasing, you have to be very specific about what you&aposre trying to deliver. And the problem with that is that there usually isn&apost actually a correct answer. The nature of software delivery in this era of continuous delivery is that there never really is a target or specific point where the app or library is "done". Software is very much alive and constantly evolving. Even if you strive to deliver a stable, finished product targeting a clear definition of done, all of its dependencies will be constantly shifting underneath, forcing you to reckon with the reality that your target is moving.
Your agents will always find more plausible-looking problems to fix in your software. And if you don&apost stop them, they will just continue piling "fixes" for these into your codebase, without ever stopping to consider whether they add any value to the overall system or product. Your agents have endless suggestions for what should be improved, but no discernment about which of these are actual net improvements and no sense of whether the extra complexity they entail is worth the ongoing operational and maintenance burden.
Being a software engineer now mostly means bringing this discernment to your projects.
I could have generated random questions for this exercise but there is a nice publicly available dataset called SQuAD. This has a set of 100K+ questions which can be answered by a model. Let me pick 20 random questions from the set. The questions are such that no context is needed for answering those. These are 20 questions with actual answers and 10 made up questions. I used Opus 5 to make up some unanswerable questions as the actual dataset unanswerable questions depend on the context in the dataset which we are not using here.
Let&aposs write a simple scorer. It will only check if one of the expected answers is fully present within the LLM response. Also, if the LLM response contains "Not Known" it will match an unanswerable question.
Now armed with the function to get an answer from a LLM endpoint and a scorer, we can write a simple score-question function.
(defn- score-question
[config prompt question]
(let [system-prompt (:content prompt)
q (:question question)
a (:answers question)
actual (get-answer config system-prompt q)]
{:question q
:expected-answers a
:actual actual
:score (scorer a actual)
}))
Results from Qwen 3.0 0.6B
The following table shows results from running the evals against a Qwen 3.0 0.6B model. It is a simple model so, the results are not very impressive. Even then they are still good for such a small model - 13 answers correct out of 30, a 43.3% correctness rate. The prompt given to the model was:
Answer questions concisely. There is no need for full sentences. Say, Not Known if you do not know or are unable to infer.
And, that prompt shows up partially in some of the answers - like the question about the European population killed by the Black Death. There are some interesting hallucinations also like the Portugese city where the Rhine reaches the sea. Our scorer also shows its limitations where case mismatches cause an answer to be marked as fail. Like the 20.8% gas question. Also, the model sometimes gives correct answers albeit partial (like Newton instead of Isaac Newton)
Gemma being a larger model scores better 15/30 - around 50%. Again the limitations of our scorer show up which flags correct answers as wrong due to the case mismatch or punctuation issues. Let&aposs look at alternate scorers to fix this issue.
The F1 scorer combines values of precision and recall to generate a score of the answer. Where Precision (P) = matching tokens/predicted tokens and Recall (R) = matching tokens/expected tokens. Precision punishes padding of answers, whereas Recall punishes omission of answers.
The F1 score is defined as 2PR/(P + R)
Ideally we should normalize the generated answers to improve the precision and recall metrics but to keep things simple I will just do a lower case of the words.
With the f1 scorer we can see better matches. We have 14 perfect scores and if we include partial matches 21/30 answers are good. Again, the f1 scorer is better than the exact match scorer but still leaves a lot to be desired. Semantically similar words will still be flagged as incorrect answers. Negations of answers are ignored. This leads us into more complex scorers like semantic scoring or LLM as a judge which we will look at in the next post.
This is a cool example of the kind of problem that Clojure makes so much simpler to solve. Having a coherent model of time and not sharing mutable state makes parallel computing so much easier, because you’re not trying to coordinate multiple mutators contending for a shared resource. If you just don’t share the resource and let the language level primitives handle serializing operations for you, you can stop worrying about entire classes of bugs and just focus on your problems.
The ideas parallel servers reinvent by hand - immutable snapshot, pure read phase, changes as data - are the language&aposs defaults. If writing from many threads is difficult, then don&apost write: all game logic is pure functions (fn [world events]) that read the snapshot in parallel and return changes as deltas, merged in one place. The merge preserves order, so the parallel run is bit identical to the sequential one, and no mechanic cares how many cores it runs on.
I think it’s so cool when people experience the benefits of Clojure’s paradigm in real projects. It’s often a tough sell at first but it really is amazing how far you can get with pure functions operating on immutable data. My favourite way to fix bugs is to just make them impossible by default, which often means rethinking the way your program models the world and time. It can be a bit trippy at first but it pays off.
Software rarely gets simpler as it grows. Every new feature, integration, or business twist adds a bit more complexity. Sometimes that’s because business needs keep changing, but how you design your system plays a huge role in how simple or difficult things become later.
Functional programming tackles this complexity directly. Rather than spreading state everywhere and connecting components until it feels right, it relies on small functions that process data and produce new output. It’s like drawing clear lines on a map, so you always know where data’s flowing. This makes code much easier to read, test, and change even as the system grows.
Based on the book “Applied Higher-Order Functions,” the content below explains how functional programming principles, especially higher-order functions, reduce software complexity, simplify architecture, and make technology less confusing.
Big question: Why does software complexity grow faster than added features?
Why Does Software Complexity Grow Faster Than New Features?
It grows faster because each new feature increases complexity. Developers add new dependencies, more ways for components to interact, and more decisions to make.
Even if a feature looks simple at first, teams soon realize the trickiest part isn’t the growing codebase, it’s all those unseen connections and the unpredictable behavior that remain unnoticed.
Growing Codebases
As codebases expand, developers spend more time just understanding what’s already there. Productivity drops, errors appear, and onboarding someone new starts feeling like giving them a mysterious challenge.
What makes things truly difficult is when dependencies aren’t obvious or when multiple components can alter the same piece of data. Suddenly, a small update breaks out in a random place.
Hidden Dependencies
The system becomes unpredictable when components depend on unclear resources such as
Global variables.
Shared settings.
External services.
Objects changed elsewhere.
Even small edits can generate unexpected side effects.
Mutable Shared State
When any part of the app modifies shared state, finding where a value changed becomes challenging. And when everything’s tightly connected, developers can’t fix one thing without causing trouble in a different spot.
Debugging becomes a long process because the problem might be hiding in code a developer hardly realizes exists.
The problem is not the number of lines. The issue is that any function can change totalPrice, making it harder to know where a value came from or why it changed.
Hard to Debug
In complex systems, problems do not stay in one place. They affect other areas too.
As everything is connected, a bug in one part of the program can spread to other parts. So it would be difficult to find where the problem started.
Tight Coupling
Changing one small thing can accidentally break many other parts of the program. A small change can ripple out and force changes in places they didn’t expect.It impacts in
Less flexibility.
Harder testing.
Limited reuse.
More bugs.
Why Architecture Matters
Complexity isn’t always caused by building larger applications; it often results from how the software is structured. Good architecture
Creates clear data flow.
Decreases dependencies.
Components adapt independently, creating zero unintended side effects.
What’s the Difference Between Data-Centric and Function-Centric Architecture?
Data-centric and function-centric architectures organize software in fundamentally different ways. In a data-centric style, developers will see objects that don’t just store information they also hold the logic for changing it.
On the other hand, function-centric architecture focuses on functions that take in data, transform it, and produce something new in a clear, step-by-step process.
Traditional Data-Centric Design
So, how does data-centric design actually work out? In this approach, objects manage both their data and everything developers can do to that data. At first, it works fine, especially in smaller projects. But as apps grow, problems appear.
Here’s what often goes wrong.
Business code is scattered and harder to track.
State changes aren’t obvious—they’re hidden, which makes debugging tough.
Developers end up with strong links between modules, so changes in one place affect everything else.
The data flow becomes hard to follow.
Even simple changes start to require major effort.
If developers add more features, operations start depending on several objects interacting with each other. At this point, even experienced developers can struggle to figure out what’s really going on.
As the application grows, more responsibilities often get added to the same object.
Function-Centric Design
A function-centric architecture shifts the focus away from “who owns what” and instead asks, “What transformation do we need?” Functions take data, change it, and return something new. Developers are not worrying about hidden side effects, just about pure transformations.
Here’s what stands out:
Functions work on data—they don’t own it.
Inputs and outputs are clear from the start.
Fewer side effects.
Business logic is kept apart from data structure.
Developers can reuse the same functions without duplication.
The logic stays small and focused. This makes it easier to test and reuse.
Key Differences
Why Function-Centric Design Scales Better
Function-centric systems are easier to manage as they grow. The code is just simpler to work with—and to fix.
Teams get some real advantages:
Better code reuse—combine small functions instead of duplicating logic.
Testing is simple, since each function can stand on its own.
Behavior is predictable. Developers know what will happen. The inputs and outputs are clear.
Maintaining the code takes less work—changes are usually limited to a few functions.
Teams work together more smoothly—less tangled dependencies to manage.
This approach doesn’t mean abandoning object-oriented programming. Instead, teams get another tool for structuring big systems, one that leans on reliable, predictable functions and clear data transformations. That means as an app grows, it’s less likely to become uncontrollable.
How Does Functional Programming Reduce Cognitive Load?
It makes code easy to understand and manage. In large projects, developers have to remember many things—
Where data changes.
How different parts work together.
What affects the system, and much more.
With Functional programming, it is easy to break code into small functions. Each function does one clear job. This makes it easier for developers to understand and update the code.
So when developers look at a function, they immediately notice:
What information goes in.
What comes out.
Exactly what it is meant to do.
That means fewer surprises, fewer mistakes, and faster fixes.
When developers use functional programming, they just don’t have to work as hard to understand a huge codebase.
Why Are Pure Functions Easier to Maintain?
Pure functions always produce the same result. They avoid tampering with data beyond their own sphere, and nothing external affects them.
This means developers can:
Understand what a function does without looking at a dozen other files.
Reuse it wherever they need.
Test it with ease.
Change it without worrying they have mistakenly broken something else.
As your project gets more complicated, relying on pure functions stops the code from turning into a tangled mess.
Same input = same output. No hidden dependencies. Easy to test.
How Does Clear Data Flow Improve Software Design?
Functional programming makes data flow clear.
Developers can see what information goes into a function, what the function does, and what result comes out. Nothing important is hidden in the background.
When something breaks, it’s easier to fix since developers know precisely where the problem occurred—and they don’t risk breaking everything else in the process.
A Function Pipeline Example:
(defn validate-user [user]
(assoc user :valid true))
(defn normalize-user [user]
(update user :name clojure.string/lower-case))
(defn save-user [user]
(println "Saved:" user))
(def user {:name "JOHN"})
(-> user
validate-user
normalize-user
save-user)
Data moves like this: User → Check → Clean → Save. Anyone reading the code can easily follow what happens.
Few Hidden Side Effects
Sometimes changing one piece of code accidentally breaks something else.
Functional programming makes new results instead of changing data. So there are
Fewer bugs.
Less chance of breaking other features.
More reliable software.
Easier maintenance.
Simpler Debugging
When every function has one clear job, finding bugs becomes much easier.
Developers know:
What goes into the function.
What should come out.
This helps teams:
Find problems faster.
Write better tests.
Spend less time debugging.
Fix bugs without creating new problems.
Faster Onboarding for Developers.
With small, focused functions, new developers can start without reading all the code. They
Learn how things work.
Identify component roles.
Contribute faster.
That means:
New hires are productive faster.
Teams work better together.
Everyone is a little more confident when making changes.
Predictable Code Builds Better Software
Predictable code is much easier to work with.
Consistent functions let developers update without surprises. So teams
Keep the app smooth.
Make changes with confidence.
Find and fix bugs faster.
Grow and maintain big projects more easily.
Reducing Mental Effort Leads to Better Systems
Good software isn’t just about writing more code. Code should be easy to read and understand. Small functions make software easy. Clear flow makes testing easy. Predictable results make improving easy.
How Does Functional Programming Help Software Teams Scale?
When projects grow, developers use small functions. They avoid big and complex coding. Each function does one clear job.
Easier Code Reviews.
Smaller functions mean faster, easier code reviews. When developers check someone’s code, it’s clear:
What input they use.
What output they’ll get.
If the logic actually makes sense.
If they accidentally created an unexpected outcome.
Reviews go faster, and problems get caught before they escalate.
Smaller Functions Encourage Reuse.
Functions that perform a single operation can be used in other parts of the application. So
Functional programming keeps code parts separate, so one change doesn’t break everything else. So developers
Make changes safely.
Test each part separately.
Improve the code when needed.
Avoid future problems.
Better Collaboration of Teams.
With defined roles, developers work on components without interfering. Projects move without any interruptions. So teams have,
Less conflict between code changes.
Each part has one clear role.
Faster development.
Better teamwork.
Reducing Technical Debt.
Messy code creates debt. Functional programming prevents it with small, clean functions. So it is easy for teams to
Manage the code.
Handle large projects with less confusion.
Make easy updates.
How Do Higher-Order Functions Make Software Architecture More Flexible?
Instead of copying the same workflow over and over, developers simply provide the part that changes. The rest of the logic stays in one place.
So they don’t end up with 12 slightly different copies when they only needed one. They replace only a single part, keep everything else as-is, and keep the code clean and easy to update.
Without Higher-Order Functions
(defn process-payment [payment]
payment) ;; validate, calculate fees, save payment
(defn process-refund [refund]
refund) ;; validate, calculate fees, save refund
The same workflow gets copied multiple times. It is difficult to maintain different versions over time.
Duplicate code needs more work. Developers have to update each copy. Higher-order functions let developers write shared logic one time—they simply replace whatever behavior is different.
Developers can:
Write common parts once.
Reuse them everywhere.
Spend less time on maintenance.
Cleaner Abstractions.
Higher‑order functions create reusable blocks.
This makes software:
More modular.
Easier to extend.
Simpler to test.
Less dependent on tightly connected code.
Developers build bigger apps by reusing small code instead of rewriting.
Flexibility That Lasts
As software grows, it’s easy for the same code to get repeated. Over time, this makes the code harder to manage. Higher‑order functions let developers reuse code.
Higher‑order functions keep software flexible. Teams reuse workflows, change only parts, and add features with less code.
When Should You Use Functional Programming?
Functional programming keeps software simple when it grows. It fits best with large projects, teams with lots of developers, or any system that’s always changing.
Large Enterprise Applications
Big business apps keep growing—more features, more complexity. If nobody’s paying attention, the code soon becomes disorganized.
Functional programming assists in maintaining clarity with
small, single-purpose functions.
Clear data flow.
Reusable code.
The result? Fewer hidden tangles, fewer surprises, and it’s much safer to add new components without disrupting existing parts that already work.
Distributed Systems
Modern apps are made up of lots of services that interact continuously. Teams want each service to be easy to read and test—and avoid behaving oddly.
Functional programming makes that happen. It keeps services clear and predictable, and makes the whole system easier to handle as it grows.
APIs
APIs aren’t designed to perform complex tasks: they get a request, process some data, and send something back. Functional programming helps here, too, by making API code consistent and simple.
It’s easier to test, update, and trust—which prevents many problems as the API workload increases.
A practical example of API Request Processing:
Data Pipelines
Think about data pipelines—they’re just data passing through a bunch of steps. Functional programming makes each step simple and easy to reuse and test.
Debugging is easier, because developers can often identify exactly where the issues occur.
Financial Software
Financial apps must not make mistakes. They need all calculations to be completely reliable and simple to follow.
Functional programming makes code easier to understand. Reviewing or testing code gets a lot simpler, and accuracy is easier to secure.
AI Workflows
AI systems often go through lots of stages.
With functional programming, developers can reuse, test, and change each step without confusing the rest of the workflow. It just makes changes and improvements smoother.
Predictability Matters More as Systems Grow
As your system grows, the hard part isn’t adding new features—it’s just understanding what’s already there. That’s where functional programming is advantageous.
Developers know how each function behaves.
Data flows through clear paths.
They can test things in isolation.
There are fewer chances to break things with even small changes.
Functional programming makes software easier to understand as it grows.
Conclusion: Why Is Functional Programming Better for Complex Software Systems?
It helps teams stay coordinated and lets developers spend more time building new things—not just untangling old ones.
Sure, functional programming isn’t perfect for every single project. But if teams want software that stays simple, scalable, and easy to improve as it grows, it’s the best option.
Continue Learning
This article is built on ideas from Applied Higher Order Functions. If you want to dig deeper into how higher-order functions and function-centric design lead to cleaner, more scalable code—with real examples—this book is a great next step.
I&aposd like to thank all the sponsors and contributors who make this work possible. Without you, the projects below would not be as mature or would not exist or be maintained at all! So a sincere thank you to everyone who contributes to the sustainability of these projects.
Open the details section for more info about sponsoring.
Sponsor info
If you want to ensure that the projects I work on are sustainably maintained, you can sponsor this work in the following ways. If you work for a company that uses my OSS, please ask your employer, that would be even better. Thank you!
In the past two months it was summertime in Europe. Due to a couple of heatwaves, it was the perfect time to spend inside and enjoy my new air conditioning, while coding ;-).
The first half of July was mostly spent on improving performance and compatibility of SCI on CLJS. SCI now JIT-compiles interpreted function bodies to JavaScript at runtime, which closes a lot of the gap with compiled ClojureScript: a tight numeric loop went from ~175ms to ~7ms, over 20 times faster than the interpreter. Implementing core protocols on custom types now also works. There&aposs hardly anything you can&apost do in SCI that you can do in compiled CLJS. I released new versions of scittle and nbb that take full advantage of this.
Also in the middle of July, clj-kondo got a pretty cool enhancement. It infers types of function arguments from how they are used. E.g. when you write (defn foo [x] (inc x)) we can infer that foo is a function that takes a number. I took this principle as far as I could while preventing false positives. Of course, clj-kondo supports the latest Clojure 1.13 destructuring changes too.
In August I had the pleasure of giving a talk about Reagami at Func Prog Sweden. In the talk I gave an interactive demo of how to use Reagami in a Squint project through a REPL. I also went into detail on the algorithm that powers the fast DOM diffing. While preparing for the talk, I added SSR to Reagami too. You can view the talk on YouTube:
The last few weeks of August I created babashka.ffi, a new namespace in babashka to call C libraries. See my previous blog post: Babashka 1.13.220 gets FFI. To validate the design I wrote four libraries with it: babashka.sqlite, babashka.duckdb, babashka.postgres and filewatcher. Each one exercised a different corner of the API, along with some examples based on raylib. PacMan is particularly cool:
Right now I&aposm looking forward to giving a babashka
workshop at the Clojure/conj together with Rahul Dé. We&aposre still polishing the workshop material behind the scenes and I&aposm excited to see how it&aposs turning out. I&aposm sure it&aposll be a lot of fun and hope to catch many of you there.
In between all of this, I also worked on squint. It now supports the core protocols, so you can plug in your own collections and use them with core functions. E.g. you can use Immutable.js with Squint. I&aposm thinking about lightweight immutable persistent data structures for squint, but so far I haven&apost had much need for them, outside of Advent of Code puzzles.
The above was all about making existing projects better. But I also had a few new creative ideas:
Buzz: a cross client-server framework that lets you write web-apps on the JVM or babashka without any JS tooling, while still having full JS expressivity via Squint. I wrote tube-pod and multi-snake with it.
Cljbang.el: A Clojure-like language that runs as Emacs Lisp
Here are some highlights per project. See each project&aposs CHANGELOG.md for the full list.
Babashka: native, fast-starting Clojure interpreter for scripting.
1.13.220: Add experimental babashka.ffi: call C functions in shared libraries straight from babashka and JVM Clojure! See the guide
1.13.220: On Linux, the install script installs the dynamic binary by default. It installs the static binary on musl systems and on systems with glibc older than 2.17. The --static and --dynamic options override the automatic selection
1.13.220: :exec-args can sit directly on a task, not only under :cli, the way (exec ...) already reads it. Before, it was ignored on an :exec-fn or :cmd task
1.13.220: A task&aposs :cli spec adds to the runner-level :tasks {:cli {:spec ...}} instead of replacing it. An option from the runner level keeps its coercion and default, and --help lists it under Inherited options
1.13.220: A task with :exec-fn runs when another task :depends on it. Before, it did nothing
1.13.220: Options declared by an :exec-fn task named in :depends also parse for the CLI task that runs, with their coercion and default. --help lists them under Inherited options
1.13.220: :cmd can be a symbol naming a var that holds the command tree, like :cli. Its namespace loads on demand
1.13.220: Shell completion offers inherited options (via :depends) too
1.13.220: SCI: call site caching for instance and static methods, constructors and fields. Interop calls are up to 5x faster
1.13.219: Tasks get automatic --help and shell completions, through the new :exec-fn and :cmd keys. See the blog post! These task keys should be considered experimental and may change in a future version of babashka, depending on feedback from the community
1.13.219: Clojure 1.13 map destructuring: :keys!, :syms!, :strs!, & inside a directive, :select, :all and :defaults. Adds req! and some-vals to clojure.core
#1321: support implementing the clojure.core/Inst protocol on records, types and reify, and with extend-protocol and extend-type
#2054: a proxy of java.io.Writer supports the one-argument write and append, so binding *out* to it works
#1918: fall back to $HOME when the OS does not supply a home directory, e.g. for LDAP users in the static binary
#1994: fix :eval and :print options of clojure.main/repl being ignored in the interactive REPL (@jeroenvandijk)
Bump jline to 4.4.0: security hardening, a rewritten signal path for the FFM terminal, Kitty keyboard protocol
#2021: bump http-kit to 2.9.0-beta4, which fixes four security advisories
babashka.ffi: call C functions in shared libraries from Clojure. New library, also usable from JVM Clojure. See the guide and the examples. The API is experimental
Vectors map to arrays in both directions, maps map to JSON. Bring your own JSON library through :read-json and :write-json
clj-kondo export with a with-conn hook, CI on three operating systems
filewatcher: watch files and directories from babashka
Built on babashka.ffi: FSEvents on macOS, inotify on Linux, ReadDirectoryChangesW on Windows, and polling everywhere
The same event types on all three platforms, modeled after chokidar
A watcher keeps the process alive until close
SCI: Configurable Clojure/Script interpreter suitable for scripting
ClojureScript JIT compilation. SCI on CLJS compiles interpreted function bodies to JavaScript at runtime via js/Function. This is enabled by default and needs no configuration. When JIT is enabled, loops and numerical computations become much faster (and, in unrestricted contexts, JS interop too)
When eval is unavailable (e.g. under a Content Security Policy) SCI falls back to the interpreter. Results, error messages and error locations should be identical. And of course, it works under :advanced compilation
You can turn JIT off at runtime with js/globalThis.SCI_DISABLE_JIT = true before loading SCI, or in your Google Closure compile settings with :closure-defines {sci.core/disable-jit true}
More CLJS JIT performance improvements. Up to 20x on arithmetic-dense code for >2 arity. Keyword lookups, instance? and js globals no longer fall back to the interpreter
ClojureScript: native protocol support (#639). SCI code can implement CLJS protocols on deftype, defrecord and reify, and host code calling protocol methods on such instances dispatches into the sci implementations. Works under :advanced compilation
#1063: CLJS: deftype and defrecord fields are JS accessors on the type&aposs prototype: (.-field x) works on instances, (set! (.-field x) v) mutates deftype fields
New :unrestricted option on init and eval-string: when true, evaluated code may mutate built-in vars and CLJS instance interop skips :classes checks. The option applies only to the context it was passed to
BREAKING: enable-unrestricted-access! now throws. Use the :unrestricted option instead. The old function set a process-global flag that leaked into nested contexts
Support async functions by adding :async true in the attr map of defn
Caches resolved JVM instance methods per call site for performance
Fix babashka#2030: aset on a primitive array was reflective and 170x slower than aset-double
Errors thrown inside a loop now report a located stack frame for the loop form instead of a frame without location (all platforms, including babashka)
clj-kondo: static analyzer and linter for Clojure code that sparks joy.
Type checker: infer the type of a function param from how it is used in the body. E.g. (defn f [s] (subs s 1)) (f 42) will warn, since the evidence (subs s 1) tells us that s should be a string.
Type checker: infer the value type of a destructured map key from how it is used in the body. E.g. (defn f [{:keys [x]}] (inc x)) (f {:x "foo"}) will warn. A key whose use rejects nil and that has no :or default is required.
Type checker: a destructured binding gets the value type of its key when the map&aposs type is known, including through function return maps. E.g. (defn cfg [] {:port "8080"}) (let [{:keys [port]} (cfg)] (inc port)) will warn.
Type checker: a key missing from a map literal is provably nil, also through destructuring, keyword access chains and function return maps. E.g. (inc (:y {})) will warn.
Type checker: narrow the type of a local in the then-branch of if or the body of when when it is guarded by a known predicate. E.g. (if (string? x) (inc x) ...) will warn.
Built-in analysis now uses Clojure 1.13.0-alpha4. Param type inference over the core sources grows the arg type coverage of clojure.core from 23 to 150 vars. E.g. (interleave 1 [2]) and (mod "a" 2) will warn.
#721: NEW linter: :constant-condition: warn on a condition whose truthiness is the same on every run. On by default. Replaces :condition-always-true, whose config and ignores still apply to always-true conditions, and takes over the cond catch-all warning from :unreachable-code
Clojure 1.13 CLJ-2961: infer required keys from :keys!, :syms! and :strs! and report them at call sites
#2874: Clojure 1.13 CLJ-2964: support :select in map destructuring. The bound map&aposs keys are known to the type checker
Clojure 1.13 CLJ-2966: support :defaults in map destructuring, error when used without :or
#2943: Type checker: when an :analyze-call hook rewrites a call, clj-kondo checks the arity of the original function but not its parameter types.
#2900: :discouraged-var: new per-var :positions option (a set or vector of :call and/or :value) to limit the warning to call position or value position. A var passed to a higher-order function such as map counts as :value.
#2851: NEW linter: :seq-rest: suggest using (next x) over (seq (rest x)). Defaults to :off (@tomdl89)
#1882: built-in support for clojure.test.check.clojure-test/defspec
#2877: warn when #_ before an unmatched reader conditional discards the next form. E.g. [#_#?(:cljs 1) 2] reads as [] in :clj and will warn.
Vars defined in comment forms no longer count for :shadowed-var, :unused-private-var and :inline-def.
Performance: use a record for var usages: 13.5% less allocation, ~5-10% faster linting. More performance work by @alexander-yakushev
The minimum Clojure version to run clj-kondo on the JVM is now 1.11.
#197: :positional spec marker: positional args get their own Arguments: help section and may not be passed as options
#197: :restrict-args: error on positional args not consumed by :args->opts
#219: :cmd-aliases on a table entry or tree node gives a command one or more alternative names.
A short option that declares a non-boolean :coerce takes the rest of its token as its value, like getopt: -J-Dfoo=bar binds "-Dfoo=bar", -p80 binds 80. Flag letters may precede the valued option in a cluster: with :b a flag and :a valued, -ba x parses as -b -a x
#216: in a cluster of flags, where no letter takes a value, an interior hyphen is an error instead of silently ending option parsing.
Help: show the dispatch-level :spec options under Inherited options:. The parser always accepted these options, but help did not show them
Help: format-command-help accepts :spec, the dispatch-level spec, so a standalone call shows the same options as dispatch
dispatch: the command named on the command line wins over the :exec-args of its ancestors. A value the user typed at an ancestor level still wins over both
Add ordered :enum values for validation, help and completion
Support :doc and :epilog as a vector of lines, joined with newlines
#199: fix hang on variadic arguments that weren&apost "collected" (e.g. (repeat :k))
#203: parse-opts* resolves :spec so its :coerce/:collect entries steer parsing like in parse-opts
Completion: the fish snippet registers with --keep-order, so fish offers options in the order they are emitted, long option before its short alias, rather than sorting short options first
zsh completion: offer a command&aposs options without typing a dash first, by opting the registered program names out of zsh&aposs prefix-needed style
Thanks to @lread for continued documentation review and maintenance
Preparatory release before adding immutable + persistent collections in squint.immutable. Added a lot of protocols and made sure core functions work properly with them
Add the ILookup, IAssociative, IMap, ICounted, IKVReduce, ICollection, IEmptyableCollection and IEquiv protocols. get, assoc, contains?, find, dissoc, count, reduce-kv, conj, empty and = dispatch to them on custom types. Plain objects and arrays keep their fast paths
Add the IStack, IIndexed, IVector, IWriter and IPrintWithWriter protocols, write-all, and an ITransientVector-pop! slot; nth, peek, pop, pop!, subvec, vec, vector?, sequential?, set?, map?, seq, = and printing dispatch to custom collection types
Add equiv, hash, hash-ordered-coll, hash-unordered-coll and the IHash protocol. hash follows equiv: plain mutable objects and arrays hash by reference
Add the IMeta and IWithMeta protocols; meta and with-meta dispatch through them and the internal meta symbol property is gone
clojure.set dispatches through the collection protocols: results keep the input&aposs type, membership tests against a protocol set are value-based, and rename-keys/map-invert no longer mutate a record
Add defrecord, record? and the IRecord marker protocol. Records store their fields as own string-keyed properties and implement the map-facing protocols, so keyword lookup, keys, seq, assoc, conj and = work through the regular core functions. assoc keeps the record type, dissoc of a basis field gives a plain map, printing gives #TypeName{:a 1}
Clojure 1.13 destructuring: :keys!/:syms!/:strs! for required keys, & inside them for keys required but not bound, :select, :all, :defaults, and :or by key
Fix #975: & {:keys [...]} now destructures a map instead of the raw rest args, and a seq destructured as a map is read as kwargs
Fix #977: recur inside try no longer emits an illegal continue
Support :as-alias in ns:require like CLJS: no runtime import, only a compile-time alias so a namespaced keyword such as ::alias/x resolves
Add :require-global and :refer-global to ns, binding globals loaded via a script tag to consts without emitting an import
Add :squint/compile-time opt-in mechanism for macro/compile-time namespaces. See doc/compile-time.md
A defmacro is compile-time only: no longer emitted to the runtime module, and :refering a macro no longer emits a runtime import for it, matching CLJS
The CLI reports the file, line and column of a compile error and exits non-zero, instead of dumping the raw exception
Fix #957: vite HMR: support ^:dev/after-load + ^:dev/before-load hooks similar to shadow-cljs
.indexOf on a lazy seq now uses reference equality like a JS array, not value equality. This diverges from CLJS but keeps = out of any bundle that only builds lazy seqs, shrinking a conj bundle from 3801 to 2215 bytes
Use Symbol.for for protocol method dispatch, so pulling in multiple copies of squint.core (e.g. via http://esm.sh/) does not break protocol dispatch
Cherry: Experimental ClojureScript to ES6 module compiler
Add cherry.test with clojure.test-compatible testing API, requirable as cljs.test or clojure.test
cherry.test/report is a multimethod dispatching on [*current-reporter* type] like cljs.test, so reporting can be extended with defmethod
Add a vite plugin with browser REPL over nREPL and ^:dev/after-load / ^:dev/before-load hot-reload hooks, sharing squint&aposs implementation: import cherry from &aposcherry-cljs/vite.js&apos
Add reify, defmulti/defmethod and the vswap! macro. #&aposfoo emits foo&aposs value, like squint
Dynamic vars compile to squint&aposs box scheme, so set! and binding work across ESM modules. cljs.core dynamic vars are exported as accessor boxes proxying the real var
defprotocol:extend-via-metadata impls resolve under the fully qualified method symbol, so replicant&aposs mutation-log renderer works: replicant&aposs own test suite passes under cherry
Fix deftype implementing cljs.core protocols such as Inst, IIterable and IAtom: their marker properties were Closure-renamed in the precompiled core and missing from the emitter&aposs core protocol set. The externs list and the set are now generated from cljs.core&aposs protocols (bb gen-externs) and the build fails on drift
Fix #190: share PROTOCOL_SENTINEL with coexisting CLJS runtimes in the same JS realm
Share the macro scan and macro lookup with squint. Namespaces flagged {:squint/compile-time true} load only their compile-time part into the macro environment, like squint
CLI: --help/-h, argument validation and error messages via babashka.cli&aposs dispatch, like squint. Adds watch and nrepl-server commands, shell tab completion, and reads options from cherry.edn instead of squint.edn
Fix emitted import specifiers on Windows: backslashes are normalized via the path resolution now shared with squint
No JIT, so hot code is slower than Node.js, Bun or Deno, but the binary is small, startup is fast and memory use stays low. A Hono app serves around 30k requests per second locally, using less memory than the same app on Node.js or Bun
An install script for macOS, Linux and Windows, and dev release binaries
Clojure git and Maven deps, a module table covering url and util, @babashka/fs, and a test runner
Experimental
Buzz: write a web application with the JVM or babashka only
New project. Server state is watched and updated from client code. The UI compiles through squint and renders with Reagami, so no ClojureScript toolchain and no Node.js
Rendering is asynchronous by default and coalesces at 20ms, and a failing render is contained to its own connection
Examples: a whiteboard, a tap viewer, and a Datalevin browser with a CodeMirror query editor
Highly experimental, the API will change
tube-pod: turn YouTube videos into a private podcast
New project, written with Buzz. Add a link in the browser, tube-pod downloads the audio with yt-dlp, writes an RSS feed and serves both
Rsyncs the audio and the feed to a remote after each change, since a laptop is asleep when you want to listen
Reagami: A minimal zero-deps Reagent-like for Squint and CLJS
Add reagami.ssr to render hiccup to an HTML string on the JVM, Babashka, Squint and CLJS. See Server-side rendering
reagami.core/render (the regular render function) now hydrates a server-rendered page. It adopts the existing DOM instead of clearing the root
Add create-reagami-app. Run npm create reagami-app my-app to create a Vite project with hot reload and a browser nREPL
Breaking: :on-render now takes a map: (fn [{:keys [node lifecycle state save]}]). Call save with a value to keep it for the next call, and read it back as state. In previous versions, the hook took three arguments and its return value became the state
Move reordered nodes with moveBefore where the browser has it, so a moved subtree keeps its iframe state, animations, focus and selection (#54)
Set value, checked, selected and disabled on a tag with a hyphen as attributes, not as JS properties. A custom element observes attributes, so a property never had any effect. Native elements still handle them as properties
Custom events, e.g. :on-rated, now reach the element through addEventListener, because a browser only wires an on* property for standard events
Add web component example. A <todo-list> custom element, used from Squint, from JavaScript with and without Reagami
Fix memory leak with :on-render nodes and other :on-render improvements
cljbang: a Clojure-like language that runs as Emacs Lisp
Compiles Clojure forms to Emacs Lisp forms and evaluates them in the running Emacs. No subprocess and no transpiled text, following the same approach as squint
Namespaces with per-namespace aliases, multiple arities in fn and defn, loop/recur with a tail position check, try/throw/ex-info, case, atoms, syntax quote including nesting, &form and &env in macros, regex and set literals, #_, edn/read-string, slurp and spit
el! for calling Emacs Lisp names that are not valid Clojure symbols
ClojureScript JIT compilation. Nbb now bundles a SCI that compiles interpreted function bodies to JavaScript at runtime via js/Function. This is enabled by default. This makes loops, numerical computations and JS interop much faster
Nbb now ships babashka.fs as a built-in library. The full file system API (glob, copy, move, create-dirs, delete-tree, with-temp-dir, path helpers and more) is available via (require &apos[babashka.fs :as fs]), matching Babashka
Support implementing CLJS protocols (e.g. ILookup, etc) on deftype and defrecord
Support editscript: CLJS deftype/defrecord field interop, set! on ^:unsynchronized-mutable fields, add cljs.core type classes like PersistentHashMap, write-all and goog.math.Long
SCI now covers most CLJS capabilities, so nbb should run existing CLJS libraries unless they rely on very specific macros that require the JVM. If you have anything that does not run, please report it in #nbb!
Scittle: Execute Clojure(Script) directly from browser script tags via SCI
ClojureScript JIT compilation. Scittle now bundles a SCI that compiles interpreted function bodies to JavaScript at runtime via js/Function. This is enabled by default
Range requests: inclusive Content-Range last-pos per RFC 9110, suffix ranges (bytes=-N, previously a 500), clamping last-pos beyond EOF, reading the full range, and a test suite (@slagyr)
Reduced the core.async virtual thread memory corruption I reported upstream to a pure Java repro. GraalVM 25.0.3-ea.04 fixes it, and the pipeline test is back on now that the compile NPE is gone too
Enable the Ristretto JIT for runtime-loaded bytecode, and update the benchmarks for it
Clojure code runs without a JDK present, with the boot class loader warning suppressed
Pick up pom.xml when there is no deps.edn, and recompile Java sources when a dependency changed
New prototype. Compiles cherry expressions on the JVM and evaluates the resulting JS in an embedded GraalJS context
Two variants: a default Truffle JIT build, and a 49MB --small build without it
clj-kondo-browser: a static Clojure source browser built from clj-kondo analysis
New prototype. Renders a codebase as a static HTML page where every symbol links to its definition and usages, scope-aware, so a local is linked only within its scope
Runs clj-kondo as a pod and gets the classpath from deps.clj
grasp: Grep Clojure code using clojure.spec regexes
It’s been a while since I’ve been working on Lazuli – the editor plug-in for Pulsar (and probably VSCode in the near future) where I try to bring the Clojure REPL experience – that is, running code inside your editor and printing the result inside your editor, without having to copy-paste between two different tabs – to other languages.
If you didn’t read the previous post, the TL;DR; is: Lazuli is basically “Chlorine, but for more languages than Clojure”. Ruby was the first target, for two reasons – first, because I decided to go back to the Ruby/Rails stack; and second, because it’s probably one of the easiest languages to implement that on.
Since then, I’ve added support for Python, and I’ve been experimenting with JavaScript and Elixir. Each of them ended up being its own can of worms, and I think it’s worth writing down what I found – because some of the “obvious” solutions turn out not to work at all.
A quick refresher on watch points
I explained watch points in the previous post, but here’s a very short version: a watch point is a binding around a method or function – all the local variables, self/this/whatever the language calls it, class variables, and so on. When you evaluate something in the editor, the nREPL tries to match a watch point at that line, so if you evaluate self.some_method it knows where you are – the instance, the class variables you have defined – and it evaluates the code as if it was running inside your already-running process, be it a webserver, an app, etc.
I originally called them “watch points” precisely so I would not bind the name to a single language. The concept is the same everywhere – only the implementation changes. And that’s what this whole post is about – the different levels of pain each language gives you when trying to implement them.
Python – the easy one
At my job I started to work with Python, so I did the same thing I did for Ruby: implemented an nREPL server, and added support for parsing the code in duck-repled. Then, at the nREPL layer, I had to implement some kind of “watch point”, but using what Python offers me.
For Ruby, I used binding to define how the server captures the context. For Python, I used the frame object that the setprofile API gives you – it plays essentially the same role. Both languages support manually-defined watch points and automatically-defined ones, and they map surprisingly well to the same concept. Updating the watch points (and patch, and imports) are harder in Python because in Ruby, you can evaluate something passing binding as a parameter – you can’t do that in Python, and instead you need to convert the frame to locals and globals “dict” objects, pass those around, and then replace the watch point with the result (so that it’s not a frame anymore, but a dict containing the values). But it works, although it does feel like cheating (you are essentially faking an environment to the Python eval function) and sometimes can cause some weird issues – for example, let’s suppose I have this code:
class Example:
def m1(self, a, b):
return a + b
def m2(self, a, b):
return a - b
Supposing I have a watch point on m1, then I add an import re at the first line, before the class. That will “alias” an re inside m1, but not insidem2. Lazuli supports “patching” so I can “patch” m1 to use a new implementation, but if I do evaluate a code that adds a watch point on m2 later… it’ll gladly say that re doesn’t exist. The reason is that the “import” only exists for things that have watch points (it’s confusing, I know) because namespaces in Python don’t get profiled and don’t get their own watch points (so when you evaluate an import, it’ll only update existing watch points, not new ones that might appear later – and for the interpreter, that current namespace can’t be changed).
The printing part of Python was actually easier than Ruby. In Ruby, there is no one-size-fits-all for inspect – so many gems override how things are printed that I gave up trying to intercept the output, and instead I decided to parse the Ruby result and instance variables, properties, and everything else into a structure based on what Ruby actually gives me. Not perfect, but it works. In Python, not many people change the way outputs are presented, so the parser-based approach was way less painful (I decided to implement a structure similar to hiccup, but containing what most languages offer – so arrays and strings are supported, and nothing else. To say that it’ll print a “number” in the screen, you return ["number", "10"] for example).
So – Python: mostly a rerun of Ruby, with fewer surprises. Good.
Then I decided to try JavaScript.
JavaScript – the DevTools Protocol saves us (almost)
JavaScript is where things start getting complicated.
The whole idea of Lazuli, and of the nREPL I’m building, is that you’re running your normal code, and you just add a thin server that can somehow connect to whatever is running right now and then evaluate commands. You don’t change anything in your codebase – you add a require, import, or similar, and whatever you bring needs to be as unintrusive as possible – for example, it can’t depend on additional libraries, because otherwise you need them installed locally, and that’s not how some languages work anyway (the first version of the Ruby nREPL depended on a BEncode library, and that was a no-go – I would have to include the nREPL into the Gemfile for example).
So, what are the main problems in JavaScript? Well, you can’t just “connect into” a JS virtual machine – you need to open a server. But if you’re in a web app, you can’t start a local socket in your machine (or any server for that matter) your webpage needs to be connected to a server, and it needs to be a WebSocket. But the socket needs to be running in the server that is offering the JavaScript, so completely unacceptable (and even if it was, it would not work – it would need to “rewrite” the JS before sending to the browser, but a server can, and will, send multiple JS files – which ones is the “right one” to connect to the websocket?)
To solve that, Lazuli uses the DevTools Protocol. Both Node.js and the browsers support it – you can tell a browser that DevTools is available for your localhost machine, and, surprisingly, that just works (well, it opens a devtools channel for every page but you can just filter for the one you want and that will work). But we gain access to more things by using this protocol – one of them being the debugging API, which is actually how we add watch points to the JavaScript nREPL.
But, as always, there’s a problem. Well, actually, three of them.
Problem 1: source maps
JavaScript is not a compiled language, but most people use some kind of bundler. If you’re using React, for example, you’ll write JSX inside your JavaScript, and that will get transformed – changing lines, changing names, wrapping things.
So when you get an exception (or when you’re trying to figure out where a watch point should go), you have to parse that through a source map. And unfortunately, for the Lazuli project, we can’t escape this. There are ways to avoid minifying code – in fact, on Pulsar we actually do that – but it’s simply not how most JavaScript development works nowadays. The solution is to try to do a “reverse source map”. Essentially: if I evaluate something like a in my editor, that has to map to something that was already transpiled to the final JavaScript code, and then I evaluate that transpiled name over there. It works, but it’s flimsy and still not perfect – for example, supposing a const value = 10 gets transpiled to var a=10, then later the bundler found that this variable isn’t referenced anymore and decided to reuse const otherValue = 20 to a=20 – the “original” value, containing 10, will be lost forever, and there’s no way we can avoid this. Most bundlers don’t do that in development, luckily, but it’s still an issue that might happen.
Problem 2: functions that don’t exist
The second problem is way more complicated. And it was a surprise for me.
If you have three functions in a file, and only two of them are used, then the third one will never exist. Not “will not be called” – literally does not exist. For some reason, it seems that either the JavaScript engine garbage collects the function even if it’s in a top-level global namespace, or – more likely – it simply never actually compiles it to bytecode. For example, in the code below:
function unused(a, b) {
return a + b
}
function functionOne(a, b) {
return a / b
}
export function functionTwo(a, b) {
return functionOne(a + b, 2)
}
Now, supposing I have a watch point on functionTwo. Changing the code of functionTwo to call functionOne twice works; inspecting a and b on both functionTwo and functionOne works too. Trying to call parseInt inside functionTwo also works, but trying to call unused won’t work – ever. Even just typing unused and evaluating it will return undefined, because the JS virtual machine simply “erases” that function.
This might pose a problem for future Lazuli features – if the whole point is to be able to interactively evaluate anything, and if half of what you wrote isn’t there anymore, that’s a hole can of works. That might be possible to mitigate with the same technique as Python, though – maybe we can “fake” it by making “evaluate top block” produce a “global-ish” identifier, then update manually each watch point so that this identifier is in scope – but it’s still kind of hard to make it work.
Problem 3: namespaces and patching
And that’s not all. JavaScript is a very different language in the context of namespaces and files – there’s some magic going on around ESM that’s different from CommonJS and other stuff, and it’s close to impossible to “patch” a function. There’s a Chrome DevTools API that will try to patch things, but it only works with some very strict constraints that are not really clear what they are (and I mean very strict – for example, adding a new line won’t work).
One thing I’m thinking about is to make a Babel transformer that injects functions with their own inspect handlers, and somehow keeps a global state of these functions so that they can never be garbage collected. This might work, but I don’t yet know if it’s a good idea – but if it does, we could have Lazuli working perfectly in a JavaScript environment, with just some changes to whichever bundler you’re using (it’s not fully unintrusive as I wanted, but if it’s the only way, that might be the path).
Elixir – the interesting one
Elixir is the last language I’m working with, because I find it very interesting, and I’m checking whether it’s possible to replicate the Ruby experience there.
Some things seem easy. Some things are hard. And some things might be impossible.
Private functions are invisible
The first problem is that private functions in Elixir are not visible to my plugin. Elixir has some ways to get the bindings and the environment around a call, and context – but because I don’t know Elixir that well yet, I don’t fully understand:
what’s the reason for using one of those over the other,
why we have to use both sometimes,
and how (or if) I can capture the whole context, including private functions.
For now, seems that private functions (defined with defp) are not “compiled” into the bindings and the env. That complicates things, because I can get the “local variables” part of my watch point, but if I try to call a public function, that works… but private ones don’t. I could, theoretically, re-create the private function as a “public” one while I am evaluating the watch point just to know what is the result, but then I get into the second problem:
The VM is immutable, kinda
Another problem with Elixir is that the BEAM is immutable – kinda. You can’t just patch a function and make every caller in the runtime use the new version. That literally isn’t possible at the VM level. You can have some of this approach with GenServer and other constructs, but I don’t fully believe that this will actually patch stuff in production the way REPL-driven development expects – because your code needs to be a “GenServer” already, so it’s essentially easier to just save the file and hot-reload it.
The issue is – a REPL-driven development is meant to be a way to test solutions without needing to save the file. I can write “fragments”, evaluate them, patch functions, evaluate them, repeat, until I’m satisfied with the output – then I will save the code and allow either the hot-reload of my environment do its magic, or reload the whole file and check if I didn’t introduce any bug. With this approach, a “evaluate top block” might not even be useful for Elixir, honestly.
Bindings only exist at call time
Elixir does have the bindings of a function – but only when the function is called. So you know which local parameters are passed in, you know which global ones are used, but you don’t actually know what are the variables that you create over time inside the function.
Python, with locals and globals, have the same problem. But we can bypass that by storing the frame object, and when I evaluate the code in the editor the first time then it’s converted to the dicts. Unfortunately, this won’t work in Elixir, and I’m not sure it’ll work at all, ever – even if we had some way to capture “just before the end of the function”, we would still capture only when nothing crashed – and capturing when something crashed is probably the biggest reason for a watch point now.
Shadowing
And here’s the one that might complicate things a lot: variables are shadowed. Suppose I have this code:
module Something do
def example() do
a = 10
fun = fn -> a + 1 end
a = 20
fun2 = fn -> a + 1 end
end
end
If we define a watch point on this example function, then start to evaluate line per line, we’ll update the watch point with a=10, fun=<function>, then we’ll update the a to be 20, and then define a fun2. The issue is – fun.(), in this case, will return 11 – because that was the value of a at the time – and fun2.() will return 21 for the same reason. This is the happy path… but if I want to understand why the value of fun was 11, I can try to select its inner body – that is, a + 1 – and then evaluate it. BUT – watch points are, unfortunately, tied to a single point in a specific file and line, and they are updated when we evaluate code – meaning that evaluating that selection would return 21, confusing things.
The solution could be to add some “metadata” for each evaluation, meaning that it’ll redefine where that variable was defined, and then evaluate could capture the scopes but only consider variables defined before or at the current line. I don’t think this is easy or simple to do (considering that changing the code inside the editor also needs to update where variables were defined and also the editor will need to be “deletion and change aware” (if we rewrite a to a1, what happens?) and there might be even more trade-offs that I didn’t think about. Considering that I don’t know Elixir that well that might cause such a huge amount of bugs that this might be too difficult for now.
Final thoughts
Lazuli is growing, and the project – in my view, at least – is quite interesting. It might be bringing some of the superpowers from Clojure REPL-driven development that people love to other languages.
I’m still not comfortable with the level of tests that I have, because I really want to avoid breaking existing REPLs when I update the plug-in, and vice versa – and I also want to keep adding languages, some of which I don’t personally use at work (I don’t use Python anymore, and I only use Elixir for personal experiments).
Another thing about Lazuli is that it sometimes feels like it’s moving in the diametrically opposite direction of what people want to do nowadays. Lazuli is a project to bring the code and the developer closer together – to the point that you’re evaluating code live while you’re typing it. But we live in a world of LLMs, where people want to distance themselves from the code. I, honestly, don’t think that’s the right position to be betting on – especially when LLMs are not perfect yet, and I don’t know if they ever will be. There is still a lot of misinformation, broken promises, and hype around AI-assisted coding – some people trust too much on the ability of the AI agents. Some are even comparing source code to assembly language and to the binary that runs on your computer, which is completely absurd – compilers are (supposedly) deterministic, and LLMs are not – by design.
So maybe, if the whole LLM boom proves to be a great mistake, and people end up with millions (or even billions) of lines of code that they don’t understand, and it simply doesn’t work… maybe Lazuli can help in the future.
Or maybe I’m completely wrong and I’ll move to a different position.
But here’s the thing: I still believe that as developers, we need to understand our creation – be it written by us, or by a machine.
Have you been thinking you need another conference to go? Maybe in lovely Durham, North Carolina? We thought you might, so we made one. Later this year we’re hosting DatomicConf 2026.
Join us on Friday, December 11, 2026, in Durham, North Carolina, for a one-day, single-track conference dedicated to Datomic and the ideas and community around it. There will be news from the Datomic team, real-world Datomic use cases and opportunities to connect with other people who are building reliable, thoughtful systems.
Registration is free but limited, so if you’re planning on attending, register now.
If you’re interested in the possibility of a livestream, let us know at the livestream interest form.
All the details right now are at conf.datomic.com. We’ll soon share more information about hotels and visiting Durham. Check in on the #datomic channel at the Clojurians Slack for updates or reach out to us at conf@datomic.com with any questions.
Today babashka 1.13.220 is released, with a new babashka.ffi namespace for calling C libraries directly from Babashka scripts. The babashka.ffi library is also available as a standalone library for JVM Clojure, so you can use it in your Clojure projects as well. Note that the API is still experimental, although no changes are currently planned. It just needs more exposure and your feedback :). Here&aposs a small demo.
Calling C
This example loads libz from your system and requests the version.
To get a feeling for how to use it in larger, non-trivial projects, read the library guide. Some of the API decisions like defcfn are clearly inspired by coffi, so I want to thank Joshua Suskalo for leading the way with his excellent library. But babashka.ffi is not simply a copy of coffi. It does a few things differently. You can provide an explicit library (or a function or delay that resolves to one) to defcfn for example. Also it has a place concept (inspired by Specter&aposs paths) that efficiently lets you read from and write to structs and unions. Like coffi, babashka.ffi builds on java.lang.foreign and makes you manage memory explicitly through arenas. One benefit of this is that you&aposll get exceptions rather than segfaults that tear down your REPL and you can use with-open to release allocated memory.
Install
To use babashka.ffi and libraries that build on it, you have to use a dynamically linked version of babashka. On Mac and Windows this was always the default. On Linux, the static binary was preferred historically since it did not depend on your system&aposs libc and zlib. In this release we flip this default to a mostly-static binary: all the shared C libraries that babashka needs are statically linked, and glibc is the only dynamically linked part. The aarch64 binary, although it carries -static in its name, was already built this way. Babashka on Linux is built in a container that pins the glibc version to the lowest one possible so it should work on all mainstream LTS versions of Linux today. If you still prefer the fully static binary, you can use the install script with the --static flag. If you use a package manager or a GitHub Action to install babashka, it may not yet be up to date with this new policy. If that is the case, feel free to open an issue at the babashka GitHub repo and I&aposll reach out to get this fixed. Meanwhile you can install babashka using the installer script on GitHub to a temporary directory to get a second installation of babashka with FFI enabled:
To validate the design of babashka.ffi even more, a couple of new libraries were born. These libraries mostly resemble existing pods but now use FFI to fulfill similar use cases.
One cool thing you could not do with a pod before is defining a Clojure function in SQLite:
(require &apos[babashka.sqlite :as sq])
(sq/with-conn [db nil]
(sq/create-function! db "initials"
(fn [s] (apply str (map first (clojure.string/split s #" ")))))
(sq/query db ["select initials(?) i" "gerald jay sussman"]))
;;=> [{:i "gjs"}]
Tasks: :exec-fn composition
This release also has some really nice task improvements: :exec-fn tasks now compose through :depends. A task can depend on another CLI task, and the dependency&aposs options parse, coerce and show up in --help and shell completion:
Ask five engineers what "decoupling" means, and you will get five abstract answers about SOLID principles, hexagonal layers, microservice boundaries, or dependency inversion interfaces.
Almost nobody talks about decoupling from the point of view of the data itself.
The Fundamental Law:
If you decouple the data, the logic decouples automatically.
If you only decouple the logic while sharing mutable data, you haven't decoupled anything.
What does data actually experience as it moves through a running system? Is it continuously tethered across shared memory, or does it move across clean, discrete boundaries?
Understanding the physics of data decoupling—specifically independent memory pointers and stop-and-start boundaries—not only transforms how you structure production software, but also unlocks how we solve the two biggest bottlenecks in modern engineering: Human Snippet Tunnel Vision and AI Context Amnesia.
1. The Physics of Coupling: The Shared Pointer Trap
In a tightly coupled codebase, modules don't just depend on each other conceptually—they are physically tethered in RAM.
❌ COUPLED DATA (Continuous Live Tether / Shared Mutable Pointer):
Pointer A (package auth) ────┐
├──► [ RAM Memory Slot: 0x7FFE4A20 ]
Pointer B (package payment) ─┘ Data: { UserID: 42, Status: "Active", Balance: 100 }
* Danger: If auth.go mutates the status or alters the memory layout,
payment.go reads corrupted state or fails at runtime without warning.
When multiple packages hold pointers to the same mutable memory block:
Temporal Coupling: Package A and Package B must execute in lockstep. You cannot delay, retry, or parallelize one without coordinating locks.
Invisible Side Effects: Changes made inside auth.go propagate silently across the heap into payment.go.
The "Decoupling Illusion": Even if you wrap both packages in clean interfaces, if they are still passing shared mutable pointers underneath, they are not decoupled.
2. The Decoupled Mental Model: Stop-and-Start Boundaries
True data decoupling happens when data moves in discrete "stops and starts" across explicit boundaries.
Instead of sharing a live pointer, each system holds an independent pointer pointing to its own isolated memory allocation:
✅ DECOUPLED DATA (Independent Pointers & Stop-and-Start Handoff):
[ Stage 1: Auth Engine ]
Pointer A ──► [ Local Buffer 1: { UserID: 42, Status: "Active" } ]
│
▼ (Serialization / Value Handoff)
[ Boundary / SQLite / Queue / Channel ] <── "STOPS" (Data at rest)
▲
│ (Deserialization / Local Allocation)
[ Stage 2: Payment Engine ]
Pointer B ──► [ Local Buffer 2: { UserID: 42, Status: "Active" } ]
Why Independent Pointers Win:
Spatial Isolation:Pointer A lives only in Auth's scope; Pointer B lives only in Payment's scope. If Pointer A is mutated or garbage collected, Pointer B remains 100% intact.
Temporal Independence: The handoff boundary acts as a temporal air gap. Auth can run at 10:00:01 AM, write to the boundary, and shutdown. Payment can wake up at 10:00:05 AM and process the payload.
Reference by Identity (IDs), Not RAM Addresses: Instead of passing raw RAM addresses (0x7FFE4A20), decoupled systems pass IDs (UserID: 42 or node_id: "ast_func_402"). Each component queries or constructs what it needs.
3. Where Does the Data "Stop"? (RAM vs. Disk)
You can place your stop-and-start boundaries in two places depending on your performance and persistence needs:
As codebases scale past tens of thousands of lines, this data flow problem triggers two simultaneous breakdowns:
+-------------------------------------------------------------------------+
| YOUR ENTIRE REPOSITORY |
| [auth.go] [payment.go] [user.go] [db.go] [queue.go] |
| |
| +---------------------------------------+ |
| | YOUR IDE VIEWPORT (30-50 lines) | |
| | Editing line 42 in auth.go... | |
| +---------------------------------------+ |
| |
| * Blind to cross-package blast radius & contract mutations! * |
+-------------------------------------------------------------------------+
1. The Human Problem: Snippet Tunnel Vision (The Straw Problem)
Standard IDEs (VS Code, JetBrains) show 30 to 50 lines of code at a time. Trying to comprehend complex data flows through a 50-line viewport is like peering into a skyscraper through a drinking straw. You cannot see the blast radius of your changes.
2. The AI Problem: Context Window Amnesia (The Overflow Problem)
Autonomous AI coding agents (Claude Code, Cursor, Windsurf) struggle when developers dump 50 raw source files into the prompt window:
Token Inflation: $20+/hr in API fees burning context on boilerplate.
Mental Pointer Tracking: The LLM is forced to mentally simulate live data pointers across 50 text files, leading directly to hallucinations and broken imports.
5. The Solution: Treating Code Itself as Decoupled Relational Data
When I ran into these two friction points on large projects, I realized the answer wasn't to write another static linter or dump more raw text into an LLM prompt.
The answer was to apply data decoupling principles to the codebase itself:
The Stop-and-Start Boundary: Parse the repository's Abstract Syntax Tree (AST) and LSP symbols into a local, relational SQLite database (synapse.db), and let the parser terminate.
Independent AI Pointers via MCP: Instead of forcing an AI agent to read 50 raw text files, expose the SQLite database via a local Model Context Protocol (MCP) server. The agent runs recursive SQL queries in 2 milliseconds, retrieving exact dependency graphs without swamping its context window.
Independent Spatial Canvas: Wire the relational tables into a local 2D visual canvas (http://127.0.0.1:8080). When an engineer or AI agent refactors a module, the canvas lights up the blast radius and traces data taint flows in real time.
6. How Different Languages Decouple Data
Every major language runtime has wrestled with this problem, producing some ingenious data-decoupling mechanics:
Many developers still use JSON.parse(JSON.stringify(obj)) for deep copies, which silently strips functions, undefined, Date objects, and crashes on circular references. Modern JS includes structuredClone(), which creates a 100% isolated heap allocation and correctly clones circular graphs, Map, Set, ArrayBuffer, and Blob (though functions and DOM nodes still throw a DataCloneError). Even faster: Transferable Objects (postMessage(buffer, [buffer])) completely transfer memory ownership from the main thread to a Web Worker, instantly zeroing out the sender's pointer for zero-copy concurrency.
2. Erlang & Elixir (BEAM): The Zero-Shared-Heap Actor Model
Unlike Java, Node, or Go (where threads share a single global heap), every single Erlang/Elixir process has its own private heap and private garbage collector (with off-heap reference counting for large binaries >64 bytes). When one process sends a message to another, the BEAM VM physically copies the bytes across process heaps. There is literally no shared mutable memory in the entire VM—making deadlocks and race conditions structurally impossible.
3. Clojure: Persistent Data Structures (HAMT)
How do you update an immutable collection with 1,000,000 items without copying the entire array every time? Clojure uses Hash Array Mapped Tries (HAMT). When you "modify" an immutable map, Clojure shares 99.9% of the existing tree nodes (structural sharing) and only allocates a tiny new path of 3–4 nodes. Because of its 32-way branching factor (M=32), you get a brand-new, decoupled immutable snapshot in Olog₃₂n (effectively bounded O(1)) time with minimal memory overhead.
4. Rust: Compile-Time Move Semantics
Rust takes a different route: instead of copying memory or running a garbage collector, it enforces single ownership at compile time. When you pass data to a new function, Rust moves ownership and marks the original pointer as invalid in the compiler. If you try to read from the old pointer on the next line, the code won't even compile—giving you zero-cost pointer isolation with zero runtime overhead.
5. Go: Channels & Value Receivers
In Go, structs are value types by default (b := a copies top-level fields, though beware that inner slices, maps, or pointers still share underlying backing storage). When pairing goroutines, Go favors channels (ch <- msg) to transfer data across memory boundaries without shared locks: "Do not communicate by sharing memory; instead, share memory by communicating."
6. SQLite as the Universal Polyglot Air Gap
When you need to decouple across completely different languages (e.g. a Go compiler engine, a Python AI model, and a TypeScript web browser), in-memory pointers are impossible. Storing state in a local SQLite database acts as a universal relational boundary. It utilizes the OS page cache for sub-2ms reads, ensures ACID serialization, and lets any tool or language query the state with independent pointers and zero runtime coupling.
Conclusion: Drawing the Line in the Sand
Decoupling isn't about design patterns or complex class hierarchies—it’s about drawing a line in the sand for your data.
On one side of the line: Your memory, your pointers, and your execution scope.
On the other side of the line: Their memory, their pointers, and their execution scope.
At the line itself: Clean, stop-and-start data boundaries.
When you draw clear lines in the sand, you eliminate invisible regressions, free your systems to scale independently, and give both yourself and your AI agents the clarity to build with confidence.
I’ve been exploring these mechanics while building Go-Synapse—a local, 2D AST canvas and SQLite MCP engine. How do you handle data boundaries and pointer ownership in your own architecture? Drop your thoughts in the comments below!
<!DOCTYPE html>
<html>
<head>
<title>Tarn Barford</title>
<meta charset="utf-8"/>
<link rel="icon" type="image/x-icon" href="/favicon.ico">
<link href="/style.css" media="screen" rel="stylesheet" type="text/css" />
<link rel="alternate" type="application/atom+xml" title="Journals of Tarn Barford" href="/atom" />
<link href="/highlight.css" media="screen" rel="stylesheet" type="text/css" />
<link href="/highlight-console.css" media="screen" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="container">
<div id="header">
<div id="header">
<p>From the <a href="/journal">Journals</a> of <a href="/">Tarn Barford</a></p>
<h1>
vim-slime
</h1>
<p>
Mar 26, 2012
</p>
</div>
</div>
<div id="post_content">
<html><body><p>Today I found the awesomeness that is <a href="https://github.com/jpalardy/vim-slime">vim-slime</a>, it's been an exciting day for me.
<a href="http://common-lisp.net/project/slime/">Slime</a> is the "The Superior Lisp Interaction Mode for Emacs", I can almost hear the emacs crowd laughing.</p>
<p>For those that use vim and haven't used Slime, vim-slime or <a href="https://github.com/vim-scripts/VimClojure">something similar</a>, this is why it's awesome:</p>
<p><strong>Text can be sent from any process to the stdin of a <a href="http://www.gnu.org/software/screen/">gnu screen</a> or <a href="http://tmux.sourceforge.net/">tmux</a> session.
The process in this case is vim and the screen/tmux session is a terminal</strong>.</p>
<p>Screen is a <a href="/journal/oh-screen-where-have-you-been">really neat</a> terminal multiplexer (you can run multiple terminals in a terminal window).
The multiplexed shell processes are children of the screen process, which itself is not a child of the terminal window process.
This means a screen process and its child processes keep running if you close the terminal window.
Later you can re-connect to it, this is what makes vim-slime possible.</p>
<p>Here is an screen shot, on the left is me in gVim writing some awful Clojure <a href="#footnote-1">[1]</a>.
On the right is a screen buffer in which I started a Clojure REPL.
When I want to try run some code I can send any vim text selection to the REPL in a keystroke (or two).</p>
<p><img alt="vim slime screenshot" src="screenshot.jpg"/></p>
<p>It doesn't have to be a Clojure REPL either, we can send anything to a screen shell.
We could run git commands, find, grep, sed, etc. Like with the Clojure REPL we can even interact with any terminal programs that use STDIN.</p>
<p>This concept can be taken even further,
You can even connect to a tmux session over SSH and share a terminal or a <a href="http://remotepairprogramming.com/remote-pair-programming-with-tmux-and-vim-the">terminal program like vim to do remote pairing</a>!</p>
<p>Hopefully remote pairing is the topic of my next post as there are a couple geographically distant people I know who are keen to do some pair hacking.
I stand to learn a lot!</p>
<p><a name="footnote-1">[1]</a>
I learnt almost everything I know about Lisp from <a href="http://www.ccs.neu.edu/home/matthias/BTLS/">The Little Schemer</a>. Great book.</p></body></html>
</div>
<div id="comments">
</div>
</div>
<div id="footer">
<p> </p>
<p>Questions, comments, suggestions? Email me, <a href="mailto:tarn@tarnbarford.net">tarn@tarnbarford.net</a> (<a href="/pgp.txt">public key</a>)</p>
<p> </p>
</div>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Tarn Barford</title>
<meta charset="utf-8"/>
<link rel="icon" type="image/x-icon" href="/favicon.ico">
<link href="/style.css" media="screen" rel="stylesheet" type="text/css" />
<link rel="alternate" type="application/atom+xml" title="Journals of Tarn Barford" href="/atom" />
<link href="/highlight.css" media="screen" rel="stylesheet" type="text/css" />
<link href="/highlight-console.css" media="screen" rel="stylesheet" type="text/css" />
<style>
#swipe-canvas {
position: relative;
width: 900px;
height: 300px;
}
#swipe-results {
font-size: 30px;
padding-left: 50px;
padding-left: 50px;
}
#swipe-results ul {
margin: 0px;
padding: 0px;
}
#swipe-results li {
float: left;
background-color: #DDDDDD;
list-style-type: none;
padding: 10px;
margin: 5px;
border-radius: 5px;
}
#swipe {
position: relative;
}
#swipe-loading {
position: absolute;
height: 50px;
width: 300px;
top: 85px;
left: 300px;
background-color: darkgray;
border-radius: 10px;
text-align: center;
padding-top: 20px;
border: black;
border-width: 5px;
}
</style>
</head>
<body>
<div id="container">
<div id="header">
<div id="header">
<p>From the <a href="/journal">Journals</a> of <a href="/">Tarn Barford</a></p>
<h1>
Swipe Keyboard
</h1>
<p>
Apr 06, 2014
</p>
</div>
</div>
<div id="post_content">
<html><body><p>When I first tried a <a href="http://www.swype.com/">Swype</a> keyboard I was impressed how effective it
was. Even though I don't use the feature on my phone I was interested in how it
could be built, so I <a href="https://github.com/tarnacious/swipe-keyboard">implemented this otherwise useless swipe-able keyboard</a> below. It probably doesn't work on mobile devices, but works on modern
browsers with mouse pointers (although I've only really tried Chrome and
Firefox).</p>
<div id="swipe">
<canvas height="300px" id="swipe-canvas" width="900px"></canvas>
<div id="swipe-results"></div>
<div style="clear: both"></div>
<h2 id="swipe-loading">Loading<noscript>Javascript is Required</noscript></h2>
</div>
<p>I initially tried to solve this using the technique Peter Norvig famously uses
in his <a href="http://norvig.com/spell-correct.html]">spell checker</a>. He takes a sequence of characters and
generates a set of word candidates by adding, removing and swapping characters
in the original sequence, the generated candidates are removed if they are not
found a dictionary. This can work but to be effective too many combinations
need to be generated.</p>
<p>If the dictionary is indexed into a <a href="http://en.wikipedia.org/wiki/Trie">trie</a> the number of combinations
generated can be reduced significantly by traversing the trie and only
generating valid letter combinations. This is a pretty bare implementation of
that, it requires: </p>
<ul>
<li>The first and last characters of the initial sequence are used </li>
<li>Intermediate characters in the initial sequence can be repeated or discarded </li>
<li>No characters are added or swapped</li>
</ul>
<p>Basically, if you swipe through all the characters in a word in order, then the
word will be found if it is in the index regardless how many characters are
swiped in between. It is surprisingly quick and effective.</p>
<p>This implementation uses <a href="https://raw.github.com/first20hours/google-10000-english/master/google-10000-english.txt">these 10000 words</a>, I intended to use digital
books but never got around to it as these words demonstrate the concept well
enough.</p>
<p>This is the first thing I've written in <a href="https://github.com/clojure/clojurescript">ClojureScript</a> or
<a href="https://github.com/clojure/clojurescript">Clojure</a>, so my code my vary from non-idiomatic to shamblolic. I
initially used a <a href="http://clojuredocs.org/clojure_core/clojure.zip/zipper">zipper</a> to build the trie with immutable data
structures, but found the indexing took to long with my zipper implementation
so I <a href="https://github.com/tarnacious/swipe-keyboard/commit/6edd7b26e78121fbe8586b3f0ef54ca8277d9e32">switched to using native Javascript maps</a>.</p>
<p>I found that <a href="https://github.com/clojure/core.async">core.async</a> library is really awesome, the <a href="http://docs.closure-library.googlecode.com/git/index.html">Google
closure library</a> and <a href="https://developers.google.com/closure/compiler/">compiler</a> integration with <a href="http://leiningen.org/">Leiningen</a> the <a href="https://github.com/emezeske/lein-cljsbuild">cljsbuild plug-in</a> to be impressive. My main pains
were the slow JVM start-up time, the advanced closure compiler build of the web
worker script fails silently when run (but the main script works fine when
compiled with the advanced compiler), and at times I felt some compile time
type checking would be nice.</p>
<p>I would like to extend this experiment to index the word occurrence counts and
proceeding word counts in original text and rank the found words as most
likely. Support casing, umlauts, special characters, spelling correction and
compound words in the indexing and lookup. I think a live lookup while swiping
would also be possible.</p>
<p>Overall this was fun, turned out OK I think, and was a great learning
experience.</p></body></html>
</div>
<div id="comments">
</div>
</div>
<div id="footer">
<p> </p>
<p>Questions, comments, suggestions? Email me, <a href="mailto:tarn@tarnbarford.net">tarn@tarnbarford.net</a> (<a href="/pgp.txt">public key</a>)</p>
<p> </p>
</div>
<script src="swipe.js" type="text/javascript"></script>
</body>
</html>
What is going on?
