Inside chachaml: Core Capabilities for AI-Native Workflows in Clojure Part 2

This is Part 2 of our series, Clojure Meets Production MLOps: How chachaml Delivers AI-Native Workflows.

If you missed Part 1, Why Machine Learning Projects Fail in Production—and How chachaml Solves the MLOps Gap, we covered a common issue in machine learning. Models are easier to build than they are to operate. We also looked at why MLOps has become the real bottleneck and how chachaml helps address it.

In this article, we focus on the platform itself. You’ll see how chachaml supports AI-native workflows and the capabilities that help teams move from experiments to production.

Next, in Part 3, From Prototype to Production: The Business Case for chachaml, we’ll look at the practical side of adoption and how these capabilities can help teams deliver AI systems more efficiently.

The Architecture Behind chachaml

➜ Where chachaml Fits in Modern ML Infrastructure

Machine learning involves more than model training. Data moves through several stages before reaching production.

Chachaml steps in to handle the core details of machine learning operations, sitting right between the data pipelines and deployment. 

• Orchestrate workflow.
• Manage experiment.
• Track artifact.
• Team collaboration.

This makes it easier to track what was trained, how it was trained, and what got to production. 

➜ Integrating with Existing Clojure Infrastructure

chachaml works with existing Clojure systems, including:

• Datomic applications.
• Event-driven architectures.
• JVM services.
• Existing data pipelines.➜ Acting as the Coordination Layer
• Chachaml brings everything together in one spot. 

Teams can add MLOps without changing infrastructure. 

Core Capability #1: Pipeline Orchestration

➜ Why ML Pipelines Become Unmanageable

As the machine learning projects get bigger, teams start to notice elements being overlooked. Common problems include,

• Manual steps for execution.
• Inconsistent runs.
• Reproducibility issues.

Without a structured process, it’s hard to know what was run, when it was run, and whether the same results can be reproduced later.

➜ How chachaml Pipelines Solve This

chachaml provides a structured way to run machine learning workflows. It supports,

• Workflow execution.
• Step dependencies.
• Tracked pipeline runs.

This helps teams move from ad hoc scripts to repeatable processes. The result is, 

• Greater reliability.
• Better repeatability.
• Easier auditing and troubleshooting.

Example: Pipeline definition

(pipe/run-pipeline! "train-and-deploy"
  [{:name "preprocess" :fn (fn [_ctx] preprocessed-data)}
   {:name "train"      :fn (fn [ctx] (train (:prev-result ctx)))}
   {:name "evaluate"   :fn (fn [ctx] (eval-model (:prev-result ctx)))}
   {:name "register"   :fn (fn [ctx] (reg/register-model ...))}])

➜ New Pipelines UI in v0.7.0

Version 0.7.0 adds a dedicated Pipelines page.

Now everyone can see exactly what’s happening in the pipeline, track the status in real time, and actually get a sense of what’s running—no need to dig through endless logs or scripts. That means

• More Pipeline visibility.
• Workflow observation.
• Better teamwork with coordination.

Teams get one clear view of the ML workflow—no misunderstandings.

Core Capability #2: Experiment Tracking That Fits the REPL

➜ Why Experiment Tracking Matters

Training a model is easy. Understanding why it performed better than another model is harder.

Teams need a way to compare runs, track changes, and reproduce results. Without that, experiments quickly become difficult to manage. Experiment tracking helps with

• Model comparisons.
• Team collaboration.
• Reproducibility.

➜ Native Tracking Workflows

chachaml brings experiment tracking directly into the Clojure workflow. Teams can track,

• Runs.
• Parameters.
• Metrics.
• Notes.

Everything is stored together, making it easier to understand how a model was trained and why it performed the way it did.

➜ deftracked and Automatic Run Capture

One of chachaml’s most useful features is deftracked.

It automatically captures function executions as tracked runs. This provides,

• Function-level tracking.
• Minimal setup.
• A workflow that feels natural to Clojure developers.

Example: deftracked usage

(deftracked train-model [config data]
  (ml/log-params config)
  (let [model (fit data config)]
    (ml/log-metric :accuracy (evaluate model))
    model))

(train-model {:lr 0.01} training-data)
;; => auto-creates a run, logs params + metric, no extra wiring

Developers can add tracking without changing how they normally write code.
➜ Batch Metric Logging for Large Training Runs

When teams run big training jobs, they end up with tons of metrics. 

chachaml handles this by letting them batch metric logs, so they don’t waste time or resources. It keeps things moving more quickly and smoothly. Among the advantages are,

• Better performance.
• Improved scalability.
• Lower logging overhead.

This keeps experiment tracking practical even for larger workloads.

Core Capability #3: Model Lifecycle Management

➜ The Problem with Ad-Hoc Model Storage

Training a model is only part of the job. Teams also need to know which version is deployed, how it was trained, and where it came from. Without a structured system, problems appear like

• Version confusion.
• Deployment mistakes.
• Missing history and provenance.

As more models move through development and production, managing them becomes harder.

➜ Registry-Based Model Management

chachaml includes a model registry that helps teams manage models throughout their lifecycle. It supports,

• Model versioning.
• Promotion workflows.
• Production stages.

Teams now have a clear route from experiment to deployment. It guarantees that the correct model is put into production. 

➜ Comparing Model Versions Over Time

The registry also makes it easier to understand how models change over time. Teams can,

• Browse version history.
• Compare model versions.
• Review historical performance.

This helps teams make better deployment decisions.

Example: Model registry workflow

(reg/register-model "iris-classifier" {:artifact "model" :stage :staging})
(reg/promote! "iris-classifier" 1 :production)
(reg/load-model "iris-classifier")     ;; loads latest production version
(reg/diff-versions "iris-classifier" 1 2)

Core Capability #4: AI-Native MLOps Through MCP

➜ Why Traditional MLOps Interfaces Are Changing

Developers aren’t working the same way anymore. 

With AI coding assistants and agent-based tools now part of daily workflows, users prefer not to search through dashboards or dig into logs. They’d rather just ask a question in plain English and get an answer on the spot. 

As AI agents become more capable, operational tools need a way to work with them directly.

➜ chachaml’s MCP Server Changes the Game

One of the most distinctive features of chachaml is its built-in MCP (Model Context Protocol) server.

It exposes 16 tools through JSON-RPC and the standard MCP protocol, allowing AI agents to interact directly with machine learning data and workflows. Agents can

• Run query.
• Experiments query.
• Inspect models.
• Access the model registry.

The MCP server works with tools like

• Claude Code

Example: Connecting chachaml MCP to Claude Code

// claude_desktop_config.json
{
  "chachaml": {
    "command": "clojure",
    "args": ["-M:mcp"],
    "cwd": "/path/to/your/project"
  }
}

// Available tools: list_runs, best_run, compare_runs,
//   list_experiments, get_model, list_models, and 10 more

• Continue
• Other MCP-compatible clients

So instead of clicking around and searching manually, developers talk to the system and get what they need right away. 

➜ Example Workflow

Let’s say a developer asks, “Which experiment had the highest accuracy last month?” 

The AI agent pushes that request to chachaml through MCP. 

chachaml digs through runs, experiments, and model metadata, finds the answer, and provides it. The agent returns the answer.

No manual searching. No custom queries. No switching between tools.

➜ Why This Matters for Future AI Engineering

Most MLOps platforms were designed around dashboards and user interfaces.

chachaml adds another option: direct access through AI agents.

Machine learning runs, experiments, models, and registry data become available via natural-language requests.

As AI-assisted development becomes more common, this approach becomes increasingly useful.

chachaml isn’t just designed for machine learning workflows. It’s designed for workflows that AI agents can understand and use.

Core Capability #5: Chat With Your ML Data

➜ From Dashboards to Conversations

No more searching through multiple dashboards and run histories just to find a simple answer.

There’s a /chat interface where teams can ask for details about experiments, models, or runs—just by typing questions in simple language. 

No more searching between multiple screens. Need some info? Just ask. 

➜ Practical Examples

Questions might look like:

• Which run performed best?
• Show experiments with accuracy above 0.95.
• Compare model versions.
• Which model is currently in production?

The system finds the relevant data and returns the answer directly.

➤ Backed by Real Tool Calls, Not Hallucinations

Example: Querying ML data via chat

(chat/ask
  "Which run in experiment 'iris' has the best accuracy?"
  {:provider :anthropic
   :api-key "sk-ant-..."
   :model "claude-sonnet-4-20250514"})

;; => {:answer "Run abc123 has the highest accuracy at 0.94..."
;;     :iterations 2}

The chat interface isn’t guessing.

Behind the scenes, chachaml uses MCP tools to query experiments, runs, models, and registry data.

Every response comes straight from the database—no guesses, no made-up info, just real data. 

That matters a lot when developers need precise details to make decisions. They get a workflow that feels conversational, yet it always stays connected with what’s actually happening in their projects. 

Core Capability #6: Production Visibility Through the Web UI

➜ Why ML Teams Need More Than Logs

Logs can tell teams what happened. They don’t always tell what’s happening.

As machine learning projects get bigger, teams really need a clear view of what’s going on—whether that’s experiments, models, pipelines, or deployments. 

Without this kind of visibility, tracking bugs becomes a major issue, and important details get scattered across tools and forgotten.

Common challenges include,

• Fragmented observability.
• Difficult debugging.
• Limited visibility across workflows.

A central interface makes it easier to understand the system’s state.

➜ Inside the Eight-Page UI

chachaml includes a web UI built for day-to-day machine learning operations.

Teams can access:

• Runs dashboard
• Experiment views
• Search
• Model registry
• Compare page
• Chat page
• Pipeline page
• Artifact views

This gives developers, data scientists, and operators a shared view of machine learning activity.

➜ Visual Analytics for Production Teams

The UI includes tools for analyzing and comparing results over time. Features include

• Vega-Lite charts.
• Metric curves.
• Comparison overlays.
• CSV exports.

With one dedicated place to track progress, teams can compare their runs, spot performance trends, and figure out what went wrong—without digging through endless log files or searching through giant spreadsheets. 

All components coexist, so monitoring models and seeing how they change over time just makes a lot more sense. 

Core Capability #7: Alerts, Collaboration, and Operational Monitoring

➜ Detecting Problems Before Users Do

No machine learning model stays perfect. Data changes, people do new things, and suddenly what worked yesterday starts to fall apart.

The sooner teams catch these problems, the easier they are to fix. 

Suppose a team sets up an alert when model accuracy dips below a set threshold—this lets them jump in and solve the problem before users even notice anything is wrong. 

➜ Slack Alert Integration

With chachaml, teams can integrate automated Slack alerts via webhooks. It’s easy to set up rules like “Ping me if accuracy drops below 0.90.” 

Example: Slack alert configuration

(alerts/set-alert! "accuracy-drop"
  {:experiment  "production"
   :metric-key  :accuracy
   :op          :<
   :threshold   0.9
   :webhook-url "https://hooks.slack.com/services/..."})

(alerts/check-alerts!)
;; => Slack message fired when accuracy drops below 0.90

That way, 

• Teams can jump on problems faster.
• Keep track of model health.
• Improve their oversight. 

Teams don’t have to wait until something goes wrong after deployment—now they will know right away when there’s a problem. 

➜ User Attribution and Team Accountability

When the projects start getting bigger, it’s really helpful to see who did what—who ran an experiment, trained a model, or pushed out a deployment. 

chachaml comes with tools like 

• CHACHA_USER

Example: HTTP Write API (curl / Python / Go)

# Log a run from any language via HTTP
RUN_ID=$(curl -s -X POST http://localhost:8080/api/w/runs \
  -H 'Content-Type: application/json' \
  -d '{"experiment":"iris","name":"from-python"}' | jq -r .id)

curl -X POST http://localhost:8080/api/w/runs/$RUN_ID/metrics \
  -H 'Content-Type: application/json' \
  -d '{"accuracy":0.94}'

• My Runs views
• Ownership tracking

Teams get a full record across runs, experiments, and models, so everyone can tell who’s responsible for what and when it happened. 

Core Capability #8: Open Integration Across Languages and Teams

➜ Not Everyone Needs to Be Writing Clojure

Most engineering teams use more than one language. Even if the company leans heavily on Clojure, it will probably still need Python or Go for machine learning work. 

That’s just how things go. chachaml gets it; it’s flexible enough to fit into whatever environment teams are already using. 

➜ HTTP Write APIs

Chachaml comes with HTTP write APIs so outside apps can push their data straight into the platform. 

Teams can log runs, metrics, parameters, and artifacts from:

• Python.
• Go.
• curl.
• Any tool that can make HTTP requests.

It’s super easy for teams to plug in without disrupting their current workflows. 

Chachaml serves as a central hub for tracking and managing all machine learning tasks in one place, rather than replacing preferred tools. 

➜ sklearn Interoperability

Many machine learning teams already use Python libraries such as scikit-learn.

Through libpython-clj2, chachaml can work with those libraries while keeping tracking and management inside the Clojure ecosystem.

It includes helpers such as:

• tracked-fit!

Example: sklearn interoperability via libpython-clj2

(sk/train-and-evaluate!
  (lm/LogisticRegression :max_iter 200)
  X-train y-train X-test y-test
  :register-as "iris-classifier"
  :stage       :staging)

;; => trains in Python sklearn, logs metrics + registers model in chachaml

• tracked-predict
• evaluate!

Teams can train and evaluate models using familiar Python tools while still recording experiments and results in chachaml.

➜ Shared Artifact Storage with S3

Machine learning projects generate many artifacts, including models, datasets, reports, and evaluation results.

chachaml supports S3-compatible storage, including

• MinIO.
• DigitalOcean Spaces.
• AWS S3.

Teams can keep all their artifacts in one place and share them, no matter what environment they’re working in. 

It’s just simpler that way—everyone stays aligned, and moving work from development to staging to production doesn’t mean teams have to rethink how they handle their artifacts. 

Storage, Scaling, and Production Readiness

➜ From SQLite to PostgreSQL

Most teams get started with a simple setup. With chachaml, teams can use SQLite for small projects or local development. Thanks to SQLite’s WAL mode, teams achieve better concurrency without complicating deployment. 

When the team grows and the demands increase, switching to PostgreSQL is easy. Teams don’t have to change how they work — the API and workflows stay exactly the same. And when teams need to scale up, chachaml collaborates with HikariCP for connection pooling. 

So it handles bigger workloads and more users without difficulty. 

➜ Dockerized Deployment

Getting started doesn’t take much effort, either. chachaml offers Docker-based deployment options, so starting the app, database, and web interface simultaneously is pretty simple. 

If a team wants to move from a local setup to something everyone can access, there’s no need to rebuild its infrastructure—everything just works. 

The post Inside chachaml: Core Capabilities for AI-Native Workflows in Clojure Part 2 appeared first on Flexiana.

Permalink

I am sorry, but everyone is getting syntax highlighting wrong

Translations: Russian

Syntax highlighting is a tool. It can help you read code faster. Find things quicker. Orient yourself in a large file.

Like any tool, it can be used correctly or incorrectly. Let’s see how to use syntax highlighting to help you work.

Christmas Lights Diarrhea

Most color themes have a unique bright color for literally everything: one for variables, another for language keywords, constants, punctuation, functions, classes, calls, comments, etc.

Sometimes it gets so bad one can’t see the base text color: everything is highlighted. What’s the base text color here?

The problem with that is, if everything is highlighted, nothing stands out. Your eye adapts and considers it a new norm: everything is bright and shiny, and instead of getting separated, it all blends together.

Here’s a quick test. Try to find the function definition here:

and here:

See what I mean?

So yeah, unfortunately, you can’t just highlight everything. You have to make decisions: what is more important, what is less. What should stand out, what shouldn’t.

Highlighting everything is like assigning “top priority” to every task in Linear. It only works if most of the tasks have lesser priorities.

If everything is highlighted, nothing is highlighted.

Enough colors to remember

There are two main use-cases you want your color theme to address:

1. Look at something and tell what it is by its color (you can tell by reading text, yes, but why do you need syntax highlighting then?)
2. Search for something. You want to know what to look for (which color).

1 is a direct index lookup: color → type of thing.

2 is a reverse lookup: type of thing → color.

Truth is, most people don’t do these lookups at all. They might think they do, but in reality, they don’t.

Let me illustrate. Before:

After:

Can you see it? I misspelled return for retunr and its color switched from red to purple.

I can’t.

Here’s another test. Close your eyes (not yet! Finish this sentence first) and try to remember what color your color theme uses for class names?

Can you?

If the answer for both questions is “no”, then your color theme is not functional. It might give you comfort (as in—I feel safe. If it’s highlighted, it’s probably code) but you can’t use it as a tool. It doesn’t help you.

What’s the solution? Have an absolute minimum of colors. So little that they all fit in your head at once. For example, my color theme, Alabaster, only uses four:

• Green for strings
• Purple for constants
• Yellow for comments
• Light blue for top-level definitions

That’s it! And I was able to type it all from memory, too. This minimalism allows me to actually do lookups: if I’m looking for a string, I know it will be green. If I’m looking at something yellow, I know it’s a comment.

Limit the number of different colors to what you can remember.

If you swap green and purple in my editor, it’ll be a catastrophe. If somebody swapped colors in yours, would you even notice?

What should you highlight?

Something there isn’t a lot of. Remember—we want highlights to stand out. That’s why I don’t highlight variables or function calls—they are everywhere, your code is probably 75% variable names and function calls.

I do highlight constants (numbers, strings). These are usually used more sparingly and often are reference points—a lot of logic paths start from constants.

Top-level definitions are another good idea. They give you an idea of a structure quickly.

Punctuation: it helps to separate names from syntax a little bit, and you care about names first, especially when quickly scanning code.

Please, please don’t highlight language keywords. class, function, if, elsestuff like this. You rarely look for them: “where’s that if” is a valid question, but you will be looking not at the if the keyword, but at the condition after it. The condition is the important, distinguishing part. The keyword is not.

Highlight names and constants. Grey out punctuation. Don’t highlight language keywords.

Comments are important

The tradition of using grey for comments comes from the times when people were paid by line. If you have something like

of course you would want to grey it out! This is bullshit text that doesn’t add anything and was written to be ignored.

But for good comments, the situation is opposite. Good comments ADD to the code. They explain something that couldn’t be expressed directly. They are important.

So here’s another controversial idea:

Comments should be highlighted, not hidden away.

Use bold colors, draw attention to them. Don’t shy away. If somebody took the time to tell you something, then you want to read it.

Two types of comments

Another secret nobody is talking about is that there are two types of comments:

1. Explanations
2. Disabled code

Most languages don’t distinguish between those, so there’s not much you can do syntax-wise. Sometimes there’s a convention (e.g. -- vs /* */ in SQL), then use it!

Here’s a real example from Clojure codebase that makes perfect use of two types of comments:

Disabled code is gray, explanation is bright yellow

Light or dark?

Per statistics, 70% of developers prefer dark themes. Being in the other 30%, that question always puzzled me. Why?

And I think I have an answer. Here’s a typical dark theme:

and here’s a light one:

On the latter one, colors are way less vibrant. Here, I picked them out for you:

Notice how many colors there are. No one can remember that many.

This is because dark colors are in general less distinguishable and more muddy. Look at Hue scale as we move brightness down:

Basically, in the dark part of the spectrum, you just get fewer colors to play with. There’s no “dark yellow” or good-looking “dark teal”.

Nothing can be done here. There are no magic colors hiding somewhere that have both good contrast on a white background and look good at the same time. By choosing a light theme, you are dooming yourself to a very limited, bad-looking, barely distinguishable set of dark colors.

So it makes sense. Dark themes do look better. Or rather: light ones can’t look good. Science ¯\_(ツ)_/¯

But!

But.

There is one trick you can do, that I don’t see a lot of. Use background colors! Compare:

The first one has nice colors, but the contrast is too low: letters become hard to read.

The second one has good contrast, but you can barely see colors.

The last one has both: high contrast and clean, vibrant colors. Lighter colors are readable even on a white background since they fill a lot more area. Text is the same brightness as in the second example, yet it gives the impression of clearer color. It’s all upside, really.

UI designers know about this trick for a while, but I rarely see it applied in code editors:

If your editor supports choosing background color, give it a try. It might open light themes for you.

Bold and italics

Don’t use. This goes into the same category as too many colors. It’s just another way to highlight something, and you don’t need too many, because you can’t highlight everything.

In theory, you might try to replace colors with typography. Would that work? I don’t know. I haven’t seen any examples.

Using italics and bold instead of colors

Myth of number-based perfection

Some themes pay too much attention to be scientifically uniform. Like, all colors have the same exact lightness, and hues are distributed evenly on a circle.

This could be nice (to know if you have OCD), but in practice, it doesn’t work as well as it sounds:

OkLab l=0.7473 c=0.1253 h=0, 45, 90, 135, 180, 225, 270, 315

The idea of highlighting is to make things stand out. If you make all colors the same lightness and chroma, they will look very similar to each other, and it’ll be hard to tell them apart.

Our eyes are way more sensitive to differences in lightness than in color, and we should use it, not try to negate it.

Let’s design a color theme together

Let’s apply these principles step by step and see where it leads us. We start with the theme from the start of this post:

First, let’s remove highlighting from language keywords and re-introduce base text color:

Next, we remove color from variable usage:

and from function/method invocation:

The thinking is that your code is mostly references to variables and method invocation. If we highlight those, we’ll have to highlight more than 75% of your code.

Notice that we’ve kept variable declarations. These are not as ubiquitous and help you quickly answer a common question: where does thing thing come from?

Next, let’s tone down punctuation:

I prefer to dim it a little bit because it helps names stand out more. Names alone can give you the general idea of what’s going on, and the exact configuration of brackets is rarely equally important.

But you might roll with base color punctuation, too:

Okay, getting close. Let’s highlight comments:

We don’t use red here because you usually need it for squiggly lines and errors.

This is still one color too many, so I unify numbers and strings to both use green:

Finally, let’s rotate colors a bit. We want to respect nesting logic, so function declarations should be brighter (yellow) than variable declarations (blue).

Compare with what we started:

In my opinion, we got a much more workable color theme: it’s easier on the eyes and helps you find stuff faster.

Shameless plug time

I’ve been applying these principles for about 8 years now.

I call this theme Alabaster and I’ve built it a couple of times for the editors I used:

• VS Code
• JetBrains IDEs
• Sublime Text (twice)

It’s also been ported to many other editors and terminals; the most complete list is probably here. If your editor is not on the list, try searching for it by name—it might be built-in already! I always wondered where these color themes come from, and now I became an author of one (and I still don’t know).

Feel free to use Alabaster as is or build your own theme using the principles outlined in the article—either is fine by me.

As for the principles themselves, they worked out fantastically for me. I’ve never wanted to go back, and just one look at any “traditional” color theme gives me a scare now.

I suspect that the only reason we don’t see more restrained color themes is that people never really thought about it. Well, this is your wake-up call. I hope this will inspire people to use color more deliberately and to change the default way we build and use color themes.

Permalink

Statistics made simple

I have a weird relationship with statistics: on one hand, I try not to look at it too often. Maybe once or twice a year. It’s because analytics is not actionable: what difference does it make if a thousand people saw my article or ten thousand?

I mean, sure, you might try to guess people’s tastes and only write about what’s popular, but that will destroy your soul pretty quickly.

On the other hand, I feel nervous when something is not accounted for, recorded, or saved for future reference. I might not need it now, but what if ten years later I change my mind?

Seeing your readers also helps to know you are not writing into the void. So I really don’t need much, something very basic: the number of readers per day/per article, maybe, would be enough.

Final piece of the puzzle: I self-host my web projects, and I use an old-fashioned web server instead of delegating that task to Nginx.

Static sites are popular and for a good reason: they are fast, lightweight, and fulfil their function. I, on the other hand, might have an unfinished gestalt or two: I want to feel the full power of the computer when serving my web pages, to be able to do fun stuff that is beyond static pages. I need that freedom that comes with a full programming language at your disposal. I want to program my own web server (in Clojure, sorry everybody else).

Existing options

All this led me on a quest for a statistics solution that would uniquely fit my needs. Google Analytics was out: bloated, not privacy-friendly, terrible UX, Google is evil, etc.

What is going on?

Some other JS solution might’ve been possible, but still questionable: SaaS? Paid? Will they be around in 10 years? Self-host? Are their cookies GDPR-compliant? How to count RSS feeds?

Nginx has access logs, so I tried server-side statistics that feed off those (namely, Goatcounter). Easy to set up, but then I needed to create domains for them, manage accounts, monitor the process, and it wasn’t even performant enough on my server/request volume!

My solution

So I ended up building my own. You are welcome to join, if your constraints are similar to mine. This is how it looks:

It’s pretty basic, but does a few things that were important to me.

Setup

Extremely easy to set up. And I mean it as a feature.

Just add our middleware to your Ring stack and get everything automatically: collecting and reporting.

(def app
  (-> routes
    ...
    (ring.middleware.params/wrap-params)
    (ring.middleware.cookies/wrap-cookies)
    ...
    (clj-simple-stats.core/wrap-stats))) ;; <-- just add this

It’s zero setup in the best sense: nothing to configure, nothing to monitor, minimal dependency. It starts to work immediately and doesn’t ask anything from you, ever.

See, you already have your web server, why not reuse all the setup you did for it anyway?

Request types

We distinguish between request types. In my case, I am only interested in live people, so I count them separately from RSS feed requests, favicon requests, redirects, wrong URLs, and bots. Bots are particularly active these days. Gotta get that AI training data from somewhere.

RSS feeds are live people in a sense, so extra work was done to count them properly. Same reader requesting feed.xml 100 times in a day will only count as one request.

Hosted RSS readers often report user count in User-Agent, like this:

Feedly/1.0 (+http://www.feedly.com/fetcher.html; 457 subscribers; like FeedFetcher-Google)

Mozilla/5.0 (compatible; BazQux/2.4; +https://bazqux.com/fetcher; 6 subscribers)

Feedbin feed-id:1373711 - 142 subscribers

My personal respect and thank you to everybody on this list. I see you.

Graphs

Visualization is important, and so is choosing the correct graph type. This is wrong:

Continuous line suggests interpolation. It reads like between 1 visit at 5am and 11 visits at 6am there were points with 2, 3, 5, 9 visits in between. Maybe 5.5 visits even! That is not the case.

This is how a semantically correct version of that graph should look:

Some attention was also paid to having reasonable labels on axes. You won’t see something like 117, 234, 10875. We always choose round numbers appropriate to the scale: 100, 200, 500, 1K etc.

Goes without saying that all graphs have the same vertical scale and syncrhonized horizontal scroll.

Insights

We don’t offer much (as I don’t need much), but you can narrow reports down by page, query, referrer, user agent, and any date slice.

Not implemented (yet)

It would be nice to have some insights into “What was this spike caused by?”

Some basic breakdown by country would be nice. I do have IP addresses (for what they are worth), but I need a way to package GeoIP into some reasonable size (under 1 Mb, preferably; some loss of resolution is okay).

Finally, one thing I am really interested in is “Who wrote about me?” I do have referrers, only question is how to separate signal from noise.

Performance. DuckDB is a sport: it compresses data and runs column queries, so storing extra columns per row doesn’t affect query performance. Still, each dashboard hit is a query across the entire database, which at this moment (~3 years of data) sits around 600 MiB. I definitely need to look into building some pre-calculated aggregates.

One day.

How to get

Head to github.com/tonsky/clj-simple-stats and follow the instructions:

Let me know what you think! Is it usable to you? What could be improved?

Permalink

The Rest of the Story: June Edition - JVM Weekly vol. 181

Before we get into the technical weeds: is it this hot where you are too? I’m writing this from London, where the thermometer is closing in on 34 degrees, the sky is perfectly cloudless, and the islanders are reacting as if the apocalypse had begun (air conditioning here remains a largely theoretical concept - in the Kensington Event Center it was hot like in the the all-inclusive Hotel in Egypt).

But enough about the weather, because June in the JVM world ran just as hot.

This time the month split neatly into two halves. On one side, the JDK did that slow, unglamorous work that keeps Java in the game: a native Argon2 moved closer to the standard library, Babylon started lowering Java straight onto Tensor Cores, and the Vector API went through an honest, public reckoning with its own limitations. On the other side, the tooling layer reorganized itself around agents: JetBrains and Microsoft, in the same week, bet on almost perfectly opposite strategies, Spring AI reached 2.0 GA, and the enterprise MCP story I keep saying someone should finally write up keeps coming together. Let’s get started!

1. June: The Rest of the Story

I’ll start with a topic that, if you want to understand where AI-assisted programming is heading, says more than any chat-window demo: in that same stretch of June, JetBrains and Microsoft bet on almost perfectly opposite strategies for the agent harness, the layer that actually runs long, multi-step agent sessions.

JetBrains open-sourced Mellum2, a 12-billion-parameter coding model that activates only 2.5 billion parameters per token thanks to a Mixture-of-Experts architecture, routing each token through a subset of 64 experts. JetBrains’ Nikita Pavlichenko and Anton Semenkin call it a “focal model”: fast, specialized, and deliberately not racing the frontier models on breadth. The benchmarks back up that niche: on a single H100 it matches Qwen2.5-7B almost exactly in single-request mode (192 vs 193 tokens/s), but under concurrent load (and only that kind counts in production) it beats Qwen2.5-7B by 21% and Qwen3-8B by 79%, while its “thinking” variant hits 78.4% on EvalPlus.

A small model only needs one H100…

The technical report is honest about the cost: once you shift evaluation toward broad reasoning (GPQA Diamond, MMLU-Redux), the larger models take the lead again. Mellum2, though, runs on infrastructure you control. Claude Code and Codex run locally but route inference through the Anthropic and OpenAI APIs; Cursor’s work on Composer stays tied to Cursor’s platform (now with an xAI layer outside your walls). Mellum2 is a bet that ownership and on-prem will matter as AI moves deeper into the SDLC.

BTW: last Thursday I was at AI Tinkerers in Warsaw, where Damian Bogunowicz was talking about exactly this, Mellum2.

And then, as if for contrast, Microsoft went for full consolidation. Ji Dong, Senior PM on GitHub Copilot for JetBrains, announced that Copilot CLI is becoming the default agent harness in GitHub Copilot for JetBrains, with the IDE’s own local harness slated for deprecation. The reasoning is pure platform economics: maintaining a separate harness for JetBrains meant features and models landed there later than on Copilot’s other surfaces, so folding everything into Copilot CLI buys faster parity and (they claim) higher-quality results. CLI sessions run independently in the background while the IDE starts, monitors, and steers them, which incidentally makes parallel agent runs the default, and existing local sessions convert automatically.

Put them side by side and you have the whole tension of this moment. JetBrains: own your model, own your harness, run it yourself. Microsoft: one harness to rule them all, ship faster by not maintaining variants. Both approaches are reasonable and, interestingly, both can be the future at the same time

Remember how last month I joked that someone should write a roundup of the coalescing agentic JVM ecosystem? It keeps coalescing. And meanwhile Markus Eisele added an argument complementary to March’s MCP feature in Open Liberty: instead of rewriting a Jakarta EE 10/11 monolith into microservices, turn it into an MCP server, using the Adapter pattern to draw a hard trust boundary so agents get capability-scoped tools rather than raw access to the database and filesystem. He follows that with the Java Agent Skills Kit proposal, which lands squarely on the SKILL.md / SkillsJars thread we’ve been tracking: stop encoding agent behavior as one big, brittle prompt string, and start treating capabilities as separate, versionable, unit-testable components.

Bringing SOLID to agentic workflows is a sentence that’s either the future or a consultant’s slide. This month I’m leaning toward the former.

(It’s getting crowded - I really do need to finally write that roundup article, don’t I? 😉)

Sonatype is tightening the screws on OSS publishing to Maven Central. For now softly (the UI shows how far over the limit you are), and from August “for real.”

Brian Fox (CTO of Sonatype, and at one time chair of Apache Maven) laid it out on the company blog. The schedule is two-stage: from June 16 a “soft” phase with notifications only is underway, and from August 11, 2026 hard enforcement kicks in, meaning publishing is paused until usage comes down, an exception is approved, or a paid option is purchased. Three monthly metrics per organization are counted (file count, size, and number of releases), and as three-month averages at that, so a one-off spike or an urgent CVE fix won’t get you booted on their own.

Where this comes from: over the last 90 days, 10% of namespaces accounted for more than 88% of published files and more than 90% of the space taken by new releases, so the target is commercial-scale patterns (huge artifacts, very frequent releases, Central as the last-mile for SDKs), not ordinary OSS. For those over the threshold there are three routes: cut unnecessary publishing (Fox writes plainly that this is not the place for every CI build), request an exception for an unusual OSS project, or move to paid Maven Central Publisher Pro.

If you maintain anything with a regular release cycle, these two months until August are a good moment to look into the Usage Center and check which side of the threshold you’re sitting on.

That one of them happens to be my employer’s namespace, I can confirm firsthand: the problem isn’t purely theoretical. Both org.virtuslab and com.softwaremill, are already over the threshold, so we are observing development closely.

Nice shut out opportunity - VirtusLab is a team behind Scala 😉

On the security front, which always gets lost in the JEP noise, native Argon2 is becoming a Preview feature. JEP 8377081 moved from Draft to Submitted, bringing RFC 9106-compliant password hashing into the JDK itself.

All three variants (Argon2d, Argon2i, and the recommended Argon2id) come through the SecretKeyFactory SPI, and Argon2ParameterSpec exposes memory cost, time cost, and parallelism, so you can tune resistance to brute-force attacks from GPUs and ASICs. The practical benefit is one fewer reason to drag Bouncy Castle into your dependency tree just to hash a password like it’s 2015, and the JDK’s cryptographic foundation finally steps down from compute-bound PBKDF2 to something memory-hard.

Following March’s quietly impressive HPKE and ML-DSA bundle in JDK 26, the platform’s posture around post-quantum and modern hashing improves release by release.

The most interesting thing in Java’s AI/ML story that is not a wrapper over someone else’s API: Project Babylon is putting Java onto GPU Tensor Cores. Using Code Reflection and the Heuristic Accelerator Toolkit, Babylon can now lower Java functional interfaces to NVVM IR and PTX and perform half-precision matrix multiplication directly on the HMMA.m16n8k16.f16 instruction, with no JNI and no hand-written CUDA.

The HAT API introduces tensor types (hat.T16x16x16) that use CodeModel analysis to optimize tiling and data flow. Still very experimental, but the direction, namely a unified model in which OpenJDK lowers high-level Java straight to hardware-specific IR, is one I’d keep an eye on.

March gave us the “Java is fast, your code might not be” discussion; June gave us the JDK team admitting that the modern API doesn’t always win. Two threads on panama-dev are worth your time. The first is a scalability wall: VectorMask.toLong() and fromLong() rest on 64-bit primitives, capping masks at 64 lanes, which falls apart against ARM SVE’s ambitions reaching 2048 bits, unless the signatures change.

The second, more humble, is JMH audits showing that the Vector API still doesn’t confidently beat C2 auto-vectorization, with float-to-float conversions where the scalar loop C2 generated was leaner than the explicit Vector API version (drowning in guards and uncommon-trap paths). There’s even the counterintuitive finding that holding vector constants in static final fields costs more than building them locally inside the loop.

Speaking of Leyden, which gave us the GC-independent AOT object cache in JDK 26 (JEP 516), the project now wants AOT caching for custom class loaders. The proposal extends the AOT/CDS benefits beyond the system and platform loaders to “safe and reasonable” non-subclassed URLClassLoader instances, identifying them during training runs, recreating them in the assembly phase, and storing linked Class mirrors in the cache so that getClassLoader() resolves correctly in production.

On top of that, anything that deviates from standard URLClassLoader behavior is excluded for cache stability, which is just the usual CDS caution, but it’s a real step toward fast startup for the many frameworks that lean heavily on non-standard loading.

On the governance front: Thomas Stuefe nominated Robert Toyonaga (Red Hat and IBM) for OpenJDK Committer status on the basis of 17 non-trivial patches in HotSpot Runtime, centered on Native Memory Tracking and JDK Flight Recorder, which is exactly the diagnostics you appreciate at 2 a.m. during a production memory leak.

Two more things from the corners of language design and polyglottery. Project Amber debated erasure-based union types (Integer | Float reducing to the nearest common ancestor), a “fast path” with no JVM changes to exhaustive matching in switch without formal sealed hierarchies. The community is split on whether erasure-only convenience won’t later create friction with Valhalla’s long-term goals for specialized generics, which is a fair worry.

And in the “de-JVM-ing” genre of functional languages that gave us swc4j and WebAssembly4J last month, ClojureWasm is a from-scratch runtime in Zig and Clojure that skips both the JVM and GraalVM Native Image, targeting sub-1 MB Wasm binaries with an allocator tuned for persistent data structures.

Finally, a Quarkus cluster and a nice callback. Quarkus is moving to Vert.x 5 in version 4, reworking its reactive core and leaning on SmallRye Common I/O and a NIO.2 Zip filesystem provider to attack archive-scanning bottlenecks (the public API stays stable; internal SPIs and handler APIs will change).

On top of that, the new Quarkus Pi4J extension moves Pi4J context initialization and hardware providers to the build phase, giving CDI-injected GPIO/I2C/SPI/PWM and sub-second native startup on a Raspberry Pi, which is a credible “Java instead of Python or C++” argument for IoT (and a tidy continuation of Pi4J joining Commonhaus in March).

And here’s the promised callback: when Floci was a GitHub All-Star last month, I noted that the community wanted Quarkus DevServices integration. Markus Eisele delivered exactly that, wiring Floci in as Dev Services for S3, SSM, and SQS right into the quarkus dev loop, without LocalStack’s login-token tax. The ecosystem listens, as you can see.

Oh, and one more for the “human-readable text as a weapon” file: Ken Kousen described a delightfully malicious escalation of using property-based testing with jqwik on both sides of the prompt-injection war, namely engineers fuzzing LLMs for injection vulnerabilities with @Provide and Arbitraries, and, on the dark side, developers poisoning their own code with data-nuking payloads for when an autonomous “vibe coder” swallows them.

The lesson is unchanging and unspectacular: sandbox anything that lets an LLM touch your filesystem.

A small note to close on: 👩🏻💻 Marit van Dijk (Java Champion and Developer Advocate at JetBrains) has kicked off a new series on the IntelliJ IDEA YouTube channel, where she talks with people from the JVM community.

The first episode is a conversation with Moritz Halbritter from the Spring team about JSpecify, the standardized nullness annotations that, after being adopted in Spring Framework 7 and Spring Boot 4 (plus support in IntelliJ itself), are shifting from a curiosity into an everyday tool for null-safety in Java.

Episode 1 on YouTube

2. Release Radar

Kotlin 2.4.0

Kotlin 2.4.0 is the release where the headline feature of the last few versions stops being preview. It promotes context parameters to Stable, and the opt-in -Xcontext-parameters flag disappears, so if you have @OptIn(ExperimentalContextParameters::class) scattered around, you can clean it up. They were still experimental when I covered 2.3.20 in March, so this is the moment they move into the “put it in your public API” zone. The use case is the one that’s hit everyone: dragging a logger, a transaction handle, or a tenant ID through five layers of functions just so the deepest one can read it. The compiler injects the context as the first argument at compile time (zero runtime overhead versus a plain call), it works with suspend functions, and unlike ThreadLocal it survives on iOS and JS targets in KMP, because there’s no thread-local there to break:

// Context parameters - Stable in 2.4.0; the -Xcontext-parameters flag is gone

context(tx: DatabaseTransaction)
suspend fun saveOrder(order: Order) {
    tx.persist(order)   // tx is resolved from the surrounding context, not passed in
}

The second genuinely pleasant quality-of-life change is explicit backing fields going Stable, collapsing that “private mutable plus public read-only” dance every Compose and StateFlow codebase is full of into a single declaration:

// Before: standard Compose/StateFlow boilerplate

private val _uiState = MutableStateFlow(UiState())
val uiState: StateFlow<UiState> = _uiState

// After (2.4.0): one declaration, explicit backing field

val uiState: StateFlow<UiState>
    field = MutableStateFlow(UiState())

Beyond the language there’s plenty here: kotlin.uuid.Uuid is now Stable in the common standard library (with new isSorted() / isSortedBy() checks), Kotlin/JVM can emit Java 26 bytecode with annotations in metadata enabled by default, and Kotlin/Native gains Swift Package Manager dependencies via a swiftPMDependencies block, closing out the CocoaPods-replacement story I flagged in March, plus Swift export (now in Alpha) mapping suspend functions to Swift async and Flow to AsyncSequence.

Kotlin/Wasm incremental compilation is Stable and on by default, and experimental WebAssembly Component Model support lands for cross-language Wasm interop. One administrative morsel worth knowing: each kotlin-stdlib release line on the JVM now gets an 18-month security support window with backported fixes. Compatible with Gradle 9.5.0.

Release Notes

Kotlin Toolchain 0.11 (formerly Amper)

Kotlin Toolchain 0.11 (covered by Joffrey Bion ) is the release where Amper stops being Amper. If you missed the KotlinConf keynote, the headline goes like this: Amper grew into Kotlin Toolchain and jumped straight to Alpha, which in JetBrains-speak means “we’re committing to maintaining this, so test away.”

TLDR:

At the heart of the change is a single kotlin command, conceived as one entry point into all of Kotlin: with it you create a project, build, run, test, package, and publish, and in the future also format your code and generate documentation. No build-tool decisions up front, no plugin wiring before you write your first line. The CLI can now be installed globally (sdk install kotlintoolchain) and called from outside the project directory, with the wrapper still pulling the version matched to a given repo. The long-awaited publishing of JVM libraries to Maven, including Maven Central, also enters preview, where the Toolchain takes on the whole tedious ritual (sources and javadoc JARs, PGP signatures, POM metadata, checksums) and reduces a release to kotlin publish mavenCentral, or with publishingMode: auto to a fully automated pipeline.

My favorite little detail, very on-theme for this edition: the new generated section in plugin.yaml, which registers generated sources, resources, classes, and cinterop definitions in one readable place, designed so these artifacts are recognizable at a glance not only to humans but also to AI agents and other tools. This is exactly the same thinking as JobRunr’s context7.json (more on that shortly), just built straight into the plugin model. The rest, for extension authors, is new checks and commands declarations (with the kotlin check and kotlin do commands), public entry points to tasks that are private by default, and a few fresh references like ${project.rootDir}.

One migration gotcha to close on: you can’t upgrade automatically from old Amper - the wrappers have to be swapped manually via kotlin update --create.

Release Notes

Spring AI 2.0

The third heavyweight release: Spring AI 2.0 reached GA, and being Spring, it came with a small cloud of accompanying articles. The core is the exit from experimental to stable: ChatModel, EmbeddingModel, and VectorStore are now a provider-neutral abstraction layer, so switching between OpenAI, Anthropic, and a local Ollama stops being a refactor, while the polished VectorStore abstractions and automatic metadata filtering aim squarely at cutting RAG boilerplate.

On that base, two newer things stand out: SelfCorrectingStructuredOutputConverter, which closes the feedback loop around malformed JSON (when the output fails schema validation, it packs the original instruction, the bad output, and the exception into a new prompt and retries up to a configurable retryCount)...

…and a composable tool-calling API, which replaces imperative tool binding with declarative ToolCallback beans, automating schema generation and threading state through ToolContext.

Craig Walls also documents the edges, namely ElevenLabs’ SpeechModel for streamed TTS and the Embabel abstraction layered over Spring AI and LangChain4j. Spring AI didn’t arrive alone, by the way: in the same window Spring Boot 4.1.0 shipped (more in the Radar below).

Spring Boot 4.1.0 & Spring Data 2026.0.0

Spring Boot 4.1.0 shipped alongside Spring AI 2.0 GA and is a fairly routine bump. For me the strongest point is first-class Spring gRPC support, now with reference documentation and a @GrpcAdvice annotation for centralized gRPC exception handling, which finally gives gRPC the same autoconfigured “convention over configuration” treatment REST has enjoyed for a decade.

On the security side, both reactive and blocking HTTP clients can be equipped with an InetAddressFilter that blocks outbound requests to specific addresses, a built-in hardening primitive against SSRF that you’d otherwise write by hand or pull from a library. The observability story keeps deepening (updated OpenTelemetry support, including OTel SDK environment variables, plus KafkaListenerObservationConvention beans applied automatically to the container factory), there’s file rotation support for Log4j, context now propagates automatically to @Async methods on separate threads, and a new @AutoConfigureWebServer slice annotation helps tests that need a real embedded server.

On the cleanup side: everything deprecated in 4.0 is removed, the layertools jar mode is gone, the withdrawn Apache Derby integration is deprecated (migrate to H2 or HSQL), and LiveReload in Devtools is deprecated with no replacement. It also rolls up all the fixes from 4.0.7. Treat it and Spring Data 2026.0.0 as one coordinated upgrade.

Release Notes

Gradle 9.6.0

Gradle 9.6.0 is a minor update. It improves Configuration Cache hit rate by precisely tracking project properties supplied via the org.gradle.project.<name> system properties and ORG_GRADLE_PROJECT_<name> environment variables: previously, changing any such value invalidated the cache, even if that property was never read during configuration.

There’s also a neat fix with an infrastructural flavor that fits perfectly with Gradle’s own AWS migration story (worth reading the writeup of the AWS-powered build engine behind 200 million plugin downloads a month): Gradle’s log files generated a large volume of I/O, and on network block storage like AWS EBS that triggered IOPS throttling and real slowdowns. 9.6.0 reworks that implementation, delivering significant gains on low-IOPS storage.

Looking toward Gradle 10, both implicit and explicit (findProperty(), property(), hasProperty()) lookups resolved from parent projects now emit deprecation warnings, with an opt-in NO_IMPLICIT_LOOKUP_IN_PARENT_PROJECTS preview to adopt the future behavior early, and lazy property types relax the requirement for an exact type match in the Groovy DSL.

Release Notes

Helidon 4.5.0

Helidon 4.5.0 is a release on the 4.x line following the solid LTS 4.4 (the one that brought Java Verified Portfolio support, LangChain4j agentic patterns, MCP 1.1, and the plan to rename the next line to Helidon 27 in step with JDK 27). 4.5.0 itself delivers hardened value encryption for shared-secret configuration in Helidon Config, closes out the project’s effort around API stability with dedicated stability annotations, and carries the usual fixes and dependency updates. Java 21 remains the floor, Java 25 recommended.

Release Notes

JobRunr 8.7.0

JobRunr 8.7.0 (announced by Nicholas D’hondt) makes the library noticeably easier to embed.

And since ClawRunr is built on it, “easier to embed” now reads as “a better foundation for the agentic JVM stack.”

The headline is lazy server initialization: JobRunr ships full starters for Spring, Quarkus, and Micronaut, but everyone else had to fight a greedy Fluent API that started the Background Job Server and Dashboard the moment you called useBackgroundJobServer().

Now both start on initialize(), the new getBackgroundJobServer() / getDashboardWebServer() accessors hand you the servers to manage their lifecycle, and the old configuration-ordering constraints disappear (builder calls can go in any order). On top of that: the dashboard adopts the system color scheme by default and links directly to each release’s release notes, and Jackson3JsonMapper now deserializes common collection types (List.of(...), Set.of(...), HashMap, and friends) out of the box.

My favorite little detail, very on-theme for this edition: the repo added a context7.json so that AI coding assistants pull the exact, up-to-date JobRunr configuration.

JobRunr Pro adds batch continuations created from within a batch, plus a round of dashboard security hardening (the license key validated server-side, authorization enforced on Server-Sent Events streams, stricter API key checks for the Multi-Cluster Dashboard).

Release Notes

A2A Java SDK 1.0.0

The A2A Java SDK reached GA at 1.0.0, and it matters, because Agent2Agent (an open standard under the Linux Foundation) is a protocol that lets agents from any language, framework, or vendor discover each other’s capabilities, delegate tasks, and collaborate over JSON-RPC, gRPC, or HTTP+JSON. Getting to 1.0 required a real spec-conformant transition: AgentCard now advertises a supportedInterfaces list instead of separate url/transport fields, kind type discriminators are gone, the entire spec module was modernized onto Java records, and JSpecify null-safety annotations were adopted throughout - the Maven coordinates also moved to org.a2aproject.sdk on Maven Central.

The 1.0.0 release itself adds a new integration test kit, a Quarkus-based agent for testing interoperability between SDKs, and exposes HTTP response headers through the A2AHttpResponse interface and A2AClientHTTPError.

Following it comes A2A SDK for Jakarta Servers 1.0.0-RC1 (WildFly integration, under org.wildfly.a2a), Both ADK for Java and the Spring AI A2A integration already use this SDK, so it’s effectively the interop layer for the agentic ecosystem this whole edition is circling.

Documentation | GitHub

3. GitHub All-Stars

June was a strong month for the “zero-config Java” itch and the “look inside the JAR” genre. Four picks.

jhostty - “Probably the Most Portable Terminal in the World,” in One File

Here’s JBang itself reminding everyone why it still sets the bar. jhostty, from Max Rydahl Andersen is a full-featured terminal emulator in a single Java file (~940 lines) on Java 25, embedding the Ghostty engine as a hardware-accelerated JavaFX control via GhosttyFX (0.1.169) by vlaaad. No build system, no IDE, no project setup, just JBang and one file: jbang jhostty@maxandersen (JBang pulls Java 25 automatically).

The feature list is absurdly complete for a script the author says was written “in a few hours a Sunday morning”: freely nested horizontal and vertical splits, ten themes switchable on the fly (the whole UI adapts, context menus and split dividers included), independent per-pane zoom (keyboard, scroll wheel, trackpad pinch, with the percentage in the title bar), every system font (terminal favorites like JetBrains Mono and Fira Code listed first), drag-and-drop of files and URLs, clickable link detection, shell integration (⌘F/Ctrl+F), and a native macOS menu bar.

A perfect “signal” project for how far GhosttyFX and JBang have come: native-quality, low-latency desktop UI tucked into one file you launch with a single command.

Nuts - “The Package Manager Java Never Had”

After nine years of development, Nuts (Network Unified Tool System) reached a round 1.0.0.0, and its author describes it without false modesty as “the package manager Java never had.” The idea: manage dependencies at runtime, not build time, by reading Maven descriptors directly and solving the perennial fat-JAR problem at the source, since only the JARs and dependencies (and, for native binaries, only the assets) actually needed land on the target machine.

Nuts requires no custom descriptors or build tool and doesn’t change classloading behavior: it just resolves the dependency tree, builds the classpath, and runs the app. What makes it distinctive is a shared workspace across all applications, the ability to keep multiple versions of the same app side by side, and automatic provisioning of the required JDK, so nuts install myapp pulls the latest version along with its dependencies and runtime while optimizing network and disk. The author offers the best analogy himself: it’s npm/nvm or uv, but for the Java ecosystem.

The ecosystem is still firmly in container-and-fat-JAR territory, so approach it as a thought-provoking alternative rather than a Monday-morning migration, but the nine-year gestation and a 1.0 milestone show.

Jet - A Turnkey Java HTTP Client and Server, Without the Kotlin Runtime

Jet, by Jacob Peterson, is, in its own words, “a simple, lightweight, modern, turnkey Java web client and server library” (Java 25, MIT-licensed, already on Maven Central at 3.3.0). The contrarian “Java-first” pitch: Javalin-style fluent DX without the mandatory kotlin-stdlib dependency. The architecture is modular: Common provides native HTTP header models and data structures (something the author flags competitors lack), URL building, and I/O utilities; Server is JetServer.Builder, a Handle/Handler/Router/Route system, session support, and, nicely, built-in Let’s Encrypt certificates; on top of that sit a dedicated OpenAPI annotations module and a Gradle plugin that generates the spec (with a README section arguing why a plugin rather than an annotation processor). A Client module is on the roadmap. A detail very on-theme for this edition: the README has its own “Personal Note About AI” section where the author addresses AI’s role in building the library.

Early days (~5 stars), but the niche for Java purists who find Spring Boot overkill and Javalin’s Kotlin roots a dealbreaker is real.

Marshal - Behavioral Supply-Chain Security for JVM Dependencies

Marshal does behavioral supply-chain security for JVM dependencies, with a very concrete model: it watches how packages change on Maven Central and scores every update on a 0-100 risk scale. A maintainer swap, a dropped GPG signature between versions, a sudden jump in dependency count, these all surface the day a version is published, long before a CVE exists.

It plugs in as a GitHub Action on pull requests touching pom.xml or build.gradle(.kts): it detects the build tool, scans the dependency changes, and comments on the PR with a finding (e.g. “javax.activation:activation, ORANGE 55/100, GPG signature dropped between versions”), and with threshold: red plus a required check it can block the merge. Crucially, if dependencies can’t be resolved (because the build fails), the check fails rather than passing silently, on the principle that a scanner that can’t analyze your project shouldn’t report it as clean.

It targets teams that auto-merge dependency updates directly. Still early (v0.1.0, a handful of stars), but the direction, catching the intentional backdoors that CVE-scanning alone misses, is the right one.

PS: And yes, I’m finally accepting that the JVM’s agentic ecosystem has coalesced enough that I have to stop joking about writing that overview and just write it. Consider this your warning. 😉

PS2: JVM Weekly landed at the top of Hacker News last Friday. Wow! Thank you all ❤️

PS3: From the “JVM Weekly in strange places” series - Stansted Express.

Permalink

On programming languages, targets, and platforms

I started as a Java developer, but for some time now, I have broadened my horizons. Recently, I thought about how early languages were dedicated to a single target and platform, and now they are broadening their focus. In this post, I want to write down my thoughts in the hope that it may be useful to others, probably to my future self.

Definitions

You may have been wondering about the title terms. I'm pretty sure that if you read this post, you have a pretty good picture of what a programming language is. Some may disagree on some finer points or raise a hair-splitting one, but it's not a PhD thesis, only a post on my blog. I must define what I mean by target and platform in the context of this post before going further.

Target
A target only makes sense in the context of compiled programming languages. For example, C's target is native code, and Java's is bytecode.

Platform
A platform is the system that will ultimately run the target. Native code runs on the operating system; bytecode on the JVM.

Early programming languages

Early programming languages had a single target and platform. I mentioned C and Java, but Ruby, Python, JavaScript, etc., were all the same.

I believe it was the case for a long time. It changed at some point, though.

Multi-target is the new black

The first time I heard about multi-target was in Scala. Scala came from the era of single-target and targeted bytecode on the JVM platform. However, in 2015, Martin Odersky announced Scala.js, which added JavaScript to Scala's target.

The original article was published on InfoWorld, but it seems to have redirection issues nowadays. Here's the introduction on a copy:

Scala, developed as a functional and object-oriented language for the JVM, is now multiplatform, with developers using it in abundance on JavaScript via Scala.js, Scala founder Martin Odersky says.

With Scala.js, developers write code in Scala, and the code is then compiled to JavaScript, analogous to using Microsoft's TypeScript. Developers can leverage their Scala skills in Web development. "[Scala is] very popular on JavaScript now," Odersky said at the Scala Days conference in San Francisco.

—- Scala.js lets you compile Scala to JavaScript

While Scala was the first I had heard about, other languages started to target JavaScript: I know at least Kotlin and Clojure, two originally JVM-bound languages. From that point on, it seemed every language started to add more targets. When they didn't, third parties tried to do it.

I believe that Kotlin was the epitome of such a strategy. It started with JavaScript, but the team later added native with LLVM and is currently working on WebAssembly, as far as I know. Java developers weren't left behind. The GraalVM project, managed by Oracle, allows them to generate native code. A third-party project, TeaVM, targets JavaScript and WebAssembly. It seems that nowadays lots of languages target Wasm.

Here's a small excerpt of languages and what targets and platforms they support at the time of this writing. It can't be exhaustive and obviously focuses on the JVM ecosystem, which I happen to know better.

Again, this is just a snapshot of projects I know at the time of this writing.

Discussion

Programming languages were initially focused on a single target and platform. With time, more and more languages broaden their horizon. It comes either as part of the language itself or is an effort from third parties.

It made sense so far. The bigger your organization's investment in a programming language, the harder it is to pivot to another one. In that light, GraalVM's native is a value proposition for Java-heavy organizations using Kubernetes. The JVM was meant for long-running tasks, while Kubernetes is built on the idea of stopping pods and starting new ones.

However, with the fast progress of AI, I wonder whether the trend will continue. Some might question the value. Instead of GraalVM'ifying existing Java applications, why not rewrite them directly in Rust, which natively compiles to machine code? This won't happen for core applications at the beginning, but I expect some may experiment with peripheral ones. If the experience nets benefits in terms of resource usage, then it may gnaw at core apps.

I'm very interested in the next few years to understand whether I'm imagining things, or if it will be a complete upheaval of the landscape.

To go further:

• Scala.js lets you compile Scala to JavaScript
• GraalVM
• TeaVM
• DartNative
• rustc_codegen_jvm
• Zig and WebAssembly
• SwiftWasm

Originally published at A Java Geek on June 21^st, 2026.

Permalink

ClojureScript vs. CoffeeScript for Processing sketches

DECEMBER 2015 UPDATE: I’ve run some tests with the latest ClojureScript and made some minor changes to the code, which significantly improved performance. Read this post for updated numbers. Keeping this post around for archival reasons.

The set up

This whole thing started not as a performance test, but as me experimenting with ClojureScript and Quil while reading Matt Pearson’s Generative Art. As such, it is not the most scientific of comparisons, and instead born out of my notes when exploring how to do sketches with ClojureScript for the web.

Consider the following example:

It initializes a number of circles with a random radius and movement direction, and then each frame we:

• Update their positions,
• Evaluate overlap if any of the circles overlaps with any other, and finally,
• When two overlap, draw a circle at the overlap center, with a radius of the overlap amount.

As you can see, it’s unlike a visualization where you take a set of existing items and display them, but it requires us to alter a significant amount of data every frame. I wrote it first on ClojureScript but my initial implementation had some serious performance issues, so after optimizing a bit I decided to rewrite it on CoffeeScript for a performance comparison.

Some notes:

• I’ve written Clojure before, even if I’m still a relatively new user.
• This was the first bit of CoffeeScript that I wrote, and I mostly picked it because I wanted to compare on the browser and I’m not too keen on Javascript.

This means that chances are both pieces lend themselves to optimization, but I don’t expect one language had an unfair advantage over the other (unlike, say, if I’d written one of the examples in C#, where I have a much better idea of the optimization trade-offs).

Performance

The initial ClojureScript implementation used maps for everything, which was horribly slow - I got about 9fps for 100 circles. A quick profile showed that a lot of time was being spent on accessing the map elements, and changing it so that the circle information was passed as a datatype improved performance by 300%.

As David Nolen commented on Twitter:

@argesric there's a reason ClojureScript always supplies lower level facilities - this is how the data structures are written

— David Nolen (@swannodette) January 7, 2015

Here’s the frames-per-second that Chrome reported for each implementation:

Unsurprisingly, CoffeeScript comes out ahead on something that requires a lot of data modification.  What did surprise me was how much better it scaled: it took 350 wandering circles for the performance of the CoffeScript version to drop as far as the performance the ClojureScript implementation had with only 150.

It could be argued that the comparison is unfair, since CoffeeScript gets to be mutable and if performance is the main constraint, then the GC cost of immutability is likely to bite you when doing an example such as this one.  Valid points, but I precisely wanted to push ClojureScript to see how it behaved in this case, and as I mentioned only thought about doing a CoffeeScript variant for comparison afterwards.

I then tried to find out where that time was going…

Profiling

… which brought me to the next issue - the ClojureScript version has significantly more noise in the profiler, making optimization more difficult. Check for instance this trace:

Most of the time is being spent on garbage collection or on core functions.  Our own code is in random_circles.cljs, which as you can see is way down the list. There must be some of our own functions involved in there, of course, but we need to dig really deep to figure out which of them are.

By comparison, the equivalent CoffeeScript trace is pretty straightforward.

Even without digging in at all it’s clear that most of the time in the ClojureScript version is going to garbage collection and the collection functions themselves. This is an advantage that CoffeeScript also has on this scenario, since it gets to just use Javascript’s native arrays and we don’t need to continually dispose of calculated data for the circles (even though we do it for the intersections).

Some parting thoughts

Code length: the CoffeeScript implementation is 95 lines long, ClojureScript is 165. That I did not expect. The latter has a different indentation and could probably be compacted, but I’m not sure how readable the actual functions would end up without extracting some datatype values on let.

Mutability: I could write the ClojureScript version with mutable datatypes, but that would also make the code longer and less readable.  I may try it for performance’s sake - I expect however that while it would remove the GC hit, the time spent on the CLJS function cost would still remain.

Reducing iterations: ClojureScript implementation can likely be improved by having the functions that iterate over circles be less independent so that we perform all actions at once. In this way, instead of doing first a pass that figures out the overlap and then another that draws it, we could do a single pass that takes care of both.

P5.js vs. Processing.js: One somewhat embarrassing note is that I realized too late that I had written the ClojureScript example against Processing.js and the CoffeeScript against P5.js… which turns out aren’t quite the same thing. As you’ve seen on the Profiling section, however, the actual rendering library was not the main performance bottleneck. I also found an article pointing out that Processing.js actually has a performance advantage when rendering, so I don’t expect rewriting the CoffeeScript version to use it would make any significant difference in favor of ClojureScript.

The source: Right, sources. CoffeeScript version, requiring P5.js; and the ClojureScript implementation, requiring Quil.

Permalink

Extending types from Clojure while working on the REPL

On the datatypes chapter of Programming Clojure 2nd edition (page 155) there’s the following bit where the CryptoVault is extended to support the default input streams:

The calls to spit/slurp didn’t work on my tests at first, even reloading the namespace.

After a few tests, it turns out that it was because I was using an instance of the type that was created before I had extended the type, so it didn’t have the necessary associations.

It appears that an instance keeps the function associations of when it was created, instead of referring to the latest definition… probably because it’s an entirely different type that just happens to have the same name.

Somewhat surprising at first but makes sense immediately if you think of the type not as being reloaded, but redefined.

Permalink

Clojure Meets Production MLOps: How chachaml Delivers AI‑Native Workflows ( Part 1)

This is the first article in our three-part series, Clojure Meets Production MLOps: How chachaml Delivers AI-Native Workflows.

In this article, we look at a problem many ML teams run into. Building a model is one thing. Running it reliably in production is another. We cover why MLOps has become the biggest challenge for many teams and how chachaml helps close that gap for Clojure developers.

In Part 2, Inside chachaml : Core Capabilities for AI-Native Workflows in Clojure, we’ll look at the platform itself and the features that make production workflows easier. Then, in Part 3, From Prototype to Production: The Business Case for chachaml , we’ll discuss what this means for teams that want to turn AI projects into long-term business value.

The Clojure ML Problem Nobody Talks About

➜ Machine Learning Has Matured—MLOps Has Become the Real Bottleneck

Machine learning is easier than ever. Open-source tools, cloud services, and pretrained models have made it possible for almost any team to build models.

But building a model is only the first step. The harder part is running that model in production. 

Teams have many responsibilities. They need to track experiments, manage multiple model versions, monitor performance, retrain models as needed, and ensure everything remains reproducible. 

That’s exactly what MLOps tries to handle—and honestly, it’s more important now than ever. 

📌 Gartner’s 2024 survey says only 41% of generative AI projects and 42% of traditional AI projects ever reach production. Most of them get stuck at the prototype phase and don’t go any further. 

📌 McKinsey’s 2024 report shows that 72% of organizations now use AI somewhere in their business—and 65% use generative AI regularly.

As AI adoption grows, so does operational complexity.

Clojure teams face an even tougher situation. Python has mature MLOps tools such as MLflow, while Clojure developers often have two choices:

1. Stay in Clojure and work with limited MLOps tooling.
2. Move parts of the workflow to Python and manage multiple technology stacks.

Neither option is ideal.

As machine learning becomes part of modern software systems, Clojure teams need tools that help them operate machine learning systems in production—not just build models.

➜ Why Clojure Teams Face a Different Challenge

Clojure runs on the JVM, so you get a massive selection of libraries immediately. It’s great for data-heavy projects, too. The REPL makes it easy to try ideas and run tests instantly.

What Clojure Does Well

• Teams get access to all the JVM tools and libraries. 
• Data processing? It’s fast and reliable in Clojure. 
• Use the REPL for fast feedback. 
• Its functional approach works well with data pipelines.
• It integrates easily with any systems already running on the JVM. 

Where Things Get Hard

The challenge isn’t building machine learning models. The challenge is running them in production.

Python has mature MLOps tools for experiment tracking, model management, deployment, and monitoring. Clojure has fewer options. 

As a result, teams often run into gaps when they need:

• Experiment tracking.
• Pipeline orchestration.
• Model lifecycle management.
• Deployment workflows.
• Monitoring and observability.
• Production-ready MLOps.

As a result, many teams use Clojure for applications and data processing but rely on Python tools for ML operations. 

The Trade-Off

Many Clojure teams end up choosing between two imperfect options:

1. Stay in Clojure
   • Keep architectural consistency.
   • Use the existing JVM stack.
   • Share deployment and operational infrastructure.
   • Work with fewer MLOps tools.
     
2. Move Part of the Workflow to Python

• Access mature MLOps tooling.
• Add another language to the stack.
• Keep separate infrastructure. 
• Increase operational complexity.

Neither option is perfect.

As machine learning becomes part of production systems, Clojure teams need tools that support the entire machine learning lifecycle, not just model training. 

➜ The Hidden Cost of Python-First MLOps

Many Clojure teams fill MLOps gaps by using Python tools. Sure, teams get mature platforms as part of the deal, but they also commit to a different set of challenges. 

• Everyone is switching between Clojure and Python.
• Multiple deployment pipelines.
• Tracking logs and metrics in different observability systems.
• Teams of engineers and data scientists move into separate silos. 

Eventually, machine learning creates its own ecosystem—carrying along its own tools, workflows, and infrastructure.

This is the exact gap chachaml was designed to solve.

What is chachaml?

➜ Introducing a Clojure-Native MLOps Platform

chachaml is a Clojure-native MLOps library developed within the Flexiana ecosystem.

It’s built for teams that want to run machine learning systems in production without moving their workflows to another language or stack.

This isn’t a notebook tool.

And it’s not just for experimenting with models.

chachaml handles the operations side of ML. Teams track experiments and manage models. They can maintain workflow consistency and operate pipelines.

Simply put, it brings MLOps to Clojure. 

➜ REPL-First by Design

Most MLOps tools are built around Python workflows. chachaml takes a different approach.

Because it’s designed for Clojure, it works naturally with the REPL. Within the workflow, developers can test modifications, conduct experiments, and evaluate outcomes.

That means:

• Immediate feedback.
• Faster debugging.
• Interactive experimentation.
• Native Clojure development experience.

For Clojure teams, this can feel much more natural than switching between notebooks, scripts, and external tools.

➜ Designed for Teams, Not Just Individuals

Machine learning projects rarely stay with one developer for long.

Share, review, deploy, and maintain models. Teams need a common hub for experiments, artifacts, and workflows. 

chachaml supports:

• Shared storage and artifacts.
• Team-based workflows.
• Production deployments.
• Multi-user environments.

It’s built for production ML teams, not just solo experiments.

Why chachaml Represents a Major Milestone for the Clojure Ecosystem

➜ Moving Beyond Experimental ML

The Clojure ecosystem has long supported:

• Data science experimentation.
• Model development and training.
• Data analysis and processing.

But production ML requires more than building models. Teams also need:

• Deployment pipelines.
• Model versioning.
• Monitoring and observability.
• Governance and management. 

chachaml provides:

• Production-ready ML infrastructure.
• End-to-end ML lifecycle management.
• A path from experimentation to operations.

In short, chachaml acts as the infrastructure layer that helps the Clojure ecosystem move toward mature, production-grade machine learning.

➜ Reducing Dependence on External MLOps Platforms

Many teams rely on separate MLOps tools for deployment and operations. This often creates:

• Additional infrastructure.
• More integration work.
• Complex operations.
• Multiple systems to maintain.

Chachaml helps reduce that dependency by offering:

• Greater ownership of ML infrastructure.
• Unified technology stack.
• Simple operations.
• JVM-native deployment.

Teams handle machine learning workflows without adding unnecessary platforms.

➜ Building ML Systems Without Leaving Clojure

Chachaml lets your teams handle both ML development and operations without ever leaving the Clojure ecosystem.

Teams can keep one codebase for everything—apps and ML systems live side by side. Everyone runs deployments, monitoring, and engineering the same way. 

It will result in,

• Fewer silos.
• Easy collaboration between teams.
• Operational standards everyone follows.
• Move from building to shipping a lot faster. 

If your organization already uses Clojure, chachaml turns machine learning into just another part of your engineering toolkit. No extra languages, no awkward hand-offs—just a natural fit. 

The post Clojure Meets Production MLOps: How chachaml Delivers AI‑Native Workflows ( Part 1) appeared first on Flexiana.

Permalink

Clojure Deref (Jun 23, 2026)

Permalink

Penpot for Developers: The Open-Source Design Tool That Speaks Your Language

Most design tools treat developers as an afterthought. You get handed a file, you squint at a spec panel, and you manually translate someone else's pixels into code that drifts out of sync the moment the design changes.

Penpot is an open-source design and prototyping platform where design is expressed as actual code — SVG, CSS, HTML, and JSON, the same web standards you already ship. No proprietary .fig lock-in, no "designer dialect" to interpret. It's MPL-2.0 licensed, written largely in Clojure/ClojureScript with a Rust WebAssembly renderer, and at the time of writing sits around 47k stars on GitHub.

Here's what's actually in it for you as a developer.

1. Inspect mode gives you real, ready-to-use code

Every design in Penpot has an Inspect tab that exposes the underlying SVG, CSS, and HTML. Because the design is web standards under the hood, what you copy is what you ship — not an approximation a plugin reverse-engineered. This removes the translation layer that usually causes design-to-implementation drift.

2. Layouts that behave like real CSS

Penpot supports native CSS Grid and Flexbox layouts. You design responsive interfaces using the same layout models that exist in the browser, so the structure you see in the canvas maps onto the box model you'll actually write. Less "why doesn't this reflow like the mockup" friction.

3. An MCP server for AI-driven design-to-code

This is the part worth paying attention to in 2026. Penpot ships an official MCP (Model Context Protocol) server, now integrated directly into the main repo under /mcp.

What it enables: any MCP-compatible AI client — Claude Code, Cursor, Claude Desktop, Copilot-style tools — can read and modify your Penpot design files programmatically. Because designs are already structured, machine-readable code, the agent isn't guessing from a screenshot. It works with the real component tree, styles, and tokens.

The workflows people are building with it include:

• Translating a board into production-ready semantic HTML and modular CSS, honoring your design tokens
• Generating interactive prototypes from existing designs
• Turning a rough scribble into a component that respects your design system
• Auto-generating design-system documentation from a file
• Code-to-design (not just design-to-code) and design-to-documentation round trips

Quick start for Claude Code against a local server:

claude mcp add penpot -t http http://localhost:4401/mcp

The MCP core is written in TypeScript for type-safe interaction with the Penpot Plugin API, and supports both a hosted (remote) setup and a self-hosted local setup. One safety note worth repeating: the MCP key is a personal, non-recoverable token — treat it like a password, and start your agent with read-only prompts (list, inspect, analyze) before letting it write changes to a focused page.

4. Native design tokens as a single source of truth

Penpot has first-class native Design Tokens, plus Components and Variants. Tokens act as one source of truth shared between design and development, which means no manual token exports and no separate plugin to keep your color/spacing/type scales in sync. Combined with the MCP server, your design system can become a direct context source for AI-generated code.

5. An open API, plugins, and webhooks

If you want to automate or integrate, Penpot is programmable:

• Plugin system with access to the full workspace — read and write designs programmatically
• Open REST API accessible via access tokens
• Webhooks to wire Penpot into your existing toolchain

No app-store review process or corporate gatekeeping to extend the tool.

6. Self-host it anywhere

Penpot is deployment-agnostic. Use the hosted SaaS at design.penpot.app, or run it on your own infrastructure with Docker, Kubernetes, Elestio, and other options. For teams under compliance constraints (healthcare, finance, government), this means your design IP can live entirely on servers you control — no third-party cloud required.

Why a developer might actually care

The short version: Penpot collapses the gap between design and development by refusing to invent its own format. Web-native output, real CSS layouts, native design tokens, an open API, and an MCP server that makes the whole thing AI-actionable add up to a design platform that speaks the languages you already work in — and that you can own end to end.

Figma still leads on polish, plugin breadth, and prototyping depth, so this isn't a "rip and replace" pitch. But if data ownership, design-code alignment, or AI-in-the-loop workflows matter to you, Penpot is worth a serious look.

Getting started

• Reference: https://github.com/penpot/penpot

Permalink

MrAnderson 0.6

MrAnderson 0.6 is out. If you’ve never heard of it, that’s perfectly fine - it’s the kind of tool that’s supposed to do its job quietly and stay out of sight. cider-nrepl has been using MrAnderson to inline its dependencies (bundle private copies of them that can’t clash with yours) for about as long as cider-nrepl has existed, and most of the time you’d never know it was there.

The trouble is the “most of the time”. MrAnderson has always had a few rough edges, and every so often one of them would draw blood during a cider-nrepl release and I’d find myself wondering whether we should just rip the whole thing out. I wasn’t even joking about it - at one point I went and opened a PR to remove MrAnderson from cider-nrepl entirely. That’s roughly how fond of it I was feeling.

What actually pulled me back was a different project. I’d started migrating cider-nrepl’s build from Leiningen to tools.deps, and MrAnderson, being a Leiningen plugin, was squarely in the way - which forced the question of whether to drop dependency inlining altogether or make it work outside of Leiningen. Poking at that turned into poking at MrAnderson’s long-standing issues more broadly, and the grown-up answer won out: fix the thing instead of tearing it out. This release is the result of that effort. It even ships a Leiningen-free entry point (mranderson.core/inline-deps, #42) so it can be driven straight from tools.build, which is what got the ball rolling in the first place.

Before I get into the gory details, huge thanks to Benedek Fazekas, who wrote MrAnderson and has carried it for years. I showed up with a pile of questions and a longer pile of patches, and he was nothing but supportive throughout - he even made me a co-maintainer along the way. None of what follows would have happened without him. Thank you, Benedek!

What problem is this even solving?

cider-nrepl is nREPL middleware, which is a fancy way of saying it gets loaded into your project’s JVM, right next to whatever dependencies you happen to be using. So picture this: cider-nrepl needs one version of rewrite-clj, your project pins a different one. One of them wins, the other breaks, and you get a baffling error that has nothing to do with anything you were working on.

A normal library can shrug and tell you to bump a version. Tooling that injects itself into essentially every Clojure project on the planet doesn’t have that luxury - it has to coexist with everyone’s choices. The trick MrAnderson pulls is to make cider-nrepl stop depending on rewrite-clj in the usual sense at all. Instead, it copies rewrite-clj’s source straight into cider-nrepl under a private prefix:

;; rewrite-clj.zip becomes
cider.nrepl.inlined-deps.rewrite-clj.v1.rewrite-clj.zip

and then rewrites every reference to point at the private copy. Now cider-nrepl carries its own sealed-off rewrite-clj that can’t possibly collide with yours. In the JVM world this is usually called “shading”; in Clojure-land we call it dependency inlining.

Why this is much harder than it looks

“Rename a namespace everywhere” sounds like a job for find-and-replace. It is absolutely not. Clojure refers to a namespace in a startling number of shapes, and they don’t all want the same treatment:

• ns forms, with :require/:use, :as aliases, :refer, and prefix lists
• fully-qualified vars in code (some.ns/a-fn)
• Java interop: :import, fully-qualified class names, type hints, constructors
• defrecord/deftype, which compile down to Java classes
• reader conditionals in .cljc, data readers, (load "..."), resource paths
• metadata and docstrings (which can, of course, contain example code)

And here’s the part that turns it from tedious into genuinely tricky: a dotted, fully-qualified name can be either a namespace/var reference or a Java class/record reference, and the two get munged differently. A namespace keeps its dashes; a class has to follow the JVM’s package rules, where dashes become underscores. Frequently the only way to tell which is which is to look at the surrounding context. On top of all that, a dependency can ship compiled Java .class files that need their own, separate repackaging - and the two mechanisms have to agree with each other.

MrAnderson does a lot of this transformation textually rather than with a full program analyzer, mostly for speed. That’s a reasonable trade-off, but it’s also exactly where the gremlins live.

A taste of what got fixed

Here are a few of those gremlins, now squashed:

• Records and dashes (#73) - cider-nrepl uses the prefix cider.nrepl.inlined-deps, dash and all. A reference like (instance? instaparse.gll.Failure x) points at a record, which compiles down to a Java class - so the prefix has to follow Java’s package rules, and the dash becomes an underscore (inlined-deps becomes inlined_deps). An ordinary namespace reference, on the other hand, keeps the dash. MrAnderson used to get this wrong and emit a broken reference; it now tells the two apart from the shape of the symbol, the way Clojure’s own reader does.
• Over-eager imports (#52) - if a dependency shipped its own class in the clojure.lang package, MrAnderson would happily prefix every clojure.lang import it saw, including core classes like clojure.lang.Var. Imports are now rewritten by exact class, not by package.
• Mixed imports (#33) - (:import [com.acme.impl Deftyped JavaClass]) can mix a deftype-generated class with a real Java class, and the two need different prefixes. The import now gets split so each class lands in the right place.
• Imports nobody was rewriting (#54, #92) - imports of inlined classes in a dependency’s top-level namespace, and in your own project’s sources, were simply being skipped.
• load statements (#61) - libraries that split a namespace across (load "...")ed files (hello, tools.deps.alpha) embed a resource path, not a symbol. Those paths weren’t being repointed, so the inlined library couldn’t find its own pieces.
• The genuinely scary one (#88) - if an inlined library shared a root namespace with your project and shipped AOT .class files, MrAnderson could delete your project’s own sources while cleaning up. That one I was happy to see the back of.

There’s also a good deal of less glamorous work underneath: the Java-class repackaging used to be hardwired to a target/ directory and had no test coverage at all, which is a big part of why these bugs lived as long as they did. It’s now decoupled and tested. My favourite detail: MrAnderson builds its own releases by inlining its own dependencies, so cutting 0.6 was itself a full-scale integration test of the whole thing.

What’s next

There’s still plenty of room for improvement here. The body of each namespace is still rewritten textually rather than properly parsed, and I’d love to see that grow into something more principled one day. Some of the trickier corner cases (the dash-vs-underscore guessing in particular) really want a proper analyzer rather than clever pattern matching. But the first few steps have been encouraging, and for the first time in years I’m not eyeing the eject lever.

If you rely on MrAnderson, give 0.6 a spin - and if you hit a rough edge I missed, the issue tracker is right there. Bug reports with a minimal repro are worth their weight in gold.

That’s all from me for now. Keep hacking!

Permalink

A Survival Themed Probability Cheatsheet Zine

A printable PDF of an 8-panel foldable probability zine built with Clay.

Permalink

Cartels of Mediocrity

Reproducing a sociology simulation paper about social norms and the tendency towards low-quality exchanges and suboptimal outcomes.

The Unaccountability Machine¹ by Dan Davies is broadly about how and why dysfunctional systems produce outcomes nobody seems to want. It also contains a tidy introduction to Cybernetics² and ideas like the viable system model³ and requisite variety⁴.

Reading that got me interested in more systems and sociology topics, and eventually led me to these papers:

1. The LL game: The curious preference for low quality and its norms (Gambetta & Origgi, 2012)⁵
2. Social Norms and the Dominance of Low-Doers (Proietti & Franco, 2018)⁶

Despite knowing nothing about game theory, sociology, or behavioral modeling, I thought both papers were approachable and engaging. After 25 years in industry it was refreshing to see a formal academic take on organizational cliques. And the papers were timely with a few things in my life:

• I’d been prototyping some game ideas involving simulated agent populations and emergent behavior with my kid, and the second paper does exactly that
• I’d been looking for a reason to try Clerk⁷, a Clojure visual notebook tool
• It was performance review time at work

So I committed to reproducing the paper’s findings in Clojure and its tables/figures (plus new ones) using Clerk:

• Clojure source: https://github.com/taylorwood/low-doers
• Clerk notebooks: fundamentals and figures

The phenomenon

Professionally, have you ever felt like your hard work wasn’t furthering your career, or even irritating some colleagues? Ever thought things might be easier if you coasted a bit? Maybe you needn’t even feel bad about it if your peers had the same mindset.

The dissonance is reduced by interacting always with the same people, whom one can trust for not challenging one’s standards. L-doers segregate themselves in mutual admiration societies.

This is the mindset from which Gambetta & Origgi’s “cartels of mediocrity” arise. The LL game abstract:

We investigate a phenomenon which we have experienced as common when dealing with an assortment of Italian public and private institutions: people promise to exchange high quality goods and services (H), but then something goes wrong and the quality delivered is lower than promised (L). While this is perceived as ‘cheating’ by outsiders, insiders seem not only to adapt but to rely on this outcome. They do not resent low quality exchanges, in fact they seem to resent high quality ones, and are inclined to ostracise and avoid dealing with agents who deliver high quality. This equilibrium violates the standard preference ranking associated to the prisoner’s dilemma and similar games, whereby self-interested rational agents prefer to dish out low quality in exchange for high quality. While equally ‘lazy’, agents in our L-worlds are nonetheless oddly ‘pro-social’: to the advantage of maximizing their raw self-interest, they prefer to receive low quality provided that they too can in exchange deliver low quality without embarrassment. They develop a set of oblique social norms to sustain their preferred equilibrium when threatened by intrusions of high quality. We argue that cooperation is not always for the better: high quality collective outcomes are not only endangered by self-interested individual defectors, but by ‘cartels’ of mutually satisfied mediocrities.

And later in the LL game:

Our basic point so far can be summed up thus: if you give me L but in return you tolerate my L we collude on L-ness, we become friends in L-ness, just like friends we tolerate each other’s weaknesses. But if you give me H that leaves you free to disclose my L-ness and complain about it. So you are not my friend, I fear and resent you, and if I cannot punish you for producing H, at least I avoid dealing with you. While in an ordinary world it is L-doers who are punished by avoidance and exclusion, in an L-dominated world it is H-doers who are ostracised. Essentially, the L-exchange can be seen as a cartel of mediocrities who pretend to be H.

There seems to be two forces that could contrast our supposed natural inclinations to L-ness and promote quality, one is the passion for a job well-done, the intrinsic pleasure found in employing and testing one’s skills at some task; the other is competition, succeeding at which carries extrinsic rewards. Generically, these forces fail if the algebraic sum of rewards and punishments for H-ness is lower than the sum of rewards and punishments for L-ness. Even H-prone individuals are ultimately driven to choose L (or to become eccentric and isolated ‘perfectionists’ or to migrate) if they systematically fail to gain any reward from their effort. In short, L spreads if it pays off. This remains a tautology, however, unless we can understand the conditions that affect the relative payoffs of H and L.

I recommend reading the paper just for the examples of different H/L exchanges, their psychosocial underpinnings, and the colorful Italian anecdotes⁸.

Social Norms and Low-Doers takes these ideas further with social agent simulation in NetLogo⁹. After modeling the social decay of L-worlds, it tests different regimes of rewards/sanctions to foster high-quality exchange:

Social norms play a fundamental role in holding groups together. The rationale behind most of them is to coordinate individual actions into a beneficial societal outcome. However, there are cases where pro-social behavior within a community seems, to the contrary, to cause inefficiencies and suboptimal collective outcomes. An explanation for this is that individuals in a society are of different types and their type determines the norm of fairness they adopt. Not all such norms are bound to be beneficial at the societal level. When individuals of different types meet a clash of norms can arise. This, in turn, can determine an advantage for the “wrong” type. We show this by a game-theoretic analysis in a very simple setting. To test this result – as well as its possible remedies – we also devise a specific simulation model. Our model is written in NETLOGO and is a first attempt to study our problem within an artificial environment that simulates the evolution of a society over time.

The game

A society of agents that collaborate with each other, accumulate payoffs, age, retire, and get replaced by new hires over time. Their collaborations consist of a simple mutual exchange of abstract “goods”:

Assume for simplicity that goods can be produced at two levels of quality, High (H) and Low (L). H is both more rewarding to receive and more costly to produce than L; H takes more time, effort, skills and organisation.

Agents independently choose how much effort to put in: H (high) or L (low). Neither knows the other’s choice in advance. The four possible outcomes from any agent’s perspective:

Every agent has a type, which describes their ranking of preferred outcomes, e.g. HH > HL > LH > LL. All preference orderings of those four outcomes become 24 possible agent types in all, each classifiable along two axes: selfishness and mindedness. A type is selfish when free-riding (LH) is its top choice, and high-minded when it ranks mutual-high (HH) above mutual-low (LL).

Proietti & Franco focus on two types to start: hs1 and ls1. Both are selfish (their top preference is giving L and receiving H) but they differ in their mindedness (which they’d prefer if selfishness isn’t on the table.)

• hs1 (high-minded) prefers mutual-high (HH) over mutual-low (LL). Starts out playing H.
• ls1 (low-minded) prefers mutual-low (LL) over mutual-high (HH). Starts out playing L.

Such agents are arguably likely to be found in a competitive society where individuals are incentivized to participate in many activities (for example improving their CV by publishing, teaching, participating to conferences and research projects) while at the same time economizing their efforts and getting the most out of them.

Payoffs follow the preference ordering:

hs1 ranks HH > LL; ls1 flips them. Both rank the sucker outcome (HL) worst and free-riding (LH) best.

Agents can reconsider their strategy after an exchange. Each agent tracks two running tallies per partner (which are reset whenever the agent changes strategy):

• Shortfall: cumulative gap between actual payoff and what both playing baseline would have earned.
• Balance: running difference between actual payoff and what the opposite action would have earned with this partner.

When the shortfall crosses a threshold, a negative balance triggers a switch in the agent’s baseline strategy.

The asymmetry for the hs1 vs ls1 collaborations is apparent from the opening play: hs1 starts at H and can fall short, but ls1 starts at L and in an LL exchange already earns its second-best outcome.

sequenceDiagram
    participant hs1
    participant ls1

    Note over hs1: plays H (baseline)
    Note over ls1: plays L (baseline)

    hs1->>ls1: H
    ls1->>hs1: L

    Note over hs1: earns 1 (sucker)
shortfall +2, balance −1
    Note over ls1: earns 4 (free-rider)

    Note over hs1: shortfall ≥ threshold
balance < 0 → switches to L

    loop Every subsequent round
        hs1->>ls1: L
        ls1->>hs1: L
        Note over hs1: earns 2, shortfall grows
but switching back to H
would only earn 1 --- stuck
        Note over ls1: earns 3 (preferred LL)
    end

The simulation

Proietti & Franco implement the model simulation in NetLogo, and I based my reproduction on their published source code¹⁰.

Each simulated year:

1. Reset everyone’s annual earnings
2. Retire the oldest agents and replace them with fresh ones
3. Every pair plays one round; earnings are gated by a collaboration probability
4. Age everyone
5. Mark anyone past the retirement age for mandatory retirement
6. Mark a bottom % earners for early retirement

The institution hires 50/50 from hs1/ls1. The mechanism that tips the balance is in step 6: ls1 agents systematically out-earn hs1 agents over time, so hs1 agents have shorter careers/earlier retirements. Try-hards burn out and low-doers inherit the org: fresh hs1 replacements (hired 50/50) start out playing H, get exploited again, and the cycle repeats.

The agent state is a plain map; the memory map is keyed by partner id and tracks the shortfall/balance running tallies.

(defn agent [id type]
  {:my-id id :type-of-academic type :age 0 :total-payoff 0 :memory {}})

Each round, both agents decide independently whether to offer H or L, then record the exchange:

(defn play-round
  ([a1 a2] (play-round a1 a2 default-params true))
  ([a1 a2 params pays-out?]
   (let [act1 (decide a1 (:my-id a2) params)
         act2 (decide a2 (:my-id a1) params)
         [p1 p2] (typ/payoffs (:type-of-academic a1)
                              (:type-of-academic a2)
                              [act1 act2])]
     [(record-exchange a1 (:my-id a2) act1 act2 p1 pays-out?)
      (record-exchange a2 (:my-id a1) act2 act1 p2 pays-out?)])))

The decide function implements the reconsider rule: an agent plays its baseline until shortfall crosses the threshold, then a negative balance flips it to the opposite action (H-to-L or vice versa.)

(defn- reconsidering? [baseline shortfall threshold]
  (and (= baseline :H) (>= shortfall threshold)))

(defn decide
  [agent partner-id {threshold :change-of-strategy-threshold
                     :keys [reward-tick? reward-pct sanction-pct reward-maxes]}]
  (let [t    (:type-of-academic agent)
        base (typ/baseline-action t)
        m    (get-in agent [:memory partner-id])
        last (:last-action m base)]
    (if (reconsidering? base (:difference-from-optimal m 0) threshold)
      (let [rb (if reward-tick?
                 (reward-balance t (:recon-exch m {})
                                 (reward-prob (:window-hh agent 0)
                                              (:max-hh reward-maxes))
                                 (reward-prob (:window-ll agent 0)
                                              (:max-ll reward-maxes))
                                 (double (or reward-pct 0))
                                 (double (or sanction-pct 0)))
                 0.0)]
        (if (neg? (+ (:balance m 0) rb))
          (other last)
          last))
      base)))

The reward-tick? branch is used later: when a reward/sanction regime is active, the expected reward/sanction differential (rb) is added to the balance before the switch test.

I think it’s important to note, as in the paper, the experiments below are conducted on a fully-connected network of agents and each connection has a probability of collaboration. Obviously this modeling approach doesn’t reflect more hierarchical or siloed organizational structures. To that end they also model scale-free networks where agents have fewer connections and find those conditions more favorable for hs1.

The outcomes

Given a society of 20 agents hired evenly from hs1/ls1, H actions begin around 20% of exchanges, then collapse to a ~9% steady state within the first decade or two and remain there for 1,500 simulated years.

Time axis is log-scale to show early collapse and the long flat tail. Faint blue lines are each RNG seed's 5-year rolling mean; bold blue line is all-seed median.

Career churn

The agent turnover driving that collapse shows up in the agent career tenures and retirement reasons. Plotting tenure-at-retirement, hs1 (left) piles up on the short end while ls1 (right) generally lasts longer.

hs1 dominates the early-retirement side (bottom-% earners forced out under sustained low payoffs), while agents that survive to the mandatory retirement age reach it at similar tenures regardless of type. ls1’s longer careers come from rarely capitulating to H, not from outlasting hs1 once it does.

Left: early exits driven by low cumulative payoff (hs1-heavy). Right: agents that reached the mandatory retirement age (both types, similar tenures).

Robustness

The paper’s finding holds across different simulation parameters for society sizes and strategy-change thresholds. (See original: 4.3)

H-action rate after 1,500 years across a range of society sizes and change-of-strategy thresholds (15% quantile retirement).

Table 4: adjusted payoffs. hs1's LL jumps to 8 vs ls1's 7.

Table 4 breaks that asymmetry by raising hs1’s LL payoff above ls1’s, so an hs1 stuck in mutual-low no longer trails its partner as it does under normal payoffs, yet the simulated outcome is largely unchanged. (See original: 4.7)

Raising hs1's LL payoff above ls1's (Table 4) breaks the payoff asymmetry that promotes low-doer dominance, yet ls1 still dominates.

What can be done?

The paper tests two basic strategies for preventing the collapse into low-quality exchanges:

1. Change who you hire: the society’s composition of H vs L, selfish and non-selfish
2. Change their incentives: rewards for HH exchanges and sanctions for LL

Hiring filters

The first lever is the mix of agent types in the society. (See original: 4.10)

Even at 70–90% hs1, the H-rate declines with network size, across multiple change-of-strategy thresholds.

Unsurprisingly, hiring more hs1 agents helps, but as the paper notes:

Furthermore, it is quite challenging for a policy maker or employer to succeed in hiring such a high percentage of high-minded individuals.

Heroes & Saints

What works better is changing the kind of high-minded agent you hire. These two non-selfish, high-minded types protect the H equilibrium:

• hn1 (hero): always plays H, even as the sucker, though it still ranks free-riding (LH) on top in preference.
• hn2 (saint): same behavior as the hero, but goes further and ranks LH below LL, so it wouldn’t even want to exploit.

Both types resist the capitulation mechanism entirely: since they always play H regardless, their shortfall/balance never push them to capitulate. Swap hs1 for either and the H-rate hovers around 50%.

Under such conditions, the efficiency of an institution can be sustained if high-minded people are not selfish, we may call them “heroes” or “saints”.

Change the type, not the count: hs1 collapses to ~9%, while hn1/hn2 hold near 50%.

Reliably hiring a smaller contingent of heroes (hn1) and saints (hn2) may be more effective than hiring mostly high-minded, selfish agents (hs1), but it is similarly challenging for most organizations; mathematically, we can’t all work “the best of the best.”

Rewards & Sanctions

The second lever leaves the agent mix alone and changes the incentives (the reward-tick? branch of decide above).

As it turns out, rewarding HH exchanges has little effect, but sanctioning LL exchanges pulls capitulated hs1 agents back to H: the looming penalty enters the reconsider calculation and tips the balance.

Frequency also matters: sanctions every round push H-rates above 65%, while sanctions every three rounds barely help at all. Sanctioning LL is what helps. Frequent sanctions help more (bottom-left), and hiring more hs1 pushes the effect toward ~90% (bottom-right).

Rewards and sanctions (the paper's Tables 5-8).
Each panel plots the steady-state H-rate against LL-sanction strength; color is the HH-reward. Top row sanctions every third year (f=3), bottom row every year (f=1); left column hires 50% hs1, right column 65%. Error bars are 95% CI over 5 seeds.

Sanctions only fire when an agent is in range of a reconsideration, so the more hs1 agents you start with, the more often the penalty has something to penalize. Hiring 65% hs1 lifts the baseline and reduces the sanction-frequency impact, recovering much of the sanction benefit even at the every-three-years cadence where 50/50 hiring collapses.

With selective hiring and balanced incentives, the H-rate climbs to ~90% (which is the most effective policy tested in the paper.)

Takeaways

These papers model societies/organizations that tend toward an equilibrium where low effort becomes pro-social behavior despite the undesirable outcomes. They contend “cartels of mediocrity” form against those offering higher quality exchanges.

To my mind, the top insight is that pro-social conformists and disillusioned try-hards (not free-riders) are the true drivers of decay: agents who’d genuinely prefer mutual excellence get captured by a social norm, “rationally” capitulating once sufficiently exploited. The high-minded burn out earlier and churn, and are replaced with new hires, keeping the organization flush with new subjects.

The secondary insight would be that sanctions on low-quality work are far more effective than rewards for high-quality work.

My suggestion when faced with organizational dynamics like these is to take pride in doing high-quality work even if peers do not! (But also, work within your means¹¹. Don’t sacrifice your sanity and burn out.)

Notes

Permalink

Automatic help and completions in Babashka CLI

org.babashka/cli {:mvn/version "0.11.73"}

#!/usr/bin/env bb
(require &apos[babashka.cli :as cli]
         &apos[clojure.string :as str])

;; stand-ins; a real tool would shell out to git
(def ^:private branches ["main" "develop" "feature/login" "release/2.0"])
(def ^:private remotes  ["origin" "upstream" "fork"])

(defn clone [{:keys [opts]}]
  (println "Cloning" (:url opts)
           (when (:depth opts) (str "(depth " (:depth opts) ")"))))

(defn log [{:keys [opts]}]
  (println "Showing" (or (:max-count opts) "all") (name (:format opts)) "log entries"))

(defn checkout [{:keys [opts]}]
  (println (if (:create opts) "Creating and switching to" "Switching to") (:branch opts)))

(defn remote-add [{:keys [opts]}]
  (println "Added remote" (:name opts) "->" (:url opts)))

(defn remote-remove [{:keys [opts]}]
  (println "Removed remote" (:name opts)))

(defn remote-list [_]
  (run! println remotes))

(def tree
  {:spec {:verbose {:coerce :boolean :desc "Be verbose" :alias :v}}
   :cmd
   {"clone"
    {:fn clone :doc "Clone a repository into a new directory"
     :spec {:url   {:desc "Repository to clone from" :require true}
            :depth {:desc "Create a shallow clone with N commits" :coerce :long}}
     :args->opts [:url]}
    "log"
    {:fn log :doc "Show commit logs"
     :spec {:format    {:desc "Output format" :coerce :keyword
                        :validate #{:oneline :short :full}
                        :default :short}
            :max-count {:desc "Limit the number of commits" :coerce :long :alias :n}}}
    "checkout"
    {:fn checkout :doc "Switch branches"
     :spec {:branch {:desc "Branch to switch to" :coerce :string
                     :complete-fn (fn [{:keys [to-complete]}]
                                    (filter #(str/starts-with? % to-complete) branches))
                     :require true}
            :create {:desc "Create the branch before switching" :coerce :boolean :alias :b}}
     :args->opts [:branch]}
    "remote"
    {:doc "Manage the set of tracked repositories"
     :cmd-order ["add" "remove" "list"]
     :cmd
     {"add"
      {:fn remote-add :doc "Add a remote"
       :spec {:name {:desc "Remote name" :require true}
              :url  {:desc "Remote URL" :require true}}
       :args->opts [:name :url]}
      "remove"
      {:fn remote-remove :doc "Remove a remote"
       :spec {:name {:desc "Remote name" :coerce :string
                     :complete-fn (fn [{:keys [to-complete]}]
                                    (filter #(str/starts-with? % to-complete) remotes))
                     :require true}}
       :args->opts [:name]}
      "list" {:fn remote-list :doc "List the existing remotes"}}}}
   :epilog "Docs: https://example.com/mygit"})

(defn -main [& args]
  (cli/dispatch tree args {:prog "mygit" :help true}))

(apply -main *command-line-args*)

chmod +x mygit.clj

(require &apos[babashka.deps :as deps])
(deps/add-deps &apos{:deps {org.babashka/cli {:mvn/version "0.11.73"}}})
(require &apos[babashka.cli] :reload)

$ ./mygit.clj clone https://example.com/repo.git --depth 1
Cloning https://example.com/repo.git (depth 1)

$ ./mygit.clj checkout -b feature/login
Creating and switching to feature/login

$ ./mygit.clj log -n 5 --format oneline
Showing 5 oneline log entries

$ ./mygit.clj remote add origin https://example.com/repo.git
Added remote origin -> https://example.com/repo.git

Usage: mygit [options] <command>

Commands:
  clone    Clone a repository into a new directory
  log      Show commit logs
  checkout Switch branches
  remote   Manage the set of tracked repositories

Options:
  -v, --verbose  Be verbose
  -h, --help     Show this help

Run "mygit <command> --help" for more information on a command.

Docs: https://example.com/mygit

Usage: mygit checkout [options] <branch>

Switch branches

Options:
      --branch  Branch to switch to (required)
  -b, --create  Create the branch before switching
  -h, --help    Show this help

Run "mygit --help" for global options.

Usage: mygit remote [options] <command>

Manage the set of tracked repositories

Commands:
  add    Add a remote
  remove Remove a remote
  list   List the existing remotes

Options:
  -h, --help  Show this help

Run "mygit remote <command> --help" for more information on a command.

Run "mygit --help" for global options.

Usage: mygit remote add [options] <name> <url>

Add a remote

Options:
      --name  Remote name (required)
      --url   Remote URL (required)
  -h, --help  Show this help

Run "mygit --help" for global options.

$ ./mygit.clj statys
Unknown command: statys

Commands:
  clone    Clone a repository into a new directory
  log      Show commit logs
  checkout Switch branches
  remote   Manage the set of tracked repositories

Run "mygit --help" for more information.

source <(./mygit.clj org.babashka.cli/completions snippet --shell zsh)

$ ./mygit.clj remote <TAB>
add     -- Add a remote
remove  -- Remove a remote
list    -- List the existing remotes

$ ./mygit.clj log --format <TAB>
full  oneline  short

$ ./mygit.clj remote remove <TAB>
origin  upstream  fork

$ ./mygit.clj checkout <TAB>
main  develop  feature/login  release/2.0

$ ./mygit.clj org.babashka.cli/completions complete --shell zsh -- remote &apos&apos
add	Add a remote
remove	Remove a remote
list	List the existing remotes
--help	Show this help
-h	Show this help

Permalink

The Hidden Elegance of Gradient Noise

How would you go about rendering a scene reminiscent of dark teal water, lit from somewhere below, with thousands of faint cyan filaments drifting and swirling across it? Your instinct might be to reach for a shader or to create a particle simulation, but you could render the whole thing using just a couple hundred lines of arithmetic instead. That's precisely what we're going to do in this post by rendering these filaments using the same function Ken Perlin wrote in 1985 to fake textures on a computer that couldn't draw them for real, which we know today as Perlin noise.

I'll walk you through a moving-water visualization to illustrate what Perlin noise actually is, and how a single noise value can be used to steer thousands of particles into curving currents to create a flowing surface. The snippets use the Squint ClojureScript dialect, but the ideas are language-agnostic.

What is Perlin noise?

Naively using random values is the wrong approach for creating a natural-looking texture. Pure randomness at every pixel will produce boring static that's chaotic and grainy. Real surfaces such as marble or water are smooth because neighbouring points tend to be correlated. A piece of marble that's bright here is probably still fairly bright a millimetre over there.

Perlin noise provides a way to generate that kind of structured pseudo-randomness. It's a deterministic function from a point in space to a scalar value, with three properties that make it magical for graphics, which are as follows.

Nearby inputs give nearby outputs without seams, leading to smooth transitions. The same seed always gives the same output, so the texture ends up being stable across frames. And it has no preferred direction, making it look isotropic, unlike a simple grid of blurred random dots.

Under the hood it's just gradient noise generated in three steps. First, we need a tile space in the form of a grid, and then we plant a pseudo-random gradient vector at every corner to provide a direction. For any point inside a cell, we need to figure out how strongly each corner's gradient points toward it using a dot product. Finally, we just blend the contributions of the surrounding corners.

A naive linear blend would leave ugly visible creases at every grid line. Perlin, instead, passes the interpolation parameter through a fade curve which is a polynomial shaped so that it starts and ends flat, allowing the value to ease gently into each corner:

(defn fade [t]
  (* t t t (+ (* t (- (* t 6) 15)) 10)))

The formula above is just 6t⁵ − 15t⁴ + 10t³ with its first derivative being zero at both t = 0 and t = 1, which is precisely what guarantees the output is smooth across cell boundaries. Linear interpolation itself is likewise dead simple:

(defn lerp [t a b]
  (+ a (* t (- b a))))

The gradient lookup hashes a corner to one of a fixed set of directions and returns the dot product against the point's offset within the cell:

(defn grad [hash x y z]
  (let [h (bit-and hash 15)
        u (if (< h 8) x y)
        v (if (< h 4) y (if (or (== h 12) (== h 14)) x z))]
    (+ (if (zero? (bit-and h 1)) u (- u))
       (if (zero? (bit-and h 2)) v (- v)))))

A small seeded PRNG shuffles an identity permutation table at construction time to decide which gradient each corner gets, making the field reproducible. A caller doesn't need to worry about any of this and simply passes their desired x, y, and z to noise3 to get back a smooth value. Perlin's raw output sits roughly in [-1, 1], and the implementation remaps it to [0, 1] so that downstream consumers can scale it linearly into their own positive range:

(/ (+ 1 n) 2)

And that's the whole noise engine in a nutshell. Now that we have our noise, let's see what we can do with it to create a smooth animation.

From a number to a current

Smooth scalar values are nice, but what if we wanted to create an animation which moves in a particular direction? Well, to do that we just have to treat the noise value as an angle to give us a compass heading. Next, we multiply by a full turn (2π) so that the entire [0, 1] range maps to every possible direction:

(defn create-flow-field 
  [{:keys [noise noise-scale force-scale time-scale]
    :or {noise-scale 0.003 force-scale 1 time-scale 0.15}}]
  (let [noise3 (:noise3 noise)]
    {:force-at
     (fn [x y t]
       (let [theta (* (noise3 (* x noise-scale) (* y noise-scale) (* t time-scale))
                      js/Math.PI 2)]
         #js {:x (* (js/Math.cos theta) force-scale)
              :y (* (js/Math.sin theta) force-scale)}))}))

And with that trick we get a flow field which we can ask for a velocity vector of a pixel at (x, y). Since the underlying noise is smooth, nearby pixels get nearly identical headings and the field ends up looking like a coherent map of currents, complete with eddies, calm spots, and converging streams.

The noise-scale knob controls the zoom factor of the flow. Scaling the coordinates down before sampling samples the noise at a coarse resolution, creating swirls that are broad and slow. On the other hand, scaling up produces nervous little vortices.

A keen reader will have noticed that the function takes a third coordinate, t, that we'll come back to later. For now, I'll leave a hint that it's going to be our secret ingredient for motion.

Drawing the curves

To actually see the current we have to drop particles into the field and let them drift. Each particle needs to keep track of its previous position as it's moved by its local current, so that we can draw a short line segment from where it was to where it landed:

(defn update-particle! [p force]
  (set! (.-lifetime p) (dec (.-lifetime p)))
  (if (neg? (.-lifetime p))
    (respawn! p)
    (do
      (set! (.-prevX p) (.-x p))
      (set! (.-prevY p) (.-y p))
      (set! (.-x p) (+ (.-x p) (.-x force)))
      (set! (.-y p) (+ (.-y p) (.-y force)))
      (wrap! p (.-width p) (.-height p))))
  p)

When we run that for a few thousand particles over a thousand frames in a row, they trace a curve through the field, and since the field is smooth and continuous, neighbouring particles trace neighbouring curves. The collective result has a look of flow lines following a current similar to the way dye disperses in moving water.

Each segment itself is just a stroked line, tinted by a second, finer noise pass so the colour shimmers across the surface instead of reading as just flat cyan:

(defn- draw-segment! [p noise2]
  (let [v     (noise2 (* (.-x p) 0.004) (* (.-y p) 0.004))
        hue   (+ 185 (* v 30))
        light (+ 55 (* v 25))]
    (set! (.-strokeStyle ctx) (str "hsla(" hue ", 80%, " light "%, 0.3)"))
    (doto ctx
      (.beginPath)
      (.moveTo (.-prevX p) (.-prevY p))
      (.lineTo (.-x p) (.-y p))
      (.stroke))))

So the shape of the motion comes from the noise field while the colour comes from an independent one sampled at a different scale. Thus, we have two channels of the same primitive, doing two different jobs.

Two additions that make it move

Everything we've done so far produces a frozen flow field. Next, we'll need to make two small changes to turn it into a living animation.

1. Time is just a third dimension

Remember the unused t in force-at, which we were going to come back to? Well, what I didn't mention is that Perlin noise can be defined for any number of dimensions, and the implementation here is actually 3D. The first two dimensions are in space, but the third one is time. Each frame, we advance t a tiny bit, and because the noise is smooth in all directions, the entire current field ends up drifting as a result. Eddies migrate, streams bend, while calm patches open and close. The field smoothly evolves from one frame to another as we increment the counter:

(swap! state update :time inc)

The time-scale parameter governs how fast that evolution happens, and we want to keep it small to produce gentle change rather than a strobe. And that's how using an extra noise dimension as the clock turns a static render into an animation. In case you're wondering, you can generalize it freely, and a 3D animation can be similarly created using 4D noise.

2. Trails decay into the deep

The last step is to make sure that our trails fade over time to create continuous motion as old trails fade out, and new ones appear over time. To achieve a shimmering effect we want to avoid fully clearing the canvas. Instead, every frame paints a translucent dark rectangle over the scene before drawing the new segments:

(set! (.-fillStyle ctx) "rgba(3, 18, 26, 0.03)")
(.fillRect ctx 0 0 width height)

A value of 0.03 alpha is doing a huge amount of work creating the effect of old line segments slowly getting drowned. A particle's recent trail glows bright, one from half a second ago starts to fade, and then it's gone completely. The result is a cheap, accidental motion-blur that gives the surface its reflective, continuously flowing quality.

Tuning this alpha number shifts the whole mood, with higher values making trails vanish almost instantly, while lower ones smear into long ghostly streaks.

Tying the edges together

Another thing to consider is how to keep the surface believable at the borders. Here, we can have particles that drift off one edge reappear on the opposite side using a toroidal, seamlessly tiling wrap. When a particle wraps, its previous position needs to wrap along with it to avoid drawing ugly streaks across the canvas:

(defn- wrap-delta [v extent]
  (cond (>= v extent) (- extent)
        (neg? v)     extent
        :else        0))

In a flow field, particles spiral into a handful of attractor orbits and drain out of the rest of the canvas. To keep the whole surface populated, each particle has to have a randomized lifetime so that when it expires, it can respawn at a fresh random location with its lifetime reset. Lifetimes also need to be jittered from the start so that respawns stay staggered rather than all firing on the same frame.

The loop

Here's how the whole machine runs frame by frame:

(defn draw []
  (let [{:keys [width height particles time]} @state
        force-at (:force-at field)
        noise2   (:noise2 noise)
        n        (alength particles)]
    ;; Fade old trails toward the deep.
    (set! (.-fillStyle ctx) "rgba(3, 18, 26, 0.03)")
    (.fillRect ctx 0 0 width height)
    (set! (.-lineWidth ctx) 1)
    (set! (.-lineCap ctx) "round")
    ;; For each particle: sample the current, drift, draw its segment.
    (dotimes [i n]
      (let [p (aget particles i)]
        (update-particle! p (force-at (.-x p) (.-y p) time))
        (draw-segment! p noise2)))
    ;; Advance the clock and schedule the next frame.
    (swap! state update :time inc)
    (reset! raf-id (js/requestAnimationFrame draw))))

If you read the function from top to bottom, you can see the exact steps that are happening. First, a dark background is painted on the canvas, then the noise field is sampled for each of a couple thousand particles to see which way the water flows. The particles are nudged that way, and a faint coloured line is drawn behind each. Doing that sixty times a second creates the final animation.

Why it works

Now we can see how the whole animation is put together. The noise gives us the basis for the flow field, the third dimension provides an arrow of time, and the fade creates a sense of motion. All of these ideas compose into something that feels far more complex than the sum of its parts.

The full source can be seen on Squint playground, and the version running above is served from perlin-flow.js generated from Squint.

Permalink