Planet Clojure

My experience with Cursor and Clojure-MCP

Today, there are many ways to use Generative AI for coding. From tools running in the terminal like Claude code, Codex, Kira, Ampcode, Cline… to using Anthropic, ChatGPT, Gemini, Mistral … in the browser to a more integrated experience with Cursor or VS Code with Copilot. The previous list is not exhaustive; there is literally an explosion of tooling implementing the same OODA loop. A build-from-scratch description here with its naive Clojure implementation here.

What I found missing is developer reports on the use of these tools in real-life situations. I get it. It’s hard to compare those tools with each other: GenerativeAI non-deterministic nature, various AI models, various clients using it with tools or MCP, the quality of the prompt written by the developer, the state of your codebase … and so on. Add to that the hype train of everything needs AI versus what AI actually does today is hard to challenge.

So here is my report after using Cursor and Clojure-MCP for a few months. I hope it inspired others to do the same and that we can all grow together with this wave of tools.

Cursor without clojure-mcp

Clojure being a niche language, the initial experience was frustrating. It could not get the parentheses right: more than half the time, one was missing, and the more the edits happened, the more functions started to be mismatched with each other. Even for the simplest tasks, it was often wrong. It was pre-Claude-4 (today’s best coding model with GPT5, as far as I know), maybe it could figure it out better now? Maybe not.

Cursor agent mode made it even worse than without agent: the clj-kondo hint about wrong parentheses confuses the agent that starts to move parentheses here and there until it decides to either give up or rewrite the entire file…

Even simple things took way too long, and debugging the parentheses imbalance in Lisp is not something I enjoy particularly.

Arrive clojure-mcp

https://github.com/bhauman/clojure-mcp

You might recognize the author’s name because he also created figwheel.

From the README:

Clojure MCP connects AI models to your Clojure development environment, enabling a remarkable REPL-driven development experience powered by large language models (LLMs).

With it, the agent experience got a lot better. No more imbalance parentheses with dedicated tools to edit s-expression, and it speeds up the agent loop considerably by using the REPL. The agent can try a solution directly (ie. without JVM restart) and see if it works or not. Pretty much a human will do. After a few tries, it writes the solution in the file. I don’t have metrics on the speed of Clojure-MCP versus the vanilla cursor edit, but from my impressions, it goes a LOT faster to write good code solutions.

Context of my ongoing project

To give some context (which matters a lot, but more on that later), on what I’m working on. I start a new project from scratch using this great template: clojure-stack-lite. It comes with everything you want for starting a web project: Integrant, Deps, Babahka tasks, Tailwind, CI, clj-kondo, Reitit… Kudos to Andrey, it is a great work and makes it easy to start something new.

Why does that matter for my AI setup? With a small codebase, it’s harder to get lost. Or with fewer tokens, the code proposals are better. With a working reload workflow, Clojure-MCP can iterate more easily until it finds a working solution, and if it goes in the wrong direction, you can just (user/reset) and you are clear to go. With different codebase maturity, you will get a different experience.

Retrospectively, another interesting aspect of this codebase is that with HTMX, my templates are present in the codebase like the rest of the code. Due to how verbose HTML (ie. hiccup) and Tailwind are, the codebase quickly gets dominated by views, which significantly increases the amount of tokens (higher cost and latency of every model call). I would be curious to compare it with a solution where the templates are in different files, like Mustache/Handlebars or something like that, making it clear where the views are and where the logic is. I guess I could force it in the code, but having less power in the templating somehow deserves me here.

Worth mentioning, I started from a few HTML mockups that I built with Claude-4. I use this mockup in my prompt, so the AI should have a good idea about the expected visual side with a syntax close to Hiccup.

Building full features

The first few features I built entirely with the Agent, I used Claude-4 as a model (let’s start with the best, right?), and how boy, it felt like magic. What I thought would take me a few days to implement was done in a few hours with some minor changes and follow-up prompting to help the agent go in the direction I want. I imagine it’s matching the “vibe coding” experience. You still need to know and explain clearly what you want, but a lot of things are figured out by the model and by trial and error with the REPL.

I forgot to mention that I’m also using playwright-mcp, which allows the agent to reproduce what I see in my browser and fix the error that can appear in the console. Feels like magic when you are describing something, and the Agent changes the layout according to your instructions. As a side effect, I rarely open the Tailwind documentation despite not being familiar with it.

Over time, the code accumulates small problems

Nothing major, but I realize that the agent did not refactor the code on its own, like a human would. I had to actively do it. If I don’t, and since the Agent takes inspiration from my existing codebase to produce new code, my codebase will exponentially compound in the wrong direction.

An example of a missing refactor that a human would have naturally done: introducing UI components like button and title. Instead, the agent keeps repeating the same code over and over, which increases the code length and makes it hard to read (1k lines of hicup is not handy to browse => if a function does fit on my screen, it’s way too long)

All your tech debt will be repeated over and over forever.

No tech debt will stay hidden.

The view in the handler that is small enough that you did not want to create a namespace for it, the function in a let that you already repeated a few times instead of doing a defn. The Agent will reproduce your mistakes continuously until you actively instruct it to refactor itself, or (often faster) refactor it yourself as soon as you see something looking remotely like a code smell.

Another way to think about it → if you put garbage in, you get garbage out. With a smile because the AI models are always approving you're doing!

Over time, namespaces are becoming larger as the agent does not make architectural changes on its own.

My project has become sizable, something like a small SaaS built by a company with full-time employees. Then it becomes impossible to build full features in one prompt. Instead, I had to target prompts for specific areas of the feature I want to build. I think the Agent is now reading too many tokens to iterate effectively. Something I can observe in the Cursor analytics: prompting for “simple” things now takes a few million tokens. Note that starting from 1 September, I cannot observe it anymore… the feature now comes bundled with Cursor 1.5+, whatever that is 🤦

Anyway, back on building features. Having large namespaces including code smells makes my codebase not AI-ready. Cursor rules and memory are a way to help the agent, but complementary to that, you need to refactor and have a clear architecture. Like for a human developer, your namespace needs to make sense, stay small, have comments and docstring when it matters (Models have the tendency to add useless comments like get-name : return the name. Yeah, thank you, Sherlock… I remove these comments as soon as I see them). Stay ahead on the upkeep of the codebase, make it work better with AI (and for humans too!). Try to think that every time you are starting a new agent, it is like a new junior hire that knows nothing about your codebase. The junior is relentless and has the entire internet in its brain.

Cursor auto mode

I get the Cursor intention here, but unfortunately, since it’s a black box, it’s hard to know when to use it. Sometimes it produces good code, sometimes it’s pure garbage with the linter screaming at you. Clojure being a niche language, I’m guessing that the routing between the model done with the auto is not matching the complexity of the task accurately? For example, refactoring the copy/paste in a test file to turn it into fixtures and reusable helpers is not complex per se, but apparently too complex for the model to pick up by auto. The fact that Auto is not showing which model he ends up using makes it a tool hard to use. I hope that the cursor team will improve it because I think it makes sense not to always use claude-4/gpt-5 for doing something. Reading a file with a top model is just a waste of tokens/money/infrastructure.

Related to that, Cursor is being updated frequently, but there is no clear changelog for it. With all the non-deterministic factors of having an agent coding for you, it adds an extra random factor: do I do things better today, or does Cursor change something 🤔. Coding with AI means you have to accept experimenting with a lot with fast-moving tools anyway, so what’s the problem with another vector of randomness? Still, it’s somewhat frustrating not to know what you are even using. Hopefully, tools will get better over time to trace your AI-assisted work and make suggestions to improve your technique (is there already a company working on that?). Some of the companies building AI agents are already working on thread sharing, which will bring explainability as a side effect.

Cursor auto-complete

The initial feature that was talked about when Cursor got out.

By default, I disable it. I find the current UX distracting with the constant highlights and the mixing of normal auto-completion (powered by LSP, or VS Code itself) with AI suggestions. The only case that it can eventually be useful is for writing docstring (careful of hallucination) and doing repetitive changes that cannot be done quickly with the IDE or that are difficult to explain by prompt. The case where you have to edit 20 function calls to add an argument, or tedious stuff like this.

Clojure-MCP learning curve

The MCP comes with a lot of tools; some are used by the model itself, and others need to be explicitly mentioned in your prompting. I have not tried them all yet, but one I discovered a few weeks ago, and that I now use often, is code_critique. I think it’s the easiest feature you can start with if you have no prior experience with AI-assisted coding. The prompt I use is a variation of “Use code_critique to review a list of files”. It works extremely well, making great suggestions following Clojure best practices, and it can implement them correctly most of the time. This makes me think that I should improve my prompting to drive the agent in the right direction. For reference, the code_critique prompt.

Using multiple agents at the same time

I experimented a bit with that, but not as much as I would like.

How it works: You can start multiple Cursor threads (or use the cursor-cli) to work on different features at the same time. The agent iterates like it does with a single Thread.

I think it’s another game compared to single-agent coding. There are a few requirements to start:

Your codebase needs to be organized enough that you are confident that the 2 agents are not gonna write in the same files. Of course, you also need great documentation to be confident that the Agent knows where is what.
Your mental state needs to be in a deep flow. Writing software is often a task that requires stacking information on top of each other until you get to work on the piece you need. Imagine stacking X times the amount of information to follow the construction of different features. It’s where our childhood playing Starcraft will make you shine!

I do it occasionally for small things like updating dependencies, refactoring a precise file, or writing unit tests on a consolidated feature, but I did not build 2 distinct features at the same time. I think this setup is not great for that because the filesystem with git is unique, as well as the JVM. Theoretically, you can have multiple users interacting with the same REPL, but in practice, they might misguide each other, adding more non-deterministic elements to the party.

I think a more practical setup is to have multiple clones of the same project and multiple REPL starts. So, replicating 2 developers working on different features, merging with the latest branch when they are done.

Another practical scenario, which brings us back to making architectural choices to make the best of AI agents. For example, a better separation of frontend and backend. With HTMX, the boundaries are blurred. With a SPA and a backend, once you get a clear input/output contract between the frontend and the backend, 2 agents can work in parallel to build those.

Still, it’s something I want to experiment with in the future when the tooling becomes more mature, and when my codebase will get big enough for productivity gain.

Last words

I see AI agents with repl-driven development as a powerful tool to significantly increase productivity. I feel that I’m only scratching the surface of what is possible today, and the tooling is still really young. In the near future, software development will be vastly different than what it used to be.

Other Clojure-specific tools I have not used (yet), but could fit your workflow better than Cursor and clojure-mcp:

https://github.com/PEZ/backseat-driver A ChatGPT like in VSCode
https://github.com/karthink/gptel A simple LLM client for Emacs
https://github.com/editor-code-assistant/eca AI pair programming capabilities agnostic of editor

For the Clojurist not using agentic coding, I hope it gives you a reality check on it. For those we do, please share what is working for you 🙂

Permalink

Complex multimethod processing, in Clojure

Code

;; complex_multi_method_processing.clj

(defn arg-type [arg]
  (cond
    (string? arg) :string
    (number? arg) :number
    (fn? arg) :function
    (instance? java.time.LocalDate arg) :date
    :else :other))

(defn dispatcher [args]
  (map arg-type args))

(defmulti process-args (fn [& args] (dispatcher args)))

(defmethod process-args '(:number) [& args]
  '(:number))

(defmethod process-args '(:string :string) [& args]
  '(:string :string))

(defmethod process-args '(:number :number) [& args]
  '(:number :number))

(defmethod process-args '(:number :function :function) [& args]
  '(:number :function :function))

(defmethod process-args '(:number :number :function :function) [& args]
  '(:number :number :function :function))

(defmethod process-args '(:string :function :function) [& args]
  '(:string :function :function))

(defmethod process-args '(:string :string :function :function) [& args]
  '(:string :string :function :function))

(defmethod process-args '(:date :function :function) [& args]
  '(:date :function :function))

(defmethod process-args '(:date :date :function :function) [& args]
  '(:date :date :function :function))

(defmethod process-args '(:other :function :function) [& args]
  '(:other :function :function))

(defmethod process-args '(:other :other :function :function) [& args]
  '(:other :other :function :function))

(defmethod process-args :default [& args]
  (str "Many arguments (" (count args) "): " (clojure.string/join ", " args)))

;; (defmethod process-args '(:default :function :function) [& args] ;; I thought this would work like magic
;;   "Anything and function argument provided")

Notes

rswan

Permalink

The Alchemist's Endgame: My Final Synthesis of p-adic Clojure and Legacy Code.

"I used p-adic distance and functional programming to analyze 50-year-old COBOL.

And surprisingly… it worked better than any traditional parser."

🌪️ The Problem: COBOL is Too Big to Parse

Legacy COBOL systems are beasts:

5 million+ lines of code
Naming conventions like WS-CUST-ID, PRINT-HEADER, ORD-TOTAL
No documentation. No schema. No mercy.

Traditional approaches fall short:

Build a parser → slow, fragile, breaks on dialect variations
Manual analysis → human error, not scalable
Regex matching → misses subtle relationships

What if… we didn't build structure — but discovered it using mathematics?

🌀 The Mathematical Foundation: p-adic Distance

Building on the p-adic ultrametric structures from Part 1, we apply the same prefix-based distance concept to COBOL variable names instead of binary/byte arrays.

The key insight: variables with similar prefixes are "closer" in p-adic space - perfect for discovering naming patterns in legacy code.

Bypassing Abstract Syntax Trees

Traditional parsers build Abstract Syntax Trees (AST) - hierarchical representations of program structure. But for legacy analysis, we need something different: structure discovery rather than structure imposition.

Where ASTs require complete grammar knowledge, ultrametric spaces let us discover relationships through distance mathematics alone. The hierarchy emerges naturally from the data itself.

🚀 Implementation: p-adic Analysis in Clojure

Step 1: Transform COBOL Names into Tokens

(defn tokenize-name [s]
  "Split COBOL variable names on common delimiters"
  (clojure.string/split s #"[.-_]"))

;; Examples:
(tokenize-name "WS-CUST-ID")     ;; => ["WS" "CUST" "ID"]
(tokenize-name "PRINT.HEADER")   ;; => ["PRINT" "HEADER"] 
(tokenize-name "ORD_TOTAL_AMT")  ;; => ["ORD" "TOTAL" "AMT"]

Step 2: Compute p-adic Distance Between Variables

(defn common-prefix-length [a b]
  "Count matching prefix tokens between two token vectors"
  (->> (map vector a b)
       (take-while (fn [[x y]] (= x y)))
       count))

(defn p-adic-distance [base-tokens other-tokens p]
  "p-adic ultrametric distance: closer prefixes = smaller distance"
  (let [prefix-len (common-prefix-length base-tokens other-tokens)]
    (/ 1 (Math/pow p (inc prefix-len)))))

;; Example distances with p=2:
(let [base ["WS" "CUST" "ID"]
      vars [["WS" "CUST" "NAME"]    ;; prefix=2 → distance=1/8
            ["WS" "ORDER" "ID"]     ;; prefix=1 → distance=1/4  
            ["PRINT" "HEADER"]]]    ;; prefix=0 → distance=1/2
  (map #(p-adic-distance base % 2) vars))
;; => (0.125 0.25 0.5)

Step 3: Hierarchical Clustering via `group-by`

The magic happens when we use group-by with prefix length - essentially creating a distance-aware hash-map:

(defn analyze-cobol-structure [base-var var-names p]
  "Cluster COBOL variables by p-adic distance hierarchy"
  (let [base-tokens (tokenize-name base-var)]
    (->> var-names
         (map #(vector % (tokenize-name %)))
         (group-by (fn [[_ tokens]] 
                     (common-prefix-length base-tokens tokens)))
         (sort-by first >)  ;; Sort by depth (deeper first)
         (map (fn [[depth items]]
                {:depth depth
                 :distance (/ 1 (Math/pow p (inc depth)))
                 :members (map first items)
                 :count (count items)})))))

This approach creates what we might call an ultrametric hash-map - where keys aren't just equal or unequal, but exist in a measurable distance relationship. Unlike traditional hash-maps that only support exact key matches, this structure enables proximity-based lookups and hierarchical organization.

Step 4: Real-World COBOL Example

(def cobol-variables
  ["WS-CUST-ID" "WS-CUST-NAME" "WS-CUST-ADDR" "WS-CUST-PHONE"
   "WS-ORDER-ID" "WS-ORDER-DATE" "WS-ORDER-TOTAL"
   "PRINT-HEADER" "PRINT-DETAIL" "PRINT-FOOTER"
   "DB-CONNECT" "DB-CURSOR" "FILE-INPUT" "FILE-OUTPUT"])

(analyze-cobol-structure "WS-CUST-ID" cobol-variables 2)

Output (corrected):

({:depth 2, :distance 0.125, :members ("WS-CUST-ID"), :count 1}
 {:depth 1, :distance 0.25,  :members ("WS-CUST-NAME" "WS-CUST-ADDR" "WS-CUST-PHONE"
                                       "WS-ORDER-ID" "WS-ORDER-DATE" "WS-ORDER-TOTAL"), :count 6}  
 {:depth 0, :distance 0.5,   :members ("PRINT-HEADER" "PRINT-DETAIL" "PRINT-FOOTER"
                                       "DB-CONNECT" "DB-CURSOR" "FILE-INPUT" "FILE-OUTPUT"), :count 7})

🔥 Why This Works Better Than Traditional Approaches

1. No Grammar Required

Traditional parsers need complete COBOL grammar definitions
p-adic approach works on naming patterns alone
Handles dialect variations and legacy quirks gracefully

2. Computational Efficiency

Traditional AST parsing requires recursive tree traversal and grammar validation
Our approach: Direct mathematical computation using prefix comparison
Distance calculation scales linearly with variable name length
No need to build or maintain complex parse trees

3. Discovers Hidden Structure

Reveals relationships invisible to regex matching
Strong triangle inequality ensures consistent groupings
Mathematical foundation provides confidence in results

4. Structure-Preserving Data Access

Unlike traditional hash-maps where get only works with exact keys, our ultrametric approach enables "approximate lookups" - finding the closest structural matches when exact matches fail. This is invaluable for legacy code analysis where variable naming inconsistencies are common.

🔬 From Clusters to System Architecture

The clustering analysis above shows relationships relative to a single base variable. To discover the complete system hierarchy, we analyze multiple base patterns in parallel and merge the results:

(defn discover-system-hierarchy [all-variables base-patterns p]
  "Discover complete system structure by analyzing multiple base patterns"
  (->> base-patterns
       (pmap (fn [base-pattern]
               (let [matching-vars (filter #(clojure.string/starts-with? % base-pattern) 
                                          all-variables)]
                 (when (seq matching-vars)
                   {:pattern base-pattern
                    :subsystem-size (count matching-vars)
                    :internal-structure (analyze-cobol-structure 
                                        (first matching-vars) matching-vars p)}))))
       (remove nil?)
       (sort-by :subsystem-size >)))

;; Discover the complete system architecture
(def base-patterns ["WS-CUST" "WS-ACCT" "WS-ORDER" "DB-" "PRINT-" "ERR-"])
(discover-system-hierarchy cobol-variables base-patterns 2)

This parallel analysis reveals how individual clusters combine into the larger system architecture - transforming local similarity measurements into global structural understanding.

🚀 Scaling Up: Enterprise Analysis

For production systems with thousands of base patterns:

(defn enterprise-cobol-analysis [all-variables p threshold]
  "Automatically discover base patterns and analyze at scale"
  (let [;; Extract potential base patterns from variable prefixes
        base-candidates (->> all-variables
                            (map tokenize-name)
                            (mapcat #(take 2 %))  ; Consider 1-2 token prefixes
                            frequencies
                            (filter #(>= (second %) threshold))  ; Min occurrence threshold
                            (map first))

        ;; Analyze each significant pattern
        analysis-results (discover-system-hierarchy all-variables base-candidates p)]

    {:total-variables (count all-variables)
     :base-patterns-found (count base-candidates)
     :major-subsystems (take 10 analysis-results)
     :coverage-ratio (/ (apply + (map :subsystem-size analysis-results))
                       (count all-variables))}))

📊 Real Results: Revealing the System's Hidden Hierarchy

When applied to a real-world banking system (5M+ LOC, ~50,000 variables), the parallel analysis revealed the complete architectural structure:

WS-* (Workspace Data - 12,000+ variables)
- WS-CUST-* (Customer record - ~800 variables)
- WS-CUST-ID, WS-CUST-NAME, WS-CUST-ADDR-LINE1, ...
- WS-ACCT-* (Account details - ~1,500 variables)
- WS-ACCT-BALANCE, WS-ACCT-TYPE, WS-ACCT-LAST-TRN-DATE, ...
- ... and 10 other major sub-clusters
DB-* (Database Mapping - 9,000+ variables)
- DB-CUSTOMER-TBL-* (Maps to CUSTOMER table)
- DB-TRANSACT-HST-* (Maps to TRANSACTION_HISTORY table)
ERR-* (Error Handling - ~500 variables)
- ERR-MSG-TEXT, ERR-CODE, ERR-MODULE-ID, ...

Key Insights from this Structure:

The parallel analysis automatically identified relationships across different naming conventions
Cross-references between WS-CUST-* and DB-CUSTOMER-TBL-* became visible through distance measurements
Previously undocumented subsystems like ERR-* emerged from the mathematical clustering

🎯 Key Takeaways

Mathematics Reveals Structure: p-adic distance finds patterns without parsing
Functional Programming Scales: Clojure's built-ins handle complexity elegantly
Legacy Systems Have Hidden Gold: Decades-old code contains discoverable patterns
Simple Tools, Powerful Results: group-by + mathematical insight goes far
Beyond Traditional Data Structures: Distance-aware hash-maps open new possibilities

🔗 What's Next?

This approach opens doors to:

Database Schema Analysis: Apply p-adic clustering to SQL table relationships
Code Similarity Detection: Use ultrametric spaces for refactoring candidates
API Consistency Checking: Discover naming pattern violations in REST endpoints
Cross-System Integration: Map legacy COBOL structures to modern APIs using distance-preserving transformations

The mathematical foundation is solid, the implementation is elegant, and the results speak for themselves.

🔄 Full Circle: Second Chances with Better Math

This mathematical approach might even work for other systematic naming conventions I've tackled before - database schemas, API endpoints, even file system hierarchies. The same principles that revealed hidden structure in 50-year-old COBOL could unlock patterns in any domain where naming follows implicit rules.
An experimental implementation is available here.

Have you used unconventional mathematical approaches to tackle complex systems? What patterns might benefit from distance-based analysis? Share your experiences in the comments!
Buy me a coffee if this helped! ☕

Permalink

August 2025 Clojure Support and Q2 Project Updates

Greetings all! We have updates from two Q2 projects and a new report from Toby Crawley who provides ongoing support and maintenance for Clojure. You’ll find a brief description of the all these projects below.

Toby Crawley: Clojure Support and Maintenance
Toby’s report includes links to Clojure Changelogs for March through July 2025 as well as an overview of fixes and updates. He monitors community channels on a regular basis. If you have any issues or questions about Clojars, you can find him in the #clojars channel on the Clojurians Slack, or you can file an issue on the main Clojars GitHub repository.

Karl Pietzrak: Code Combat
This project will focus on adding Clojure(Script) to CodeCombat. See Wiki page at https://github.com/codecombat/codecombat/wiki/Aether

Siyoung Byun: SciCloj Building Bridges to New Clojure Users
Scicloj aims to improve the accessibility of Clojure for individuals working with data, regardless of their programming backgrounds. The project aims to develop standardized templates to encourage greater consistency across the documentation of existing Scicloj ecosystem libraries, making those libraries more robust and user-friendly.

Clojure Support and Maintenance: Toby Crawley

This is an update on the work I’ve done maintaining Clojars in March through July 2025 with the ongoing support of Clojurists Together.

Most of my work on Clojars is reactive, based on issues reported through the community or noticed through monitoring. If you have any issues or questions about Clojars, you can find me in the #clojars channel on the Clojurians Slack, or you can file an issue on the main Clojars GitHub repository.

You can see the CHANGELOG for notable changes, and see all commits in the clojars-web and infrastructure repositories for this period. I also track my work over the years for Clojurists Together (and, before that, the Software Freedom Conservancy.

Below are some highlights for work done in March through July:

I finally addressed issues with running out of memory on occasion. It turned out to be the in-memory session store; we were using aging sessions, but we were generating enough sessions in 48 hours (the session ttl) to exhaust the heap. Adjusting the ttl to 1 hour solved the problem, but a better long-term solution would be to not create a session until a user logs in, as that is all we need a session for. Clojars currently creates a session for each visit to the site.
Clojars was storing uploads in /tmp during deploys, and there is no signal to when a deploy is complete, so we can’t delete them at the end of the deploy. This was causing the server to run out of disk space, so I moved upload storage to a larger partition, and made tmp file cleanup happen more often.
We had some client that was repeatedly connecting to Clojars, then failing TLS negotiation, then trying again. This caused our AWS load balancer expense to increase by several hundred dollars, so I blocked that IP address from accessing Clojars.
I upgraded a few dependencies to address some CVEs.
I worked on spiking out how to implement using Problem Details (rfc9457) to return deploy validation failures to the client. See this issue for more details.

I worked with Ambrose Bonnaire-Sergeant on some security (& other) fixes:

Fixing, then inlining a deps.edn alias we used to override versions to resolve CVEs. We weren’t actually using the alias when building the uberjar, and then realized we didn’t need the alias at all, as those dependencies could be top-level.
Adding a pom.xml to the repository to allow Dependabot to detect vulnerable dependencies.
Importing/adding clj-kondo configurations for dependencies to give better linting.

Code Combat: Karl Pietzrak

Q2 2025 $2k. Report No. 1, Published 9 September 2025

Successfully deployed CodeCombat on our own private cloud (GCP)
Some of CodeCombat’s dependencies are very old:
- for sandboxing user code, Aether, hasn’t been updated since 2017
- closer.js, which contains some Clojure support, hasn’t been touched since 2014.
  We are in the process of updating these so build in a modern context, as many of their dependencies are dead.

Progress update to follow in our next report.

SciCloj Building Bridges to New Clojure Users: Siyoung Byun

Q2 2025 $2k. Report No. 2, Published 14 August 2025

In the time since my previous update I have continued to focus on the community expansion side of the Scicloj initiative. The work and effort continues as a long-term goal alongside other dedicated contributors in the Scicloj community.

Macroexpand 2: Connecting Clojure Data Practitioners

I co-hosted and prepared a three-hour-long online discussion session called Macroexpand 2: Connecting Clojure Data Practitioners as part of Macroexpand gatherings on August 9th. Working with co-host Daniel Slutsky, I planned the event agenda and reached out to Clojurians from diverse backgrounds and positions to invite them to join and share their insights at this gathering.

This initiative represents a step forward in positioning Scicloj not only as a data science community, but as a platform for community expansion, networking, and outreach in the Clojure ecosystem by providing dedicated space and time for discussing and sharing knowledge between individuals and organizations.

Purpose of the Gathering

The gathering aimed to share real-world experiences and challenges, provide networking opportunities for fellow Clojurians while identifying and developing concrete solutions to common problems, and shape clearer direction for follow-up sessions.

Key Outcomes

The gathering brought together participants from different organizations with varied backgrounds, giving us an opportunity to collect valuable insights as individuals, groups, and as a community. The event included participant introductions, lightning rounds where some attendees presented ongoing projects, community expansion efforts, educational outreach initiatives, and discussions about potential tools or platforms for facilitating hiring and job opportunities within the Clojure ecosystem.

Through discussions about success stories of using Clojure in data work, current challenges, and hiring market impressions, we identified several key insights: including the critical importance of integration libraries and tools for other languages (particularly with an example with Python integration).

Other insights included the value of staying connected within the community not only for technological updates but also for accessing new job and collaboration opportunities, and the importance of using streamlined communication channels to maintain community connections. We also explored actionable strategies the community could implement to support and sustain a healthy, thriving Clojure ecosystem.

We partially recorded the gathering, and will publish an edited version soon.

Further Vision

This gathering represents an ongoing initiative rather than a one-time event. Following this successful first session, we are confident that such community discussions are both important and needed. I will continue organizing future gatherings, with formats evolving based on participants inputs and feedbacks along the way. I am excited about the potential impact of this initiative, and look forward to contributing and sharing its outcome in the future.

Macroexpand 2025: Two Upcoming Online Conferences

Macroexpand 2025(two online conferences) will be held in October 2025 hosted by Scicloj. I have been working as a co-organizer to prepare for the conferences, and this initiative will continue.

About the Conferences

Macroexpand-Noj (October 17-18, 2025) focuses on practical applications of the Noj toolkit in real-world data science, featuring talks on any data analysis related projects and methology, updates on the tool, documentation efforts. And Macroexpand-Deep (October 24-25, 2025) is dedicated to AI system research and applications in Clojure, exploring deep learning networks, large language models, vector embeddings, and supporting infrastructure.

Current Work

As one of the organizers, I am actively preparing and organizing both conferences. I published a post about the upcoming conferences on Clojure Civitas, including a call for speakers.

I will continue to assist potential speakers in preparing talks, brainstorm talk ideas with interested participants, reach out to community members, help others conceptualize their projects for presentation, contribute to Scicloj organizing team discussions, and plan the upcoming conferences. This remains an ongoing initiative with significant potential for community impact.

Permalink

August 2025 Clojars Support and Q2 Project Updates

Greetings all! We have updates from two Q2 projects and a new report from Toby Crawley who provides ongoing support and maintenance for Clojars. You’ll find a brief description of the all these projects below.

Toby Crawley: Clojars Support and Maintenance
Toby’s report includes links to Clojars Changelogs for March through July 2025 as well as an overview of fixes and updates. He monitors community channels on a regular basis. If you have any issues or questions about Clojars, you can find him in the #clojars channel on the Clojurians Slack, or you can file an issue on the main Clojars GitHub repository.

Karl Pietzrak: Code Combat
This project will focus on adding Clojure(Script) to CodeCombat. See Wiki page at https://github.com/codecombat/codecombat/wiki/Aether

Clojars Support and Maintenance: Toby Crawley

This is an update on the work I’ve done maintaining Clojars in March through July 2025 with the ongoing support of Clojurists Together.

Below are some highlights for work done in March through July:

I finally addressed issues with running out of memory on occasion. It turned out to be the in-memory session store; we were using aging sessions, but we were generating enough sessions in 48 hours (the session ttl) to exhaust the heap. Adjusting the ttl to 1 hour solved the problem, but a better long-term solution would be to not create a session until a user logs in, as that is all we need a session for. Clojars currently creates a session for each visit to the site.
Clojars was storing uploads in /tmp during deploys, and there is no signal to when a deploy is complete, so we can’t delete them at the end of the deploy. This was causing the server to run out of disk space, so I moved upload storage to a larger partition, and made tmp file cleanup happen more often.
We had some client that was repeatedly connecting to Clojars, then failing TLS negotiation, then trying again. This caused our AWS load balancer expense to increase by several hundred dollars, so I blocked that IP address from accessing Clojars.
I upgraded a few dependencies to address some CVEs.
I worked on spiking out how to implement using Problem Details (rfc9457) to return deploy validation failures to the client. See this issue for more details.

I worked with Ambrose Bonnaire-Sergeant on some security (& other) fixes:

Fixing, then inlining a deps.edn alias we used to override versions to resolve CVEs. We weren’t actually using the alias when building the uberjar, and then realized we didn’t need the alias at all, as those dependencies could be top-level.
Adding a pom.xml to the repository to allow Dependabot to detect vulnerable dependencies.
Importing/adding clj-kondo configurations for dependencies to give better linting.

Code Combat: Karl Pietzrak

Q2 2025 $2k. Report No. 1, Published 9 September 2025

Successfully deployed CodeCombat on our own private cloud (GCP)
Some of CodeCombat’s dependencies are very old:
- for sandboxing user code, Aether, hasn’t been updated since 2017
- closer.js, which contains some Clojure support, hasn’t been touched since 2014.
  We are in the process of updating these so build in a modern context, as many of their dependencies are dead.

Progress update to follow in our next report.

SciCloj Building Bridges to New Clojure Users: Siyoung Byun

Q2 2025 $2k. Report No. 2, Published 14 August 2025

Macroexpand 2: Connecting Clojure Data Practitioners

Purpose of the Gathering

Key Outcomes

We partially recorded the gathering, and will publish an edited version soon.

Further Vision

Macroexpand 2025: Two Upcoming Online Conferences

Macroexpand 2025(two online conferences) will be held in October 2025 hosted by Scicloj. I have been working as a co-organizer to prepare for the conferences, and this initiative will continue.

About the Conferences

Current Work

As one of the organizers, I am actively preparing and organizing both conferences. I published a post about the upcoming conferences on Clojure Civitas, including a call for speakers.

Permalink

Too many tools: How to manage frontend tool overload

People naturally struggle to perform actions and continue day-to-day tasks with an abundance of choices . Too many indistinguishable choices reduce clarity in the decision-making process and simplicity in every individual task. This abundance of options has negatively affected every field of software engineering, including frontend development.

In the past, developing a web app was simple, especially with the limited availability of stable, simple frontend development tools. In 2010, any developer could easily start building a web app to solve any general business requirement using JavaScript, jQuery, PHP, HTML, and CSS. They could use Adobe Dreamweaver or any syntax-highlighting-enabled code editor, and quickly ship the final product via FileZilla FTP.

Now, developers need to construct an optimal development toolkit for themselves by choosing tools from dozens of frontend frameworks, libraries, languages, build tools, testing frameworks, package managers, code editors, deployment tools, and AI tools, even before writing a single code line.

Even after initiating a codebase, they have to observe rapidly changing tools to constantly improve codebases by also migrating to new tools if needed.

The abundance of tools has made modern frontend development chaotic. And it’s started negatively affecting the frontend developer experience.

Let’s study how the growth of frontend development created so many tools, and how to manage tool overload.

Why did the early frontend development era (2009–2013) feel simpler?

Frontend development went through a special period where any developer could easily build and ship a generic product without facing tool overload. In the 2009–2013 era, frontend development felt simpler compared to its current state.

That’s due to the following factors:

Constrained defaults created a shared baseline

Ask most experienced, senior web developers from the 2009–2013 era about the tools they used in that period. The majority will mention JavaScript, jQuery, PHP, HTML, and CSS. These were the only popular, easily accessible, fully-featured, deployment-friendly, simple tools that were available.

ASP.NET, JSP, and Python were available for web development in this era, but they couldn’t compete with the previously-mentioned stack due to a lack of public awareness, development complexity, limited developer productivity-focused features, and limited third-party hosting options.

Developers could easily initiate a web project without facing a recommended tool overload since those tools were the industry default.

Developer expectations were modest, and projects weren’t complex

“The past programmers were the real programmers” is true.

Past developers didn’t seek libraries to avoid just ten lines of code, code generators or AI tools to skip reading API documentation, and frameworks/boilerplates to skip mastering codebase architecture .

They were always ready to learn, experiment, discuss, and code. The modest developer expectation was a major reason to stick to the existing tool set.

Almost all past general web apps had simple codebase architectures, UIs, and implementations. Developers didn’t want to seek tools to reduce project complexity:

Standards were simple, and the experiments were fewer

In this era, browsers didn’t implement the same behavior for several JavaScript, HTML, and CSS features (e.g., browsers had different CSS vendor prefixes).

Browser compatibility was a major factor to consider during development, but all browsers implemented a generic feature set based on well-accepted web standards to build simple web apps:

CSS2 and CSS3 versions’ features were enough to achieve fair usability in UIs
XHTML wasn’t as semantically-enhanced as HTML5, but it was simple and had enough features
ECMAScript didn’t bring frequent features and enhancements, but had enough stable features under ES3 and ES5
The XMLHttpRequest API for AJAX programming, and other basic web APIs, like the DOM API

Browsers didn’t have frequent releases with so many experimental web standard implementations and deprecations. Developers didn’t have to learn advanced web standards every month to make sure that their codebases stayed up to date.

The trade-off: limited flexibility, but less decision fatigue

There were a few choices while selecting frontend development tools, and they also came with limited features. Here are all the widely used frontend development tools in the 2009–2013 period:

UI design: XHTML, CSS, jQuery UI, and Bootstrap
UI programming: JavaScript, jQuery, PHP, JSP, and ASP.NET (to generate HTML)
Code editors: Adobe Dreamweaver, Sublime Text, Notepad++. Developers could use any code editor, even the one that comes with the operating system, since they had modest expectations from a code editor
Deployment: FTP (FileZilla) to a third-party host or using a Bash script if developers have their own servers

Most developers used JavaScript, jQuery, PHP, HTML, and CSS without build tools, package managers, frameworks, and many libraries to build any general web app. Their choices were limited, but they didn’t spend hours researching which tool to use. They shipped products pretty fast:

What caused the explosion of tools post-2015?

After going through the simplest frontend development era from 2009 to 2013, the developers entered an innovative era in the 2014–2015 period.

A silent war waged between traditional, simple web development and the growth of single-page apps (SPAs). The exponential growth of frontend development tools was mostly due to the following factors:

Rapid ECMAScript releases and evolving web standards

ECMAScript had only three major specification releases (ES2, ES3, and ES5) between the first release of the JavaScript implementation in the Netscape Navigator 3.0 browser and 2015. ECMAScript didn’t create any major releases with new language features during the 2009–2015 period, so that period was undoubtedly the most stable period of JavaScript.

No build tools were required to fix JavaScript version compatibility since web apps used stable, mature JavaScript ES5 and earlier features.

ECMAScript started creating new major releases every year right after 2015 with the release of the notable ES6. Thousands of polyfills, several transpilers, build tools, and package managers were born to fix JavaScript compatibility issues:

npm search keyword

WebSockets became popular. WebAssembly and WebRTC were introduced. Many built-in web APIs were improved with feature extensions (e.g., the DOM API integrated various observer APIs) after 2015. The developer community built many tools to improve the developer experience with new web standard implementations.

SPA and TypeScript adoption fuel ecosystem growth

In early frontend development, developers used the page-based technique for building web apps. They decomposed the app into multiple pages, linked relevant scripts and stylesheets to each page, and adjusted routing by configuring web servers.

This approach was beginner-friendly and didn’t require additional UI development, state management, routing, caching libraries, or frontend frameworks . Vanilla JavaScript and jQuery were enough to handle the whole client-side app logic.

Around 2016, SPAs (single-page apps) became popular with the RESTful API communication pattern, and the rapid growth of React, Angular, Vue, and Svelte. The JavaScript developer community and tech companies started building state management solutions, routers, UI kits, boilerplates, and various libraries for popular frontend libraries.

Several frameworks, including Remix, Next.js, Gatsby, and Docusaurus, were born to turn the popular React library into a framework to compete with frameworks like Angular and Vue. The popularity of TypeScript also created many packages, such as type collections, project starter kits, validators, and code generators. TypeScript development made other tools like module bundlers, build tools, and linters popular, as well.

SPA and TypeScript adoption created a spike in the new tool innovation and development timeline. Here is how a few development choices in 2011 have turned into so many in 2025:

Comparison between Stack Overflow Survey’s 2011 web development options (left) and 2025 options (right). See how a few choices expanded to many and created a tool fragmentation in the developer ecosystem

Open-source funding and GitHub hype-driven tool proliferation

Open-source development was a popular, effective development strategy from the initialization of the free software movement, initiated by Richard Stallman in 1983. This model became popular in top tech companies and worldwide developer communities after 2015, with the promising growth of GitHub as the world’s largest Git repository hosting platform.

Top tech companies started:

Open-sourcing tools they’d built for internal development requirements to boost the organization’s popularity and let the open-source community improve the project
Motivating employees to innovate new open-source projects and contribute back to the open-source tools the company uses
Funding open-source projects

Developers started using open-source contributions to show their programming expertise and find new career opportunities. Some developers became tool maintainers by contributing to open-source full-time.

Meanwhile, GitHub released more features and turned the traditional Git hosting platform into a futuristic, fully-featured social coding and deployment platform, and introduced more tools under GitHub Actions.

The evolved open-source development was a major reason for the rapid growth of new frontend development tools.

Generative AI accelerated the creation of new tools

By pre-training AI with existing human-created knowledge, generative AI introduced a promising solution. Developers could now automate literally any manual task in the software development lifecycle.

ChatGPT’s explosion in 2022 was just the start. Devs started using publicly available generative AI models and APIs to create numerous automation tools that help them ship products faster after:

Coding: AI-powered code editors and plugins that can write or improve code based on natural language instructions
Design: Diagram generators, sketch/prototype generators, and AI software architect agents
CI: AI-powered code reviewing bots, security testers, and test generators
Conversion: Design to code tools, e.g., Builder.io, a Figma to code tool, and idea to product tools
Project management: Task estimators, task description generators, and task scheduling tools

Thousands of AI tools came to the software market, promising productivity enhancements. The result has been an AI tool war that’s leading us towards AI tool overload.

The dominant ChatGPT in 2022 started losing popularity due to new competitor tools, creating an AI tool fragmentation:

state of javascript update showing tools

How many tools are we really talking about?

Is the number of tools enough to reduce clarity in modern frontend development decision-making?

According to Wikipedia, there are more than 3.1 million packages on the NPM registry. The majority of them are frontend development tools, including libraries, polyfills, frameworks, utilities, etc. If we sum up all the frontend development tools that exist outside of NPM to NPM packages, we indeed get millions. However, only a small percentage of these are actually popular and usable.

Studying developer surveys, like the State of JavaScript and the Stack Overflow annual developer survey, and exploring technical newsletters, like JavaScript Weekly, are good approaches to estimate the number of tools that we can include in modern developer decision-making.

The following table summarizes the available popular frontend tools based on popular developer surveys:

Category	Popular tools
Frontend libraries/frameworks	React, Angular, Vue, Svelte, Preact, Lit, Solid, Alpine.js, HTMX, Qwik, Stencil.js, Remix, Nuxt.js, Flutter, React Native, Astro, Gatsby, Docusaurus,
UI and styling	Tailwind, Bootstrap, Ant Design, Materialize, MUI, Bulma, Foundation, Open Props, UnoCSS, Semantic UI, Pico.CSS, Chakra UI, Blaze UI, Vuetify, daisyUI, Quasar, styled components, Emotion, clsx, cva
Building and packaging	Vite, Webpack, esbuild, Rollup, Bun, Parcel, Turbopack, Rspack, Rsbuild, Farm
Code editors	VsCode, JetBrains IDEs, Visual Studio, Vim, Notepad++, Neovim, Claude Code, Zed, Sublime Text, VsCodium, Windsurf, Cursor, Fleet, LiteXL
Frontend languages	JavaScript, TypeScript, CoffeeScript, Flow, Elm, Dart, Kotlin, Python, Go, PureScript, ClojureScript
Testing	Jest, Vitest, Testing Library, Mocha, Jasmine, Cypress, Playwright, Pupeteer, Chai, Selenium, Storybook
Desktop/mobile app development (for turning web apps to native desktop/mobile apps)	Electron, Nw.js, Tauri, Wails, NodeGUI, Flutter, React Native, Neutralinojs, Compose Multiplatform, Electrino, Webview C++, Electrobun, Buntralino, Cordova, Ionic
Package managers	npm, pnpm, Yarn, Bun, ni, Cotton, Deno
State management	Redux, Zustand, Jotai, Recoil, Valtio, MobX, ngRx, NGXS, RxJs, Akita, Pinia, Vuex, Nano Stores
Server communication	Fetch, Axios, Got, SuperAgent, Ky, Needle, Wretch, xior, WebSocket, Socket.IO, Pusher, Apollo, gRPC

There are about 10 frontend development tool categories, and each category has more than 10 options.

That puts us at about 100+ popular frontend development tools. Apart from these primary choices, there are millions of dependency packages on NPM and other package registries. Moreover, the modern AI tool war started introducing several developer-productivity-focused, competitive AI tools every single day. Similar to the growth in NPM registry, this flooded the ecosystem with AI-powered developer tools.

This number is enough to complicate modern frontend development, harden decision-making, and affect the overall developer experience!

Why the abundance of tools can hurt DX

New, innovative tools bring more flexibility to frontend development choices and improve developer experience, but tool overload can negatively affect developer experience:

Decision fatigue: too many choices slow down projects

In 2010, developers didn’t have to research the optimal frontend toolkit for building a web app — they could build anything with a small collection of tools. But now, developers have to decide which framework, libraries, build tool, boilerplate, code editor, UI kit, programming language, styling language, and test runner to use before even writing a single code line.

The abundance of tools slows down project initialization. Their competitive, indistinguishable features make it difficult to know where to begin.

Steep learning curves, especially for newcomers

Each popular frontend development tool comes with different concepts, features, and development practices. They often put a unique, thick abstraction layer over built-in web APIs and development techniques. Mastering any frontend tool is hard, especially for newcomers.

Frontend development tools usually improve productivity in large-scale projects, but the unique concepts that come with each tool complicate modern frontend development.

For example, beginners can implement a simple, dynamic status text with native web technology as follows, without learning new abstract concepts over the traditional DOM:

<div class="app">
  <span id="status">Wait...</span>
</div>

<script>
  setTimeout(() => {
    document.querySelector('#status')
      .innerText = 'Ready.';
  }, 2000);
</script>

In React, beginners will have to learn the component lifecycle, JSX, Hooks, and build tools to do the sample implementation:

import React, { useState, useEffect } from 'react';

export function App() {
  const [label, setLabel] = useState('Wait...');

  useEffect(() => {
    const timer = setTimeout(() => {
      setLabel('Ready.');
    }, 2000);
    return () => clearTimeout(timer);
  }, []);

  return (
    <div className='App'>
      <span>{label}</span>
    </div>
  );
}

Ecosystem fragmentation leads to unstable codebases

Frontend development tools come with different APIs, coding styles, paradigms, and dependencies, and make frequent new feature implementations and deprecations. In a fragmented ecosystem, where too many tools exist to solve literally anything, developers often tend to integrate various rapidly changing tools with their codebases to boost their coding productivity.

As a result, frequently changing, unstable features, different coding styles, APIs, paradigms, and patterns mix up and affect the stability and quality of codebases.

Tool churn creates maintenance and migration costs

Frontend tools come with various promising benefits and proven DX enhancements. Dev teams often integrate these new tools by discarding existing ones

For example, a sample web app project initiated in 2005 can go through the following rapid frontend development tooling additions and removals, known as tool churn:

Year	Tool churn
2005	Uses vanilla JavaScript, HTML, PHP, and CSS
2010	Integrate jQuery for AJAX and DOM manipulation
2011	Use jQuery UI elements
2015	Use React components in HTML/PHP pages
2016	Migrate to modern React using Create React App and use a Node.js RESTful backend Use Mocha for testing
2017	Rewrite UI using MUI components
2018	Started using Redux
2019	Migrate to TypeScript
2020	Use Jest for testing Replace Redux with Recoil Start using Rollup
2021	Use React SWR for caching Switch to React context API
2022	Start using Vite
2023	Replace React SWR with TanStack Query
2024	Migrate to Next.js

In the above example, tool churn is low before 2015 and skyrocketed after 2015. The above example focuses on UI development-focused tool churn, but tool churn happens with code editors, extensions/plugins, and any other development tool.

Tool churn is time-consuming, challenging, costly, and is burning out developers at work with unstable codebases.

How to manage DX in an era of tool overload

Developers love to stay up-to-date with technology and learn new technologies, but they don’t like to get burned out at work due to an overload of modern tools and tool churn.

Devs don’t like to dive deep into documentation and online communities of dozens of dependencies before choosing a dependency package. We all like to get benefits from modern tools, but don’t like to let them ruin the developer experience.

Here is how you can manage developer experience with tool overload:

Lean on community conventions and proven ecosystems

Design patterns become popular not solely because a top tech giant uses them. Specific technologies gain developer attention not solely because they’re backed by a top tech giant.

The community success of a specific development convention or technology is a key reason behind its popularity. For example, Node.js became so popular not solely because LinkedIn, Netflix, and eBay started using it . Its proven productivity was a key reason behind its popularity.

This is true for all popular technologies, conventions, and development techniques such as MVC, REST, GraphQL, SPA, NoSQL, Docker, Vue, and Next.js.

Adhering to top community conventions and proven ecosystems guarantees a good developer experience for general projects. For example, React, Vue, and Angular are proven tools for general use cases:

According to the State of JavaScript 2024 survey results, React, Vue, and Angular are the top three UI libraries

Pick tools with healthy documentation and support, not just hype

A sudden spike in hype of a specific technology due to an effective marketing strategy doesn’t always guarantee its long-term success. Apart from the quality of the solution, healthy documentation, and community support guarantee long-term success. Imagine what React, Vue, or Angular would be without proper documentation and developer support, or Node.js without up-to-date API documentation?

With healthy documentation and good developer support, developers can:

Do a quick feasibility study before deciding on a tool without diving too much into implementation documentation, GitHub issues, or release histories
Avoid time-consuming migration decisions by requesting developer support to solve critical issues.
Upgrade to new versions productively using official migration guides and tools:
React 19 came with an official, well-written migration guide to let developers smoothly transition to the new version

Optimize for team needs and stability over novelty

The goal of maintaining a stable, healthy codebase is to offer a usable, stable product for users. Using new tools doesn’t always guarantee codebase stability and satisfy the team’s needs.

Sometimes, an old but well-maintained tool can work better than a new one. Choosing stable, mature, well-maintained tools by discussing with the team improves overall developer experience within the team. You don’t need to choose a new, futuristic, rapidly changing tool just to go with trends.

Adopt change only when the benefit is clear and measurable

Mindlessly using trending tools, conventions with frequent migrations and rewrites typically don’t add any benefits to codebases or developer experience. These habits just increase tool churn, waste developer hours, make codebases unstable, and negatively impact everyone’s productivity.

Evaluating specific tools with experiments or quick research — then integrating them only if they bring clear benefits — will help reduce unwanted tool churn.

For example, in an open-source project that I maintain, I still use a Rollup-based build script added by a contributor about four years ago. I don’t need to look for another build tool since switching the build tool doesn’t bring any measurable benefit to the project:

A part of the Neutralino.js Rollup-based build script on GitHub

Encourage continuous learning without chasing every new trend

Continuous learning to be up-to-date and chasing every new trend are two different things. Learning new technology using newsletters, blogs, podcasts, and techtalks is great. But if we start randomly chasing new technology, then there is a problem.

For example, if you notice trending CSS-in-JS libraries, you shouldn’t instantly think about introducing them to your codebases. Instead, you can learn their benefits with experiments and consider using them only if you face issues with the current styling technique and notice an optimal situation to apply.

Learning about — but not chasing — new trending tools makes you an up-to-date developer. Just don’t go overboard.

Conclusion

The growth of development tools increased the flexibility of frontend development approaches and improved the developer experience. However, the abundance of tools has made everything chaotic and started negatively affecting developer experience, creating decision fatigue, steep learning curves, and increasing tool/code churn.

It’s the developer’s right to choose the right tool for the right scenario or decide not to use a tool for a specific scenario at all. Developers define their own DX with their choices and expectations.

When everyone uses a specific trending tool, no one is forcing you to use it unless your company recommends it. But still, you can discuss concerns about the recommended tool. Choosing the right tool based on the development scenario, regardless of the hype, brings a better developer experience for everyone.

Focus on clarity, stability, team satisfaction, and continuous learning, but don’t chase trending technologies and let them ruin the team’s developer experience .

The post Too many tools: How to manage frontend tool overload appeared first on LogRocket Blog.

Permalink

A fake Clojure Object equals to what you want

Imagine you write a unit test where you compare two maps:

(is (= {:some {:nested {:id ...}}}
       (get-result)))

Turns out, this map has a random UUID deep inside so you cannot blindly compare them with the “equals” function:

(defn get-result []
  {:some {:nested {:id (random-uuid)}}})

(is (= {:some {:nested {:id ???}}}
       (get-result)))

This won’t work because the nested :id field will is random every time.

What to do? Most often, people use libraries for fuzzy matching, DSLs, etc. Well, a single case still doesn’t mean you should drag in another library. Apparently, it could be solved with a dummy object that equals to any UUID:

(def any-uuid
  (reify Object
    (equals [_ other]
      (uuid? other))))

(= any-uuid (random-uuid))
true

(= any-uuid 42)
false

Now replace the value in your map, and the test will pass:

(is (= {:some {:nested {:id any-uuid}}}
       (get-result)))

It works the same for numbers:

(def any-number
  (reify Object
    (equals [_ other]
      (number? other))))

(= any-number 42)
true

(= any-number -99)
true

The only caveat is, this dummy object must be the first one in the = function. It does equal to any object on the left but the opposite is false: a normal UUID doesn’t equal to a fake UUID.

For the rest, it short and trivial, and no other libraries are needed.

Permalink

Emmy, the Algebra System: Classical Mechanics Chapter One

Permalink

Dragon Curve Fractal - Complex & Bits

A familiar image is re-discovered in a sequence of complex numbers, and the binary bits of natural numbers.

Permalink

Clojure Deref (Sep 9, 2025)

Welcome to the Clojure Deref! This is a weekly link/news roundup for the Clojure ecosystem (feed: RSS).

Upcoming Events

Clojure/Conj 2025: Schedule is now live
Clojure South 2025: Tickets On Sale
Macroexpand 2025: Oct 17-18, 24-25

Podcasts, videos, and media

Mailman: event driven made easy (by Wout Neirynck) - London Clojurians
Calling a function before it’s defined, in Clojure - Clojure Diary

Blogs, articles, and projects

nREPL 1.4 - Bozhidar Batsov
Zero runtime cost styles in ClojureScript | Roman Liutikov, Software Engineer - Roman Liutikov
C4 - Classic Clojure Compiler Contemplation - David Miller
C4 - AST me anything - David Miller
C4 - Symbolic of what? - David Miller
C4 - ISeq clarity - David Miller
C4 - Functional anatomy - David Miller
2025 Board Nominations and Our Annual Meeting - Kathleen Davis
OSS updates July and August 2025 - Michiel Borkent
Developing a Space Flight Simulator in Clojure - Jan Wedekind
Relaunching Yakread: an algorithmic reading app - Jacob O’Bryant

Libraries and Tools

New releases and tools this week:

fulcro-rad 1.6.16 - Fulcro Rapid Application Development
fulcro-rad-semantic-ui 1.4.7 - Semantic UI Rendering Plugin for RAD
devcontainer-templates 2.0.0 - Devcontainer templates for Clojure
babashka 1.12.208 - Native, fast starting Clojure interpreter for scripting
dot-clojure 1.4.0 - My .clojure/deps.edn file
aleph 0.9.2 - Asynchronous streaming communication for Clojure - web server, web client, and raw TCP/UDP
clojure-cli-config 2025-09-04 - User aliases and Clojure CLI configuration for deps.edn based projects
project-templates 2025-09-04 - Clojure CLI Production level templates for seancorfield/deps-new
datomic-mcp-server - 🚀 Datomic MCP Server
monkeyci 0.19.8 - Next-generation CI/CD tool that uses the full power of Clojure!
qclojure 0.15.0 - A functional quantum computer programming library for Clojure with backend protocols, simulation backends and visualizations.
next-jdbc 1.3.1070 - A modern low-level Clojure wrapper for JDBC-based access to databases.
stripe-clojure - Clojure SDK for the Stripe API.
honeysql 2.7.1350 - Turn Clojure data structures into SQL
unused-deps - Find unused deps in a clojure project
sfv - A 0-dependency Clojure library for parsing and generating Structured Field Values for HTTP (RFC 9651/8941)
eca 0.50.2 - Editor Code Assistant (ECA) - AI pair programming capabilities agnostic of editor
deep-diamond 0.36.1 - A fast Clojure Tensor & Deep Learning library
kindly 4-beta19 - A small library for defining how different kinds of things should be rendered
fulcro 3.9.0-rc10 - A library for development of single-page full-stack web applications in clj/cljs
clay 2-beta54 - A REPL-friendly Clojure tool for notebooks and datavis

Permalink

Tutorial on Advanced P-adic Structures with Clojure: Monadic and Parallel Enhancements.

🔧 Learning by Rebuilding: This intentionally reinvents some familiar patterns (with-open, etc.) to explore mathematical computing applications. The real novelty is in the p-adic mathematics - the infrastructure is just educational exploration! 🧮

Introduction: Connecting the Dots 🔗

Welcome to the continuation of our high-performance computing journey! In our previous tutorial on 3D spatial data sorting with Morton codes, we explored parallel computation and memory-efficient data structures. Today, we fulfill that promise by applying these advanced techniques to p-adic structures, creating a powerful fusion of mathematical theory and high-performance computing.

This tutorial builds directly upon concepts from our previous p-adic introduction and the parallelization concepts from the Morton codes tutorial.

What Makes This a Natural Progression 🌟

The beauty of functional programming lies in its ability to abstract computational patterns. The techniques we developed for spatial data processing translate beautifully to mathematical computation, demonstrating the power of well-designed abstractions.

From Spatial Sorting to Mathematical Computation 📊

In our Morton codes tutorial, we mastered:

Parallel chunk processing for spatial data
Memory-efficient data representations
Thread pool management for concurrent operations
Performance optimization techniques

Now we apply these same principles to p-adic mathematics, creating a robust computational framework that handles complex mathematical operations with the same efficiency we achieved for spatial data sorting.

The transition from spatial indexing to mathematical computation isn't just about changing domains - it's about recognizing that many computational patterns are universal. Whether we're sorting 3D points or computing p-adic valuations, we need efficient data structures, parallel processing, and robust error handling.

The Parallelization Promise Fulfilled ⚡

(defn find-critical-points-monadic [vectorized p parallel-level]
  "Monadic Critical Point Detection - Fixed thread pool usage bug"
  (with-managed-resource
    (->ThreadPoolResource parallel-level)
    (fn [thread-pool]
      (try
        (let [chunk-size (max 1 (quot (count vectorized) parallel-level))
              chunks (partition-all chunk-size vectorized)
              
              futures (mapv
                        (fn [chunk]
                          ;; BUG FIX 1: Specify the managed thread pool
                          (CompletableFuture/supplyAsync
                            #(keep
                               (fn [v]
                                 (let [grad-result (discrete-gradient-simple v)
                                       val-result (p-adic-valuation-monadic v p)]
                                   ;; Return the result only if both calculations succeed
                                   (when (and (is-ok? grad-result) (is-ok? val-result))
                                     {:vector v
                                      :gradient (extract-value grad-result)
                                      :p-adic-valuation (extract-value val-result)})))
                               chunk)
                            thread-pool))
                        chunks)
              
              results (mapcat #(.get ^CompletableFuture %) futures)]
          (ok (vec results)
              {:critical-count (count results)
               :parallel-level parallel-level}
              [{:level :info :message (str (count results) " critical points found")}]))
        (catch Throwable t
          (err t {} [{:level :error :message "Critical point detection error"}])))))

Just as we parallelized spatial indexing, we now parallelize p-adic computations, demonstrating how general-purpose parallel patterns can be applied to mathematical domains.

The key insight is that mathematical operations often exhibit natural parallelism. Computing p-adic valuations across large datasets, finding critical points in ultrametric spaces, and performing matrix operations all benefit from the same parallel processing techniques we used for spatial data.

Enhanced Architecture: Monads Meet Parallelism 🗂️

Modern functional programming teaches us that composition is key to building robust systems. By combining monadic error handling with parallel computation, we create a framework that's both mathematically rigorous and computationally efficient.

Combining Functional and Parallel Paradigms 🔄

Our architecture now integrates:

Value extraction with extract-value and extract-error
Metadata tracking for computational context
Logging capabilities for debugging and analysis
Timing information for performance monitoring

This isn't just about adding features - it's about creating a coherent system where each component enhances the others. Monadic composition ensures that errors propagate cleanly through parallel computations, while metadata tracking gives us insights into performance bottlenecks.

Monadic Operations 💎

;; bind with logs
(defn bind [r f]
  (if (is-ok? r)
    (try
      (let [result (f (extract-value r))
            combined-logs (concat (:logs r) (:logs result))]
        (if (is-ok? result)
          (->OkResult (extract-value result) 
                      (merge (:metadata r) (:metadata result))
                      combined-logs)
          (->ErrResult (extract-error result)
                       (merge (:metadata r) (:metadata result))
                       combined-logs)))
      (catch Throwable t 
        (->ErrResult t 
                     (:metadata r)
                     (conj (:logs r) {:level :error :message (.getMessage t)}))))
    r))

(defn mapr [r f]
  (bind r (fn [v] (ok (f v) {} [{:level :info :message "Map operation"}]))))

;; Monad including performance metrics
(defn timed-bind [r f]
  (if (is-ok? r)
    (let [start-time (System/nanoTime)]
      (try
        (let [result (f (extract-value r))
              end-time (System/nanoTime)
              duration-ms (/ (- end-time start-time) 1000000.0)
              timing-metadata {:execution-time-ms duration-ms}]
          (if (is-ok? result)
            (->OkResult (extract-value result)
                        (merge (:metadata r) (:metadata result) timing-metadata)
                        (concat (:logs r) (:logs result)))
            result))
        (catch Throwable t (err t (:metadata r) (:logs r)))))
    r))

(defmacro mlet
  "Extended monadic let: Automatically collects logs and metrics"
  [bindings & body]
  (if (empty? bindings)
    `(ok (do ~@body) {} [{:level :info :message "mlet completion"}])
    (let [[sym expr & rest] bindings]
      `(timed-bind ~expr (fn [~sym] (mlet ~rest ~@body))))))

The timed-bind operation automatically tracks execution time, while mlet provides a clean syntax for monadic composition with automatic logging and timing.

These operations form the foundation of our computational framework. By wrapping mathematical operations in monadic contexts, we gain automatic error handling, logging, and performance monitoring without cluttering our mathematical code.

Advanced Resource Management 🛡️

One of the biggest challenges in high-performance computing is resource management. Memory leaks, thread pool exhaustion, and resource contention can quickly derail even the most elegant algorithms.

Managed Resource Protocol 🔧

(defprotocol ManagedResource
  (acquire [this] "Acquires the resource")
  (release [this resource] "Releases the resource")
  (describe [this] "Describes the resource"))

(defrecord ArenaResource [arena-type]
  ManagedResource
  (acquire [_] 
    (case arena-type
      :confined (Arena/ofConfined)
      :shared (Arena/ofShared)
      :auto (Arena/ofAuto)))
  (release [_ arena] 
    (when arena (.close ^Arena arena)))
  (describe [_] (str "Arena resource of type: " arena-type)))

(defrecord ThreadPoolResource [thread-count]
  ManagedResource
  (acquire [_] (ForkJoinPool. thread-count))
  (release [_ pool] 
    (when pool 
      (.shutdown ^ForkJoinPool pool)
      (.awaitTermination ^ForkJoinPool pool 5 java.util.concurrent.TimeUnit/SECONDS)))
  (describe [_] (str "ThreadPool with " thread-count " threads")))

Our resource management system handles:

Memory arenas for off-heap memory management
Thread pools for parallel computation
Automatic cleanup with proper error handling

The protocol-based approach gives us flexibility while ensuring consistent resource management patterns. Whether we're dealing with memory arenas or thread pools, the same acquisition and cleanup patterns apply.

Safe Resource Usage 🛟

(defn with-managed-resource [resource-spec body-fn]
  "Manages a resource using the resource-spec and executes the body-fn"
  (let [start-time (System/nanoTime)]
    (try
      (let [resource (acquire resource-spec)
            acquisition-time (- (System/nanoTime) start-time)]
        (try
          (let [result (body-fn resource)
                execution-time (- (System/nanoTime) start-time acquisition-time)]
            (log-result 
              (if (satisfies? ResultType result) result (ok result))
              :info
              (str "Resource management: " (describe resource-spec)
                   " acquired=" (/ acquisition-time 1000000.0) "ms"
                   " executed=" (/ execution-time 1000000.0) "ms")))
          (finally
            (try
              (release resource-spec resource)
              (catch Throwable release-ex
                (println "Warning: Resource release error:" (.getMessage release-ex)))))))
      (catch Throwable t 
        (err t {} [{:level :error :message "Resource management failed"}])))))

This ensures resources are properly acquired and released, even in case of exceptions.

The macro approach provides a clean, idiomatic way to handle resources while maintaining the functional programming principles that make Clojure code so elegant. It's the difference between hoping resources get cleaned up and guaranteeing it.

P-adic Computations with Vector API ⚡

Modern CPUs provide powerful SIMD (Single Instruction, Multiple Data) capabilities through vector instructions. The Java Vector API gives us access to these capabilities while maintaining type safety and performance.

Enhanced Valuation Computation 📈

(defn p-adic-valuation-monadic [^IntVector v ^int p]
  "p-adic valuation calculation within a monad - exception-safe"
  (try
    (let [result (if (= p 2)
                   ;; p=2 special case: bit operation optimization
                   (let [packed (.convert v VectorOperators/I2L (LongVector/SPECIES_256))
                         zero-mask (.eq packed (.zero (LongVector/SPECIES_256)))]
                     (if (.allTrue zero-mask)
                       Integer/MAX_VALUE
                       (.reduceLanes (.lanewise packed VectorOperators/TRAILING_ZEROS_COUNT)
                                     VectorOperators/MIN)))
                   ;; General p-adic valuation
                   (let [zero-vec (.zero (.species v))
                         p-vec (.broadcast (.species v) p)]
                     (if (.allTrue (.eq v zero-vec))
                       Integer/MAX_VALUE
                       (loop [current v valuation 0 max-iter 32]
                         (if (or (zero? max-iter)
                                 (.anyTrue (.ne (.mod current p-vec) zero-vec)))
                           valuation
                           (recur (.div current p-vec) (inc valuation) (dec max-iter)))))))]
      (ok result 
          {:computation-type (if (= p 2) :bit-optimized :general)
           :p-value p}
          [{:level :debug :message (str "p-adic valuation calculated: p=" p " result=" result)}]))
    (catch Throwable t 
      (err t {} [{:level :error :message "p-adic valuation calculation error"}]))))

Key features:

Specialized handling for p=2 using bit operations
General p-adic valuation using algebraic operations
Vectorized computation using Java Vector API
Monadic error handling with detailed logging

The beauty of this implementation lies in its adaptability. For p=2, we use efficient bit operations, but for arbitrary primes, we fall back to general algebraic methods. The Vector API ensures that both approaches benefit from SIMD acceleration.

Data Preparation and Alignment 🎯

(defn prepare-aligned-data-enhanced [data vector-lane-count]
  "Monadic version of data preprocessing - with validation"
  (try
    (when (empty? data)
      (throw (IllegalArgumentException. "Cannot process empty data")))
    
    (let [species (IntVector/SPECIES_256)
          aligned (->> data
                       (map #(cond 
                               (coll? %) (vec %)
                               (number? %) [%]
                               :else (throw (IllegalArgumentException. 
                                             (str "Invalid data type: " (type %))))))
                       (map #(take vector-lane-count (concat % (repeat 0))))
                       (mapv int-array)
                       (mapv #(IntVector/fromArray species % 0)))]
      (ok aligned 
          {:data-count (count data)
           :vector-lane-count vector-lane-count
           :aligned-count (count aligned)}
          [{:level :info :message (str "Prepared " (count aligned) " vectors")}]))
    (catch Throwable t 
      (err t {} [{:level :error :message "Data preparation error"}]))))

This function handles data validation, type conversion, and vector alignment for optimal SIMD performance.

Data alignment might seem like a low-level concern, but it's crucial for SIMD performance. Misaligned data can cause significant performance penalties, so we handle alignment automatically while providing clear error messages when alignment isn't possible.

Ultrametric Space Construction 🌐

Ultrametric spaces are fundamental to p-adic analysis, but constructing them efficiently requires careful attention to both mathematical properties and computational performance.

Distance Matrix Computation 🎯

(defn compute-distance-matrix-monadic [aligned-data p]
  "Ultrametric distance matrix calculation in a monad - Fixed return value bug"
  ;; BUG FIX 2: Fixed mlet to return the correct map
  (let [n (count aligned-data)
        results (make-array Double/TYPE n n)]
    (mlet [computation-result
           (reduce
             (fn [acc [i j]]
               (bind acc
                 (fn [_]
                   (mlet [vi (ok (nth aligned-data i))
                          vj (ok (nth aligned-data j))
                          diff (ok (.sub vi vj))
                          val-result (p-adic-valuation-monadic diff p)]
                     (let [val (extract-value val-result)
                           distance (if (>= val Integer/MAX_VALUE) 0.0 (Math/pow p (- val)))]
                       (aset results i j distance)
                       (aset results j i distance)
                       (ok distance)))))) ; wrap in ok for bind
             (ok nil)
             (for [i (range n) j (range (inc i) n)] [i j]))]
      ;; Return the final result map in the body of mlet
      (ok {:distance-matrix results
           :dimensions [n n]
           :p-prime p}))))

Our implementation:

Uses monadic composition for error handling
Leverages vector operations for performance
Handles edge cases (like zero vectors)
Provides detailed metadata about the computation

The challenge with distance matrix computation is that it scales quadratically with input size. By using vectorized operations and parallel processing, we can handle much larger datasets than naive implementations would allow.

Parallel Critical Point Detection 🔍

The critical point detection implementation was already shown earlier in the "Parallelization Promise Fulfilled" section, demonstrating:

Chunk-based parallel processing
Managed thread pool resources
Graceful error handling
Detailed performance metrics

Critical point detection is naturally parallel - we can process different regions of the space independently. The key is balancing chunk size to minimize coordination overhead while maximizing CPU utilization.

Hodge Theory Integration 🎭

Hodge theory provides a bridge between algebra and geometry, and its integration with p-adic methods opens up fascinating computational possibilities.

Monadic Hodge Modules 🎨

(defrecord MonadicHodgeModule [species p-prime operations metadata])

(defn create-monadic-hodge-module [p-prime]
  "Generates a Hodge module in a monad"
  (try
    (let [species (IntVector/SPECIES_256)
          operations {:add VectorOperators/ADD
                      :sub VectorOperators/SUB  
                      :mul VectorOperators/MUL
                      :and VectorOperators/AND
                      :or VectorOperators/OR
                      :xor VectorOperators/XOR
                      :min VectorOperators/MIN
                      :max VectorOperators/MAX}
          metadata {:creation-time (System/currentTimeMillis)
                    :p-prime p-prime
                    :vector-width (.vectorBitSize species)}]
      (ok (->MonadicHodgeModule species p-prime operations metadata)
          metadata
          [{:level :info :message (str "Hodge module created: p=" p-prime)}]))
    (catch Throwable t 
      (err t {} [{:level :error :message "Error creating Hodge module"}]))))

We've created a mathematical framework that:

Encapsulates vector species and operations
Tracks mathematical metadata
Provides monadic interfaces for mathematical operations

The modular approach allows us to build complex mathematical structures from simpler components while maintaining clear interfaces and error handling throughout.

Filtration Operations 🌊

(defn filtration-monadic [hodge-module levels vectors]
  "Monadic Filtration"
  (mlet
    [species (ok (:species hodge-module))
     p-prime (ok (:p-prime hodge-module))
     level-masks (ok (mapv #(IntVector/broadcast species (int (Math/pow p-prime %))) levels))]
    (mapv (fn [level-mask]
            (mapv #(.and % level-mask) vectors))
          level-masks)))

This implements p-adic filtrations with proper monadic composition and error handling.

Filtrations are sequences of nested subspaces, and computing them efficiently requires careful attention to both mathematical structure and computational complexity. Our monadic approach ensures that errors in any stage of the filtration computation are handled gracefully.

Complete Analysis Pipeline 🔄

Bringing all these components together, we create a comprehensive analysis pipeline that demonstrates the power of compositional design.

Integrated Ultrametric Analysis 🧮

(defn ultrametric-analysis-monadic-enhanced
  [data p & {:keys [parallel-level analysis-type memory-limit-mb]
             :or {parallel-level (.. Runtime getRuntime availableProcessors)
                  analysis-type :full
                  memory-limit-mb 1024}}]
  "A complete monadic ultrametric analysis pipeline"
  
  ;; Memory check
  (let [available-memory (- (.maxMemory (Runtime/getRuntime))
                           (.totalMemory (Runtime/getRuntime)))
        memory-threshold (* memory-limit-mb 1024 1024)]
    (if (< available-memory memory-threshold)
      (err (RuntimeException. "Insufficient memory") 
           {:available-memory available-memory :required-memory memory-threshold})
      
      (mlet
        [;; Phase 1: Ultrametric space construction
         ultrametric-space (build-ultrametric-space-monadic-enhanced data p)
         
         ;; Phase 2: Morse analysis
         critical-points (find-critical-points-monadic 
                           (:vectorized ultrametric-space) p parallel-level)
         
         ;; Phase 3: Topology analysis
         topology (ok {:euler-characteristic (count critical-points)
                       :critical-count (count critical-points)})
         
         ;; Phase 4: Witt elimination (conditional)
         witt-result (if (#{:full :witt} analysis-type)
                       (parallel-witt-elimination-monadic
                         (:distance-matrix ultrametric-space) p parallel-level)
                       (ok nil))]
        
        ;; Assemble the final result
        {:ultrametric-space ultrametric-space
         :morse-analysis {:critical-points critical-points
                          :topology topology}
         :witt-elimination witt-result
         :analysis-metadata {:p-prime p
                             :data-size (count data)
                             :parallel-level parallel-level
                             :analysis-type analysis-type}}))))

Our complete pipeline:

Validates memory requirements
Builds ultrametric spaces
Performs Morse analysis
Computes topological features
Optionally performs Witt elimination

Each stage of the pipeline builds on the previous ones, with monadic composition ensuring that errors are handled cleanly and resources are managed properly. The optional Witt elimination demonstrates how the pipeline can be extended with additional mathematical operations.

Practical Examples and Testing 🧪

Theory is important, but practical examples demonstrate how these abstractions work in real applications.

Example Usage 💡

(defn detailed-example []
  "Detailed execution example"
  (let [data (vec (range 1 21))
        result (ultrametric-analysis-monadic-enhanced 
                 data 3 
                 :parallel-level 2 
                 :analysis-type :full)]
    (if (is-ok? result)
      (do 
        (println "=== Analysis Successful ===")
        (println "Metadata:" (:metadata result))
        (println "Logs:" (take 5 (:logs result)))
        (pp/pprint (select-keys (extract-value result) 
                                [:analysis-metadata])))
      (do 
        (println "=== Analysis Failed ===")
        (println "Error:" (extract-error result))
        (println "Logs:" (:logs result))))))

This example demonstrates the complete workflow from raw data to mathematical insights, showing how the various components work together in practice.

Performance Testing 📊

(defn performance-comparison []
  "Performance comparison test"
  (let [test-sizes [50 100 200]
        results (for [size test-sizes]
                  (let [data (vec (take size (repeatedly #(rand-int 1000))))
                        start-time (System/nanoTime)
                        result (ultrametric-analysis-monadic-enhanced data 2 :analysis-type :ultrametric-only)
                        end-time (System/nanoTime)
                        duration (/ (- end-time start-time) 1000000.0)]
                    {:size size
                     :duration-ms duration
                     :success (is-ok? result)
                     :metadata (when (is-ok? result) (:metadata result))}))]
    (println "=== Performance Results ===")
    (doseq [r results]
      (println (format "Size %d: %.2fms %s" 
                       (:size r) (:duration-ms r) 
                       (if (:success r) "Success" "Failure"))))))

Performance testing isn't just about measuring speed - it's about understanding the trade-offs between different approaches and ensuring that our optimizations actually improve real-world performance.

Key Advantages of This Approach ✨

The integration of monadic error handling, parallel computation, and mathematical rigor creates a framework that's greater than the sum of its parts.

1. Mathematical Rigor Meets Practical Computation 🔬

Our implementation maintains mathematical correctness while providing practical computational capabilities.

By embedding mathematical operations in monadic contexts, we ensure that numerical errors, edge cases, and computational limitations are handled explicitly rather than hidden.

2. Exceptional Safety 🛡️

The monadic approach ensures that errors are handled gracefully and resources are managed safely.

Safety isn't just about preventing crashes - it's about providing meaningful error messages, maintaining data integrity, and ensuring that partial results are clearly marked as such.

3. Performance Optimization ⚡

Vector API usage and parallel computation provide significant performance benefits.

Performance optimization in mathematical computing isn't just about speed - it's about enabling computations that would otherwise be intractable, opening up new possibilities for mathematical exploration.

4. Extensibility 🔧

The modular design makes it easy to extend with new mathematical operations or computational strategies.

The protocol-based approach and monadic composition mean that new mathematical operations can be added without modifying existing code, demonstrating the power of good abstraction design.

Conclusion and Next Steps 🎯

In this tutorial, we've built upon our basic p-adic implementation to create a robust, high-performance computational framework. The monadic approach provides exceptional safety and composability, while the vector and parallel operations ensure computational efficiency.

The journey from basic p-adic arithmetic to a full computational framework demonstrates how functional programming principles scale from simple functions to complex systems. By maintaining clear abstractions and compositional design, we've created something that's both powerful and maintainable.

What to Explore Next: 🚀

GPU Acceleration: Integrate GPU computation for even better performance
Distributed Computing: Extend to cluster computing environments
Interactive Visualization: Add real-time visualization capabilities
Additional Mathematical Structures: Implement related mathematical concepts

Each of these directions builds on the foundation we've established, demonstrating how good architectural decisions pay dividends as systems grow and evolve.

💻 Complete Implementation

Want to see this all in action? I've created a comprehensive implementation that puts all these concepts together:

🔗 Full P-adic Ultrametric Implementation with AVX2

This is my battle-tested implementation that combines:

✅ Monadic error handling
✅ AVX2 vectorization
✅ Parallel processing
✅ Ultrametric space construction
✅ Advanced p-adic computations

I've run extensive tests on this code, so I'm confident it demonstrates all the concepts we've discussed in a production-ready format. Feel free to use it as a reference implementation or starting point for your own p-adic adventures! 🚀

🤔 Why Not Just `with-open`?

Fair question! For 90% of cases, with-open is perfect. The differences here are admittedly subtle:

ThreadPool safety: with-open calls .close() immediately, while this does graceful shutdown with awaitTermination
Unified metrics: Automatic timing and logging for all resource types
Resource specs: Reusable resource specifications vs inline creation

Is it worth the complexity? Probably not for production code. But for exploring composition patterns in mathematical computing, it's been educational! 🎓

In real code, I'd likely just use `with-open` with some wrapper functions.

Thanks for following along on this mathematical and computational journey! If you found this tutorial helpful, please give it a ❤️ and share your own experiences with p-adic computing in the comments below. 💬

Buy me a coffee if this helped! ☕

Permalink

Calling a function before it’s defined, in Clojure

Code

;; calling_function_before_definition.clj

(declare b)

(defn a []
  (println "a")
  (b))

(defn b []
  (println "b"))

(a)

Notes

rswan

Permalink

Relaunching Yakread: an algorithmic reading app

I've recently finished a year-long rewrite of the Yakread codebase and have released it under a source-available license. Yakread is a reading app that makes heavy use of algorithmic recommendation/filtering. I originally launched it in 2022 during the last leg of my time as a full-time entrepreneur. It's written with Biff, a Clojure web framework that I also created during that time.

I'm publishing Yakread's source code mainly so that it can serve as a non-toy example project for Biff users. It's about 10K lines of code as of writing. I'm also using Yakread to experiment with potential framework features before adding them into Biff.

The app

I like reading stuff on the internet. Social media tends to be pretty shallow, though. Long-form content (articles, blogs, newsletters) is better on average but can take more work to manage: RSS readers and email inboxes tend to get filled up pretty easily, and sorting things chronologically benefits the most frequent publishers. I've found there's a certain amount of mental overhead associated with long-form content that, when you have only a few minutes to read something, often makes it easier to just pull up Reddit.

Yakread is my attempt to make reading long-form content as frictionless as reading social media. It's structured as a daily email with links to articles. New users start out getting five links a day to articles that were liked by other Yakread users. You can also add your own newsletter/RSS subscriptions to Yakread, and there's support for bookmarking individual articles to read later a la Pocket or Instapaper. Posts from these content sources are also compiled algorithmically so that:

Blogs that publish a few times per year don't get buried by daily newsletters and other frequent publishers.
Subscriptions that you interact with the most don't get buried by dozens of other subscriptions that you signed up for on a whim.
Articles you miss get resurfaced repeatedly, so you don't feel like you have to "keep up" with anything.

The web app also features a "for you" feed, similar to the daily emails, which lets you read on-demand. There are pages that list your content chronologically in case you want to read something specific: the recommendation algorithm is there as a default, but it's not the only way to read. There's a "favorites" page which lists articles that you've thumbs-upped.

Yakread is monetized through native ads (mostly for newsletters) and a "premium" subscription which removes ads.

The code

The README has a section describing the parts of Yakread's code structure that differ from regular Biff projects. Here are a few high-level points, written without assuming any prior knowledge of Biff.

"The algorithm":

For recommending new articles (i.e. not ones from your own subscriptions or bookmarks), Yakread uses Spark MLlib's collaboritive filtering implementation. There are controls layered on top that bias the results toward articles that have been recommended a fewer number of times across all users (exploration vs. exploitation).
Ads are selected via the same collaborative filtering model. The predicted rating for each ad is treated as a probability that the user will click on that ad, then we calculate the expected value of showing each ad (i.e. probability of a click multiplied by how much the advertiser is bidding for each click) and charge the advertiser (in the case of a click) via a second-price auction.
The algorithm for selecting articles from your subscriptions and bookmarks is a few hundred lines of custom code, which e.g.: computes a pair of "affinity" scores (lower bound and upper bound) for each of your subscriptions based on your previous interactions; ranks subscriptions based on affinity score, again with controls for exploration vs. exploitation; ranks articles based on how recently they were recommended and how recently they were published; figures out the right balance between recommending subscription posts and recommending bookmarks.

Everything else:

Yakread is a server-side rendered app that uses htmx. There's nothing too fancy going on in the UI (the biggest form in the app has 8 fields), so I'm keeping it simple.
It uses XTDB for the database. You could think of XTDB's immutable architecture kind of like "distributed SQLite." Queries operate on a local point-in-time snapshot of the data, so you can run multiple queries while handling a given request without worrying about network latency (pretty helpful for a recommender system).
The app's data model is organized via Pathom. I sometimes think of Pathom as "data-oriented dependency injection." It gives you the benefit of ORM model objects from OOP languages but without having to pass around a database connection. You specify up front what entities and fields you want, then Pathom wires up the data in the correct shape for you.
I use state machines to separate pure application logic from effectful code. Application logic returns data describing the effects it needs to perform (Pathom queries, network requests, database transactions, etc), then the machine transitions to other states that perform the effects, then results are passed to the next pure logic state, and so on. This makes unit tests easy to write since you never have to mock anything or check the results of side effects.
The tests are largely inline snapshot tests.
Yakread currently deployed as a monolith to a single DigitalOcean droplet which handles both web requests and background jobs. If/when the time comes to deploy a separate worker (which probably needs to happen soon...), the same deployment artifact can be ran with a BIFF_PROFILE=worker env var set.
I deploy Yakread with rsync. Even though there's only a single web server, most deploys can be done without downtime via the REPL: after rsync finishes, new code is evaluated while the application runs. Full restarts typically only happen when new dependencies are added.
Yakread does a lot of email: I use SubethaSMTP to receive email newsletters (set an MX record and open up port 25) and MailerSend for sending the daily digests.

The theory

I've been interested in recommender systems for a long time, starting with music and then moving into written content. In both domains I've been attracted to the idea of a system that can handle most of the tedious organizational work while only requiring you to do the part that only you, as the human, can do: give feedback on which things you like and don't like. I think there's plenty of unrealized potential for recommender systems to help people learn, enjoy life, and coordinate.

Recommender systems often get a bad rap, and for good reason: most people's exposure to them comes in the form of large companies trying to shove the digital equivalent of potato chips down their throat. However I see that as a problem of business incentives rather than a problem with algorithmic recommendation in general; there aren't any technical barriers to writing algorithms that serve up salad instead of potato chips.

So I don't think a mass return to reverse-chronological feeds is the answer: competition is. Ideally we'd have a larger distribution of companies offering recommendation-powered services that had to compete based on the quality of their results. Instead we mostly have a few behemoths that are optimized to squeeze every bit of interaction from you that they can, the long-forgotten sanctity of your notifications tab be damned.

I hope that Yakread makes the internet a little bit better in that regard. Although I've pretty much given up on trying to take over the world, I still like the idea of being part of a movement. And there is interesting stuff happening: Bluesky, for instance, has had far more success than I thought it would when it was announced back in 2019. The popularity of email newsletters is encouraging.

The internet is still young. Maybe the next decade can be a phase of building digital public infrastructure.

Permalink

Developing a Space Flight Simulator in Clojure

In 2017 I discovered the free of charge Orbiter 2016 space flight simulator which was proprietary at the time and it inspired me to develop a space flight simulator myself. I prototyped some rigid body physics in C and later in GNU Guile and also prototyped loading and rendering of Wavefront OBJ files. I used GNU Guile (a Scheme implementation) because it has a good native interface and of course it has hygienic macros. Eventually I got interested in Clojure because it has more generic multi-methods as well as fast hash maps and vectors. I finally decided to develop the game for real in Clojure. I have been developing a space flight simulator in Clojure for almost 5 years now. While using Clojure I have come to appreciate the immutable values and safe parallelism using atoms, agents, and refs.

In the beginning I decided to work on the hard parts first, which for me were 3D rendering of a planet, an atmosphere, shadows, and volumetric clouds. I read the OpenGL Superbible to get an understanding on what functionality OpenGL provides. When Orbiter was eventually open sourced and released unter MIT license here, I inspected the source code and discovered that about 90% of the code is graphics-related. So starting with the graphics problems was not a bad decision.

Software dependencies

The following software is used for development. The software libraries run on both GNU/Linux and Microsoft Windows.

Clojure the programming language
LWJGL provides Java wrappers for various libraries
- lwjgl-opengl for 3D graphics
- lwjgl-glfw for windowing and input devices
- lwjgl-nuklear for graphical user interfaces
- lwjgl-stb for image I/O and using truetype fonts
- lwjgl-assimp to load glTF 3D models with animation data
Jolt Physics to simulate wheeled vehicles and collisions with meshes
Fastmath for fast matrix and vector math as well as spline interpolation
Comb for templating shader code
Instaparse to parse NASA Planetary Constant Kernel (PCK) files
Gloss to parse NASA Double Precision Array Files (DAF)
Coffi as a foreign function interface
core.memoize for least recently used caching of function results
Apache Commons Compress to read map tiles from tar files
Malli to add schemas to functions
Immuconf to load the configuration file
Progrock a progress bar for long running builds
Claypoole to implement parallel for loops
Midje for test-driven development
tools.build to build the project
clj-async-profiler Clojure profiler creating flame graphs
slf4j-timbre Java logging implementation for Clojure

The deps.edn file contains operating system dependent LWJGL bindings. For example on GNU/Linux the deps.edn file contains the following:

{:deps {; ...
        org.lwjgl/lwjgl {:mvn/version "3.3.6"}
        org.lwjgl/lwjgl$natives-linux {:mvn/version "3.3.6"}
        org.lwjgl/lwjgl-opengl {:mvn/version "3.3.6"}
        org.lwjgl/lwjgl-opengl$natives-linux {:mvn/version "3.3.6"}
        org.lwjgl/lwjgl-glfw {:mvn/version "3.3.6"}
        org.lwjgl/lwjgl-glfw$natives-linux {:mvn/version "3.3.6"}
        org.lwjgl/lwjgl-nuklear {:mvn/version "3.3.6"}
        org.lwjgl/lwjgl-nuklear$natives-linux {:mvn/version "3.3.6"}
        org.lwjgl/lwjgl-stb {:mvn/version "3.3.6"}
        org.lwjgl/lwjgl-stb$natives-linux {:mvn/version "3.3.6"}
        org.lwjgl/lwjgl-assimp {:mvn/version "3.3.6"}
        org.lwjgl/lwjgl-assimp$natives-linux {:mvn/version "3.3.6"}}
        ; ...
        }

In order to manage the different dependencies for Microsoft Windows, a separate Git branch is maintained.

Atmosphere rendering

For the atmosphere, Bruneton’s precomputed atmospheric scattering was used. The implementation uses a 2D transmittance table, a 2D surface scattering table, a 4D Rayleigh scattering, and a 4D Mie scattering table. The tables are computed using several iterations of numerical integration. Higher order functions for integration over a sphere and over a line segment were implemented in Clojure. Integration over a ray in 3D space (using fastmath vectors) was implemented as follows for example:

(defn integral-ray
  "Integrate given function over a ray in 3D space"
  {:malli/schema [:=> [:cat ray N :double [:=> [:cat [:vector :double]] :some]] :some]}
  [{::keys [origin direction]} steps distance fun]
  (let [stepsize      (/ distance steps)
        samples       (mapv #(* (+ 0.5 %) stepsize) (range steps))
        interpolate   (fn interpolate [s] (add origin (mult direction s)))
        direction-len (mag direction)]
    (reduce add (mapv #(-> % interpolate fun (mult (* stepsize direction-len))) samples))))

Precomputing the atmospheric tables takes several hours even though pmap was used. When sampling the multi-dimensional functions, pmap was used as a top-level loop and map was used for interior loops. Using java.nio.ByteBuffer the floating point values were converted to a byte array and then written to disk using a clojure.java.io/output-stream:

(defn floats->bytes
  "Convert float array to byte buffer"
  [^floats float-data]
  (let [n           (count float-data)
        byte-buffer (.order (ByteBuffer/allocate (* n 4)) ByteOrder/LITTLE_ENDIAN)]
    (.put (.asFloatBuffer byte-buffer) float-data)
    (.array byte-buffer)))

(defn spit-bytes
  "Write bytes to a file"
  {:malli/schema [:=> [:cat non-empty-string bytes?] :nil]}
  [^String file-name ^bytes byte-data]
  (with-open [out (io/output-stream file-name)]
    (.write out byte-data)))

(defn spit-floats
  "Write floating point numbers to a file"
  {:malli/schema [:=> [:cat non-empty-string seqable?] :nil]}
  [^String file-name ^floats float-data]
  (spit-bytes file-name (floats->bytes float-data)))

When launching the game, the lookup tables get loaded and copied into OpenGL textures. Shader functions are used to lookup and interpolate values from the tables. When rendering the planet surface or the space craft, the atmosphere essentially gets superimposed using ray tracing. After rendering the planet, a background quad is rendered to display the remaining part of the atmosphere above the horizon.

Templating OpenGL shaders

It is possible to make programming with OpenGL shaders more flexible by using a templating library such as Comb. The following shader defines multiple octaves of noise on a base noise function:

#version 410 core

float <%= base-function %>(vec3 idx);

float <%= method-name %>(vec3 idx)
{
  float result = 0.0;
<% (doseq [multiplier octaves] %>
  result += <%= multiplier %> * <%= base-function %>(idx);
  idx *= 2;
<% ) %>
  return result;
}

One can then for example define the function fbm_noise using octaves of the base function noise as follows:

(def noise-octaves
  "Shader function to sum octaves of noise"
  (template/fn [method-name base-function octaves] (slurp "resources/shaders/core/noise-octaves.glsl")))

; ...

(def fbm-noise-shader (noise-octaves "fbm_noise" "noise" [0.57 0.28 0.15]))

Planet rendering

To render the planet, NASA Bluemarble data, NASA Blackmarble data, and NASA Elevation data was used. The images were converted to a multi resolution pyramid of map tiles. The following functions were implemented for color map tiles and for elevation tiles:

a function to load and cache map tiles of given 2D tile index and level of detail
a function to extract a pixel from a map tile
a function to extract the pixel for a specific longitude and latitude

The functions for extracting a pixel for given longitude and latitude then were used to generate a cube map with a quad tree of tiles for each face. For each tile, the following files were generated:

A daytime texture
A night time texture
An image of 3D vectors defining a surface mesh
A water mask
A normal map

Altogether 655350 files were generated. Because the Steam ContentBuilder does not support a large number of files, each row of tile data was aggregated into a tar file. The Apache Commons Compress library allows you to open a tar file to get a list of entries and then perform random access on the contents of the tar file. A Clojure LRU cache was used to maintain a cache of open tar files for improved performance.

At run time, a future is created, which returns an updated tile tree, a list of tiles to drop, and a path list of the tiles to load into OpenGL. When the future is realized, the main thread deletes the OpenGL textures from the drop list, and then uses the path list to get the new loaded images from the tile tree, load them into OpenGL textures, and create an updated tile tree with the new OpenGL textures added. The following functions to manipulate quad trees were implemented to realize this:

(defn quadtree-add
  "Add tiles to quad tree"
  {:malli/schema [:=> [:cat [:maybe :map] [:sequential [:vector :keyword]] [:sequential :map]] [:maybe :map]]}
  [tree paths tiles]
  (reduce (fn add-title-to-quadtree [tree [path tile]] (assoc-in tree path tile)) tree (mapv vector paths tiles)))

(defn quadtree-extract
  "Extract a list of tiles from quad tree"
  {:malli/schema [:=> [:cat [:maybe :map] [:sequential [:vector :keyword]]] [:vector :map]]}
  [tree paths]
  (mapv (partial get-in tree) paths))

(defn quadtree-drop
  "Drop tiles specified by path list from quad tree"
  {:malli/schema [:=> [:cat [:maybe :map] [:sequential [:vector :keyword]]] [:maybe :map]]}
  [tree paths]
  (reduce dissoc-in tree paths))

(defn quadtree-update
  "Update tiles with specified paths using a function with optional arguments from lists"
  {:malli/schema [:=> [:cat [:maybe :map] [:sequential [:vector :keyword]] fn? [:* :any]] [:maybe :map]]}
  [tree paths fun & arglists]
  (reduce (fn update-tile-in-quadtree
            [tree [path & args]]
            (apply update-in tree path fun args)) tree (apply map list paths arglists)))

Future work

Although the hard parts are mostly done, there are still several things to do:

control surfaces and thruster graphics
launchpad and runway graphics
sound effects
a 3D cockpit
the Moon
a space station

It would also be interesting to make the game modable in a safe way (maybe evaluating Clojure files in a sandboxed environment?).

Conclusion

You can find the source code on Github. Currently there is only a playtest build, but if you want to get notified, when the game gets released, you can wishlist it here.

Anyway, let me know any comments and suggestions.

Enjoy!

Updates

Submitted for discussion to Reddit here
See HackerNews discussion of this project here

Permalink

C4 - Functional anatomy

We look at the implementation of functions in ClojureCLR and how evaluation/compilation translates a source code definition of a function to the underlying class representation. (The first of several posts on this topic.)

The universe, and everything

There are a number of interfaces and classes that are used with in ClojureCLR (and this is parallel to ClojureJVM). We will cover all of these in detail:

Graph of all function-related types

IFn

The essence of a function is that you can invoke it on (zero or more) arguments. If you want a class to represent a function, the minimum requirement is to implement the IFn interface.

public interface IFn 
{
    object invoke();

    object invoke(object arg1);

    object invoke(object arg1, object arg2);

    object invoke(object arg1, object arg2, object arg3);

    // ...

    object invoke(object arg1, object arg2, object arg3, object arg4, object arg5, 
                    object  arg6, object arg7, object arg8, object arg9, object arg10, 
                    object arg11, object arg12, object arg13, object arg14, object arg15, 
                    object arg16, object arg17, object arg18, object arg19, object arg20);

    object invoke(object arg1, object arg2, object arg3, object arg4, object arg5, object arg6, object arg7,
                            object arg8, object arg9, object arg10, object arg11, object arg12, object arg13, object arg14,
                            object arg15, object arg16, object arg17, object arg18, object arg19, object arg20,
                            params object[] args);

    object applyTo(ISeq arglist);
}

(In ClojureJVM, IFn also implements interfaces Callable and Runnable; no equivalents exist in ClojureCLR.) There are invoke overloads for zero to twenty arguments, and a final overload that takes arguments past the 20 count in a params array. The applyTo method is used to define the behavior of apply.

IFnArity

This interface is not in ClojureJVM. I implemented it in ClojureCLR to provide a way to interrogat a function about what arities it supports. It was required to deal with dynamic callsites. That’s going to take us a long way from what we need to understand here, so I’m going to ignore it. (Okay, I’m avoiding it partly because I don’t remember what purpose it serves. Homework for me. ) For reference only:

public interface IFnArity
{
    bool HasArity(int arity);
}

AFn

If you want something to use as a function in ClojureCLR, just create a class that implements IFn and create an instance of it. You can put that any place a function can go.

As a practical matter, you should avoid that. To implement IFn, you need to provide implementations for all of the invoke methods – most of which are just going to throw a NotImplementedException – as well as applyTo.

The abstract base class AFn supplies default implementations of all the invoke methods and applyTo. The default implementations throw an ArityException, derived from ArgumentException, that indicates the function does not support the arity of the invoke.

public abstract class AFn : IFn, IDynamicMetaObjectProvider, IFnArity
{
    #region IFn Members

    public virtual object invoke()
    {
        throw WrongArityException(0);
    }

    public virtual object invoke(object arg1)
    {
        throw WrongArityException(1);
    }

    public virtual object invoke(object arg1, object arg2)
    {
        throw WrongArityException(2);
    }

    // ...

}

AFn cleverly provides a default implementation of applyTo that uses the invoke methods to do the work. It does this by counting the number of arguments in the ISeq passed to applyTo, and calling the appropriate invoke method. If there are more than 20 arguments, it collects the first 20 into individual variables and the rest into an array, and calls the final invoke overload.

public virtual object applyTo(ISeq arglist)
{
    return ApplyToHelper(this, Util.Ret1(arglist,arglist=null));
}

The call to Util.Ret is a trick required by the JVM: it returns the value of its first argument. The second argument is there to ensure that the arglist variable in the caller is set to null after the call. This is to help the garbage collector reclaim the memory used by the ISeq if it is no longer needed. As it turns out, this is not really necessary in the CLR, but I never got around to deleting it – it occurs in a lot of places, as ApplyToHelper illustrates:

public static object ApplyToHelper(IFn fn, ISeq argList)
{
    switch (RT.BoundedLength(argList, 20))
    {
        case 0:
            argList = null;
            return fn.invoke();
        case 1:
            return fn.invoke(Util.Ret1(argList.first(),argList=null));
        case 2:
            return fn.invoke(argList.first()
                    , Util.Ret1((argList = argList.next()).first(),argList = null)
            );
        case 3:
            return fn.invoke(argList.first()
                    , (argList = argList.next()).first()
                    , Util.Ret1((argList = argList.next()).first(), argList = null)
            );
        case 4:
            return fn.invoke(argList.first()
                    , (argList = argList.next()).first()
                    , (argList = argList.next()).first()
                    , Util.Ret1((argList = argList.next()).first(), argList = null)
            );    

        // ...
}

If you want to create a function class yourself, I highly recommend basing it on AFn. As an example, in the test code for ClojureCLR.Next, I use F# object expressions to create functions to test things like reduce functionality:

let adder =
    { new AFn() with
        member this.ToString() = ""
      interface IFn with
        member this.invoke(x, y) = (x :?> int) + (y :?> int) :> obj
    }

AFunction

The Clojure (CLR+JVM) compiler does not generate AFn sublasses directly. It generates classes derived from AFunction (for functions that are not variant) and RestFunction (for functions that are variant.) A variant function is one that has a & parameter amongst its options:

(fn 
   ([x]  ... one-arg definition ... )
   ([x y] .. two-arg definition ... )
   ([x y & ys] .. rest-arg definition ... ))

AFunction is an abstract base class derived from AFn:

public abstract class AFunction : AFn, IObj, Fn, IComparer
{
    // ...
}

It implements interface IObj, providing metadata attachment capability. This is done not simply by having a metadata field, but by implementing the withMeta method to return an instance of an AFunction+MetaWrapper. The latter class has a field for the metadata and a pointer back to the originating function. I guess they wanted to save the space of the field for the metadata.

It implements interface IComparer so that an instance of the class can be passed to things like sort routines, comparers for hash tables, etc. The Compare method just calls the two-argument invoke:

        public int Compare(object x, object y)
        {
            Object o = invoke(x, y);

            if (o is Boolean)
            {
                if (RT.booleanCast(o))
                    return -1;
                return RT.booleanCast(invoke(y, x)) ? 1 : 0;
            }
            return Util.ConvertToInt(o);
        }

AFunction also includes some support for a method implementation cache used in implementing protocols. That’s for discussion at another time.

When creating a derived class of AFunction, the compiler defines overrides of the invoke methods for each arity defined in the function declaration.

RestFn

On to RestFn. Ah, RestFn. The source file is almost 5000 lines long. I still have nightmares. I took the JVM version and did a bunch of search/replace operations. (For the ClojureCLR.Next, written in F#, I wrote a program to generate it.)

Consider the following:

(fn 
   ([x y]        ... two-arg definition ... )
   ([x y z ]     ... three-arg definition ... )
   ([x y z & zs]  ... rest-arg definition ... ))

This function should fail if called on zero or one arguments. It should call the supplied definitions when called on two or three arguments. And it should apply the variant definition when called with four or more arguments.

A key notion here is the required arity of the function. For this example, the required arity is two. This is the minimum number of arguments covered by one of the cases in the definition. If we call invoke() or invoke(object arg1), the call should fail.

RestFn uses a very clever trick to implement this functionality in a general way. By ‘clever’, I mean that every time I look at this code I have to take an hour to learn anew what it is actually doing. I hope by writing it down here, I can make this go faster in the future.

RestFn defines standard overrides for all the invoke overloads and applyTo. These are all defined in terms of other methods, the doInvoke methods that are designed to take a final argument that is an ISeq containing the rest args, if needed..

For the example, the compiler would provide overrides for

invoke(arg1, arg2)
invoke(arg1, arg2, arg3)
doInvoke(arg1, arg2, arg3, args)

that implement the code supplied in each case of the function definition. Ask yourself: What should be the default implementations for all the other invoke and doInvoke overloads?

For invoke the behavior is different for those with fewer than the required arity and those with more:

If there are fewer than the required arguments, the call should fail.
If there are more than the required arguments, the call should delegate to doInvoke(arg1, arg2, arg3, args)

The distinguishing factor is the required arity. For an invoke with fewer arguments than the required arity, we should fail. If we have more arguments than the required arity, then we should call the overload of doInvoke that takes the required arity plus the rest argument.

Thus, all the invoke overrides are variants on this theme:

public override Object invoke(Object arg1, Object arg2, Object arg3, Object arg4)
{
    switch (getRequiredArity())
    {
        case 0:
            return doInvoke(
                ArraySeq.create(
                    Util.Ret1(arg1, arg1 = null), 
                    Util.Ret1(arg2, arg2 = null), 
                    Util.Ret1(arg3, arg3 = null), 
                    Util.Ret1(arg4, arg4 = null)));
        case 1:
            return doInvoke(
                Util.Ret1(arg1, arg1 = null), 
                ArraySeq.create(
                    Util.Ret1(arg2, arg2 = null),
                    Util.Ret1(arg3, arg3 = null), 
                    Util.Ret1(arg4, arg4 = null)));
        case 2:
            return doInvoke(
                Util.Ret1(arg1, arg1 = null), 
                Util.Ret1(arg2, arg2 = null), 
                ArraySeq.create(
                    Util.Ret1(arg3, arg3 = null), 
                    Util.Ret1(arg4, arg4 = null)));
        case 3:
            return doInvoke(
                Util.Ret1(arg1, arg1 = null), 
                Util.Ret1(arg2, arg2 = null), 
                Util.Ret1(arg3, arg3 = null), 
                ArraySeq.create(
                    Util.Ret1(arg4, arg4 = null)));
        case 4:
            return doInvoke(
                Util.Ret1(arg1, arg1 = null), 
                Util.Ret1(arg2, arg2 = null), 
                Util.Ret1(arg3, arg3 = null), 
                Util.Ret1(arg4, arg4 = null), 
                null);
        default:
            throw WrongArityException(4);
    }
}

This invoke will be invoked by call in the code with four arguments: (f 1 2 3 4). If f had a clause that takes four arguments, then we woudl be calling f’s override of the four-argument invoke. So it does not have such a case. Let’s say f has a required arity of 2. We have enought arguments to supply the minimum. Then we would end up calling doInvoke(1,2, [3, 4]), which f has an overridden. If, instead, f had a required arity of 12, then we do not have arguments. The default case kicks in and we throw an exception: Wrong number of arguments.

The default implementations of doInvoke all return null. They will never be called. We will only ever call an override of doInvoke, the one matching the required arity (+ 1 for the rest args).

I’ll leave applyTo as an exercise.

Static invocation

In the previous post C4: ISeq clarity, I touched upon the notion of static invocation of functions. Static invocation is an efficiency hack. It allows a call such as

(f 1 2 3)

to bypass the usual dynamic dispatch that does a lookup of the current value of the #'f Var and instead links in the code directly to an invocation of a static method on the class defined for f. For this to happen, the first requirement is that the function allows direct invocation, the constraints being:

The function is not nested inside of another function.
The function does not close over any lexical variables from an outer scope.
The function does not use the this variable.

When these conditions are met, for each invoke the function defines, there will be a staticInvoke method of the same arity with the actual function definition. The invoke just calls the staticInvoke of the same arity.

I provide more detail on some of the issues of static linking in a previous post outside this series, The function of naming; the naming of functions.

Primitive urges

In the same two posts mentioned just above, I also touched upon primitive invocation. If one of the invoke overloads is typed so that its argument types and return types contain only ^long or ^double or (the default) ^object type hints and at are not all ^object, then we will create have the class representing the function implement an interface such as

public interface ODLLD
{

    double invokePrim(object arg0, double arg1, long arg2, long arg3);
}

In invocations where we know the types of the arguments, we can avoid boxing by calling the invokePrim method directly.

The interface is named by the type of each argument plus the return type. ODLLD = four arguments, of types Object, double, long, and long, with a return type of double. These interfaces are in the clojure.lang.primifs namespace. One to four arguments are accommodated. If you care to count, that comes to 358 interfaces. (Eventually, I’d like to replace these with the corresponding Function interfaces. We do have real generics in C#. And in ClojureCLR.Next, I’d like to get rid of the restriction to just long and double primitives.)

And that pretty much covers the classes that implement functions in ClojureCLR. Except …

The mysterious Fn

If you look above carefully, you will note that AFunction (and hence, indirectly, RestFn) implements an interface named Fn. This is a marker interface – no methods:

public interface Fn
{
}

I have found only one use of Fn in the Clojure code. Over in core_print.clj you will find this:

(defmethod print-dup clojure.lang.Fn [o, ^System.IO.TextWriter w]
  (print-ctor o (fn [o w]) w))

That’s it. It makes sure if you print a function with *print-dup* set to true, it prints out a constructor call. Really.

(defn f [x] (inc x))  ; => #'user/f
(bind [*print-dup* true] (prn f)) ; prints: #=(user$f__4237. )

Makes my day.

Code generation

If you have all of that digested, code generation is not as hard as one might think.
Some complexity is handled in source code macroexpansion. If you have a defn with fancy arg destructuring

(defn destr [& {:keys [a b] :as opts}]
  [a b opts])

That gets macroexpanded to

(def destr (clojure.core/fn ([& {:keys [a b], :as opts}] [a b opts])))

And that fn expresssion gets macroexpanded to

((clojure.core/fn ([& {:as user/opts, :keys [user/a user/b]}] [user/a user/b user/opts])))

One more round:

(fn* ([& p__4247] (clojure.core/let [{:keys [a b], :as opts} p__4247] [a b opts])))

Which, let’s face it, is just

(fn* [& arg] ...line noise---)

fn* is the underlying primitive for all function definitions. There’s a little work regularizing syntax:

(fn* [x] ...))

is the same as

(fn* ([x] ...))

You’d have that nesting anyway if you had multiple arities overloaded.

There can be an optional name to use:

(fn* myfunc ([x]))

But after that, it’s pretty much just a bunch of method bodies to parse. The fn* parses into an FnExpr that contains a list of FnMethod instances, one for each arity overload. When we generate code, the FnExpr becomes a class, the FnMethod instance each contribute a method (or several if we have static or primitive hackery). And there we are.

I’m lying

The actual complexity involved in code-gen for FnExpr and FnMethod is daunting. That’s going to take another post.

Permalink

Cursor without clojure-mcp

Arrive clojure-mcp

Context of my ongoing project

Building full features

Over time, the code accumulates small problems

All your tech debt will be repeated over and over forever.

Cursor auto mode

Cursor auto-complete

Clojure-MCP learning curve

Using multiple agents at the same time

Last words

🌪️ The Problem: COBOL is Too Big to Parse

🌀 The Mathematical Foundation: p-adic Distance

Bypassing Abstract Syntax Trees

🚀 Implementation: p-adic Analysis in Clojure

Step 1: Transform COBOL Names into Tokens

Step 2: Compute p-adic Distance Between Variables

Step 3: Hierarchical Clustering via group-by

Step 4: Real-World COBOL Example

🔥 Why This Works Better Than Traditional Approaches

1. No Grammar Required

2. Computational Efficiency

3. Discovers Hidden Structure

4. Structure-Preserving Data Access

🔬 From Clusters to System Architecture

🚀 Scaling Up: Enterprise Analysis

📊 Real Results: Revealing the System's Hidden Hierarchy

🎯 Key Takeaways

🔗 What's Next?

🔄 Full Circle: Second Chances with Better Math

Clojure Support and Maintenance: Toby Crawley

Code Combat: Karl Pietzrak

SciCloj Building Bridges to New Clojure Users: Siyoung Byun

Macroexpand 2: Connecting Clojure Data Practitioners

Purpose of the Gathering

Key Outcomes

Further Vision

Macroexpand 2025: Two Upcoming Online Conferences

About the Conferences

Current Work

Clojars Support and Maintenance: Toby Crawley

Code Combat: Karl Pietzrak

SciCloj Building Bridges to New Clojure Users: Siyoung Byun

Macroexpand 2: Connecting Clojure Data Practitioners

Purpose of the Gathering

Key Outcomes

Further Vision

Macroexpand 2025: Two Upcoming Online Conferences

About the Conferences

Current Work

Why did the early frontend development era (2009–2013) feel simpler?

Constrained defaults created a shared baseline

Developer expectations were modest, and projects weren’t complex

Standards were simple, and the experiments were fewer

The trade-off: limited flexibility, but less decision fatigue

What caused the explosion of tools post-2015?

Rapid ECMAScript releases and evolving web standards

SPA and TypeScript adoption fuel ecosystem growth

Open-source funding and GitHub hype-driven tool proliferation

Generative AI accelerated the creation of new tools

How many tools are we really talking about?

Why the abundance of tools can hurt DX

Decision fatigue: too many choices slow down projects

Steep learning curves, especially for newcomers

Ecosystem fragmentation leads to unstable codebases

Tool churn creates maintenance and migration costs

How to manage DX in an era of tool overload

Lean on community conventions and proven ecosystems

Pick tools with healthy documentation and support, not just hype

Optimize for team needs and stability over novelty

Adopt change only when the benefit is clear and measurable

Encourage continuous learning without chasing every new trend

Conclusion

Upcoming Events

Podcasts, videos, and media

Blogs, articles, and projects

Libraries and Tools

Introduction: Connecting the Dots 🔗

What Makes This a Natural Progression 🌟

From Spatial Sorting to Mathematical Computation 📊

Step 3: Hierarchical Clustering via `group-by`

🤔 Why Not Just `with-open`?

In real code, I'd likely just use `with-open` with some wrapper functions.