Compiling Clojure to WebAssembly

I recently watched The Future of Write Once, Run Anywhere: From Java to WebAssembly by Patrick Ziegler & Fabio Niephaus. This conference talked demo'd compiling a Java application in the browser, and running that locally. Not only that, but they also compiled Java bytecode to web assembly, and simpy ran it with node command.

I have a Clojure application, cljcc ( a toy C compiler ) which I always wanted a web version of. I was thinking of implementing the core library to be common across clojure/script, but after seeing the above talk wanted to try directly compiling to WASM. I have mentioned the steps I followed for the same in this post.

Compilation

For compiling to WASM, it's just about changing the backend target in the Graal VM native image tool. It builds on the setup I described in this post.

Reference Babashka Task Runner

;; added a new babashka task
lib:build:wasm {:doc "Builds native WASM image."
                  :depends [lib:build:jar]
                  :task (shell {:dir "target/lib"}
                               "native-image"
                               "--tool:svm-wasm"
                               "-jar" "cljcc-lib.jar"
                               "-o" "cljcc-lib-wasm"
                               "--features=clj_easy.graal_build_time.InitClojureClasses"
                               "--initialize-at-build-time"
                               "--verbose")}

It first builds the Clojure codebase to a jar file ( the :depends step ), and then use the option --tool:svm-wasm in the native image tool, which generates WASM.

Right now it's only supported in the latest early access version of jdk.

# uses sdkman for managing jdk version
sdk use java 25.ea.18-graal # specified in the original conference talk
# might also need brew install binaryen

This generates three files.

# /target/lib
cljcc-lib-wasm.js      cljcc-lib-wasm.js.wasm cljcc-lib-wasm.js.wat  cljcc-lib.jar

bun run cljcc-lib-wasm.js # Runs the main function

Frontend

I wanted to use the js and wasm files in simple frontend application. User can enter C source and specify the compilation stage, which will call the js file above and show the output.

At first, I was sort of confused. Java programs will have void main(args ... ) method as the starting point to binary. As this function doesn't return anything, how should I capture the output of the program ?

Luckily this was simple, as output stdout, stderr are translated to the browser's console.log, error. Simply copied all calls to console methods, and stored them in a local variable.

var originalConsoleLog = console.log;
var capturedLogs = [];
console.log = function() {
    var args = Array.from(arguments).map(arg =>
        typeof arg === 'object' ? JSON.stringify(arg, null, 2) : String(arg)
    );
    var message = args.join(' ');

    capturedLogs.push(message);
    originalConsoleLog.apply(console, arguments);
};

The other problem was how to actually use the generated js and wasm files. The js file is filled with IIFEs. Just loading the file immediately runs the entire compiler.

The generated js file has a top level GraalVM object. It has the main methods, such as run, which starts the WASM instantiation, loads the arguments etc. I commented out the IIFEs which execute the GraalVM.run method in the generated js file. Attached this GraalVM object to window, so I could access it from other files and then manually called the .run methods present on this object.

async function runCompiler(source, stage) {
    try {
        const args = [source, "linux", stage];
        const config = new GraalVM.Config();
        await GraalVM.run(args,config).catch(console.error);
    } catch (e) {
        // ...
    }
}

I don't think this is the ideal way to go about this. But it works for now.

Added a few buttons to specify the compilation stage, code examples and it's done ! The generated wasm file is 23 megabytes !, so the first execution takes a significant time as it pulls the file.

I have only implemented a subset of C's features, ( following the book Writing a C Compiler by Nora Sandler).

You can play with the compiler at cljcc.shagunagrawal.me.

Permalink

Making a simple Datalog Engine in Clojure

I started reading about Datalog and watched this conference talk Domain Modeling With Datalog by Norbert Wojtowicz. The talk models Github using datalog. It starts out simple, but by the end models a significant portion of Github's core features.

I was only familiar with SQL like databases, with data being conformed to a schema. This idea of modelling data using entity, attribute, value triples was very new to me. What I felt was more interesting was how it actually figured out the result. For e.g. the relations specified in the query can be in any order, can include variables, refer to each other etc. I wanted to implement this query engine.

The goal for me was to implement some of the examples presented in the talk. It's not optimized, doesn't have much features and probably has some bugs, but I had a lot of fun building it. I did have some classes on Prolog and logic programming in college, but didn't pay that much attention. I only ever did basic things like, mary's father is bob, bob's father is dan, then is dan an ancestor of mary ? I didn't realize it could be used as a query language for a database.

This post mentions the examples given in the talk, and how I implemented them.

Source Code: kaepr/dedukt

References

Throughout this project, I took references from the below articles/talks/book.

Triples

The data for this system is described by using only 3 things,

Entity, Attribute, Value

The datastructure for the database is a list of the above triples. Anytime some new information needs to be added, a new triple is added to this database. It's append only.

For my usecase, I will just store it in memory, as a list of list.

(def db
  [[11 :attr/name "some-value"]
   [22 :attr/name "some-value-2"]
   ;; more entries
   [33 :attr/name "other-value-3"]])

Above is an example of such a database, with some random entities, attribute and values. Each row is called a datum, which is basically a fact we have in our the system. The database is a list of facts about the data.

Each row, in the entity column has unique ids. Different numbers means they are different things.

Below is the example from the talk, which is much more clear. It models user's in Github.

(def db 
  [[11 :user/name "richhickey"]
   [22 :user/name "tonsky"]
   [33 :user/name "pithyless"]])

Each entity is a separate user ( based on the id ). Entity has some attribute attached to it, which is the keyword. And that attribute has a value to it. The current example only shows strings, but it can be anything.

Query

The query looks similar to SQL queries.

;; syntax
[:find <var-name> <var-name>
 :where [e a v]
         ...
        [e a v]]

;; example
[:find ?name
 :where 
 [11 :user/name ?name]]

There's a find clause, which finds ?name. This ?name is a user defined variable. The question mark in front of the variable name is sort of a standard practice. This is the unknown value, for which we are trying to find the values for. where clause expects a list of triples. Notice in this triple, ?name is used inside the where clause.

So this query will return all the values in all the triples, for which 11 is the entity, and :user/name is the attribute. So all usernames for a user with id 11.

The output is

#{["richhickey"]}

All outputs are returned as a set of tuples.

For now, let's just implement this query, and use the db defined above.

There's a small helper function to convert the query in vector form to map.

(defn parse-query
  "Returns query in map form.

  Each clause becomes a key inside map."
  [q]
  (let [find-index (.indexOf q :find)
        where-index (.indexOf q :where)
        vars (subvec q (inc find-index) where-index)
        wheres (subvec q (inc where-index))]
    {:find vars
     :where wheres}))

The naive solution ( just enough to make the example work ) is

(defn match
  "Returns the first matching fact and where clause."
  [[e a v :as _fact] where-clauses]
  (first
   (filter
    (fn [[we wa]]
      (and (= e we)
           (= a wa)))
    where-clauses)))

(defn run [db q]
  (let [{:keys [find where]} (parse-query q)
        rf (fn [acc fact]
             (if-let [matched (match fact where)]
               (conj acc [(nth fact 2)])
               acc))]
    (reduce rf #{} db)))

(run db q)
;; #{["richhickey"]}

It simply takes in the list of facts and where clauses, checks whether the entity and attribute match. If yes, then the value is a possible answer. This will work for different values of entity or attribute.

Let's see the second query from the talk.

[:find ?id
 :where [?id :user/name "tonsky"]]

Now the query will return all the id's with the given attribute and value. It's definitely possible to modify the above code to handle this case. But even with just two queries it's possible to see that is not the correct way forward. The implementation will become way too complex if I continued implementing in such a way.

The above solution made a lot of assumptions. For e.g. it assumed that value is the only thing we might query, what if there are multiples entities and attributes with same values ? What about multiple where clauses ?

Below are some more example queries.

[:find ?id ?name
 :where [?id _ "tonsky"]] ;; find all id? and ?name for any attributes, but value "tonsky"
 
[:find ?attr
 :where [_ ?attr _]] ;; find all attributes 

There are many more such variations, and it's not possible to implement this by hand. The query runner must somehow figure out what to query, what to filter on, the constraints etc on it's own.

Datalog

Here is datalog's wikipedia definition

Datalog is a declarative logic programming language. While it is syntactically a subset of Prolog, Datalog generally uses a bottom-up rather than top-down evaluation model. This difference yields significantly different behavior and properties from Prolog. It is often used as a query language for deductive databases.

Example

# datalog program consists of facts
edge(a, b); # there'a edge from a->b
edge(b, c); # edge from b->c
edge(d, e); # edge from d->e

# and also rules, which deduce new facts
path(x, y) :- edge(x, y)
path(x, y) :- edge(x, z), path(z, y)

The above example has a list of facts, e.g. edges in a graph. It also has rules, which define a path. A path is either a edge between two nodes, or through intermediary nodes.

Based on this information, a datalog query engine can figure out the path.

For e.g. if we ask the datalog system

# is there is path, using the facts and rules defined above ?
?- path(a, c)

The result will be true, as it figured out that there does exist a path from a to c based on the given facts. It will return false for path(a, d)

Datalog will also return the possible answers. E.g.

# find all possible values for Y, which you can figure out a path ( from a )
?- path(a, Y)

The Y above is a variable. Now the datalog engine returns all possible values of Y such that a path to a exists. In this case, it would be b, c.

Compare this to querying a database, and we can see the similarities.

For the query

[:find ?id
 :where [?id :user/name "tonsky"]]

the datalog engine will have to figure out the relations between the data. Then it will do something like

?- users(X, :user/name, "tonsky")

So it will return all possible values of X, such that the attribute was :user/name and name is "tonsky".

But how to make this engine, and how to represent the facts which this engine can understand.

How does the triple store translate to facts above?

The triple store defined in db above gets transformed as such.

[e a v] -> a(e, v)

The attributes inside the triple store becomes the predicates.

So the database of triples becomes something like

# triple store translated to datalog facts
user_name(11, "richhickey").
user_name(22, "tonsky").
user_name(33, "pithyless").
user_email(11, "rich@e.com").

# query
user_name(UserId, "richhickey")?
user_email(UserId, Email)?

Above example has facts of user names and email and a query. Notice the UserId common in both queries. Datalog engine figures out this common variable and returns the email for the user who is named "richhickey".

There's a browser based datalog engine at Online Datalog Engine.

Query engine

Let's start with simple queries.

(def db 
  [[11 :user/name "richhickey"]
   [22 :user/name "tonsky"]
   [33 :user/name "pithyless"]])

[:find ?attr
 :where [22 ?attr "tonsky"]] ;; #{[:user/name]}

[:find ?id
 :where [?id :user/name "tonsky"]] ;; #{[22]}

[:find ?id ?name 
 :where [?id :user/name ?name]] ;; all entries

All of these are similar, just with different variables. There is only a single entry in the where clause. Let's call the clause inside where a pattern. The pattern, which is itself a triple has either constants or variables. Variables are Clojure symbols, starting with ?. Constants are any valid Clojure value. This pattern will be checked against the database, which is a list of facts.

The pattern is checked against every fact. This can be handled generically by using pattern matching. This is unlike the first solution, where I just check for the values.

(defn match
  "Returns the fact, if it succesfully matches to the pattern.
   Otherwise nil."
  [pattern fact]
  (let [term-match? (fn [p f] (or (= '_ p) (variable? p) (= p f)))
        matched? (reduce (fn [acc [p f]]
                           (and acc (term-match? p f)))
                         true
                         (map vector pattern fact))]
    (when matched? fact)))

(match ['?id '_ "tonsky"] [22 :user/name "tonsky"])
;; [22 :user/name "tonsky"]

Above function tries to match a pattern against a fact. It simply checks for each item in the triple. If it's a variable, then it's matches. If it's a constant, then it's checked by value. There's a special match all case, using the _ symbol in the pattern.

This match is run against all the facts in the database. If a value is found, great, otherwise there are no solutions.

This works, but will return the entire triple. We only want the values for symbols specified in the find clause.

This is done by the select function.

(defn select [find-vars pattern fact]
  (let [pattern-to-fact (map vector pattern fact)
        f (fn [fv]
            (some (fn [p-f]
                    (when (= (first p-f) fv) (second p-f))) pattern-to-fact))]
    (mapv f find-vars)))

It uses the pattern and the provided fact, and the list of variable names. Then merges to return the values of the variables.

Finally it's the run function. Lets run's this on the database and queries above.

(defn run [db q]
  (let [{:keys [find where]} (parse-query q)
        fact (some #(match (first where) %) db)
        selected (select find (first where) fact)]
    (set [selected])))

(def db 
  [... same as above ... ])

(run db [:find '?attr
         :where [22 '?attr "tonsky"]])
;; => #{[:user/name]} correct !

(run db [:find '?id
         :where ['?id :user/name "tonsky"]])
;; => #{[22]} correct !

(run db [:find '?id '?name
         :where ['?id :user/name '?name]])
;; => #{[11 "richhickey"]} wrong :(

The first two examples are correct. By using pattern matching, it was possible to offload the logic for figuring out which variable and it's value we are trying to find.

The third is also technically correct, but we are only returning the first matching result. Ideally it should return all values. It's a simple change, just return all matching facts for now.

Below is the modified run.

(defn run [db q]
  (let [{:keys [find where]} (parse-query q)
        facts (filter #(match (first where) %) db)
        values (mapv #(select find (first where) %) facts)]
    (set values)))

(run db [:find '?id
         :where ['?id :user/name '_]])
;; => #{[11] [22] [33]}

(run db [:find '?id '?name
         :where ['?id :user/name '?name]])
;; => #{[11 "richhickey"] [22 "tonsky"] [33 "pithyless"]}

So at this point, we are able to support multiple variables. Multiple facts in the database. A catch all variable _. And the result returns all possible values which satisfy the single where predicate we provide.

Multiple where predicates

Let's complicate the domain model and the possible queries. Each user will now have an email. We will update the database definition, and see an example query.

(def db
  [[11 :user/name "richhickey"]
   [22 :user/name "tonsky"]
   [33 :user/name "pithyless"]
   [11 :user/email "rich@e.com"]
   [22 :user/email "nikita@e.com"]
   [33 :user/email "norbert@e.com"]])

[:find ?email
 :where [?id :user/name "richhickey"]
        [?id :user/email ?email]] ;; another where predicate 

Each user, identified by their numeric id, now has two attributes. name and now an email. The query includes another predicate. The interesting part is the ?id variable, which is used in both predicates.

The query is asking, find me the email of some user. This user has the name "richhickey". The query engine will figure out, that user with id 11 has name as "richhickey". Now using this updated information, it will find the triple, where entity id is 11 and the attribute is email, and will returns "rich@e.com".

Datalog queries also have no concept of order. Even if the order of where predicates is switched, the query engine will still figure it out.

There could be multiple ways of solving this. I will keep things simple and sort of brute force the solution.

For each pattern (where predicate), let's find all possible answers. Instead of simply returning the result, it will be returned with the variable symbol used in the pattern. This will be used later to join the possible values together.

(defn find-variable-facts
  "Returns subset of fact, which are assigned as variables in the pattern."
  [pattern fact]
  (keep (fn [[p f]] (when (variable? p) f))
        (map vector pattern fact)))

(defn variable->fact
  "Return a map from variable to fact, with a given pattern and fact."
  [pattern fact]
  (zipmap (filter variable? pattern)
          (find-variable-facts pattern fact)))

(variable->fact
   '[?id :user/name ?name]
   '[11 :user/name "richhickey"])
;; => {?id 11, ?name "richhickey"}

(defn all-matches
  "Returns variable->fact mapping from all facts, with a given pattern."
  [pattern facts]
  (map #(variable->fact pattern %)
       (filter #(match pattern %) facts))) ;; reuses the `match` defined before

(all-matches
  '[?id :user/email ?email]
  db) ;; db defined before
;;=>({?id 11, ?email "rich@e.com"}
;    {?id 22, ?email "nikita@e.com"}
;    {?id 33, ?email "norbert@e.com"})

So we have for each pattern ( variables ), all the facts which satisfy it. Till now it's similar to the old solution ( apart from having a variable symbol map). As we have multiple patterns now, these list of variable bindings must be merged.

(defn merge-variable->facts
  [variable->facts-1 variable->facts-2]
  (for [vf1 variable->facts-1
        vf2 variable->facts-2
        :when (every? (fn [k] (= (get vf1 k)
                                 (get vf2 k)))
                      (set/intersection (set (keys vf1))
                                        (set (keys vf2))))]
    (merge vf1 vf2)))

(comment

  (merge-variable->facts
   '[{?id 11}]
   '[{?id 11 ?email "rich@e.com"}
     {?id 22 ?email "tonsky@e.com"}])
;; => ({?id 11, ?email "rich@e.com"})
  ())

This is the core logic, and what I meant sort of by brute force. Given two patterns, we generate solutions for those two patterns. These solutions are just list of the variable bindings, generated using all-matches above.

Now to join these two lists of variable bindings, we make sure to keep only the valid combinations. E.g. if two patterns share a variable, then we keep only the variable bindings, where the values are common between them.

First is generating all solutions, by taking the cartesian product. This combines every variable binding map from the first result, with all the bindings from the other result. This is what I meant by brute force.

Now we filter these combinations. If there's a common variable in both results, then it's checked by value. Only keep the combination if the values match. If a variable exists in only one of the variable binding map, keep it. It might be used by the subsequent patterns .

Let's combine all the above functions inside the process-patterns function.

(defn process-patterns [patterns facts]
  (let [initial-matches (all-matches (first patterns) facts)
        result (reduce (fn [acc pattern]
                         (merge-variable->facts acc
                                                (all-matches pattern facts)))
                       initial-matches
                       (rest patterns))]
    result))

(process-patterns
  '[[?id :user/name "richhickey"]
    [?id :user/email ?email]]
  db)
;; ({?id 11, ?email "rich@e.com"})

And with the modified select function, we update the run function.

(defn select
  [variables result]
  (->> result
       (mapv #(vec (vals (select-keys % variables))))
       set))

(defn run [db q]
  (let [{:keys [find where]} (parse-query q)
        result (process-patterns where db)]
    (->> result
         (select find))))

We can now run queries which includes multiple variables across multiple clauses.

(def db
  [[11 :user/name "richhickey"]
   [22 :user/name "tonsky"]
   [33 :user/name "pithyless"]
   [11 :user/email "rich@e.com"]
   [22 :user/email "nikita@e.com"]
   [33 :user/email "norbert@e.com"]
   [44 :org/name "clojure"]
   [55 :repo/slug "clojure/clojure"]
   [55 :repo/owner 44]
   [66 :repo/slug "tonsky/datascript"]
   [66 :repo/owner 22]])

(run db
     '[:find ?email
       :where
       [?id :user/email ?email]
       [?id :user/name "richhickey"]])
;; #{["rich@e.com"]}

(run db
     '[:find ?user ?email
       :where
       [?id :user/email ?email]
       [?id :user/name ?user]])
;;#{["pithyless" "norbert@e.com"]
;   ["tonsky" "nikita@e.com"]
;   ["richhickey" "rich@e.com"]}

(run db
     '[:find ?repo
       :where
       [?p :user/name "tonsky"]
       [?r :repo/owner ?p]
       [?r :repo/slug ?repo]])
;; #{["tonsky/datascript"]}

Limitations

The current implementation is very basic. It generates all possible options for each clause, and filters the valid ones. It works for simple data, but it does not support the major feature of datalog engine which is rules, and recursively querying data.

For example, querying all repositories

[:find ?name ?repo
 :where 
   (or [?p :org/name ?name]
       [?p :user/name ?name])
   [?r :repo/owner ?p]
   [?r :repo/slug ?repo]]

This will return names of the owner and the repository. But owners, can either be user, or organizations. Hence the or. This kind of query is not supported.

The other major feature is recursively querying.

For e.g. let's say we define a relation between repositories using fork.

["repo-2" :fork "repo-1"] ;; repo 2 is a fork of repo 1
["repo-3" :fork "repo-2"] ;; repo 3 is a fork of repo 2
["repo-4" :fork "repo-3"] ;; repo 4 is a fork of repo 3

Now we can have a query like finding all parents of "repo-4". It will only return "repo-3", but it should return all of the repos. (I don't think forks have this behavior in Github, but you get the point)

This requires the query engine to use rules ( i.e. fork ), and query all the possible solutions.

I am yet to implement these features. Studying more about logic programming for now, and will try to implement the rest.

There's also Learn Datalog Today. It's a tutorial to using Datalog. It's broken down into chapters, adding more features in each chapter. Quite similar to the conference talk I mentioned in the beginning. I might use this website and it's challenges next time when trying to implement more features.

Permalink

A constraint language, bedframe, and air handling trouble

Hi friends!

I had an absolute blast in New York City visiting all of y'all who replied to my last email. Shout out to those who offered to host me, even after I typo’d my dates and suggested I’d be in town for a month rather than a week — true Internet friendship right there!

NYC has a creative energy and depth that I haven’t felt elsewhere — London and Taipei by comparison have much better transit infrastructure (and aren’t as deafeningly loud), but New York definitely feels to me more like the “capital of the world” with all the mixing of social/economic classes, variety of national backgrounds, and enormous variety of snacks.

I also love how New Yorkers comment out loud and otherwise interact with strangers to navigate public situations together.

A simple constraint language

In between hanging with friends in New York, I found a few hours for programming and knocked out a new codeCAD prototype. So far this one is more of a Clojure library rather than full interpreter/language. The key idea is to lazily solve numeric variables only when the ground value is actually required (i.e., when calling OpenCascade C++ code that doesn’t know anything about constraints).

This allows one to use numeric variables and constraints fluidly across functions, with parameters acting as “in/out” variables.

Here’s an example (hypothetical syntax) showing how the relationship between a circle’s radius and diameter could be encapsulated within the circle function:

(defn circle [center: Point
              radius: Length = 1
              diameter: Length = 2]

  (constraint (= radius (* 0.5 diameter)))

  ;; imagine some call here that actually draws the circle
  ;; and so requires radius and diameter to be solved...
)

some-point = [0, 0]

;; numeric variable, initial value of 5
dia = ?5

(circle some-point
        :radius     5
        :diameter dia)

;; now when you use the dia variable, it's going to always be 10
;; This is forced by the circle fn constraint and radius argument
dia ;; => 10

My hypothesis is that this might be very useful for all sorts of reusable geometric stuff. We’ll see how that works out as I keep building it out and developing my own lil’ OpenSCAD-like system.

Bed completed

Back in July we discussed flatpack bed ideas, and I’m happy to report that I’ve completed a first prototype:

For the past few weeks it has successfully met the design objectives of:

  1. lifting the mattress off the floor
  2. creating a ton space where we can hide boxes

I’m pretty chuffed with how it looks, too. I wish I could take credit for the wild striping pattern, but that wasn’t my idea — one sheet just arrived like that (apparently within spec for CP/CP birch plywood) and we decided to roll with it.

The bed design is simple, with symmetric headboard/footboard panels joined to rails with M6 screws / threaded inserts and some metal dowels for additional shear support and to ease assembly:

I designed for the 180x200 cm Ikea mattress we already had, and the frame is pretty much all Ikea internally:

  • a pair of Lonset slats
  • resting on a Skorva metal midbeam
  • attached via metal plates that’re screwed into the headboard/footboard
  • laterally braced with thin metal strips

The latter parts aren’t available for purchase, but Ikea offers spare parts — and I can truthfully say that I’ve lost parts of the Malm bed I purchased in my 20’s.

Ikea spare parts order confirmation

Everything was cut out on my friend’s CNC:

This took two days, as we spent most of the first day debugging why his CNC couldn’t cut reasonable circles. This turned out to be backlash from a loose X-axis pinion/rack — I’m now two-for-two diagnosing this exact problem on the CNC routers I’ve cajoled my way into operating here in Amsterdam. Weird.

The pieces were then cleaned up and the edges rounded over using Bosch’s adorable trim router (highly recommended) and the MFT/Paulk-style workbench I designed and had CNC cut at the same time:

(The bed has MFT-style 20mm holes on 96mm centers, since I figured that if something went wrong during the CNC operations, I might as well end up an extra workbench top.)

Each leg is a pair of panels, with the inside one cut short for the mattress. The panels joined through the headboard/footboard with, uh, 3D-printed spacers and tube:

So far it seems to be holding up just fine; no idea how to delegate credit between PCTG’s stronger-than-PLA layer adhesion, the bearing friction of the spacers, and the axial metal screw within the tubes.

On the front of the legs, the tubes are hidden by brass cabinet pulls (which also serve as the nut into which the axial screw clamps the assembly tight).

Next I expect we’ll build some attachments for the bed like a large padded headboard, side table cup holders, plant stands, jacket hangers, etc.

Fan troubles

I used my powered air respirator while finishing the bed, but I found it a bit stuffy and so I’ve been trying to increase the airflow. I don’t have a great way of quantifying the airflow directly, so as a proxy I’ve used an FNB58 inline USBC tester to measure the power consumption of the fan running at full power for one minute.

Here are the results:

BFB1012H  0.214 Wh unimpeded
          0.118 Wh filter and hose

BFB1012UH Unable to run, draws too much current from powerbank even at lowest setting

BFB1012VH 0.213 Wh unimpeded
          0.091 Wh filter and hose
          0.101 Wh adapter (no filter) and hose
          0.106 Wh dual adapter (no filters) and hose
          0.113 Wh dual adapter but no hose.
          0.117 Wh hose only

I started with the “H” fan, but it felt stuffy so I bought the “UH” variant based on the AliExpress listing’s description: “super violent turbo blower”. Unfortunately, this turned out to draw too much current: nominally it’s 6A at 12 V => 72 W, and even at the lowest PWM speed it still triggers some kind of overcurrent shutdown mode in my USB-C power bank (rated at 22.5W, oops).

So then I ordered the “VH”, but from what I can tell it’s no better than the original “H”.

It seems like the airflow restrictions caused by the filter adapter and/or hose limit the fan to about half the power consumption of the unencumbered fan, which presumably means the airflow is substantially less than the rated capacity.

What should I try next?

  • A different model of centrifugal-style blower fan?
  • Pairing up two fans I already have into a single (larger?) hose?
  • A totally different style of fan?

If you know about moving air, I’d love to hear from you!

My RSS feed

Did you know my website has an RSS feed?

My friend Ivan reported that occasionally the items would all show up as “unread” in his RSS reader. I’ve dug into the issue, which turned out to be related to deploying my website/newsletter from different timezones — the static site generator I use calls Ruby’s Time.parse method, which always parses into a local datetime. Luckily this carefree, just-do-something dynamic language attitude also makes for an easy fix:

# from https://github.com/nanoc/nanoc/blob/1955f7c0bc86de9ad51b4bb62530754c65bc9e61/nanoc/lib/nanoc/helpers/blogging.rb#L250-L263
# modified so that the date is always assumed to be UTC midnight, so that published item IDs don't change as my laptop changes time zones while I'm traveling.
module Nanoc::Helpers
  module Blogging
    def attribute_to_time(arg)
      t = Time.parse(arg)
      Time.new(t.year, t.month, t.day, t.hour, t.min, t.sec, "+00:00")
    end
  end
end

if "4.11.6" != Gem.loaded_specs["nanoc"].version.to_s
  throw "Nanoc version changed, check that feed id duck punching works."
end

Misc. stuff

Permalink

Bury Buffer

I’ve been spending a lot more time recently with Emacs. While I never gave up on org-mode, my Emacs usage had otherwise been in decline. A few months ago, I started experimenting with some “new” affordances: vertico, marginalia, orderless, consult and embark. The combined UX of these tools was impressive, and unlike similar functionality in, say, IntelliJ, could be used in far more contexts. When it comes to mainstream software, not much has changed since I first played around with an Apple IIgs nearly 40 years ago - disjointed islands that the user must travel between over and over again. Emacs delivers a non-feudal vision of personal computing. Perhaps Jef Raskin was right all along.

Coming back to everyday Emacs usage resurfaced one of my frustrations: Emacs window management. Without getting into the weeds, I devised a system that works for me both on large screens with many window splits, and on a MacBook Air where I have at most one split. The following snippet was arrived at with the help of Claude running in gptel. I’m a Elisp noob, so I apologize in advance for any LLM vagaries.

(defun my/get-display-window
  (source-window &optional create-if-needed)
  (save-excursion
    (goto-char (window-start source-window))
    (or
      (window-in-direction 'right source-window)
      (let ((below-window (window-in-direction 'below source-window)))
        (when (and below-window
                   (not (window-minibuffer-p below-window)))
          below-window))
       (when create-if-needed
         (split-window source-window nil 'below)))))

The (goto-char ...) bit is necessary since the “right” window is the one to the right of the cursor not the current window. Without this bit you run into trouble if there is a horizontal split next to the current window.

Integrating this with a particular package eliminates the behavior of a “random” buffer being chosen for display. For example with magit:

(defun my/magit-display-buffer (buffer alist)
  (when-let ((target-window (my/get-display-window
                              (selected-window) t)))
    (set-window-buffer target-window buffer)
    target-window))

(add-to-list 'display-buffer-alist
  '("\\(magit-revision:\\|magit-diff:\\)"
     (my/magit-display-buffer)
    (inhibit-same-window . t)))

This setup combined with binding bury-buffer makes using Emacs windows superior to pretty much anything else I have on my computer. What is bury-buffer you ask? Every window has its own list of buffers it has displayed, bury-buffer pushes the top buffer for a window to the bottom of the list. So if some unruly package displays a buffer in some undesirable window, you can just bury it to get back to where you were.

The three other tools I find useful for dealing with Emacs windows:

  • avy, to jump to any location in any window quickly
  • ace-window, to jump to any window + some useful commands
  • tab bar mode, for managing a group of windows like ERC

I have quite a bit more to say about Emacs, but there are some ClojureScript things afoot as well, stay tuned!

Permalink

Gaiwan: September Recap

We had a busy month at Gaiwan! Here are some things that happened in September.

Blog Updates

First, Arne released a blog post on his stance on AI which sums up how we approach AI and LLMs at Gaiwan:

In addition, Laurence released his series of 30 articles about Fennel.

GitHub - humorless/fennel-fp-neovim: A series of articles exploring Fennel and Functional Programming in Neovim
A series of articles exploring Fennel and Functional Programming in Neovim - humorless/fennel-fp-neovim
alt

Lastly, we migrated all the blog posts on Lambda Island to our main website; you can check it out here. Check out some our favorite old posts!

Improve your code by separating mechanism from policy
by Arne Brasseur Years ago when I was still a programming baby, I read about a concept, a way of thinking about code, and it has greatly influenced how I think about code to this day. I’ve found it tremendously helpful when thinking about API design, or how to
alt
The Art of Tree Shaping with Clojure Zippers
This is a talk I did for the “Den of Clojure” meetup in Denver, Colorado. Enjoy! Captions (subtitles) are available, and you can find the transcript below, as well as slides over here. For comments and discussion please refer to this post on r/Clojure. Transcript Arne: My name is
alt

Open Source Updates

We released two new libraries:

  • co.gaiwan/mcp-sdk our take on a MCP (Model Context Protocol) implementation. We&aposve been playing around with the very cool Clojure MCP, but Clojure MCP only supports the STDIO transport. Here&aposs an example of how to use mcp-sdk to provide a clojure_eval tool. Check out the README for more information!
  • lambdaisland/dbus-client DBUS has become the IPC (inter-process communication) mechanism of choice on Linux. It&aposs used extensively to make things work smoothly in desktop environments and is also used on the system level, for example: to control Systemd resources. We found that the Java bindings were difficult to work with from Clojure, because they do runtime reflection on concrete interface implementations. Clojure is highly dynamic, unlike Java, so we created something that could talk to DBUS more directly. We&aposve pushed out a lot of code recently, and it&aposs now at the stage where early adopters can start experimenting with it.

Also, this past month, we closed up some issues and moved some projects over to GH actions in preparation for a new release soon for Kaocha / 考察 [kǎo chá]. Stay tuned for more open source news!

Upcoming Events

altMeetup Flyer for Arne&aposs talk about the Gaiwan Stack

Arne will be speaking at the London Clojurians online meetup about the Gaiwan Stack on Nov 11. As an online meetup, you can join from anywhere. Please sign up here: https://www.meetup.com/london-clojurians/events/311341652/

Let&aposs Talk! 🍵 ☕

At Gaiwan, we&aposre looking to have more conversations with our community. One of the reasons we&aposre called Gaiwan is that we want to savor a cup a tea, ideally with good company. So whether you have questions or suggestions about our open source projects or our stance on AI, just hit reply and reach out.

Permalink

2025 Board Nominations: Time to Vote

Clojurists Together is in the process of electing new board members to fill the four volunteer positions that are expiring this year. These positions are for two-year terms (approximately Nov. 2025 - Oct. 2027).

The board is responsible for governing the organization, selecting and administering the projects that are sponsored, and interacting with sponsors. It also facilitates decision-making and implementation for the future plans of Clojurists Together.

We received 10 applications from the Clojure Community. 4 are from current board members who would like to continue to serve. All of the candidates are talented Clojurists that we would love (and be honored) to have on our board.

The board has nominated all 10 candidates for the current board election (in alphabetical order of last names):

Next Steps

All current Clojurists Together Members should have received a link to their ballot to vote via email. If you haven’t received a ballot by 9th October (and you’ve checked your spam folder to see if it’s hiding there), please contact kdavis@clojuriststogether.org.

Voting will close on 16th October at midnight Pacific Time. The top four candidates, as voted on by members will fill our open slots. Please note that our bylaws stipulate that “there may not be more than two members under the control or close influence of the same entity. If this threshold is crossed then members must stand down until there are no more than two members from the same entity”. This year, there are 3 candidates from the same organization. So depending on the election results, we may need to make an adjustment after the votes are in.

We will announce the winners shortly afterward and you’ll be able to meet them at the Annual members meeting at 10 am Pacific time, October 29, 2025

Your Vote is Important!! Please Vote!

Meet the Candidates

Felix Barbalet

https://github.com/xlfe/

Company: Psithur
I’ve really enjoyed contributing to the Clojurists Together board over the past two years, and I hope I’ve brought a helpful view and been able to make a difference to help grow the Clojure ecosystem.

I’ve been using Clojure professionally for 5+ years and I’m now running a small software/data company which uses full stack Clojure. Clojure has been the continuation of a life-long learning journey for me, and I don’t see that slowing down. The kind of things I’m interested in are figuring out how to raise the profile of Clojure as a smart choice for large enterprises all the way down to startups, and to help reduce the barriers to entry to Clojure as a profession. I want to see Clojure continue to grow and I’d love to see more corporate support to bring Clojure to more people.

Bio: I’ve spent >15 years working as a technologist in a range of roles including analytics, data science, data engineering, software architecture and software engineering. I have experience across highly-regulated industries (Government), consulting and startups. I have a B Economics (Honours) with specialisation in advanced econometrics, statistics, game-theory and behavioral economics and a graduate certificate in Cyber Security.


Daniel Compton

https://github.com/danielcompton
x.com/danielwithmusic

Company: Whimsical
I’ve been a board member of Clojurists Together since the beginning, as well as the secretary/treasurer. I have been the main board member running day-to-day operations of Clojurists Together since we began.

Bio: I’m Daniel Compton. I live in New Zealand and have been a board member of Clojurists Together since it began, as well as the secretary/treasurer. I have been the primary board member running day-to-day operations of Clojurists Together since we began. I’d like to continue to serve the community through running Clojurists Together.


Rafal Dittwald

https://github.com/rafd

Company: Bloom Ventures
I see this as an opportunity to serve and give back to our great community. Clojurists Together is a critical part of the Clojure ecosystem, and I want to see it and the community grow. I believe CT could serve as the foundation, for, well, a Clojure Foundation, supporting the Clojure commons in various new ways - such as financial host on Open Collective, an intellectual property host, and maybe experiment with community participation in selecting projects (via, say, quadratic funding). …if other members are supportive, of course.

Bio: Rafal has been working with Clojure for over 10 years, primarily building web applications (many of which are open-source). He organizes Clojure Camp and has given a few talks (such as Solving Problems the Clojure Way). He has been a Clojurists Together member since its inception. He has served on the board of 3 for-profit corporations and 2 non-profit corporations.


Arthur Fücher

https://www.linkedin.com/in/arthur-fucher/
https://github.com/afucher ; https://www.youtube.com/@clojure-br
I am also member of InnerSource Commons (https://innersourcecommons.org/about/members/)

Company: Nubank
I’ve been working with Clojure for over five years, including three years on platform engineering and developer experience, building tools and infrastructure for developers. I also contribute to open-source Clojure projects and organize ClojureBR, the Brazilian Clojure community, which promotes meetups and content in Portuguese.

Clojure and its community have given me a lot, and I’d like to give back by helping Clojurists Together strengthen the Clojure ecosystem globally and bring the perspective of Brazil, where many companies and developers are producing high-quality open-source. My experience in tooling, open source, and community organizing equips me to evaluate projects, contribute ideas, and help make the organization more inclusive and impactful.

Bio: Arthur Fücher is a Clojure developer who in recent years has focused on improving developer experience and platform tooling. He contributes to open-source Clojure projects and organizes ClojureBR, the Brazilian Clojure community. Arthur is passionate about Clojure and dedicated to growing its community, sharing knowledge, and supporting initiatives that help more people discover and use the language.


Heather Moore-Farley

https://github.com/tophat8855

**Company:**Peerspace
I’ve been on the board for the past 4 years and have a good idea of the time commitment and work involved. I love working with Clojure and like learning about and funding projects that are important for the Clojure ecosystem.

Bio: Heather has been a software engineer for over 10 years and has been using Clojure and Clojurescript for almost 8 years. When she’s not coding, she’s biking and drumming.


Felippe da Motta Raposo

https://github.com/felippemr/
https://www.reanard.com/
https://www.linkedin.com/in/felippe-raposo/

Company: VitableHealth
I am inexperienced with Clojure and could provide a beginner’s perspective on projects and community initiatives that help people accelerate their learning experience. I could also help with creating targeted industry guides for fintech and healthtech – helping people clearly understand Clojure’s value proposition outlined to their specific needs.

Bio: Felippe has over a decade of experience building software, from a database-as-a-service platform managing thousands of resources to large-scale systems that power automated money movement in the venture ecosystem — and more recently, in healthcare. He’s contributed to open source projects and spoken at Python community events in Brazil. Outside of work, Felippe enjoys reading with his kids and has recently introduced his six-year-old son to the Captain Tsubasa manga.

I currently live in Sandy, UT. USA.


Christoph Neumann

https://christophneumann.dev/

Company: Nubank
I believe that it’s vital to fund open source development for Clojure to grow and remain viable for years to come. I’d like to join the board because I love the work Clojurists Together is doing, and I would like to help. I think I’m a good fit because I’ve been active in the Clojure community for a decade, and I’m the developer advocate on the Clojure core team.

Bio: Christoph is a Clojure fanatic, entrepreneur, and the Clojure team’s developer advocate. After earning a MS in Computer Science and most of a PhD, Christoph left academia to take functional programming and user-centric design back into industry. He went on to make exceptionally-reliable, high-performance Clojure systems for live sports and esports production. He’s worked with some of the biggest names in the industry, including the NFL, Wimbledon, Call of Duty, and The Overwatch League.

With his friend, Nate Jones, he hosts the “Functional Design in Clojure” podcast. For more than 100 episodes, they have helped object-oriented developers escape the perils of side effects through immutable, functional programming. Visit christophneumann.dev for links to everything.


Max Penet

https://github.com/mpenet

Company: Exoscale
I’m committed to continuing to make a positive impact on the community through CT’s work. I genuinely enjoy helping out whenever possible, especially through the various discussion channels. Having benefited greatly from this community myself, it feels only fair to give back by contributing my time and effort. Given my long-standing involvement and reliance on a healthy, thriving Clojure community for both personal and professional projects, I believe I could be a good fit for this role.

Bio: I currently work as a software engineer at Exoscale, a European cloud provider that depends heavily on Clojure. I’ve been an active member of the Clojure community and have used Clojure professionally for over a decade across various fields. Over the years, I have authored, contributed to, and maintain several open-source libraries. I also stay closely engaged with the community on a near-daily basis, always aiming to offer help whenever possible. Most importantly, I deeply appreciate what CT is doing and would love to be able to help sustain that effort for two more years.


Howard M. Lewis Ship

https://lewisship.net/
https://github.com/hlship

Company: Nubank
I have long experience contributing to the open source community as an individual and as a representative of my employers (such as Walmart and Nubank), and have a new role at Nubank leading our open source enablement team, where part of my mandate is to engage with existing open source organizations and communities. I’ve been interested in Clojure since approximately 2010 and actively using it professionally since 2013.

Bio: I’ve been involved in open source projects since at least 2003; starting with Java (and the Apache Tapestry framework, which I founded) and continuing more recently with Clojure projects, including the Lacinia GraphQL framework, the Pedestal web framework, and org.clj-commons/pretty, as well as other less prominent projects.

I’m very much interested in contributing back to the Clojure open source community, as I think these efforts are critical to the long-term viability of the Clojure language.


Daniel Slutsky

https://github.com/daslu

Company: consultant at Daniel Slutsky LTD, full-time contractor for endor.global.

I deeply care about the Clojure open-source communities and Clojure growth, and these topics have been my main focus during the last few years, at Scicloj and elsewhere.

Bio: I’m Daniel Slutsky, a mathematician and data science consultant who has been working with Clojure since 2013. I co-organize Scicloj, where our mission is “Helping Clojure grow in new fields” - including academia, research, education, and domains beyond traditional software engineering.

My approach to community building draws from years of experience in local social activist groups, emphasizing inclusive collaboration and helping newcomers find their place in action-oriented projects.

I help maintain several open-source projects including Clay (literate programming), Noj (data science toolkit), Tableplot (data visualization) and ClojisR (Clojure-R bridge). Through mentorship programs, conferences like SciNoj Light and Macroexapnd-2025, and ongoing study groups focused on tooling, AI, and real-world applications, I work to create accessible pathways for people to explore Clojure in their fields of interest.

As a recipient of Clojurists Together grants in 2024 and having collaborated with a few other grantees, I’ve seen firsthand how this support enables community-driven work that expands Clojure’s reach. I would welcome the opportunity to help CT in supporting projects that both strengthen our existing open-source ecosystem and welcome new voices from diverse fields.

Permalink

Commenting, i.e. adding docstrings to multimethods in Clojure

Code

;; commeting_multimethods.clj

(defn arg-type [arg]
  (cond
    (string? arg) :string
    (number? arg) :number
    :else :other))

(defn dispatcher [args]
  (map arg-type args))

(defmulti add

  "Adds two strings or nubers.

     Usage
     ----------

     ```clojure
     (add 1 2)
     (add \"a\" \"b\")
     ```
    "

  (fn [& args] (dispatcher args)))

(defmethod add '(:number :number) [a b]
  (+ a b))

(defmethod add '(:string :string) [a b]
  (str a b))


(doc add)

(add 1 2)
(add "a" "b")

The workaround I did before, which is not needed:

;; multimethod_commenting_work_around.clj

(defn arg-type [arg]
  (cond
    (string? arg) :string
    (number? arg) :number
    :else :other))

(defn dispatcher [args]
  (map arg-type args))

(defmulti multi-add (fn [& args] (dispatcher args)))

(defmethod multi-add '(:number :number) [a b]
  (+ a b))

(defmethod multi-add '(:string :string) [a b]
  (str a b))

(defn add
  "Adds two strings or nubers.

   Usage
   ----------

   ```clojure
   (add 1 2)
   (add \"a\" \"b\")
   ```
  "
  [a b]
  (multi-add a b))

(doc add)

(add 1 2)
(add "a" "b")

Notes

Permalink

core.async and Virtual Threads

core.async 1.9.829-alpha2 is now available, which adds support for Java virtual threads (ASYNC-262).

Threads must block while waiting on I/O operations to complete. "Parking" allows the platform to unmount and free the underlying thread resource while waiting. This allows users to write "normal" straight line code (without callbacks) while consuming fewer platform resources. Clojure core.async go blocks until now used an analyzer to rewrite code with inversion of control specifically for channel parking operations (the ! async ops like >!). Other blocking operations (!! channel ops or arbitrary I/O ops) are not allowed.

Since Java 21, virtual threads implement I/O parking in the Java platform itself - that capability is a superset of what go blocks provide by supporting all blocking I/O operations. Because virtual threads are a superset of go block capabilities, go blocks can now be reimplemented using virtual threads without changing their semantics.

Using virtual threads

This release reimplements go blocks using virtual threads when available (Java 21+). go blocks retain their existing semantics (! channel ops park, blocking I/O not allowed) but do not require loading or running the analyzer. core.async is faster to load (when using Clojure >= 1.12.3) and faster to compile go blocks (no IOC). No code or configuration changes are required.

io-thread

io-thread was added in a previous core.async release and is a new execution context for running both channel operations (parking or blocking) and blocking I/O operations (which are not supported in go). Since alpha2, io-thread blocks also run in virtual threads.

Virtual thread control

A new system property clojure.core.async.vthreads has been added with these values:

  • (unset, default) - core.async will opportunistically use virtual threads when available (≥ Java 21) and will otherwise use the old analyzer impl. io-thread and :io thread pool will run on platform threads if virtual threads are not available. If AOT compiling, go blocks will always use IOC (no change).

  • target means that you are targeting virtual threads. At runtime from source, go blocks will throw if vthreads are not available. When AOT compiling, go blocks are always compiled to be run on vthreads and will throw at runtime if vthreads are not available (Java <21).

  • avoid means that vthreads will not be used by core.async - you can use this to minimize impacts if you are not yet ready to utilize vthreads in your app. If AOT compiling, go blocks will use IOC. At runtime, io-thread and the :io thread pool use platform threads.

Note: existing IOC compiled go blocks from older core.async versions continue to work (we retain and load the IOC state machine runtime - this does not require the analyzer), and you can interact with the same channels from both IOC and virtual thread code.

Feedback wanted!

We are very interested in feedback on performance of existing core.async programs, whether that is observable latency or throughput of the code, or differences in heap consumption and cleaning. Please give us feedback in #core-async on Clojurians Slack!

Permalink

Senior Software Developer at Horizon Investments

Senior Software Developer at Horizon Investments


We are Horizon. We empower financial advisors by providing technology-enabled investment product and by helping them make AND communicate investment and financial planning decisions. We are looking for a talented full-stack web engineer to join our small team.

The Senior Software Developer role at Horizon Investments is focused on writing quality full-stack code and actively participating in the full software development life cycle. Programming is the primary responsibility, though other opportunites may be available based on experience, interest and need. This means that the role of Senior Software Developers at Horizon may include responsibilities like: dev ops, dev ex, report automation, and other tangential elements of the software development lifecycle.

Responsibilities

  • contribute to the firm’s codebase by writing well-tested, durable code.

  • participate in team meetings and provide insight to the SDLC.

  • collaborate with team members on improving the software development process.

  • contribute to the product ideation and business analysis processes.

  • suggest features and changes to applications where appropriate.

  • report issues and bugs, and help resolve them.

Skills

  • able to write concisely and clearly when collaborating with teammates.

  • strong logical programming skill with Clojure(script), Python, and/or other high-level languages.

  • comprehensive knowledge of web architecture and application patterns

  • experience building polished user interfaces with a variety of style oriented paradigms like: SASS, LESS, Tailwind, Tachyons, etc.

  • a firm grasp of git,

  • an insatiable desire to solve hard problems quickly,

  • a positive, self-starting attitude,

  • a teachable spirit and an eagerness to learn (and teach),

  • an understanding of, or interest in, financial technology, financial planning, investments, and/or markets.

Permanent US Residents only

Permalink

Building Software Faster with LLMs: Part 5 - Learning and Knowledge Accumulation

Note: This article series was written by me, with LLMs helping to refine the style and structure.

This is the final article in my series on working with LLMs at scale. Part 1 through Part 4 focused on building with LLMs—managing sessions, ensuring quality, coordinating parallel work.

This article explores a different theme: the place of learning in the era of LLMs.

Series Navigation: ← Part 4: Rapid Evolution

The Learning Paradox

Here’s the uncomfortable truth: to be effective with LLMs today, you need to know the craft. When using Claude to write code, I rely on decades of software engineering experience—I know what good code looks like, what patterns to avoid, what trade-offs matter.

But people training today don’t have the same need to learn that I did. Why spend years mastering the intricacies of memory management, algorithmic complexity, or API design when an LLM can handle most of it?

This creates a void in expertise. If fewer people develop deep craft knowledge because LLMs handle the implementation details, where does the next generation of experts come from? The ones who can steer the LLMs, evaluate their output, and make good architectural decisions?

It’s likely a temporary problem. There’s no reason to think that I myself—and my ways of thinking—cannot eventually be replaced by LLMs. The “steering” and “judgment” I provide today may be automated tomorrow.

But for now, we’re in this transition. And I still value learning deeply. It’s been a core motivation for me throughout my career.

So here’s how I use LLMs to accelerate my own learning, working with my fallible human brain that still needs repetition, reinforcement, and deliberate practice to internalize complex ideas.

Prompting as a Learned Skill

I’ve had over 2,000 conversations with AI. I’m still learning how to prompt better. It’s a skill, and one I recommend others invest in.

Good prompts aren’t just about getting answers—they’re about shaping the learning experience to match how you learn best. Here are some examples from my own learning journey.

Example 1: Catching Up on React

I used React when it first started, but it’s changed dramatically. Instead of asking “explain React hooks,” I framed it as a structured learning path:

I used React at the beginning when it started but now it has changed a lot.
Do you think you can help me be up to speed with all the concepts in a few
bite-size lectures with example code?

I am interested to cover at least: hooks, effects, callbacks, routing,
state management, async operations, axios, reusable components, performance,
and the history of the framework.

Key elements of this prompt:

  • Context: I’m not a beginner, I’m catching up
  • Format: Bite-size lectures with code
  • Scope: Specific topics I need to cover
  • Goal: Get up to speed, not comprehensive mastery

This gave me a structured curriculum tailored to my experience level.

Example 2: Understanding Anthropic

When I wanted to learn about Anthropic as a company, I didn’t ask for a Wikipedia-style overview. I asked for serialized, digestible stories:

I want you to give me bite-size stories about Anthropic, the AI company.
The stories have to come in chronological order, one at a time (wait for me
to say continue). They have to be few paragraphs long to cater to the low
attention span of human beings in the 21st century.

Cover: the origin of the company, the founders, key people, Claude, ethics,
Sonnet and Opus, Claude Code, etc. Aim for 10 stories total for now.

Key elements:

  • Format: Serialized stories, not a wall of text
  • Pacing: One at a time, controlled by me
  • Tone: Acknowledges modern attention spans
  • Coverage: Specific topics I care about
  • Scope: 10 stories sets expectations

This turned learning into an interactive experience rather than passive reading.

Pattern 1: Heavily-Annotated Code (Worksheets)

When learning a new codebase or pattern, I ask Claude to generate worksheets—code with extensive inline comments explaining every decision.

This works because code with explanations serves as a learning artifact you can read, annotate, and reference later. The key is being specific about what you want explained:

Generate [concept] code with inline comments explaining:
- Why this approach (vs alternatives)
- Common mistakes
- Performance considerations
- When to use / not use
- How it connects to [related concept]

I’ve used this pattern to learn:

  • Rust’s ownership system
  • Clojure’s transducers
  • Kubernetes operators
  • Distributed system patterns
  • Database indexing strategies

Pattern 2: Flashcards for Spaced Repetition

Reading annotated code helps understanding, but doesn’t ensure retention. For that, I use flashcards with spaced repetition.

The Flashcard System

I built a flashcard CLI that implements spaced repetition algorithms (SM-2, Leitner boxes).

But manually creating flashcards is tedious. So I ask Claude to generate them.

Example: Learning System Design

Generate flashcards for distributed systems concepts. Format as JSON.
Focus on: CAP theorem, consensus algorithms, partitioning strategies.

Each card should have:
- A precise question
- A complete answer
- Tags for organization
- Notes with additional context

Claude generates:

[
  {
    "question": "In the CAP theorem, what does 'Partition Tolerance' actually mean?",
    "answer": "The system continues to operate despite network partitions (messages between nodes being lost or delayed). You must choose: consistency (all nodes see same data) or availability (all requests get responses) during a partition.",
    "tags": ["cap-theorem", "distributed-systems", "fundamentals"],
    "notes": "Common misconception: You can't 'give up' partition tolerance—network partitions will happen. You choose between CP (consistent but not available during partition) or AP (available but inconsistent during partition)."
  },
  {
    "question": "What is the key difference between Raft and Paxos consensus algorithms?",
    "answer": "Raft uses leader election + log replication (simpler to understand). Paxos uses proposal numbers + acceptors (more flexible but complex). Raft is easier to implement correctly.",
    "tags": ["consensus", "raft", "paxos", "algorithms"],
    "notes": "Raft was explicitly designed to be more understandable than Paxos. In practice, most systems use Raft (etcd, Consul) or Raft-like algorithms (Zab in ZooKeeper)."
  },
  {
    "question": "When should you use hash-based partitioning vs range-based partitioning?",
    "answer": "Hash partitioning: Uniform distribution, good for random access. Range partitioning: Preserves ordering, good for range queries. Choose based on your query patterns.",
    "tags": ["partitioning", "sharding", "database"],
    "notes": "Real example: User IDs → hash partition (uniform load). Timestamps → range partition (range queries like 'last 24 hours'). Hybrid approaches exist (compound keys)."
  }
]

I import these directly into my flashcard system:

cat system_design_cards.json | ./flashcards bulk-create --project system-design

Now I can quiz myself:

$ ./flashcards quiz --project system-design --limit 5

Question 1/5: In the CAP theorem, what does 'Partition Tolerance' actually mean?

[Show answer? y/n]

The Spaced Repetition Loop

Here’s how the system works:

  1. Generate flashcards from concept (Claude)
  2. Import into flashcard CLI
  3. Quiz yourself (spaced repetition schedules reviews)
  4. Rate confidence (1-5): Low = see again soon, High = see much later
  5. Repeat over days/weeks until internalized

The beauty: I can generate flashcards for any technical topic in minutes. No manual card creation. No finding existing decks that don’t quite match my needs.

The Meta-Pattern: AI as a Learning Compiler

These patterns share a common insight:

LLMs can transform any learning goal into a personalized learning artifact.

You provide a concept to learn, your preferred style (flashcards, annotated code), and your knowledge gaps. Claude generates materials tailored to you—at the right complexity, in the format you learn best from.

Practical Workflow

My typical learning workflow:

  1. Generate annotated code: Complete implementation with explanatory comments
  2. Extract flashcards: Key concepts become cards for spaced repetition
  3. Quiz over time: Spaced repetition embeds knowledge long-term

This takes 30-60 minutes to generate materials, then ongoing review. Traditional learning (reading papers, watching videos, manual note-taking) would take hours or days to achieve the same depth.

The Flashcard System Details

Since I mentioned my flashcard system, here’s how it works:

Features:

  • Multiple spaced repetition algorithms (SM-2, Leitner, fixed intervals)
  • Project organization (tag by topic)
  • Adaptive quiz sessions (repeat failed items immediately)
  • JSON import/export (easy to generate with LLMs)
  • CLI for terminal-based studying

Usage:

# Create from Claude-generated JSON
flashcards bulk-create --file cards.json --project rust

# Start adaptive quiz session
flashcards quiz --project rust --limit 10

# Review a specific card
flashcards review <id> --confidence 4

# See statistics
flashcards stats --project rust

The adaptive sessions are key: if you get a card wrong (confidence < 3), it immediately re-queues it. You practice until you get it right, then spaced repetition takes over for long-term retention.

Key Learnings

After a year of using LLMs for learning:

Annotated code beats plain code. The “why” matters more than the “what.”

Flashcards with spaced repetition work. Generate hundreds of cards in minutes, internalize them over weeks.

Active learning beats passive. Quizzing yourself is more effective than rereading notes.

Conclusion: Beyond Code Generation

This series has been about working with LLMs at scale—managing multiple sessions, ensuring quality, building coordination patterns. But it’s also been about a fundamental shift in how we work with code and knowledge.

LLMs aren’t just code generators. They’re:

  • Productivity multipliers (Part 2)
  • Quality enforcers (Part 3)
  • Coordination layers (Part 3)
  • Learning accelerators (Part 5)

The developers who thrive in this new world won’t be the ones who use LLMs as fancy autocomplete. They’ll be the ones who build workflows, tooling, and patterns that let them work at 10x their previous speed while maintaining quality and continuously learning.

My system—tmux integration, telemetry, memento, smoke tests, code review workflows, flashcards—is just one approach. Yours will look different based on your needs, your style, your projects.

But the core principles apply:

  • Make the invisible visible (ergonomics, telemetry)
  • Enforce quality automatically (smoke tests)
  • Share context between sessions (memento)
  • Learn continuously (flashcards, annotated code)

The future of software development is human-AI collaboration. Not humans being replaced, but humans being amplified. The question isn’t whether to use LLMs—it’s how to use them most effectively.

I hope this series has given you ideas, tools, and patterns you can adapt to your own workflow.

Now go build something amazing.


If you’re interested in any of the tools or patterns mentioned in this series, feel free to reach out. I’m happy to discuss what you find compelling and share more details.

Permalink

Building Software Faster with LLMs: Part 3 - Higher-Level Abstractions

Note: This article series was written by me, with LLMs helping to refine the style and structure.

In Part 1 I described the pain points of working with multiple LLM sessions. Part 2 covered the ergonomics layer that made individual sessions manageable.

But ergonomics alone isn’t enough when you’re running 5-10 parallel Claude sessions. You need coordination, quality enforcement, and shared context. This article covers the higher-level abstractions that make LLM teams actually work.

Series Navigation: ← Part 2: Ergonomics | Part 4: Rapid Evolution →

The Smoke Test Paradigm: Designing Software for Rapid Iteration

Here’s the key insight: software design principles that help human developers also help LLMs. The same things that trip up human coders—complex interfaces, tight coupling, unclear contracts—trip up LLMs too.

When building software that LLMs will write and modify, the classic principles still apply:

  • Modular code: Small, well-defined components
  • Simple interfaces: Clear inputs and outputs
  • Loose coupling: Changes in one area don’t cascade
  • Fast feedback: Know immediately when something breaks

The difference is velocity. LLMs can iterate 10x faster than humans—but only if the feedback loop is tight. That’s where smoke tests become critical.

Why Smoke Tests Over Unit Tests?

I tried comprehensive unit test suites. They worked, but the overhead was crushing:

  • Writing tests took longer than writing features
  • Tests became brittle as code evolved
  • Mocking and fixtures added complexity
  • False positives made me ignore failures

The problem: unit tests are designed for human-paced development. When Claude can refactor an entire module in 30 seconds, waiting 5 minutes for a full test suite kills momentum.

Instead, I adopted smoke tests: simple, end-to-end checks that verify the system works. Run in seconds. Clear pass/fail. No ambiguity.

Example from my flashcards project (test/smoke_test.sh):

#!/bin/bash
# Smoke test: Does the basic workflow work?

# Create a flashcard
./flashcards create \
    --question "What is 2+2?" \
    --answer "4" \
    --project "math"

# Get quiz items
./flashcards quiz --limit 1 | grep "What is 2+2?"

# Review it
./flashcards review <id> --confidence 5

# Check it's in the list
./flashcards list | grep "What is 2+2?"

echo "✅ Smoke test passed!"

That’s it. No mocking. No fixtures. No complex assertions. Just: Does it work end-to-end?

The Make Test Convention

Every project has a Makefile with a test target:

test:
	@echo "Running smoke tests..."
	@./test/smoke_test.sh
	@echo "✅ All tests passed"

Claude knows this convention. After every code change, it automatically runs make test. If tests fail, Claude must fix them before continuing.

This simple pattern has caught hundreds of regressions. Claude refactors a function? Tests catch it. Claude renames a variable? Tests catch it. Claude adds a feature? Tests verify it.

Why This Works

Smoke tests have unique advantages for LLM workflows:

  1. Fast: Run in seconds, not minutes
  2. Clear failures: “Command failed” is unambiguous
  3. Self-documenting: Reading the test shows how the system should work
  4. Easy to maintain: When features change, tests are obvious to update
  5. Catches real issues: Integration problems that unit tests miss

The trade-off: you don’t get fine-grained coverage. But in my experience, that’s fine. I’d rather have 90% confidence in 5 seconds than 99% confidence after 5 minutes of test runs.

Memento: Shared Context Between Sessions

The core challenge of parallel LLM sessions: they don’t know about each other.

Session A refactors the authentication system. Session B adds a new feature that uses authentication. Session A’s changes break Session B’s code—but Session B has no idea until tests fail.

I needed a shared knowledge base. Enter memento.

What Is Memento?

Memento is my note-taking system built on org-roam, which implements the Zettelkasten method for networked thought. I expose it to Claude via MCP (Model Context Protocol).

Think of it as a shared brain for all Claude sessions—a personal knowledge graph where notes link to each other, concepts build on each other, and every LLM session can read and contribute to the collective knowledge.

Memento note system showing interconnected knowledge graph

Key features:

  • Public notes tagged with PUBLIC are accessible via MCP
  • Searchable with full-text search
  • Structured with org-mode properties and tags
  • Version controlled in git
  • Persistent across sessions

The Global Context Pattern

Every Claude session starts by reading the claude-global-context note:

;; Automatically loaded by Claude at session start
(mcp__memento__note_get :note_id "claude-global-context")

This note contains:

  • My coding preferences
  • Project structure
  • Common pitfalls
  • Tools available (memento, MCP servers, custom scripts)
  • Reminders (never access ~/.roam directly, always use MCP)

As I discover patterns, I add them to this note. Every future Claude session gets that knowledge automatically.

Example from my global context:

## 🧪 Testing Approach:
- Write tests for new features
- Rely on smoke tests for projects (trigger with `make test`)
- **Whenever all tests pass after a change, make a commit with a descriptive message**

## 🔧 ELISP DEVELOPMENT WITH DOOMSCRIPT:
See the note tagged `elisp` for patterns and testing approaches

Session-Specific Context

For complex projects, I create dedicated notes:

  • memento-clojure-patterns: Clojure idioms and anti-patterns
  • appdaemon-testing-guide: How to test Home Assistant automations
  • mcp-server-patterns: How to build reliable MCP servers

When Claude works on these projects, I explicitly reference the notes:

Read the note `mcp-server-patterns` and apply those patterns
to this new server implementation.

Claude reads the note, absorbs the context, and applies it. The next Claude session working on the same project does the same thing—they’re building on shared knowledge.

Coordination Patterns (Experimental)

I’m experimenting with explicit coordination notes for parallel sessions:

# working-on-memento-refactor

## Current State
- Session A: Refactoring CLI argument parsing (IN PROGRESS)
- Session B: Adding new `bulk-update` command (WAITING)
- Session C: Updating tests (COMPLETED)

## Decisions Made
- Use argparse instead of manual parsing (Session A, 2025-09-28)
- All commands must support JSON output (Session B, 2025-09-27)

## Upcoming Work
- [ ] Migrate all commands to new arg structure
- [ ] Add integration tests
- [ ] Update documentation

Each session reads this note before starting work. Session A updates its status when done. Session B sees that and can proceed safely.

This is informal right now—I’m still exploring better patterns. Some ideas:

  • Barrier functionality: Session B blocks until Session A completes
  • Lock mechanism: Only one session can modify a file at once
  • Dependency tracking: Session C depends on Session A and Session B

I’m considering building an MCP server specifically for project coordination. Something like:

# Hypothetical coordination MCP server
mcp_coordinator.claim_file("src/parser.py", session_id="A")
# Other sessions get an error if they try to edit it
mcp_coordinator.add_barrier("refactor-complete", required_sessions=["A", "B"])
mcp_coordinator.wait_for_barrier("refactor-complete")  # Blocks until A and B finish

The Supervisor Pattern: Orchestrating LLM Teams

When I need major changes, I run multiple Claude sessions in parallel:

  • Session A: Implements feature X
  • Session B: Writes tests for feature X
  • Session C: Updates documentation
  • Session D: Reviews changes from A, B, and C

This is the supervisor pattern—but instead of manually coordinating, I use an LLM to generate prompts for other LLMs.

The Meta-LLM Approach

Planning parallel work is itself an LLM task. I have Claude generate the work breakdown and individual prompts:

  1. I describe the goal to a planning session: “Implement feature X with tests and docs”
  2. The planner LLM creates:
    • A work plan broken into phases (represented as a DAG)
    • Individual prompt files for each parallel task
    • Memento-based coordination scheme
    • A supervisor prompt for monitoring progress
  3. I review and launch using my automation tools

This meta-approach scales much better than manual coordination. The planner understands dependencies, estimates complexity, and generates consistent prompt structures.

The Tooling: claude-parallel

I built claude-parallel to automate the workflow:

# Step 1: Generate the plan
claude-parallel plan -P myproject -p "requirements.txt"

# This launches a planning Claude session that:
# - Breaks work into phases and tasks
# - Creates prompt files in ~/.projects/myproject/prompts/
# - Generates plan.json with the dependency DAG
# - Creates a supervisor.txt prompt for monitoring

# Step 2: Dispatch work to parallel sessions
claude-parallel dispatch -p prompts/phase-1-task-auth.txt src/auth.py
claude-parallel dispatch -p prompts/phase-1-task-tests.txt tests/test_auth.py

The dispatch command automatically:

  • Creates a new tmux window
  • Changes to the file’s directory
  • Launches Claude with the prompt
  • Monitors completion via memento notes

Tmux Automation

For complex projects with many parallel sessions, I use generate_tmuxinator_config:

# Generate tmuxinator config from prompt files
generate_tmuxinator_config -n myproject prompts/*.txt > ~/.config/tmuxinator/myproject.yml

# Launch all sessions at once
tmuxinator start myproject

This creates a tmux session with:

  • One window per prompt file
  • Proper window naming for easy navigation
  • All sessions starting in the correct directory

How I Do It Today

  1. Write high-level requirements in a text file
  2. Run claude-parallel plan to generate work breakdown
  3. Review the generated prompts (adjust if needed)
  4. Launch sessions via claude-parallel dispatch or tmuxinator
  5. Use memento for coordination (automatically set up by the planner):
    • Sessions read/write status notes
    • Sessions check phase completion before starting
    • Blocker notes communicate issues
  6. Rely on smoke tests to catch integration issues
  7. Monitor via tmux status indicators (see Part 2) or run the supervisor prompt

Persona-Driven Architecture

Assigning roles to sessions improves output quality, but I use personas differently than you might expect.

I use Robert C. Martin (Uncle Bob) as the planner and architect. When breaking down a complex feature into parallel tasks, I ask the planner session:

You are Robert C. Martin (Uncle Bob). Review this feature request and break it
down into clean, well-separated tasks for parallel implementation. Focus on
SOLID principles and clear interfaces between components.

This gives me a work breakdown that follows clean architecture principles: small, focused components with clear responsibilities.

Then for the worker sessions (the ones actually implementing the tasks), I experiment with different prompts. Sometimes specific personas help:

  • “You are obsessed with performance and correctness” for algorithm-heavy code
  • “You are paranoid about edge cases and defensive programming” for input validation
  • “You value simplicity above all else, avoid any unnecessary complexity” for utility functions

Other times, I just use the task description from the planner without additional persona framing. I’m still experimenting with what works best for different types of work.

What’s Missing

Current gaps in my supervisor pattern:

  1. No automatic conflict detection: I manually ensure sessions don’t edit the same files
  2. No rollback mechanism: If Session A breaks tests, I manually revert
  3. No progress tracking: I eyeball tmux windows instead of having a dashboard
  4. No automatic merging: I manually integrate changes from parallel sessions

These are ripe for automation. The MCP coordination server would solve 1-3. Number 4 might need a specialized “merger” session that reads changes from all other sessions and integrates them.

Knowledge Accumulation Over Time

Traditional LLM conversations are ephemeral. Each session starts fresh. But with memento, knowledge compounds.

Example workflow:

  1. Week 1: I discover that MCP servers should validate input strictly
  2. I add to global context: “MCP servers must validate all inputs and return clear error messages”
  3. Week 2: Claude builds a new MCP server, automatically applies that pattern
  4. Week 3: I discover another pattern (connection pooling), add it to global context
  5. Future sessions: Apply both patterns automatically

Over months, my global context evolved from 50 lines to 500+ lines of hard-won knowledge. New Claude sessions are more productive from day one.

The Memento Notes Index

To make knowledge discoverable, I maintain a memento-notes-index:

## Development & Technical Guides

- **mcp-server-patterns**: Patterns for building reliable MCP servers
- **smoke-test-paradigm**: Why smoke tests work better than unit tests
- **elisp-testing-guide**: Fast testing with doomscript
- **code-review-guide**: How to review code and log issues for AI

## Quick Lookup by Use Case

- Building MCP servers → `mcp-server-patterns`
- Emacs development → `elisp-testing-guide`
- Testing frameworks → `smoke-test-paradigm`

When Claude asks “How should I structure this?”, I can say: “Check the notes index for relevant guides.”

Key Learnings

  • Smoke tests catch 90% of issues with 10% of the effort
  • Shared context prevents reinventing the wheel
  • Personas improve output quality
  • Informal coordination works for 3-5 sessions
  • Capture every discovery in memento

What’s Next

The patterns in this article work but aren’t fully automated. I’m manually coordinating sessions, manually managing shared context, manually merging changes.

Part 4 covers experiments and works-in-progress: the project explorer tool, Emacs integration for code review, diff workflows, and ideas that didn’t quite work out.

Part 5 shifts to learning: using Claude to generate flashcards, worksheets, and annotated code for studying complex topics.


Continue Reading: Part 4: The Way We Build Software Is Rapidly Evolving →


If you’re interested in any of the tools or patterns mentioned in this series, feel free to reach out. I’m happy to discuss what you find compelling and share more details.

Permalink

Clojure South: Nubank brings the community together in Brazil and beyond

When Nubank began its journey, we knew that scaling infrastructure alone wouldn’t be enough — we also needed predictability, clarity, and simplicity. Choosing Clojure was part of that path. The functional language brings concepts like immutability, a fast-feedback ecosystem powered by the REPL, and elegant ways to explore problems — all of which aligned perfectly with what we set out to build.

Cognitect, the organization that maintains Clojure and Datomic, has been an indirect partner in this engineering culture, one that values not only what we deliver but also how we deliver it.

With that in mind, we’re excited to announce the second edition of Clojure South, scheduled for October 6–7, 2025, in São Paulo and organized by Nubank. The conference was designed to be a moment of connection, hands-on learning, and exchange among those already using Clojure & Datomic, the curious, those considering adoption, and anyone eager to grow as a developer.

What to expect from the program

The Clojure South agenda is designed to deliver value from the very start, with practical content, inspiring talks, and opportunities for technical reflection. On October 6, the focus will be on workshops tailored to different levels of experience.

In the morning, Christoph Neumann, Clojure Developer Advocate, will lead an Introduction to Clojure workshop. This hands-on session will cover the essentials: syntax, immutable data structures, and REPL-driven development. With live coding and exercises, participants will leave able to write their own programs in Clojure.

In the afternoon, Carolina Silva and Hanna Figueiredo, Software Engineers, will host a Datomic workshop, exploring the database as a fact-oriented system. Participants will dive into transactions, Datalog queries, working with temporal data, and how immutable data modeling enables new ways of thinking about history, auditing, and system evolution.

On October 7, the conference will highlight inspiring presentations from both Brazilian and international speakers. Among the keynotes, Alessandra Sierra, Principal Software Engineer, will reflect on the 12-year journey of the Component library in her talk “12 Years of Component.” Alex Miller, also a Principal Software Engineer, will explore the “Joy of Data,” showing how Clojure makes it easy to treat data as values that can be transformed and composed seamlessly. 

 Across both days, there will be simultaneous translation in Portuguese, Spanish, and English to ensure that language is never a barrier to participation. The program will also include moments for informal connection, such as a welcome coffee, coffee breaks, and a happy hour, all designed to strengthen ties between participants, speakers, and companies.

Why it matters

The program offers two dimensions of value. The first is practical experience: workshops go beyond listening, giving attendees the chance to apply, test, make mistakes, and learn. This helps internalize concepts that often remain abstract.

The second is real-world visibility: seeing how companies apply Clojure & Datomic in production, understanding concrete challenges, and learning from practitioners who have already done it. This shortens the gap between “curiosity” and “adoption,” whether for small teams or large organizations.

Clojure South 2025 will take place at Nubank’s Spark office, located at Av. Manuel Bandeira, 346 — Vila Leopoldina, São Paulo. The venue offers great facilities and is surrounded by hotels, public transportation, and infrastructure that makes attendance more convenient.

The official website includes all details about registration, ticket pricing, food options, the venue, and how to get there.

Conclusion

For us at Nubank, organizing Clojure South is a way to put into practice something we deeply believe: technology makes its greatest impact when it goes beyond code — when it involves culture, community, and shared experiences. We want to contribute to making Clojure and Datomic more accessible and widely known, because they are powerful tools for building complex, ever-evolving systems.

Whether you already code in Clojure, are curious about functional programming, or simply want to learn about technical solutions that tackle real challenges of scale and evolution, Clojure South is for you. Registrations are already open! Join us in São Paulo this October, bring your curiosity and challenges, and let’s start a new chapter for the Clojure community in South America.

The post Clojure South: Nubank brings the community together in Brazil and beyond appeared first on Building Nubank.

Permalink

The Vibe Coding Loop

The following is an excerpt from the forthcoming book Vibe Coding: Building Production-Grade Software With GenAI, Chat, Agents, and Beyond by Gene Kim and Steve Yegge.


The vibe coding loop looks similar to the traditional developer loop. But when you’re coding with AI, every step becomes critical. As you’ll see soon, you can’t fall asleep at the wheel. If you do, you’ll soon wind up with frustrating and expensive rework, a theme we continue to explore throughout Part 2 of the book. Let’s talk about the vibe coding loop. Here’s what it can look like:

  • Frame your objective: Give your AI collaborator a clear, concise overview of what outcome you’re aiming for. Be specific about what success looks like and why you’re building it.
  • Decompose the tasks: Break down what you’re trying to do into clear, achievable steps. In general, the smaller the steps, the better chance AI has to succeed. Even as AI grows more capable, small steps are always a good idea. Don’t hesitate to ask it to subdivide the big tasks (e.g., “Here’s what I’m trying to do. Propose a plan.”).
  • Start the conversation: Ask AI to generate a plan to achieve your goal, or give it instructions to get it started, such as what you practiced in the last chapter.
  • Review with care: The solution your AI comes up with might look correct, but until you have established a basis for trusting it, you need to review it.
  • Test and verify: You’re responsible for the quality of the code, whether you wrote it or AI did. This works best when writing your tests and expectations before generating the code—advocates of test-driven development (TDD) will rejoice. Fail fast, fix fast, and ask AI to help you spot subtle mistakes that might linger unnoticed.
  • Refine and iterate: Continue iterating until you achieve your goal.

By the way, once you’re somewhat experienced with this vibe coding loop, there is one more critical step to add:

  • Automate your own workflow: Begin automating away chunks of your workflow. Any friction creates huge opportunity costs. And any time you spend typing or copying/pasting/slinging slows down your vibe coding loop. If you’re doing anything manually, that is a cost you pay every time you try to vibe code.

Automating this toil will not only make you faster but will speed up your ability to experiment and innovate. We’ll talk more about the unexpectedly high benefits of this later in the book. (Hint: It’s the O in FAAFO). And if, at any time, you’re typing a lot or manually searching through data structures, stop and ask yourself: “Could I ask AI to help with this?” The answer is usually yes, and you’ll be faster and have more fun. 

War Story: Gene’s Video Excerpter

For the past fifteen years, Gene has been taking screenshots whenever he finds something interesting in podcasts or YouTube videos, hoping to revisit those moments eventually, maybe to write about someday or to further research an interesting fact. In practice, he rarely used them. It was too tedious to search through the screenshots, locate the original content, and find the exact quote he needed. The juice didn’t seem worth the squeeze. Optimistically, he held out hope that it might be someday and kept making screenshots. For fifteen years! We mentioned this story briefly in the Preface, but now we’ll show the details of how Gene was able to vibe code his way to success.

In our first vibe coding pairing session together, we set out to build something that could create video excerpts (clips) of YouTube videos directly from Gene’s screenshots. He would be able to dig up a picture and, with the click of a button, post that excerpt from the video. His new tool would also use the video transcript to add overlaid captions (subtitles) onto the clips.

We used ffmpeg, a super-powerful command-line tool that can process, convert, and manipulate video and audio files in almost any format. It’s notorious for having extremely complex command-line options and syntax, which makes the operations difficult to write and almost impossible to read afterward. With this complexity in mind, we were going to find out if AI could come to the rescue.

In the following sections, we’ll walk you through how Gene went through the vibe coding loop multiple times, using a chat assistant to build what he wanted. We recorded the forty-seven minutes it took for him to build it.

Frame the Objective

First, Gene explained to Steve what he was trying to build. He needed a tool to automate the process of creating a “highlights reel” from his extensive collection of video highlights, which were video screenshots he had taken on his phone. Before starting our session, he had converted those screenshots into the following data: the YouTube channel and video, as well as the start and end times of the video clip he wanted generated. He also had movie files and transcripts of those YouTube videos.

He aimed to create captioned video .mp4 files, with the transcript converted into subtitles that showed up in the video frame, so he could share on social media. Gene felt his thousands of screenshots were a treasure trove of the wisdom of others, of interesting research material, and of miscellaneous topics that people would be interested in. This tool would finally let him start sharing that accumulated wisdom.

Decompose the Tasks

Given the objective, Gene now needed to decompose his problem into tasks that he could implement with AI. He came up with the following tasks, which could be implemented and validated using AI:

  • Download the YouTube video and transcripts. (Gene had already done this using the fantastic yt-dlp.)
  • Extract a specified segment from the downloaded video using ffmpeg, based on the highlight’s start and end time stamps.
  • Extract the corresponding transcript for that segment from the existing transcript file.
  • Generate subtitles from the transcript text and time stamps.
  • Overlay those subtitles onto the video segment using ffmpeg.

For this project, Gene chose to use Claude via the Sourcegraph AI assistant inside his IntelliJ IDE, though any assistant (and any model) would have worked. This session occurred before autonomous agents, so he was vibe coding using regular chat. A skill that remains useful today with agents, because some problems will always best be solved with chat.

Gene’s vibe coding loop looked like this: He would type his prompt in the assistant window. AI would generate some code in the chat. Gene would copy and paste that answer into his editor, or in some cases, smart-apply it directly into the code with a button click. Ask, answer, integrate, over and over. And it worked! Boy, did it ever. As we shall see.

Task 1: Start Simple—Video Extraction

Gene’s first task was to extract a segment of the source video file. Here was his starting prompt:

Given an excerpt beginning and end (in seconds), give me the ffmpeg command to extract that portion of the video. Go ahead and shell out and put that into a file /tmp/output.mp4.

A short prompt, but it got the job done. No need to look up any ffmpeg documentation, no need to learn the command-line arguments, no need to learn time unit conventions. AI handled all the details. Within minutes, Gene and Steve had working code that could extract video clips. He opened the video file, and it looked great. Given the simple nature of this task, Gene decided tests were not needed. Gene was convinced that we could rely on ffmpeg working correctly, so we moved onto the next task. (You decide whether that was a good decision.) 

Task 2: Processing the Transcript

Next, Gene moved on to processing the transcript data. Given the start and end time of the highlight, he needed to extract the relevant transcript portions. Here was the prompt he used:

Here’s the video transcript (it’s a JSON array of objects). Write a function that, given a list of start and end ranges, extracts all the relevant entries in the transcript.

AI generated the function, which Gene copied into his Clojure code base. Although it ran correctly, this was a nontrivial function, so we needed test cases. This function computed intersections of time ranges in the transcript and seemed to have lots of places where the code might go wrong.

Gene gave our AI assistant another prompt: “Write some tests.” It generated several interesting test cases, exercising the different ways that time ranges might overlap. And indeed, one test case failed. 

This was a genuine teachable moment for both of us. Our AI assistant was sure that the failed case was due to an off-by-one error in the code. But we discovered the code itself was correct; it was the generated test cases that were wrong. So much for tests that “look good.”

This reminded us that AI is not always reliable. We had to stay vigilant and verify its answers—especially because AI almost always sounds confident and correct and explains why it’s correct in lengthy detail. In this case, it was right when it generated the initial code but completely wrong in guessing why the tests were failing.

We soon had a tested function, which, given a list of transcript start/end ranges, would correctly extract the text for that part of the transcript. So far, so good. 

Task 3: Caption Generation

Finally, we needed to add captions. This meant taking the transcript file and inserting it as captions that could be seen in the video frames. This was a large enough task that we decomposed it into the following subtasks:

First, we asked ChatGPT what caption formats ffmpeg supports. (Answer: SRT and ASS formats, which neither Gene nor Steve knew about before. And now we do!)

Gene then asked ChatGPT, “Give examples of SRT and ASS transcript files.” Gene chose the SRT transcript format because it had fewer fields and looked simpler to implement. Again, there is no need to become an SRT file format specialist. We then asked ChatGPT to generate the SRT file from the transcript segments.

Gene wrote this prompt:

Write a function to transform my list of transcript entries (a JSON array) into an SRT file.

Our AI assistant generated the code to do it, and it chose a great function name (which is sometimes more difficult than writing the function). Finally, we needed the subtitle text to be placed into the video frames. We learned that ffmpeg calls these “captions.” 

Modify the ffmpeg command to generate captions, using the specified SRT caption file.

If you watch the session recording, you can hear Gene gasp the moment he opens the video and sees the video excerpt with overlaid captions. We had not been vibe coding for long, barely over half an hour. And we hadn’t written many prompts. On the recording, Gene declared, “This is freaking incredible,” plus lots of expletives we had to censor out.

The Result

In a total of forty-seven minutes of pair programming using vibe coding techniques with chat, Gene had built a working video clip generator that achieved his goal:

  • Extract a portion of the source YouTube videos using the start/end time stamps.
  • Transform the podcast transcript file into caption texts and output to an SRT caption file, which ffmpeg can use as input.
  • Generate captioned text in the video frames using ffmpeg using the SRT caption files to overlay captions onto the extracted file.

Not bad for an hour’s work. It turned into an hour because, upon closer inspection, Gene and Steve noticed that two lines of captions were being displayed, and there was something wrong with the caption timing. They spent a few minutes trying to fix it, and then Gene promised to work on it that evening.

The next day, after Gene got his code working, he texted Steve: “Holy cow, I got this running! I had so much fun generating and posting excerpts, extracting every quote I found inspiring.” Steve had not expected that Gene—who is not a professional programmer—would have accomplished this in under an hour. Gene had finally created a way to plunder his fifteen-year-old treasure trove.

What’s better is that it turns out the video Gene was using for testing the code was a talk by Dr. Erik Meijer (whom you may recall from Part 1). When Gene posted a twelve-part series of his favorite quotes from that talk on social media, Dr. Meijer responded: “This looks amazing. Thanks for doing this. It helps grasp the talk even faster than just watching at 2x speed.”

Gene’s tweet got nearly a quarter million views. Clearly others were finding his treasure trove and excerpt format valuable. This is the kind of impact vibe coding can unlock.

Okay, if you’re super experienced, Gene’s programming feat might sound mundane. It’s mostly new code in a small code base, and the final product was smaller than what some professional developers might commit multiple times a day. Some of you could have written this whole program in a quarter of the time it took us pairing with vibe coding.

That’s fair. But it’s also not the point. The takeaway here is not “Oh ho, ha ha. AIs will never replace real programmers.” The point is that we were able to build it at all. The program never would have been written the old way, but Gene did it in under an hour (fast) with AI.

For Gene, this was a life-changing experience. Gene achieved FAAFO. He had considered this sufficiently so far from reach that he had never bothered trying (ambitious). After creating this program, he used it several times a week because it unlocked the value of thousands of interesting moments he captured while listening to podcasts. Best of all, it was fun, and it set in motion writing tons of other utilities, some of which he uses multiple times daily.

Here are some other takeaways from this early vibe coding session:

  • AIs are capable of handling small to medium tasks, including in less popular programming languages, and using fairly complex Unix command-line tools.
  • You interact with AI as if it were a senior pair programmer who’s so distracted that they can make serious mistakes from time to time.
  • Clojure is the future of programming languages. Ha, ha! We’re just conducting a test to see if you’re still reading. But we both do like Clojure a lot.

We did this little test in September 2024 (almost prehistoric AI times). Given all the advances in coding agents, we know we could complete this project today in a fraction of the time. A coding agent could doubtless have solved this problem in a couple of minutes. As AI improves, it will be able to handle larger and larger tasks. It’s possible that Gene’s video excerpting program could have been implemented in one shot—if not today, sometime in the future. But like when giving tasks to humans, the larger the task you have AI take on, the more that can go wrong.

The relevant skill is no longer code generation (i.e., typing out code by hand), but being able to articulate your goals clearly and create good specifications that AI can implement. Because of this, the principles here continue to apply to larger projects as AI’s capabilities scale up.

Bonus Task: Detecting the YouTube Progress Bar

In the Preface, Gene mentioned that he had his first inkling of how powerful chat programming could be as early as February 2024. While we’re talking about chat programming, here is a slightly expanded explanation of what happened.

For the non-iOS YouTube screenshots, he could ask the new ChatGPT-4 vision model to extract the current playback time displayed in the video player controls (e.g., “1:45”). But screenshots from the iOS YouTube app were different. They only showed a red progress bar with no visible time stamp. Without that timing information, he couldn’t automatically determine where in the video to create his excerpts.

On a whim, Gene typed into ChatGPT: “Here’s a YouTube screenshot. There’s a red progress bar under the video player window. Write a Clojure function that analyzes the image. March up the left side of the image to find the red progress bar.” The AI-generated code used Java 2D graphics libraries—ImageIO, BufferedImage, Color classes—which Gene had never used before. Gene hadn’t used bitmap functions since writing Microsoft C++ code in 1995. When the function correctly identified the progress bar on row 798 of the image on the first try, Gene sat slack-jawed.

Next, he extended the solution. “On that row, march right until you see a non-red pixel,” he prompted, and AI delivered code that calculated the exact playback percentage from the progress bar’s position. What would have taken him days of studying graphics APIs—if he’d attempted it at all—was working in under an hour. This code transformed thousands of iOS screenshots from unusable artifacts into valuable time stamps.

That’s what changed Gene’s life in 2024 and set the stage for his exciting adventure with Steve a year and a half later. Truly, FAAFO.

Onward

Gene’s video excerpting tool shows the vibe coding loop in action. By breaking down a complex task, collaborating with AI through conversation, and iteratively building a solution, Gene accomplished in under an hour what might never have happened otherwise. 

But, as valuable as this chat-based approach proved to be, it only scratches the surface of what’s possible with vibe coding. Later in the book, we’ll examine the prompts that Gene used and show what made them effective.

Before we do that, we’ll look at what we can do with autonomous, agentic coding assistants, or “coding agents,” and how they alter the vibe coding loop. 


Stay tuned for more exclusive excerpts from the upcoming book Vibe Coding: Building Production-Grade Software With GenAI, Chat, Agents, and Beyond by Gene Kim and Steve Yegge on this blog or by signing up for the IT Revolution newsletter.

The post The Vibe Coding Loop appeared first on IT Revolution.

Permalink

Copyright © 2009, Planet Clojure. No rights reserved.
Planet Clojure is maintained by Baishamapayan Ghose.
Clojure and the Clojure logo are Copyright © 2008-2009, Rich Hickey.
Theme by Brajeshwar.