Python to Wisp: The Lisp That Stole Python's Indentation

Python to Wisp: The Lisp That Stole Python's Indentation

Every Lisp programmer has heard the complaint: "too many parentheses." Every Python programmer has heard the praise: "the indentation makes it readable." Wisp asks a dangerous question: what if we put them together?

Wisp is an indentation-sensitive syntax for Scheme — specifically GNU Guile Scheme — standardized as SRFI-119. It takes the structure of Lisp and expresses it through indentation, the way Python does. The outer parentheses vanish. What remains is code that looks strangely like Python but thinks like Lisp — with macros, tail-call optimization, exact arithmetic, and homoiconicity.

This article walks through every major topic in the official Python tutorial and shows how Wisp approaches the same concept. Two languages, both indented, separated by philosophy.

1. Whetting Your Appetite

Python sells itself on readability, rapid prototyping, and an enormous ecosystem. It's the default choice for scripts, web backends, data science, and teaching.

Wisp sells itself on a different proposition: the readability of Python's visual structure combined with the raw power of Scheme. Scheme is one of the most theoretically clean programming languages ever designed — tail-call optimization, first-class continuations, hygienic macros — but its parenthesized syntax scares newcomers away. Wisp removes that barrier.

Wisp runs on GNU Guile, a mature Scheme implementation that ships with many GNU/Linux distributions. It gives you access to Guile's full ecosystem: real OS threads (no GIL), arbitrary-precision arithmetic, a sophisticated module system, and the ability to reshape the language through macros.

The trade-off? Python has PyPI. Wisp has the Guile ecosystem and whatever you can call via FFI. This isn't a contest for library count — it's a contest for expressiveness per line.

2. The Interpreter / The REPL

Python:

>>> 2 + 2
4
>>> print("Hello, world!")
Hello, world!

Wisp:

> + 2 2
4
> display "Hello, world!"
Hello, world!

To use Wisp interactively, launch guile and switch languages with ,L wisp. Or run scripts directly: guile --language=wisp -s hello.w.

The syntax is prefix notation — the function comes first, then its arguments — but without the outer parentheses that traditional Scheme requires. The line display "Hello, world!" is equivalent to Scheme's (display "Hello, world!"). Indentation replaces the parens.

3. An Informal Introduction

Numbers

Python:

>>> 17 / 3       # 5.666...
>>> 17 // 3      # 5
>>> 17 % 3       # 2
>>> 5 ** 2       # 25

Wisp:

/ 17 3            ; 17/3 — an exact rational!
quotient 17 3     ; 5
remainder 17 3    ; 2
expt 5 2          ; 25

The first difference is striking. (/ 17 3) in Scheme returns 17/3 — an exact rational number, not a floating-point approximation. Scheme distinguishes exact and inexact numbers at the type level. If you want the float, you ask for it explicitly: (exact->inexact (/ 17 3)) gives 5.666....

Operators take multiple arguments naturally:

+ 1 2 3 4 5       ; 15
* 2 3 4            ; 24
< 1 2 3 4          ; #t (chained comparison)

And Scheme gives you arbitrary-precision integers for free. No int vs long distinction, no overflow:

expt 2 1000        ; a 302-digit number, exactly

Strings

Python:

word = "Python"
word[0]        # 'P'
word[0:2]      # 'Py'
len(word)      # 6
f"Hello, {word}!"

Wisp:

define word "Wisp"
string-ref word 0       ; #\W (a character)
substring word 0 2      ; "Wi"
string-length word       ; 4
format #f "Hello, ~a!" word  ; "Hello, Wisp!"

Scheme strings are mutable (unlike Python's), though idiomatic Scheme treats them as values. The format function uses ~a as a placeholder — #f means "return a string" and #t means "print directly":

format #t "Hello, ~a!\n" word   ; prints: Hello, Wisp!

Lists

Python:

squares = [1, 4, 9, 16, 25]
squares[0]              # 1
squares + [36, 49]      # [1, 4, 9, 16, 25, 36, 49]
squares.append(36)      # mutates!

Wisp:

define squares '(1 4 9 16 25)
list-ref squares 0              ; 1
append squares '(36 49)         ; (1 4 9 16 25 36 49)

Scheme lists are linked lists — immutable by convention, efficient at the head. append returns a new list without modifying the original. There's no .append mutation. If you want indexed access, Scheme has vectors:

define sq #(1 4 9 16 25)
vector-ref sq 0                 ; 1

The '(1 4 9 16 25) is a quoted list — the quote ' prevents Scheme from trying to call 1 as a function.

4. Control Flow

if / cond

Python:

if x < 0:
    print("Negative")
elif x == 0:
    print("Zero")
else:
    print("Positive")

Wisp:

cond
  : < x 0
    display "Negative"
  : = x 0
    display "Zero"
  else
    display "Positive"

In Wisp, cond handles multi-branch conditionals. Each branch starts with a : (which creates the nested parentheses Scheme expects). The else clause catches everything.

The simple if takes a condition, a then-branch, and an else-branch:

if : < x 0
  display "Negative"
  display "Non-negative"

And because if is an expression, not a statement, you can use it anywhere:

define label
  if : < x 0
    . "negative"
    . "non-negative"

The . (dot) prevents the string from being treated as a function call — it marks a bare value rather than an application.

for loops

Python:

for word in ["cat", "window", "defenestrate"]:
    print(word, len(word))

Wisp:

for-each
  lambda : word
    display word
    display " "
    display : string-length word
    newline
  '("cat" "window" "defenestrate")

Scheme doesn't have a built-in for loop — iteration is done through for-each (for side effects) or map (for transformation). The lambda defines an anonymous function applied to each element.

Or using a named let (Scheme's idiomatic loop):

let loop
  : words '("cat" "window" "defenestrate")
  when : not : null? words
    display : car words
    display " "
    display : string-length : car words
    newline
    loop : cdr words

This is recursion that looks like a loop. car gets the first element, cdr gets the rest, and loop calls itself with the remaining list. Guile guarantees tail-call optimization, so this uses constant stack space — unlike Python, which would hit a recursion limit.

range

Python:

list(range(0, 10, 2))  # [0, 2, 4, 6, 8]

Wisp:

use-modules : srfi srfi-1

iota 5 0 2             ; (0 2 4 6 8)

Guile's iota from SRFI-1 generates numeric sequences. The arguments are count, start, and step — slightly different from Python's start, stop, step.

while

Python:

a, b = 0, 1
while a < 10:
    print(a)
    a, b = b, a + b

Wisp:

let loop : : a 0
             b 1
  when : < a 10
    display a
    newline
    loop b {a + b}

There's no while keyword here — it's a named let that recurses. The {a + b} uses Wisp's curly-brace infix syntax (from SRFI-105), which lets you write math the familiar way. Without it, you'd write (+ a b).

Pattern Matching

Python 3.10+:

match command:
    case "quit":
        quit_game()
    case "help":
        show_help()
    case _:
        print("Unknown")

Wisp (using Guile's match):

use-modules : ice-9 match

match command
  : "quit"
    quit-game
  : "help"
    show-help
  : _
    display "Unknown"

Guile's match supports destructuring, guards, and nested patterns — similar to Python's structural pattern matching but available since long before Python 3.10.

Functions

Python:

def factorial(n):
    """Return the factorial of n."""
    if n == 0:
        return 1
    return n * factorial(n - 1)

Wisp:

define : factorial n
  "Return the factorial of n."
  if : = n 0
    . 1
    * n : factorial {n - 1}

Functions are defined with define. The : factorial n creates (factorial n) — a function named factorial taking one argument. The docstring goes right after the parameter list. The last expression is the return value.

The Wisp version also benefits from tail-call optimization if rewritten in tail-recursive style:

define : factorial n
  let loop : : i n
               acc 1
    if : = i 0
      . acc
      loop {i - 1} {acc * i}

This handles (factorial 100000) without breaking a sweat. Python would need sys.setrecursionlimit or an iterative rewrite.

Default and Keyword Arguments

Python:

def greet(name, greeting="Hello"):
    print(f"{greeting}, {name}!")

Wisp:

use-modules : ice-9 optargs

define* : greet name #:optional (greeting "Hello")
  format #t "~a, ~a!\n" greeting name

Guile uses define* (from ice-9 optargs) for optional and keyword arguments:

define* : connect host #:key (port 8080) (timeout 30)
  format #t "Connecting to ~a:~a (timeout ~a)\n" host port timeout

connect "example.com" #:port 443
; => Connecting to example.com:443 (timeout 30)

#:optional for positional defaults, #:key for keyword arguments. More explicit than Python's unified syntax, but equally capable.

Lambda Expressions

Python:

double = lambda x: x * 2
sorted(words, key=lambda w: len(w))

Wisp:

define double
  lambda : x
    * x 2

sort words
  lambda : a b
    < (string-length a) (string-length b)

In Wisp, lambda is a full block — no single-expression restriction like Python's lambda. The indented body can contain anything.

5. Data Structures

Lists (Linked Lists)

Scheme lists are cons-cell linked lists — the fundamental data structure of all Lisps:

define fruits '("apple" "banana" "cherry")
car fruits                    ; "apple" (first)
cdr fruits                    ; ("banana" "cherry") (rest)
cons "mango" fruits           ; ("mango" "apple" "banana" "cherry")
length fruits                 ; 3
list-ref fruits 1             ; "banana"

Vectors (Arrays)

For indexed access, Scheme has vectors:

define v #(10 20 30 40 50)
vector-ref v 2               ; 30
vector-length v               ; 5

Hash Tables (Dictionaries)

Python:

tel = {"jack": 4098, "sape": 4139}
tel["guido"] = 4127
del tel["sape"]

Wisp:

define tel : make-hash-table

hashq-set! tel 'jack 4098
hashq-set! tel 'sape 4139
hashq-set! tel 'guido 4127
hashq-remove! tel 'sape

hashq-ref tel 'jack          ; 4098

Hash tables in Guile use explicit functions rather than syntax. It's more verbose, but Scheme compensates with association lists for small mappings:

define tel
  ' : (jack . 4098)
      (sape . 4139)

assoc 'jack tel               ; (jack . 4098)

Association lists are simple, immutable, and idiomatic for small key-value collections.

Sets

Guile doesn't have a built-in set type, but SRFI-113 provides them, or you can use sorted lists with lset- operations from SRFI-1:

use-modules : srfi srfi-1

define a '(1 2 3 4)
define b '(3 4 5 6)

lset-difference equal? a b     ; (1 2)
lset-union equal? a b          ; (1 2 3 4 5 6)
lset-intersection equal? a b   ; (3 4)

List Comprehensions

Python:

[x**2 for x in range(10) if x % 2 == 0]

Wisp:

use-modules : srfi srfi-42

list-ec : : i (index 10)
  if : = 0 : remainder i 2
  expt i 2
; => (0 4 16 36 64)

SRFI-42 provides eager comprehensions. The : lines create the nested structure that Scheme expects.

Or idiomatically with map and filter:

map
  lambda : x
    expt x 2
  filter even? : iota 10

Looping Techniques

Python:

for i, v in enumerate(["tic", "tac", "toe"]):
    print(i, v)

Wisp:

let loop : : i 0
             words '("tic" "tac" "toe")
  when : not : null? words
    format #t "~a ~a\n" i : car words
    loop {i + 1} : cdr words

Or using SRFI-42:

do-ec : : i (index 3)
  format #t "~a ~a\n" i
    list-ref '("tic" "tac" "toe") i

6. Modules

Importing

Python:

import math
from os.path import join
import json as j

Wisp:

use-modules : ice-9 regex

use-modules
  : srfi srfi-1
    #:select : iota fold

use-modules
  : ice-9 popen
    #:prefix popen-

Guile's module system uses use-modules. The #:select option imports specific symbols, and #:prefix namespaces them — similar to Python's from X import Y and import X as Y.

Defining Modules

Python:

# mymodule.py
def greet(name):
    return f"Hello, {name}!"

Wisp:

define-module : myproject mymodule
  . #:export : greet

define : greet name
  format #f "Hello, ~a!" name

The #:export clause explicitly lists what the module makes public — like Python's __all__ but mandatory and standard.

The Script Entry Point

Python:

if __name__ == "__main__":
    main()

Wisp scripts use a shell header that invokes Guile with the right flags:

#!/usr/bin/env bash
exec guile -L . -x .w --language=wisp -e main -s "$0" "$@"
!#

define : main args
  display "Hello from the command line!\n"

The -e main flag tells Guile to call the main function with command-line arguments.

7. Input and Output

String Formatting

Python:

name = "World"
f"Hello, {name}!"
"{:.2f}".format(3.14159)

Wisp:

define name "World"
format #f "Hello, ~a!" name           ; "Hello, World!"
format #f "~,2f" 3.14159              ; "3.14"

Guile's format uses tilde-based directives: ~a for display, ~s for write (with quotes), ~d for integer, ~f for float, ~% for newline. It's closer to Common Lisp's format than to Python's f-strings.

File I/O

Python:

with open("data.txt", encoding="utf-8") as f:
    content = f.read()

with open("output.txt", "w") as f:
    f.write("Hello\n")

Wisp:

use-modules : ice-9 textual-ports

; Read entire file
define content
  call-with-input-file "data.txt" get-string-all

; Write to file
call-with-output-file "output.txt"
  lambda : port
    display "Hello\n" port

Or using with-input-from-file for a more Python-with-like pattern:

with-input-from-file "data.txt"
  lambda ()
    let loop : : line (read-line)
      when : not : eof-object? line
        display line
        newline
        loop : read-line

Scheme's port system is lower-level than Python's file objects but equally capable. Every I/O function accepts an optional port argument — display "hello" port writes to a specific destination.

JSON

Guile has a JSON module:

use-modules : json

define data
  json-string->scm "{\"name\": \"Ada\"}"
; => (("name" . "Ada"))

scm->json-string '((name . "Ada"))
; => "{\"name\":\"Ada\"}"

8. Errors and Exceptions

Try / Catch

Python:

try:
    x = int(input("Number: "))
except ValueError as e:
    print(f"Invalid: {e}")
finally:
    print("Done")

Wisp:

catch #t
  lambda ()
    ; try body
    define x
      string->number : read-line
    when : not x
      error "Invalid number"
  lambda : key . args
    ; catch handler
    format #t "Error: ~a ~a\n" key args

display "Done\n"

Guile uses catch and throw (or the newer with-exception-handler / raise-exception). The handler is a lambda that receives the error key and arguments.

The more modern style:

use-modules : ice-9 exceptions

with-exception-handler
  lambda : exn
    format #t "Error: ~a\n" : exception-message exn
  lambda ()
    string->number "not-a-number"
  . #:unwind? #t

Raising Exceptions

Python:

raise ValueError("something went wrong")

Wisp:

error "something went wrong"

Or with structured keys:

throw 'value-error "something went wrong"

Custom Error Types

Python creates exception classes. Guile uses symbol keys:

throw 'insufficient-funds "Need more money"
  . balance amount

Or with the newer condition system from R7RS:

use-modules : ice-9 exceptions

define-exception-type &insufficient-funds &error
  make-insufficient-funds
  insufficient-funds?
  : balance insufficient-funds-balance
    amount insufficient-funds-amount

More ceremony than Python's class hierarchy, but also more structured — each field is explicitly named and accessible.

9. Classes and Objects

This is where the languages diverge most sharply.

Python's Object-Oriented Approach

class Dog:
    kind = "canine"

    def __init__(self, name):
        self.name = name

    def speak(self):
        return f"{self.name} says woof!"

rex = Dog("Rex")
rex.speak()

Wisp's Approach: GOOPS

Guile has GOOPS (GNU Object-Oriented Programming System), a powerful CLOS-style object system:

use-modules : oop goops

define-class <dog> ()
  kind
    . #:init-value "canine"
    . #:getter dog-kind
  name
    . #:init-keyword #:name
    . #:getter dog-name

define-method : speak (dog <dog>)
  format #f "~a says woof!" : dog-name dog

define rex : make <dog> #:name "Rex"
speak rex                ; "Rex says woof!"

GOOPS supports multiple inheritance, multiple dispatch (methods dispatch on all argument types, not just the first), metaclasses, and method combinations. It's more powerful than Python's object system, but also more verbose.

Inheritance

Python:

class Puppy(Dog):
    def speak(self):
        return f"{self.name} says yip!"

Wisp:

define-class <puppy> (<dog>)

define-method : speak (dog <puppy>)
  format #f "~a says yip!" : dog-name dog

The Alternative: Just Use Data

Many Scheme programmers avoid GOOPS entirely and use plain data structures with functions — similar to Clojure's philosophy:

define : make-dog name
  ' : (kind . "canine")
      (name . ,name)

define : dog-speak dog
  format #f "~a says woof!" : assoc-ref dog 'name

define rex : make-dog "Rex"
dog-speak rex

This is simpler and often sufficient.

10. The Standard Library

Python's Batteries vs. Guile's Foundations

Python ships with modules for everything. Guile ships with fewer but more fundamental tools:

Regex

use-modules : ice-9 regex

define m : string-match "[0-9]+" "hello 42 world"
match:substring m              ; "42"

Threading — No GIL

This is Guile's secret weapon. Where Python has the GIL limiting true parallelism, Guile has real POSIX threads:

use-modules : ice-9 threads

call-with-new-thread
  lambda ()
    display "Hello from another thread!\n"

par-map
  lambda : x
    * x x
  iota 10
; => (0 1 4 9 16 25 36 49 64 81)

par-map is parallel map — it distributes work across OS threads. No multiprocessing workaround, no async/await complexity. Just threads.

Web Server in 10 Lines

use-modules : web server

run-server
  lambda : request request-body
    values
      ' : (content-type . (text/plain))
      . "Hello, World!"

Guile ships with a built-in HTTP server and client. No framework needed for basic use.

11. Advanced Standard Library

Output Formatting

use-modules : ice-9 pretty-print

pretty-print
  ' : (name . "Ada")
      (languages . ("Python" "Wisp" "Scheme"))

Multi-threading (Continued)

use-modules : ice-9 threads

define mutex : make-mutex
define counter 0

define : increment
  lock-mutex mutex
  set! counter {counter + 1}
  unlock-mutex mutex

; Spawn 100 threads
let loop : : i 100
  when : > i 0
    call-with-new-thread increment
    loop {i - 1}

Exact Decimal Arithmetic

Python needs the decimal module. Scheme has exact rationals built-in:

+ 1/10 2/10               ; 3/10 (exact!)
= (+ 1/10 2/10) 3/10      ; #t
* 7/10 105/100             ; 147/200 (exact!)

No Decimal class, no imports. Exact fractions are a primitive type.

12. Virtual Environments and Packages

Python has venv and pip. Guile has a different model.

Installation:

# Guile is often pre-installed on GNU/Linux
sudo apt install guile-3.0

# Wisp
sudo apt install guile-wisp
# or from source

Dependencies:
Guile uses a load-path model rather than per-project virtual environments:

export GUILE_LOAD_PATH="/path/to/library:$GUILE_LOAD_PATH"
guile --language=wisp -s myapp.w

For project-specific dependencies, the community uses GNU Guix (a functional package manager) or manual load-path management. It's less ergonomic than pip install, but aligns with the GNU ecosystem's philosophy.

There's no PyPI equivalent. Libraries are distributed through GNU Guix, OS packages, or directly as source. This is Wisp's weakest point compared to Python.

13. Floating-Point Arithmetic

Python:

>>> 0.1 + 0.2 == 0.3
False

Wisp:

= {0.1 + 0.2} 0.3             ; #f (same problem with floats!)

Same IEEE 754, same surprises. But Scheme has an escape hatch Python doesn't — exact rationals are a first-class type, not a library:

= {1/10 + 2/10} 3/10          ; #t (exact!)
exact->inexact 1/3              ; 0.3333333333333333
inexact->exact 0.1              ; 3602879701896397/36028797018963968

That last line reveals the actual value stored for 0.1 — a ratio of two large integers. Scheme lets you move freely between exact and inexact worlds.

14. Interactive Editing

Python's REPL supports readline history and tab completion.

Guile's REPL offers similar features plus meta-commands:

,help              ; show all REPL commands
,describe display  ; show documentation
,time (fib 30)     ; benchmark an expression
,profile (fib 30)  ; profile an expression
,L wisp            ; switch to Wisp syntax
,L scheme          ; switch back to Scheme

You can switch between Wisp and Scheme syntax in the same session. This is useful for learning — write in Wisp, see what Scheme it corresponds to.

15. The Secret Weapon: Macros

Like all Lisps, Wisp supports macros — compile-time code transformations. But because Wisp code looks like indented pseudocode, Wisp macros are unusually readable for Lisp macros.

Example: A Python-style for-in loop

define-syntax-rule : for-in var lst body ...
  for-each
    lambda : var
      body ...
    lst

; Now use it:
for-in word '("cat" "window" "defenestrate")
  display word
  newline

You just added a for-in construct to the language. It's not a function that takes a callback — it's real syntax that the compiler expands before execution.

Example: Unless

define-syntax-rule : unless condition body ...
  when : not condition
    body ...

unless : = 1 2
  display "Math still works\n"

Example: Time-it

define-syntax-rule : time-it body ...
  let : : start (current-time)
    body ...
    format #t "Elapsed: ~a seconds\n"
      - (current-time) start

time-it
  let loop : : i 1000000
    when : > i 0
      loop {i - 1}

Why This Matters

In Python, if you need a new control structure, you write a function with callbacks or a context manager — a workaround. In Wisp, you write a macro that generates the exact code you want, with zero runtime overhead. The result is indistinguishable from a built-in language feature.

This is the deep reason Lisp syntax exists: when code and data have the same structure, programs that write programs become natural.

The Big Picture

Closing Thoughts

Wisp is not trying to replace Python. It's trying to answer a question: can Lisp be approachable?

The answer is yes. Wisp code reads like indented pseudocode. You can show it to someone who has never programmed and they'll follow the structure. But underneath that gentle surface lies one of the most powerful programming models ever devised — one where functions are data, data is code, and the language reshapes itself to fit your problem.

The ecosystem gap is real. You won't find a Wisp equivalent of NumPy or Django. But for systems programming, scripting, language design, concurrent servers, or any domain where you want to think differently about code — Wisp offers something Python cannot: a Lisp you can read at a glance.

display "Welcome to Wisp.\n"
display "Where indentation meets imagination.\n"

Get started: install Guile, install Wisp, and type ,L wisp in the Guile REPL. The parentheses are gone. The power remains.

This article was written as a companion to the official Python tutorial. Every section maps to a chapter in that tutorial, showing the same concepts through Wisp's indentation-sensitive Lisp lens. For more on Wisp, visit draketo.de/software/wisp.

Permalink

Python to Clojure: A Gentle Guide for Pythonistas

Python to Clojure: A Gentle Guide for Pythonistas

Python is the world's most popular general-purpose language. Clojure is a quiet powerhouse — a modern Lisp on the JVM that has earned fierce loyalty among developers who discover it. Both languages prize simplicity, but they mean very different things by the word.

This article walks through every major topic in the official Python tutorial and shows how Clojure approaches the same idea. Whether you're a Pythonista curious about functional programming or a polyglot looking for a side-by-side reference, this guide is for you.

1. Whetting Your Appetite

Python sells itself on readability, rapid prototyping, and "batteries included." It eliminates the compile-link cycle, encourages experimentation in the REPL, and runs everywhere.

Clojure shares the same REPL-driven culture, but takes a more opinionated stance: data is the universal interface. Where Python says "everything is an object," Clojure says "everything is data." Clojure runs on the JVM (and in the browser via ClojureScript), giving it access to the entire Java ecosystem — a different kind of "batteries included."

Both languages let you sit down and be productive in minutes. The difference is in what they optimize for: Python optimizes for readability of imperative code; Clojure optimizes for simplicity of data transformation.

2. The Interpreter / The REPL

Python has an interactive interpreter invoked with python:

>>> 2 + 2
4
>>> print("Hello, world!")
Hello, world!

Clojure has the REPL (Read-Eval-Print Loop), which serves the same purpose but is even more central to the workflow. Most Clojure developers keep a REPL connected to their editor at all times:

user=> (+ 2 2)
4
user=> (println "Hello, world!")
Hello, world!

The parentheses aren't noise — they're the syntax. Every expression is a list where the first element is the function. This uniformity is what makes Lisp macros possible, and it takes about a day to stop noticing the parens.

3. An Informal Introduction

Numbers

Python:

>>> 17 / 3      # 5.666...
>>> 17 // 3     # 5 (floor division)
>>> 17 % 3      # 2
>>> 5 ** 2      # 25

Clojure:

(/ 17 3)       ;=> 17/3  (a rational! no precision lost)
(quot 17 3)    ;=> 5
(rem 17 3)     ;=> 2
(Math/pow 5 2) ;=> 25.0

Right away, a philosophical difference appears. Clojure gives you a ratio (17/3) by default rather than silently losing precision. Clojure also has arbitrary-precision integers out of the box — no special int vs long distinction to worry about.

Strings

Python:

word = "Python"
word[0]        # 'P'
word[0:2]      # 'Py'
len(word)      # 6
f"Hello, {word}!"

Clojure:

(def word "Clojure")
(get word 0)          ;=> \C  (a character)
(subs word 0 2)       ;=> "Cl"
(count word)          ;=> 7
(str "Hello, " word "!")

Both treat strings as immutable. In Python, strings are sequences of characters; in Clojure, strings are Java strings, but you can also treat them as sequences of characters when needed via seq.

Lists

Python:

squares = [1, 4, 9, 16, 25]
squares[0]              # 1
squares + [36, 49]      # [1, 4, 9, 16, 25, 36, 49]
squares.append(36)      # mutates!

Clojure:

(def squares [1 4 9 16 25])
(get squares 0)              ;=> 1
(conj squares 36)            ;=> [1 4 9 16 25 36] (new vector!)
(into squares [36 49])       ;=> [1 4 9 16 25 36 49]

This is the first big fork in the road. Python lists are mutable; Clojure vectors are persistent and immutable. conj doesn't change squares — it returns a new vector that efficiently shares structure with the old one. No defensive copying, no "did someone mutate my list?" bugs.

4. Control Flow

if / cond

Python:

if x < 0:
    print("Negative")
elif x == 0:
    print("Zero")
else:
    print("Positive")

Clojure:

(cond
  (neg? x) (println "Negative")
  (zero? x) (println "Zero")
  :else (println "Positive"))

In Clojure, if is an expression that returns a value (no need for a ternary operator). cond handles the multi-branch case.

for loops

Python:

for word in ["cat", "window", "defenestrate"]:
    print(word, len(word))

Clojure:

(doseq [word ["cat" "window" "defenestrate"]]
  (println word (count word)))

But here's the thing — doseq is for side effects. When you want to transform data (which is most of the time), you use map, filter, or for (which is a list comprehension, not a loop):

(for [word ["cat" "window" "defenestrate"]]
  [word (count word)])
;=> (["cat" 3] ["window" 6] ["defenestrate" 13])

range

Python:

list(range(0, 10, 2))  # [0, 2, 4, 6, 8]

Clojure:

(range 0 10 2)  ;=> (0 2 4 6 8)

Clojure's range is lazy — it doesn't allocate memory for all elements upfront.

Pattern Matching

Python 3.10+:

match status:
    case 400:
        return "Bad request"
    case 401 | 403:
        return "Not allowed"
    case _:
        return "Something's wrong"

Clojure (with core.match):

(match status
  400 "Bad request"
  (:or 401 403) "Not allowed"
  :else "Something's wrong")

Clojure's pattern matching via core.match is a library, not built-in syntax — which is itself a philosophical statement. In Lisp, you can add any syntax you want via macros.

Functions

Python:

def fib(n):
    """Return Fibonacci series up to n."""
    a, b = 0, 1
    result = []
    while a < n:
        result.append(a)
        a, b = b, a + b
    return result

Clojure:

(defn fib [n]
  "Return Fibonacci series up to n."
  (->> [0 1]
       (iterate (fn [[a b]] [b (+ a b)]))
       (map first)
       (take-while #(< % n))))

The Clojure version reads as a pipeline: start with [0 1], repeatedly produce the next pair, extract the first element, and keep going while it's less than n. No mutation, no accumulator variable, no while loop. Just data flowing through transformations.

Default Arguments, *args, **kwargs

Python:

def greet(name, greeting="Hello"):
    print(f"{greeting}, {name}!")

def log(*args, **kwargs):
    print(args, kwargs)

Clojure:

(defn greet
  ([name] (greet name "Hello"))
  ([name greeting] (println (str greeting ", " name "!"))))

(defn log [& args]
  (println args))

Clojure uses multi-arity functions for defaults and & args for variadics. For keyword arguments, the idiomatic approach is to pass a map:

(defn create-user [{:keys [name email role] :or {role "member"}}]
  {:name name :email email :role role})

(create-user {:name "Ada" :email "ada@example.com"})

Lambda Expressions

Python:

double = lambda x: x * 2
sorted(pairs, key=lambda p: p[1])

Clojure:

(def double #(* % 2))        ; reader macro shorthand
(sort-by second pairs)       ; or: (sort-by #(nth % 1) pairs)

In Clojure, anonymous functions are so common they have a shorthand: #(...) with % for the argument. Named functions and anonymous functions are treated identically — there's no second-class lambda.

5. Data Structures

Lists, Tuples, and Vectors

Python needs both lists (mutable) and tuples (immutable). Clojure needs only vectors (immutable, indexed) and lists (immutable, sequential). The mutability question simply doesn't arise.

Dictionaries / Maps

Python:

tel = {"jack": 4098, "sape": 4139}
tel["guido"] = 4127
del tel["sape"]
{x: x**2 for x in range(5)}

Clojure:

(def tel {"jack" 4098 "sape" 4139})
(assoc tel "guido" 4127)       ;=> new map with guido added
(dissoc tel "sape")            ;=> new map without sape
(into {} (map (fn [x] [x (* x x)]) (range 5)))

Again: assoc and dissoc return new maps. The original is untouched. Clojure maps can also use keywords as keys (:jack instead of "jack"), which double as accessor functions — a small but delightful ergonomic touch:

(def person {:name "Ada" :age 36})
(:name person)  ;=> "Ada"

Sets

Python:

a = {1, 2, 3, 4}
b = {3, 4, 5, 6}
a - b            # {1, 2}
a | b            # {1, 2, 3, 4, 5, 6}
a & b            # {3, 4}

Clojure:

(require '[clojure.set :as set])
(def a #{1 2 3 4})
(def b #{3 4 5 6})
(set/difference a b)     ;=> #{1 2}
(set/union a b)          ;=> #{1 2 3 4 5 6}
(set/intersection a b)   ;=> #{3 4}

Virtually identical semantics, different syntax. Clojure sets are also immutable and can be used as functions: (a 3) returns 3 (truthy), (a 7) returns nil (falsy).

List Comprehensions

Python:

[x**2 for x in range(10) if x % 2 == 0]

Clojure:

(for [x (range 10) :when (even? x)]
  (* x x))

Clojure's for is a list comprehension, not a loop. It supports :when for filtering, :let for local bindings, and multiple binding forms for nested iteration — all lazily evaluated.

Looping Techniques

Python:

for i, v in enumerate(["tic", "tac", "toe"]):
    print(i, v)

for q, a in zip(questions, answers):
    print(q, a)

Clojure:

(doseq [[i v] (map-indexed vector ["tic" "tac" "toe"])]
  (println i v))

(doseq [[q a] (map vector questions answers)]
  (println q a))

Or more idiomatically, just use map to transform and print later:

(map-indexed (fn [i v] [i v]) ["tic" "tac" "toe"])
(map vector questions answers)

6. Modules and Namespaces

Python:

import math
from os.path import join
import numpy as np

Clojure:

(require '[clojure.string :as str])
(require '[clojure.java.io :as io])
(import '[java.util Date])

Python organizes code into modules (files) and packages (directories with __init__.py). Clojure organizes code into namespaces — each file declares a namespace with ns, and dependencies are explicit:

(ns myapp.core
  (:require [clojure.string :as str]
            [cheshire.core :as json])
  (:import [java.time LocalDate]))

There's no from X import * equivalent in Clojure, and that's by design. Explicit is better than implicit — a principle Pythonistas already appreciate.

The if __name__ == "__main__" Pattern

Python:

if __name__ == "__main__":
    main()

Clojure doesn't need this pattern because the REPL workflow means you rarely "run a file." When you do need an entry point, you declare a -main function:

(defn -main [& args]
  (println "Hello from the command line!"))

7. Input and Output

String Formatting

Python:

name = "World"
f"Hello, {name}!"
"{:.2f}".format(3.14159)

Clojure:

(str "Hello, " name "!")            ; simple concatenation
(format "Hello, %s!" name)          ; printf-style
(format "%.2f" 3.14159)             ; "3.14"

Clojure uses Java's String.format under the hood. There's no f-string equivalent, but str for concatenation and format for templates cover the same ground.

File I/O

Python:

with open("data.txt", encoding="utf-8") as f:
    content = f.read()

with open("output.txt", "w", encoding="utf-8") as f:
    f.write("Hello\n")

Clojure:

(slurp "data.txt")                         ; read entire file
(spit "output.txt" "Hello\n")             ; write entire file

;; For line-by-line processing:
(with-open [rdr (clojure.java.io/reader "data.txt")]
  (doseq [line (line-seq rdr)]
    (println line)))

slurp and spit — arguably the best-named I/O functions in any language. For structured work, with-open mirrors Python's with statement.

JSON

Python:

import json
data = json.loads('{"name": "Ada"}')
json.dumps(data)

Clojure (with cheshire or clojure.data.json):

(require '[cheshire.core :as json])
(json/parse-string "{\"name\": \"Ada\"}" true)  ;=> {:name "Ada"}
(json/generate-string {:name "Ada"})             ;=> "{\"name\":\"Ada\"}"

The true argument keywordizes the keys — JSON maps become idiomatic Clojure maps instantly.

8. Errors and Exceptions

Try / Catch

Python:

try:
    x = int(input("Number: "))
except ValueError as e:
    print(f"Invalid: {e}")
finally:
    print("Done")

Clojure:

(try
  (Integer/parseInt (read-line))
  (catch NumberFormatException e
    (println "Invalid:" (.getMessage e)))
  (finally
    (println "Done")))

Since Clojure runs on the JVM, it catches Java exceptions. The structure is almost identical to Python's try/except/finally.

Raising Exceptions

Python:

raise ValueError("something went wrong")

Clojure:

(throw (ex-info "something went wrong" {:type :validation}))

ex-info is idiomatic Clojure — it creates an exception that carries a data map. Instead of defining custom exception classes, you attach structured data to a generic exception. This is the "data over classes" philosophy in action:

(try
  (throw (ex-info "bad input" {:field :email :value "not-an-email"}))
  (catch clojure.lang.ExceptionInfo e
    (println (ex-data e))))
;=> {:field :email, :value "not-an-email"}

Custom Exceptions

Python:

class InsufficientFunds(Exception):
    def __init__(self, balance, amount):
        self.balance = balance
        self.amount = amount

Clojure rarely defines custom exception classes. Instead:

(throw (ex-info "Insufficient funds"
         {:type :insufficient-funds
          :balance 100
          :amount 150}))

One generic mechanism, infinite data shapes. No class hierarchy to maintain.

9. Classes and Objects

This is where Python and Clojure diverge most dramatically.

Python's Object-Oriented Approach

class Dog:
    kind = "canine"             # class variable

    def __init__(self, name):
        self.name = name        # instance variable

    def speak(self):
        return f"{self.name} says woof!"

rex = Dog("Rex")
rex.speak()  # "Rex says woof!"

Clojure's Data-Oriented Approach

(def rex {:kind "canine" :name "Rex"})

(defn speak [dog]
  (str (:name dog) " says woof!"))

(speak rex)  ;=> "Rex says woof!"

No class. No self. No constructor. Just a map and a function. The dog is its data.

Inheritance vs. Composition

Python:

class Animal:
    def __init__(self, name):
        self.name = name

class Dog(Animal):
    def speak(self):
        return "Woof!"

class Cat(Animal):
    def speak(self):
        return "Meow!"

Clojure uses multimethods or protocols for polymorphism:

;; Multimethod approach — dispatch on data
(defmulti speak :type)
(defmethod speak :dog [animal] "Woof!")
(defmethod speak :cat [animal] "Meow!")

(speak {:type :dog :name "Rex"})   ;=> "Woof!"
(speak {:type :cat :name "Whiskers"}) ;=> "Meow!"

;; Protocol approach — like interfaces
(defprotocol Speakable
  (speak [this]))

(defrecord Dog [name]
  Speakable
  (speak [this] (str name " says Woof!")))

(speak (->Dog "Rex"))  ;=> "Rex says Woof!"

Multimethods dispatch on any function of the arguments (not just type), making them more flexible than traditional OOP dispatch.

Iterators and Generators

Python:

class Reverse:
    def __init__(self, data):
        self.data = data
        self.index = len(data)

    def __iter__(self):
        return self

    def __next__(self):
        if self.index == 0:
            raise StopIteration
        self.index -= 1
        return self.data[self.index]

def reverse(data):
    for index in range(len(data) - 1, -1, -1):
        yield data[index]

Clojure:

(reverse "spam")  ;=> (\m \a \p \s)
(rseq [1 2 3 4])  ;=> (4 3 2 1)

Clojure's sequences are lazy iterators. There's no iterator protocol to implement — every collection already participates in the sequence abstraction. Need a custom lazy sequence? Use lazy-seq:

(defn countdown [n]
  (when (pos? n)
    (lazy-seq (cons n (countdown (dec n))))))

(countdown 5)  ;=> (5 4 3 2 1)

Generator Expressions

Python:

sum(x*x for x in range(10))

Clojure:

(reduce + (map #(* % %) (range 10)))
;; or with the threading macro:
(->> (range 10) (map #(* % %)) (reduce +))

The threading macro ->> reads like a Unix pipeline and is one of Clojure's most beloved features.

10. The Standard Library

Python's "Batteries Included"

Python ships with modules for OS interaction, file I/O, regex, math, dates, HTTP, email, testing, logging, threading, and much more — all in the standard library.

Clojure's Approach

Clojure's standard library is smaller but remarkably powerful for data manipulation. For everything else, you tap into:

1. The entire Java ecosystem — need HTTP? Use java.net.http. Need dates? Use java.time. Need crypto? Use javax.crypto.
2. Clojure community libraries — managed via deps.edn or Leiningen.

Concurrency — Where Clojure Truly Shines

Python's threading story involves the GIL and careful locking. Clojure was designed for concurrency from day one:

• Atoms — uncoordinated, synchronous state updates
• Refs — coordinated, transactional state (Software Transactional Memory)
• Agents — asynchronous state updates
• core.async — CSP-style channels (like Go's goroutines)

(def counter (atom 0))
(swap! counter inc)      ;=> 1  (thread-safe, no locks)
@counter                 ;=> 1

;; Process 1000 items in parallel:
(pmap expensive-fn (range 1000))

11. Advanced Standard Library

Output Formatting

Python: pprint, textwrap, locale
Clojure: clojure.pprint/pprint (built-in), Java's Locale

(require '[clojure.pprint :refer [pprint]])
(pprint {:a 1 :b {:c [1 2 3] :d "hello"}})

Templating

Python: string.Template
Clojure: clojure.core/format, or libraries like Selmer for HTML templating.

Multi-threading

Python:

import threading
t = threading.Thread(target=worker)
t.start()

Clojure:

(future (worker))  ; runs in a thread pool, returns a deref-able future
@(future (+ 1 2))  ;=> 3

Logging

Python: logging.warning("Watch out!")
Clojure:

(require '[clojure.tools.logging :as log])
(log/warn "Watch out!")

Decimal Arithmetic

Python: decimal.Decimal("0.1") + decimal.Decimal("0.2")
Clojure:

(+ 0.1M 0.2M)  ;=> 0.3M  (BigDecimal literal with M suffix)

Clojure has literal syntax for BigDecimals (M suffix) and BigIntegers (N suffix) — no imports needed.

12. Virtual Environments and Packages

Python uses venv and pip:

python -m venv myenv
source myenv/bin/activate
pip install requests
pip freeze > requirements.txt

Clojure uses deps.edn (official) or project.clj (Leiningen):

;; deps.edn
{:deps {org.clojure/clojure {:mvn/version "1.12.0"}
        cheshire/cheshire {:mvn/version "5.13.0"}}}

clj -M -m myapp.core  # run with dependencies resolved automatically

There's no virtual environment concept because dependencies are resolved per-project from the deps.edn file. Maven coordinates ensure reproducibility. No activate/deactivate dance.

13. Floating-Point Arithmetic

Both languages sit on top of IEEE 754 doubles:

>>> 0.1 + 0.2 == 0.3
False

(= (+ 0.1 0.2) 0.3)  ;=> false

Same hardware, same surprise. But Clojure offers escape hatches as first-class citizens:

;; Ratios — exact arithmetic, no precision loss
(+ 1/10 2/10)      ;=> 3/10
(= (+ 1/10 2/10) 3/10)  ;=> true

;; BigDecimals
(+ 0.1M 0.2M)      ;=> 0.3M

Clojure's ratio type means you can do exact fractional arithmetic without importing anything. This alone has saved countless financial applications from rounding bugs.

14. Interactive Editing

Python supports readline-based history and tab completion in its interactive interpreter.

Clojure developers typically use nREPL connected to their editor (Emacs + CIDER, VS Code + Calva, IntelliJ + Cursive). The experience goes far beyond line editing — you can evaluate any expression in your source file, inspect results inline, and navigate documentation without leaving your editor.

The Big Picture

Closing Thoughts

Python is a phenomenal language. Its readability, ecosystem, and community have earned it the top spot for good reason. But if you've ever felt the friction of debugging shared mutable state, wrestling with class hierarchies, or wishing your data transformations could be simpler — Clojure might be the language that makes you see programming differently.

You don't have to abandon Python. Many developers use both: Python for its unmatched ML/AI ecosystem and scripting ergonomics, Clojure for systems where correctness, concurrency, and data transformation matter most.

The best way to start is to fire up a REPL. Try Clojure's official getting started guide, or experiment in the browser at repl.it. The parentheses will feel strange for an hour. Then they'll feel like home.

This article was written as a companion to the official Python tutorial. Every section maps to a chapter in that tutorial, translated through the lens of Clojure's philosophy: simple, data-driven, and functional.

Permalink

LISP Prolog and Evolution

I just saw David Nolen give a talk at a LispNYC Meetup called:

Prolog

• Unification
• Backtracking

LISP

• Garbage collection
• Functional programming
• Homoiconicity code is just a form of data
• REPL
• Minimal syntax, you program in abstract syntax trees

Causes for LISP Losing Ground

LISP vs. Haskell

Similarity between Haskell and LISP

A few Haskell and LISP differences

• LISP is homoiconic, Haskell is not
• LISP has a very advanced object system CLOS
• Haskell uses monadic computations

Evolution and the Selfish Gene

The Sudden Rise of Clojure

Typed LISP

• Typed Racket
• Typed Clojure
• Shen

Permalink

Why am I learning Clojure Script?

Notes

• Learn Clojure Script
• Step 1, Hello World! with Clojure Script
• Step 2, Adding aliases
• Stock Trends

Permalink

Using Generative AI tooling with Clojure

Clojure is easy to read for humans and AIs

Code written in Clojure is expressive and concise, but is still easy to reason about. In his “History of Clojure” paper Rich Hickey, the original author of Clojure states his motivations for building a new programming language:

“Most developers are primarily engaged in making systems that acquire, extract, transform, maintain, analyze, transmit and render information—facts about the world … As programs grew large, they required increasingly Herculean efforts to change while maintaining all of the presumptions around state and relationships, never mind dealing with race conditions as concurrency was increasingly in play. And we faced encroaching, and eventually crippling, coupling, and huge codebases, due directly to specificity (best-practice encapsulation, abstraction, and parameterization notwithstanding). C++ builds of over an hour were common.”

As a result, Clojure programs are very much focused on dealing with data and do it safely in concurrent programs. We have by default immutable data structures, easy to use literal representations of the most common collection types (lists, vectors, sets and maps) and a very regular syntax. A typical Clojure program has way less ceremony and boilerplate, not to mention weird quirks to deal with compared to many more programming languages such as Java, C#, Typescript or Python.

This means that large language models have less to deal with when reading or writing Clojure code. We have some evidence in Martin Alderson’s article that Clojure is token efficient compared to most other popular programming languages.

When we author code with generative AI tools, a developer reviewing it has less code to read in a format that easy to reason about.

Clojure MCP boosts Agentic development workflows

The REPL driven workflow speeds up the feedback cycle in normal development modes. Read Eval Print Loop is a concept in many programming languages in the LISP family such as Common Lisp, Scheme and Clojure. It allows the developer to tap into and evaluate code in a running instance of the application they are developing. With good editor integration, this allows smooth and frictionless testing of the code under development in an interactive workflow.

With the addition of MCP (Model Context Protocol) agents have gained access to a lot of tooling. In May 2025 Bruce Hauman announced his Clojure MCP that provides agents access to the REPL. Now AI agents such as Claude Code, Copilot CLI and others can reach inside the application as it is being developed, try code changes live, look at the internal state of the application and benefit from all of the interactivity that human developers have when working with the REPL.

It also provides efficient structural editing capabilities to the agents, making them less error-prone when editing Clojure source code. Because Clojure code is written as Clojure data structures, programmatic edits to the source code are a breeze.

We can even hot load dependencies to a running application without losing the application state! This is a SKILL.md file I have added to my project to guide agents in harnessing this power:

---
name: adding-clojure-dependencies
description: Adds clojure dependencies to the project. Use this when asked to add a dependency to the project
---

To add dependencies to deps.edn do the following:

1. Find the dependency in maven central repo or clojars
2. Identify the latest release version (no RCs unless specified)
3. Add dependency to either main list (for clojure dependencies),
   :test alias (for test dependencies) or :dev alias (for development dependencies). Use the REPL and
   `rewrite-edn` to edit the file
4. Reload dependencies in the REPL using `(clojure.repl.deps/sync-deps)`

In my experience, with the Clojure MCP coding agents have a far easier time troubleshooting and debugging compared to having them just analyze source code, logs and stacktraces.

As a developer, we can also connect to the same REPL as the coding agent, making it easy to step in and aid the agent when it gets stuck. In my workflows, I might look at the code the agent produced and test it in the REPL as well, make changes as required and instruct the agent to read what I did. This gives another collaborative dimension to standard prompting techniques that are normally associated with generative AI development.

Getting AI to speak Clojure

Generating and analyzing code with AI tooling is just one way to apply AI in software development. As developers, we should understand the potential for embedding AI functionality at the application level too. LLMs seem to be good at understanding my intentions, even if they don’t necessarily produce the right results. One possibility is to take input provided by a human and enrich it with data so that further processing becomes easier. For the sake of experiment, let’s look at a traditional flow of making a support request.

The user goes to the portal and their first task is to identify the correct topic under which this support request belongs to. They usually have to classify the severity of the issue as well. Then they describe what problem they have and add their contact information. With this flow, there’s a large chance that the user misclassified their support request, causing delays in getting the work in front of the right person, making the user experience poor and causing frustrations in the people handling the support requests. From the user’s point of view they don’t care about which department picks up the request, so this system is pushing the support organization’s concerns to the end user.

What if we could avoid all that and have the request routed automatically to the right backlog? Enter OpenAI’s Requests API which can handle text, image and various file inputs to generate text or JSON outputs. The json_schema response format is interesting in particular because we can express the desired result format in a manner that we can then use to process the response programmatically down the line.

In the Clojure world, we often use Malli to define our data models. We can use malli.json-schema to transform our malli schemas into JSON schema that the endpoint understands, and then use malli.transform to translate the response from JSON back to Clojure data.

A similar idea has been shown previously by Tommi Reiman, author of Malli.

Note that the choice of model can have a big effect on your output!

(require '[malli.json-schema :as mjs])
(require '[malli.core :as m])
(require '[malli.transform :as mt])
(require '[cheshire.core :as json])
(require '[org.httpkit.client :as http])

(defn structured-output
  [api-endpoint-url api-key malli-schema input]
  (let [;; Convert Malli schema to JSON Schema
        json-schema (mjs/transform malli-schema)

        ;; Build request body
        body {:model "gpt-4o" ;; consult your service provider for available models
              :input input
              :text {:format {:type "json_schema"
                              :name "response"
                              :strict true
                              :schema json-schema}}}

        ;; Make HTTP request
        response @(http/post
                   (str api-endpoint-url "/v1/responses")
                   {:headers {"Authorization" (str "Bearer " api-key)
                              "Content-Type" "application/json"}
                    :body (json/generate-string body)})

        ;; Parse response
        parsed-response (json/parse-string (:body response) true)

        ;; Extract structured data from response
        content (-> parsed-response :output first :content first :text)
        parsed-data (json/parse-string content true)

        ;; Decode using Malli transformer
        result (m/decode malli-schema parsed-data (mt/json-transformer))]

    ;; Validate and return
    (when-not (m/validate malli-schema result)
      (throw (ex-info "Response does not match schema"
                      {:schema malli-schema
                       :result result})))
    result))

(structured-output
   (get-base-url)
   (get-api-key)
   [:map
    [:department [:enum :licences :hardware :accounts :maintenance :other]]
    [:severity [:enum :low :medium :critical]]
    [:request :string]
    [:contact-information [:map
                           [:email {:optional true} :string]
                           [:phone {:optional true} :string]
                           [:raw :string]]]]
   "Hi, my laptop diesn't let me login anymore. Can't work. What do? t. Hank PS call me 555-1343 because I can't access email")
; => {:department :hardware,
;     :severity :critical,
;     :request "Laptop doesn't let me login anymore. Can't work.",
;     :contact-information
;     {:raw "Hank, phone: 555-1343, cannot access email",
;      :phone "555-1343"}}

Notice how the model was able to process the human input that contained typos and informal, broken language. One doesn’t usually have to use any of the more powerful and expensive models to do this level of work. The older json_object response format has some capabilities and supports even lighter models. See OpenAI’s documentation for reference.

I think methods like this make it easy to embed LLM-enabled functionality in Clojure applications, giving them capabilities that are normally very hard to implement using traditional methods.

DISCLAIMER

If you were to implement the above example in your application as is, you’d be sending personal data to OpenAI. As developers, we should always consider the larger implications. You can deploy models locally in your region and keep data residency and processing within the EU region for example by using Solita’s FunctionAI in UpCloud or other service providers.

Further reading

If you’re new to agentic development, Prompt engineering 101 is a great starter for how to get past the first hurdles.

Permalink

I Used Claude to Mine 34 Clojure Adoption Interviews. Here is What It Found.

I pointed Claude at eight years of case study interviews and asked it to find the patterns. The results were more consistent - and more interesting - than I expected.

Permalink

Verifikation von Algorithmen mit Z3 – Teil 1

Ein Algorithmus soll effizient Lösungen zu gegebenen Problemen berechnen. In der Praxis besteht das Argument, dass der Algorithmus auch die richtigen Lösungen berechnet, meist aus einer Menge von Testfällen, einer kurzen Rechtfertigung in natürlicher Sprache oder einem Stoßgebet. Selbst eine große Menge von Testfällen – auch wenn diese bspw. mithilfe von QuickCheck erstellt wurden – kann aber keine Aussage über die allgemeine Korrektheit eines Softwarestücks machen. Um wirklich sicher zu gehen, müssen wir die Struktur des Codes betrachten, darüber Aussagen treffen und formal argumentieren, dass diese Aussagen auch allgemeine Geltung haben. Ein Mittel, um das zu tun, lernen wir in dieser dreiteiligen Artikelserie kennen: Wir nutzen den SMT-Solver Z3 als automatisierten Theorembeweiser. Diese Artikel sollen einen ersten Einblick geben in die Inhalte unserer neuen iSAQB-Advanced-Level-Schulung zu Formalen Methoden. In der Schulung kommen neben Z3 weitere Verifikationswerkzeuge wie VeriFast, Forge und Liquid Haskell zur Sprache.

Da wir formale Aussagen über Code treffen wollen, könnten wir versucht sein, die Aussagenlogik zu nutzen. In der Aussagenlogik kann man sich mit Variablen, „wahr“ und „falsch“ und ein paar Verknüpfungen logische Ausdrücke basteln (x oder (nicht y und z)). Das ist zwar ein nettes Vehikel der theoretischen Informatik, allerdings kann man damit leider für unsere Zwecke ziemlich wenig Interessantes aussagen. Die nächste Stufe solcher formalen Systeme wäre die Prädikatenlogik, bei der zusätzlich zu den Werkzeugen der Aussagenlogik Prädikate und Quantoren existieren. Die Mächtigkeit der Prädikatenlogik ist meistens ausreichend, um Korrektheitsbedingungen von Software zu beschreiben.

Die Herausforderung, Formeln in der Aussagenlogik auf ihre Erfüllbarkeit zu prüfen, nennt sich „SAT“ (für Satisfiability) und ist zwar im Worst-Case sehr rechenaufwendig, aber immerhin machbar. Dahingegen gibt’s für die Prädikatenlogik keinen allgemeinen Algorithmus, der Formeln mit Sicherheit auf Erfüllbarkeit überprüfen kann. Für jeden erdenklichen Prüfalgorithmus wird es immer Formeln geben, bei denen dieser sich in einer Endlosschleife verfängt.

Die Aussagenlogik ist also zu ausdrucksschwach, lässt sich aber automatisch ausrechnen. Die Prädikatenlogik ist ausdrucksstark genug, lässt sich aber nicht mehr automatisch ausrechnen. Gibt es vielleicht noch was dazwischen? Ja, gibt es und es nennt sich SAT Modulo Theories – SMT. Sehr frei übersetzt: SAT aber mit noch bisschen mehr Werkzeugen. Mit SMT-LIB-2 gibt’s einen Standard für die Sprache von SMT. Z3 ist ein sog. SMT-Solver, also ein Prüfalgorithmus für SMT-Formeln.

SMT-LIB-2 ist eigentlich gedacht als einheitliches Austauschformat zwischen SMT-Libraries auf der einen Seite und SMT-Solvern auf der anderen Seite. Zu diesem Zwecke basiert die Syntax von SMT-LIB-2 auf S-Expressions. Der Standard sagt sogar ganz explizit:

The choice of the S-expression syntax and the design of the concrete syntax was mostly driven by the goal of simplifying parsing, as opposed to facilitating human readability.

Diese Argumentation ist Teil der Kraft, die stets einfache Parser will und dabei einfache Lesbarkeit schafft: Als Scheme-, Racket- oder Clojure-Programmierende ist uns diese Syntax gerade recht. Man kann in SMT-LIB-2 direkt ziemlich gut programmieren und das werden wir in diesem Blogartikel demonstrieren. Als Anschauungsbeispiel wollen wir Geraden rasterisieren, um sie auf den Bildschirm zu malen.

Ich zieh ’ne Line, ihr zieht Leine

Eine Linie hat einen Start- und einen Endpunkt und verhält sich dazwischen sehr geradlinig. In der abstrakten Welt der Mathematik gibt es darüber gar nicht viel mehr zu wissen, doch wenn wir solche Linien auf einen Computerbildschirm zeichnen wollen, haben wir das Problem, dass dieser in der Regel aus einem Raster an Pixeln besteht; wir müssen die Punkte auf der idealen Linie also irgendwie in dieses starre Pixelkorsett zwängen. Um dieses Rasterisierungsproblem im folgenden noch etwas einfacher zu machen, bestimmen wir, dass der Startpunkt immer bei der Koordinate (0, 0) und dass der Endpunkt (dx, dy) rechts oben davon liegen soll – dass also dx > 0 und dy >= 0 sind. Eine solche Linie entspricht weitgehend einer Geradenfunktion f mit f(x) = dy/dx * x. Solche Funktionen f können wir in x-Richtung ganz einfach diskretisieren – der erste Schritt zur Rasterisierung –, indem wir sie nur für ganzzahlige x aufrufen. Im allgemeinen werden die Ergebnisse – also die y-Werte – aufgrund des Bruchs dy/dx dann aber rationale Zahlen und keine Ganzzahlen sein. Um auch in y-Richtung zu diskretisieren, müssen wir also geeignet runden.

Der simple Algorithmus als Spezifikation

Wir können solche Linien bzw. Geraden in SMT-LIB-2 direkt modellieren indem wir eine neue sog. „Sorte“ (als Programmierende würden wir sagen: Typ) einführen:

(declare-datatypes ()
                   ((Line (mk-line (line-dx Int)
                                   (line-dy Int)))))

Ab jetzt ist Line ein Bezeichner für die neue Sorte, mk-line ist der Konstruktor mit zwei Argumenten und line-dx und line-dy sind die zwei Getter.

Nicht jeder Ganzzahlwert für dx ist zulässig. Der eine andere Punkt (außer (0, 0)) muss auch ein anderer Punkt sein. Diese Einschränkung beschreiben wir in einer Funktion, die prüft, ob dx ungleich Null ist. Wir wollen uns hier außerdem weiter einschränken auf „flache“ Linien (dx >= dy) nach oben rechts (dx >= 0 und dy >= 0) (warum wir uns einschränken und warum das keine echte Einschränkung ist, wird später klar). Insgesamt sieht die Validierungsfunktion so aus:

(define-fun line-valid ((l Line))
  Bool
  (and
   (> (line-dx l) 0)
   (>= (line-dx l)
       (line-dy l))
   (>= (line-dy l)
       0)))

Für eine solche valide Linie können wir die (rationale) Steigung durch eine Divison berechnen. Diese Division ist zulässig, weil dx echt größer Null ist.

(define-fun line-slope ((l Line))
  Real

  (/ (line-dy l)
     (line-dx l)))

Mithilfe des line-slope finden wir den exakten (rationalen) y-Wert für einen gegebenen x-Wert mit einer einfachen Multiplikation heraus. Die Rasterisierung in x-Richtung ist damit automatisch sichergestellt, denn x kann nur ganzzahlige Werte annehmen:

(define-fun line-exact-y ((l Line) (x Int))
  Real
  (* (line-slope l) x))

Wenn wir jetzt wissen möchten, welcher Pixel zu diesem y-Wert korrespondiert, dann müssen wir nur noch runden. Die eingebaute Funktion to_int macht eine Real-Zahl zu einer Int-Zahl, rundet dabei allerdings immer ab. Um bspw. 0.6 zu 1 aufzurunden, addieren wir vor dem to_int-Aufruf einfach noch 0.5 drauf.

(define-fun round-to-nearest ((x Real))
  Int

  (to_int (+ x 0.5)))

(define-fun line-rounded-y ((l Line) (x Int))
  Int
  (round-to-nearest (line-exact-y l x)))

Um in der Welt der puren Funktionen zu bleiben, zeichnen wir nicht direkt per Seiteneffekt auf den Bildschirm, sondern berechnen erst eine Liste mit allen y-Werten von links nach rechts. Hier soll’s ja nicht um Effekte gehen, sondern um Algorithmen. Das Berechnen einer solchen Liste macht der übliche rekursive Algorithmus. Den kann man auch in SMT-LIB-2 hinschreiben:

(define-fun-rec draw-simple-acc ((l Line) (x Int) (todo Int))
  (List Int)
  (ite (<= todo 0)
       ;; done
       nil
       ;; recurse
       (insert (line-rounded-y l x)
               (draw-simple-acc l (+ x 1) (- todo 1)))))

(define-fun draw-simple ((l Line) (todo Int))
  (List Int)
  (draw-simple-acc l 0 todo))

draw-simple nimmt als Parameter eine Linie (also im wesentlichen eine Geradensteigung) und ein Ziel auf der x-Achse (todo). Wir rufen dann die Funktion draw-simple-acc auf mit dem zusätzlichen initialen Akkumulator 0. Dieser Akkumulator beschreibt die gerade betrachtete x-Koordinate, für welche wir die passende y-Koordinate entlang der Linie berechnen wollen. nil ist die leere Liste und mit insert bauen wir aus einer alten Liste und einem Listenelement eine neue Liste zusammen (oft heißt das Ding cons oder in Clojure conj).

Wir können den Algorithmus ausprobieren, indem wir ans Ende der Datei noch einen (check-sat)-Aufruf machen. Danach können wir uns Ergebnisse mithilfe von (simplify ...) ausdrucken. Dieser Code:

(check-sat)
(simplify (draw-acc (mk-state-1 (mk-line 7 5) 0) 4))

liefert dieses Ergebnis:

sat
(insert 0 (insert 1 (insert 1 (insert 2 nil))))

Wir sehen, dass bei diesen „flachen“ Geraden manchmal ein Schritt nach oben gemacht wird, manchmal verbleibt die rasterisierte Linie aber auch für ein paar Schritte auf derselben Höhe.

Die Funktion draw-simple-acc macht eine Menge Dinge gleichzeitig: Abbruchbedingung prüfen, Business-Logik aufrufen, den einen Teil der Business-Daten in die Liste einfügen und den anderen Teil der Business-Daten an den nächsten rekursiven Aufruf weiterleiten. Im folgenden werden wir die Business-Logik dieses Algorithmus nach und nach optimieren, während der buchhalterische Rest des Algorithmus gleich bleibt. Deshalb faktorisieren wir draw-simple-acc schon jetzt in zwei Teile. Der eine Teil ist schon fertig – wir nennen diesen Teil den Rahmenalgorithmus. Der andere Teil ändert sich mit jeder Optimierung – wir nennen diesen Teil die Businesslogik. Die Businesslogik besteht immer aus vier Elementen:

1. Es gibt einen Typen (Sorte) für den Zustand, welcher im Rahmenalgorithmus als Akkumulator fungiert.
2. Es gibt eine Schrittfunktion, die einen alten Akkumulator in den nächsten Akkumulator überführt.
3. Es gibt eine Extraktorfunktion, die aus dem Zustandsobjekt den y-Wert für die aktuelle Iteration rausholt.
4. Um mit der Rechnerei starten zu können, brauchen wir eine Funktion, die uns ein initiales Zustandsobjekt baut.

In Code sieht das dann so aus: Der Zustand enthält eine Linie und das x, was vorher der Akkumulator war.

(declare-datatypes ()
                   ((State-1 (mk-state-1 (state-1-line Line)
                                         (state-1-x Int)))))

Ähnlich zum Linienobjekt sind nur manche Zustandsobjekte valide. Wir definieren entsprechend wieder eine Validierungsfunktion:

(define-fun state-1-valid ((st State-1))
  Bool
  (and
   (line-valid (state-1-line st))
   (>= (state-1-x st) 0)))

Wir definieren außerdem die Extraktorfunktion:

(define-fun state-1-exact-y ((st State-1))
  Real

  (line-y (state-1-line st)
          (state-1-x st)))

(define-fun state-1-y ((st State-1))
  Int
  (round-to-nearest
   (state-1-exact-y st)))

Die Schrittfunktion zählt einfach nur das x hoch. Das Linienobjekt wird nicht verändert:

(define-fun step-1 ((st State-1))
  State-1
  (mk-state-1
   (state-1-line st)
   (+ 1 (state-1-x st))))

Die Funktion, die das initiale Zustandsobjekt baut, ist mit dem Konstruktor mk-state-1 bereits gegeben. Wir können diese Teile jetzt in den Rahmenalgorithmus einsetzen. Das sieht dann so aus:

(define-fun-rec draw-1-acc ((st State-1) (todo Int))
  (List Int)
  (ite (<= todo 0)
       ;; done
       nil
       ;; recurse
       (insert (state-1-y st)
               (draw-1-acc (step-1 st) (- todo 1)))))

(define-fun draw-1 ((l Line) (todo Int))
  (List Int)
  (draw-1-acc (mk-state-1 l 0) todo))

Als funktional Programmierende würden wir den Rahmenalgorithmus natürlich gern als Funktion höherer Ordnung hinschreiben. Das geht in SMT-LIB-2 leider nicht.

Addition statt Multiplikation

Dieser Algorithmus ist einfach zu verstehen, arbeitet allerdings sehr verschwenderisch. Er macht mit der Funktion state-1-y in jedem Schritt eine Multiplikation. Das ist teuer, zumindest teurer als es sein müsste. Wir wissen ja, dass wir in jeder Iteration nur einen x-Schritt nach rechts gehen. Zur Optimierung dieses Algorithmus können wir uns die meiste Arbeit dieser wiederholten Multiplikation sparen, indem wir in jedem Schritt einfach nur einmal die Steigung auf den vorher berechneten y-Wert draufaddieren. Das erfordert natürlich, dass wir uns den y-Wert auch gemerkt haben, also packen wir diesen in ein neues Zustandsobjekt State-2. Wir definieren den Extraktor gleich mit:

(declare-datatypes ()
                   ((State-2 (mk-state-2 (state-2-line Line)
                                                     (state-2-x Int)
                                                     (state-2-exact-y Real)))))

(define-fun state-2-y ((st State-2))
  Int
  (round-to-nearest
   (state-2-exact-y st)))

Die zugehörige Validierungsfunktion muss jetzt auch noch prüfen, ob der y-Wert im Zustand auch zur angegebenen Linie und dem x-Wert passt.

(define-fun state-2-valid ((st State-2))
  Bool
  (and
   (line-valid (state-2-line st))
   (>= (state-2-x st) 0)
   (= (state-2-exact-y st)
      (line-y (state-2-line st)
              (state-2-x st)))))

Die neue Schrittfunktion macht jetzt nur noch eine Addition und keine ganze Multiplikation mehr:

(define-fun step-2 ((st State-2))
  State-2

  (mk-state-2
   (state-2-line st)
   (+ 1 (state-2-x st))
   (+ (state-2-exact-y st)
      (line-slope
       (state-2-line st)))))

Und jetzt fehlt nur noch die Funktion, die für eine gegebene Linie den initialen Zustand berechnet:

(define-fun init-state-2 ((l Line))
  State-2
  (mk-state-2 l 0 0.0))

Diese Funktionen können wir wieder in den Rahmenalgorithmus einsetzen. Wir nennen das Ergebnis draw-2 (und führen es hier nicht noch mal aus).

Das haben wir jetzt alles so hinprogrammiert, aber ist es auch richtig? Um diese Frage beantworten zu können, müssen wir uns erst mal klar machen, was „richtig“ hier überhaupt bedeutet. Ich würde vorschlagen, der Algorithmus arbeitet richtig, wenn er unter Absehung technischer Details die gleichen Ergebnisse wie der einfachere Algorithmus oben produziert. Formalisiert würden wir also gern solche eine solche Aussage prüfen:

(assert
 (forall ((l Line)
          (todo Int))
         (= (draw-1 l todo)
            (draw-2 l todo))))

Das kann man in SMT-LIB-2 so hinschreiben und Z3 rennt sogar los – es wird aber nie fertig. Mit einem solchen Programm befinden wir uns in einem Logikfragment, das für Z3 nicht mehr entscheidbar ist. Wir können aber eine Spezifikation aufschreiben, die funktional äquivalent und trotzdem entscheidbar ist. Der erste Teil davon ist einfach: Wir wollen sagen, dass der initiale Zustand, der aus init-state-2 rausfällt ein valider Zustand ist und außerdem das erwartete y zurückgibt, wenn man den Extraktor darauf anwendet. Wir wollen natürlich diese Aussage wieder prüfen für alle Linien. Dennoch müssen wir kein forall verwenden. Mit forall suchen wir nach einer Garantie der Allgemeingültigkeit unserer Formel. Anstatt aber die Allgemeingültigkeit zu prüfen, können wir auch das Gegenteil unserer Formel behaupten und dann fragen, ob diese negierte Formel erfüllbar ist:

(declare-const l Line)
(assert (line-valid l))

(assert
 (not (state-2-valid (init-state-2 l))))

(check-sat)

Wir können dieses SMT-LIB-2-Programm mit Z3 prüfen lassen. Raus kommt: unsat. Das klingt unbefriedigend, ist aber das Ergebnis, das sagt, dass unsere Verifikation erfolgreich ist. Die Aussage „Unsere Formel gilt nicht“ ist nicht erfüllbar, für kein einziges Linienobjekt l, d.h. unsere Formel gilt für alle l.

Das wichtigere Korrektheitskriterium betrifft step-2. Wir wollen sagen, dass step-2 sich so verhält wie step-1, abgesehen von einigen technischen Details. Formalisiert:

(declare-const st State-1)
(assert (state-1-valid st))

(assert
 (not
  (= (step-2
      (into-state-2 st))
     (into-state-2
      (step-1 st)))))

(check-sat)

Hier steht, dass es egal ist, ob wir für einen gegebenen State-1 zuerst mit into-state-2 in die State-2-Welt gehen und dann step-2 aufrufen, oder ob wir zuerst step-1 aufrufen und dann in die State-2-Welt eintauchen. Dieses Korrektheitskriterium ist ein typisches „kommutierendes Diagramm“: Egal welchen Pfad man verfolgt, man landet immer bei demselben Ergebnis. Dass das gilt, bestätigt uns Z3 mit einem weiteren unsat.

Die Korrespondenz zwischen den Welten von State-1 und State-2, welche wir als Korrektheitskriterium herangezogen haben, ist noch recht offensichtlich. Dafür hätten wir vielleicht Z3 gar nicht gebraucht. Wir wollen unseren Algorithmus im nächsten Artikel aber noch weiter verbessern. Die Art der Korrektheit bleibt dabei immer dieselbe: Wir sagen, unsere neuen Algorithmen sollen sich wie der offensichtlich richtige erste Algorithmus verhalten.

Permalink

Clojure Deref (Feb 10, 2026)

Permalink

On Dyslexia, Programming and Lisp.

Dyslexia is a difference in the way the brain processes language. It is often defined as the difference between intelligence (vaguely defined via the fake IQ test) and a person's ability to read. That is, a dyslexic with normal intelligence will underperform compared to their peers in learning how to read. There are many different theories on what the cause is, but most people believe that there are fundamental differences between the brains of people who have dyslexia and those that do not. In Maryanne Wolf's book, Dyslexia, Fluency, and the Brain, several essays point to a physical difference in the density of specialized brain cells that the brain uses to process letter shapes. It has become clear recently from fMRIs that dyslexics actually develop unique neural pathways for reading that are located in the right side of the brain compared to the normal left-side-dominated reading pathways. However, a root cause has not been identified, and the jury is still out. Dyslexia varies considerably, with up to 15% of the population experiencing some form of it.

The experience of dyslexia is very interesting. I am dyslexic, and I was diagnosed in the 3rd grade. My dyslexia was more "intense" than some, having both verbal and visual effects. I remember trying to learn to read and struggling with all the letter rotation on the page. PQ DB IL letters all looked the same to me.

The best way to describe it is that my brain was attempting to process these letters for their meaning and would automatically rotate the letter. When you think about it, it actually makes sense. When you pick up an object with your hand that you don't understand, what do most people do? They turn it to look at all sides of the object. That's what my brain does with letters.

Because English letters are not "fixed," there is no inherent difference between the bottom and top of the letter; my brain could not fix the position of the letter on the page. The rotation was intense for me, as I was experiencing rotation in "3D." For me, this made the letter move more on the page and also made the letters even more confusing for me.

It made reading very difficult until my brain was able to fix their position. I did this by adopting what I called the "Sky line" approach. Each word makes a unique shape, like the skyline of buildings in a city. By memorizing the shape of each word in the English language, I could fix the position of the word on the page and overcome this rotation problem. Interestingly enough, once I memorized the shape of most of the words in English, I found this method to be very fast. It may be because I can see the shape of the word quickly, not relying on letter shape.

Frankly, rotation was a nightmare for me, literally. As a child, I remember having dreams/nightmares where I was falling and spinning. I would fall forever, spinning and spinning. I could not orient myself. Once, in this nightmare, I realized my solution. There was no fixed position. My desire to fix the position of the letter, or myself, was a fool's errand. Rotation itself was the only constant; the rotation, in a way, was a fixed position. It's a little difficult to describe, and this "realization" did come to me as a dream, so take it for what you will. I guess I am still spinning.

Anyway, moving on from that, in college I developed an interest in software engineering. Taking a class in C and then quickly learning Python. C and its family of languages are interesting. They are frankly a nightmare of obscure scribbles that had little meaning to me. I think a classic example of this is the ternary conditional operator. This is an operator that I still struggle with. I rapidly found that Python's whitespace-based syntax was much easier on my spinning brain. Most of the obscure squiggles are removed, and I was able to focus on the semantic meaning of the program. I found that my Sky Line approach for visual memorization works very well for programming languages. Each function or class defines its own shape. Remembering each shape in a large codebase is easier than remembering how to spell my own name.

After college I learned about Lisp, specifically the Clojure programming language. After learning about Clojure, I dove headlong into Scheme and its family of S-expression languages. At last, I had found a language that was seemingly designed for my spinning brain.

Lisp-based languages use S-expressions. These S-expressions form a constant and regular pattern. All expressions are enclosed in parentheses (). The first symbol is always the function or the macro. The rest of the expressions are arguments to that function or macro. Lisp programming is layers and layers of S-expressions, each with their own shape, making them easy to memorize and locate. I no longer have to worry about the rotation of the ternary conditional operator.

Scheme fascinates me even further. Many of the coding conventions in Scheme encourage people to use full and descriptive function names. I believe this gave my mind a deeper ability to organize the skyline shapes. Maybe you can think of this like the difference between 32-bit and 64-bit pointers in garbage collectors. The longer the function name, the more functions I can store in my mind's working memory.

These experiences leave me wondering what other characteristics we could add to our programming languages that would help people like myself. In general, people with "normal," left-dominated reading pathways have constructed languages. In doing so, they have built language that works effectively for them. These languages are tangles of obscure squiggles, a spinning brain nightmare. This tangle allows for denser packing of meaning into a "smaller" space, easy for normies, or so they think. But I can't help but notice the popularity of languages like Ruby and Python. Both of these languages take approaches that reduce the visual complexity.

I wonder if by reducing the visual complexity, we can instead increase the amount of semantic complexity we can hold in our minds. Dyslexia is common, and many people experience some form of the issues I describe above. Perhaps, we are all suffering from the tyranny of obscure squiggles.

Permalink

future - Clojure

Code

;; future.clj

(future
  (println "This runs in background")
  (Thread/sleep 1000)
  (println "Done after 1 second"))

(let [f (future
          (Thread/sleep 2000)
          "Result")]
  (println "Immediate return")  ; Prints immediately
  (println @f)) ; Blocks here for 2 seconds if needed

;;;;;;;;;;;;;

(def my-future (future (+ 1 2 3)))

;; Block until result is ready
(println @my-future)  ; Prints: 6

;; Or with timeout (returns nil if not ready in 1000ms)
(deref my-future 1000 :timeout)

;;;;;;;;;;;;;

(do
  (def my-future-3 (future (Thread/sleep 2000) (+ 1 2 3)))

  ;; Or with timeout (returns nil if not ready in 1000ms)
  (deref my-future-3 100 :timeout))

;;;;;;;;;;;;;

(do
  (def my-future-4 (future (Thread/sleep 20) (+ 1 2 3)))

  ;; Or with timeout (returns nil if not ready in 1000ms)
  (deref my-future-4 100 :timeout))

Permalink

Limit concurrent HTTP connections to avoid crippeling overload

Even the fastest web servers become bottlenecks when handling CPU-intensive API work. Slow response times can cripple your service, especially when paired with an API gateway that times out after 29 seconds.

I solved this using a simple middleware that eliminated 504 - Gateway Timeout responses and significantly reduced unnecessary load on my service API.

I assumed that if a single request takes 5 seconds on average, at least five requests could complete before hitting Amazon API Gateway’s 29-second timeout:

In practice, the behavior was completely different (though in retrospect, it makes perfect sense). CPU resources are divided equally among all concurrent requests, causing all responses to slow down proportionally:

I found myself in a situation where even a mediocre load would make the system incapable of responding within the time limit. The API gateway would return 504 - Gateway Timeout on behalf of my service, while my unaware service would occupy CPU resources for responses that would never be used for anything, slowing everything even further.

A sure way to contribute to climate change and get a high cloud bill, while delivering zero value.

Oh wait…
a caller is very likely to want to retry a request, indicating a temporary problem, which the HTTP response code 504 - Gateway Timeout does. Now multiply your already high cloud bill by the retry count.

In other words: A disaster. ☠️

An entirely different architecture, maybe involving a queue or some async response mechanism, would probably have been a better solution. But sometimes, we need to work with what we’ve got.

Since my CPU load was fairly consistent across requests, I could predict how many concurrent connections could complete within the timeout limit.

With the following middleware, I limit concurrent active connections to ensure high CPU utilization while still responding within the timeout:

(defn wrap-limit-concurrent-connections
  "Middleware that limits the number of concurrent connections to ´max-connections´,
   via the atom `current-connections-atom`.
   This means that the middleware can be applied in several different places
   while still sharing an atom if necessary."
  [handler current-connections-atom max-connections]
  (fn [request]
    (let [connection-no (swap! current-connections-atom inc)]
      (try
        (if (>= max-connections connection-no)
          (handler request)
          {:status 503 :body "Service Unavailable"})
        (finally
          (swap! current-connections-atom dec))))))

The middleware implementation is very naive and assumes that, the service only exposes work with a similar load profile so that the same middleware (and coordination atom) can be reused across the service.

Though the middleware does make the 504 - Gateway Timeout responses go away, they are replaced with slightly fewer 503 - Service Unavailable errors. The important part is that the maximum possible number of 200 - OK are allowed to pass through, making the system partially responsive while scaling up (deploying more instances).

I ran tests to find the right value for max-connections that matches the given work and hardware the service was running on.

Endpoints with low CPU intensity, such as health checks, should not be wrapped in the middleware. You don’t want a service instance terminated and restarted just because the health check can’t communicate: I’m still doing important stuff.

A more sophisticated rate-limiting middleware is possible using the same scaffolding as above. Maybe something that times requests and reduces concurrency as response time goes up, or something with different weights instead of just incrementing and decrementing by one. But if this starts getting hairy, you might be better off with an entirely different architecture.

Use with caution. 💚

Permalink

Full Stack Engineer (mid- to senior-level) at OpenMarkets Health

Permalink

Python Only Has One Real Competitor

Python Only Has One Real Competitor

Clickbait subtitle: "and it's not even close"

Python is the undisputed monarch in exactly one domain: Data Science.

The story, as I understand it, goes something like this:

Python has very straight-forward interop with native code. Because interop is straightforward, libraries like numpy and pandas got made. Around these an entire Data Science ecosystem bloomed.

This in turn gave rise to interactive notebooks with iPython (now "Jupyter"), plotting with Matplotlib, and machine learning with PyTorch and company.

There are other languages and platforms - like R, MATLAB, etc. - which compete for users with Python. If you have a field biologist out there wading through the muck to measure turtles, they will probably make their charts and papers with whatever they learned in school.

But the one glaring weakness of these competitors is that they are not general purpose languages. Python is. This means that Python is also widely used for other kinds of programs - such as HTTP servers.

So for the people who are training machine learning models to do things like classify spam it is easiest to then serve those models using the same language and libraries that were used to produce them. This can be done without much risk because you can assume Python will have things like, say, a Kafka library should you need it. You can't really say the same for MATLAB.

And in the rare circumstance you want to do something that is easiest in one of those alternative ecosystems, Python will have a binding. You can call R with rpy2, MATLAB with another library, Spark with PySpark, and so on.

For something to legitimately be a competitor to Python I think it needs to do two things.

1. Be at least as good at everything as Python.
2. Be better than Python in a way that matters.

The only language which clears this bar is called Clojure.

That's a bold claim, I know. Hear me out.

Clojure has a rich ecosystem of Data Science libraries, including feature complete numpy and pandas equivalents in dtype-next and tech.ml.dataset. metamorph.ml for Machine Learning pipelines, Tableplot for plotting, and Clay for interactive notebooks.

For what isn't covered it can call Python directly via libpython-clj, R via ClojisR, and so on.

Clojure is also a general purpose language. Making HTTP servers or whatever else in Clojure is very practical.

What starts to give Clojure the edge is also the answer to the age-old question: "Why is Python so slow?"

Python is slow because it cannot be made fast. The dark side of Python's easy interop with native code is that many of the implementation details of CPython were made visible to, and relied upon by, authors of native bindings.

Because all these details were relied upon, the authors of the CPython runtime can't really change those details and not break the entire Data Science ecosystem. This heavily constrains the optimizations that the CPython runtime can do.

This means that people need to constantly avoid writing CPU intensive code in Python. It is orders of magnitude faster to use something which delegates to the native world than something written in pure Python. This affects the experience of things like numpy and pandas. There is often a "fast way" to do something and several "slow ways." The slow ways are always when too much actual Python code gets involved in the work.

Clojure does not have this problem. Clojure is a language that runs on the Java Virtual Machine. The JVM can optimize code like crazy on account of all the souls sacrificed to it. So you can write real logic in Clojure no issue.

There's a reason Python's list is implemented in C code but Java can have multiple competing implementations, all written in Java. Java code can count on some aggressive runtime optimizations when it matters.

This also means that if you use Clojure for something like an HTTP Server to serve a model, you can generally expect much better performance at scale than the equivalent in Python. You could even write that part in pure Java to make use of that trained pool of developers. Anecdotally, startups often switch from whatever language they started with to something that runs on the JVM once they get big enough to care about performance.