This is going to be the first post in a series about Clojure and how to get started with it. In this post, I’ll talk about what Clojure is, why it’s interesting to learn, and how to get a Clojure environment set up on your system. In subsequent posts, I’ll dive into the basics of Clojure and give you some pointers for how to get started with it. What is Clojure? Clojure is a functional progra...
This is going to be the first post in a series about Clojure and how to get started with it. In this post, I’ll talk about what Clojure is, why it’s interesting to learn, and how to get a Clojure environment set up on your system. In subsequent posts, I’ll dive into the basics of Clojure and give you some pointers for how to get started with it. What is Clojure? Clojure is a functional progra...
tl;dr: you can generate very small (less than 1k) JS artifacts from ClojureScript with some tradeoffs. I worked out a list of rules to follow and made the cljs-ultralight library to help with this.
Most of the web apps I build are rich front-end UIs with a lot of interactivity. Quite often they are generating audio in real time and performing other complicated multimedia activites. This is where ClojureScript and shadow-cljs really shine. All of the leverage of a powerful LISP with its many developer-friendly affordances (editor integration, hot-loading, interop, repl) brought to bear, allowing me to quickly iterate and build with fewer bugs.
On many projects I find myself also needing a small amount of JavaScript on a mostly static page. An example would be a static content page that has a form with a submit button that needs to be disabled until particular fields are filled. It seems a bit excessive to send a 100s of kilobyte JS file with the full power of Clojure's immutable datastructures and other language features just to change an attribute on one button.
In the past I resorted a tiny bit of vanilla JS to solve this problem. I have now discovered I can use ClojureScript carefully to get most of what is nice about the Clojure developer experience and still get a very small JS artifact.
Here's an example from the Jsfxr Pro accounts page. What this code does is check whether the user has changed a checkbox on the accounts page, and shows the "save" (submit) button if there are any changes.
(ns sfxrpro.ui.account)
(defn start {:dev/after-load true} []
(let [input (.querySelector js/document "input#update-email")
submit-button (.querySelector js/document "button[type='submit']")
initial-value (-> input .-checked)]
(aset submit-button "style" "display" "none")
(aset input "onchange"
(fn [ev]
(let [checked (-> ev .-target .-checked)]
(aset submit-button "style" "display"
(if (coercive-= checked initial-value)
"none"
"block")))))))
(defn main! []
(start))
This code compiled to around 500 bytes. It has since been updated to do a bunch of different more complicated stuff and today it compiles to 900 bytes. I'll talk about some of the special weirdness and language tradeoffs in a second, but first here is the shadow-cljs config I used.
{:builds {:app {:target :browser
:output-dir "public/js"
:asset-path "/js"
:modules {:main {:init-fn sfxrpro.ui/main!}}
:devtools {:watch-dir "public"}
:release {:output-dir "build/public/js"}}
:ui {:target :browser
:output-dir "public/js"
:asset-path "/js"
:modules {:account {:init-fn sfxrpro.ui.account/main!}}
:devtools {:watch-dir "public"}
:release {:output-dir "build/public/js"}}}
The first build target :app is for the main feature-rich app which does all the complicated stuff. I am fine with this being a larger artifact as it does a lot of things for the user including realtime generation of audio samples.
The second target :ui creates a file called account.js which is just 900 bytes. It gets loaded on the accounts page which is statically rendered. The reason for two completely separate build targets is otherwise the compiler will smoosh all of the code together and bloat your artifact size. It is easiest just to keep both codebases completely separate.
When compiling I found it useful to have a terminal open watching the file size of account.js so I could see real time when the size ballooned out and figure out which code was making that happen.
So what tricks do we have to use in the code to get the artifact size down? Here is a brief list of rules to follow to stay small. If you break any of these rules your artifact size will balloon.
vec or hash-map or list for example. Instead you have to use native JavaScript data structures at all times like #js {:question 42} and #js [1 2 3]. That also means you will have to use aget and aset instead of get and assoc. It means we are dropping immutablity and other data type features.= operator. I know that sounds mad but what you can use instead is ClojureScript's coercive-=. This function does a native-style surface level JavaScript comparison. This means you have to give up the value based equality comparison you can use on deeply nested datastructures in Clojure.partial. Other clever built-ins like juxt are probably going to be bad for file size too. As far as I can tell it's anything that uses immutable Clojure types under the hood. For the specific case of partial you can use #(...) instead to do what you need.js/console.log instead of print.(.map some-js-array some-func) instead of (map some-func some-js-array)Generally as far as possible you should stick with native JavaScript calls and data types.
If all of this sounds onerous remember that the idea here is to only do this in situations where you have a small codebase giving the user some small amount of interactivity on a web page. So that's the tradeoff. You still get LISP syntax, editor integration, hot-loading, repl, and lots of other nice Cloure stuff, but you have to forgo immutable datastructures and language features like partial.
I have created a small library called cljs-ultralight to help with the UI side of things. It uses browser calls and returns JS data types. You can use it to perform common UI operations like attaching event handlers and selecting elements, without incurring too much overhead.
The library applied-science.js-interop also works with these techniques. Require it like this: [applied-science.js-interop :as j] and you can use j/assoc! and j/get and friends. Note if you use j/get-in or other functions that take a list argument, you should instead pass a JavaScript array which works well.
Also note that @borkdude has a couple of very interesting projects under way in this space. Check out squint and cherry for more details.
New is always better.
– Barry Stinson, Senior Clojure developer
Over the years I’ve heard countless complaints about nREPL and its numerous perceived flaws. I dawned me today that it might be a good idea to write this “living”1 article that will try to catalog and address some of them.
I’ll focus mostly on the common complaints in the Clojure community, but I’ll also touch upon some of the broader perceived issues with the nREPL protocol itself.
A lot has been said about the choice of Bencode for encoding data between the client and the server. The format was very primitive they said. JSON is better! EDN is better! X(ML) is better!
Of course, back when this decision was made EDN didn’t even exist and for most editors dealing effectively with JSON was a very tall order. For clients like vim and Emacs it was much easier to write a simple bencode implementation (typically a few hundred lines of code) than to have to create a fast (and correct) JSON parser in VimScript or Emacs Lisp. Admittedly, I struggled for a while to get even the CIDER bencode parser working properly!
Bencode has many limitations, but it was the widest common denominator and in my opinion its choice contributed a lot to the early success of nREPL.
I find it funny that even though the Clojure nREPL implementation eventually provided JSON (Transit) (see https://github.com/nrepl/fastlane) and EDN (built-in) transports, practically no one uses them.2
I also can’t expect any language-agnostic protocol to adopt a niche format like EDN and expect any form of wide adoption.
REPL stands for something - Read, Eval, Print, Loop.
It does not stand for - Eval RPC Server/Window.
It is not something I “feel” about. But I do think we should be precise about what we call REPLs.
While framing and RPC orientation might make things easier for someone who just wants to implement an eval window, it makes the resulting service strictly less powerful than a REPL.
– Rich Hickey (https://groups.google.com/forum/?pli=1#!msg/clojure-dev/Dl3Stw5iRVA/D_Kcb-naOx4J)
The above excerpt is from a 2015 conversation around the introduction of Clojure’s socket REPL server and what (nREPL) problems does it solve. I’ve seen this conversation cited many times over the years to support the thesis that nREPL is problematic in one regard or another. So, is nREPL a real REPL or not? Does this even matter if it does what you need it to do?
Well, nREPL was never meant to be a “real REPL”. It was meant to be a REPL
server that makes it easy for people to build development tools (e.g. Clojure
editor plugins) upon. And framing requests and responses makes a huge difference
there. Sure, it might look appealing to be relying only on eval for
everything, but you still need to be able to process whatever you got from
eval. It’s also pretty handy to be able to match requests and responses that
originated from them.
I guess nREPL’s approach was eventually “redeemed” when the socket REPL didn’t gain broad
tool adoption, and was followed up by prepl that framed the input. Still, it
didn’t gain broad adoption for development tooling as was hard to deal with its
unstructured output.
Now things have come full circle with a recently introduced Clojure REPL (framed request, framed response).
To wrap it up here - we can either be dogmatic or practical. I always go for being practical - it might not be a real REPL, but it sure gets real work done.
Can’t argue with that one. That obviously puts nREPL at an disadvantage when it comes to Clojure’s built-in socket REPLs or REPLs that get “injected” by clients (e.g. the now defunct unrepl).
On the bright side - this also untangles nREPL from the Clojure’s release cycle. And makes contributing to the project a lot easier.
By the way, I always felt this particular concern to be quite overblown as nREPL is small self-contained library with no external dependencies. Also it’s easy to have it only in development profile (where it’s typically used). Technically it’s also possible to “upgrade” any socket REPL to an nREPL, but we never finished the work in this direction. (see https://github.com/nrepl/nrepl/commits/injection) I guess there wasn’t enough demand for this.
One more thing - tools like CIDER and Calva have been injecting nREPL during development automatically for quite a while, which blurs quite a bit the line between built-in and third-party. The point I’m trying to make here is that there’s a fine line between potential issues and real issues. I prefer to focus on the latter group.
Maybe they do. Maybe they don’t. Obviously they are concept that predates nREPL and people have been divided about it pretty much its entire existence.
I’m a bit biased, having worked with middleware for many years (long before I got interested in Clojure) and I’ve always liked the idea of being able to process requests and responses like a pipeline transforms them. But I also realize that comes with a certain learning curve and the ordering of the pipeline steps is not always clear, which can result in various problems.
I guess no middleware is more controversial than piggieback, the one responsible for ClojureScript support. Love it or hate it, there’s no denying that the decision to organize nREPL’s functionality around middleware made it possible to provide ClojureScript support (ClojureScript didn’t even exist at the time when nREPL was created) without having to touch the nREPL codebase. It also made possible some fairly interesting workflows like hosting a Clojure and a ClojureScript on the same nREPL server. And, of course, this also means that anyone can provide a replacement for piggieback if they wish to.
At the end of the day, however, one should not forget that middleware is just an implementation detail in the reference Clojure nREPL server and you can implement the nREPL protocol without middleware. Babashka’s nREPL server is a nice example of such an approach.
I was hoping that by now there would also be more native ClojureScript nREPL servers, but alas. Maybe this year?
No, it’s not. Just take a look at the nREPL protocol ops (sans session management ops):
completionsevalinterruptload-filelookupstdinWhich one of them seems Clojure-specific to you?
Yeah, the protocol design was obviously influenced by Clojure and we never gave much thought on how to standardize nREPL certain responses, but nREPL was always meant to be a language-agnostic protocol.
Admittedly it didn’t gain much traction outside of the Clojure & broader Lisp community, that’s a different matter altogether.
Well, that’s more or less true as of today, but I plan to address this eventually. In particular the structure of responses is mostly
undocumented, which is forcing some people reverse engineer it from existing nREPL implementations. That response structure also needs
a bit of consideration as ops like lookup have responses that are somewhat linked to common Clojure var metadata.
If you’ve always wanted to contribute to open-source projects - that’s a great way to get started.
Basically, we need a standard test suite that checks whether nREPL implementation conform to the nREPL protocol specification. This would make it much easier to build nREPL servers and to verify their proper work.
That’d be certainly nice to have and I think it’s very doable. Perhaps one of you, dear readers, is looking for a fun small OSS project?
I came across the following statement today:3
First of all, nREPL is mainly designed for Clojure, tested against Clojure, and implements features that Clojure needs in a pretty specific Clojure way.
That’s certainly not how I see things. I believe Chas Emerick wouldn’t agree with the assessment either.
Many misconceptions about nREPL originate from a conflation of the nREPL protocol and the reference Clojure implementation. When I took over nREPL I expanded the documentation greatly and made the protocol more prominent here and there. Still, a lot of damage has already been done and we’ll need to do more to separate those clearly in the minds of most people.
I see this quite linked to documenting the protocol clearly and building a standard nREPL compatibility test suite.
No software is perfect and neither is nREPL. It grew organically over the course of many years and it definitely accumulated a bit of weirdness and cruft along the way. And a lot of charm, plus a track record of stability and robustness.
Would we have done everything the same way knowing what we know today? Probably not. But nREPL still gets the job done and it’s still one of the better options for tool authors who don’t want to reinvent the wheel every time.
I get that it’s always fun to complain about something4 and to work on exciting new alternatives5, but I think in the spirit of Clojure it’s also not a bad idea to appreciate the value of stability and reliability. And keep improving a good thing.
Next time you’re very frustrated with nREPL consider trying to find a solution for your problem that everyone in the community might benefit from. New is always better until it’s not. In the (n)REPL we trust!
P.S. If I forgot to mention something you hate about nREPL, please share it in the comments.
I’ll be updating it from time to time. ↩
Admittedly we never adjusted the protocol structure for them, meaning those transports don’t make use of something that can’t be represented with Bencode. ↩
See https://andreyorst.gitlab.io/posts/2023-03-25-implementing-a-protocol-based-fennel-repl-and-emacs-client/ ↩
I’m often guilty of this myself. ↩
And I have no doubt that some of the alternatives are better than nREPL in one way or another. ↩
Sometimes, in a vast and healthy developer ecosystem of a language like Clojure, it can be difficult to know where to get started when you want to build an application. The target audience of this series is developers who have some Clojure syntax in their hands and want to start building applications.
Over a series of posts we'll build stagehand, a web application for managing an inventory of servers. An inventory is, in the style of Ansible, a collection of data including system and network information.
Specifically, the Servers in our inventory can be:
We'll give it a frontend for common tasks (CRUD, running playbooks) and an API for use by other programs.
This article will cover:
neil to initialize the projectdeps.edn and the Clojure CLIringThese articles will assume readers have some familiarity with the Clojure language. For a quick introduction check out this primer. For a more complete guide checkout the only book for the brave and true.
For every topic I'll provide an introduction and include references to more authoritative or comprehensive sources. By the end readers should have a bit more familiarity and a folder of bookmarks to dig into.
Clojure programmers generally prefer to build applications by composing libraries together rather than using frameworks.
There are Clojure frameworks, they provide a reliable foundation to build on top of. For developers new to Clojure, however, exploring the starter template of a framework feels like figuring out how to get an alien spaceship running.
I believe that a gradual introduction to foundational libraries is a more productive starting point.
To follow along you'll need to install a few things. Links to installation directions here:
bash scripts" resolution bb script to manage your deps.ednFor me, a Clojure dev environment isn't complete without Babashka and neil. These two projects have done a lot in making Clojure more accessible.
Before continuing ensure these commands run without errors:
clojure -M -e '(println "Clojure Online")'
bb -e '(println "Bash? Bash who?")'
neil --version
The choice of text editor is a personal one.
I use Neovim with Conjure. For more detail about my setup check out this post.
For a more complete description of editor options you can check out:
This project will use deps.edn to manage its dependencies, and we'll use neil to manage deps.edn!
neil can:
deps-new templatenreplneil version patchneil version major 3 --forceneil version minor --no-tagMichiel Borkent (@borkdude), the author of babashka, neil, clj-kondo, and many others wrote a great introduction to neil here that goes into more depth.
We'll start by using neil new to initialize the project using a template.
neil new --help
# Usage: neil new [template] [name] [target-dir] [options]
#
# Runs the org.corfield.new/create function from deps-new.
#
# All of the deps-new options can be provided as CLI options:
#
# https://github.com/seancorfield/deps-new/blob/develop/doc/options.md
#
# ...snip...
# The provided built-in templates are:
#
# app
# lib
# pom
# scratch
# template
# ...snip...
The options for deps-new and the default templates can be found here for later reference.
The scratch template includes nearly nothing. Perfect!
Templates can accept options to customize their behavior. We'll use the --scratch option to modify the path of the initial source file the template creates.
neil new scratch stagehand --scratch stagehand/app
# Creating project from org.corfield.new/scratch in stagehand
Let's take a look at our new project:
cd stagehand/
tree
# .
# ├── deps.edn
# └── src
# └── stagehand
# └── app.clj
#
# 2 directories, 2 files
# Not much here. How many lines of code?
wc -l **/*
# 4 deps.edn
# 12 src/stagehand/app.clj
# 16 total
Two files with just sixteen lines of code between them! Might as well include it all here:
;; deps.edn
{:paths ["src"]
:deps {}
:aliases
{:neil {:project {:name stagehand/stagehand}}}}
;; src/stagehand/app.clj
(ns stagehand.app
"FIXME: my new org.corfield.new/scratch project.")
(defn exec
"Invoke me with clojure -X stagehand.app/exec"
[opts]
(println "exec with" opts))
(defn -main
"Invoke me with clojure -M -m stagehand.app"
[& args]
(println "-main with" args))
The docstrings on the functions above show that we can run our new project by either executing a function or by running -main:
clojure -X stagehand.app/exec :name "Rattlin"
# exec with {:name Rattlin}
clojure -M -m stagehand.app Hello World
# -main with (Hello World)
It's working! Those commands are a bit opaque though. The next section will provide some context.
The Clojure CLI is the companion to the deps.edn file. Its main job is to:
classpath so that your source code and libraries are available to the JVMHere are the commands we just ran, with notes on the flags and arguments.
clojure -X stagehand.app/exec :name "Rattlin"
# -X => eXecute
#
# stagehand.app/exec => the `exec` function from the `stagehand.app` namespace
# found on the classpath.
# The function name is not important, though it should
# take a map as a single argument
#
# :name "Rattlin" => `:key "Value"` pairs that are rolled into a map
# and passed to the called function as its only argument.
# In this case that map will look like:
# {:name "Rattlin"}
clojure -M -m stagehand.app Hello World
# -M => Say to yourself, 'Ah, we're using `clojure.main` here.
# So all further options are for `clojure.main`'
#
# -m, --main => Specify a namespace to look for a function named `-main` to
# execute
#
# stagehand.app => The namespace we're going to look for `-main` in
#
# Hello World => Arguments to pass to the `-main` function, as seq of strings
I highly recommend reviewing these resources for a more comprehensive explanation:
clojure.main - Official ReferenceThe scratch template doesn't include a .gitignore file. Let's copy one from the app template:
# assuming you're in the root of the stagehand directory
pushd ..
neil new app the-giver
cp the-giver/.gitignore $OLDPWD
rm -r the-giver
popd
Let's save our game:
git init
git add .gitignore deps.edn src/
git commit -m 'Getting started'
Making this a repo will make it easier to see what the next few commands are adding to our project by using git diff.
nreplOne of Clojure's greatest selling points is its support for the editor connected REPL.
Working at the REPL feels like playing with a Rubik's Cube. It's constantly in your hands. The feedback is instant. In comparison, developing compile-and-run languages feels like setting up a bunch of dominos over and over. Though with TDD you can get that loop to look like this:
REPL driven development and TDD are not mutually exclusive. Use your REPL to setup your dominos! Or something!
To use the REPL from our text editor we'll use nrepl. neil can help us out here too:
neil add nrepl
{:paths ["src"]
:deps {}
- :aliases
- {:neil {:project {:name stagehand/stagehand}}}}
+ :aliases
+ {:neil {:project {:name stagehand/stagehand}}
+
+ :nrepl ;; added by neil
+ {:extra-deps {nrepl/nrepl {:mvn/version "1.0.0"}}
+ :main-opts ["-m" "nrepl.cmdline" "--interactive" "--color"]}}}
This command has added an nrepl alias to our deps.edn file. Aliases are another feature of the Clojure CLI that enables certain tasks to specify extra dependencies or add another entrypoint to the program.
In this case we see that the nrepl alias specifies an additional dependency on nrepl/nrepl from Maven at version 1.0.0. The :main-opts key is a hint to us that this alias should be run with clojure -M.
clojure -M:nrepl
# Explantion:
# -M => Using `clojure.main` here!
#
# :nrepl => Use the `:nrepl` alias in our deps.edn file so that
# the extra dependency gets loaded, and all the options
# specifed in `:main-opts` get passed to `clojure.main`
For demonstrations of working at the REPL check out:
If we don't add a test runner now we probably never will.
neil add test
tree test/
test
└── stagehand
└── stagehand_test.clj
1 directory, 1 file
Running a git diff will show that neil added an alias to our deps.edn file:
{:paths ["src"]
:deps {}
:aliases
{:neil {:project {:name stagehand/stagehand}}
:nrepl ;; added by neil
{:extra-deps {nrepl/nrepl {:mvn/version "1.0.0"}}
- :main-opts ["-m" "nrepl.cmdline" "--interactive" "--color"]}}}
+ :main-opts ["-m" "nrepl.cmdline" "--interactive" "--color"]}
+
+ :test ;; added by neil
+ {:extra-paths ["test"]
+ :extra-deps {io.github.cognitect-labs/test-runner
+ {:git/tag "v0.5.0" :git/sha "b3fd0d2"}}
+ :main-opts ["-m" "cognitect.test-runner"]
+ :exec-fn cognitect.test-runner.api/test}}}
The test alias adds in the cognitect-labs/test-runner for running our tests. We can run our new test by:
# Using the clojure CLI
clojure -M:test
# or with neil
neil test
We're writing a web application, so we need a way to handle HTTP requests and serve up some HTML. For that we'll use ring.
Ring is the current de facto standard base from which to write web applications in Clojure.
What ring provides:
The ring wiki is great and worth going through end-to-end.
We'll use neil to find the rings, neil to bring them all, and in the deps.edn bind them... ahem
# Neil can help you find libraries with a `search` command
neil dep search ring
# :lib ring/ring-core :version 1.9.6 :description "Ring core libraries."
# :lib ring/ring-codec :version 1.2.0 :description "Library for encoding and decoding data"
# :lib ring/ring-servlet :version 1.9.6 :description "Ring servlet utilities."
# :lib ring/ring-jetty-adapter :version 1.9.6 :description "Ring Jetty adapter."
# :lib ring/ring-devel :version 1.9.6 :description "Ring development and debugging libraries."
# --- snip ---
# We'll start with the minimum set to get off the ground
neil add dep ring/ring-core
neil add dep ring/ring-jetty-adapter
# Let's see how this changes the deps.edn file:
git diff deps.edn
diff --git a/deps.edn b/deps.edn
index 87caaea..4e6b5cd 100644
--- a/deps.edn
+++ b/deps.edn
@@ -1,5 +1,6 @@
{:paths ["src"]
- :deps {}
+ :deps {ring/ring-core {:mvn/version "1.9.6"}
+ ring/ring-jetty-adapter {:mvn/version "1.9.6"}}
:aliases
{:neil {:project {:name stagehand/stagehand}}
With this in place we can start hacking on this application. Start your REPLs!
clojure -M:nrepl
# nREPL server started on port 59171 on host localhost - nrepl://localhost:59171
# nREPL 1.0.0
# Clojure 1.11.1
# OpenJDK 64-Bit Server VM 17.0.4.1+1
# Interrupt: Control+C
# Exit: Control+D or (exit) or (quit)
# user=>
There's a prompt for you to type Clojure forms into, but we won't be using that very much if at all. Instead we'll evaluate expressions from our text editor.
As the output mentions, there is an nrepl server listening locally on a random port, 59171 in this case. Editors with Clojure support know to look for connect to this server by by referencing the .nrepl-port file, which was created when we ran the previous command.
cat .nrepl-port
# 59171
Refer to your editor specific documentation about managing your connection to the nrepl server and evaluting forms.
Let's get to "Hello World" with ring. Edit src/stagehand/app.clj and type along:
;; file: src/stagehand/app.clj
(ns stagehand.app
"Server Inventory Management"
;; To start working with ring we need a server+adapter
;; Jetty is a good default choice
(:require [ring.adapter.jetty :as jetty]))
;; Adapters convert between server specifics to more general ring
;; requests and response maps. This allows you to change out the server
;; without updating any of your handlers.
;; We'll store the reference to the server in an atom for easy
;; starting and stopping
(defonce server (atom nil))
;; Any function that returns a response is a "handler."
;; Responses are just maps! Ring takes care of the rest
(defn hello
[_request]
{:status 200
:headers {"Content-Type" "text/plain"}
:body "Hello World\n"})
;; note: the `_` in `_request` indicates the argument is unused, while
;; still giving it a useful name
;; Same as above, the 404 handler just returns a map
;; with the "Not Found" status code
(defn not-found
[request]
{:status 404
:headers {"Content-Type" "text/plain"}
:body (str "Not Found: " (:uri request) "\n")})
;; app is the main handler of the application - it'll get
;; called for every request. It will route the request
;; to the correct function.
;;
;; For routing we'll start by just matching the URI.
;; We'll add in a real routing solution in the next blog post
(defn app
[request]
(case (:uri request)
"/" (hello request)
;; Default Handler
(not-found request)))
;; start! the Jetty web server
(defn start! [opts]
(reset! server
(jetty/run-jetty (fn [r] (app r)) opts)))
;; note: the anonymous function used as the handler allows us to revaluate the
;; `app` handler at the REPL to add additional routes / logic without
;; restarting the server or process.
;;
;; Another option is to pass in the handler as a var, `#'app`
;; For a deeper explanation check here:
;; https://clojure.org/guides/repl/enhancing_your_repl_workflow
;; stop! the server and resets the atom back to nil
(defn stop! []
(when-some [s @server]
(.stop s)
(reset! server nil)))
;; -main is used as an entry point for the application
;; when running it outside of the REPL.
(defn -main
"Invoke me with clojure -M -m stagehand.app"
[& _args]
(start! {:port 3000 :join? true}))
;; This is a "Rich" comment block. It serves as a bit of documentation and
;; is convenient for use in the REPL. All the code above is available for use,
;; including our handlers!
(comment
;; Just call the handler by providing your own request map - no need
;; to actually run the server
(app {:uri "/"})
; {:status 200,
; :headers {"Content-Type" "text/plain"},
; :body "Hello World"}
;; For use at the REPL - setting :join? to false to prevent Jetty
;; from blocking the thread
(start! {:port 3000 :join? false})
;; Evaluate whenever you need to stop
(stop!)
;; At the REPL, *e is bound to the most recent exception
*e)
With your nrepl connected editor, evaluate the call to start! in the comment block at the bottom of this file to get the server going. With this we should be able to verify our server is up and our handlers are working as expected:
curl http://localhost:3000
# Hello World
curl http://localhost:3000/bird
# Not Found: /bird
The raw content of an HTTP request looks like:
GET / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.86.0
Accept: */*
The actual request is easier to write than most plain-text data formats, like JSON or YAML. Unfortunately programs need this to be in a form they can understand. Ring handles translating HTTP requests into Clojure maps.
Let's add a handler to print the request map as our handler sees it.
First we'll add in clojure.pprint to pretty-print the request map:
(ns stagehand.app
"Server Inventory Management"
(:require [ring.adapter.jetty :as jetty]
+ [clojure.pprint :refer [pprint]]))
Add a dump function above app:
(defn dump [request]
{:status 200
:headers {"Content-Type" "text/plain"}
:body (with-out-str (pprint request))})
Update app with an /dump route
(defn app
[request]
(case (:uri request)
"/" (hello request)
+ "/dump" (dump request)
;; Default Handler
(not-found request)))
Reevaluate these functions in your editor/REPL and make a request. Add some extra fields to see how ring handles it:
curl -v 'http://localhost:3000/dump?test=true&something=extra&something=else'
* Trying 127.0.0.1:3000...
* Connected to localhost (127.0.0.1) port 3000 (#0)
# Our request
> GET /dump?test=true&something=extra&something=else HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/7.86.0
> Accept: */*
>
# The response
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Sun, 19 Mar 2023 01:28:04 GMT
< Content-Type: text/plain
< Transfer-Encoding: chunked
< Server: Jetty(9.4.48.v20220622)
<
{:ssl-client-cert nil,
:protocol "HTTP/1.1",
:remote-addr "127.0.0.1",
:headers
{"accept" "*/*", "user-agent" "curl/7.86.0", "host" "localhost:3000"},
:server-port 3000,
:content-length nil,
:content-type nil,
:character-encoding nil,
:uri "/dump",
:server-name "localhost",
:query-string "test=true&something=extra&something=else",
:body
#object[org.eclipse.jetty.server.HttpInputOverHTTP 0x6ad325e0 "HttpInputOverHTTP@6ad325e0[c=0,q=0,[0]=null,s=STREAM]"],
:scheme :http,
:request-method :get}
The request map Ring produces provides a bit of additional context and breaks out various parts for easy access. There's definitely room for improvement, such as automatically parsing the :query-string. We'll address this in the next section with middleware.
The ring wiki describes the request and response maps in greater detail.
There's a more complete version of this dump handler in the
ring/ring-devellibrary:ring.handler.dump/handle-dump.ring-develhas some very useful functions to aid with development. We'll probably revist this library in a later post.
Middleware offers a way to address cross-cutting concerns across groups of handlers. Middleware can add additional information to a request/response map, or even transform the body of the request.
We'll add some parameter parsing middleware to the entire application. Thankfully ring/ring-core includes middleware to handle this.
First we'll apply the ring.middleware.params/wrap-params to parse any query parameters and form bodies. The full docstring is included here, it's shorter and more complete than anything I could write:
ring.middleware.params/wrap-params [handler] [handler options] ──────────────────────────────────────────────────────────────────────── Middleware to parse urlencoded parameters from the query string and form body (if the request is a url-encoded form). Adds the following keys to the request map: :query-params - a map of parameters from the query string :form-params - a map of parameters from the body :params - a merged map of all types of parameter Accepts the following options: :encoding - encoding to use for url-decoding. If not specified, uses the request character encoding, or "UTF-8" if no request character encoding is set.
The wrap-params middleware above uses string values as keys in the maps it creates. Generally keywords are preferred as keys for easier/faster value access. There's middleware for that too, again here's the docstring:
ring.middleware.keyword-params/wrap-keyword-params [handler] [handler options] ──────────────────────────────────────────────────────────────────────── Middleware that converts the any string keys in the :params map to keywords. Only keys that can be turned into valid keywords are converted. This middleware does not alter the maps under :*-params keys. These are left as strings. Accepts the following options: :parse-namespaces? - if true, parse the parameters into namespaced keywords (defaults to false)
We'll require these namespaces, do a bit of renaming, and then finally apply our middleware using the threading macro.
(ns stagehand.app
"Server Inventory Management"
(:require [ring.adapter.jetty :as jetty]
+ [ring.middleware.params :refer [wrap-params]]
+ [ring.middleware.keyword-params :refer [wrap-keyword-params]]
[clojure.pprint :refer [pprint]]))
----
- (defn app
+ (defn main-handler
[request]
(case (:uri request)
"/" (hello request)
"/dump" (dump request)
;; Default handler
(not-found request)))
+ (def app
+ (-> #'main-handler
+ wrap-keyword-params
+ wrap-params))
Note that #'main-handler is using a var-quote. This makes it so that changes to main-handler are picked up in the REPL.
Middleware is applied in a bottom to top fashion, so first wrap-params does its work, followed by wrap-keyword-params, and then our main-handler.
Reevaluate the file and run that request from earlier:
curl 'http://localhost:3000/dump?test=true&something=extra&something=else'
{ ...omitted...
+ :params {:test "true", :something ["extra" "else"]},
+ :form-params {},
+ :query-params {"test" "true", "something" ["extra" "else"]},
:query-string "test=true&something=extra&something=else",
}
It's working! One thing to note is that query parameters with the same name become a vector in the :params map.
Before writing your own middleware, check the list of standard middleware or thrird party libaries on the ring wiki.
We're off the ground! There's not much of stagehand here yet though. The next article will add a few more libraries and start giving adding in some initial functionality.
The next few articles will cover:
reititThis article was heavily influenced by:
The ClojureDoc guide to Basic Web Development was rewritten by the amazing Sean Corfield shortly before this article was published. It covers a lot more ground, give it a read!
This is step-by-step process to setup vim to be a great (Clojure) editor. This post covers:
A few years ago I didn't see the value in using Neovim over Vim. That has recently changed. I learned that Neovim:
treesitterInstallation instructions are here.
When I first started using Neovim I went missing for three days. When I was found by the seaside I got to show the rescue party a neat Lua based configuration and an okay setup.
Thankfully there have been some community efforts to help prevent folks from getting lost out there.
I know use nvim-lua/kickstart.nvim to get started. It's intended to be the starting point for your custom configuration. To build up some foundational knowledge checkout these resources:
Here's the quickstart, assuming *NIX or MacOS:
# Backup anything:
mv ~/.config/nvim{,.bak} || echo ' -- Nothing to backup'
# Clone the repo to ~/.config/nvim
git clone https://github.com/nvim-lua/kickstart.nvim.git ~/.config/nvim
# Start nvim, watch the initial setup go
nvim
# Restart nvim when its done
treesitterFirst we'll add Treesitter support for Clojure:
nvim ~/.config/nvim/init.lua
Search for Configure Treesitter - this is line 287 for me. Adding languages to the ensure_installed portion has treesitter automatically fetch and install support for these languages.
I recommend adding:
markdownbashhtml, css, if you do web stufffennel, a Lua powered Lisp, more details later onclojureThe updated block should resemble:
-- ~/.config/nvim/init.lua
-- ...some ways down...
-- [[ Configure Treesitter ]]
-- See `:help nvim-treesitter`
require('nvim-treesitter.configs').setup {
-- Add languages to be installed here that you want installed for treesitter
ensure_installed = {
'c', 'cpp', 'go', 'lua', 'python', 'rust', 'tsx', 'typescript', 'help', 'vim',
'markdown', 'bash', 'html', 'css', 'fennel', 'clojure'
},
---- snip
Save the file and restart nvim. You should see notifications in the status bar that Treesitter is installing support for these languages.
The main plugin of interest is Conjure which enables interactive development, connecting Neovim to your running program.
For an in-depth demonstration, check out Conjure's author, Oliver Caldwell, and his talk Conversational Software Development: What, Why and How
Additional plugins add support for structural editing. These are:
If you're giving structural editing a try, read through the READMEs above and reference this gist.
Custom plugins are added to: ~/.config/nvim/lua/custom/plugins/init.lua.
After these plugins restart nvim so they install and load.
-- ~/.config/nvim/lua/custom/plugins/init.lua
-- You can add your own plugins here or in other files in this directory!
-- I promise not to create any merge conflicts in this directory :)
--
-- See the kickstart.nvim README for more information
return {
-- Conjure!
'Olical/conjure',
-- Structural editing, optional
'guns/vim-sexp',
'tpope/vim-sexp-mappings-for-regular-people',
'tpope/vim-repeat',
'tpope/vim-surround',
}
localleaderConjure makes heavy use of the localleader, typically bound to comma. The kickstart configuration sets this to <space>, but I've found it doesn't seem to break anything if I set it as comma. This nicely separates Conjure keymaps from the rest of our setup.
To make the change, open ~/.config/init.lua and edit:
-- ~/.config/nvim/init.lua
-- around line 43 or so
vim.g.maplocalleader = ','
To try out your new setup, and learn more about Conjure, execute this command from nvim:
:ConjureSchool
You can install additional language servers, linters, and formatters using Mason:
:Mason
This configuration uses Lazy as a package manager.
:Lazy
Use Telescope to explore files, help pages, key maps, the local buffer... just about everything!
:Telescope keymaps
""" Here's some to get you started:
" [s]earch [f]iles
<Space>sf
" [s]earch [g]rep - might require ripgrep
<Space>sg
" Recent files
<Space>?
" Current buffer
<Space>/
" Switch open buffers
<Space><Space>
Since you probably arrived here to get set up for writing more Clojure, you might also be interested in Aniseed and Fennel.
Aniseed allows you to write your Neovim configuration, as well as plugins in Fennel, a Lisp which compiles to Lua. Conjure itself is written this way.
Oliver Caldwell has his own magic-kit to get you started on that journey. I'll look for you by the sea.
There's also cajus-nvim, another Aniseed based starter kit.
I am happy that first edition of my Clojure book is launched, one can get it here. I thank all those who have helped me to achieve this. Though I have put myself as the author many have contributed to it, for example the AsciiDoctor project, Language Tools, the Free Software Foundation and so on, to name a few.
I am committed to learn Clojure more to make myself a better software engineer, and I will be documenting my work here. I plan to release video tutorial of my book soon, it too will be libre and gratis. I might release 2nd edition of my book by Jan 2024 where it will contain more sections, and hopefully all the tasks documented in gitlab should have been done.
In 2017 I started to learn more about the universe of Functional Programming. This paradigm was gaining traction and most object-oriented programming languages were adding more and more features inspired by this paradigm, including the language that I used the most: Java.
After the JDK 8 release in March 2014, it became common to hear Java developers using terms like: functional programming, streams, optional, map, flat map, etc. But many people around me still ignored these new features and, I confess, it took me a while to adopt them. The ideas sounded very interesting, but putting them into practice turned out to be more difficult than I expected.
After much trial and error, I decided to dig deeper into the concepts. The book Functional Thinking helped me take my first steps in the right direction.
In parallel I decided to learn a functional-first programming language instead of trying to partially apply the functional paradigm in an object-oriented language. After doing a lot of research, I chose to learn Elm. The fact that it is a pure and immutable functional language caught my attention. Also, it is focused on webapps development and, until then, I hadn't found any solution for developing web pages that I liked.
After going through the whole Introductory Guide to the Elm Language and reading the Elm in Action book, I already felt quite comfortable developing webapps in this paradigm. I liked Elm so much that I started a project to teach programming to beginners using this language and made the first classes available on the website elm.dev.br (in Brazilian Portuguese).
But there was a serious problem that I still had to face: Elm is a language designed for webapps development and works very well for that, but I was looking for a general purpose solution, that could also be used in backends development. So I started hunting for another language again.
Elm is a statically typed language inspired by Haskell. The natural step would be to use Elm on the frontend and Haskell on the backend. And that's what I tried to do. I read with some difficulty the Learn You a Haskell for Great Good! book (available for free here) and learned a lot of cool stuff. But creating a complete backend using Haskell proved to be more than I could chew. So I decided to look for alternatives...
During this whole process the word Lisp kept popping into my head! From time to time I would come across a video of someone influential in the community talking about it (like this video or this twit by John Carmack, founder of id Software). It felt mystical. And the fact that Nubank adopted Clojure brought a very real and pragmatic case study of the use of a Lisp dialect here in Brazil.
Until then I was postponing studying it because I was prioritizing statically typed languages and the most famous dialects of Lisp are dynamic languages. But in early 2021 I finally decided to give it a chance. I chose the language Clojure and started reading the Getting Clojure book. Unlike my Haskell studies, I managed to read this book in just a few days! At the same time, I started taking Clojure classes at Alura (a popular online programming school in Brazil), which helped me to see how to program in this language in hands-on. It was a good combination: in the book I learned the concepts of the language more deeply and in the course I reviewed these concepts and learned the more practical parts.
Lisp is not a programming language, but a family of languages with many dialects. The most famous dialects include Common Lisp, Clojure, Scheme and Racket. So after deciding that I was going to learn Lisp, I had to choose one of its dialects.
Clojure stood out to me for two reasons:
The experience of programming in Clojure was quite liberating. Practice TDD together with REPL Driven Development (technique quite widespread inside the Clojure community) makes the feedback loop very fast. The fact that Clojure is a dynamic language also contributes to that.
Another characteristic of Clojure is that it is an impure language, that is, we can have side effects at any time. The main advantage of this is that it makes the language easier to learn (although it brings with it a host of other problems that don't happen in purer languages like Elm or Haskell).
But although it is an impure language, it encourages a series of good practices that significantly reduces the potential problems of this approach.
Clojure might be intimidating at first, but after a few hours you get used to it's syntax and it's actually quite easy to learn. That's why I also chose this language to share with other developers the basic fundamentals of functional programming.
If books are your thing, I recommend starting with Getting Clojure, which as I said earlier, is a great way to understand the basic principles behind Clojure. But if you are looking for a free option, you may start with the online version of the Clojure for Brave and True book. Another option more focused on the foundations of the paradigm and that addresses languages other than Clojure is the book Functional Thinking, by Neal Ford.
I created a 10 hours course on Introduction to Functional Programming with Clojure with optional payment (yes, you can enroll for free if you want to!). But right now it's only available in my native language: Brazilian Portuguese. 😉
And you, what is your favorite paradigm? Have you tried programming using the functional paradigm? What were your main difficulties? Share your experiences in the comments!
Did you like this text? Checkout my other articles at: https://segunda.tech/tags/english and follow me on twitter.
Notes
clj꞉user꞉> ; Use `alt+enter` to evaluate
;; strategy pattern
(+ 4 3)
7
clj꞉user꞉>
(* 4 3)
12
clj꞉user꞉>
((if true + *) 4 3)
7
clj꞉user꞉>
((if false + *) 4 3)
12
clj꞉user꞉>
(defn add [a b] (+ a b))
#'user/add
clj꞉user꞉>
(defn multiply [a b ] (* a b))
#'user/multiply
clj꞉user꞉>
((if false add multiply) 4 3)
12
clj꞉user꞉>
((if true add multiply) 4 3)
7
Greetings Clojurists Together community! Read the latest updates for the following 2022 projects: Clj-kondo, ClojureDart, Kaocha, and Practicalli.
A range of updates, new content and tool reviews and testing. Reviewed 75 solutions for 44 students on Exercism.io over the last two week, adding that advice and code walk-throughs to the Practicalli Clojure book. \
Contributions to other projects
Created a fennel based configuration for Cloure development with Neovim, with a focus on REPL driven dvelopment, using LSP and as distraction free as possible.
All configuration written in fennel, except for a smal lua file to boostrap aniseed fennel to lua compiler.
Started a book to help use Neovim and the Practicall/neovim-config for those new to Neovim (and vim)
LSP completion testing shows inconsistent completion on aliases for required libraries
In this post, I’ll give updates about open source I worked on during January 2023.
Sponsors But first off, I’d like to thank all the sponsors and contributors that make this work possible! Top sponsors:
If you want to ensure that the projects I work on are sustainably maintained, you can sponsor this work in the following ways. Thank you!
If you’re used to sponsoring through some other means which isn’t listed above, please get in touch.
Attention
If you are using Github Sponsors and are making payments via Paypal, please update to a creditcard since Github Sponsors won’t support Paypal after February 23rd 2023. Read their statement here. If you are not able to pay via a creditcard, you can still sponsor me via one of the ways mentioned above.
Native, fast-starting Clojure interpreter for scripting. New releases in the past month: 1.0.170 - 1.1.173 Highlights:
I contributed changes to the following libraries to make them compatible with babashka:
The new babashka http-client project mostly replaces babashka.curl. This month the default client was improved to accept gzip and deflate as encodings by default, reflecting what babashka.curl did. Also babashka.http-client is now available as a built-in namespace in babashka v1.1.171 and higher.
Static analyzer and linter for Clojure code that sparks joy. Three new releases with many fixes and improvements in the last month. Check the changelog for details. Some highlights:
This is a new project and gives you access to a subset of instaparse via a pod. Instaparse was request a few times to have as a library in babashka and instaparse-bb is a good first step, without making a decision on that yet. See the relevant discussion here.
Remove unused Clojure vars. In the 0.3.5 version, Carve got the following updates:
CLI to transform between JSON, EDN, YAML and Transit using Clojure Version 0.4.23:
File system utility library for Clojure. Fs has gotten a few new functions:
See changelog for more details.
A CLI to add common aliases and features to deps.edn-based projects. This month there were several small fixes, one of them being to always pick stable versions when adding or upgrading libraries. See full changelog for details.
Light-weight static blog engine for Clojure and babashka. The blog you’re currently reading is made with quickblog. Version 0.2.3 was released with contributions from several people, mostly enabling you to tweak your own blog even more, while having good defaults. Instances of quickblog can be seen here:
If you are also using quickblog, please let me know! A collection of ready to be used SCI configs for e.g. Reagent, Promesa, Re-frame and other projects that are used in nbb, joyride, scittle, etc. See recent commits for what’s been improved.
Edamame got a new function: parse-next+string which returns the original string along with the parsed s-expression.
I added the ability to build and deploy 4ever-clojure to Github Actions. Every time a commit is merged, the site is automatically updated.
The following projects also got updates, mostly in the form of maintenance and performance improvements. This post would get too long if I had to go into detail about them, so I’ll briefly mention them in random order:
Discuss this post here. Published: 2023-02-05 Tagged: clojure oss updates
In this post, I’ll give updates about open source I worked on during February 2023.
Native, fast-starting Clojure interpreter for scripting. New release: 1.2.174. Highlights:
Babashka compatibility in external libs I worked together with the maintainers of the following libraries to make them compatible with babashka:
Static analyzer and linter for Clojure code that sparks joy. New release: 2023.02.17. Some highlights:
Configurable Clojure/Script interpreter suitable for scripting and Clojure DSLs. This month:
File system utility library for Clojure. Highlights:
Clojure library for shelling out / spawning sub-processes This month I looked into wrapping output of processes with a prefix so when ran in parallel, you can easily distuingish them. A preliminary solution is in this thread.
Interact with clojure-lanterna from babashka. A very experimental 0.0.1 release was published. You can try it out by playing tetris in the console with babashka:
bb -Sdeps ‘{:deps {io.github.borkdude/console-tetris {:git/sha “2d3bee34ea93c84608c7cc5994ae70480b2df54c”}}}’ -m tetris.core
Scripting in Clojure on Node.js using SCI Finally nbb has gotten support for passing maps to keyword argument functions:
Several other improvements have been made in the area of macros and resolving JS library references and resolving dependencies in an nbb.edn file, relative to an invoked script which is not in the current directory. See changelogs here.
VSCode CLJS scripting and REPL (via SCI) This month I contributed a built-in version of rewrite-clj to joyride, so joyriders can rewrite their code from within VSCode.
Showcase CLJS libs using SCI A little project to show how you can use SCI to showcare your CLJS library in an interactive way.
The following projects also got updates, mostly in the form of maintenance and performance improvements. This post would get too long if I had to go into detail about them, so I’ll briefly mention them in random order:
These are some of the other projects I’m involved with but little to no activity happened in the past month.
Discuss this post here. Published: 2023-03-01 Tagged: clojure oss updates
This final update covers January 2023. This month we merged our changes for a better hot reload (see Update #2) and then went on bug fixing and doing small improvements for a while.
A problem kept bothering us: in our cljd.flutter (well in January it was stillcljd.flutter.alpha2) we have :watch to react to updates to all kind of stateful objects and sub to narrow the scope of changes we are interested in for a given “watchable”. This is very useful as it allows to change the frequency at which a piece of UI is updated: one can have a big atom holding a lot of state (and this changing frequently) and watch a slower changing subset of it.
However sub only allowed to narrow one watchable. This led to awkward code were every other watchables (IO or framework related) updates had to be shoe-horned in a single canonical atom with the help of listeners (whose lifecycles have to be tied to specific parts of the widget tree) and hair-pulling naming decisions on paths inside this atom.
We first extended sub to allow to apply a function across several watchables latest values, somehow behaving like a join. It worked but felt quite stiff and not very pleasing to use. sub relied on the f & args update convention (as used in swap!, update-in etc.) but there’s no agreed upon convention for when one has several input and thus it doesn’t mesh as well as the single-input version with the core lib.
This dissatisfaction led us to develop “cells”. Cells as in spreadsheets. So cells are expressions which recompute their value when their dependencies values change. Obviously we made cells watchable. A cell is defined by using the $ macro (because it caches its latest value) and inside (dynamically, not lexically) a cell one can take (<!) from other cell or any other watchable.
Example: assuming now is an atom (a plain old one) updated at 60fps, then ($ (.-day (<! now))) is a cell recomputed at 60fps but yielding a new value once a day. So any widget (or other cell) depending on this cell will only be updated once a day.
This generalizes nicely to multiple dependencies and even to dynamic dependency graphs. Another interesting fact to know about cells is that a cell is recomputed only when it’s watched (and this cascades transitively: an unwatched cell doesn’t watch its dependencies…). Another consequence of cells is that the push for a single big canonical atom goes away.
This changed positively the way we write apps. So much that in our February workshop we talked only about cells, not subs.
Dec. 2022 Update:
spy.com.lambdaisland as well as lambdaisland, see https://github.com/lambdaisland/open-source/issues/91.2.4.--notification-timeout or:kaocha.plugin.notifier/timeout. Note that this feature doesn’t work for terminal-notifier on macOS, with Java’s built-in TrayIcon, or with notify-send on certain Linux desktop environments and distributions.--watch. Previously, profiles would be respected on the initial load, but not after watch reloaded the configuration.java.lang.IllegalArgumentException: No matching clause: [] exception when :kaocha.spec.test.check/syms is a collection.Jan. and Feb. 2023 Update: After a refreshing break, we’ve hit the ground running in 2023, merging approximately 20 PRs from both Gaiwan and community contributors, fixing many bugs and some CI/cljdocs issues. Several new libraries support Babashka, including Kaocha. We hope this and other efforts will expand the pool of Clojure libraries and tools available for use in Babashka. Most of the focus has been on Kaocha, which saw eight separate releases over the past two months. We’ve also released minor updates for Launchpad, embedkit, and Deja-fu.
--watch and Neovim by bumping Beholder to 1.0.2:as-aliaskaocha.repl/config accepts a :profile key when specifying extra options.kaocha.plugin.capture-output/bypass macro, for temporarily bypassing output capturing.:each fixture could swallow thrown exceptions, resulting in a test being treated as passing when it should have been reported as an error. Fixed by changing how :each fixtures wrap the test function in execution. (thanks @NoahTheDuke)--watch with the default Beholder watcher.distance-in-words now renders approximate weeks; month ranges were adjusted
Welcome to the Clojure Deref! This is a weekly link/news roundup for the Clojure ecosystem. (@ClojureDeref RSS)
Two cool new projects you should check out this week - the Clojure Spring Challenge 23 provides fun puzzles you can solve. And my colleague Jarrod Taylor released the first game in the Clojure Arcade!
E89 SRE With Rahul De - ClojureStream podcast
Clojure Arcade: Mac Man Quick Start - Jarrod Taylor
Building Re-frame 06: Evolution of Re-frame Subscription - Engineering with V
Digging into deeply nested sequence in Clojure - Clojure Diary
Comparing Clojure and Javascript - Kari Marttila
Streaming PostgreSQL Large Objects using Clojure - Konstantinos Tzanidakis
Developing Ionic application using Clojurescript and Helix - To-do App - Marko Kocic
Creating an Uberjar for a Clojure Full Stack Application - Kari Marttila
Recife Temporal Properties - Paulo Rafael Feodrippe
clojure-mode meets tree-sitter - Bozhidar Batsov
London Clojurians Community - a relatively brief history - John Stevenson
New releases and tools this week:
clj-kondo 2023.03.17 - Static analyzer and linter for Clojure code that sparks joy
panas.reload - a hot reload for babashka serving html+css (or htmx)
splint 0.1.119 - Splint is a Clojure static code analyzer and linter
edamame 1.3.20 - Configurable EDN/Clojure parser with location metadata
web-security 1.0.0-28 - Decoupled web security implementations for Clojure
spacemacs-config - rich Clojure & LSP config for Spacemacs
clerk-doc - Turn clojure files into markdown
openai-clojure 0.4.0 - Clojure functions to drive the OpenAI API
gh-release-artifact 0.2.1 - Upload artifacts to Github releases idempotently
http-client 0.1.8 - HTTP client for Clojure and Babashka built on java.net.http
cli 0.6.50 - Turn Clojure functions into CLIs!
babashka 1.3.175 - Native, fast starting Clojure interpreter for scripting
honeysql 2.4.1006 - Turn Clojure data structures into SQL
ring-lib 1.2.1-55 - Opinionated implementations for Clojure ring handler
deps.clj 1.11.1.1257 - A faithful port of the clojure CLI bash script to Clojure
tools.deps 0.18.1308 - Deps as data and classpath generation
Clojure CLI 1.11.1.1257
datahike 0.6.1539 - A durable Datalog implementation adaptable for distribution
vizdeps 1.0 - Visualize Leiningen dependencies using Graphviz
next-jdbc 1.3.862 - A modern low-level Clojure wrapper for JDBC-based access to databases
xtdb 1.23.1 - General-purpose bitemporal database for SQL, Datalog & graph queries
nginx-clojure 0.6.0 - Nginx module for embedding Clojure or Java or Groovy programs, typically those Ring based handlers
math.combinatorics 0.2.0 - Efficient, functional algorithms for generating lazy sequences for common combinatorial functions
At Solita we have a long-standing tradition of code camps - with records dating back to 2012. Last event of the kind was held seven years ago, but like fine wine, a good thing can’t be rushed. Here, a code camp is sort of a hackathon, a gathering of devs hacking away at some task, with a focus on Fun. At Oulu, we were located at Tervatoppila Saunamaailma (Saunaworld), in a nice cozy cottage-like atmosphere - just 10 minutes drive from the centrum. Of course, the saunas were booked as well to wash off the hacker’s blood, sweat, and tears - that’s a given. We also had catering and, most importantly, drinks.
Frontyard of Saunamaailma
I arrived in due time, found myself a spot at a table, grabbed a drink, and got set up. I had thought about looking at the templates beforehand, but coming from a game jam background I opted not to. To me, part of the fun is the hectic and manic hacking, observing as your initially well-thought and clean code throughout the event further and further breaks down, losing any semblance of coherence, finally embracing the chaos and becoming one with it. Surely preparing ahead would be cheating.
I decided to use Clojure, as I figured banging out stuff in the REPL - an interactive development environment that allows you to hook into the running application and evaluate code at runtime - could be very efficient in this type of fast hacking type environment. I haven’t used Clojure too much in this type of real-time context, so that was relevant to my interests as well. For web development, Clojure is my daily driver, so I’m pretty familiar with the language. As the allotted time was around five hours, that seemed advantageous. Granted, code camps offer a great opportunity to experiment with new technologies without pressure, with the trade-off being that you probably won’t accomplish much, especially if starting cold.
Our table, hacking away
The task for the code camp was to write a bot that was able to navigate a store, pick up the items with the best discounts, and when done, exit via the cash register to bank their score. Over time, the bots get exhausted, losing life, which can be replenished by potions (curiously presented as beer). If the bot’s life goes to zero, the bot is removed from the playing field. Some items can also be utilised as weapons, which can be used to attack other bots, for shenanigans. The store is represented as a top-down 2d map, where items spawn randomly. It has walls that must be navigated past and can have traps that cause damage.
The game itself was run on an Azure-based server instance. The bots communicated with the server with simple REST API, sending a movement, pick-up, or use action. Once per second, the game was updated with the latest actions for the bots. A local server for dev was also available, but I figured I’d be fine with the cloud setup.
The more difficult map
I cloned the repository, opened up the Clojure template, and fired up the REPL. I ran the function (api/register “Seppo”) and got a response. Perfect. My bot appeared on the playing field but died quickly since I wasn’t quick enough to send actions. Next up, some random movement. First though, since the server had chat available, that needed to be used. All would fear the war cry “Hail seitan”
I quickly got some random movement going, so it was time to focus on the Plan. I figured most would concentrate on optimising their score, so my goal instead was to focus on gathering as many weapons as possible and use them to blast my opponents off the map, keeping myself alive as long as possible. Any time a weapon spawns, my bot would beeline to it and use it. If no weapons were available, focus on collecting potions to keep the health up, otherwise just wander randomly.
My alkoritmi - drunkenness had to be implied in code as I was on NA beers
I needed a way to get the closest weapon and the closest potion. I implemented Manhattan distance calculation (sum of absolute x and y differences from my position to item position) and implemented some dumb movement - just check whether the x or y difference is greater, and move a step closer in that direction. Worked fine unless there were walls, which my bot stubbornly wanted to ignore and push through.
With luck, I got suitable spawns and managed to move toward the correct items. Great! However, item spawns proved to be an issue - bots were ignoring potions so over time the map was full of just potions. I figured I’ll just take my weapon search code, adjust it slightly and for when there are no weapons on the playing field, look for potions. This would both keep my health up and open up the map for more weapon spawns. I made additions for potions and set my bot loose again, and finding items seemed to work ok. Now, only to pick them up.
We had multiple rooms of people hacking
I had the biggest issues with my pick-up code for some reason. My bot would move correctly to the item but just wouldn’t for the life of it pick it up, it would just move on, notice there was an item close by, and move back to it, ad nauseam. It was sending the correct action, it was at the correct place, but just wouldn’t pick the item up. What the hell was going on? By chance I was looking at the server interface when other bots were operating and noticed that picking up items was not instant - it required multiple turns. And yes, the documentation stated as such, I just missed it.
At this point, it should be good to expand a bit on my bot’s lifecycle. Basically, I started the bot, it ran its course until it crashed due to an exception. For example, sending an action for a dead bot was an error. Once I got my movement working, a bot could theoretically live for a long time, which was not good for my development. I implemented a turn counter as a hard limit. With x turns done, just quit. Later I moved to atoms for different state handling. Whenever I needed to keep a track of a new thing - add an atom. It got a bit messy. Chaotic. So, keeping track of what my bot was doing was at times challenging.
Nevertheless, with enough checks and adding new state atoms, I got my bot to stay put when on top of an item. It managed to pick up items! I added use-action and it did manage to use items as well! It didn’t put any consideration if anyone else was on the stage, it just blasted mindlessly whenever it could. It was something! I did see it get a kill a couple of times as well, so mission success.
Then the map changed.
We had a set of different maps, with different terrain difficulties. My bot operated ok on a simple map. Any more complicated, the luck factor with bot and item spawns became significant as I didn’t have proper navigation yet. And the traps, those my bot chose to merrily ignore. The more difficult maps had higher item spawns, so that was a benefit. A slight benefit, but still. Needless to say, my bot was not super effective with a more difficult terrain with no real pathfinding. With the clock ticking, I decided to refocus my efforts, got some dessert and a drink, and just ran around trying to cause mayhem, until it was time to present our solutions.
At Oulu, we have for some reason become quite Clojure-oriented over time, with many of us have had exposure. There were a couple of Elixir solutions, which did quite well. At least one C# .Net 7 solution also performed very well, reaching the top score momentarily and then getting stuck in the harder map. Other used languages were Rust, TypeScript, Python and JS with Ramda-library. All in all, there was quite a wide spectrum of languages used, which is always nice to see at this kind of event. Few dipped their toes into unfamiliar territory experimenting with new tech, which fit the nature of the event perfectly. No completely off-the-rocker deep-end esoteric languages this time!
Giving presentations about our solutions
So, how did it go?
Well, my strategy of going hog wild, blasting everyone off the map wasn’t the best option. Weapons were single-use, did limited damage, and cost money, so the bot ran out of resources quickly. The item spawn rate was map dependent as well and in the easiest maps there just weren’t enough weapons - I implemented a feature that would drink potions, just to get them out of way for more weapons to spawn.
I realised too late, that picking items took multiple turns - what can you say, my bot got bored easily. It tried to pick a weapon up but just wandered off. I’m not sure why it did that, as my code should’ve just stayed put when on an item. Weird bug. Which leads to…
Debugging in the cloud is hard! There’s only the game state to rely on for debugging purposes and you get no feedback on whether your move was good until the game ticks. In hindsight, a local setup could’ve been helpful for debugging.
Embracing the chaos is fun but there are some drawbacks - mutations, spikes growing on your power armour, stuff like that. I was finding myself - unintentionally - exploring ways to write code imperatively in a functional language. This likely spawned from me messing in the REPL and lifting code into my codebase without caring about the code architecture overall. Global mutations are an easy way to keep state, yet messy and obviously abundant with heresy, send the inquisitors.
I never got around to implementing proper navigation, just relied on luck for “getting there” the straightest line possible. Which was a completely fine approach, unless walls were in the way. This caused too much I thought about A* but figured I’ll do it later. Later came too late. There was a ready-made A* implementation for Clojure I could’ve used.
Fine-tuning my bot I would implement some sort of asynchronous loop that could handle multiple bots at the same time, to make debugging and developing a bit smoother and easier. My current solution was synchronous and blocking, relying on a turn counter. I didn’t use the REPL to its fullest potential - actually developing in the app at runtime. Still, working with the REPL overall was great and it did remove some obstacles to testing my code.
Overall, the event was great fun! While my solution never really got “there” I did enjoy the same joy of programming I previously have gotten from game jams that I regrettably so rarely have time for now. Unburdened by the shackles of billable hours, just fast-paced hacking the code together, to succeed and to fail. The food, drinks, and sauna were great, and the atmosphere at the location was completely different from the office environment, cozy, homely, and warm. I would greatly encourage anyone to take part in these types of events, as it is a great detox from all the serious work. It’s also a great way to try out new stuff pressure-free! Bots of Black Friday-repository is available with the server and templates - fork and try it out!
There’s been a rumour that the next event of the kind would be happening later this year, so why not join us!
In a previous post I shared an HTTP file server that can hold a request for a file until the file changes. Besides live reloading this blog as I write posts, I've also used it for writing Tampermonkey scripts.
Tampermonkey is a browser extension that injects custom JavaScript into webpages. Rather than use Tampermonkey's built-in code editor, we can load our script from the file system, though we'd need to grant permission to read local files and I prefer to serve files from a particular directory through a local server. Once serving the script file, we can load it by adding a header comment to the Tampermonkey script:
// @require http://localhost:8000/script.jsWhile the technique that follows works with plain JavaScript using the built-in eval function, I chose to use Scittle to run Clojure scripts; here's a Tampermonkey header comment that loads a CDN version of Scittle:
// @require https://cdn.jsdelivr.net/npm/scittle@0.5.14/dist/scittle.jsNow, in script.js, we can write:
scittle.core.eval_string('(js/alert "Hello, world")')and when we load a page with the script injected, it pops up the expected dialog.
Besides the trouble of embedding a string of Clojure code within JavaScript, I've found that Tampermonkey requires some fiddling to clear its cache of external scripts. To avoid that, we can serve our Clojure code from the file system, putting it in the same directory as script.js and updating script.js to load the Clojure file and evaluate it:
fetch('http://localhost:8081/tools.cljs')
.then(response => response.text())
.then(script => {
scittle.core.eval_string(script));
console.log('Loaded')
};Now the script loads anew every time a webpage is loaded.
To add live reloading and evaluate script changes without having to refresh the page, serve the files from a server that holds requests until the file updates (such as the one in the previous post). Then we can add a simple reload loop by fetching the file accordingly (in the case of my server, by adding ?on-update to the URL), and when it returns fetching it again:
const liveReload = () => {
fetch('http://localhost:8081/tools.cljs?on-update')
.then(response => response.text())
.then(() => {
scittle.core.eval_string(script);
console.log('Reloaded');
})
.finally(liveReload);
liveReload();Now when changes to the Clojure script file are saved, the Tampermonkey script gets the new code, evaluates it, and applies the changes without requiring the page to be refreshed.
To add UI, we can bring in React and Reagent (CDN paths are given in the Scittle docs) by including them as Tampermonkey header comments.
My full scaffolding script, with some error handling to quit reloading when the server shuts down, looks like this:
const basePath = 'http://localhost:8081/tools.cljs';
fetch(basePath)
.then(response => response.text())
.then(script => {
scittle.core.eval_string(script);
console.log('Loaded');
})
let failures = 0;
const liveReload = () => {
fetch(`${basePath}?on-update`)
.then(response => response.text())
.then(script => {
failures = 0;
scittle.core.eval_string(script);
console.log('Reloaded');
})
.catch(error => {
console.error(error);
failures++;
})
.finally(() => {
if (failures < 20) liveReload();
})
};
liveReload();With this setup I wrote about 500 lines of Clojure to add a toolbar of shortcut actions to a complex website, and the process was quite smooth, especially being able to leverage Clojure's defonce to preserve UI state between reloads. If I'd had to reload the entire page every time I wanted to test a new version of the script, iterating on the code would have taken much longer.