Clojure Deref (Mar 15, 2024)

Welcome to the Clojure Deref! This is a weekly link/news roundup for the Clojure ecosystem (feed: RSS). Thanks to Anton Fonarev for link aggregation.

Blogs, articles, and projects

Libraries and Tools

New releases and tools this week:

  • pretty 2.3 - Library for helping print things prettily, in Clojure - ANSI fonts, formatted exceptions

  • ring 1.12.0 - Clojure HTTP server abstraction

  • hirundo 0.1.33 - Helidon 4.x RING adapter - using loom/java21+

  • clojure-lsp-intellij 2.1.0 - Intellij Plugin for Clojure & ClojureScript development via Language Server (LSP) made in Clojure

  • datalevin 0.9.3 - A simple, fast and versatile Datalog database

  • squint 0.7.96 - Light-weight ClojureScript dialect

  • Clojure-Sublimed 3.7.0 - Clojure support for Sublime Text 4

  • pg2 0.1.5 - A fast PostgreSQL driver for Clojure

  • calva 2.0.422 - Clojure & ClojureScript Interactive Programming for VS Code

  • timbre-json-appender 0.2.12 - JSON appender for Timbre

  • trace 1.1 - Better REPL based debugging output

  • vl-gallery-edn - vega-lite gallery in EDN format

  • deps.clj 1.11.2.1446 - A faithful port of the clojure CLI bash script to Clojure

  • guardrails 1.2.4 - Efficient, hassle-free function call validation with a concise inline syntax for clojure.spec and Malli

  • metamorph 0.2.4 - A Clojure library designed to providing pipelining operations

  • cli 0.8.58 - Turn Clojure functions into CLIs!

  • metamorph.ml 0.7.3 - Machine learning functions for metamorph based on machine learning pipelines. Part of scicloj.ml

  • clj-kondo 2024.03.13 - Static analyzer and linter for Clojure code that sparks joy

  • clojure-lsp 2024.03.13-13.11.00 - Clojure & ClojureScript Language Server (LSP) implementation

  • clay 2-alpha82 - A tiny Clojure tool for dynamic workflow of data visualization and literate programming

  • bubble-config - An aero powered config with environments aimed at Babashka tasks

  • yamlscript 0.1.42 - Programming in YAML

  • shadow-cljs-ext - Loading the shadow-cljs UI in browser devtools

  • easyreagent - Easy React Components for Reagent

Permalink

PG2 release 0.1.5: Migrations

PG2 version 0.1.5 ships its own migration engine through the pg2-migration package. Like Migratus or Ragtime, it allows to grow the database schema continuously, track changes and apply them with care.

Table of Content

Installation

;; lein
[com.github.igrishaev/pg2-migration "0.1.5]

;; deps
com.github.igrishaev/pg2-migration {:mvn/version "0.1.5"}

Concepts

Migrations are SQL files that are applied to the database in certain order. A migration has an id and a direction: next/up or prev/down. Usually it’s split on two files called <id>.up.sql and <id>.down.sql holding SQL commands. Say, the -up file creates a table with an index, and the -down one drops the index first, and then the table.

Migrations might have a slug: a short human-friendly text describing changes. For example, in a file called 002.create-users-table.up.sql, the slug is “Create users table”.

Naming

In PG2, the migration framework looks for files matching the following pattern:

<id>.<slug>.<direction>.sql

where:

  • id is a Long number, for example 12345 (a counter), or 20240311 (date precision), or 20240311235959 (date & time precision);

  • slug is an optional word or group of words joined with - or _, for example create-users-table-and-index or remove_some_view. When rendered, both - and _ are replaced with spaces, and the phrase is capitalized.

  • direction is either prev/down or next/up. Internally, down and up are transformed to prev and next because these two have the same amount of characters and files look better.

Examples:

  • 001-create-users.next.sql
  • 012-next-only-migration.up.sql
  • 153-add-some-table.next.sql

Above, the leading zeroes in ids are used for better alignment only. Infernally they are transferred into 1, 12 and 153 Long numbers. Thus, 001, 01 and 1 become the same id 1 after parsing.

Each id has at most two directions: prev/down and next/up. On bootstrap, the engine checks it to prevent weird behaviour. The table below shows there are two rows which, after parsing, have the same (id, direction) pair. The bootstrap step will end up with an exception saying which files duplicate each other.

Filename Parsed
001-some-trivial-slug.next.sql (1, next)
001-some-simple-slug.next.sql (1, next)

A migration might have only one direction, e.g. next/up or prev/down file only.

When parsing, the registry is ignored meaning that both 001-Create-Users.NEXT.sql and 001-CREATE-USERS.next.SQL files produce the same map.

SQL

The files hold SQL expressions to be evaluated by the engine. Here is the content of the 001-create-users.next.sql file:

create table IF NOT EXISTS test_users (
  id serial primary key,
  name text not null
);

BEGIN;

insert into test_users (name) values ('Ivan');
insert into test_users (name) values ('Huan');
insert into test_users (name) values ('Juan');

COMMIT;

Pay attention to the following points.

  • A single file might have as many SQL expressions as you want. There is no need to separate them with magic comments like --;; as Migratus requires. The whole file is executed in a single query. Use the standard semicolon at the end of each expression.

  • There is no a hidden transaction management. Transactions are up to you: they are explicit! Above, we wrap tree INSERT queries into a single transaction. You can use save-points, rollbacks, or whatever you want. Note that not all expressions can be in a transaction. Say, the CREATE TABLE one cannot and thus is out from the transaction scope.

For granular transaction control, split your complex changes on two or three files named like this:

# direct parts
001-huge-update-step-1.next.sql
002-huge-update-step-2.next.sql
003-huge-update-step-3.next.sql

# backward counterparts
003-huge-update-step-3.prev.sql
002-huge-update-step-2.prev.sql
001-huge-update-step-1.prev.sql

No Code-Driven Migrations

At the moment, neither .edn nor .clj migrations are supported. This is by design because personally I’m highly against mixing SQL and Clojure. Every time I see an EDN transaction, I get angry. Mixing these two for database management is the worst idea one can come up with. If you’re thinking about migrating a database with Clojure, please close you laptop and have a walk to the nearest park.

Migration Resources

Migration files are stored in project resources. The default search path is migrations. Thus, their physical location is resources/migrations. The engine scans the migrations resource for children files. Files from nested directories are also taken into account. The engine supports Jar resources when running the code from an uberjar.

The resource path can be overridden with settings.

Migration Table

All the applied migrations are tracked in a database table called migrations by default. The engine saves the id and the slug or a migration applied as well as the current timestamp of the event. The timestamp field has a time zone. Here is the structure of the table:

CREATE TABLE IF NOT EXISTS migrations (
  id BIGINT PRIMARY KEY,
  slug TEXT,
  created_at timestamp with time zone not null default current_timestamp
)

Every time you apply a migration, a new record is inserted into the table. On rollback, a corresponding migration is deleted.

You can override the name of the table in settings (see below).

CLI Interface

The migration engine is controlled with both API and CLI interface. Let’s review CLI first.

The pg.migration.cli namespaces acts like the main entry point. It accepts general options, a command, and command-specific options:

<global options> <command> <command options>

General options are:

-c, --config CONNFIG     migration.config.edn      Path to the .edn config file
-p, --port PORT          5432                      Port number
-h, --host HOST          localhost                 Host name
-u, --user USER          The current USER env var  User
-w, --password PASSWORD  <empty string>            Password
-d, --database DATABASE  The current USER env var  Database
    --table TABLE        :migrations               Migrations table
    --path PATH          migrations                Migrations path

Most of the options have default values. Both user and database names come from the USER environment variable. The password is an empty string by default. For local trusted connections, the password might not be required.

The list of the commands:

Name Meaning
create Create a pair of blank up & down migration files
help Print a help message
list Show all the migrations and their status (applied or not)
migrate Migrate forward (everything, next only, or up to a certain ID)
rollback Rollback (the current one, everything, or down to a certain ID)

Each command has its own sub-options which we will describe below.

Here is how you review the migrations:

<lein or deps preamble> \
    -h 127.0.0.1 \
    -p 10150 \
    -u test \
    -w test \
    -d test \
    --table migrations_test \
    --path migrations \
    list

|    ID | Applied? | Slug
| ----- | -------- | --------
|     1 | true     | create users
|     2 | false    | create profiles
|     3 | false    | next only migration
|     4 | false    | prev only migration
|     5 | false    | add some table

Every command has its own arguments and help message. For example, to review the create command, run:

lein with-profile +migrations run -m pg.migration.cli -c config.example.edn create --help

Syntax:
      --id ID             The id of the migration (auto-generated if not set)
      --slug SLUG         Optional slug (e.g. 'create-users-table')
      --help       false  Show help message

Config

Passing -u, -h, and other arguments all the time is inconvenient. The engine can read them at once from a config file. The default config location is migration.config.edn. Override the path to the config using the -c parameter:

<lein/deps> -c config.edn list

The config file has the following structure:

{:host "127.0.0.1"
 :port 10150
 :user "test"
 :password #env PG_PASSWORD
 :database "test"
 :migrations-table :migrations_test
 :migrations-path "migrations"}

The :migrations-table field must be a keyword because it takes place in a HoneySQL map.

The :migrations-path field is a string referencing a resource with migrations.

Pay attention to the #env tag. The engine uses custom readers when loading a config. The tag reads the actual value from an environment variable. Thus, the database password won’t be exposed to everyone. When the variable is not set, an exception is thrown.

Commands

Create

The create command makes a pair of two blank migration files. If not set, the id is generated automatically using the YYYYmmddHHMMSS pattern.

lein with-profile +migration run -m pg.migration.cli \
  -c config.example.edn \
  create

ls -l migrations

20240312074156.next.sql
20240312074156.prev.sql

You can also provide a custom id and a slug as well:

lein with-profile +migration run -m pg.migration.cli \
  -c config.example.edn \
  create \
  --id 100500 \
  --slug 'some huge changes in tables'

ll migrations

100500.some-huge-changes-in-tables.next.sql
100500.some-huge-changes-in-tables.prev.sql
20240312074156.next.sql
20240312074156.prev.sql

List

The list command renders all the migrations and their status: whether they are applied or not.

lein with-profile +migration run -m pg.migration.cli -c config.example.edn list

|    ID | Applied? | Slug
| ----- | -------- | --------
|     1 | true     | create users
|     2 | true     | create profiles
|     3 | true     | next only migration
|     4 | false    | prev only migration
|     5 | false    | add some table

Migrate

The migrate command applies migrations to the database. By default, all the pending migrations are processed. You can change this behaviour using these flags:

... migrate --help

Syntax:
      --all           Migrate all the pending migrations
      --one           Migrate next a single pending migration
      --to ID         Migrate next to certain migration
      --help   false  Show help message

With the --one flag set, only one next migration will be applied. If --to parameter is set, only migrations up to this given ID are processed. Examples:

... migrate           # all migrations
... migrate --all     # all migrations
... migrate --one     # next only
... migrate --to 123  # all that <= 123

Rollback

The rollback command reverses changes in the database and removes corresponding records from the migration table. By default, only the current migration is rolled back. Syntax:

... rollback --help

Syntax:
      --all           Rollback all the previous migrations
      --one           Rollback to the previous migration
      --to ID         Rollback to certain migration
      --help   false  Show help message

The --one argument is the default behaviour. When --all is passed, all the backward migrations are processed. To rollback to a certain migration, pass --to ID. Examples:

... rollback               # current only
... rollback --one         # current only
... rollback --to 20240515 # down to 20240515
... rollback --all         # down to the very beginning

Lein examples

Lein preamble looks usually something like this:

> lein run -m pg.migration.cli <ARGS>

The pg2-migration library must be in dependencies. Since migrations are managed aside from the main application, they’re put into a separate profile, for example:

:profiles
{:migrations
 {:main pg.migration.cli
  :resource-paths ["path/to/resources"]
  :dependencies
  [[com.github.igrishaev/pg2-core ...]]}}

Above, the migrations profile has the dependency and the :main attribute. Now run lein run with migration arguments:

> lein with-profile +migrations run -c migration.config.edn migrate --to 100500

Deps.edn examples

Here is an example of an alias in deps.edn that prints pending migrations:

{:aliases
 {:migrations-list
  {:extra-deps
   {com.github.igrishaev/pg2-migration {:mvn/version "..."}}
   :extra-paths
   ["test/resources"]
   :main-opts
   ["-m" "pg.migration.cli"
    "-h" "127.0.0.1"
    "-p" "10150"
    "-u" "test"
    "-w" "test"
    "-d" "test"
    "--table" "migrations_test"
    "--path" "migrations"
    "list"]}}}

Run it as follows:

> clj -M:migrations-list

You can shorten it by using the config file. Move all the parameters into the migration.config.edn file, and keep only a command with its sub-arguments in the :main-opts vector:

{:aliases
 {:migrations-migrate
  {:extra-deps
   {com.github.igrishaev/pg2-migration {:mvn/version "..."}}
   :extra-paths
   ["test/resources"]
   :main-opts ["migrate" "--all"]}}}

To migrate:

> clj -M:migrations-migrate

API Interface

There is a way to manage migrations through code. The pg.migration.core namespace provides basic functions to list, create, migrate, and rollback migrations.

To migrate, call one of the following functions: migrate-to, migrate-all, and migrate-one. All of them accept a config map:

(ns demo
  (:require
   [pg.migration.core :as mig]))

(def CONFIG
  {:host "127.0.0.1"
   :port 5432
   :user "test"
   :password "secret"
   :database "test"
   :migrations-table :test_migrations
   :migrations-path "migrations"})

;; migrate all pinding migrations
(mig/migrate-all CONFIG)

;; migrate only one next migration
(mig/migrate-one CONFIG)

;; migrate to a certain migration
(mig/migrate-to CONFIG 20240313)

The same applies to rollback:

;; rollback all previously applied migrations
(mig/rollback-all CONFIG)

;; rollback the current migration
(mig/migrate-one CONFIG)

;; rollback to the given migration
(mig/rollback-to CONFIG 20230228)

The read-disk-migrations function reads migrations from disk. It returns a sorted map without information about whether migrations have been applied:

(mig/read-disk-migrations "migrations")

{1
 {:id 1
  :slug "create users"
  :url-prev #object[java.net.URL "file:/.../migrations/001-create-users.prev.sql"]
  :url-next #object[java.net.URL "file:/.../migrations/001-create-users.next.sql"]}
 2
 {:id 2
  :slug "create profiles"
  :url-prev #object[java.net.URL "file:/.../migrations/foobar/002-create-profiles.prev.sql"]
  :url-next #object[java.net.URL "file:/.../migrations/foobar/002-create-profiles.next.sql"]}
 ...}

The make-scope function accepts a config map and returns a scope map. The scope map knows everything about the state of migrations, namely: which of them have been applied, what is the current migration, the table name, the resource path, and more.

The function create-migration-files creates and returns a pair of empty SQL files. By default, the id is generated from the current date & time, and the slug is missing:

(create-migration-files "migrations")

[#object[java.io.File "migrations/20240313120122.prev.sql"]
 #object[java.io.File "migrations/20240313120122.next.sql"]]

Pass id and slug in options if needed:

(create-migration-files "migrations" {:id 12345 :slug "Hello migration"})

[#object[java.io.File "migrations/12345.hello-migration.prev.sql"]
 #object[java.io.File "migrations/12345.hello-migration.next.sql"]]

Conflicts

On bootstrap, the engine checks migrations for conflicts. A conflict is a situation when a migration with less id has been applied before a migration with greater id. Usually it happens when two developers create migrations in parallel and merge them in a wrong order. For example:

  • the latest migration id is 20240312;
  • developer A makes a new branch and creates a migration 20240315;
  • the next day, developer B opens a new branch with a migration 20240316;
  • dev B merges the branch, now we have 20240312, then 20240316;
  • dev A merges the branch, and we have 20240312, 20240316, 20240315.

When you try to apply migration 20240315, the engine will check if 20240316 has already been applied. If yes, an exception pops up saying which migration cause the problem (in our case, these are 20240316 and 20240315). To recover from the conflict, rename 20240315 to 20240317.

In other words: this is a conflict:

id        applied?
20240312  true
20240315  false
20240316  true  ;; applied before 20240315

And this is a solution:

id        applied?
20240312  true
20240316  true
20240317  false ;; 20240315 renamed to 20240317

Permalink

Clojure Data Structures Tutorial

Clojure's collections are central to Clojure programming. While in most languages, you might find collections somewhere in the standard library, in Clojure, they are front and center.

Permalink

With YAMLScript, YAML Becomes a Proper Programming Language

Does YAML data need to be programmed? Many think so, including one of the creators of YAML itself.

Ingy döt Net, has started a project to bringing scripting capabilities to the data serialization language, in a project called YAMLScript.

Ingy döt Net is also working on another programming Language, Lingy.

SUSE Engineer Tina Müller dropped the news in her annual FOSDEM talk last month.

With YAMLScript, all valid YAML code is valid YAMLScript code. Plus, all YAMLScript function code, since it is itself in the YAML syntax, can be directly embedded into YAML files, or loaded in from other files.

The new programmable functionality will include “excellent interpolation features,”  such as merge, filter, and concatenate.  And people can create their own “generators” to manipulate data on the fly, döt Net promised.

The work is still in the early stages, with the latest release of the compiler being 0.1.41 released earlier this week.

But the idea is that it will solve “most programming things people want to do with YAML,” döt Net said, through the conference medium of Müller.

YAML Itself Is Not a Programming Language

YAML wasn’t originally designed for programming, even though many people wish it were.

A misunderstood language since its debut in 2004, YAML was the work of döt Net, Oren Ben-Kiki and Clark Evans.

As per its name, YAML (now short for “YAML Ain’t Markup Language”) actually is not a markup language (such as HTML for the web), but rather a data serialization language, a superset of JSON in fact.

YAML (current version 1.2.2) offers multiple ways to string data.

A flow sequence can written as a comma-separated list within square brackets:

- [  Bob Marley, Peter Tosh, Bunny Wailer]


Or! Data can be written as block collections using indentation for scope:

-Gene Clark
-Roger McGuinn
-David Crosby


Behind this simple format, however, lies a bewildering array of options and rules of how data can be further marked up, as well as how it could be interpreted.

But, to date, it has not been used as a programming language. By itself, YAML has no idea what a “variable” or “function” is.

${{ … }} Is Not YAML

But even frequent users can be forgiven for assuming YAML was a programming language.

In infrastructure management software, YAML data is often festooned with executional code overlaid by templating software. VMware‘s Saltstack embeds YAML in its Salt State Files this way, and people assume the coding is in YAML, Müller said, demonstrating with code snippets.

Red Hat‘s Ansible does something similar, embedding YAML as strings in its configuration cade, Müller observed.

Both of those automation tools use Jinja templates to add in the coding. Many people think the overlays are actually YAML, she said. They flood the YAML channel (#chat:yaml.io on matrix; libera.chat#yaml on irc) with queries about their broken code.

GitHub Actions Workflow also embeds YAML, within the center of ${{ … }} strings.

Again, not YAML.

VMware, Red Hat and GitHub aren’t alone in jumping ahead of YAML. The static limitations of YAML have been particularly acute with Kubernetes, where it is used as a configuration format.

In 2020, Amazon Web Services released cdk8s, a framework for capturing Kubernetes configuration data so it can be shared for different uses.

And Pulumi has built its core value proposition around the ability to manage configuration data, even configuration data for Kubernetes, not with clumsy YAML files, but with programming code itself.

YAML Is Now a (Functional) Programming Language

döt Net’s YAMLScript compiler/interpreter (“ys“) can be downloaded with curl and unwrapped at the command line. Or it can be imported as a library to any one of a number of languages, such as Python.

The ys interpreter ingests any valid YAML code as YAMLScript. The user only need to preface YAML functional code with: !yamlscript/v0.

Rosetta Code shows all the many ways you can say “Hello World” in YAMLScript:

!yamlscript/v0

say: "Hello, world!"

=>: (say "Hello, world!")

=>: say("Hello, world!")

say:
=>: "Hello, world!"

say: ("Hello, " + "world!")

say: ."Hello," "world!"

say "Hello,": "world!"

say "Hello," "world!":


Beneath the covers, YAMLScript compiles to Clojure code, run by the Small Clojure Interpreter (SCI).

Since Clojure is a functional programming language, one based on a Lisp syntax — “even though it commonly doesn’t look like one syntactically,” the docs note — YAML is then, technically, a functional programming language (instantly making it one of the mostly widely used functional languages after Microsoft Excel).

Tina Müller demonstrates YAMLScript at FOSDEM.

The post With YAMLScript, YAML Becomes a Proper Programming Language appeared first on The New Stack.

Permalink

January & February 2024 Long-Term Project Updates

We’re excited to post the first reports of the year from our 2024 long-term developers. There is a lot of great work to catch-up on, so dive in!

Bozhidar Batsov: CIDER, Clojure-mode, clojure ts-mode
Michiel Borkent: clj-kondo, babashka, SCI, squint, nbb, CLI, and more
Toby Crawley: clojars
Thomas Heller: shadow-cljs
Kira McLean: Scicloj Libraries
Nikita Prokopov: Humble UI, Clj-reload, Datascript, and more
Tommi Reiman: Malli
Peter Taoussanis: Telemere, Tempel, and more

Bozhidar Batsov

Another busy couple of months with quite a few achievements, both big and small. This time around I even managed to write a few articles about the bigger achievements! Here are the highlights for this period:

On top of this I wrote an article about the positive impact that community-contributed funding has had on CIDER and friends. Thanks to everyone for their continued support!

P.S. Some of you might be interested to hear that in this period I’ve also become of Flycheck’s maintainers. I’ve spent quite a bit of time working on Flycheck, and I see it very connected to the rest of my projects (e.g. there’s flycheck-elgot and flycheck-clj-kondo which many Clojure programmers user)


Michiel Borkent

In this post I’ll give updates about open source I worked on during January and February 2024. To see previous OSS updates, go here.

Sponsors

I’d like to thank all the sponsors and contributors that make this work possible. Like you can read on Bozhidar Batsov’s blog these aren’t the easiest times for Open Source sponsored software. I have no reason to complain, but I did see a similar drop in sponsoring in the last year. I’m thankful for those who sponsored my projects in the past and even more for those who keep doing so! Without you, the below projects would not be as mature or wouldn’t exist or be maintained at all.

Current top tier sponsors:

Additional sponsor info: If you want to ensure that the projects I work on are sustainably maintained, you can sponsor this work in the following ways. Thank you!
Clojurists Together
Github Sponsors
The Babaska or Clj-kondo OpenCollective
Ko-fi
Patreon
If you’re used to sponsoring through some other means which isn’t listed above, please get in touch. On to the projects that I’ve been working on!

Updates

Here are updates about the projects/libraries I’ve worked on last two months.

  • clj-kondo: static analyzer and linter for Clojure code that sparks joy. Released 2024.02.12
    • #2276: New Clojure 1.12 array notation (String*) may occur outside of metadata
    • #2278: bigint in CLJS is a known symbol in extend-type
    • #2288: fix static method analysis and suppressing :java-static-field-call locally
    • #2293: fix false positive static field call for (Thread/interrupted)
    • #2093: publish multi-arch Docker images (including linux aarch64)
    • #2274: Support clojure 1.12 new type hint notations
    • #2260: calling static field as function should warn, e.g. (System/err)
    • #1917: detect string being called as function
    • #1923: Lint invalid fn name
    • #2256: enable assert in hooks
    • #2253: add support for datomic-type-extensions to datalog syntax checking
    • #2255: support :exclude-files in combination with linting from stdin + provided --filename argument
    • #2246: preserve metadata on symbol when going through :macroexpand hook
    • #2254: lint files in absence of config dir
    • #2251: support suppressing :unused-value using :config-in-call
    • #2266: suppress :not-a-function linter in reader tag
    • #2259: ns-map unmaps var defined prior in namespace
    • #2272: Report var usage in if/when condition as always truthy, e.g. (when #'some-var 1)
  • squint: CLJS syntax to JS compiler
    • #472: Use consistent alias
    • #474: fix JSX fragment
    • #475: don’t crash watcher on deleting file
    • Add simple-benchmark
    • #468: Keywords in JSX should render with hyphens
    • #466: Fix doseq expression with set! in function return position
    • #462: Add "exports" field to package.json
    • #460: escape &lt; and > in JSX strings
    • #458: don’t emit null in statement position
    • #455: don’t export non-public vars
    • Fix infix operator in return position
    • Allow playground to use JSX in non-REPL mode
    • Add transducer arity to all existing core functions
  • babashka: native, fast starting Clojure interpreter for scripting. Two releases in the past two months with the following changes:
    • #1660: add :deps-root as part of hash to avoid caching issue with deps.clj
    • #1632: fix (.readPassword (System/console)) by upgrading GraalVM to 21.0.2
    • #1661: follow symlink when reading adjacent bb.edn
    • #1665: read-string should use non-indexing reader for compatibilty with Clojure
    • Bump edamame to 1.4.24
    • Bump http-client to 0.4.16
    • Bump babashka.cli to 0.8.57
    • Uberjar task: support reader conditional in .cljc file
    • Support reader conditional in .cljc file when creating uberjar
    • Add more javax.net.ssl classes
    • #1675: add hash-unordered-coll
    • #1658: fix command line parsing for scripts that parse --version or version etc
    • Add clojure.reflect/reflect
    • Add java.util.ScheduledFuture, java.time.temporal.WeekFields
    • Support Runnable to be used without import
    • Allow catch to be used as var name
    • #1646: command-line-args are dropped when file exists with same name
    • #1645: Support for clojure.lang.LongRange
    • #1652: allow bb.edn to be empty
    • #1586: warn when config file doesn’t exist and --debug is enabled
    • #1410: better error message when exec fn doesn’t exist
    • Bump babashka.cli to 0.8.55 which contains subcommand improvements
    • Bump deps.clj to 1.11.1.1435
    • Bump babashka.fs to 0.5.20
    • Compatibility with plumbing.core
    • Compatibility with shadow.css by improving tools.reader compatibility
    • #1647: Allow capturing env vars at build time (only relevant for building bb)
  • process: Clojure library for shelling out / spawning sub-processes
    • #123: exec now converts :env and :extra-env keywords (@lread)
    • #140: accept java.nio.file.Path as :dir argument
    • #148: accept Path in :out, :err and :in
    • Support :out :bytes (@hansbugge)
  • babashka.json: babashka JSON library/adapter
    • Released version 0.1.6 which fixes :key-fn + read behavior for cheshire
  • tools-deps-native and tools.bbuild: use tools.deps directly from babashka
    • Upgraded the underlying tools.build version to the latest version used in tools.build (the very latest wasn’t compatible with tools.build!)
  • edamame: Configurable EDN/Clojure parser with location metadata
    • Support new ^[String] metadata notation which desugars into ^{:param-tags [String]}
    • Add :map and :set options to coerce map/set literals into customizable data structures, for example, an ordered collections to preserve key order.
  • nbb: Scripting in Clojure on Node.js using SCI
    • Add cljs.test/run-test macro
    • Add cljs.core/Atom
    • Add promesa promesify
  • http-client: babashka’s http-client
    • #45: query param values are double encoded
  • CLI: Turn Clojure functions into CLIs!
    • Fix #82: prefer alias over composite option
    • Add :opts to :error-fn input
    • Fix command line args for test runner --dirs, --only, etc
    • Fix --no-option (--no prefix) in combination with subcommands
    • Prioritize :exec-args over spec :defaults
    • dispatch improvements (@Sohalt, @borkdude):
      • The :cmds order of entries in the table doesn’t matter
      • Support parsing intermediate options: foo --opt1=never bar --opt2=always
  • SCI: Configurable Clojure/Script interpreter suitable for scripting and Clojure DSLs
    • Bump edamame
    • Add hash-ordered-coll
    • read-string should use non-indexing reader

Other projects

There are many other projects I’m involved with but that had little to no activity in the past month. Check out the Other Projects section (more details) of my blog here to see a full list.
Published: 2024-02-29
Tagged: clojure oss updates


Toby Crawley

January 2024

Commit Logs: clojars-web, infrastructure

I upgraded the PostgreSQL database from v12.1 to v15.5.

February 2024

Commit Logs: clojars-web

This month was a smattering of small changes/fixes:


Thomas Heller

Time was mostly spent on doing maintenance work and some bugfixes. As well as helping people out via the typical channels (e.g. Clojurians Slack).

Current shadow-cljs version: 2.27.5 Changelog

Notable Updates:

  • Tweaked watch recompile logic to also account for indirect var usages via macros. The usual recompile logic compiles a changed namespace, and then everything which directly required it. This is sometimes insufficient since macros can insert references to other namespaces, without the namespace using the macro having direct knowledge (or require) of that ns. Now all direct accessed vars (after analysis) are collected and used for this logic, making it much more accurate.
  • Added limited support for npm modules using import() in their code. This is still fairly limited in support, and does not perform actual code splitting as webpack might, but it made some npm modules usable again that would previously just crash.
  • Worked a lot on shadow-grove. Spent way too much time benchmarking, but I feel confident now that performance is quite good and potentially the fastest CLJS react-equivalent available. Also updated the playground, which I plan on using as the basis for documentation. Still a couple of things to work out, but overall in good shape.

Kira McLean

This is a summary of the work I did on open source projects in January and February 2024. It was published as a blog post on my website here.

Clojure Tidy Tuesdays

The main thing I spent my time working on over the past couple of months was a collection of tutorials and guides for working with data in Clojure. The R for Data Science online learning community publishes toy datasets every week for “Tidy Tuesdays” with a question to answer or example article to reproduce. I’ve been going through them in Clojure, and it’s proven a great tool for uncovering areas for future development in the Clojure data science ecosystem.

Other Work

The explorations with the Tidy Tuesday data have been revealing areas where I think we could benefit from more ergonomic ways to work with tablecloth datasets. I started two little projects each with a couple of little wrappers around existing functions to make them easier to use with tablecloth datasets. So far I’m calling them tcstats (for statistical operations on datasets) and tcutils (with miscellaneous dataset manipulation tools that aren’t built-in to tablecloth directly).

I am also still working on the Clojure Data Cookbook. I nudged it forward ever so slightly these last couple of months, and I plan to finish it despite the remaining holes in Clojure’s data science stack. I would love to also fill these in eventually, but the Cookbook will be a living document that can easily evolve and be updated as new tools and libraries are developed.

Lastly, one of the main missing pieces I’m discovering we really need to work on in Clojure’s data science ecosystem is a robust yet flexible graphics library. There are a few great solutions that already exist, but they take different approaches to graphing that can make them a bit clumsy to work with when it comes time to build more complex visualisations. My dream is to implement a proper grammar of graphics in Clojure, giving the Clojure data ecosystem a “profressional quality” graphics library, so to speak. Anyway.. there is still tons of work to do here so I’m grateful for the ongoing funding that will allow me to continue to focus a large amount of time on it for the foreseeable future.


Nikita Prokopov

Hello guys and gals, Nikitonsky here with some Winter 2024 updates.

New library! Clj-reload is a smarter way to reload Clojure code during development:
Main goal was to replace and improve over tools.namespace:

  • Do not reload everything on first repl/refresh
  • Allow users to register hooks on namespace unload
  • Support in-ns and standalone require and use
  • Not reload namespaces that were never loaded before
  • Keep defonce values/protocols/types between reload

And it worked! Humble UI and my website are already migrated to it, and it feels fantastic to being able to use defonce again, without creating a separate namespace just to hold that one value.

Filipe Silva is working on CIDER integration, so I hope we’ll see more adoption soon.

If you’ve been using tools.namespace, give clj-reload a try. You might like it better. If you haven’t, it’s time to reconsider your REPL workflow. We have some convincing points in the README.

Oh, and DO check out our new logo. I’m pretty happy with how it turned out.

Humble UI:

  • Implemented tricky lazy-loading system
  • Improved deftype+ stacktraces
  • Migrated to VDom:
    • gap
    • label
    • image
    • svg
    • padding
    • rect, rounded-rect
    • clip, clip-rrect
    • halign, valign, center
    • vscroll, vscrollbar
    • column, row
    • hoverable, clickable
    • button, toggle-button (new!)
    • slider
  • Migrated to clj-reload
  • Jakub Dundalek managed to run HumbleUI app from GraalVM native image

JWM:

  • macOS: fix fullscreen. Yes, I had to fix something that I didn’t touch for a year and it broke anyways.

Humble Deck:

  • Now supports multiple decks

humble-starter:

  • Simplest possible project to draw a window on a screen

DataScript:

  • Query: shortcircuit clause resolution when result is guaranteed to be empty #459 via @galdre

Clojure Sublimed:

  • Detect namespace from in-ns forms

Sublime Executor:

  • Support colored output (I can finally see colored diffs in failing tests)

Blogging:

Talks:

Onward and upward!



Tommi Reiman

Worked on a big new release of Malli and wrote a blog post about it.

Reitit should be now feature-complete for OpenAPI, will cut out release soon.

Helped users to use and adopt the libs.

0.14.0 (2024-01-16)

  • Better development-time tooling
    • malli.dev/start! captures all malli-thrown exceptions, see [README](README.md#development-mode for details
    • does not log individual re-instrumentation of function vars
    • BREAKING: changes in malli.dev.virhe and malli.pretty extension apis, wee #980 for details
  • New m/deref-recursive to recursive deref all schemas (not :refs)
  • FIX: Malli generates incorrect clj-kondo spec for :fn schemas #836 via #987
  • Support for Var references #985, see [guide](README.md#var-registry for details.
  • BREAKING: m/coerce and m/coercer throw ::m/coercion instead of ::m/invalid-input
  • New Guide for Reusable Schemas
  • Less printing of Var instumentation
  • BREAKING: qualified symbols are valid reference types #984
  • Fixing mt/strip-extra-keys-transformer for recursive map encoding #963
  • Support passing custom :type in into-schema opt for :map and :map-of #968
  • mu/path->in works with :orn, :catn and :altn.

Something else

Teppo the Dog enjoying the Sun at Näsijärvi.

spring-fi

Peter Taoussanis

Open source update

A big thanks to Clojurists Together, Nubank, lambdaschmiede, and other sponsors of my open source work!

It’s been a productive start to the year! Have been focused almost entirely on open source. Output has included:

7x library releases

This includes http-kit v2.8.0-RC1 (tons of new stuff in here!), Tempel v1.0.0-RC1, Nippy v3.4.0-RC2, and Timbre v6.5.0.

See here for the full list, and also here for a new GitHub-hosted roadmap of my major upcoming open source work.

New Tempel explainer video

I recorded a short video to explain the new/upcoming Tempel data security framework.

This hopefully helps makes it clearer where Tempel can be useful.

The ultimate goal is to try make it feasible for more Clojure devs to incorporate data security in their apps, and/or at least get more Clojure devs thinking about ways to protect their users’ data.

It can be easier than you expect, and I detail one example pattern in the video.

BTW please let me know if there’s interest in me doing more of these kinds of videos in future.

London Clojurians Talk

On 20 Feb I gave an online talk at the London Clojurians Group. The talk’s now on YouTube.

Had a good time, was fun talking with folks there. A big thanks to Bruno Bonacci for organizing and hosting!

The talk was non-technical, titled:

Some controversial truths: challenging some commonly-held ideas about Clojure, software engineering, and startups; and sharing the 1 secret that will solve all your problems.

Part of the talk’s intention was to discuss some of the trade-offs that Clojure users/businesses should be aware of, but it looks like the Q&A after was unfortunately not recorded.

Please ping if there’s interest in me posting a write-up to summarize some of the key points discussed.

Interview with Daniel Compton

Had a really nice chat with Daniel Compton about my open source work and other projects. (Thanks Daniel!)

To avoid possible confusion re: discussed dates/timelines, please note that this was recorded at the end of last year (2023).

Lots of work On Telemere

Have been putting in a lot of work on the upcoming Telemere structured telemetry library for Clojure/Script.

Along with Tempel, this’ll be my first all-new Clojure library in 7+ years.

Very happy with how it’s coming along so far, and looking forward to sharing more closer to release (hopefully this April).

In some ways Telemere represents the culmination and refinement of many years of ideas from some of my other libraries - notably Timbre and Tufte.

I think the result is going to be really nice - and something possible only in Clojure.

Upcoming work

All major upcoming work is now documented live here. In addition to all the usual maintenance and support, my biggest objectives for this year are definitely Telemere and a majorCarmine update.

- Peter Taoussanis

Permalink

The Design of Biff

I gave a presentation for the London Clojurians meetup this week. It starts out with a standard "what is Biff" overview, then in the second half I discuss my thought process/motivations for a few of Biff's features.

Permalink

Clojure Deref (Mar 8, 2024)

Welcome to the Clojure Deref! This is a weekly link/news roundup for the Clojure ecosystem (feed: RSS). Thanks to Anton Fonarev for link aggregation.

From the core

Last week CVE-2024-22871 detailed in GHSA-vr64-r9qj-h27f was filed. I’ve added the relevant info to the latter link so read there for the details. The important takeaway here is that you should never read serialized objects from an untrusted source (usually this is via ObjectInputStream.readObject() if you’re grepping your source code). There are an open-ended number of ways an attacker can craft malicious objects using a variety of languages and libraries.

Today we released Clojure 1.11.2 and 1.12.0-alpha9 with a fix for the specific problem identified in that service advisory - the change is small (converting some infinite loop cases to throw instead). Along with this, there is a new version of the Clojure CLI that defaults to Clojure 1.11.2, and new versions of some contrib libraries.

We have been working hard on the last remaining feature pieces for Clojure 1.12. Due to some useful feedback on prior alphas, we found an issue with the new class array syntax (e.g. String*) - while this is an invalid class name in Java, it (and literally any name) are valid class names in the JVM, and in particular, Clojure deftype is an example way that you can create such a class. We considered a wide range of options but we have settled on an alternate syntax, with array classes ending in ::N (where N is the dimension). :: is not valid in Java class names, and not (until now) valid in Clojure symbol names, so there is no chance of conflict with this syntax.

With method values, we heard all of your feedback regarding reflective support for uniform method symbols. I think we’ve found some interesting new angles on this that will allow us to support reflection in many cases, which I will save for another post. For those interested specifically in avoiding reflection, we will address that more directly in a future release. And finally, we are zero-ing in final work for implicit function conversion. All of this work is past the planning and deep into implementation, and it should start to land soon.

Finally, this is not really Clojure core work but a result of some LLM explorations at Nubank, but I’ve submitted a patch to the MultiPL-E LLM coding benchmark to add support for Clojure. We’re starting to use this ourselves in evaluating Clojure capabilities of different models, and hopefully others can get use out of it as well.

Blogs, articles, and projects

Libraries and Tools

New releases and tools this week:

  • fireworks 0.1.1 - Fireworks is a color printer for Clojure, ClojureScript, and Babashka

  • futurama 1.0.2 - Futurama is a Clojure library for more deeply integrating async abstractions with core.async

  • clara-rules 1.4.0-SNAPSHOT - Performance focused forward-chaining rules in Clojure

  • clojure-lsp 2024.03.01-11.37.51 - Clojure & ClojureScript Language Server (LSP) implementation

  • llama.clj 0.8.2 - Run LLMs locally. A clojure wrapper for llama.cpp

  • clojure 2024-03-03 - Practicalli Clojure REPL Driven Development

  • deps-try 0.11.1 - Try out Clojure libraries via rebel-readline

  • honeysql 2.6.1126 - Turn Clojure data structures into SQL

  • template-processor - Clojure library which provides StringTemplate-like functionality

  • dinodoc 0.2.0 - Docusaurus-powered documentation system for Clojure and other languages

  • Clojure-Sublimed 3.6.0 - Clojure support for Sublime Text 4

  • nbb 1.2.187 - Scripting in Clojure on Node.js using SCI

  • overarch 0.12.0 - A data driven description of software architecture based on UML and the C4 model

  • clj-kondo 2024.03.05 - Static analyzer and linter for Clojure code that sparks joy

Permalink

Top Paying Programming Technologies 2024

Delve into the realm of high-paying programming technologies as we explore the top earners in the field. Consider whether acquiring expertise in these technologies aligns with your goals for advancing your development skills.

1. ZIG - $103,611

Frustrated with C's complexity but still crave its performance? Zig might be the language for you. Designed as a modern C successor, Zig offers a simpler and safer experience while maintaining C's low-level control and speed. It ditches error-prone features like macros and preprocessors, and includes safeguards against memory leaks and buffer overflows that plague C development. Zig integrates seamlessly with existing C code, so you can leverage your C libraries or even write Zig code usable from C programs. Unlike some languages, Zig doesn't force external dependencies. Its built-in toolchain handles building and managing your project, streamlining development. Performance-focused features like manual memory management and low-level control give you fine-tuned efficiency in your programs. To top it off, Zig boasts powerful compile-time features for generic and efficient code. Overall, Zig provides a compelling alternative for system programming, embedded development, or any task where C's performance is desired with a more modern and safer approach.

2. Erlang - $99,492

Erlang is a unique language designed for building large, reliable systems that need to be constantly available. Imagine massive telecommunication networks or always-on messaging apps – that's Erlang's sweet spot. It excels at handling concurrency, meaning it can manage many tasks simultaneously without breaking a sweat. Erlang's secret weapon is its lightweight processes that can be easily restarted if they encounter errors. This "let it crash" philosophy ensures the overall system keeps running smoothly. Plus, Erlang is built for distribution, allowing you to spread your application across multiple machines for even greater scalability. If you're looking to build fault-tolerant, high-performance systems that can't afford downtime, Erlang's a powerful tool to consider.

3. F# - $99,311

F# is a versatile programming language that combines the best of multiple worlds. It blends functional programming with object-oriented features, offering a clean and concise way to write code. Unlike traditional languages where data can be constantly modified, F# emphasizes immutability, meaning data remains unchanged after creation. This leads to fewer errors and easier reasoning about how your code behaves. F# shines in areas like data manipulation, asynchronous programming (handling multiple tasks at once), and building robust systems. Its focus on clarity and correctness makes it a favorite for complex financial applications, scientific computing, and functional programming enthusiasts.

4. RUBY - $98,522

Ruby is a programmer-friendly language known for its simplicity and readability. Unlike some languages that require mountains of code, Ruby lets you achieve complex tasks with fewer lines. This focus on developer productivity makes it a great choice for beginners and experienced programmers alike. Ruby is also quite versatile. It tackles web development with frameworks like Rails, but its uses extend to data analysis, automation tools, and even creating desktop applications. If you're looking for a language that's easy to learn, fun to use, and powerful enough for various projects, Ruby is a strong contender.

5. Clojure - $96,381

Clojure offers a unique blend of power and expressiveness for programmers. It's a dialect of Lisp, which means code can be treated as data, allowing for flexible manipulation. But Clojure's core is functional programming, where you avoid modifying data and instead create new versions. This leads to cleaner, less error-prone code. Despite its functional heart, Clojure allows for mutations when needed. It also integrates seamlessly with Java, making it a great choice for working with existing Java libraries and tools. So, if you value clean, expressive code, functional programming benefits, and the ability to leverage the Java ecosystem, Clojure might be worth exploring.

6. Elixir - $96,381

Elixir is a language designed to build robust and scalable applications. It combines the strengths of two worlds: Erlang, known for its fault tolerance and concurrency, and Ruby, famous for its developer friendliness. Elixir inherits Erlang's ability to handle many tasks simultaneously and self-heal from errors. This makes it ideal for constantly running systems like chat applications or e-commerce platforms. On the other hand, Elixir borrows Ruby's clean syntax and focus on programmer productivity. This makes it easier to learn and write than Erlang, while still offering the power and reliability needed for large-scale projects. So, if you're looking for a language that balances ease of development with the ability to build fault-tolerant, scalable applications, Elixir is a great choice.

7. Lisp - $96,381

Lisp (List Processor) is a unique and influential programming language that stands out from the crowd. Unlike most languages that use different structures for code and data, Lisp relies on "lists" for everything. This makes Lisp incredibly flexible, as you can manipulate programs like data and even modify them on the fly. This very feature made Lisp a forerunner in Artificial Intelligence research, where symbolic manipulation and easy code changes were crucial.

8. Scala - $96,381

Scala is a powerful programming language designed to address limitations in Java. It combines object-oriented and functional programming paradigms, offering flexibility and expressiveness. Scala is known for its conciseness, allowing you to write less code to achieve more.

One of its strengths is safety. Scala is statically typed, which means the compiler catches errors before you even run the program. Additionally, features like immutable data structures help prevent bugs.

Scala integrates seamlessly with Java, so you can leverage existing Java libraries and even mix Scala and Java code within the same project.

If you're looking for a language that's powerful, concise, safe, and plays well with Java, Scala is a great option for web development, data analysis, or building scalable applications.

9. Perl - $94,540

Perl is a veteran scripting language known for its versatility and text manipulation prowess. It's particularly adept at handling complex text data, making it a favorite for tasks like web scraping, data munging, and system administration. Perl's claim to fame is its powerful regular expressions, which are like supercharged search patterns for sifting through text.

However, Perl's flexibility can sometimes be a double-edged sword. Its code can be concise but also cryptic for beginners due to its shorthand syntax. Despite its age, Perl still sees use in web development, bioinformatics, and system automation, but for new projects, languages like Python might be more approachable due to their clearer syntax.

10. Go - $92,760

Go, also sometimes called Golang, is a relatively young language created at Google. It prioritizes simplicity, reliability, and performance. Go is designed to be easy to learn and use, with a clean and concise syntax. This focus on readability makes code easier to maintain and collaborate on. Despite its simplicity, Go is a powerful language well-suited for building backend systems, web applications, and even command-line tools. One of its strengths is concurrency, which allows your program to handle many tasks simultaneously without a struggle. Go also excels at performance, offering fast execution speeds thanks to its compiled nature. If you're looking for a language that balances ease of use, performance, and concurrency for building modern applications, Go is a great choice.

11. OCaml - $91,026

OCaml (pronounced oh-CAML) is a powerful language that blends multiple programming paradigms. It combines functional programming, with its emphasis on immutability and pure functions, with object-oriented features for structured code organization. This fusion offers a balance between flexibility and reliability. OCaml is known for its strong static typing system, which catches errors early in the development process, leading to more robust and secure applications.

Performance is another perk. OCaml compiles to efficient machine code, making it suitable for tasks where speed matters, like system programming, financial applications, and scientific computing. Beyond its technical strengths, OCaml boasts a mature ecosystem with a wide range of libraries and tools, making development smoother. If you're looking for a versatile language that prioritizes safety, performance, and offers a blend of functional and object-oriented features, OCaml is definitely worth considering.

12. Objective-C - $90,000

Objective-C, once a dominant force in mobile development, is a powerful language specifically designed for Apple's operating systems, like iOS and macOS. It bridges the gap between high-level languages and the underlying hardware, giving programmers more control over system resources. This fine-grained control was essential in the early days of mobile development, allowing developers to optimize performance and squeeze the most out of limited device resources.

However, Objective-C's complexity can be a drawback. Its syntax can be verbose and error-prone, making code challenging to maintain and debug. Additionally, its reliance on manual memory management can lead to memory leaks if not handled carefully.

With the rise of Swift, Apple's newer language, Objective-C's popularity has declined. Still, some legacy applications and parts of Apple's own frameworks are written in Objective-C. So, if you're working on an existing iOS or macOS project with a large Objective-C codebase, or need to interact with Apple's older frameworks, understanding Objective-C remains valuable. But for new projects, Swift is generally the preferred choice due to its simpler syntax, improved safety features, and better integration with modern development practices.

13. Rust - $87,012

Rust is a powerful language gaining traction for its focus on speed, memory safety, and concurrency. Unlike some languages that prioritize ease of use over control, Rust empowers you to write efficient code while ensuring it's free of memory errors like crashes or data corruption. This makes Rust ideal for building reliable systems that need to perform well, especially in areas like embedded systems, operating systems, and high-performance web services.

While Rust's focus on safety leads to more complex syntax compared to some languages, it also offers features like automatic memory management, preventing memory leaks that plague other languages. This can save development time and improve the overall stability of your application. The Rust community is known for its helpfulness and extensive documentation, making it easier to learn and navigate the language's intricacies. So, if you're looking for a language that prioritizes performance, memory safety, and building robust applications, Rust is a compelling choice, especially for projects where control and efficiency are paramount.

14. Swift - $86,271

Swift is a modern programming language developed by Apple specifically for building apps on their devices. It offers a compelling alternative to the older Objective-C, aiming for simplicity, safety, and performance. Swift boasts a clean and readable syntax, making code easier to write, understand, and maintain compared to Objective-C's verbose style.

One of its key strengths is safety. Swift incorporates features to prevent common programming errors like memory leaks and crashes that can plague Objective-C development. This focus on safety reduces development headaches and leads to more reliable applications.

Despite its emphasis on safety, Swift doesn't sacrifice performance. It compiles to efficient code, making it suitable for creating fast and responsive apps for iPhones, iPads, Macs, and other Apple devices. Additionally, Swift plays well with existing Objective-C code, allowing you to integrate older libraries or even mix Swift and Objective-C within the same project if needed.

So, if you're looking to develop modern applications for Apple's ecosystem, Swift is the language of choice. It offers a great balance between readability, safety, and performance, making it easier to write robust and efficient apps for Apple devices.

15. Groovy - $86,271

Groovy is a dynamic language that rides on the coattails of Java, offering a familiar base with a more concise and readable twist. Imagine Java with less verbosity and a sprinkle of sugar. This makes Groovy a great choice for developers already familiar with Java who want to improve their productivity and write more expressive code.

Groovy seamlessly integrates with Java. You can use Groovy code within existing Java projects and leverage existing Java libraries. This smooth integration makes it a popular choice for extending Java applications or building microservices.

Beyond its Java connection, Groovy shines in web development. It works well with frameworks like Grails, which streamline the process of building web applications. Groovy also boasts powerful metaprogramming features, allowing you to write code that manipulates itself at runtime, leading to more dynamic and flexible applications.

So, if you're looking for a language that's familiar to Java developers, offers a more concise syntax, and integrates seamlessly with the Java ecosystem, Groovy is a strong contender. It's particularly well-suited for web development, scripting tasks within Java projects, and building microservices.

16. Bash/Shell (all shells) - $85,672

Bash, along with sh, zsh, and ksh, are all flavors of what's called a shell. Imagine them as text-based interpreters that act as the go-between for you and the operating system's core. Programmers use shells for a number of reasons. First, shells provide direct access to the operating system's kernel, the central program that controls everything. This granular control lets them manage files, run programs, and automate tasks with a high degree of precision. Shells are also great for writing scripts, which are basically lists of commands strung together. These scripts can automate repetitive tasks, saving programmers tons of time. On top of that, shells are highly customizable. You can change how they look, set up shortcuts for frequently used commands, and even create your own functions to expand their abilities. Plus, shells come standard on most Unix-based systems, including macOS and Linux, making them a universal tool for programmers regardless of their specific environment. Finally, shells are lightweight compared to graphical user interfaces (GUIs). This makes them efficient to use, even on machines with lower resources. While GUIs might be easier to learn at first, shells offer a powerful alternative for programmers thanks to their direct control, scripting abilities, and universal presence in the Unix world.

17. Haskell - $85,672

Haskell is a unique breed of programming language known for its focus on functional programming. Unlike traditional languages where data can be constantly modified, Haskell emphasizes immutability – data remains unchanged after it's created. This leads to fewer errors and makes reasoning about how your code behaves much easier.

Haskell shines in areas like data manipulation, handling multiple tasks simultaneously (concurrency), and building robust systems. Its focus on clarity and correctness makes it a favorite for complex financial applications, scientific computing, and for programmers who enjoy the elegance of functional programming.

18. Apex - $81,552

Apex is a specialized language designed to work exclusively with the Salesforce platform. It combines elements of Java and Salesforce's Visualforce templating language, making it a powerful tool for customizing and extending Salesforce's capabilities. Because Apex is native to Salesforce, it integrates seamlessly with the platform's features and objects. This smooth integration allows developers to create applications that leverage the full potential of Salesforce, without worrying about compatibility issues.

Another advantage of Apex is its object-oriented approach. By utilizing object-oriented programming principles, Apex allows developers to model their code based on real-world concepts. This structure makes complex Salesforce applications easier to understand, maintain, and manage over time.

Furthermore, Apex excels at manipulating Salesforce data and automating tasks. Developers can build custom logic to trigger actions based on events, update records automatically, or integrate Salesforce with external systems. Apex's power extends beyond data manipulation. It can be used to create custom buttons, triggers, and Visualforce pages, effectively extending the Salesforce user interface. This allows developers to tailor the platform to specific business needs, improving user experience and efficiency.

In essence, Apex empowers developers to build custom applications and significantly extend the functionality of the Salesforce platform. However, it's important to note that Apex's usefulness is primarily within the Salesforce ecosystem.

19. PowerShell - $81,311

PowerShell is a multi-talented tool that blends the power of a command-line interface (CLI) with the flexibility of a scripting language. Designed primarily for Windows environments, it empowers administrators and developers to automate tasks, configure systems, and manage resources efficiently.

PowerShell's strength as a programming language lies in its ability to automate repetitive tasks on Windows systems. Imagine writing scripts to handle user accounts, software installation, or system configuration settings – PowerShell lets you do this and more, saving you significant time and effort. Unlike traditional scripting languages, PowerShell operates on the concept of objects. Everything from user accounts to network settings is treated as an object, making your scripts more organized, reusable, and easier to maintain, especially for complex tasks.

PowerShell comes with a rich library of built-in commands (cmdlets) designed to interact with various aspects of Windows. You can further extend its functionality by creating custom cmdlets or leveraging existing modules from the active PowerShell community. Additionally, PowerShell integrates seamlessly with other Microsoft technologies like Active Directory and Azure, making it a natural choice for automating tasks within these environments. While its roots are in Windows, PowerShell's functionality extends to Linux and macOS as well, adding to its versatility for managing multi-platform environments.

In essence, PowerShell's automation capabilities, object-oriented approach, rich functionality, and integration with Microsoft tools make it a compelling language for scripting and programming tasks, particularly for system administration and management, especially on Windows systems.

20. SAS - $81,000

SAS (originally standing for Statistical Analysis System) isn't just a programming language, it's a software suite specifically designed for data enthusiasts. It empowers you to not only crunch numbers but also to manipulate, analyze, and visualize your data in powerful ways.

Imagine going beyond basic calculations like means and medians. SAS lets you perform a wide range of statistical tests, from simple analyses to complex regressions and modeling techniques. But SAS doesn't stop there. It tackles data management too, efficiently importing data from various sources, cleaning and transforming it as needed, and handling even massive datasets with ease.

If you're looking to gain insights from your data, SAS is a valuable tool. It can summarize your data into clear and concise tables and reports, and it can create informative visualizations like charts and graphs to bring your data to life. SAS even ventures into predictive analytics. By leveraging historical data, SAS can help you build models to forecast future trends or outcomes, making it a valuable asset in areas like marketing, finance, and risk management.

So, what makes SAS a popular choice for data analysis? Several factors contribute to its success. First, SAS offers a vast library of procedures and tools, providing a one-stop shop for various statistical and data manipulation needs. Second, SAS strikes a balance between power and ease of use. While it offers a user-friendly interface for basic tasks, it also provides a programming language for advanced users who crave customization and automation. Finally, SAS boasts industry-specific solutions and expertise in various domains like healthcare, finance, and government.

However, SAS can be a pricier option compared to some free, open-source alternatives. Additionally, the SAS programming language itself can have a steeper learning curve.

Overall, SAS is ideal for organizations that require a comprehensive and user-friendly suite for in-depth data analysis, business intelligence, and even venturing into predictive modeling. Its industry-specific solutions and established track record make it a trusted choice for many companies looking to unlock the power of their data.

21. Lua - $80,690

Lua is a lightweight and powerful scripting language designed to be embedded within other applications. Imagine a small, efficient engine that can be seamlessly integrated into existing software. This makes Lua ideal for extending the functionality of programs or adding scripting capabilities to game engines and web applications.

Despite its small size, Lua packs a punch. It's known for its speed and ease of use, with a clean and concise syntax that makes it beginner-friendly. This focus on simplicity makes Lua code easier to read, understand, and maintain. Furthermore, Lua boasts automatic memory management, freeing programmers from the burden of manual memory handling, which can lead to errors and crashes.

So, if you're looking for a language that's lightweight, fast, and easy to learn, Lua is a great option for embedding scripting capabilities into your applications. Its focus on speed and ease of use makes it a popular choice for game development, web applications, and even extending the functionality of existing software.

22. Nim - $80,000

Nim is a versatile programming language designed to be efficient, expressive, and elegant. It combines the strengths of established languages like Python and Ada, offering a clean syntax and strong static typing. This focus on readability and type safety makes Nim code easier to understand, maintain, and less prone to errors.

Nim shines in various areas. For system programming tasks that require close control over hardware resources, Nim's efficiency and ability to compile to native code make it a compelling choice. On the other hand, Nim's expressiveness and metaprogramming features allow developers to write concise and flexible code. This makes it well-suited for building web applications and even scripting tasks.

Another perk of Nim is its ability to compile to various targets, including C, C++, and JavaScript. This allows developers to leverage existing libraries written in these languages and seamlessly integrate Nim code into different environments.

Overall, Nim offers a compelling combination of readability, efficiency, and versatility. Whether you're building system-level applications, web services, or scripting tools, Nim is a language worth considering, especially if you value clean syntax, strong typing, and the ability to target multiple platforms.

23. Raku - $79,448

Raku (formerly Perl 6) is a versatile language offering a unique blend of features. It excels at text processing with built-in regular expressions, but also embraces object-oriented, functional, and metaprogramming paradigms. This flexibility allows developers to tackle various tasks, from web development and data analysis to system scripting and automation. Raku's focus on readability with a clean syntax and optional static typing makes code easier to understand and maintain, while its powerful features empower you to write concise and expressive programs.

24. Python - $78,331

Python's readability and versatility make it a popular choice for beginners and experts alike. This general-purpose language is known for its clear syntax, resembling plain English, making it easier to learn and write code. Beyond its beginner-friendliness, Python excels in web development, data science, automation, and scripting thanks to its vast libraries and frameworks. Whether you're building a simple website, analyzing complex data, or automating tasks, Python's ease of use and extensive capabilities make it a powerful tool for programmers of all levels.

25. Kotlin - $78,207

Kotlin is a modern, general-purpose language designed for building cross-platform applications. It offers a concise and expressive syntax, making code cleaner and easier to write compared to Java. Kotlin integrates seamlessly with Java, allowing you to leverage existing Java libraries and even mix Kotlin and Java code within the same project. This makes it a popular choice for Android development, but its strengths extend to web development, desktop applications, and more, attracting programmers who value readability, safety, and the ability to build modern applications for various platforms.

26. APL - $77,500

APL, known for its unique set of symbols, condenses complex operations into concise expressions. This focus on brevity makes APL ideal for mathematicians and data scientists who value expressing complex logic in a compact way. However, its unconventional syntax can be challenging to learn for beginners.

27. Crystal - $77,104

Crystal is a general-purpose language aiming to bridge the gap between readability and performance. Inspired by Ruby's syntax, it offers a familiar feel for Ruby developers. But unlike Ruby, Crystal compiles to native code, making it much faster. This speed advantage, combined with its readable syntax and growing ecosystem, makes Crystal a compelling choice for those who want to build performant applications without sacrificing code clarity.

28. TypeScript - $77,104

TypeScript is an extension of JavaScript that adds optional static typing. This means you can define the data types your variables will hold, making your code more reliable and easier to understand. Think of it as JavaScript with a safety net – you get the flexibility of JavaScript with the ability to catch errors early in the development process, leading to more robust and maintainable applications.

29. Assembly - $77,010

Assembly language is a low-level programming language for a computer or other programmable device specific to a particular computer architecture in contrast to most high-level programming languages, which are generally portable across multiple systems. Assembly language is converted into executable machine code by a utility program referred to as an assembler like NASM, MASM, etc.

30. Fortran - $76,104

Fortran, a pioneer in scientific computing, remains a popular language for high-performance applications. Its clean syntax and efficient compilation make it ideal for numerical computations, simulations, and scientific data analysis. Even today, Fortran is a dominant force in scientific computing, used for tasks like weather prediction and engineering simulations, thanks to its speed and extensive libraries of specialized functions.

31. Cobol - $76,000

COBOL, though not as widely used as some modern languages, is still a critical player in the business world. Designed specifically for business applications, COBOL boasts a clear, English-like syntax that makes it readable even for non-programmers. This readability, along with its robust data handling capabilities, makes it ideal for managing vast amounts of business data. The real power of COBOL lies in its legacy. Many core financial systems, government programs, and enterprise applications are built on COBOL, and due to their critical nature, they continue to be maintained and updated. So, COBOL remains relevant for programmers who need to work with these existing systems or develop new applications that seamlessly integrate with them.

32. C# - $74,963

C#, pronounced "C-Sharp", is a versatile object-oriented language popular for building modern applications. Its clean syntax and strong typing make code readable and maintainable. C# integrates seamlessly with the .NET framework, offering a vast library of tools and features for web development, desktop applications, mobile apps, and games. This combination of readability, safety, and rich functionality makes C# a compelling choice for programmers of all levels.

33. C++ - $74,963

C++, known as "C plus plus", is a powerful language balancing high performance with versatility. It offers more control over system resources compared to higher-level languages, making it ideal for performance-critical applications like game development, system programming, and scientific computing. However, this control comes with increased complexity. Despite its steeper learning curve, C++ remains a popular choice for programmers who need the fine-grained control and raw performance it provides.

34. Julia - $74,963

Julia is a rising star in scientific computing, designed for speed and ease of use. It combines the power of a compiled language like C++ with the readability of a scripting language like Python. This makes Julia ideal for complex scientific simulations, data analysis, and machine learning tasks. Julia boasts a built-in capability for parallel computing, allowing you to harness the power of multiple cores or processors to tackle demanding computational problems efficiently.

35. R - $74,963

The R Project offers an open-source environment specifically designed for statistical computing and graphics. Imagine a powerful toolbox for data analysis, visualization, and statistical modeling. R excels at tasks like data manipulation, hypothesis testing, and creating informative charts and graphs. Its vast collection of packages further expands its capabilities, making it a popular choice for researchers, data analysts, and anyone who wants to extract insights from their data.

36. SQL - $74,963

SQL, or Structured Query Language, is a fundamental tool for interacting with relational databases. Think of it as a universal language you can use to talk to various database systems. SQL empowers you to store, retrieve, manipulate, and manage data stored in these databases. This makes it essential for tasks like generating reports, analyzing data, and keeping web applications running smoothly by interacting with the data behind the scenes.

37. C - $74,351

C is a foundational language that underpins much of modern computing. Despite its age, C remains popular for its efficiency and control over hardware resources. This fine-grained control makes it ideal for system programming, operating systems, and embedded systems where performance is critical. While higher-level languages offer more ease of use, C's ability to directly interact with hardware and its vast codebase make it a relevant language for programmers who need to squeeze maximum performance or interact with the core workings of a computer system.

38. JavaScript - $74,034

JavaScript (JS) is the lifeblood of interactive web pages. It adds dynamic behavior to websites, making them respond to user actions and create a more engaging experience. From handling form submissions to creating animations and adding real-time updates, JS is everywhere on the web. Its versatility extends beyond websites – JS is also used to build mobile apps, server-side applications, and even desktop software, making it a powerful and widely-used language for creating all sorts of interactive applications.

39. Java - $72,701

Java, renowned for its "write once, run anywhere" (WORA) philosophy, is a versatile object-oriented language. Its clear syntax and robust features make it a popular choice for building various applications. Java excels in web development, enterprise software, mobile apps, and even big data thanks to its rich ecosystem of libraries and frameworks. This combination of readability, portability, and extensive functionality makes Java a compelling language for programmers of all levels.

40. Solidity - $72,656

Solidity is a specialty language designed for building smart contracts, self-executing programs that run on blockchains like Ethereum. These contracts automate tasks and agreements securely and transparently. Solidity's syntax resembles JavaScript, making it familiar for web developers, but it focuses on creating secure and tamper-proof code for blockchain applications. So, if you want to develop decentralized applications (dApps) that leverage the power of blockchain technology, Solidity is the language to learn.

41. ADA - $71,500

Named after Ada Lovelace, Ada is a less common language known for its focus on reliability and safety. Developed for real-time systems and embedded systems (like those in airplanes or spacecraft), Ada prioritizes features that prevent errors and ensure program correctness. While not as widely used as some languages, Ada remains relevant in specific domains where safety and reliability are paramount.

42. HTML/CSS - $70,148

HTML and CSS are the building blocks of web pages. HTML provides the structure and content, like the headings, paragraphs, images, and links that make up a web page. Think of it as the skeleton of a website. CSS controls the presentation, dictating how the HTML content is displayed on the screen, including fonts, colors, layouts, and visual effects. Imagine it as the styling and design that brings the web page to life. These two languages work together to create visually appealing and informative websites.

43. Prolog - $70,000

Prolog is a logic programming language. It has important role in artificial intelligence. Unlike many other programming languages, Prolog is intended primarily as a declarative programming language. In prolog, logic is expressed as relations (called as Facts and Rules). Core heart of prolog lies at the logic being applied. Formulation or Computation is carried out by running a query over these relations.

44. Delphi - $69,608

Delphi language additions including multiline string literals, Skia support in FireMonkey offering higher performance and quality in rendering graphics and UI controls across all target platforms, a new FireDAC QBE for easier data filtering, and a new JSON wizard for data mapping.

45. GDScript - $69,608

GDScript is the built-in scripting language for the Godot game engine. Similar to Python in its syntax, it's designed to be easy to learn and use. This makes it ideal for creating game logic, handling user input, and controlling in-game objects within your Godot projects. GDScript integrates tightly with the Godot engine, offering features like automatic memory management and seamless interaction with the engine's components. So, if you're looking for a beginner-friendly language to build games with the Godot engine, GDScript is a great choice.

46. VBA - $65,698

Beyond the power of scripting VBA to accelerate every-day tasks, you can use VBA to add new functionality to Office applications or to prompt and interact with the user of your documents in ways that are specific to your business needs. For example, you could write some VBA code that displays a pop up message that reminds users to save a document to a particular network drive the first time they try to save it.

47. Visual Basic (.Net) - $65,000

Visual Basic (.NET) is a descendant of the classic Visual Basic language, designed specifically to work within the .NET framework. It offers a familiar and visually-oriented approach to programming, making it easier to learn than some text-based languages. This accessibility, combined with its .NET integration, makes Visual Basic (.NET) a popular choice for building Windows desktop applications, with access to a vast library of .NET tools and features to simplify development.

48. MATLAB - $61,735

MATLAB combines a desktop environment tuned for iterative analysis and design processes with a programming language that expresses matrix and array mathematics directly. It includes the Live Editor for creating scripts that combine code, output, and formatted text in an executable notebook.

49. PHP - $58,899

PHP is a popular open-source scripting language that excels at creating dynamic web content. Imagine adding interactivity and customization to websites. PHP seamlessly integrates with HTML to generate content on-the-fly, making websites more engaging. Beyond basic web development, PHP offers features for database interaction, user authentication, and session management. This versatility, coupled with its large community and ease of use, makes PHP a compelling choice for building websites and web applications of all shapes and sizes.

50. Dart - $55,862

Dart is a versatile language designed for building modern applications. Its clean syntax and strong typing make code readable and maintainable. The key strength of Dart lies in its ability to compile to native code or JavaScript. This means Dart code can run efficiently on various platforms, including web browsers (via compilation to JavaScript) and mobile devices (via native compilation). Additionally, Dart is the foundation of Flutter, a popular framework for building beautiful and high-performance cross-platform mobile applications. So, whether you're aiming for web, mobile, or even desktop development with a single codebase, Dart offers an efficient and feature-rich solution.

ref: Stack Overflow Developer Survey 2023

If you enjoy this article and would like to show your support, you can easily do so by making a donation through Ko-fi. Your contribution is greatly appreciated!

Buy Me a Coffee at ko-fi.com

Permalink

Feb. 2024 Short-Term Project Updates


Our first round of reports is in from some of our Q1 2024 deveopers.

Benjamin Kamphaus: UnifyBio
Chris McCormack: Sitefox
Daniel Slutsky: Scicloj
Eric Dallo: clojure-lsp
Jeaye Wilkerson: Jank
VEMV: CIDER

Benjamin Kamphaus: UnifyBio

2024 Q1 Project Report 1. Published 15 February 2024.

Accomplished so far:

  • Completed lightweight schema definition and spec, an example of which can be found here
  • Users can now stand up a local system with the plain JSON datalog query service and schema browser (see image below) using docker compose
  • Very basic quickstart steps are now included in the updated README
  • Added the ability to retract a Unify managed dataset
  • Streamed on twitch for a few weeks but paused that, moving all Unify dev to YouTube (see more below).

For the short term, my goals are:

  • Prioritize dataset lifecycle management, dataset lifecycle scenario tests, tutorials
  • Delay work on additional backends and streaming processing until I hit a stable point re: ^ schema-view *View of the schema browser, now part of the local Unify system (adapted from alzabo by Mike Travers)

At this point, Unify is in a decent shape as an alpha for prototyping/experimental work. I’d hold on using it for production, though things are stabilizing pretty quickly.

Two ways to make use of Unify that I think would be straightforward for its current state:

  • For anyone already using Datomic, who wants to import data that is in tabular form, or can be turned into a table, they could have a go at adding Unify annotations to their existing schema and writing an import config
  • For anyone wanting to start working incrementally at data modeling for a new domain, Unify can provide a good interactive environment for vetting how data out in the world, e.g. in tables, might map on to your schema, and you can visualize the data model and connections between entities as you go with the schema browser.

Edges are sharp and documentation is still limited but more detailed tutorials are coming soon!

If anyone is interested in getting stated with using Unify for data integration, or just importing tables into their Datomic schema, but are a bit wary of jumping into this blind, I’m on the Clojurians Slack as:
@Ben Kamphaus

In the channels:
[#unify, #datomic, #data-science, #data-engineering]

Notes on Streaming

I started streaming dev on Unify with Twitch but have had some frustrations with the platform and gotten feedback that YouTube would be better for streaming also, not just collecting videos. It’s not worth going into too much detail on this, mainly I want to point people to the YouTube channel if they want to follow along as I get this aspect of the work going again.

Thanks to Clojurists Together and the Rare Cancer Research Foundation for supporting this work!


Chris McCormack: Sitefox

2024 Q1 Project Report 1. Published 15 February 2024.

Hello! The first half of my Clojurists Together funded work on Sitefox is complete. I made around 50 commits to the project since the start of the year and this is a quick summary of the progress I’ve made.

My broad goal with the Clojurists Together funding is to make it safer and easier for other people to get started and build sites and apps on Sitefox. There are two components to this:
a) improving documentation and setup tooling
b) improving security and stability

Updating dependencies

I started the process of updating dependencies with an npm audit. This command finds npm packages with known vulnerabilities. I updated the npm package nodemailer due to a vulnerability, and I also express, nbb, and react. I also updated Clojure deps Promsea, and js-interop. This was the first part of the dependency updates item.

The second part of the dependency updates item was to replace the deprecated csurf module. This Express module is responsible for securing POST requests against CSRF attacks. Express is the web server that Sitefox depends upon and they have decided to no longer provide this module themselves.

Replacing this module turned out to be a bigger task than I anticipated. I started by evaluating several libraries as replacement candidates. Next I created a branch to try out one of these replacements. At first it looked like the library was a good replacement, but it was only upon more rigorous testing that I realized it had serious flaws that made it unsuitable.

This experience made it clear to me that I needed comprehensive testing of the form submission and authentication flows in order to tell if a library was a good replacement and did not introduce any bugs or regressions. This led me to work on the second part of the work.

End-to-end testing

I spent the greater proportion of the commits after this working on end-to-end tests of basic Sitefox functionality including:

  • Tests for basic server functionality - does the web server run, render pages, and respond to requests?
  • Tests for the main authentication flows such as signing up, signing in, signing out, forgot password.
  • Tests for form submission and field validation, including CSRF protection.
  • Tests for CSRF protection on Ajax POSTs.

I managed to get all of these tests implemented and passing except for the final Ajax test which I am still working on. After that I got the tests running headless on every commit via a GitHub action. This will ensure that the most basic functionality Sitefox offers won’t be inadvertently broken when I make major changes in future. In particular I can ensure the CSRF protection continues to work after upgrading to a different CSRF library.

What is next

From my initial plan I have only managed to make progress on these two items. My goal for the remainder of Q1 is:

  1. Finish the CSRF Ajax testing.
  2. Finish replacing the csurf dependency with a well-tested alternative.
  3. Update the key-value database module list and filtering code.
  4. Move the npm create helpers into the main repo, and document them properly.
  5. Tutorials showing how to get started building on top of Sitefox.

Given the progress of the first half of the quarter, I have tempered my goals a bit. If I can get these items done that will still provide a solid foundation for people wanting to get started with Sitefox and rely on it to build sites and apps.

You can see the board I am using to track progress here: https://github.com/users/chr15m/projects/13/views/1

Thanks for your interest and your support!


Daniel Slutsky: Scicloj

2024 Q1 Project Report 1. Published 29 February 2024.

February 2024 was the first of three months on the Clojurists Together project titled “Scicloj Community Building and Infrastructure”.

Scicloj is an open-source group developing Clojure tools and libraries for data and science.

As a community organizer at Scicloj, my current role is to help the emerging Scicloj stack become easier and more accessible for broad groups of Clojurians. This is done in collaboration with a few Scicloj members.

In February 2024, this has been about the following projects:

Clay

Clay is a namespace-as-a-notebook tool for literate programming and data visualization developed in collaboration with Timorthy Prately. It is used for documenting libraries and tutorials as well as an analysis tool for dynamic data exploration.

In February 2024, Clay got five new releases, mostly around the following:

  • improved compatibility of various visual elements with Quarto
  • improved support for making books
  • simplified live-reload experience
  • support for Observable visualizations
  • additional user-reported bugs and feature requests

Many thanks to generateme for the extensive testing and the numerous issues discovered.
In March 2024, the plan is mostly to continue user support and move from the alpha to the beta stage.

Noj

Noj is a project that collects a few recommended libraries for data and science in a bundle and adds some convenience layers for using them together.

During February 2024, we had some rethinking of this project’s scope and goals in discussion with Carsten Behring, who got involved in the project and has made some substantial contributions to its documentation.

My goal has been mostly to review Carsten’s work and add some documentation. For March 2024, the hope is to clarify the project’s goals and reorganize its relationships with other libraries accordingly.

Clojure Data Scrapbook

The Clojure Data Scrapbook is intended to be a community-driven collection of tutorials around data and science in Clojure.

In February 2024, I added a few tutorials and made some changes to the project structure. For March 2024, the main goal is to clarify the contribution workflow and encourage contributions.

Dev and study groups

visual-tools

In the visual-tools group, we began a new series of study sessions around ggplot2 and the grammar of graphics.

The intention is to implement a similar library in Clojure.

In February 2024, my attention has been around prototyping possible solutions, exploring ggplot and its internals, and creating a group of people who are interested in learning it. A few datavis-minded Clojurians have gathered a group and started thinking together, in meeting and over chat.

In March 2024, we will keep learning together. Hopefully, the ideas will be clarified, and some of the group members will start building something.

real-world-data

The real-world-data group will be a space for Clojure data and science practitioners to bring their own data projects, share experiences, and evolve common practices.

In February 2024, I restarted the group organization, mostly in one-on-one calls and chats with a few of the people who are planning to join.

In March 2024, we are expecting to have the first meeting.

Your feedback would help

Scicloj is in transition. On the one hand, quite a few of the core members have been very active recently, developing the emerging stack of libraries. At the same time, new friends are joining, and it seems like soon, more people will enjoy the Clojure for common data and science needs.

If you have any thoughts about the current directions, or if you wish to discuss how the evolving platform may fit your needs, please reach out.


Eric Dallo: clojure-lsp

2024 Q1 Project Report 1. Published 29 February 2024.

January 2024 Update:

clojure-lsp

The main highlight is the new custom project tree feature which shows project source-paths, dependencies, jars and namespaces! project-tree

2024.02.01-11.01.59

  • General

    • Fix binary not wokring on some aarch64 linux. #1748
    • Add new Project tree feature via the clojure/workspace/projectTree/nodes custom method. #1752
    • Fix --log-path setting to work with listen/empty arg, starting clojure-lsp server and logging properly.
  • Editor

    • Fix didChangeConfiguration throwing exception. #1749
    • Fix rename of ns causing wrong ns names because of duplicate rename actions. #1751
    • Fix range-formatting throwing exceptions when unbalanced parens are sent from client. #1758
    • Fix rename functions need to clean up LSP state or restart in some clients implementing LSP method didRenameFiles. #1755
    • Fix thread last all failed after comment form #_(...). #1745

Besides the clojure-lsp work, I worked too on some related projects that I think are important to improve clojure-lsp visibility and standardization as a tool, for example clojure-repl-intellij which without it make the use of clojure-lsp-intellij less valuable.

clojure-lsp-intellij

1.14.16 - 1.14.17

  • Bump clojure-lsp to 2024.02.01-11.01.59.
  • Add shortcuts to backward slurp and barf.
  • Add shortcut documentation to all features, check the features doc page.
  • Fix Rename feature not being available for some cases.

clojure-repl-intellij

  • Support for starting local repl besides the already available remote repl.

February 2024 Update

clojure-lsp

The main highlight is the new linter different-aliases helps guarantee consistency across alias in your codebase!
different-aliases

2024.03.01-11.37.51

  • General

    • Bump clj-kondo to 2024.02.13-20240228.191822-15.
    • Add :clojure-lsp/different-aliases linter. #1770
    • Fix unused-public-var false positives for definterface methods. #1762
    • Fix rename of records when usage is using an alias. #1756
  • Editor

    • Fix documentation resolve not working for clients without hover markdown support.
    • Added setting to allow requires and imports to be added within the current comment form during code action and completion: :add-missing :add-to-rcf #1316
    • Fix suppress-diagnostics not working on top-level forms when preceded by comment. #1678
    • Fix add missing import feature on some corner cases for java imports. #1754
    • Fix semantic tokens and other analysis not being applied for project files at root. #1759
    • Add support for adding missing requires and completion results referring to JS libraries which already have an alias in the project #1587

clojure-lsp-intellij

1.14.8 - 1.14.10

  • Fix exception when starting server related to previous version.
  • Fix some exceptions that can rarely occurr after startup.
  • Bump clojure-lsp to 2024.02.01-11.01.59.

Jeaye Wilkerson: Jank

2024 Q1 Project Report 1. Published 26 February 2024.

tl;dr I’m well ahead of schedule, so far this quarter! There’s a more detailed update here, with technical explanations: https://jank-lang.org/blog/2024-02-23-bindings/

Dynamic bindings

Dynamic bindings are implemented! I’m starting to use these within the compiler, as well, to track state. jank doesn’t have future yet, but everything is in place for binding conveyance. To go hand in hand with this, I tackled meta hints next, allowing us to do things like ^:dynamic on a var.

Meta hints

jank now supports hints in the form of ^:keyword, as well as ^{:keyword true}. Multiple hints can be specified, they can be nested, etc. Not many parts of jank are using these hints yet, since I didn’t go through all existing code to update usages, but we can do that iteratively.

What else?

Well, my quarter was booked for the following tasks:

  • 🗹 Dynamic vars
  • 🗹 Meta hints
  • ☐ Syntax quoting
  • ☐ Normalized interpolation syntax

The first two are done and the latter two remain, to be done in the next month. But is that all I did in two months? Just dynamic vars and meta hints? Nah.

Support for exceptions

I’ve added support for the special throw form, as well as for try/catch/finally. Previously, I was using inline C++ to throw exceptions, but the work on the dynamic vars gave me a good excuse to get proper exception support in there. I needed to ensure bindings were gracefully handled in exceptional scenarios!

My plan is to do a compiler deep dive, next post, going into how exceptions were implemented in jank. Stay tuned for that.

Escaped string literals

jank previously didn’t properly support strings like "foo \"bar\" spam", since they require mutation in order to unescape them. This is an interesting edge case, because all of the other tokens within jank’s lexer work based on memory mapped string views. String views allow jank’s lexer to run without any dynamic allocations at all. However, for escaped strings, we need to allocate a string, since we need to mutate it when unescaping. It’s straight-forward, but just something I hadn’t tackled yet.

Reader macros

Since I was improving the reader to support meta hints, I figured I’d add reader macro support. In jank, you can now use #_ to comment out forms. Also, you can use #{} to create sets. Finally, you can use #?(:jank foo) and #?@(:jank []) to conditionally read in jank code. The :default option is supported, too. Support for shorthand functions, regex values, and var quotes will be coming soon.

New core functions and macros

While implementing the above features, I also added support for the following vars in clojure.core:

  • assert
  • when-not
  • comment
  • zipmap
  • binding
  • with-redefs
  • drop (not lazy)
  • take-nth (not lazy)

Since jank doesn’t have lazy sequences yet, some of these functions are a little hacky. Lazy sequences come next quarter, though, as well as loop support!

Community involvement

jank received its first non-trivial code contributions this quarter, from a helpful man named Saket Patel. He’s working on a jank plugin for leiningen and it’s currently in a state where you can set up a multi-file jank project and use lein jank run to make magic happen. 😁 He’s also submitted some C++ improvements to aid in that quest and to help jank compile on Apple M1 hardware. I’ll have a more detailed demonstration of this in my next development update.

On top of that, in order to get him set up for contributing, I’ve done the following:

  • Added a Code of Conduct
  • Added a CLA, which is managed by a bot that will prompt you to sign upon submitting a PR with a large enough
  • Added automated code formatting and re-formatted the whole codebase, using clang-format. I don’t like the look of it, but at least it’s consistent

VEMV: CIDER

2024 Q1 Project Report 1. Published 7 February 2024.

January CT-sponsored work resulted in CIDER 1.13.0 (“Santiago”) and a subsequent 1.13.1 bugfix release.

Highlights have included:

  • all-new Timbre compatibility for cider-log-mode.
  • misc bugfixes, a couple of which were highly requested over the Clojurians Slack #cider channel.

There’s also been some WIP to improve our cljfmt integration.

Permalink

Principal Software Engineer at Scarlet

Principal Software Engineer at Scarlet


Meet your teammates

We are deliberately building a highly potent engineering team that is very small relative to its impact.

A council of Jedis, rather than an army of stormtroopers, if you will.

Today, it's a team of 4 highly experienced individual contributors operating as peers.

⚡️ Lightning introductions to your new teammates:

Why Scarlet?

We believe in an exciting future where software is ubiquitous in healthcare, delivering better outcomes for everybody.

Scarlet’s mission is to hasten the transition to universally accessible and affordable healthcare by safely increasing the quantity, quality and access to Medical Software.

What problem does Scarlet solve?

Unfortunately, the current regulatory environment wasn’t designed for software:

  • Getting regulatory approval can be so daunting and time-consuming that some smaller innovative projects never get off the ground.
  • It can delay the first release of software by years.
  • It can slow subsequent software iterations to the same pace at which drugs and hardware medical devices are improved.

How does Scarlet solve the problem?

We have built Scarlet from the ground up to certify Software Medical Devices.

Scarlet has the regulatory approval to certify medical software.Scarlet’s product enables its customers to achieve swift regulatory approval and perform frequent subsequent software updates.

Intriguing… what are the details of the technical solution and how is it different from the status quo?

The traditional solution to this problem is decidedly non-technical. To prove that a release is ready for certification, software builders submit documentation. Like a lot of documentation. Think 1,000 page long PDFs.

Our technical solution is non-traditional.

When using Scarlet, software builders still have to submit a lot of information. It’s the law. And every time they change something, they have to add more documentation and make sure all the existing documentation references that new documentation. But instead of wrangling a giant PDF, when using Scarlet, they submit each change as a single small file.

These files have some special sauce. On the surface, they look like Markdown, making them familiar to people who are used to the usual paper-based submission flow. But actually, they’re a special dialect of Markdown, which we call Slang™️ - short for The Scarlet Language.

One of the amazing things about Slang is that Slang files can be linked together. Users think they’re just typing information into a file, but actually, they’re populating a huge graph of connected files. We can query that graph to understand how new data relates to old data and whether there are any inconsistencies. When things don’t all line up, when the data they’re submitting would prevent them from getting certified, we can give them feedback.Because we think of Slang as a language and because our customers are editing files, it made sense to us to implement the Slang tooling as a VS Code Extension, with an LSP Server backend (ClojureScript and Clojure, respectively).

That lets our users have a great editing experience. They get version control, diffs, and lots of other stuff for free, from the editor. But it also lets us augment their experience with all of our knowledge about what their edits actually mean, and how they relate to all the existing documentation. We can help them with auto-completion, navigation, renaming, and many other goodies. Most importantly we can provide feedback in the Problems pane when they’ve entered something incorrectly.

Small changes, kept consistent with automated checks. Sound familiar? A bit like a good software development flow? This is why we think we can cut a process that typically takes a year or more down to a week or less. This is how Scarlet is going to radically change the pace of software innovation in medicine.

What principles do we emphasise in our team?

  • Small team/Big impact: We strive to make an outsized impact relative to the size of our team.
  • Openness: Proactively sharing context, so we can all make good choices
  • Problem-centricity:  Framing our conversations in terms of problems and striving to define and prioritise them well

These three principles empower us to be unreasonably effective whilst enjoying:

  • Flexible working
  • Remote-first
  • No prescribed hours
  • No holiday tracking
  • Low/no scheduled meetings
  • No stand-ups, retrospectives or agile ceremonies
  • We get together in real life twice a year for a week at our offices in London or Amsterdam
  • Asynchronous collaboration
  • We have rich async discussions
  • We flexibly have 1:1s with each other as and when it makes sense
  • High Trust and Autonomy
  • Nobody just implements prescribed solutions handed to them by someone else.
  • We all solve problems, own our choices and trust and respect each other.

What does our hiring process look like?

  • (1) Reach out to me via email (jamie@scarletcomply.com) or in Clojurians Slack (@Jamie Cox) with your GitHub, LinkedIn/CV
  • (2) We’ll do a quick sanity check on the possibility of a fit on our side. Here are some of the things which might be exciting early indications of a potential mutual fit:
  • 10++ years of professional experience
  • Highly regarded contributions in OS projects
  • Experience building dev-tools
  • Passion/motivation to solve problems in healthcare
  • Strong reference from a well-regarded source
  • (3) We’ll enable you to do a sanity check on the possibility of a fit on your side by sharing a bunch of stuff for you to peruse
  • 🍿 Introduction to Scarlet video
  • 📽️ Product demo videos
  • 📚 Product docs
  • 🎮 Access to a demo account
  • (4)  An intro call with me
  • An opportunity for you to ask me all your questions about what you’ve seen so far and anything else
  • An opportunity for me to ask you a few questions about where you’re coming from, what your superpower(s) is/are, and what you need  from your environment to be able to wield them at maximum strength
  • (5) A call with a future teammate
  • An opportunity for technical and cultural questions from both sides
  • (6) Technical challenge
  • (7) Call(s) with the remaining future teammates that you’ve not met yet
  • An opportunity for technical and cultural questions from both sides

How does Scarlet think about compensation?

Scarlet is a well-funded early-stage company, that has raised >$9M in funding and has paying customers.

Scarlet strives to pay top-of-market compensation.

That is, we aspire for compensation to be greater than, or at least equal to the next best alternative (anywhere in the world).

Scarlet is still at an early stage compared to many other companies. In practice, that means that it’s not possible to match the very top of the market yet. So, Scarlet benchmarks against the top of the market and sets compensation as close to that level as possible. Then compensation is reviewed and adjusted every 6 months, with the goal of getting to the top of the market as quickly as possible

Permalink

Copyright © 2009, Planet Clojure. No rights reserved.
Planet Clojure is maintained by Baishamapayan Ghose.
Clojure and the Clojure logo are Copyright © 2008-2009, Rich Hickey.
Theme by Brajeshwar.