C and C++ are really so fast?

During all that time I am engaged with programming, I hear that C and C++ are the speed standards. Fastest of the fastest, compiled straight to assembly code, nothing may compete in speed with C or C++. And, nobody seem to challenge that common belief.

Computing performance

Arithmetic operations with numbers, obviously, must work significantly faster in C than in any other language. But do they?
Some time ago I decided to write a set of simple benchmarks for many different languages to see, how large difference in speed really is.
Idea was simple : to find the sum of the one billion integer numbers, starting from zero, using straight-forward computing. Some compilers (rustc, for example) replace such simple cycles with formula expression, which, of course, will be evaluated in the constant time. To avoid that with such compilers. I used similar in costs operations with numbers, such as bitwise or.
After I got results, I was surprised very much. My world view turned upside down, and I had to reconsider everything I knew about speed of programming languages.
You may see my results in the table below :

Linux 64bit, 1.1 GHz CPU, 4GB RAM

All tests sources you may find here :
https://github.com/Taqmuraz/speed-table

So, as we may see, C is not very much faster than Java, difference is about 3%. Also, we see that other compiled languages are very close in arithmetic operations performance to C (Rust is even faster). Dynamic languages, compiled with JIT compiler, show worse results -- mostly because arithmetic operations are wrapped into dynamically dispatched functions there.
Interpreted dynamic languages with no JIT compiler show worst performance, not a surprise.

Memory allocation performance

After that crushing defeat, C fans would say that memory allocation in C is very much faster, because you are allocating it straight from the system, not asking GC.
Now and after I will use GC term both as garbage collector and as managed heap, depending on the context.
So, why people think, that GC is so slow? In fact, GC has pre-allocated memory, and allocation is simply moving pointer to the right. Mostly GC fills allocated memory by zeros using system call, similar to memset from C, so it takes constant time. While memory allocation in C takes undefined time, because it depends on the system and already allocated memory.
But, even considering this knowledge, I could not expect so good results from Java, which you may see in following tables :

Allocating integer arrays

Allocating instance of the class Person with one integer field.

All tests sources you may find here :
https://github.com/Taqmuraz/alloc-table

There I tested four languages in total : C, C++, Java and Lisp. And, languages with GC always show better results, though I tested them much stricter, than C and C++. For example, in Java I am allocating memory through the virtual function call, so it may not be statically optimized, and in Lisp I am checking first element of the allocated array, so compiler won't skip allocation call.

Releasing memory

C fans are still motivated to protect their beliefs, so, they say "Yes, you do allocate memory faster, but you have to release it after!".
True. And, suddenly, GC releases memory faster, than C. But how? Imagine, we made 1 million allocations from GC, but later we have only 1000 objects referenced in our program. And, let's say, those objects are distributed through all of that long span of memory. GC does stack tracing, finding those 1000 "alive" objects, moves them to the previous generation heap peak and puts heap peak pointer after the last of them. That's all.
So, no matter, how many objects you allocate, GC's work time is decided by how many of them you keep after.
And, in opposite with that, in C you have to release all allocated memory manually, so, if you allocated memory 1 million times, you have to make 1 million release calls as well (or you are going to have memory leaks). That means, O(1)-O(n) of GC against O(n) or worse of C, where n is the number of allocations happened before.

Summary

So, I want to consolidate the victory of garbage collected languages over C and C++. Here is the summary table :

Now we may see -- garbage collection is not a necessary evil, but the best thing we could only wish to have. It gives us safety and performance both.

Tribute to C

While C does show worse results on my tests, it is still an important language and it has own application field. My article does not aim for C rejection or obliteration. C is not bad, it is just not that superior as people think. Many good projects collapsed only because some people decided to use C instead of Java, for example, because they have been told that C is very much faster, and Java is incredibly slow because of garbage collection. C is good, when we write very small and simple programs. But, I would never recommend writing complex programs or games with C.

C++ is different

C++ is not simple, is not flexible, has overloaded syntax and too much complicated specification. Programming with C++ you will not implement own ideas but fight with compiler and memory errors 90% of the time.
This article does aim for rejection of C++, because speed and performance are only excuses people give for using this language in software development. Using C++, you are paying with your time, your program performance and your mental health. So, when you have choice between C++ and any other language, I hope you choose the last one.

Permalink

Slowification and Amplification

Check out my book called Grokking Simplicity and it’s the functional programming book you can recommend to beginners. If you don’t recommend it to your friends, please recommend it on Amazon.

I’m giving a talk in January at the Houston Functional Programming User Group. I’ll be presenting the last 2.5 domain modeling lenses for my upcoming book, Runnable Specifications. There are 8 sections written already, available free online. Read them now.

This issue is short because I just got back from Thanksgiving vacation. Please enjoy!

Slowification and Amplification

I get the feeling that people think I complain a lot. I notice problems, opportunities for improvement. I want to make things better, and so I tell people what I see. But, in the wrong culture, bringing up problems sounds like complaints. We’re so busy getting stuff done, and all Eric does is bring up problems that we don’t know how to solve. And so people label me as a complainer.

But identifying problems and areas for improvement is a great skill. We should all appreciate when someone on the ground, doing the work, notices an inefficiency. These signals from the trenches can be invaluable. But, too often, they’re ignored, and the source of the signals is stigmatized. That’s how it’s been at too many places where I’ve worked. It makes me feel ignored, alone, and unappreciated.

That’s why I’ve been drawn to the andon cord. When you pull it in the Toyota factory, it not only stops the line, but it calls your manager to come see the problem. And they help you solve it! You’re encouraged to bring up problems. The andon cord is pulled thousands of times per day—and they love it.

Wiring the Winning Organization describes the principles behind the andon cord in great detail. The authors call them slowification and amplification. Amplification is the boosting of signals about problems so they get to the people able to solve them, which is often as high as top-level management. Slowification refers to stopping the performance of our job for long enough to think about the job itself so we can solve problems and improve. These principles are used at high-performing teams and organizations to constantly improve.

But too often, programmers are ignored when they ask for time to improve the code, or denied the budget for a tool that will help them, or considered whiny babies for talking about how things could be better. The book details case study after case study where taking time to slow down and improve the work leads to better results in the medium and long term. I only hope I can work at such an organization one day.

The most I’ve really experienced has been the ubiquitous “retrospective”, where people get to complain but mostly nothing comes of it. Even when there is something that comes of it, it’s the usual “solve problems by adding process.” It makes me so angry I want to throw things. But that’s a story for another issue.

Permalink

How We Feel about Tech, DevX and GenAI - Solita Developer Survey 2024 Results

It is time to review the latest Solita Developer Survey results again! In this posting, we’ll examine the tools, technologies and processes we use to do our daily work at Solita.

Who Responded

This year we had 322 responses from six Solita countries. Most of the responses (201) came from Solita’s origins Finland - Denmark was the second with 80 and Poland the third with 34 responses. Sweden (4), Estonia (2) and Belgium (1) had only a minor representation. The most common job role was still a Full-stack developer (172), followed by Backend developer (108) and Software architect (66). 38% of us said, they mostly work remotely from home. Interestingly, in Denmark all respondents work at the office whereas in Poland almost everyone works remotely. Finland has a mixture of these two.

Competence Development

In the software industry, it’s essential to keep one’s skills relevant. Usually, everyday customer work offers many possibilities to develop oneself further. On top of that, 59% of the respondents say they spend at least some time on side projects or open-source contributions. That’s quite a high number. On average, when asked how well can you develop your competencies in your working time, the result was 4.52 points out of 7 which can be considered quite good. As the most popular learning platform, YouTube is still number one, followed by O’Reilly and Udemy. Of the developer conferences, our developers have followed Apply WWDC, Google I/O, AWS re:Invent and Disobey, to name but a few. On top of external conferences, Solita has its own DevDay where we spend every year a whole day with internal conference talks on multiple tracks, and have dinner and fun in the evening.

OS, IDE and Tooling

When it comes to building something, you need proper tools. The ones you know, the ones you trust, or the ones you would like to learn. As the leading operating system, Windows had a 48% share, followed by Mac 43% and Linux 9%. The most popular Linux distribution was Ubuntu, as usual. The most used IDE/editor was Visual Studio Code with 61%, followed by IntelliJ IDEA with 45%. A developer can use more than one IDE, depending on the work context. Some of our hard-core developers have always had tough discussions between Vim vs. Emacs - this time, Vim beat this race with 28 vs. 11 users.

Programming Languages and Software Stacks

For many, the most important thing in software development is the programming language. Some feel they are all just tools, while others are passionate and would not like to switch their favorite language at any cost. In 2024, our most popular language to work with was C# with 28%, followed by TypeScript at 15%, Kotlin at 11%, Java at 10%, Clojure at 7% and Python at 6%. There is a difference with the most liked and most used programming language too: the reality is that the most used one was SQL with 63% (almost all applications have a database) - also frontend behemoths with TypeScript at 61% and JavaScript at 60% were among top three most used ones. Interestingly, 2024 is the first time TypeScript has surpassed plain JavaScript - we expect this trend to continue. Also, C#, Go and Swift saw rising adoption.

When it comes to databases, PostgreSQL keeps the lead with a good margin. We also see cloud-based managed databases gain popularity, such as Azure SQL and Amazon Aurora. Oracle, MongoDB and MariaDB were in decline.

CI/CD Pipelines, Deployments and DevOps

To ensure stress-free (and almost bug-free) releases, you need proper continuous integration and deployment pipelines. Here we see a major, yet not unexpected, shift: Jenkins has given way to cloud-based solutions from the pole position to number four. Azure DevOps and GitHub Actions have taken the undisputed lead, followed by Gitlab CI. This is reflected in the wider adoption of cloud-based environments generally. When many of our customers shift their operations from on-prem to the cloud, it also impacts the tooling used for CI, packaging and deployment.

Testing, Monitoring and Security

While public cloud solutions gain popularity, also the willingness to invest in testing, monitoring and security increases. This can be seen in the results of our survey as well. In test automation maturity, the biggest winner is automated security tests almost doubling its adoption to 13% of respondents. On the monitoring side, almost all items saw a small rise, such as client-side error logging, performance monitoring, automated alerts and audit logs. In general, this probably stems from customers’ higher awareness of possible risks if logging, monitoring or security are neglected.

Generative AI - Adoption Rate and Usage

Generative AI, or GenAI, continues to be a hot topic in our industry. Solita has made a conscious effort to increase the competencies in this area; both in the tooling and the capabilities we use to help our customers. We all must become more aware and learn about these capabilities as they develop at a fast rate. Current production-level solutions rely very much on human-in-the-loop, but more agentic and autonomous applications are being explored by various industries. 38% of the respondents use GenAI daily and 26% weekly, so you can say most of us do use GenAI in our daily work. The methods vary, depending on the needs, personal know-how and even customer policies. We have internal discussion channels and info sessions on various GenAI methods and possibilities to play around with them in sandbox environments. The most popular GenAI services were GenAI chats (90%), GitHub CoPilot 59% and image creators such as Dall-E (27%). But many people go beyond these: things like OpenAI API, LLMs and neural networks, and M365 Copilot are reported to be used by many. GenAI is used for example to help decision-making, code autocompletion, aid in creating test code, explain errors and automate routine tasks.

Developer Experience (What Do We Value)

For the first time, our survey also had a section called Developer Experience, or DevX. There we wanted to know, what kind of things our experts value the most, in terms of customership, development environment, used practices and needed support. Some of the most important things for us in this perspective were:

• I can act according to my values in my work
• I can set up my development environment easily
• I can influence the improvement of development practices
• I feel comfortable asking my teammates for help
• Support from the project lead (e.g. priorities and decision-making)

Career Opportunities at Solita

We seek constantly more people who enjoy solving customer problems with tech and are eager to develop their skills further. You’ll find our open jobs at www.solita.fi/en/careers/.

Previous Solita Developer Surveys

The tech landscape evolves quite fast. To better understand how time has shaped our preferences, we recommend you read also the previous survey postings.

• Solita Developer Survey 2022
• Solita Developer Survey 2020
• Solita Developer Survey 2016

Permalink

[Boost]

10 Soft Skills que Aprendi Durante 3 Anos Criando Soluções para 100 Milhões de Usuários

Felipe Nascimento ・ Dec 1

#career #clojure #programming #developers

Permalink

State of Clojure 2024 Results

2024 Highlights

In 2024, we continued to see strong use of Clojure in a wide variety of organizations, with 73% of users using Clojure for work, especially in web development, open source, commercial services, and enterprise apps. Most were deploying those apps in either public (58%) or private (26%) cloud.

Most developers (54%) were developing for users outside their organization, in organizations of every size.

In the majority of cases, the Clojure teams were small, but teams of 100+ were also represented, likely led by Nubank, which has well over 1000 Clojure developers.

Industry-wise, the top sectors were, as usual, finance, enterprise software, consumer software, healthcare, and retail/ecommerce, with a broad range of additional responses reported - Clojure reaches everywhere there is a host platform (JVM, JavaScript, CLR).

Clojure developers mostly use either MacOS or Linux, with one of four popular development environments: Emacs, IntelliJ, VS Code, or Vim.

Clojure 1.12.0 was released in September 2024 and the survey showed rapid uptake, with 58% already using it, and 65% developing or deploying with the prior versions 1.11, and a steep drop-off after that. Clojure’s focus on stability and avoiding breaking changes makes upgrades safe and easy.

With JVM usage, we saw a significant shift from last year with the Java 21 LTS release, which 54% are now using, or even newer versions like Java 22 or 23 (26%). Usage of Java 8 LTS, the oldest supported release has dropped to just 9%. We expect to move the Clojure JVM baseline version in the next release, version TBD.

One new question this year focused on alternative Clojure dialects - these are created by enterprising community members spreading the ideas of Clojure into new hosts and niches. Babashka, a fast native-compiled scripting runtime with batteries included is being used by an eye-popping 93% of survey respondents that answered this question (about 2/3). We also saw usage of ClojureDart, Squint, Jank, and Cherry. Some additional options mentioned in the comments were Electric, Rama, nbb, sci, scittle, clojerl, and Basilisp - we’ll review those as options for next year.

Two additional questions were open response questions about web development and non-web UI frameworks. We’ll be using those responses to form concrete options for next year. Feel free to peruse the answers in the full data.

As always, we closed the survey asking who you appreciate in the Clojure community. There are way too many people and companies to mention without leaving someone out - I encourage you to read the responses! It’s been a great year with Clojure and we look forward to an amazing 2025.

Trends Over Time

Because this survey has been running since 2010 (thanks to Chas Emerick originally!), we have lots of great longitudinal data and it’s interesting to compare some of the answers over time.

Looking at the question of how Clojure developers use Clojure, we can see this has generally trended more towards using it at work. However, this year we saw an uptick of people using it for hobbies or in their studies:

One question that’s always interesting to examine is what primary language developers were using prior to Clojure. These answers have been relatively static over time - most Clojure developers come from Java, JavaScript, or Python. We can see that Ruby and C++ have fallen the last few years but C# has gotten a bit stronger. See the later section too which focuses just on new users.

We can also see how developer environments have changed over time, from OS:

To dev environment:

This is a graph of the primary JVM use over time (intermediate versions between LTS releases omitted for clarity):

Java 8 has finally relented and we are seeing strong uptake to newer versions like Java 21. For many Clojure releases we have been using Java 8 as our baseline, but we expect to move that in the next Clojure release.

Speaking of Clojure versions, Clojure’s strong record of stability and additive development makes it easy for users to upgrade when a new version is available, as with Clojure 1.12 this year:

Another interesting thing we can track is the primary way people manage their dependencies:

2024 New Users

It’s always interesting to narrow the focus just to new users (those who reported using the language for less than or up to 1 year). Applying this filter can give us a view of why and how new users are finding Clojure. For many of the questions, we see data similar to the overall user base, but in some areas new insight can be gleaned.

For example, we can look at the prior language just for new users which gives a clearer picture of where people are coming from when they start to learn Clojure:

While Java is the dominant prior language for Clojure users as a whole, new users now are primarily coming from JavaScript, Python, or Java and those trends have changed somewhat over time. One especially interesting signal is the rise of C#, which was not easy to see in the data for all users.

New users found the following challenges in learning:

It is intriguing to imagine whether the changes over time come more from what’s happening in the community or in the background of new users. For example, the difficulties of understanding functional programming vs object-oriented programming has decreased significantly over the last 3 years - is this due to a greater influx from non-OO communities, or better learning materials?

Similarly, we can look just at priorities for new users:

These don’t vary too much from the community as a whole, but spec is ranked quite a bit lower level.

It can also be useful to see which forums new users are finding useful:

In general, these are similar to the community at a whole but they are over-represented in YouTube, StackOverflow (not surprising), and Discord (probably due to Clojure Camp).

Permalink

Clojure is really nice to write

I was a happy Ruby on Rails programmer in my previous office, and out of the blue, my boss asked me to learn ReactJS. JavaScript doesn’t suck for me, it sucks, it sucks, sucks, sucks, and it was no way I am going to learn it. But my boss persisted, he seemed like a good guy, so I tried out ReactJS. The more I learned it, the more it sucked.

I was assigned ReactJS tasks, and God knows how I finished it. I hated it. I am a programmer, and you want me to do what a programmer mustn’t do, that is to code in JavaScript. It’s not a programming language, everyone uses it because they have no other choice, the browsers won’t offer first class support to any other language.

In my job prior to this sucking ReactJS job, I was shielded by jQuery and Coffeescript (wow! I loved it), But this time it was ReactJS, and it seems to make even little things complicated. We were working on Ruby on Rails project, and Rails had killed ReactJS even before it was born. I had to wonder what brains (or what kind of idiot) will it take to use React with Rails?

So I had to learn, learn I have to learn this ReactJS, the curse for my previous birth’s evil karma. So while I need to code ReactJS app, I did need a backend, but for each and every type of project I had to develop a back end. I am a lazy guy and thought: ‘why can’t I develop a simple CRUD backend that will suit a front end learner? And it should require no configuration’, and hence was born Injee - The no configuration instant database for front end developers. I wrote it using Clojure.

The agony of developing a stuff in JavaScript was soothed by developing Injee in Clojure. I reckoned that, just like I hate front end, a lot of front end developers will hate back end and Injee will help them to develop POC of their idea before reaching out to a back end developer.

I am aware of HTMX, and Injee was primarily designed to deliver CRUD payload’s vis JSON. For HTMX we need some sort of templating engine that converts JSON to HTML, hence was born the idea of Injex, that is Injee’s CRUD working well for HTMX developers. To do so this is my code change https://codeberg.org/injee/injee/compare/f63362049f3b9b20c830df1aa3266cb9d0ab1145...injex, thats it (yes more refactoring should be done, but this is it for now). As I add feature upon feature in Injee using Cloure, things just work. Nothing really breaks. It’s awesome. Why I haven’t discovered Clojure 10 years before? I feel HTMX will replace many JavaScript frameworks out there simply because it’s super productive, so Injee has a better chance of picking up if it supports HTMX, thanks to Clojure this was implemented without pain.

I was thinking of developing an admin front end for Injee with Svelte, but with this code change I can do it with HTMX! Which I think is awesome and will make me enjoy coding than deal with JavaScript. I want to thank Clojure, its creator(s), people who helped me a lot in the community and people who are trying to make this wonderful language shine.

Personally I don’t need Injee any more, the boss who told me to learn ReactJS threw me out of the job. I don’t blame him, looks like companies financial are not good, but I feel I have to develop Injee because some front end developer will benefit out of it, just like I benefited having a backend to learn ReactJS.

Permalink

Holy Dev Newsletter November 2024

Welcome to the Holy Dev newsletter, which brings you gems I found on the web, updates from my blog, and a few scattered thoughts. You can get the next one into your mailbox if you subscribe.What is happeningI’ve published my 15 min talk Mind-bending (technologies) from Ardoq conference about technologies that expand our ideas of what is possible and what computers can do for us. The good news? They are all based on Clojure or usable from Clojure! (Featuring Wolfram and Wolframite, Rama, Electric Clojure). Enjoy!

Permalink

The Kubernetes Effect

In my many years of creating Open Source software and talking about it at conferences, some of the most productive development times are often those leading up to the presentation.

In the last post, I mentioned that I was going to present a 90 minute YAMLScript tutorial at KubeCon (November 15th in Salt Lake City).

The conference was amazing and the YAMLScript tutorial was a huge success. I came away with the feeling that YAML and YAMLScript had found their community. KubeCon felt like YAMLCon!

But today's post is about the work leading up to the conference and the new data oriented features that were added to YAMLScript as a result.

At the start of October we realized there were a few things we wanted to add to the language to make it great for defining dynamic data in YAML files.

YAMLScript is a complete programming language and while you could already do almost anything with it, we knew that it had blend smoothly into the existing YAML files that people use for Kubernetes and other uses. We started by focusing on Helm charts, and seeing how well YAMLScript could fit along side the Go templating system (or replace it entirely). In the end it all worked out very well, but a few things needed to be added before tutorial time.

TL;DR for Helm Users: https://yamlscript.org/doc/helmys

In this post I want to cover some of the new (and old) features of YAMLScript that you can use to do cool things in your YAML files that are not possible with YAML alone. And remember, your YAML files are already valid YAMLScript files; but if you load them (or preprocess them) with YAMLScript, you can do a lot more in there.

What YAML Can Do Now

YAML 1.2 has a few things that let you do things a bit fancier than with a data only format like JSON:

• YAML has anchors and aliases for reusing specific nodes by naming them, and then later referring to them by name.
• YAML has a special << "merge key" that you can use to merge mappings. This was actually removed from the 1.2 spec but many popular implementations still support it (albeit with inconsistent behavior between implementations).
• YAML allows you to tag nodes and this can sometimes be used creatively, though not consistently across different YAML implementations.
• YAML supports multiple "documents" in a single file or stream, although unfortunately the spec doesn't allow you to alias nodes across documents.

YAMLScript supports all of these features too. It needs to since it claims to support the current YAML files of the world. But YAMLScript being a complete programming language lets you go so much further!

Let's take a closer look…

External (and Internal) Data Sources

Even though YAML lets you reuse nodes by name, those nodes need to be part of your YAML document. Say you have a section at the top of your YAML file that defines some default values and names them with anchors, to be aliased later throughout the file.

This is problematic because the node of defaults is also going to be a part of your data when you load it. It would be nice if you could have 2 documents in a YAML file where you define the data sources to be referred to in the first document, and then refer to them in the second document. If the loader returned the final (second) document then you could get the data you wanted without also getting the data that you don't.

Let's try it out:

# file.yaml
---
- &map1
  name: Bobbi Datamon
- &list1
  - fun
  - games
- more: stuff

--- !yamlscript/v0:
person:
  <<: *map1
  likes: *list1

Now we can load it with the YAMLScript command line interpreter, ys:

$ ys --load file.yaml
{"person":{"name":"Bobbi Datamon","likes":["fun","games"]}}

Looks like it worked, but ys --load prints the result in compact JSON. Before we discuss what happened, let's show the result in YAML:

$ ys --load --yaml file.yaml
person:
  name: Bobbi Datamon
  likes:
  - fun
  - games

Nice! ys --load gave us the data from the final document that included data from the first document.

from yamlscript import YAMLScript
ys = yamlscript.YAMLScript()
text = open("db-config.yaml").read()
data = ys.load(text)

The careful reader will have noticed that we broke the rules of YAML. We aliased nodes that were anchored in a different document. What's up?

Well, the two YAMLScript documents each have a different YAMLScript mode and that makes things work differently.

The first document has no !yamlscript/v0 tag and thus is in "bare mode". In bare mode all the rules are the same as YAML 1.2 (using the Core schema).

The second document has the !yamlscript/v0: tag. The !yamlscript/v0 tag tells YAMLScript that the document is in "code mode" (the content starts as code but can switch to data mode at any time). The : in !yamlscript/v0: tells YAMLScript to switch to data mode right away.

In code mode and data mode, aliases are very different than in bare mode. They can access anchored nodes in the same document or any previous document. Not only that, you can access parts of the anchored node with a path syntax. For instance *map1.name would produce the string "Bobbi Datamon" and *list1.1 would produce the string "games".

That means that this works the same way as the previous example:

--- &data
map1:
  name: Bobbi Datamon
list1:
- fun
- games
more: stuff

--- !yamlscript/v0:
person:
  <<:: -*data.map1
  likes:: -*data.list1

Here we only anchored the entire first document and then used the path syntax to access the parts we wanted. Note that to do this we used :: to switch to code mode and we also needed to used - to escape the values and have them be treated as YAMLScript expressions.

YAMLScript has special +++ symbol that evaluates to an array of all the prior documents in the stream. That means we don't need anchors at all:

map1:
  name: Bobbi Datamon
list1:
- fun
- games
more: stuff

--- !yamlscript/v0:
person:
  <<:: +++.last().map1
  likes:: +++.$.list1

The +++.last() function returns the last document in the stream so far (the first document in this case) The +++.$ is a shorthand for +++.last().

Another approach to take here is to make the first document use YAMLScript in code mode and define variables to use in the second document:

--- !yamlscript/v0
map1 =::
  name: Bobbi Datamon
list1 =::
- fun
- games

--- !yamlscript/v0:
person:
  <<:: map1
  likes:: list1

We use =: for assignment expressions in YAMLScript. And =:: does the same thing but toggles the mode of the value.

What if list1 was a huge list and you really wanted to keep it in a separate file?

No problem:

# big-list.yaml
- fun
- games
# ...

Now we just change the first document to load the list from the file:

--- !yamlscript/v0
map1 =::
  name: Bobbi Datamon
list1 =: load('big-list.yaml')

Not only can we access external data from a file, YAMLScript supports fetching data from the web with the curl function and also getting data from databases!

Inline Code in Data Mode

We can do the same things in a single data mode document. The trick is that we need to have a way to evaluate code in a way that doesn't affect the data.

The :: syntax is our new friend here.

This lets us do code things like define variables and even define new functions in a way that doesn't affect the data we are defining.

--- !yamlscript/v0:
::
  defn flip(array):
    reverse: array

  map1 =::
    name: Bobbi Datamon
  list1 =: load("big-list.yaml")

person:
  <<:: map1
  likes:: list1:flip

We defined a new function called flip which is a bit contrived since we could have called reverse directly; but it proves the point.

We also defined our data variables. We can actually define variables without :::

--- !yamlscript/v0:
map1 =::
  name: Bobbi Datamon
list1 =: load("big-list.yaml")

person:
  <<:: map1
  likes:: list1

In a big document, it's sometimes nice to define the data variables closer to where they are used.

--- !yamlscript/v0:
map1 =::
  name: Bobbi Datamon

person:
  <<:: map1
  list1 =: load("big-list.yaml")
  likes:: list1

In fact, since we only use map1 and list1 once, we could have just inlined them:

--- !yamlscript/v0:
person:
  <<:
    name: Bobbi Datamon
  likes:: load("big-list.yaml")

How Do I merge Thee?

Let me count the ways :)

We are still using the << merge key to merge mappings, but YAMLScript has a a standard merge function (among 100s of others).

--- !yamlscript/v0:
map1 =::
  name: Bobbi Datamon
list1 =: load("big-list.yaml")

person::
  merge::
  - ! map1
  - likes:: list1

Note the ! in front of map1. It toggles from data mode to code mode. We need to use ! for that purpose in data mode sequences. For mappings we can use key:: variable but it is just a shorthand for key: ! variable.

Another way to write that is:

person::
  merge map1::
    likes:: list1

Since the merge key is already in code mode we can just put the map1 variable there.

Sometimes you want to use a function like merge without needing to further indent the data you are applying the function to.

YAMLScript lets you put a function in a tag if you prefix it with a::

--- !yamlscript/v0:
map1 =::
  name: Bobbi Datamon
list1 =: load("big-list.yaml")

person: !:merge
- ! map1
- likes:: list1

Conditional Insertions in Mappings and Sequences

A big missing feature (and one definitely needed for Helm charts) was the ability to conditionally insert key/value pairs into mappings depending on some value being true or false.

Functionally that's a bit weird for a data language. You can always apply any function to any data node to change its value, but how do you make it control whether or not it exists at all?

You really need to apply a function to its parent mapping to make that happen…

…or do you?

YAMLScript ended up solving this using the :: syntax, with a special rule.

!yamlscript/v0:
foo: 111
::
  when a > b::
    bar: 222
baz: 333

The rule is that if the code under :: evaluates to nil then ignore it. If it evaluates to a mapping then merge it into the parent mapping.

The when function returns nil if the condition is false, and the body evaluation value if it is true.

To best understand this we can simply compile this YAMLScript to Clojure code.

$ ys -U --compile file.yaml  # or -c
(merge {"foo" 111} (merge (when (> a b) {"bar" 222}) {"baz" 333}))

You should know by now that every YAMLScript program is compiled to Clojure code and then evaluated. Well, data files that use YAMLScript are no different!

Given that merge ignores nil values, this is exactly what we want.

As we worked through the standard Helm templates we found that while this worked just fine, it was a bit verbose. We "fixed" that by letting you put the condition test "inside" the :: key:

!yamlscript/v0:
foo: 111
:when rand(100) > 50::
  bar: 222
baz: 333

I've changed a > b here to something you could actually run yourself. Note that before we never defined a or b, so that would have failed.

We can even get this into a single line by using YAML's flow style:

!yamlscript/v0:
foo: 111
:when rand(100) > 50:: {bar: 222}
baz: 333

Nice!

Now that you are up to speed, take a look at this page that shows how to completely convert a stock Helm chart to use YAMLScript instead of Go templating: https://yamlscript.org/doc/helmys

After the KubeCon we realized that this was also needed for sequences. You should be able to conditionally insert items into a sequence at any point.

All you need to do is use all of the above on a - ... sequence entry (returning a sequence or nil):

!yamlscript/v0:
- aaa
- :when rand(100) > 50::
  - foo
  - bar
- zzz

Again let's compile this to Clojure code to see exactly what it does:

$ ys -c file.yaml
(concat ["aaa"] (concat (when (> (rand 100) 50) ["foo" "bar"]) ["zzz"]))

Similar to the mapping case, but we get concat instead of merge to do the right thing with sequences.

Now we run it a couple times:

$ ys -Y file.yaml
- aaa
- zzz
$ ys -Y file.yaml
- aaa
- foo
- bar
- zzz

and Voilà!

Conclusion

I hope this post gave you some good ideas about how cleanly you can extend your YAML data files with YAMLScript. And also how this is applicable today in major YAML consumers like Helm.

Please let us know where YAMLScript can be made even better.

That's our goal!

Permalink

Testing out replicant

The SPA library landscape is pretty varied in Clojurescript, as long as you like using React underneath the hood. This particular fact is what makes replicant so interesting. Replicant has zero dependencies and is written in Clojurescript from the ground up.

Permalink

Clojure Deref (Nov 28, 2024)

Permalink

Conference Driven Developments

In my many years of creating Open Source software and talking about it at conferences, some of the most productive development times are those leading up to the presentation.

In the last post, I mentioned that I was going to present a 90 minute YAMLScript tutorial at KubeCon (November 15th in Salt Lake City).

At the start of October we realized there were a few things we wanted to add to the language to make it great for defining dynamic data in YAML files.

YAMLScript is a complete programming language and while you could already do almost anything with it, we knew that it had blend smoothly into the existing YAML files that people use for Kubernetes and other uses. We started by focusing on Helm charts, and seeing how well YAMLScript could fit along side the Go templating system (or replace it entirely). In the end it all worked out very well, but a few things needed to be added before tutorial time.

TL;DR for Helm Users: https://yamlscript.org/doc/helmys

In this post I want to cover some of the new (and old) features of YAMLScript that you can use to do cool things in your YAML files that are not possible with YAML alone. And remember, your YAML files are already valid YAMLScript files; but if you load them (or preprocess them) with YAMLScript, you can do a lot more in there.

What YAML Can Do Now

YAML 1.2 has a few things that let you do things a bit fancier than with a data only format like JSON:

• YAML has anchors and aliases for reusing specific nodes by naming them, and then later referring to them by name.
• YAML has a special << "merge key" that you can use to merge mappings. This was actually removed from the 1.2 spec but many popular implementations still support it (albeit with inconsistent behavior between implementations).
• YAML allows you to tag nodes and this can sometimes be used creatively, though not consistently across different YAML implementations.
• YAML supports multiple "documents" in a single file or stream, although unfortunately the spec doesn't allow you to alias nodes across documents.

YAMLScript supports all of these features too. It needs to since it claims to support the current YAML files of the world. But YAMLScript being a complete programming language lets you go so much further!

Let's take a closer look…

External (and Internal) Data Sources

Even though YAML lets you reuse nodes by name, those nodes need to be part of your YAML document. Say you have a section at the top of your YAML file that defines some default values and names them with anchors, to be aliased later throughout the file.

This is problematic because the node of defaults is also going to be a part of your data when you load it. It would be nice if you could have 2 documents in a YAML file where you define the data sources to be referred to in the first document, and then refer to them in the second document. If the loader returned the final (second) document then you could get the data you wanted without also getting the data that you don't.

Let's try it out:

# file.yaml
---
- &map1
  name: Bobbi Datamon
- &list1
  - fun
  - games
- more: stuff

--- !yamlscript/v0:
person:
  <<: *map1
  likes: *list1

Now we can load it with the YAMLScript command line interpreter, ys:

$ ys --load file.yaml
{"person":{"name":"Bobbi Datamon","likes":["fun","games"]}}

Looks like it worked, but ys --load prints the result in compact JSON. Before we discuss what happened, let's show the result in YAML:

$ ys --load --yaml file.yaml
person:
  name: Bobbi Datamon
  likes:
  - fun
  - games

Nice! ys --load gave us the data from the final document that included data from the first document.

from yamlscript import YAMLScript
ys = yamlscript.YAMLScript()
text = open("db-config.yaml").read()
data = ys.load(text)

The careful reader will have noticed that we broke the rules of YAML. We aliased nodes that were anchored in a different document. What's up?

Well, the two YAMLScript documents each have a different YAMLScript mode and that makes things work differently.

The first document has no !yamlscript/v0 tag and thus is in "bare mode". In bare mode all the rules are the same as YAML 1.2 (using the Core schema).

The second document has the !yamlscript/v0: tag. The !yamlscript/v0 tag tells YAMLScript that the document is in "code mode" (the content starts as code but can switch to data mode at any time). The : in !yamlscript/v0: tells YAMLScript to switch to data mode right away.

In code mode and data mode, aliases are very different than in bare mode. They can access anchored nodes in the same document or any previous document. Not only that, you can access parts of the anchored node with a path syntax. For instance *map1.name would produce the string "Bobbi Datamon" and *list1.1 would produce the string "games".

That means that this works the same way as the previous example:

--- &data
map1:
  name: Bobbi Datamon
list1:
- fun
- games
more: stuff

--- !yamlscript/v0:
person:
  <<:: -*data.map1
  likes:: -*data.list1

Here we only anchored the entire first document and then used the path syntax to access the parts we wanted. Note that to do this we used :: to switch to code mode and we also needed to used - to escape the values and have them be treated as YAMLScript expressions.

YAMLScript has special +++ symbol that evaluates to an array of all the prior documents in the stream. That means we don't need anchors at all:

map1:
  name: Bobbi Datamon
list1:
- fun
- games
more: stuff

--- !yamlscript/v0:
person:
  <<:: +++.last().map1
  likes:: +++.$.list1

The +++.last() function returns the last document in the stream so far (the first document in this case) The +++.$ is a shorthand for +++.last().

Another approach to take here is to make the first document use YAMLScript in code mode and define variables to use in the second document:

--- !yamlscript/v0
map1 =::
  name: Bobbi Datamon
list1 =::
- fun
- games

--- !yamlscript/v0:
person:
  <<:: map1
  likes:: list1

We use =: for assignment expressions in YAMLScript. And =:: does the same thing but toggles the mode of the value.

What if list1 was a huge list and you really wanted to keep it in a separate file?

No problem:

# big-list.yaml
- fun
- games
# ...

Now we just change the first document to load the list from the file:

--- !yamlscript/v0
map1 =::
  name: Bobbi Datamon
list1 =: load('big-list.yaml')

Not only can we access external data from a file, YAMLScript supports fetching data from the web with the curl function and also getting data from databases!

Inline Code in Data Mode

We can do the same things in a single data mode document. The trick is that we need to have a way to evaluate code in a way that doesn't affect the data.

The :: syntax is our new friend here.

This lets us do code things like define variables and even define new functions in a way that doesn't affect the data we are defining.

--- !yamlscript/v0:
::
  defn flip(array):
    reverse: array

  map1 =::
    name: Bobbi Datamon
  list1 =: load("big-list.yaml")

person:
  <<:: map1
  likes:: list1:flip

We defined a new function called flip which is a bit contrived since we could have called reverse directly; but it proves the point.

We also defined our data variables. We can actually define variables without :::

--- !yamlscript/v0:
map1 =::
  name: Bobbi Datamon
list1 =: load("big-list.yaml")

person:
  <<:: map1
  likes:: list1

In a big document, it's sometimes nice to define the data variables closer to where they are used.

--- !yamlscript/v0:
map1 =::
  name: Bobbi Datamon

person:
  <<:: map1
  list1 =: load("big-list.yaml")
  likes:: list1

In fact, since we only use map1 and list1 once, we could have just inlined them:

--- !yamlscript/v0:
person:
  <<:
    name: Bobbi Datamon
  likes:: load("big-list.yaml")

How Do I merge Thee?

Let me count the ways :)

We are still using the << merge key to merge mappings, but YAMLScript has a a standard merge function (among 100s of others).

--- !yamlscript/v0:
map1 =::
  name: Bobbi Datamon
list1 =: load("big-list.yaml")

person::
  merge::
  - ! map1
  - likes:: list1

Note the ! in front of map1. It toggles from data mode to code mode. We need to use ! for that purpose in data mode sequences. For mappings we can use key:: variable but it is just a shorthand for key: ! variable.

Another way to write that is:

person::
  merge map1::
    likes:: list1

Since the merge key is already in code mode we can just put the map1 variable there.

Sometimes you want to use a function like merge without needing to further indent the data you are applying the function to.

YAMLScript lets you put a function in a tag if you prefix it with a::

--- !yamlscript/v0:
map1 =::
  name: Bobbi Datamon
list1 =: load("big-list.yaml")

person: !:merge
- ! map1
- likes:: list1

In this case we can simplify it even more by adding : to the end of the tag, thus switching to code mode:

person: !:merge: -[map1, {'likes' list1}]

But I digress…

Conditional Insertions in Mappings and Sequences

A big missing feature (and one definitely needed for Helm charts) was the ability to conditionally insert key/value pairs into mappings depending on some value being true or false.

Functionally that's a bit weird for a data language. You can always apply any function to any data node to change its value, but how do you make it control whether or not it exists at all?

You really need to apply a function to its parent mapping to make that happen…

…or do you?

YAMLScript ended up solving this using the :: syntax, with a special rule.

!yamlscript/v0:
foo: 111
::
  when a > b::
    bar: 222
baz: 333

The rule is that if the code under :: evaluates to nil then ignore it. If it evaluates to a mapping then merge it into the parent mapping.

To best understand this we can simply compile this YAMLScript to Clojure code.

You should know by now that every YAMLScript program is compiled to Clojure code and then evaluated. Well, data files that use YAMLScript are no different!

$ ys -U --compile file.yaml  # or -c
(merge {"foo" 111} (merge (when (> a b) {"bar" 222}) {"baz" 333}))

Given that merge ignores nil values, this is exactly what we want.

As we worked through the standard Helm templates we found that while this worked just fine, it was a bit verbose. We "fixed" that by letting you put the condition test "inside" the :: key:

!yamlscript/v0:
foo: 111
:when rand(100) > 50::
  bar: 222
baz: 333

I've changed a > b here to something you could actually run yourself. Note that before we never defined a or b, so that would have failed.

We can even get this into a single line by using YAML's flow style:

!yamlscript/v0:
foo: 111
:when rand(100) > 50:: {bar: 222}
baz: 333

Nice!

Now that you are up to speed, take a look at this page that shows how to completely convert a stock Helm chart to use YAMLScript instead of Go templating: https://yamlscript.org/doc/helmys

After the KubeCon we realized that this was also needed for sequences. You should be able to conditionally insert items into a sequence at any point.

All you need to do is use all of the above on a - ... sequence entry (returning a sequence or nil):

!yamlscript/v0:
- aaa
- :when rand(100) > 50::
  - foo
  - bar
- zzz

Conclusion

I hope this post gave you some good ideas about how cleanly you can extend your YAML data files with YAMLScript. And also how this is applicable today in major YAML consumers like Helm.

Please let us know where YAMLScript can be made even better.

That's our goal!

Permalink

Heart of Clojure videos, attendee survey results, and diversity report 🎥 📊 💚

Hello Heart of Clojure 2024 Attendees,

Although Heart of Clojure was more than two months ago now, we are still working on documenting and evaluating our experiences there.

Read on below to get the news on our conference videos, attendee survey results, and diversity report.

Conference videos

For your viewing pleasure, we&aposve now published all 26 conference talks on Lambda Island&aposs Youtube Channel.

If you want to relive Lu Wilson&aposs call to action to being open with your (messy) work, here it is:

Attendee Survey Results

Thanks to all 56 of you who answered our Attendee Survey.

Getting your feedback and perspective on what worked and what didn&apost at our conference will help us make the next event better.

Also, it&aposs pretty gratifying to know that we did some things right!

If you&aposre curious about what other attendees thought, here&aposs a summary of the survey results:

Diversity Report

I wrote up a report on our diversity tickets program and speaker demographics in an attempt to make ourselves transparent and accountable to our commitments to diversity.

If you&aposre interested, the report is below.

I&aposm happy to hear your questions and constructive feedback about the diversity report or the attendee feedback results.

We are still working on sharing the photos and slidedecks from the conference and will be in touch soon!

Enjoy the last days of November!

Warmest greetings from Berlin,

Bettina Shzu-Juraschek

for the Heart of Clojure Orga Team

Permalink

Restrained Datalog in 39loc

by cgrand (🦋 🦣 𝕏)

Today, as promised, a follow-up to Half Dumb Datalog in 30 loc.

In the aforementioned article we complained that it was impossible for our Datalog to not conclude that Bart Simpson is his own sibling. To fix this situation we need to introduce constraints (= and not= for a start) into our toy Datalog to be able to state that Bart's sibling can't be "equal" to himself.

Don't forget: when we're not busy writing toy Datalog implementations, we are here to help: from letting you pick our brains for a couple of hours, to handling a whole project or writing a prototype or a MVP. Get in touch!
Else we are working on ClojureDart or on our app Paktol (The positive spending tracker where money goes up!)—a big feature is coming 🤫.

Expressing constraints

Before even considering implementation we should consider expression: currently we can't even state (not= x y), it would be interpreted as matching any triple, with not= being considered a variable name.

In the first installment of this Datalog series, I wrote:

A pattern is a vector of simple values (this time, including symbols for variables).

So match working on patterns represented by list is just an accident: let's reclaim lists for constraints!

A constraint is thus represented by a list whose first item is a symbol identifying the constraint type, further items are simple values (including symbols for variables).

Let's assume a constrain function taking two arguments [env constraint] and returning an environment with the constraint or nil when the constraint fails to apply.

Equipped with constrain, we can update match-patterns:

(defn constrain [env constraint] ; TODO
  nil)

(defn match-patterns [patterns dfacts facts]
  (reduce
    (fn [[envs denvs] pattern]
      (if (seq? pattern) ; 👈 if constraint, apply constraint👇
        [(->> @envs (keep #(constrain % pattern)) set delay)
         (->> denvs (keep #(constrain % pattern)) set)]
        [(-> #{} (into (for [fact facts env @envs] (match pattern fact env))) (disj nil) delay)
         (-> #{}
           (into (for [fact facts env denvs] (match pattern fact env)))
           (into (for [fact dfacts env denvs] (match pattern fact env)))
           (into (for [fact dfacts env @envs] (match pattern fact env)))
           (disj nil))]))
    [(delay #{{}}) #{}] patterns))

Representing constraints

Now we have to find a way to represent constraints inside environments maps.

Up to now, environment maps were mapping variable (symbols) to values. Now they are going to map to values or constraints sets. For simplicity we'll represent constraints sets by actual sets so it means that sets are not valid values any more -- we should generally consider that values can't be collections.

A key insight is that when one wants to add a constraint (say (not= a b)) to an environment then this constraint won't be evaluated until both a and b are bound. Thus we can bind the constraint to the first free (unbound) variable:

(defn constrain [env [op & args]]
  (let [args (map #(let [v (env % %)] (if (set? v) % v)) args)]
    (if-some [free-var (->> args (filter symbol?) first)]
      (update env free-var (fnil conj #{}) (cons op args))
      (when (apply (case op not= not= = =) args) env))))

Now we need to consider what will happen when a value is bound to a constrained variable but, first, let's extract bind out of match:

(defn bind [env p v]
  (let [p-or-v (env p p)]
    (cond
      (= p '_) env
      (= p-or-v v) env
      (symbol? p-or-v) (assoc env p v))))

(defn match [pattern fact env]
  (when (= (count pattern) (count fact))
    (reduce (fn [env [p v]] (or (bind env p v) (reduced nil)))
      env (map vector pattern fact))))

When we try to bind a value to constrained variable V, we find in the environment map a set of constraints whose first free variable is V (by definition of constrain). Thus if we bind V to an actual value, we can re-add all constraints (this will either assign them to other free variables or evaluate the constraint if all variables are bound):

(defn bind [env p v]
  (let [p-or-v (env p p)]
    (cond
      (= p '_) env
      (= p-or-v v) env
      (symbol? p-or-v) (assoc env p v)
      (set? p-or-v) (reduce constrain (assoc env p v) p-or-v)))) ; 👈 

If you are not convinced, see:

=> (constrain {} '(not= a b))
{a #{(not= a b)}} ; a is the first free var of the constraint, so it's the one being constrained
=> (bind *1 'a 42)
{a 42, b #{(not= 42 b)}} ; a is now bound and the partially evaluated constraint is put on b which has become the first free var
=> (bind *1 'b 33)
{a 42, b 33} ; success: b is bound to a value for which the constraint holds
=> (bind *2 'b 42) ; mind the *2: it's {a 42, b #{(not= 42 b)}} again
nil ; failure: b is bound to a value for which the constraint doesn't hold

It works!

We can now compute the siblings of Bart without having him among the results:

=> (q edb
     '([s] [:sibling :bart s])
     '[([:parent c p] [:father c p])
       ([:parent c p] [:mother c p])
       ([:sibling c c'] [:parent c p] (not= c c') [:parent c' p])])
#{[:lisa] [:maggie]}

I've deliberately put the constraint between the two "calls" to :parent to insist on the fact that its position doesn't matter, that we can constrain variables without knowing the evaluation order of the query. It's a desirable property as it makes the language more declarative.

What's next?

We were able to add declarative constraints to our toy Datalog without changing the code too much.

In next installments of this series I'd like to cover among other ideas negation, indices, magic sets and aggregations.

Negation and aggregation would enhance the expressiveness of our Datalog while indices and magic sets would make it practically usable.

If you like this series (or this newsletter in general) don't forget to share it widely.

Appendix: Showing some restraint in 39loc at once

(defn constrain [env [op & args]]
  (let [args (map #(let [v (env % %)] (if (set? v) % v)) args)]
    (if-some [free-var (->> args (filter symbol?) first)]
      (update env free-var (fnil conj #{}) (cons op args))
      (when (apply (case op not= not= = =) args) env))))

(defn bind [env p v]
  (let [p-or-v (env p p)]
    (cond
      (= p '_) env
      (= p-or-v v) env
      (symbol? p-or-v) (assoc env p v)
      (set? p-or-v) (reduce constrain (assoc env p v) p-or-v))))

(defn match [pattern fact env]
  (when (= (count pattern) (count fact))
    (reduce (fn [env [p v]] (or (bind env p v) (reduced nil)))
      env (map vector pattern fact))))

(defn match-patterns [patterns dfacts facts]
  (reduce
    (fn [[envs denvs] pattern]
      (if (seq? pattern)
        [(->> @envs (keep #(constrain % pattern)) set delay)
         (->> denvs (keep #(constrain % pattern)) set)]
        [(-> #{} (into (for [fact facts env @envs] (match pattern fact env))) (disj nil) delay)
         (-> #{}
           (into (for [fact facts env denvs] (match pattern fact env)))
           (into (for [fact dfacts env denvs] (match pattern fact env)))
           (into (for [fact dfacts env @envs] (match pattern fact env)))
           (disj nil))]))
    [(delay #{{}}) #{}] patterns))

(defn match-rule [dfacts facts [head & patterns]]
  (for [env (second (match-patterns patterns dfacts facts))]
    (into [] (map #(env % %)) head)))

(defn saturate [facts rules]
  (loop [dfacts facts, facts #{}]
    (let [facts' (into facts dfacts)
          dfacts' (into #{} (comp (mapcat #(match-rule dfacts facts %)) (remove facts')) rules)]
      (cond->> facts' (seq dfacts') (recur dfacts')))))

(defn q [facts query rules]
  (-> facts (saturate rules) (match-rule #{} query) set))

Permalink

The end of algorithms

Happy Thanksgiving! It’s this Thursday here in the US. I hope it finds you safe and peaceful.

Please check out the first chapters of Runnable Specifications. Someone just last week told me they didn’t realize you could already start reading it! He did and said it was just the book he was looking for. Oh, man. I rushed back home and started writing more. I need more feedback like that.

And if you want to read a completed book, Grokking Simplicity is just sitting there, waiting to be opened by you. If you already read it, tell the world you liked it by writing an Amazon review. It’s the best way to help me!

The end of algorithms

Algorithms don’t scale. Here’s a story to illustrate:

On Friday, I decided to spend some time accessing the ChatGPT API. I had a few tasks that I thought it would be good at, so I dove into it.

I had about 1k records that I wanted ChatGPT to process. I tried the stupidest thing that could possibly work, which was to submit them all at once, but that didn’t work well. So I developed an algorithm to process them one at a time, with some accumulating state that gets added to the prompt each time. My tests on the first 10 records showed signs of success, so I unleashed it on the whole 1k data set.

My algorithm stopped working. The logic of it was still sound, but now some other issues started popping up:

1. I was being rate limited. About 10% through, I was getting errors, which I was not handling.

2. There were transient failures occasionally.

3. When something did fail, I was throwing away all of the work.

4. Plus, I could track progress because now it was measured in minutes instead of seconds.

My simple accumulating loop, which treated the API like a function call, could not scale to 1k rows. And this is not the first time this has happened to me. My usual conclusion is “distributed systems introduce lots of complexity.” But today, I want to explore a different conclusion, “Algorithms don’t scale.”

One person that often talks about the problems with data structures and algorithms is Alan Kay:

The algorithms are kind of gear like, and the idea that you can smash values and data structures and get away with it implies that you're thinking small.

Part of the epiphany of Smalltalk for him was a related idea:

For the first time I thought of the whole as the entire computer and wondered why anyone would want to divide it up into weaker things called data structures and procedures.

And on how poorly algorithms scale:

Systems were starting to become much more interesting than algorithms…but trying to scale up algorithms you get a hundred million line operating systems like Windows XP.

Alan Kay has been and continues to be one of my biggest inspirations. I still learn stuff even from rewatching lectures I’ve seen a dozen times. I don’t know if it’s because his ideas are so far from my thinking that it takes me that many times to get it, or that he’s not that great at explaining things. Either way, he’s brilliant and I’m still learning.

He explains that even back in the 70’s and 80’s, people saw that systems thinking was how we could get better software. We had the example of ethernet and internet, both systems with no top-down algorithm. But languages that tried to lock down the difference between data and procedure—ALGOL and Pascal—won. Their descendants have still won today.

When Kay talks about data structures and procedures, it to me sounds very close to how Rich Hickey answered in this exchange:

Q: What needs to be the focus of programming language design in terms of concepts and constructs?

Rich Hickey: Functions and data. <drops mic>

And of course, my whole book about the functional programming paradigm divides the world into data, calculations (pure functions), and actions (impure functions). I obviously believe there is a lot of merit to the function/data view.

But is this the same as what Kay is talking about? I think we need to let our minds wander back to the 1960s to really see what he means, similarly to how when Kay says OOP, he doesn’t mean Java, which is what most people think of when they think OOP. The meaning has changed over time.

Back in the day, programs were often written to follow a complex layout in memory. There was a lot of pointer chasing. And people realized that if they organized the memory in certain ways, they’d get certain desirable properties. Some operations, like finding something in memory, or modifying something, would get faster. Or the data would take less memory. Codifying this knowledge begat data structures as a field of knowledge.

People were also realizing that certain problems were very common, such as sorting and search. If someone could just figure out the fastest way to sort, we could write down how to do it and share it. Likewise, if we wanted to find a way to traverse all the nodes in a tree, or find the shortest path, we could figure out the minimal solution. Those procedures we called algorithms.

And so programming was largely about traversing data structures in memory using algorithms to process what was found. And it sucked. The logic for what you wanted to do at each node was intertwined with the logic for traversing the data structure. If you needed to change the data structure, the entire algorithm needed to be reworked. What’s more, the programs were very hard to read and modify, let alone get right. In short, they didn’t scale.

Alan Kay was looking for something that would scale, in the same way that the internet could scale. Nodes are added and removed from the internet all day long without a bottleneck and without anyone having to understand its topology. Could he find something like that for software?

What he and his team came up with were what they called “objects”. Each object sent and received messages from other objects. The object had its own local process state that was inaccessible except through messages, which the object interpreted according to its code.

You could, of course, imitate data structures using objects, but the idea was larger. As your system scaled up, the semantics of your objects was maintained not by a better and better algorithm, but by a careful separation of concerns: Each object maintained its own invariants. Because of that, the senders to those objects could omit those concerns from their own code. So, as each semantic idea became more resiliently implemented and coherent, the users of that idea could relax their own discipline and rely on them more.

Unfortunately, when C++ and Java took on the ideas of OOP, they didn’t really convey this idea well. Classes became a convenient means of method dispatch—the mechanical “gear” idea pervaded. And perhaps it was a little more disciplined to keep the code that accessed and modified data right next to the data definition itself. But this idea of coherent objects maintaining their invariants from a chaotic outside world is only present in diffuse concentrations in the vast sea that is object oriented design advice.

Kay again:

The flaw in how things have played out is that very few in computing actually put in the effort to grok the implications of “universal scalable systems of processes”, and instead have clung to very old and poorly scalable ways to program.

I should mention that when I look at old Smalltalk code, the kind of stuff they were doing at Xerox PARC, it looks much more like what we would call functional programming today. It’s often recursive methods on tiny classes used like Algebraic Data Types.

And when I look at modern FP, it’s using ideas like reusable primitives that maintain invariants—almost like providing a service. The data structures we use in Clojure are not primitive like arrays. They guarantee a lot: immutability, structural sharing, and their participation in the core abstractions.

Here’s Alan Kay again:

It *is* like having independent subroutines that can call each other, but extended in the form of protected modules that provide “services”, and can do many helpful things internally and safely.

A well designed OOP system will feel as easy as doing a subroutine call for easy things, but can extend outwards to much more complex interactions.

For scaling etc. you want to have the invocation of “services” be a more flexible coupling than a subroutine call (for example, you should be able to do many other things while the service is happening, you shouldn’t have your control frozen waiting for the the subroutine to respond, etc.).

To wit: Kay’s OOP is like subroutine calls, but enhanced to make them work better at scale.

In other words, Java is fine, but when you put getters and setters for every field by default, you’re essentially making a mutable data structure. The encapsulation that was protecting the invariants cannot work anymore. And it becomes too easy to write large algorithms that control things from the outside.

Which brings me back to the original ChatGPT API problem I was having. My algorithm needed to be enhanced with services:

1. I needed a decent error-handling policy.

   1. I should retry transient errors.

   2. I should retry with exponential backoff when I get rate limiting backpressure.

2. I should centralize the use of the API to control the rate, but in a way that doesn’t limit how I write the program.

3. I want to maintain a running state of previously completed rows.

   1. To understand progress.

   2. To resume from later in the case of failure.

That’s a lot more stuff that I have to handle. If I try to do it as an algorithm, I’m sure I would succeed but it would be very intricate work. In an algorithm, I have to figure out a list of steps that will always succeed.

Instead, I could start to think of it as a system—relax my grip on the controls—and see it as pieces interacting. Do I need to know what order things happen in? Are there properties of the system I can guarantee with local behavior instead of with global behavior? These are the kinds of questions that start opening us up to scale.

Some additions to the discussion

On the usefulness and ubiquity of data

Rich Hickey and Alan Kay had an interesting discussion about the importance of inert data. It’s worth a read. I gave my commentary on it, then commented on my own comments. The discussion is that rich.

Relevant to the topic here, I think data is a powerful idea with a lot of pragmatic benefits. It is durable (you can easily store it or transmit it to another system) and cross-platform. It has a long history (thousands of years), so its properties are well-understood. Unfortunately, we do not have a practical way of running real software on another machine across time, so sending code is not an option.

However, I love the research question that Kay poses: Can we get to a bigger scale if we send an interpreter along with the data? Alan Kay has done some research into describing how to bootstrap the early Smalltalk that could be etched in stone.

Primitives

I personally believe that one of the failures of the Java ecosystem is to invest in educating programmers to separate concerns effectively. Instead, they blew their training budget on Dog isA Animal-style class inheritance. What we see is a bunch of programmers who cannot identify reusable abstractions if their lives depended on it.

In the functional world, we do look for those reusable abstractions. We appreciate the primitives that Clojure gives us, like atom and future, but we aren’t shy about making new ones. If there was something I could do to help Java programmers see that they do have the tools to build more abstract and scalable systems with less effort, it would be to teach them to recognize those seams for separating the problem.

Propagators

An interesting avenue I have explored and still think has a lot of potential is propagators. There are some algorithms that are just too hard to write and maintain. We need to intricately specify the order so that the right answer happens every time. Propagators give us a way to ignore the order.

Propagators do that by using associative and commutative functions that monotonically approach some answer. You form a network of these propagators, who share their current state with their neighbors. The neighbors combine those current answers to form their own current answer. Eventually, the network converges.

The nice thing about them is that they arrive at their answer without you ever having to know what order things happened in. You can reason locally about each connection in the network (make sure it’s correct), and the whole network just works to find the answer. You can add new nodes to the network and it should continue working.

Learn more about propagators from this talk by Gerald Sussman. And for a Clojure library that parses using a similar idea, see this talk by Mark Engleberg.

Data and algorithms in the small

We started by saying that algorithms don’t scale. But you need some algorithms, right? I think you do, to some extent. Even the definition for calculating the length of a linked list is an algorithm, even if it’s tiny:

(defn length [list]
  (if (empty? list)
    0
    (+ 1 (length (rest list)))))

What that means to me is that the systems thinking that Kay prefers to algorithms is built out of small algorithms. We can build a retry-with-backoff algorithm, which becomes a piece of the robust, scalable system we are ultimately building.

And I think this is also the answer that resolves the troubled discussion between Hickey and Kay: In the small, you need data and algorithms. But it only scales so far. At some point, you need to build up to systems-thinking. I happen to believe that Clojure and many contemporary languages do provide a lever for doing that better than starting from low-level languages. Clojure’s built-ins let you scale rather far—using just data and algorithms. But, as in any language, the free lunch comes to an end. At some point, you need to start finding new abstractions.

Probably the ultimate language for writing complex systems is Erlang—though I’m excited by some of the JVM developments following Virtual Threads, including structured concurrency and scoped values, which should close some of the gap between Erlang and the JVM.

Do type systems help with scale?

Alan Kay’s words:

[Work was done on Algol and later C and Pascal for] a type system that could deal with parameter matching of polymorphic procedures to new data types.

A big problem was that “data” could move around and be acted on by any procedure, even if the procedure was not helpful or at odds with the larger goals. “Being careful” didn’t scale well.

A lot of people talk about type systems helping deal with software as it scales. And I know what they mean—it still might be hard to understand or change, but at least the machine is telling you it doesn’t fail in certain ways.

I think what Alan Kay means by this statement is that people noticed this scaling issue—lots more type errors as you scale—and addressed it in an incremental fashion by having a type checker by your side. But this incremental improvement did not really scale that much farther. The real scaling came with time-sharing operating systems. Kay continues:

Meanwhile, time-sharing and multiprocessing OSs were being developed, and “being careful” did not work at all. Instead, the decision was rightly made to protect entities from each other — and themselves — via hardware protection mechanisms. This allowed processes made by many different people to coexist while being run, and it also allowed some processes to be “servers” — to provide “services” — to others.

Processes were software manifestations of whole computers — containing both processing and state — both hidden and protected.

In other words, scaling came from writing independent processes that communicated, not by locking down the correctness of the code.

More:

Now the thing to realize is that this — whole processes offering protected services — is really a good idea at any scale. First it allows much larger and more elaborate services to be done safely.

But it also makes things that weren’t safe enough at the line by line programming scales to become much more safe.

Even faulty programs can be useful if you can run them in a bubble.

Conclusions

1. Algorithms don’t scale. That’s why code starts to suck as it gets bigger.

2. We need to take global invariants and turn them into locally guaranteed invariants. This is hard, but it’s the essence of software design.

3. We build complex systems out of many small algorithms.

Rock on!
Eric

Permalink

Vote on 2025 Long-Term Funding Applications

Greetings Clojurists Together Members! This year, we are trying a new process to determine who will receive a monthly stipend of $1,500 to support their development work.

We put out the call - and we received 18 thoughtful applications for you to consider. You can also review past long-term project updates on our website to get an idea of what past long-term grant recipients have been able to accomplish with your past support (pretty amazing!)

Please review the applications below and select 5 developers to receive funding in 2025. Be on the lookout for an email that contains the link to your Ranked Vote ballot. Here we go ..in alphabetical order (by last name)….

Deadline for your vote to be counted: December 4, 2024 Midnight Pacific Time

-Michiel Borkent
-Thomas Clark
-Jeremiah Coyle
-Toby Crawley
-Eric Dallo
-Rafal Dittwald
-Dragan Djuric
-Robert Gersak
-Kira Howe (McLean)
-Roman Liutikov
-Mateusz Mazurczak and Anthony Caumond
-Jack Rusher
-Daniel Slutsky
-Adrian Smith
-Peter Strömberg
-Dan Sutton
-Peter Taoussanis
-Oleksandr Yakushev

Michiel Borkent

https://github.com/borkdude

What do you plan to work on throughout the year? My main projects are clj-kondo, babashka / SCI, and squint/cherry. I will develop new features, specified below per project. As usual I’ll also work on ongoing maintenance and offering support on Slack and Github.

Links: clj-kondo, babashka, SCI, squint

Clj-kondo

• Linter for partially extended protocols
• Extend :discouraged-var to allow specifying arities
• Ongoing maintenance, there’s always plenty to do! See the project board and see here for a list of most upvoted issues
• Experiment with adding the notion of a project classpath such that command line linting becomes easier. Possibly configurations could be read from the classpath as well.
• As clj-kondo is the foundation for the static analysis in clojure-lsp: improvements in the interaction between the two, most specifically the analysis data.

Babashka / SCI

• Maintenance of included bb libraries: SCI, babashka.fs, babashka.process, deps.clj, babashka.http-client, edamame
• Babashka tasks: prefix parallel prcoess output such that you can distuingish tasks
• Better error locations for clojure.test output (https://github.com/babashka/babashka/issues/1518)
• Report exception causes in console error report (https://github.com/babashka/babashka/issues/1515)
• Several other issues: https://github.com/babashka/babashka/issues

Squint (and cherry)

• Browser REPL support
• Source map support (some work has been done, but far from finished)
• Increase overall compatibility with CLJS
• Support dependencies from the ClojureScript ecosystem from clojars and git depsAll other related projects
• Nbb, jet, scittle, neil, edamame, rewrite-edn, carve

Why is this work important to the Clojure community? Babashka is used by 90% of Clojure Survey respondents as shown in Alex Miller’s Conj 2024 talk. Babashka and clj-kondo have 4.1k and 1.7k stars on Github respectively, their slack channel on Clojurians have almost 1800 + 1000 users. Clj-kondo and babashka are used by a large portion of the community. I think it’s fair to say they are established project.

Is there anything else you would like us to know? Thank you so much for Clojurists Together and keeping the Clojure OSS ecosystem sustainable!

Thomas Clark

https://clojurians.zulipchat.com/#user/386018

What do you plan to work on throughout the year? In a nutshell, my plan would be to help expand Clojure’s scientific ecosystem, particularly in regards to the mathematical sciences. This would take the form of developing and creating libraries, continuing and creating documentation and to initiate a serious attempt at academic outreach.

Libraries

Wolframite

Last year, CT supported Jakub Holy and I for a quarter in our attempt to resurrect and document the Wofram-Clojure bridge. We’re very happy with the progress we made in this time, but of course there are many other things that could (and should?) be done - we really feel like we’re just getting started. In particular, a key feature of Wolframite, that is missing in Wolfram itself, is the REPL experience.

In this, we want to fully integrate Wolframite with Clojure’s visual tools for all manner of datatypes that Wolfram supports. And in particular, to make data passing and memory management efficient. More functionally, we want to create a special viewer for symbolic expressions, that not only allows the user to copy and switch between maths in Wolfram, LisP and TeX forms but that automatically generates sliders for each parameter for exploration, as inspired by Wolfram’s Manipulate function.

exp (name to be decided)

In Clojure, there are now two libraries to interact with symbolic expressions and this new project would seek to integrate them: both Wolframite and Emmy. This would be powerful because although Emmy brings symbolic manipulation to Clojure and Clojurescript simultaneously, it is still missing many key features and algorithms that only Wolfram can provide. One can then imagine a situation where mathematical problems could be defined and explored in the browser (Emmy), passed to the server to be simplified algebraically (Wolframite) and then crunched numerically (Emmy).

Not only would this bridge two existing libraries but it might even lay the foundation for developing a standard for representing equations more generally.

SciCloj

More generally, there are many SciCloj libraries that I have been involved with and so with funding I would happily step up my contributions.

I can particularly see a lot of scope in the further development of Clay. I have particular notions regarding it’s use for generating presentations and can imagine creating a specific API around this purpose. I can also imagine it being the first visual tool with which I could create the special equation viewer mentioned above.

Also in regards to tableplot. Specifically, I would like to be involved in its generalization to deal with 3D plotting as I have a lot of specific use cases in this area.

Another area of interest at the moment is in improving the SciCloj API for working with common matrix operations (specifically the generation and manipulation of Hadamard-like matrices). This is an open area and it’s not clear whether this will be abstracted into a separate library or wheter this would be an extension of dtype-next.

Documentation

Another big area of focus would be improving SciCloj documentation, at both the library and project level.

Libraries

Putting aside the libraries that I have contributed code to, there are many SciCloj libraries that I use that I would be happy to contribute documentation. A bridging example would be regarding a Clojure-Blender bridge. Although I started to develop a new library for this, basilisp-blender has emerged as potentially a stronger candidate and so I would like to make detailed tutorial examples of creating ‘scientific’ outputs using this.

Projects

As discovered while writing the Wolframite documentation what is often needed is not so much a manual but examples of how to use the tool (or how to integrate many tools) in a real project. With this in mind, I would like to document the overall process of how to use the ecosystem to solve real problems, supplying both real problems and real solutions, e.g. the source and details behind my past talk and large contributions to noj.

Outreach

Outside of building library bridges to Clojure for specialist academic projects as well as collecting SciCloj talk sources in a centralised location, I would make a concerted effort to reach out to non-Clojure scientists about the benefits of Clojure.

Where I might have an advantage here is that I am a scientist before I am a programmer and so have direct connections into a world where very few people have heard of the language. With funding (see the biases section for why) I could potentially give department seminars at different institutions as well as international and online conferences: using existing academic contacts.

Furthermore, something that I am particularly excited about, I have started to initiate a grouping of research-minded Clojurians to consider publishing papers in the field. With the right support, I think this could really help raise awareness.

Why is this work important to the Clojure community?
I guess I should really have checked the following questions before filling up the previous box… :)

In regards to library creation and expansion, I think that scientific computing is an exciting growth area for Clojure. It is an area that really benefits from Clojure’s key principles and one that already has a growing number of users: users who will benefit from the changes. It is also an area that is at a tipping point in regards to reaching practical parity with other key languages and the tools that I would like to focus on are not so much now about matching competitors, but rather about providing new features and features that lead towards a completely integrated ecosystem: something that I would like to find, no matter which language provided it!

The Emmy system in particular is bringing open-source symbolic computation to both the back- and front-ends but it is missing key features and advanced libraries. A Wolfram-Emmy bridge could serve as a sure foundation and help create the real possibility of an almost unique physics programming space.

This work would also be important in regards to expanding the community itself. Following on from above, there is a large section of the numerical scientific community who are not programmers but who rely on tools like Mathematica and Matlab and so interop in these areas will be crucial for community cross-over in the future. Generalized language interop is particularly important for safe onboarding of new users and experience suggests that there is a willing ‘market’ for integrating specialist tools within more comfortable general languages like Clojure.

Beyond this however, the interactive development and documentation experience that is available now is a solution that simply needs to be shown to scientists' problems. The proposal to organise academic publishing and presentation of what the Clojure experience is like feels like an important stepping stone to the future.

Are you part of a group that is affected by systemic bias, particularly in technology? If so, can you elaborate?
Depending on what you mean by ‘systemic bias’, my disadvantage is a classical one. Living in eastern Europe and working in the public sector, I suspect my Clojure friends would be shocked at what is paid to scientists here. I don’t pretend that Hungary is as cash-strapped as some other continental countries, but compared to most other countries in ‘the west’, even those a two hour drive away, the salaries are low. If it helps, I can be more specific, but suffice it to say that this funding would go much further here and would enable a much bigger shift in what I’m able to contribute than it would for many other applicants.

Jeremiah Coyle

https://github.com/paintparty

What do you plan to work on throughout the year?
These are the 5 projects that I am planning to work on throughout the year :) All are fairly mature alpha-stage projects. Kushi and Fireworks have already been announced. I plan to announce the latest Fireworks release, and the other 3, over the next few weeks.

#1 - Kushi

Kushi is a base for building web UI with ClojureScript.

Goals for Kushi in 2025:

• Universal transpiler for kushi UI components to automatically generate components for established ClojureScript UI Rendering frameworks such as Uix, Helix, Biff, shadow-grove, etc.
• Add a quickstart repo for each supported framework Uix, Helix, Biff, shadow-grove etc., with a deluxe todo-mvp/kitchen sink example set to make it very easy to cut and paste working UI.
• Browser-based interactive design tweaking, with round trip to file system.
• Add more functionality, docs, and examples for working with the kushi.playground namespace. This gives the user a very elegant way to build their own white-labeled interactive documentation site that features Kushi’s components as well as any custom components specific to their project. It is similar to Storybook.js or Portfolio (cjohansen/portfolio). The playground namespace was used to create the current project website at kushi.design.
• Design and implement 4-5 global themes, using a variety of styles. In order to show off the power of the new theming system, I would like to add at least a couple very playful themes (sci-fi, fantasy-rpg) and incorporate them via a theme switcher into the kushi.design site.
• Add additional ui library components. Components followed by an asterisk are internally complete and just need to be turned into public functions with documentation:
  • Avatar*
  • Tabs*
  • Blockquote
  • Treeviewer
  • Carousel
  • Dropdown menu
  • Context menu (right-click)
  • Table
  • Data List
  • More card layouts (with inset margins)
  • Keyboard input / hotkey e.g. ⌘ C styled as button
  • Aspect Ratio (displays content within a desired ratio)
  • Wrapper for native color picker input

#2 Domo

Domo is a modern ClojureScript DOM utility library.

Goals for Domo in 2025:

• Curate the current API of approximately 75 public functions
• Write docstrings for all public functions
• Add simple validation and dev-time warnings for all public functions
• Figure out best approach for automated test suite
• Leverage kushi.playground to create interactive documentation

#3 - Fireworks

Fireworks is a themeable tapping library and color pretty-printing engine.

Goals for Fireworks in 2025:

• Solidify current public API
• Address all 12 current issues (mostly enhancements)
• Publish editor plugins/extensions for Emacs and VS Code. These are fairly simple extensions that just involve some basic clj-rewrite functionality for wrapping/unwrapping forms. I’ve already created initial working versions of both (emacs and VSCode) locally.
• Produce written and/or video documentation of my current live hot-reloading dev environment for JVM Clojure, with versions for both Leiningen and Deps. I recently issued a PR to add this to test-refresh. This sort of thing could also potentially be incorporated into other similar projects such as metabase/hawk and tonsky/clj-reload.
• For ClojureScript developers using Fireworks in a browser dev console, I went off the deep-end and made a dedicated Chrome extension to enable the setting of the Chrome DevTools console background and foreground color with a very nice GUI interface. Would be cool to get this working in most other Chromium-based browsers, and potentially Firefox, if there is any demand for it.
• https://github.com/paintparty/fireworks?tab=readme-ov-file#setting-the-background-color-and-font-in-chrome-devtools-clojurescript

#4 - Bling

Bling is a library for rich text in the console

Goals for Bling in 2025:

• Support automatic detection of the 3 levels of color support (16-color, 256-color, or Truecolor), using an approach similar to https://github.com/chalk/supports-color
• Support arbitrary hex colors, and their conversion, if necessary, to 256. I’ve already created an efficient Manhattan distance algorithm for doing this conversion in Fireworks
• Create more formatting templates for callouts, headers, and points-of-interest
• Add documentation about how to leverage Bling to create great-looking warnings and errors in any project. Example of using bling’s templates to create nice warnings can be found here:
• Add documentation about using Bling in conjunction with existing libraries which format Spec and Malli messages into human readable form.
• Although lower priority, it would be fun to explore enhancing the flexibility and utility of bling.core/stack-trace-preview, and exposing it as a public function, as it may be a cheap way to get give some additional context to many kinds of error messages.

#5 Lasertag

Lasertag is a utility for categorizing types of values in Clojure, ClojureScript, and Babaskha.

Goals for Lasertag in 2025:

• Solidify current public API
• Extensive coverage for all Java and Javascript types/classes
• Extensive test suite for all types and classes
• Leverage kushi.playground to make cool demo for usage in ClojureScript

Why is this work important to the Clojure community?
I believe work on these projects could benefit the Clojure community for the following reasons (listed out project-by-project):

#1 - Kushi
Kushi aims to provide a complete solution to the design layer of web UI projects.

It is an ambitious project, but I believe there is an opportunity for Kushi to offer a UI dev experience that is unique and more compelling than any equivalent in any other language. This could lead to increased organizational awareness and consideration of ClojureScript as an attractive choice for building mission-critical UI. If more companies used ClojureScript to build UI, I think it would be very beneficial for the community.

#2 - Domo
This lib has been slow-cooking over a couple years to serve various web projects, including Kushi.

The syntax is very Clojure-y, and feels much nicer than writing gnarly JS interop code. To my knowledge, there are only a few ClojureScript DOM libs (Dommy, Enlive, Enfocus), and all were written 9-10 years ago.

Compared to these libs, Domo offers even more specific functionality such as:

• copy-to-clipboard
• Getting viewport information
• Geometry for elements
• Getting screen quadrant for element or event
• Attribute querying and manipulation
• Computed styles of elements
• Adding, removing, and toggling attributes and classes
• Optional zipper-like syntax for selecting elements
• matches-media? helpers
• a11y-friendly on-mouse-down helper (faster alternative to on-click)
• Helper for keyboard-based tab navigation
• & much more!

#3 - Fireworks

Fireworks is the only lib to provide a colorizing and themeable pretty-printer that works beautifully in both Clojure and ClojureScript (and Babashka).

The output is, arguably, orders of magnitude faster and easier to read than equivalent output from clojure.pprint/pprint. This is especially true in the case of maps with data-structures as keys, or any kind of collection that features metadata.

The library also provides very powerful debugging and tapping macros. These macros provide a lightweight complement to discovery-centric, UI-heavy data exploration tools such as Portal and Flowstorm. With a simple hot-reloading setup (in clj or cljs), and minimal editor integration, Fireworks can drive an extremely compelling live-feedback dev experience without any reliance on a repl connection, or repl-related concepts. I think the continued codification, documentation, and demonstration of such a workflow could make a difference in bringing new people to Clojure. The maturation of this kind of workflow could also benefit existing users of community tools such as playback, debux, hashp, telemere, ken, spyscope, omni-trace, postmortem, and sayid.

#4 - Bling

Many mature language communities have a library for rich text printing in the terminal, for example Rich (Python), Chalk (JS), LipGloss (Go), etc.

With Clojure, the main existing option seems to be org.clj-commons/pretty. The original impetus for creating Bling came out of my experience of trying to use this lib extensively for formatting error and warning messages.

The most unique thing that Bling offers is a carefully curated palette of 11 basic colors which are readable on both light and dark backgrounds. Because these colors are located in the 16-255 ansi range, they are guaranteed to look the same on almost any terminal, regardless of the user’s theme. Most libraries leave the colorizing up to the user’s emulator profile/theme, which often leads to wildly different colorized output in the user space.

Bling also offers two simple functions for creating blocks of text in the console. Check out the readme for exhaustive visual examples. These can be semantically colored (errors, warnings, info). Bling also offers a simple function for constructing a point-of-interest diagram, like when you want to call out the namespace, line, column and show some source code with a red squiggly underline. I think the Clojure community would benefit from more library authors adding neatly-formatted, actionable warning and error messages to their codebases. It would also be great if such warnings and messages where formatted in a way that loosely followed some set of conventions. Perhaps Bling could play a small role in fostering this within the Clojure community.

#5 - Lasertag

Lasertag is a utility library that was spun out of Fireworks. I believe it to be quite unique in the current landscape of cljc libraries that deal with reflection and categorization of values.
Lasertag makes it easy to get detailed information about values, particularly in the context of interop with Java or JavaScript. See this section of the readme for an example.

I think that Lasertag could potentially offer great utility to rich-tooling projects developed in the Clojure community of the present and future.

Are you part of a group that is affected by systemic bias, particularly in technology? If so, can you elaborate? Although I did not study computer science, I do think of myself as an individual with a unique perspective and something to offer.

Toby Crawley

tobias

What do you plan to work on throughout the year?
https://github.com/clojars/

Why is this work important to the Clojure community?
Clojars provides critical infrastructure to the Clojure community. This funding would cover routine maintenance and adding new features as needed.

Eric Dallo

@ericdallo

What do you plan to work on throughout the year? I want to dedicate more time on big features for anything related to the Clojure LSP ecosystem, this includes: clojure-lsp and all integrations with editors, intellij, calva, lsp-mode, and so on. Also, I want to improve the IntelliJ experience to be as good as Cursive, improving clojure-repl-intellij and clojure-lsp-intellij projects so we can have good options for LSP in any editors.

Why is this work important to the Clojure community?
Because with that everyone coding Clojure will be using the same features improving dev xp.

Rafal Dittwald

https://github.com/clojure-camp/

What do you plan to work on throughout the year?

Clojure Camp

My primary focus will be on our topic map, particularly, creating an interface to add Clojure resources from around the web and link them to learning outcomes. Later, crowd-sourcing submissions and doing some of that linking manually. Also exploring using LLMs to automate of that process.

Additionally, I would like to mentor community contributions to other resources at Clojure Camp: developing faded example / parson’s problems for our library of exercises; a badge system; and improvements to our pairing scheduling tool and mob programming tool. I will offer up some of the funding as bounties for our community members.

And of course, I will continue to run regular mob sessions and stewarding the community.

I maintain a list of “TODOs” in this spreadsheet:

Jordan and I presented a Clojure Camp update at the Conj.

Why is this work important to the Clojure community?
Clojure has sustained itself, so far, as a language (and community) “for experts, by experts.” But for long term sustainability, we should be open to exploring other avenues for people to enter the Clojure-verse.

Clojure Camp aims to offer a learning experience to support would-be Clojure developers currently under-served by existing offerings. We achieve this by:

• translating best practices of pedagogical research of teaching programming to Clojure
• fostering a welcoming community of learners and mentors
• providing supportive infrastructure to motivate learning
  

Dragan Djuric

https://dragan.rocks/

What do you plan to work on throughout the year?
My goal with this funding in 2025 is to support Apple silicon (M cpu) in Neanderthal (and other Uncomplicate libraries where that makes sense and where it’s possible).

This will hugely streamline user experience regarding high performance computing in Clojure for Apple macOS users, which is a considerable chunk of Clojure community. They ask for it all the time, and I am always sorry to tell them that I still don’t have a solution. Once upon a time, Neanderthal worked on Mac too, but then Apple did one of their complete turnarounds with M1… This basically broke all number crunching software on macs, and the world is slow to catch up.

Several Clojure developers started exploring high performance computing on Apple, but didn’t get too far; it’s LOTS of functionality. So, having Neandeathal support Apple would enable several Clojure data processing communities to leapfrog the whole milestone and concentrate on more high-level tasks.

This affects Neanderthal (matrices & vectors), and Deep Diamond (tensors & deep learning) the most. Nvidia’s CUDA is not physically available on Macs at all, while OpenCL is discontinued in favor of Apple’s proprietary Metal (and who knows what else they’ve came up with since).

Neanderthal is a Clojure library for fast matrix computations based on the highly optimized native libraries and computation routines for both CPU and GPU. It is a lean, high performance, infrastructure for working with vectors and matrices in Clojure, which is the foundation for implementing high performance computing related tasks, including, but not limited to, machine learning and artificial intelligence.

Deep Diamond is a tensor and deep learning Clojure library that uses Neanderthal and ClojureCUDA under the hood (among other things).

So, what are the missing parts for Apple silicon?

1. A good C++/native interop. That is more or less solved by JavaCPP, but their ARM support is still in its infancy, especially regarding distribution. But it is improving.
2. A good BLAS/LAPACK alternative. There’s OpenBLAS, and there’s Apple’s Accelerate. Both support only a part of Intel’s MKL functionality. But, if we don’t insist on 100% coverage (we’re not) and are willing to accept missing operations to be slower, I could implement the most important ones in Clojure if nothing else is available.
3. A good GPU computing alternative. CUDA is not supported on Apple, and OpenCL has been discontinued by Apple. So that leaves us with Apple’s Metal, which is a mess (or so I hear). So I wouldn’t put too much hope on GPU, at the moment. Maybe much, much, later, with much, much, more experience…
4. Assorted auxiliary operations that are not in BLAS/LAPACK/Apple Accelerate, which are usually programmed in C++ in native-land. I’d have to see how many appear, and what I have to do with them.
5. Explore what’s the situation related to tensors and deep learning on Apple. I doubt that Intel’s DNNL can cover this, but who knows. Also, Apple certainly supports something, but how compatible it is with cuDNN and DNNL, is a complete unknown to me…
6. Who knows which roadblocks can pop up.

So, there’s a lots of functionality to be implemented, and there’s a lots of unknowns.

I propose to * Implement an Apple M engine for Neanderthal.* This involves:

• buying an Apple M2/3 Mac since I don’t have it (the cheapest M3 in Serbia is almost 3000 USD with VAT).
• learning enough macOS tools (Xcode was terrible back in the days) to be able to do anything.
• exploring JavaCPP support for ARM and macOS.
• exploring relevant libraries (OpenBLAS may even work through JavaCPP).
• exploring Apple Accelerate.
• learning enough JavaCPP tooling to be able to see whether it is realistic that I build Accelerate wrapper (and if I can’t, at least to know how much I don’t know).
• I forgot even little C/C++ that I did know back in the day. This may also give me some headaches, as I’ll have to quickly pick up whatever is needed.
• writing articles about relevant topics so Clojurians can pick this functionality as it arrives.

It may include implementing Tensor & Deep Learning support for Apple in Deep Diamond, but that depends on how far I get with Neanderthal. I hope that I can do it, but can’t promise it.

By the end of 2025, I am fairly sure that I can provide Apple support for Neanderthal (and ClojureCPP) and I hope that I can even add it for Deep Diamond.

Projects directly involved: https://github.com/uncomplicate/neanderthal
https://github.com/uncomplicate/deep-diamond
https://github.com/uncomplicate/clojure-cpp

Why is this work important to the Clojure community?
This will hugely streamline user experience regarding high performance computing in Clojure for Apple macOS users, which is a considerable chunk of Clojure community. They ask for it all the time, and I am always sorry to tell them that I still don’t have a solution since I don’t have a recent Mac hardware. Once upon a time, Neanderthal worked on Mac too, but then Apple did one of their complete turnarounds with M1… This basically broke all number crunching software on macs, and the world is slow to catch up.

Several Clojure developers started exploring high performance computing on Apple, but didn’t get too far; it’s LOTS of functionality. So, having Neandeathal support Apple would enable several Clojure data processing communities to leapfrog the whole milestone and concentrate on more high-level tasks.

Are you part of a group that is affected by systemic bias, particularly in technology? If so, can you elaborate?
The major obstacle that I have is that I live in a country outside USA and EU, so most work opportunities are unavailable from here (Serbia is a small country with poor economy that is still recovering from civil wars and whatnot, and the local IT market is mostly based on outrourcing). Other than that, I guess I’m in a similar position like everyone else.

Robert Gersak

https://github.com/gersak

What do you plan to work on throughout the year? https://github.com/neyho/eywa-core

Why is this work important to the Clojure community?
This project is about providing sane approach to identity access management based on OAuth2.1 specification and OpenID connect, specification that is common standard for identity management across professional services. ( it is important for everyone)

In addition, this project provides Data Modelling with out of the box GraphQL generic API exposure, that can be managed and controlled through UI. In terms of modelling entities and relations and what role can read, write, delete or own some entity or relation.

Currently this is applicable to PostgreSQL DB, but in future more similar DB will be covered.

Summary would be that this project is well suited for rapid development as well as for minimizing maintenance and change cost. It is fusion of IAM, Data Modelling and replacement for CRUD approach.

Is there anything else you would like us to know? I would like to share that IAM has been one of important subjects when working on enterprise level projects. Project applied above helped us in many ways to overcome enterprise level presence and onboard people towards common goal through usage of Data Modeling

Kira Howe (McLean)

http://github.com/kirahowe

What do you plan to work on throughout the year? I would like to continue working on stewarding Clojure’s data science ecosystem into a state of maturity. This would include specifically working on developing ggclj (a grammar of graphics implementation in Clojure – https://github.com/kirahowe/ggclj) guides and tutorials for Clojure’s data science stack (including the Clojure data cookbook and Clojure Tidy Tuesdays,https://github.com/scicloj/clojure-data-cookbook or https://github.com/kirahowe/clojure-tidy-tuesdays and conference talks. I would like to pitch some talks to broader data-focused conferences in 2025 outside the Clojure community in hopes of reaching a broader audience. These would include at least PyData Global, Fossdem, Open Data Science Conference, and Lambda world.

Why is this work important to the Clojure community?
I think this work will help expand Clojure’s user base and reach new audiences who can benefit from the unique tools and approaches Clojure brings to some of the biggest problems in data science today.

Are you part of a group that is affected by systemic bias, particularly in technology? If so, can you elaborate? I’m female and feel women are underrepresented in tech and especially in open source.

Roman Liutikov

https://github.com/roman01la

What do you plan to work on throughout the year? https://github.com/pitch-io/uix

Why is this work important to the Clojure community? UIx is one of the most dev friendly React wrappers, currently used by a number of companies known in Clojure community, including Pitch, Metosin and Cognician. UIx builds on modern React and encourages to rely more on existing React ecosystem in JS world. The library was in particular developed to be beginner friendly and close to React conceptually so that Clojure companies could onboard JavaScript developers quicker, allowing them to reuse most of their experience from JS/React world. I think UIx succeeded at this, at least judging based on feedback from companies mentioned above.

Both Pitch and Cognician hired frontend folks from JS community and they were able to pick up UIx relatively quickly. I also gave a talk on UIx at London Clojurians
I have a couple of ideas on improving the library further, preparing for React v19, also documentation needs more work and interactive examples. In general today after some years UIx is pretty stable, now I want to invest more in learning materials and templates for various types of projects, to make the library more accessible to devs.

Mateusz Mazurczak and Anthony Caumond

https://github.com/Kaspazza
https://caumond.com/#/

What do you plan to work on throughout the year? We are planning to work on discrete event simulation and discrete optimization.
Planning to add:

• A data oriented language to be able to model simulation by non-technical people.
• Add scenario management for creating multiple simulations.
• Add rendering module to give the highest possible quality insights to the modeler about the behavior of a model

Project name: Automaton simulation discrete event
Link: https://github.com/hephaistox/automaton-simulation-de

Why is this work important to the Clojure community?
First, as a little introduction:
We have over two decades of expertise in the supply chain on board and a decade of software development with software craftsmanship in our hearts. We plan to create it as an open-source alternative to existing solutions (like Rockwell Arena https://www.rockwellautomation.com/en-us/products/software/arena-simulation.html or https://www.anylogic.com/

With the difference, that thanks to Clojure and data-centric approach our software will be at much higher flexibility.
This is crucial for this kind of software, as each supply-chain industry’s needs for simulation are different. So current off-the-shelf software is either not flexible enough for their usage or to expensive to adapt (ranging in millions of euro).

Future development of this library can bring more interest in Clojure for supply-chain-related projects (especially since we are putting related topics to separate libraries, e.g., https://github.com/hephaistox/automaton-optimization

By being highly flexible in architecture, other people can extend this library to meet their needs. This can benefit college students of the supply chain and people working in supply-chain (e.g. warehouses, factories). Especially when their company or revenue is too small to require consultants and high-cost software.

This will be also a great showcase for clojure data-oriented approach, as this kind of scheduling systems are much harder to create to be so flexible using classical languages (java, c++) while keeping costs small (small team).

Is there anything else you would like us to know? As automaton-simulation-de is built with a non-monolith approach, each of its parts could be seen, on its own as a product providing value to the community.

For example:

• PRNG and probabilistic distribution clojure / clojurescript lib
• Technology agnostic discrete optimization library (random walk, descent, genetic algorithm, …)
• Scenario management,
• Scheduler engine
• Data-oriented UI components library (separating well the HTML+css part, behavior, and where the setup data comes from).
• Or other libraries that are already separated as a side-effect of the main work e.g. https://github.com/hephaistox/automaton-build for working with monorepo
  

Jack Rusher

https://github.com/jackrusher

What do you plan to work on throughout the year? I would like to devote a certain number of hours every month in 2025 to work on Clerk.

Why is this work important to the Clojure community? Clerk is used by a large number of community members for literate programming, data analysis and visualization, &c.

Daniel Slutsky

https://github.com/daslu

What do you plan to work on throughout the year? Scicloj https://github.com/scicloj/
Scicloj community building and open-source projects

Why is this work important to the Clojure community? Clojure can be a fantastic fit for many kinds of scientific and data-intensive projects. Fulfilling that potential is a years-long effort, on the technical as well as the community side.

Since early 2019, we, the Scicloj group, have been working on creating a stack of tools and libraries for data and science in Clojure. https://scicloj.github.io/

The stack we are building is addressing many needs which are useful non only in scientific context. High-performance computing, data processing, data visualization, literate programming, and literate testing are a few examples.

You may find more details about current developments in a few recent talks by Scicloj members:
Kira Howe (McLean) at London Clojurians
Sami Kallinen at Heart of Clojure
Thomas Clark at Clojure Conj

We have also been continuously working on building the Clojure community: running workshops, meetup groups, study groups and dev groups, and helping with the organization of a couple of conference. We have been mentoring many Clojurians in becoming involved in open-source and in expanding their use of Clojure to new contexts.

Based on that experience, we recently initiated the Scicloj open-source-mentorship program.
63 people have applied, and 34 of them are currently active in looking into projects and meeting regularly. The program was discussed in our recent status report (2024-10-25):

A few project mentees are new to Clojure and sometimes new to programming. We meet regularly and help them in their learning process.

We have been persistently exploring the expansion of Clojure to new fields of application and research, such as biology, linguistics, physics, statistics, and geography. We are actively working with practitioners in various fields (some new to programming and some experienced) to help them out and learn from their use cases.

Is there anything else you would like us to know? My role in Scicloj is both in community building and as an open-source maintainer of a few tools and libraries. I have been involved since the beginning (2019). Since I left my day job in 2023, Scicloj has been my main focus.

Adrian Smith

https://github.com/phronmophobic/

What do you plan to work on throughout the year?

• Grease: The goal for this year is to make a free, open source app similar to pythonista (http://www.omz-software.com/pythonista/) but for clojure. This project is already in-progress. More info at https://clojurians.slack.com/archives/C0260KHN0Q0 on the clojurians slack.
• AI tools like llama.clj: llama.clj is a library that allows running open source LLMs directly from the JVM with a clojure-friendly API.
• Dewey: Dewey is a public dataset that scans and analyzes clojure github repos weekly. These datasets are currently being used by tools like tutkain and clojure-lsp. The goal is to improve access to ecosystem data in order to be even more useful for developer tooling.
• Membrane: A pure clojure, cross-platform UI library

Why is this work important to the Clojure community? The goal for the Grease project is to make a fully scriptable iOS app. This would allow any clojure developer to write apps for their iPhone without requiring developers to jump through hoops like the Apple submission process. The app (currently code named LearnLisp) is compiled using graalvm’s native-image and scripts are executed using an embedded sci interpreter. A subgoal for this project is to make the app approachable to any developer that might be interested in learning a Lisp while also having fun making something useful for their phone. In principle, the same approach can be used to target android. A stretch goal is to also release an android app with the same features.

Many of the latest AI tools are written in python and c++. I have 15 years of experience writing python and c++. One of the goals for the next year is to continue writing clojure libraries like llama.clj that make the best AI tools available to clojure. Some examples of similar libraries are whisper.clj, usearch.clj, and clip. clj.

One of the challenges commonly cited by new clojure users is finding libraries. Dewey collects and indexes information about the clojure open source ecosystem. The dewey frontends are already quite capable of finding the right library for a particular task, but the UX needs to be improved.

The current dewey frontends:

• Library Search - Search for clojure libraries by keyword, name, author, and description.
• Cloogle - Search for any clojure function by its doc string. Queries are indexed by semantic meaning using a vector db.

The goals for this year:

• Consolidate frontends into a single, unified frontend
• Make the UI prettier
• Create an API to support developer tools
• Extract the semantic search code into its own standalone library

Membrane is a UI library written in pure clojure aimed at building complex, interactive UI applications in a functional style. One pitfall I’m trying to avoid with membrane is to solve the easy problems, but ignore the hard problems. The most recent work on membrane has been about solving some of the hard problems facing desktop UIs so that membrane can be a real alternative to the browser for complex desktop applications. These projects include clj-media (video playback), clj-cef2 (an embedded web browser based on chromium), clj-webgpu (3d graphics), and membrane.term (terminal emulator). Now that I feel comfortable that I won’t be leading users down a dead end, the goal for this year is to improve documentation and provide a good looking, high quality, component library. Membrane UIs aren’t tied to any particular graphics library. This means that membrane UIs can be embedded in any UI library that can draw shapes, texts, and rectangles.

Peter Strömberg

https://github.com/PEZ

What do you plan to work on throughout the year?
Calva - https://calva.io - I’d like to tackle repl session management, technical and ux debt and integration with CoPilot.

Why is this work important to the Clojure community? Calva runs on the important VS Code platform, which includes editors like Cursor. The uptake of Calva increases steadily and is now close to 25% of Clojurians. Especially the lacking session management and the technical debt are increasingly problematic.

Dan Sutton

https://github.com/dpsutton/

What do you plan to work on throughout the year? inf-clojure

Why is this work important to the Clojure community? The Clojure Community always benefits from editor integrations. I think the socket repl is the underused super power of Clojure. It is the great equalizer of development environments and is built into the language.

I would want to do two things:

1. Work on inf-clojure to make it more robust and accessible to both beginners and experienced devs alike. NREPL does lots of amazing and convenient stuff. But it can leave developers unable to run tests without a UI. It requires lots of configuration and takes over your main entrypoint. The socket repl is incredibly helpful. A REPL based on source is almost completely indistinguishable from one based on an AOT’ed, production jar.

2. To complement this, I will also write tutorials and create videos showcasing how easy it is to (a) start repls, (b) work with repls, and (c) lots of helpful additions that make it so powerful and flexible.
   I’ve got lots of local functions that I need to package up and get into the inf-clojure repo. Some examples:

• starting a repl: clj -J"$(socket-repl 6000)" which expands out to the command line args required for a socket repl
• m-x personal/repl [RET] 6000 will connect to a socket repl running on port 6000 and wire up a sub repl in it that shortens the displayed prompt to the last segment of the namespace and hook up clojure.pprint/pprint as the pretty printer
• save-to/eval-from registers. Using registers in emacs is lovely. Throwing different snippets into registers (t for run all tests, i for individual test, etc) is an amazing way to run. I’ve used this system for years now and it’s time it made it back into inf-clojure.
• a hotkey to require the repl-requires: (inf-clojure--send-string inf-proc "(apply require clojure.main/repl-requires)")
• a helpful debugging REPL. We have macros that create temporary things during testing. It’s often the easiest way to create some complicated state to exercise and inspect a bug. But the tests run so quickly that I can’t play around. I wanted something that would be blocking and still allow me to play around with the state: THIS IS A REPL. Using subrepls to pause the world is an amazing super power that is fundamentally incompatible with nrepl. But it’s transparently powerful with socket repls. I wrote a macro that throws all locals into a namespace and lets you introspect it.
• using a remote repl is amazing with a bit of ssh tunneling. The loveliest usecase is a failing test in Circle CI, rerunning with ssh and then connecting your REPL to your CI instance! This is straightforward with a socket repl.

I also want to highlight how queryable the Clojure runtime is. The repl helpers: source, apropos, dir, pst, doc, find-doc, add-libs, etc are INCREDIBLE. Learning to use the tools built into the language and not built into the IDE is a super power. Every Clojure REPL is an amazingly helpful repl.

I don’t want this to appear as it would be tutorial focused. I would hope that is 40% or less of my output. I want to contribute to inf-clojure, watch it grow, and share videos and articles about how it can unlock so much productivity.

Is there anything else you would like us to know? If this project sounds interesting, I’m also open to doing a shorter commitment. I wonder if there are others who have just a few months of work and we could combine to have several smaller projects.
But I could also improve this over 12 months as well. I think the clojurescript story is a bit underdeveloped. Shadow-cljs provides a socket repl that can easily upgrade. Using cljs.main is a bit harder but achieved with a different “accept” function. The documentation should be much easier.

I also want an easy way for it to just use a terminal repl easier. Using ssh tunneling is one thing, but it would be amazing to have a vterm window open and easily sending forms over to the repl.

In short, I think this tooling is a super power. I would love to build this out over a few months or the full year and share the knowledge and improve the tools.

Peter Taoussanis

https://www.taoensso.com

What do you plan to work on throughout the year?
Multiple projects, incl. further work on Telemere, Tufte, Tempel, and more

Why is this work important to the Clojure community? Have been trying to focus recent work on practical areas where Clojure either doesn’t already have great solution/s, or where Clojure might be able to offer something uniquely advantageous over other languages/platforms. There’s still a lot of low-hanging fruit re: both observability and data security for example. Beyond that, just focusing on practical tools that work at scale while being approachable for beginners.

Is there anything else you would like us to know?
Mostly, just thank you! I’ve been lucky and grateful to already receive long-term funding from Clojurists Together for 2023 and 2024.

I’d definitely be up for continuing this way if there’s interest from the community. It’s been awesome to dedicate more time to my open source, and Clojurists Together funding has undoubtedly helped with that (esp. for the more complex / high-effort stuff).

Have a lot of exciting things that I’d love to still expand on or explore in 2025. But I also understand that I’ve already received a couple years of funding now - and that there’s probably many great and deserving applicants with exciting proposals :-)

Whatever the outcome for 2025, I plan to continue to try contribute as much as I can.

Clojurists Together is a really awesome undertaking, and an inspiration. So thanks to everyone that’s been involved in helping make it reality and in keeping it running so smoothly.

Cheers! :-) - Peter

Oleksandr Yakushev

https://github.com/alexander-yakushev
https://github.com/clojure-goes-fast

What do you plan to work on throughout the year?
I plan to continue working on Clojure Goes Fast tooling. During Q3 2024, I’ve covered a lot of ground with new features and redesign of clj-async-profiler and the first release of Flamebin. This also opened a lot of ideas and opportunities for further improvement of those tools. I have a large backlog of tasks for:

• Flamebin – implement authorization and social login, client-side encrypted private flamegraphs, collaborative commenting, configuration editing and saving;
• clj-async-profiler – unified configuration, configuration reuse between flamegraphs, automatic analysis and performance improvement suggestions;
• Clojure Goes Fast knowledge base – populate the list of bad performance practices that can be later be linked to by the clj-async-profiler analyzer;
• clj-java-decompiler – CIDER integration which I wished to do in Q3 but couldn’t get to in this quarter.

Also, in 2024 I’ve once again become an active contributor and co-maintainer of CIDER, cider-nrepl, Orchard, and Compliment (250 total commits in 2024) and I want to continue doing this in 2025.

Why is this work important to the Clojure community?
Clojure Goes Fast continues to be the primary spot of performance-related projects and guides about Clojure (1.1K stars, 1.1M total Clojars downloads, 7000 unique site visitors in 2024).

CIDER is used by 40% of Clojure developers according to State of Clojure 2024 survey.

Permalink