Storyteller: A New Medium For Guiding Others Through Code

As a computer science instructor, I have long wished that there was a better way to guide others through my code. When I was first learning to program I was a big fan of traditional programming books. I have shelves and shelves of 800+ page books covering different programming languages and technologies. I have known for a while now that most learners today don't share my love of big thick books, and to be honest, I rarely read those books in their entirety. Those big books often had a lot more exposition about the code than was probably needed. As a book buyer I wanted to make sure that I was getting my money's worth so the thicker they were, the better. It is much more common these days for learners to consume blog based tutorials and videos.

Blogs and Videos

Blog-style tutorials mix code and the explanation of the code in a top-to-bottom fashion. Scrolling through these feels familiar and one can copy and paste with ease. However, linking the explanation of the code and the code itself has always been less than ideal. Often I find myself jumping around the blog post wishing I could see the entire code example while working through the explanation. Instead, I am only able to see small parts of the code and it is challenging to see how those parts relate to other parts.

Video tutorials are very popular these days. They solve some of the problems associated with blog-style tutorials. A viewer can focus on the code that is being written while listening to the explanation. However, videos have some problems too.

Viewing videos

From the perspective of the viewer, videos are not useful as a copy and paste source or a code reference. More importantly, though, they discourage the viewer from taking their time and reflecting on the material. Often when I am viewing a video tutorial I don't pause and let concepts sink in before the video moves on. Yes, I could be more disciplined and pause and rewind more often but usually I don't. The video slider is not a great search tool!

I have the same problem with audio books. I only listen to very light fiction or biography audio books because if I miss something in the first listen it is usually too hard to go back and find it. It is easier to let the audio play on. If there is a book I really want to understand I get a hard copy and read it the old fashioned way.

Making videos

From the perspective of the video creator, it is clear that not all code being developed is interesting to watch. Some of it is not really worth showing the viewer. Not all video creators can keep the narrative interesting the whole time too. I know I struggle with the 'performance' aspect of making videos (you won't find me coding on Twitch anytime soon). Many times after I am done making a video, as I review it, I wish I had mentioned something that I forgot. It is hard to go back and edit the video without scrapping it and starting over.

Storyteller

I have been working on a new medium used just for guiding viewers through code examples. It combines the best of books, blog posts, and videos. This new medium allows a developer to write code using a top-notch editor (Visual Studio Code) and then replay the development of that code in the browser. The author can add comments at important points in the evolution of the code and publish the resulting code 'playback' on a web server. The comments can include text, hand drawn pictures, screenshots, and audios/videos. The tool is called Storyteller.

Here are a few examples of a 'playback':
Adding a Border to a Picture (Python)
Enlarging a Picture (Python)
Functions with parameters (C++)

These work best on a big screen. If you are viewing a playback on a small screen you can view it in 'blog' mode (there is button in the top right to switch from 'code' mode to 'blog' mode).

I have created groups of these guided code walk-throughs to help me teach different topics to my students. These are all free and hosted on GitHub:

Sometimes I assign these as readings to my class instead of using expensive textbooks. It is a lot easier for me to write several programs than it is to find a perfect textbook (or to write my own).

I also use them for in-class demos instead of writing code live. This makes code demos flow much faster and smoother. If I make an interesting mistake while preparing the code I can still highlight it with a comment. If I make an uninteresting or embarrassing mistake I can just ignore it and the students likely will not see it.

The advantages of code playbacks:

  • The primary focus is on the code. It is always visible and easy to navigate.
  • Since the code is so accessible the explanation of it tends to be shorter and more concise.
  • I can review the code several times and add/edit comments each time I go through it. I don't have to give a perfect performance. I get multiple chances to 'get it right'.
  • Comment points highlight when the author wants the viewer to take a moment to really think about the code and reflect on it. The playback only moves forward when the viewer is ready.
  • The code mentioned in a comment can be highlighted so the viewer knows exactly where they should be looking.
  • The code can be downloaded at any point in the playback. Then a viewer can run it, change it, and add to it.
  • The tool is a language independent editor plug-in and can be used to describe programs in any language.
  • Viewers only need a web browser to go through a playback.
  • Comments can include whiteboard style drawings, screenshots, videos of running code, or use videos in place of a textual explanation.

Let me know what you think!

Links

If you would like to create your own playbacks the VS Code extension is called 'Storyteller'. You can find it here. The tool is free and open source.

The project repo can be found here. Here is some documentation. If you'd like some help getting started feel free to reach out to me.

My most recent collections of playbacks can be found here. I am currently using GitHub Pages to host them.

If you'd like to stay informed on the development of the tool or know when I release new content you can follow me on twitter @markm208 or follow the tool directly @StorytellerCode.

Permalink

Almost like Clojure

Practicing different programming languages has given me new insights in programming concepts, code styles and tools. My current crush is Clojure. I like the functional and data oriented style of it, the simplicity and minimalism. I think my favorite thing with Clojure, though, is how the REPL is used.

“What’s so cool about that repple thing?” 🧐

Testing at your fingertips 🚦

When using a code editor powered by a REPL, you can instantly evaluate variables, code blocks, functions or the entire file, just by a hitting a key combination. The result pops up right next to the cursor. This means that testing the code you currently write is at your fingertips. You are practicing a Deluxe version of Test Driven Development.

No copy-paste here 👮

It is not necessary to copy-paste code snippets into the actual REPL window or in a shell. In fact, you shouldn’t touch the REPL window at all. Because you don’t want to miss out on all the good stuff like autocomplete, hints and smart navigation that you have in your favorite editor.

For Clojurians only? 💭

I really would like to have a similar workflow in other languages too. It turns out there are some pretty good tools out there, the ones I’ve tried out so far are for JavaScript and ✨Emacs ✨.


Interactive JavaScript development 😍

Here’s me playing around with the node-zookeeper code. The cursor never leaves the editor window, the evaluation result is printed out in the Node.js REPL window to the right. More verbose than I’m used too in Clojure, but I think it works well.

Trying out code and ideas like this is a very nice workflow. I think you will find less need for debugging or console.log statements.

Keep on learning 📚

Learning a new programming language or tool can be really difficult, but also fun when the pieces finally start coming together. The knowledge gathered is something you can bring with you to other areas.



Have a look at my post about Emacs and my current configuration. The Node.js REPL feature is made possible using the js-comint package.

Photo by Marc-Olivier Jodoin on Unsplash

Permalink

Top 5 Programming Skills Worth Learning in 2021

2021 has already proven itself as another crazy year. It’s also another fantastic opportunity to learn new skills and expand on existing ones. In software development, stagnation is a sure-fire way to fall behind the curve. With new languages, technologies, best practices, and frameworks coming out more and more rapidly, this is the perfect opportunity to reevaluate where you want to go as a developer.

At the beginning of this year, I did just that. I wrote an article earlier this year about what I felt were the best languages to learn in 2021. With those, I felt that there was a nice mix for developers of any level and area of interest. However, they weren’t the languages for me to learn. Already in working with Python, Typescript, C#, and BASH, I thought about what I could do to help move my skills forward in other ways. After a lot of thought, I came up with a few programming skills that I believe are worth learning in 2021.

1. Functional Programming

This is a big one. Functional programming is a programming paradigm that revolves around the idea of moving data around, well, functionally! Instead of creating objects and callings methods on those objects, your data can be thought of as a stream, moving from function to function. It requires a different way of thinking about your programs. It's a lot easier to show it than to explain it.

Let's take a look at this Java code. It should be pretty familiar to anybody who has worked in an OOP language.

String words= "Here,are,some,words";
words = words.replace(",", " ");
words = words.trim();
words = words.toUpperCase();

Here we are replacing the commas with spaces, trimming whitespace off the end, and then setting the whole thing to uppercase.

Now, let's look at the same example in Elixir, my favorite functional programming language.

words = "Here,are,some,words"
    |> String.replace(",", " ")
    |> String.trim
    |> String.upcase

We are taking our stream and piping it into the replace function, which replaces commas with spaces. The output of that is piped into trim, which is then piped into upcase. Notice how words is assigned once, where as in our Java example, we declare it then reassign it three times. This is a key feature of Elixir. Data is immutable, meaning it cannot be changed. The string being passed into replace does not change but instead returns a new reference to a string. This new string is passed into trim, which returns a new one to upcase. Our variable words is only assigned once and our original input was never changed. This pattern of programming can help to create more robust and more maintainable software.

For those wanting to get started with functional programming, there is a lot to chose from! If you are on the JVM, you have Scala or Clojure to choose from. Scala might be easier, as it supports both OOP and functional programming paradigms. For those in the .NET side of the world, F# is a fantastic language. There is also a version of Clojure that runs on the .NET platform as well. My personal preference is Elixir, as its syntax is similar to that of Ruby’s but gives you the power of the BEAM, Erlang’s virtual machine.

You can find more information on Elixir at https://elixir-lang.org/

2. Swift, Kotlin, and Mobile Development

This one might strike some people as odd. However, I believe Swift has a place beyond just writing iOS apps. In 2015, Swift was published under the Apache 2.0 license, thus making it open source. As of now, it can be run on Mac and Linux and can run on Windows through the use of Docker.

Given this, Swift can be used to create more than just iPhone apps now. You can build APIs, command-line apps, and even do machine learning with it. Unfortunately, iOS is still restricted to Macs only, but it is a start.

The beauty in how easy the language is to write and use. It’s certainly C-inspired but feels like a modern language. It is nowhere near as clunky or verbose as Java. The same argument can be made for Kotlin, which is essentially aiming to fix the problems of Java while being a modern language. Kotlin is developed by JetBrains, so you know it’s treated as a first-class citizen in their IntelliJ platform. Kotlin can be used to build Android apps, which is excellent at but also building web apps using the Spring framework. It can be used alongside Java in existing projects, so adding it in is simple. Both of these languages blow past their predecessors in almost every way.

So why are these skills important? For one, mobile development is not going anywhere. In fact, I believe its need will continue to increase, due to the ever-growing number of systems that can now interact with phones. Apps now let you control your car, house, let you make payments, and so much more. Beyond that, both of these languages can now be used to build more kinds of applications. You essentially can build your mobile and backend in the same language now. If you learn Kotlin, you can even compile to JavaScript and use it on the web. And this isn’t even going into the amount of other smart appliances coming out. Watches, doorbells, and even washing machines are all getting smarter. Both Swift and Kotlin are great for building apps for these appliances or for building mobile apps to interface with them. These are fast, modern, and versatile languages with a lot to offer.

As a side note, Apple Silicon has already proven itself a beast and both of these languages have been shown to do very well on it. I believe we will see a wider implementation of ARM chips from all vendors eventually, so knowing these languages would be a great way to dive headfirst into these next-generation CPUs.

You can start with Swift at https://swift.org/ and Kotlin at https://kotlinlang.org/

3. Blockchain and Cryptocurrency

Blockchain technology is not going anywhere anytime soon. We've seen the kinds of disruption that Bitcoin, Ethereum, and even Dogecoin have created. Cryptocurrency has steamrolled ahead, while traditional banking and investment institutions have fought it tooth and nail. I am not going to explain the ins and outs of how the blockchain works here, but I believe that knowing how to program apps to take advantage of this technology is an extremely valuable skill to have.

Cryptocurrency essentially removes the middleman in finance. Bitcoin and Ethereum are not controlled or owned by a centralized bank or printed from the government. Because of this, they are seen as more private and a way to future-proof against inflation and any changes to the value of the dollar. This is absolutely crucial. A bitcoin is worth the same everywhere in the world.

Why is this important for us as developers? Well recently, Tesla announced they would be accepting Bitcoin as payment and not converting back into a stable coin or into USD. This is a massive step forward for cryptos. If things keep moving this way, there will come a time when developers need to know and understand how to build apps that can integrate with the blockchain. As more people start investing in and taking an interest in cryptocurrency, the demand for being able to use it for commerce in websites and apps will only grow.

My advice for getting started would be to learn Solidity, the programming language used for building apps to take advantage of Ethereum. You will also need a knowledge of languages like Python and JavaScript to be able to let your users interact with your applications. Below is a simple Hello, World app taken from Solidity's website.

pragma solidity ^0.8.0;
contract MyContract {
    function helloWorld() public pure returns (string memory) {
        return "Hello, World!";
    }
}

More information can be found at https://soliditylang.org/

4. GraphQL

Moving away from broader language categories into more design and architecture, GraphQL aims to solve a couple of the problems of REST APIs. It was designed by Facebook to in order help them streamline getting data from their servers to their client apps.

Instead of declaring endpoints for your API, you have one GraphQL endpoint that you pass your queries and mutations to. A query is essentially a defined set of data you want back from your server. You would use queries to define the structure of the data and how many records you want back from your server. Mutations can be thought of as a traditional create, update, or delete command. You send data to the server that you are creating or updating or whatever you want to do with it.

GraphQL aims to simplify retrieval and management between clients and servers. You no longer are fetching more or fewer data than what you expected and really don’t need to create a bunch of DTOs to manage requests and responses. You just define your object types and then you can specify what fields from it you want your server to return.

Will GraphQL replace REST? Well no, I don’t think so. Each has a specific use case. Sometimes, setting up a REST API might work better for your app. Other times you might want to use GraphQL. Why I think GraphQL is worth learning because of how much more complicated our data is becoming. Complicated data models no longer need to have complicated logic to build your dataset on your backend. With GraphQL, you tell the server exactly what you want back.

You can learn more about GraphQL at https://graphql.org/

5. Docker

Docker has been a tool that I cannot live without. Docker is a tool used to help simplify code environments. You pull down an “image” of an existing environment, say like one for Python or JavaScript, and using a file called a Dockerfile, you can copy your code into the image and run it there instead of on your machine.

There are a number of advantages to this from a developer's perspective. You, for one, can keep your development environment off your local machine. All your dependencies and code are isolated in the image. This helps prevent any headaches you might run into with running your environments locally. It also makes sharing your environment with team members easier. All you need to give them is your code and the Dockerfile and they can use the code exactly how you have it set up on your machine.

You can also set up more complicated environments using Docker Compose, where you can create environments using multiple images. You can spin up databases, message brokers, and more, all without having to download and install these pieces of software on your system. You can then reference those within your software, whether you are running that in a container or not. It greatly simplifies the process of creating and managing development environments.

This can be taken a step further with tools like Docker Swarm and Kubernetes, which make the process of setting up and configuring production environments much simpler. Being able to package and deploy software in containers like this greatly reduces the risk of taking down the whole system. Again, having all your configurations and applications isolated in this way really helps to ensure that your code will run exactly the same way no matter where it is.

Like with GraphQL, Docker and container technology helps solve complicated business infrastructure. With more and more components to applications now, and with them becoming more microservice oriented, Docker is an essential tool for developers to learn and use. The power and flexibility it gives you to control every part of your stack and deploy it with confidence are invaluable to any development workflow.

More information on Docker can be found at https://www.docker.com/

Conclusion

The world of development is constantly changing and evolving, and we as developers have to stay on top of it. How software is written and the tooling we use to create it can change overnight sometimes. It’s important to constantly be learning and adding new skills to what you know.

These skills and technologies I have discussed are a great way to do just that. More mainstream languages are introducing functional elements into their syntax. Our home appliances and phones are not only getting smarter, but more available and more complicated. Swift and Kotlin are great languages to learn in that regard. Blockchain technology will continue to grow and become more widely adopted as the demand and awareness of cryptos continue to grow. GraphQL and Docker are tools to help with how you build your apps and fetch data for them. They solve the issues of deployments, APIs, dependencies, and so much more. With these skills, you are not only helping to write better software, but you are also helping to future-proofing yourself.

(Photo by Firos nv on Unsplash)

Permalink

PF4J and Clojure

Today we’ll take a look at interesting Java library, called PF4J, which describes itself as Plugin Framework for Java. The main purpose of this library is to provide a way of detecting, initializing, and using plugins to extend your Java application with new features without the need to modify the code. The official repository has examples of how to use this library with Java, and Kotlin, but there is no Clojure example.

Recently I had to find a way how to use this library with Clojure, and it have turned a bit hard to achieve, because this library is very Javaish, if you know what I mean :). Classes, interfaces, and Java annotations are not the most common thing to see in Clojure, as we usually have no classes, interfaces or annotations, but functions, data abstractions and metadata. I’m not very experienced in Java bytecode inspection, and Clojure class generation, and I was unable to find any information on how to integrate PF4J and Clojure, so this post will explain what I’ve found so far. You know, so I would be able to refresh this if I ever again will need to use this library, or will find myself in similar situation where I need to work with annotations or class generation. And hopefully this information will be useful for others as well.

So here we will write an example plugin, in the same way how Java example plugins written in the pf4j demo application. But first, let’s look at how Java plugin is structured.

Anatomy of a plugin

Complete plugin consist of two main components - the plugin itself, and the extension point. We will take the .jar approach for storing and distributing plugins, as it is the easiest one to reach, and we can directly compare bytecode of Java plugin, and Clojure plugin. So let’s first create an extension point in Java.

Create project Greeting in your Java IDE of choice. I’m creating a maven project, so first thing we’ll need is to add pf4j dependency to our pom.xml:

<!-- snip -->
<dependencies>
  <dependency>
    <groupId>org.pf4j</groupId>
    <artifactId>pf4j</artifactId>
    <version>3.6.0</version>
  </dependency>
</dependencies>
<!-- snap -->

Now we can create extension point class org.example.Greeting, and add this code into it:

package org.example;

import org.pf4j.ExtensionPoint;

public interface Greeting extends ExtensionPoint {
    String getGreeting();
}

This is all we need for our small extension point, and we can install it in our local repo with mvn install. AFAIU, extension point’s main purpose is to help PF4J detect particular plugins, and not all plugins.

Now we need the plugin itself. This plugin depends on both pf4j and this extension point we’ve just created and installed. Create a new project JavaWelcomePlugin, and add there pf4j dependency as explained above, and the org.example.Greeting dependency:

<!-- snip -->
<dependency>
  <groupId>org.example</groupId>
  <artifactId>Greeting</artifactId>
  <version>1.0-SNAPSHOT</version>
</dependency>
<!-- snap -->

Again, create a new class GreetingsPlugin:

package org.example;

import org.pf4j.Extension;
import org.pf4j.Plugin;
import org.pf4j.PluginWrapper;
import org.example.Greeting;

public class GreetingsPlugin extends Plugin {
    public GreetingsPlugin(PluginWrapper wrapper) {
        super(wrapper);
    }

    @Override
    public void start() { System.out.println("Java plugin start"); }

    @Override
    public void stop() { System.out.println("Java plugin stop"); }

    @Extension
    public static class Welcome implements Greeting {
        public String getGreeting() {
            return "Hi from Java!";
        }
    }
}

This plugin has start and stop methods that pf4j will call during initialization, and deinitialization of the plugin, and the main part here is the nested Welcome class. This class implements our Greeting extension point, and is annotated with @Extension, which helps pf4j discover this class during build.

One more thing we need to add to our pom.xml are new manifest entries. Pf4j expects each plugin to have at least two entries in MANIFEST.mf file in the .jar archive: Plugin-Version, and Plugin-ID. In our case we will also need to add Plugin-Class entry, as we’re created a wrapper class GreetingsPlugin. Add this to the pom.xml:

<!-- snip -->
<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-jar-plugin</artifactId>
      <configuration>
        <archive>
          <manifestEntries>
            <Plugin-ID>java-welcome-plugin</Plugin-ID>
            <Plugin-Class>org.example.GreetingsPlugin</Plugin-Class>
            <Plugin-Version>1.0</Plugin-Version>
          </manifestEntries>
        </archive>
      </configuration>
    </plugin>
  </plugins>
</build>
<!-- snap -->

It’s a bit too verbose (well, it is XML after all), but here we’re just adding these three entries to our manifest file during build. Now let’s package this plugin with mvn package command.

This makes the complete example plugin in Java - much like the one in the demo application from PF4J examples. Both the interface and the plugin are written, and we can plug this code into any application, that looks for our Greeting interface. Now let’s test this plugin in small Clojure program.

Writing Clojure program that uses plugins

Let’s create small project based on the app template, that uses pf4j library, and loads this plugin we’ve just wrote in Java. I’m using Leiningen for project automation, but you may use deps or other tools if you prefer. But we will later need to edit project.clj and do some tweaks other than adding dependencies, so if you’re using deps you’re on your own here. For me, the command is:

$ lein new app pf4j-clj-app

A new project is created and we need to add org.pf4j/pf4j and org.example.Greeting dependencies to the :dependencies section in project.clj:

:dependencies [[org.clojure/clojure "1.10.1"]
               [org.pf4j/pf4j "3.6.0"]
               [org.example/Greeting "1.0-SNAPSHOT"]]

Rest of the project.clj can be left as is. Now, let’s write our -main function in the core.clj:

(ns pf4j-clj-app.core
  (:import [org.pf4j JarPluginManager]
           [org.example Greeting])
  (:gen-class))

(defn -main
  [& _]
  (let [plugin-manager (JarPluginManager.)]
    (doto plugin-manager
      (.loadPlugins)
      (.startPlugins))
    (let [greets (.getExtensions plugin-manager Greeting)]
      (println "Found" (count greets) "extensions for extension point" (.getName Greeting))
      (doseq [greet greets]
        (println (.getGreeting greet))))
    (doto plugin-manager
      (.stopPlugins)
      (.unloadPlugins))))

I’ll keep this brief - we’re importing our Greeting extension point, and the pf4j library, creating plugin-manager, load and start plugins. Then we print how many plugins we’ve found for Greeting extension point, and execute .getGreeting method for each plugin. And after that we’re stopping and unloading plugins to keep things clean.

By default pf4j searches for jar plugins in the plugin directory at the root of the project. Let’s create this directory, and copy our Java plugin there, and start our application with lein run:

$ mkdir plugins
$ cp path/to/JavaWelcomePlugin/target/JavaWelcomePlugin-1.0-SNAPSHOT.jar ./plugins/
$ lein run
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Java plugin start
Found 1 extensions for extension point 'org.example.Greeting'
Hi from Java!
Java plugin stop

It works! We’ve seen the message from the start method, the getGreeting call result, and then the stop method message. Now let’s do everything in Clojure!

Clojure ExtensionPoint implementation

To make an extension point in Clojure, let’s create new library project with lein new clj-greeting, and add needed changes to our project.clj:

(defproject org.example/Greeting "1.0-SNAPSHOT"
  :description "pf4j extension poin example in Clojure"
  :license {:name "EPL-2.0 OR GPL-2.0-or-later WITH Classpath-exception-2.0"
            :url "https://www.eclipse.org/legal/epl-2.0/"}
  :dependencies [[org.clojure/clojure "1.10.1"]
                 [org.pf4j/pf4j "3.6.0"]]
  :aot :all)

I’ve mainly changed the project name from clj-greeting to org.example/Greeting to so we could hot-swap our Java extension point to the Clojure one with simple lein install. Similarly, the version was changed to 1.0-SNAPSHOT to match the version of our Java extension point. The pf4j dependency was added to the :dependencies key. Lastly, :aot :all entry was added to make sure we AOT compile every file.

Now we can create our interface. To do so, first we need to create needed directories with src/org/example directories, and move core.clj to src/org/example/greeting.clj:

$ mkdir -p src/org/example
$ mv src/clj_greeting/core.clj src/org/example/greeting.clj
$ rmdir src/clj_greeting

Next, we add this code to greeting.clj:

(ns org.example.greeting
  (:import org.pf4j.ExtensionPoint))

(gen-interface
 :name org.example.Greeting
 :extends [org.pf4j.ExtensionPoint]
 :methods [[getGreeting [] String]])

Here, we’re creating an interface with gen-inteface and naming it to org.example.Greeting. This interface extends the org.pf4j.ExtensionPoint class, and exposes one method getGreeting which takes zero arguments, and returns a String.

Let’s package this as a jar archive with lein jar command, and see its contents with zipinfo. If you don’t have zipinfo you can use any other way to view archives. Here’s the contents:

lein jar && zipinfo -1 target/Greeting-1.0-SNAPSHOT.jar
Compiling clj-greetings.core
Created /home/alistopadov/Git/clj-greetings/target/Greeting-1.0-SNAPSHOT.jar
META-INF/MANIFEST.MF
META-INF/maven/org.example/Greeting/pom.xml
META-INF/leiningen/org.example/Greeting/project.clj
META-INF/leiningen/org.example/Greeting/README.md
META-INF/leiningen/org.example/Greeting/LICENSE
META-INF/
META-INF/maven/
META-INF/maven/org.example/
META-INF/maven/org.example/Greeting/
META-INF/maven/org.example/Greeting/pom.properties
org/
org/example/
org/example/Greeting.class
clj_greetings/
clj_greetings/core__init.class
clj_greetings/core$loading__6721__auto____171.class
clj_greetings/core$fn__173.class
clj_greetings/core.clj

We’re mainly interested in the org/example/Greeting.class entry. Let’s decompile it. This can be either done with Intellij Idea, or by using fernflower directly. The decompiled bytecode looks like this:

package org.example;

import org.pf4j.ExtensionPoint;

public interface Greeting extends ExtensionPoint {
   String getGreeting();
}

And this looks exactly like in the Java example we’ve wrote before. Great, let’s test it!

We do this by installing this extension point in place of our Java one, by using lein install command (you can remove old extension from .m2 directory for extra safety). Then we need to repackage our Java plugin, and copy it again to our Clojure application. When we run it, we should see the same output:

Java plugin start
Found 1 extensions for extension point org.example.Greeting
Hi from Java!
Java plugin stop

So, our extension point is now implemented in Clojure. Now we can write the plugin itself!

PF4J plugin in Clojure

To write a plugin let’s create another project with lein new clj-welcome-plugin. Again, we need to alter project.clj a bit:

(defproject clj-welcome-plugin "0.1.0-SNAPSHOT"
  :description "pf4j plugin example in Clojure"
  :license {:name "EPL-2.0 OR GPL-2.0-or-later WITH Classpath-exception-2.0"
            :url "https://www.eclipse.org/legal/epl-2.0/"}
  :dependencies [[org.clojure/clojure "1.10.1"]
                 [org.pf4j/pf4j "3.6.0"]
                 [org.example/Greeting "1.0-SNAPSHOT"]]
  :manifest {"Plugin-Id" "clojure-welcome-plugin"
             "Plugin-Version" "0.1.0"
             "Plugin-Class" "clj_welcome_plugin.GreetingsPlugin"}
  :aot :all)

Again, we’re adding :aot :all entry and the :manifest key with a map of keys and values of manifest entries we need to add. Thankfully, Leiningen makes it really easy to deal with this, compared to pom.xml we’ve had to modify previously.

Now, we need to create wrapper class, and the @Extension class, and here’s a problem. To my knowledge there’s no way to write nested classes in Clojure. And annotations are also a bit finicky. So let’s approach this step by step.

First thing we’ve did in Java code was importing all needed libraries, and creating a class that extended Plugin class:

package org.example;

import org.pf4j.Extension;
import org.pf4j.Plugin;
import org.pf4j.PluginWrapper;
import org.example.Greeting;

public class GreetingsPlugin extends Plugin {
    public GreetingsPlugin(PluginWrapper wrapper) {
        super(wrapper);
    }
    /* ... */
}

Let’s do the same in Clojure. We start by renaming the core.clj to greetings_plugin.clj, and importing all needed libraries, also saying that our :gen-class has no main and :extends org.pf4j.Plugin:

(ns clj-welcome-plugin.greetings-plugin
  (:import [org.pf4j Plugin PluginWrapper Extension]
           [org.example Greeting])
  (:gen-class
   :name clj_welcome_plugin.GreetingsPlugin
   :main false
   :extends org.pf4j.Plugin))

So far so good. Now we need a constructor, but in our case Clojure has already generated one, which we can see by compiling our code into jar file, and looking through the bytecode:

package clj_welcome_plugin;

/* Clojure imports */
import org.pf4j.Plugin;
import org.pf4j.PluginWrapper;

public class GreetingsPlugin extends Plugin {
   /* some initialization Clojure stuff */

   public GreetingsPlugin(PluginWrapper var1) {
      super(var1);
   }

   /* rest of the Clojure stuff */
}

Custom constructor can be provided by using the :init key in :gen-class, but I’ll left it out to the reader.

Now we can add start and stop methods:

(defn -start [_]
  (println "Clojure plugin start"))

(defn -stop [_]
  (println "Clojure plugin stop"))

And this basically completes our wrapper class, so how we add a nested class? Well, actually, from bytecode standpoint this is not a nested class, and outer class simply acts as a namespace. We can actually see that in the contents of jar file which contains compiled code of our Java plugin:

org/example/GreetingsPlugin.class
org/example/GreetingsPlugin$Welcome.class

So the inner public static Welcome class is actually a separate .class file, and we can replicate this in Clojure pretty easily. Let’s add another call to gen-class in the same file:

(gen-class
 :name ^{org.pf4j.Extension {}} clj_welcome_plugin.GreetingsPlugin$Welcome
 :prefix "impl-"
 :implements [org.example.Greeting])

(defn impl-getGreeting [_] "Hi from Clojure!")

Or we can create another file, and use :gen-class there, but to match Java plugin source code layout, I’m using the single file.

The most important thing here is the :name key. You can see that we’re providing a metadata entry ^{org.pf4j.Extension {}} which is a way to add Java annotation to a class. And we’re naming our class as clj_welcome_plugin.GreetingsPlugin$Welcome. Lastly, :implements key is used to tell, that our class implements our extension point. And because we’re using the same file, we change :prefix to "impl-", and the getGreeting method is defined as impl-getGreeting.

Let’s compile our code with lein jar again, and see if we’ve managed to replicate the .class layout in the jar archive:

$ lein jar && zipinfo -1 target/clj-welcome-plugin-0.1.0-SNAPSHOT.jar
...
META-INF/MANIFEST.MF
META-INF/leiningen/...
META-INF/maven/...
clj_welcome_plugin/GreetingsPlugin.class
clj_welcome_plugin/GreetingsPlugin$Welcome.class
...

Great! Let’s see if our GreetingsPlugin$Welcome.class actually decompiles to correct class:

package clj_welcome_plugin;

/* Clojure imports */
import org.example.Greeting;
import org.pf4j.Extension;

@Extension
public class GreetingsPlugin$Welcome implements Greeting {
   /* Clojure stuff */
   public String getGreeting() {
      Var var10000 = getGreeting__var;
      Object var1 = getGreeting__var.isBound() ? var10000.get() : null;
      if (var1 != null) {
         return (String)((IFn)var1).invoke(this);
      } else {
         throw new UnsupportedOperationException("getGreeting (clj-welcome-plugin.greetings-plugin/impl-getGreeting not defined?)");
      }
   }
}

And it is! We can see the @Extension annotation, and public class GreetingsPlugin$Welcome that implements Greeting extension point. Later, after some Clojure internal code, we can see our getGreeting method definition.

However, if we try to use this plugin, it will not work just yet:

$ cp path/to/clj-welcome-plugin/target/clj-welcome-plugin-0.1.0-SNAPSHOT.jar ./plugins/
$ lein run
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Java plugin start
Found 1 extensions for extension point 'org.example.Greeting'
Hi from Java!
Java plugin stop

Still only Java plugin runs. This took me some time to figure out why it doesn’t work - the .class files layout of the resulting .jar archives are the same. Decompiled classes, except for the Clojure stuff, look the same as Java classes.

The catch is, that when we’re building our Java plugin with mvn package, the extra file extensions.idx is created in the META-INF directory of the archive. Let’s look at this file:

# Generated by PF4J
org.example.GreetingsPlugin$Welcome

Aha! This file contains the necessary info to detect the @Extension annotated class in the jar archive, and is generated by processing Java annotation during compilation. Unfortunately, Clojure compiler doesn’t try to process our metadata as Java annotation, and hence no file is created. But we can add this file by hand, by creating META-INF/extensions.idx in the resources directory with the following contents:

clj_welcome_plugin.GreetingsPlugin$Welcome

Here, we’re indicating that our @Extension class is clj_welcome_plugin.GreetingsPlugin$Welcome. Let’s re-package our plugin with lein jar, and re-run our app:

$ cp path/to/clj-welcome-plugin/target/clj-welcome-plugin-0.1.0-SNAPSHOT.jar ./plugins/
$ lein run
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Clojure plugin start
Java plugin start
Found 2 extensions for extension point 'org.example.Greeting'
Hi from Clojure!
Hi from Java!
Java plugin stop
Clojure plugin stop

It works! Now we can make a more complex app, and extend it with plugins written both in Java and Clojure. Of course, more complex plugins well be bigger, and have more code, but the general idea will be the same.

Some ending notes

You don’t have to put both wrapper class and Extension class in the same file. I’ve mainly did this because it is in line with the Java code, so sources could be compared mostly directly. But a separate file, that sets the class name in a similar way is a more clean approach, and should be preferred.

There’s also no need to rename most of the classes, except for the @Extension one, as we have to explicitly make it into nested class via $ notation. Again, I’ve mainly did this to keep more or less same code structure with the Java examples, for easier direct comparison.

Lastly, some automation may be needed to generate extensions.idx file based on source code. Unfortunately, I’ve not found any info how to make Java annotation processing to work with Clojure metadata annotations. So I’ve did this manually, but I’ve also wrote this script, that analyzes Clojure source code as EDN, and searches for given extension point class, printing class name from gen-class form if present:

#!/usr/bin/env bb

(require '[clojure.edn :as edn])
(require '[clojure.string :as str])

(defn analyze-gen-class
  "Tests if `form` is a `gen-class` and gets the `:name` or uses the
  `default` if `:name` is not set, but `:implements` matches the
  `query`.  Replaces all dashes with underscores in case the `default`
  name came from the `ns` form."
  ([form query]
   (analyze-gen-class form query nil))
  ([form query default]
   (when (and (list? form)
              (#{:gen-class 'gen-class} (first form))
              (some #{:implements} form))
     (let [gencl (->> form
                      (drop 1)
                      (partition 2)
                      (into {} (map vec)))]
       (when-let [name (and (= (:implements gencl) query)
                            (get gencl :name default))]
         (str/replace name #"-" "_"))))))

(defn analyze-ns
  "The `ns` form already contains class name as a namespace name,
  but we check inner `:gen-class` for a `:name` override."
  [form query]
  (let [name (second form)]
    (some #(analyze-gen-class % query name) form)))

(defn analyze-form
  "Check whether current `form` is a `gen-class` or `:gen-class` expression.
  If it is, checks if gen-class has `:implements` key that is equal to
  `query` and gets the value of `:name` key.  If `form` is an `ns`
  form, finds `:gen-class` in it and checks it as above.  Recursively
  analyzes nested lists.  Terminates at first successful analysis."
  [query form]
  (when (list? form)
    (cond (= 'ns (first form)) (analyze-ns form query)
          (some list? form) (some #(analyze-form query %) form)
          :else (analyze-gen-class form query))))

(let [[query] *command-line-args*]
  (doseq [form *input*]
    (when-let [name (analyze-form
                     (edn/read-string query)
                     form)]
      (println name))))

This is a Babashka script, which analyzes S-expressions, and can be used together with another tool for parsing EDN - Jet (huge thanks to Michiel Borkent for awesome Clojure tooling!):

$ jet -c < src/clj_welcome_plugin/greetings_plugin.clj | ~/script.clj "[org.example.Greeting]"
clj_welcome_plugin.GreetingsPlugin$Welcome

Or, similarly for every .clj file in the project:

$ find -name '*.clj' -exec sh -c 'jet -c < "{}" | ~/script.clj "[org.example.Greeting]"' \;
clj_welcome_plugin.GreetingsPlugin$Welcome

For this project, the script finds the extension class name correctly from the source code we’ve gave to it, but it is more a proof of concept, and by far is not a well tested solution.

One particular point to improve is the fact that we have to pass "[org.example.Greeting]" to the script, to help it find gen-class entries that implement the interface we’re interested in. A much better solution would be to parse metadata, but unfortunately I was not able to make Jet keep metadata in resulting EDN, and I’m not sure how to properly read Clojure source code files otherwise. Although, it’s not really hard to check if all classes are present, as I highly doubt that there will be more than 5 extension classes presented in single plugin, so this solution is good enough for now.

Other than that, the process of using PF4J library is pretty straightforward, and we can fully utilize this library from Clojure projects with some hand tweaking. Thanks for reading!

Permalink

Working at Skipp - Democratize Architecture and Design with Clojure

Skipp is an Austin-based early-stage, VC-backed startup, using AI and the latest in spatial technology to simplify the renovation process and democratize access to elevated designs for the everyday homeowner. Skipp collaborates with world-class architects, designers and technologists to not only make the process of transforming a home easier and affordable but also fun and inspiring. In just hours Skipp's technology can generate real-time pricing, architect grade documentation, visualizations, and material schedules. With this approach, homeowners can not only access world-class designs for their actual space immediately, but can also efficiently manage their budget and have all the d...

Tags: Clojurescript, Design, Clojure, Datomic, Functional Programming, Startup, Aws, Python

Continue reading

Permalink

Meta Reduce 2021.0: Still in Lockdown

I’ve noticed the other day that I haven’t written any blog posts this year.1 This made me sad. Another thing that made is sad is that about a year ago I wrote this lockdown-themed article and very little has changed since then - life is still dominated by the COVID-19 pandemic and we’ve been in and out of lockdowns of various strictness ever since. At least this year we have vaccines and the end of the crisis seems to be in sight. The local authorities are predicting that most of Bulgaria’s population will be vaccinated over the next 3 months and I really hope that’s going to be the case. In the mean time - I’m going slightly madder every day…

I have to admit I haven’t made much progress with my OSS projects since the releases of CIDER 1.0 and RuboCop 1.0. I’m making slow progress on all fronts, but in general I haven’t been particularly happy with myself. Some interesting recent developments worth sharing are:

Over the past few months I’ve been trying to find better ways to support the community around my projects, that’s why I was more focused on documentation and exploring new communication channels, instead of features and fixes. I’ll probably write more on the topic soon, but I can tell you now that I really like Discord and I regret not trying it earlier. It reminds me of the early days of Slack when it was cool and simple.

A couple of days ago the results from the annual “State of Clojure” survey were published and it was nice to see I was mentioned there for my efforts to improve the Clojure tooling ecosystem. Another positive data point from the survey was that 44% of the respondents are using Emacs for Clojure development, despite the slowdown of the work on CIDER and the ever stronger competition from other editors and IDEs. Meta-X forever!

Survey Results

I have some interesting ideas for CIDER, nREPL and friends going forward. Let’s see if I’ll manage to make them a reality. I should probably run a couple of project surveys myself, as I completely dropped the ball on those.

Outside of OSS things have also been relatively quiet on my end:

  • I’ve been having some fun with algorithms and programming puzzles. This helps me feel like a real programmer, even if I don’t do much programming on my day job.
  • I finished reading the Witcher books. I have to say I was quite disappointed by the final couple of books.
  • I’ve been educating myself a bit on blockchain technologies and cryptocurrencies. I’m just curious to learn more on the topic, I don’t plan to invest anything in crypto. It’s too much hype for my taste.
  • I caught up with “The Walking Dead”. The show is getting worse every year.
  • I’ve started to watch the new “Snowpiercer” show, but I didn’t like it. I think the movie was way better.
  • I watched “Zack Snyder’s Justice League” and I have to admit is just as good as it was hyped to be.2 I didn’t even notice that it was 4 hours long.
  • I bought myself a pair of Audioengine A2+ speakers, as I wanted to enjoy better music now that I’m spending so much time at home. My initial impression of them is quite positive.
  • I’ve rediscovered “Counter-Strike” 20 years later. We’ve started playing a bit of CS with my co-workers as a bonding activity. It’s just as fun as it used to be back in high-school. And I’m still the worst player in every team.
  • I’ve also been playing on and off “Rise of the Tomb Raider”, but for some reason I’m not enjoying the game as much as the first one. Still, it’s fun, and I can definitely recommend it to people fond of games like “Uncharted”.

That’s all from me for now. As usual, I have a huge backlog of topics for “Meta Redux” and I hope that I’ll make some progress with it over the months to come. Stay safe, stay strong and keep hacking!

  1. At least not here. I did write a couple of articles over at https://emacsredux.com

  2. For DC fans. For everyone else it’s probably still one very dumb movie. 

Permalink

Clojure Developer

Clojure Developer

Skipp Renovation | Austin, TX or Brooklyn, NY
Technology to translates home renovation design into plans for specific space
$80000 - $140000

With 80% of housing stock now older than 20 years old and a growing affordability crisis, younger homeowners are renovating at an unprecedented pace. However, the process hasn't changed in 40 years and has remained a fully analog, messy and chaotic endeavor. Skipp is changing that.

With Skipp homeowners will be able to develop designs for their exact space in a way one might build and customize a car online. They will be able to access world class designs. source materials, get precise construction plans and get multiple labor bids without ever having to leave the sofa.

Skipp is building a product with world-class architects, designers, and technologists to not only make the process of transforming a home easier and affordable but also fun and inspiring. We are an early-stage, VC-backed startup using AI, data-driven development, and the latest in spatial technology to simplify the renovation process and democratize access to elevated designs for the everyday homeowner. Skipp's technology can generate real-time pricing, architect-grade documentation, visualizations, and material schedules. With our approach, homeowners can not only access world-class designs for their actual space immediately, but can also efficiently manage their budget and have all the documentation needed for a streamlined RFP and construction process.

We are looking for a Clojure backend or full-stack engineer with start-up experience. This will be an early hire for the company, one of the first non-founding members on the engineering team with an opportunity to influence the strategic technology direction and company culture.

We value talent that likes to experiment with new technologies, new techniques and new processes. We're building something that no one has seen before, and we want fresh ideas matched with a "get it done" attitude. As part of a small team, you will be expected to share your ideas, seek feedback and provide recommendations on other areas of the system. The best team members will know how it all fits together.

Our backend is written in Clojure with a Datomic database, all deployed on AWS. Clojurescript on the frontend. We also use Python for some integration and DevOps work. Overall, our technology stack is designed for rapid iteration, application of Machine Learning, and scale.

QUESTION

Given what you know about Skipp now, the AEC space, and the technologies we're using, what would you propose as a Hack day project?

QUALIFICATIONS

  • Three years or more with server-side programming languages.
  • At least one year of experience with Clojure (or Java plus another functional language)
  • Comfortable working at a startup, prior startup experience (fast-paced, rapid iteration, evolving requirements)
  • Willingness to own features from beginning to end
  • An understanding and preference for data-driven architectures
  • Must have ideas and opinions. What do _you_ want to explore?
  • Bonus: Any experience with AEC (Arch, eng, constructions) appreciated. Have you worked in the AEC industry before? Have you engaged in your own home renovations? Have you changed a light fixture or painted your home? Anything?

  • Bonus: Experience with AWS ecosystem and services such as S3, ECS, Fargate, Dynamodb, networking, etc. We use the AWS CDK for Infrastructure as Code (previously used Terraform).
  • Bonus: Previously worked with or explored Datomic
  • Bonus: Clojurescript and UI development
  • Bonus: Python (for scripting and integration)

FUNCTIONS

  • Participate in all phases of the work: design, coding, debugging, documentation, etc
  • Performs system analysis and programming activities that may require research.
  • Gather requirements from the product, marketing, and operations teams
  • Responsibility for managing your own tasks and sprints

PROJECTS

As part of a small team, you will be asked to work on a wide range of projects which will shift over time. You will have a lot of say in which projects you are interested in and will want to focus on. Potential projects include improvements to the Layout Engine, integration with the rendering engine and other tools, internal tools to support catalog maintenance, workflows, etc.

Permalink

Clojure Goodness: Create New Instance Of Java Class

Working with Java classes from Clojure code is easy. If we want to invoke methods or access fields on instances of Java classes we must first create an instance by invoking the constructor. In Clojure we can do that using the special form new or using a dot (.) notation. The new special form has the class name of the Java class we want to create an instance of as argument followed by arguments for the Java class constructor. With the dot notation we place a . after the class name followed by arguments for the class constructor to create a new instance.

Permalink

Configuring Emacs from Scratch — Intro

Configuring Emacs from Scratch — Intro

Intro

I have been using Emacs for 6 years now and for the last couple of years, Emacs has been my primary tool for writing Clojure at Helpshift Inc. Over the years, I have realized the value of Emacs. It has made me productive at work and org-mode has certainly made me more disciplined. The idea of having an editor which you can customize as it runs is itself amazing! :)

Unfortunately, I have also realized that getting on-boarded on Emacs is not an easy path. Many people try it for a few days and go back to their IDEs. So I’m going to try and write about customizing Emacs from scratch. My goal is to make this process simpler (or at least well documented) and help people get comfortable with Emacs until the point they realize its true value! :)

Installation and interactive tutorial

I’m going to use Emacs 26.1 on macOS. You can get it here for macOS, Linux or for Windows.
(The version doesn’t really matter. Most of the things written here will work for any Emacs version.)

Once you start Emacs, you will see this:

Startup screen

I would advise everyone to go through the “Emacs Tutorial” section displayed above. The Emacs way of doing things is a bit different and this interactive tutorial does a great job explaining it. It is a bit long but you won’t regret spending time on it.

But before you go through the tutorial, please consider mapping the Caps Lock key to control. Control is used a lot in Emacs, and if you don’t use Caps Lock as control, you’ll suffer from Emacs pinky. Here’s how to do it for macOS, Linux, and Windows.

If you don’t want to read this (you really should though!), here’s the gist of the tutorial:

Tutorial Summary

(If you have followed the interactive tutorial, skip this section and go to the Customizing defaults section)

  • Emacs uses the control and meta (alt/option) keys a lot. The key bindings with control and meta will be described as C-<key> and M-<key> respectively.
  • Press C-x C-c (C-x and then C-c)to exit Emacs.
  • C-g will quit the current operation. So if you are stuck and you don’t know what do to, press C-g.
  • The cursor location is called “point” in Emacs.
  • Use C-f for moving point forward, C-b for backward, C-n for going to next line and C-p for previous line.
  • C-a for going to beginning of current line. C-e for end of current line.
  • C-v for going to next page. M-v for previous page.
  • The above key bindings when used with meta will work on a word level abstractions. For example, M-f will move the cursor one word ahead (vs one character with C-f). M-a, M-e will move between sentences.
  • Files are opened in “buffers” in Emacs. There could be a buffer which is not associated with any file. But a file will always have a buffer associated with it when you open it in Emacs.
  • M-< (Meta + Less than) will take you to beginning of a buffer and M-> (Meta + Greater than) will take you to end of a buffer.
  • Press C-x k <RET> to kill a buffer.
  • Press C-x 3 to split the Emacs window horizontally. Press C-x 1 to make the split screen go away.
  • Press C-x C-f to open a file.
  • C-x C-s to save your edits.
  • When you open a few file, the previous buffer still remains active. To go to the previous buffer, press C-x b and type the buffer name (file name) and press <RET>.
  • To see the list of buffers, press C-x C-b. Press q to quit this window.
  • The term “Windows” in Emacs is not the same as the one used by your OS. In Emacs, the OS “windows” are called “frames”. The split screens in Emacs are called “windows”. For example, when you split your screen, you can say that you have 2 “windows”.
  • Use C-x o to move among windows.
  • C-d will delete a character in backward direction.
  • C-k will delete text from point to end of current line.
  • To search for text, press C-s and type the text to be searched for. Pressing C-s after that will take you to the next occurence of the text. C-s moves in the forward direction and C-r moves in the backward direction. Both have a wrapping behavior. Press RET to stop search and move the point to the current occurence.
  • If your point is on some text you want to search for, press C-s w to fill the search minibuffer with text under point.
  • You can use prefix arguments to repeat commands. Press C-u to give a prefix argument to a command. For example, to go 10 pages ahead, you can press C-u 10 C-v. Or to add a line of 72 *s, press C-u 70 *.
  • Instead of dragging to select text, press C-<SPC> (Control + Space) where you want to start dragging and then move the point to where you want to end. Text you have selected will be marked with a highlight. Press C-w to cut or M-w to copy.
  • To paste, press C-y. Pasting is called “yanking” in Emacs terms. “Cutting” is called “killing”.
  • You can also yank text that was killed previously, press C-y and then keep pressing M-y. You will go on seeing your kill history. This is because Emacs saves the text you kill in a ring data structure called “kill ring”.
  • Press C-/ to undo. Undo in Emacs is very different from other editors. (You can read about it in the Tutorial; or just play with it to know how it works.)
  • Emacs has a concept of modes. For any buffer, there is one major mode and there could be multiple minor modes. For example, if you open a python source code file, the major will be python-mode. Emacs will try to detect and use a proper major mode for any file that you open. Minor modes are modes that do small tasks such as auto indenting, spell checking, showing line numbers, etc. So more than one minor mode can be enabled per buffer.
  • The line that you see in the lower part of Emacs:
Mode line
  • It is called “mode line” and it displays useful information about the current buffer and the status of Emacs. For example, it displays which is the major mode that is currently enabled. In the above image, it is “Fundamental” mode.

Customizing defaults

Now that we have completed the interactive Emacs tutorial, we don’t want Emacs to show the tutorial message when Emacs starts. When Emacs starts up, it looks for one of these 2 files: ~/.emacs.d/init.el or ~/.emacs. If it finds one of these, it will start executing the code in these files. The code in these files is supposed to be Elisp code. Emacs Lisp or Elisp is the language created for and used by Emacs.

So now we will go ahead and create this file.

  • Press C-x f and write .emacs.d/init.el (~/ will already be there) and press <RET>. A new buffer will open up.
  • Write these contents in that buffer: (setq inhibit-startup-message t) and save (C-x C-s).
  • Quit Emacs and restart.
  • Emacs will not show the tutorial screen and instead it will start with a buffer named *scratch*. By convention, buffers which have names starting and ending with a * do not have any files associated with them.

Congratulations on your first Emacs customization! :)

Now let’s try to read and understand the Elisp code that we just wrote.

(setq inhibit-startup-message t)

In simple terms, setq is a function which sets value of the given variable to the given value. In this case, it will set inhibit-startup-message to t.

Now is a good time to use the self documenting feature of Emacs. setq is a function and we can get help on functions by pressing C-h f( C-h for help and f for functions). Press C-h f, type setq and press <RET>. A window will open up a buffer named *Help*(again, not associated with a file) which will contain information about the setq function. Press C-x 1 to maximize *scratch* again.

Now let’s see what inhibit-startup-message is. Since it is a variable, we will ask Emacs to tell us more about variables by pressing C-h v. Press C-h v, type inhibit-startup-message and press RET. Notice that for variables, their current value is also shown. This time to quit the documentation window, switch to it using C-x o and then press q.

Why was this help window quit when we pressed q? The *scratch* window not get quit on pressing q, it will just insert the letter q. Let’s find out why!

In Emacs, each key is bound to a function and there is a way to figure out which key is bound to which function. And these bindings are per buffer. So q must be bound to two different things as we see two different behaviors on pressing q.

Switch to the help buffer by pressing C-x b *Help* <RET>. Now let’s find out what q is bound to. Press C-h k and press q. “q runs the command quit-window”. Now we know why q is making the window go away; it was executing the function quit-window.

Let’s figure out what q is bound to in the *scratch* buffer. Quit the help window by pressing, guess what, q! You’ll be back on the *scratch* buffer. Now press C-h k and then q. “q runs the command self-insert-command”. self-insert-command just inserts the character that you have typed. So all the printable characters on the keyboard must be bound to “self-insert-command”. You can try verifying this. Also, you can read documentation for “self-insert-command” just like the way we read the documentation for setq.

Now we fully understand why q was acting differently in different buffers. And guess what, we did not leave Emacs even once! :)
(Something fun to try on your own: Find out what C-h k is bound to.)

Since we like to use the keyboard for doing all our tasks, the tool bar is not very useful for us.

Tool bar

So let’s disable it. Go to your init.el buffer and write (tool-bar-mode -1) on a new line. Now let’s make this code take effect without restarting Emacs.

init.el

Execute this code by going one character after the closing parenthesis. Press C-x C-e to execute this code. The tool bar is gone!
Congratulations on your first live Emacs customization! :)
(Now you should find out what C-x C-e is bound to. It is a good habit to always check which functions you are using instead of just remembering key bindings.)

Now let us disable the menu bar and the scroll bar as well. Put this code in a new line in ini.el:

(menu-bar-mode -1)
(scroll-bar-mode -1)

To clearly see which line you are on, you can have current line highlighted with this code: (global-hl-line-mode t). You will need to execute this line as we did before.

global-hl-line-mode

Another important thing which is missing is knowing the line numbers and column numbers. For that, add (line-number-mode t). You’ll see (line #, column #) coordinates of the point location in the mode line. To go to a particular line, press M-g M-g, enter the line number and press <RET>.

[If you use macOS, it is easier to use the “command” keys as “meta”. Add this line to make this happen: (setq mac-command-modifier 'meta).]

What we have in the last 4 lines of Elisp above is just changing minor modes. We have disabled minor modes which display the tool bar, the menu bar and the scroll bar; we have enabled a minor mode which highlights the current line in the buffer. If you want more information on the currently enabled major and minor modes, press C-h m. On pressing this, a minibuffer will open up. Amogst other things, this buffer will describe commonly required key bindings for the respective modes.

Now that we have seen how to configure Emacs defaults to suit your needs, you can read the “Use better defaults” section of my Emacs config and take the things that would be useful to you.
[Linux and macOS users: Please go through the “Better interaction with X clipboard”.]

Finally, let’s have a habit of writing comments for all the code that we have written. Our init.el will look like this:

https://medium.com/media/dbd6d07e8e1dc9e03bfe8134f8bf66bd/href

Conclusion

Throughout this session, we have used C-h <some key> many times. C-h is the prefix key for getting help in Emacs. We have used it quite well here and you should continue following this pattern of using help to know more about Emacs! :)
Knowing how to use the help system is crucial for mastering Emacs!

Next: Installing and configuring new packages


Configuring Emacs from Scratch — Intro was originally published in helpshift-engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Permalink

1.10.844 Release

We’re happy to announce a new release of ClojureScript. If you’re an existing user of ClojureScript please read over the following release notes carefully.

Noteworthy & Breaking Changes

  • Closure Compiler has been updated to v20210302

  • Closure Library has been updated to 0.0-20201211-3e6c510d

Note that the latest Closure Library release includes breaking changes that may impact your code:

  • goog.debug.Logger.Level has been renamed to goog.Logger.Level and the goog.log.Level/getLogger function now takes an additional argument.

  • The following JavaScript type checking predicates have been removed: goog/isString, goog/isArray, goog/isFunction, goog/isNumber, and goog/isBoolean.

  • Closure namespace loading logic has been revised, necessitating updates to some REPLs.

New Core Features

Library Property Namespaces

When consuming a JavaScript library which is exposed as a JavaScript object, you can now specify a property of this object to be used as a first-class namespace in ClojureScript. To do this, you use the $ delimiter.

For example, the SubLib property of a library object provided by npm-lib can be treated as a namespace like so:

(ns foo
  (:require [npm-lib$SubLib :as sub-lib :refer [...]]))

If needed, you can also employ string-based requires:

(ns foo
  (:require ["@npm/lib$SubLib" :as sub-lib :refer [...]]))

This feature can also be used to access default exports:

(ns foo
  (:require [npm-lib$default :as npm-lib :refer [...]]))

The $ delimiter is only needed to access the top-level object property; any nested properties are accessed via . as in the following example:

(ns foo
  (:require [react-native$NativeModules.SomeBridge :as woz]))

Notable Fixes

Accurate file name and line numbers in cljs.test

Instead of inaccurately inferring file name and line numbers from the call stack in do-report, they are now captured during macro-expansion based on metadata.

sort and sort-by retain meta

This simple change makes sort and sort-by consistent with Clojure. For example, the following evaluates to {:a true}:

(meta (sort (with-meta (range 10) {:a true})))

Floating point issues with range

Some floating point issues were addressed for range, making, for example (range 0 (+ 1 (/ 9)) (/ 9)) have 10 elements and (nth (range 0 1 0.1) 6) properly evaluate to 0.6.

#inst parsing and printing, reflect proleptic Gregorian

JavaScript employs a proleptic Gregorian date system and some bugs in ClojureScript’s #inst support is now fixed for very old dates with respect to parsing and printing #inst values.

Performance Improvements

Reduce code generated by destructure macro for maps

The code generated to implement map destructuring was optimized by introducing a shared helper to handle kw-args, reducing a portion of the Closure-optimized output from 35 bytes down to 5, a nice savings since map destructuring is frequently employed in code.

Change List

For a complete list of updates in ClojureScript 1.10.844 see Changes.

Contributors

Thanks to all of the community members who contributed to ClojureScript 1.10.844:

  • Arne Brasseur

  • Erik Assum

  • Hyunwoo Nam

  • Matthew Huebert

  • Thomas Heller

Permalink

State of Clojure 2021 Results

While a challenging year overall, 2020 was another good year for Clojure and its ecosystem. One big newsworthy item was that Cognitect joined Nubank, the world’s largest independent digital bank, to reinvent financial services across Latin America. The news was well received, and the general sentiment is summarized by a comment left by one of the survey respondents:

Great to see the Nubank acquisition of Cognitect. I think this has done wonders to bolster confidence in Clojure and Datomic for the long term. Also the subsequent support of open source developers sends a very positive message, and directly supports a healthy and robust ecosystem.
— anonymous survey respondent

Indeed Ed Wible, the co-founder of Nubank talked about Nubank’s commitment to supporting and growing Clojure (and Datomic) into the future.

In this post I’ll outline a few of the more salient points shown in the 2021 survey. Specifically, I’ll touch on the use of Clojure in production, the use of Clojure for start-ups, and how Clojure helps programmers and organizations to move quickly and confidently. As an added bonus I’ll highlight some of the amazing projects and people making the Clojure community strong.

Clojure in Production

Continuing the trend from last year, Clojure has seen growth in large companies (i.e. those having 100-1000 employees).

A breakdown of the companies sizes using Clojure in production

Nubank currently employs approximately 700 (at the time of writing) Clojure programmers, and Nubank is just one prominent example of the many banks, mortgage companies, and financial services companies using Clojure today, accounting for 22% (and growing) of the Clojure job market as reported by survey respondents. The survey shows healthy gains in the healthcare and retail markets as well.

Clojure finds adoption in a wide range of industries

Having said all of that, just where are Clojure projects deployed in production systems? Interestingly, the survey shows some nice growth in the way of public cloud and serverless deployments. The trends for AWS indicate that the cloud market is booming and so it’s natural that the general increase for that segment would show specifically for Clojure deployments as well.

Clojure (still) for Start-ups

While more large companies are adopting Clojure than ever, the sweet spot is still the smaller companies of less than 100 employees.

Clojure is still a language of choice for small companies

The reasons that start-ups choose Clojure are many and variegated:

  • Leverage - small effort, big result

  • Ready access to large existing ecosystems - Java, JavaScript, .NET

  • Scalable base - grow from PoC, to MVP, to production systems at scale

  • Moving fast - discussed in the next section

One exciting start-up of note in the Clojure ecosystem is Roam Research. Roam’s flagship product is an online note-taking tool that is designed to augment the human mind in building a set of notes that allow deep connectivity and discoverability. The enthusiasm for Roam is palpable and it’ll be fun to watch them grow.

Clojure for Moving Fast

Most survey respondents reported that they came to Clojure from the Java programming language. The fact that both languages run on the same virtual machine certainly helps smooth the path from Java to Clojure. However, many other advantages allow Clojure programmers a greater flexibility and agility in their day to day work lives using Clojure.

Functional Programming and Lispiness are great force-multipliers

First, Clojure programmers value a functional style of programming facilitating a separation of data and process. Coupled with its suite of immutable data structures, Clojure applications are often built as pipelines of data transformation functions that can be composed to implement higher-level business concepts in software. As a dialect of Lisp, Clojure provides an interactive programming environment called the REPL (Read, Eval, Print, Loop). The REPL fosters exploratory programming, allowing Clojure programmers to interactively develop solutions to sticky programming problems using a tight feedback loop. Indeed, the confluence of the REPL, immutable data structures, and functional programming allows for a development process light on ceremony, allowing programmers to focus on the problem at hand rather than the complexities foisted on them by their programming language.

The Clojure Ecosystem

Finally, Clojure would have never grown to its current size without the help of countless developers, technical writers, and big thinkers along the way. The appreciation for the amazing Clojure community is palpable within Nubank and this appreciation is increasingly being expressed as open source project funding. I’d like to close this year’s post by highlighting a number of the community leaders mentioned in the survey responses.

  • Bozhidar Batsov for continuing to improve Clojure tooling ecosystem.

  • Arne Brasseur - Kaocha is superb and his educational skills fantastic.

  • Everything Michiel Borkent aka borkdude touches is awesome!

  • Sean Corfield because he is always helpful and polite, and writes excellent documentation for important libraries.

  • Michael Fikes for being enthusiastic, engaging, and inspirational, and approachable.

  • Thomas Heller has made an amazing tool in shadow-cljs.

  • (Daniel) Higginbotham, because he made me laugh hundreds of times while introducing me to Clojure.

  • Tony Kay and contributors of Fulcro for providing a comprehensive solution to client-server state management.

  • Gene Kim for … unabashed love of the language, and for helping spread the good word.

  • London Clojurians - the online talks in the past year have been amazing.

  • Tiago Luchini, Hodur creator.

  • Carin Meier for her progressiveness: for advancing the state of the art in machine learning, and her contributions to visibility and representation in the community.

  • David Nolen is generous with explanations, direct, genuine, and never condescending or judgmental.

  • Eric Normand, as he provides great content for beginners.

  • Michael Nygard for architecture insights.

  • Dmitri Sotnikov for making web dev in Clojure accessible to the masses.

  • Peter Taoussanis - Nice, simple and complete libraries.

There are so many more who deserve praise and recognition so in the words of one of the survey respondents:

Clojure literally changed my life for the better. I am eternally grateful for the humans behind it.
— anonymous respondent

We too are grateful for everyone involved in the Clojure community and we hope to see a continuance and growth in support for those members of the community who work so hard and devote their valuable time to help Clojure and its ecosystem thrive.

Full Results

If you’d like to dig into the full results, you can find the complete set of data from this and former years here:

Thanks again for using Clojure and ClojureScript and participating in the survey!

Permalink

The beginner's way

By Ariel Alexi

An OOP developer finding her way in the functional world, what could go wrong?

So why Clojure?

This was my main question to every Clojure developer that I have met, while I was wrestling with myself about the answer to that question. During my studies for a computer science degree, I had the chance to code in various languages and try different technologies. Some of these languages were: OO languages (Python, Java, C++, and C#) and C. Although I am a junior OOP developer, developing into the functional world was always fascinating and interesting. That is why I had the question why from all the functional languages, I need to try and code in Clojure? While I was asking this question to different people, the answer was most often similar “I don’t know but..” and later on there was always the sentence of “…you have to try it and you will understand!”.

So, challenge accepted!

I am going to find out if Clojure makes it or breaks it for me. What could possibly go wrong? I am starting a new journey, a journey I think every junior developer must take - CODE AND TRY! Only by tapping away at the keyboard and trying to solve tasks, you learn what you want to do, and also what you never want to do again.

Bear in mind that I’m an Object-oriented programming (OOP) developer diving deep into the functional world, wish me luck!

What is Clojure and why should we care?

To quote Clojure.org:

Clojure is a dynamic, general-purpose programming language, combining the approachability and interactive development of a scripting language with an efficient and robust infrastructure for multithreaded programming. Clojure is a compiled language, yet remains completely dynamic – every feature supported by Clojure is supported at runtime. Clojure provides easy access to the Java frameworks, with optional type hints and type inference, to ensure that calls to Java can avoid reflection.

Where to start?

Clojure was the first language that I had to learn outside of any course at the university and don’t forget it is a functional language, YAY. It was me (with some help from my mentors) vs. Clojure.

1) Books:

There are different books I have tried reading during my study —

I found that I enjoyed reading the first book. It was a funny and interesting book for a software self-learning book. Along with the book, you get some funny and interesting assignments and read stories to make the journey more interesting. This book is not for everyone, but give it a try. If it is not your cup of tea try the second one, you can always learn in the direct and straightforward (but boring in my opinion) way with the Clojure documentation. From my point of view, you should not learn a new language by its docs. I think you should use the language docs only when you are familiar with things and you are searching for some more small details.

2) Videos

A quick intro for those who don’t know: Rich Hickey is a computer programmer and a speaker, known as the creator of the Clojure programming language.

As a part of my journey in discovering Clojure, I had an assignment to watch one of Rich Hickey’s videos every week. It may sound weird but give yourself an hour to watch them every week. Part of learning Clojure is to understand where and why we need and want it. Rich Hickey’s talks will give you a different perspective about Clojure and coding. In the beginning, I didn’t understand what everyone wanted from me, I had no opinion about them but as I continued watching more videos I started to consolidate my own opinion on the videos and the coding world as a programmer.

This is the list of Rich Hickey’s videos I watched, in the order I watched them:

I think the 3rd video is a must for every developer out there, no matter if you want to use Clojure or not.

You can find a complete list of here as well as detailed transcripts for most of them.

How was it to learn Clojure?

As an OOP developer, my first few days with Clojure were something like Conan O’Brien crying

News flash: learning a new language is hard, especially if you are a junior developer. It is difficult because learning a language that is so different from the concepts and fundamentals you already know is hard, so do give yourself some slack.

As an OOP developer, I needed to change my mindset about this new thing and its concept which means - in Clojure there are NO objects! (at least not that I recognized).

You only get data structures and functions, and it’s not an easy task for an OOP developer to throw objects out the window and act like it is all fine and easy. On the bright side, Clojure has a small box of functions and data structures you need to get familiar with and you just need to know how to manipulate them.

But don’t you worry - my opinion about Clojure has changed. Now I mostly feel like this when I’m coding:

You must have been wondering how I have changed my mind over my first month, but this I shall not reveal now. All that I wrote here, the books, and the videos are the first baby steps that usually Clojure developers take before they start to code and practice the examples in the books or the websites. Those are just the ordinary baby steps that people usually take, whereas I believe you should do it differently.

To learn more, you should read my next post.

See you there.

Permalink

Copyright © 2009, Planet Clojure. No rights reserved.
Planet Clojure is maintained by Baishamapayan Ghose.
Clojure and the Clojure logo are Copyright © 2008-2009, Rich Hickey.
Theme by Brajeshwar.