Migrating this blog from Octopress to Babashka

Octopress

Seven years ago I started writing about my adventures with Clojure. I started using Octopress back then and was still using it until today. In recent years I became frustrated with it since on every new system I had to install a version of Ruby that happened to work with Octopress, which wasn't always the case. When it did work, I started getting deprecation warnings from Ruby while compiling this blog. On top of this, Octopress seemed no longer maintained. The website says "Octopress 3.0 is Coming" but this announcement was from 2015 and the website hasn't been updated ever since. It felt like time to move on.

Requirements for replacement

I started thinking of what was essential for me when writing blog posts and came up with this list of requirements / wishes:

  • Port the existing blog with as little effort as possible
  • Writing blog posts in a convenient format: markdown is good enough, let's keep it
  • Don't use a framework like Octopress again: it isn't worth the learning curve for me
  • Don't get stuck in a language I don't use every day. The deprecation warnings in Ruby scared me and it was time-consuming to install or update the tooling that I didn't use on a daily basis.

My blog has the following essential components:

  • Archive page with list of all blog posts written so far
  • Index page that lists the most recent blog posts
  • Each post has its own HTML page
  • An atom.xml feed and a bespoke planetclojure.xml feed for Planet Clojure
  • Highlighting for Clojure code snippets: not super important, but I'd like to at least have this back after a rewrite.

The primary goal of this blog is to share some of my experiences. Of secondary importance:

  • A way for users to directly react. For now I can live with users reacting via Twitter, Clojurians Slack, etc. and I can figure this out later.
  • A fancy web design.

The language I am most fluent in is Clojure. Babashka is a scripting tool that has similar startup characteristics as Ruby so it would be nice to keep that from my Octopress experience. Let's see if I can rewrite my blog with a babashka script.

Rewriting in babashka

I started with copying each markdown file in source/_posts and moved them to a directory posts. In Octopress, blog posts start with a section like:

---
layout: post
title: Figwheel keep Om turning!
date: 2014-09-25 21:00:50 +0200
comments: true
published: true
categories: [clojure, clojurescript, figwheel, om]
---

I removed these sections replaced them with maps in a file called posts.edn:

{:title "Figwheel keep Om turning!"
 :file "figwheel-keep-om-turning.md"
 :date "2014-09-25"
 :categories #{"clojure" "clojurescript" "figwheel" "om"}
 :legacy true}

The maps are top level values so I can easily append new ones programmatically. The :legacy true flag is for blog posts that existed before the rewrite so I can create redirect pages for them. I plan to no longer include the date in the direct blog post URLs, but I don't want to break thr web for older blog posts.

Then I started writing the render.clj script which iterates over every entry in posts.edn and renders markdown files to HTML.

To do markdown rendering from babashka I'm using bootleg by Crispin Wellington which is available in the pod registry.

I had to fix a couple of things to get the rendering back that I had with Octopress. Links without markup, so https://foobar.com instead of [foobar](https://foobar.com) were rendered as an a element before, but bootleg, which uses markdown-clj doesn't do this out of the box.

Another tweak I had to implement is support line breaks in the middle of markdown link syntax, since emacs's fill-paragraph sometimes causes that to happen:

[foo
bar](https://foobar.com)

The tweaks:

(pods/load-pod 'retrogradeorbit/bootleg "0.1.9")

(require '[pod.retrogradeorbit.bootleg.markdown :as md])

(defn markdown->html [file]
  (let [markdown (slurp file)
        ;; make links without markup clickable
        markdown (str/replace markdown #"http[A-Za-z0-9/:.=#?_-]+([\s])"
                              (fn [[match ws]]
                                (format "[%s](%s)%s"
                                        (str/trim match)
                                        (str/trim match)
                                        ws)))
        ;; allow links with markup over multiple lines
        markdown (str/replace markdown #"\[[^\]]+\n"
                              (fn [match]
                                (str/replace match "\n" "$RET$")))
        hiccup (md/markdown markdown :data)
        html (-> hiccup
                 (utils/convert-to :html))
        html (str/replace html "$RET$" "\n")]
    html))

After those workarounds, it was pretty straightforward to support the most essential things my Octopress blog did.

Highlighting

Clojure syntax highlighting works pretty well with highlight.js which I include like this:

<link rel="stylesheet"
      href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/11.3.1/styles/default.min.css">
<script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/11.3.1/highlight.min.js"></script>
<script>hljs.highlightAll();</script>

After that it just pretty much worked. The Clojure syntax highlighting isn't perfect, but good enough for a blog with the occasional snippet in my opinion.

Feeds

The generation of atom.xml and planetclojure.xml (which only contains Clojure-related posts) were easy to implement with a bit of clojure.data.xml code:

(def blog-root "http://blog.michielborkent.nl/")

(defn atom-feed
  ;; validate at [https://validator.w3.org/feed/check.cgi](https://validator.w3.org/feed/check.cgi)
  [posts]
  (-> (xml/sexp-as-element
       [::atom/feed
        {:xmlns "http://www.w3.org/2005/Atom"}
        [::atom/title "REPL adventures"]
        [::atom/link {:href "http://blog.michielborkent.nl/atom.xml" :rel "self"}]
        [::atom/link {:href "http://blog.michielborkent.nl"}]
        [::atom/updated (rfc-3339-now)]
        [::atom/id blog-root]
        [::atom/author
         [::atom/name "Michiel Borkent"]]
        (for [{:keys [title date file]} posts]
          [::atom/entry
           [::atom/id (str blog-root (str/replace file ".md" ".html"))]
           [::atom/title title]
           [::atom/updated (rfc-3339 date)]
           [::atom/content {:type "html"}
            [:-cdata (get @bodies file)]]])])
      xml/indent-str))

(spit (fs/file out-dir "atom.xml") (atom-feed posts))
(spit (fs/file out-dir "planetclojure.xml")
      (atom-feed (filter
                  (fn [post]
                    (some (:categories post) ["clojure" "clojurescript"]))
                  posts)))

This feed validator gave pretty good feedback on what should and should not go into this XML file.

Rake -> bb tasks

Finally I wrote a bb.edn file with babashka tasks to create new blog posts. E.g. I created this very blog post using:

$ bb new :file migrating-octopress-to-babashka.md :title "Migrating this blog from octopress to babashka"

All implemented tasks so far:

$ bb tasks
The following tasks are available:

new     Create new blog article
render  Render blog
watch   Watch posts and templates and call render on file changes
publish Publish to blog.michielborkent.nl

This replaces the rake stuff I used to have with Octopress.

Conclusion

I'm pretty happy with how far I got with a couple of hours hacking on the babashka scripts and tasks. The meat of the work is in render.clj which is under 170 lines of Clojure. I feel in control over my own blog again. From here on I can look for a plugin which lets users respond to blog posts and perhaps a fancier design. If you anything useful to share regarding this, please let me know on e.g. Twitter.

The code for this blog is available here.

Permalink

3 Facts about Clojure every beginner must know

Are you someone who is thinking of learning Clojure and Functional Programming? I’ve just started and I hope I can help you!

I am Sameeksha — a 2021 graduate working as a Software Engineer in the data platform team at Helpshift. Clojure is the primarily used language at Helpshift. As part of my on-boarding process, I was introduced to Clojure. While learning the language there were a few things that I found were important to be drilled in the mind and especially when you are just starting. So without further ado let's get started.

Brief Introduction to Clojure

Clojure is a functional programming language, everything here revolves around functions. It belongs to the LISP (List Processing) family of languages. Functions can be passed as arguments to other functions and they can be returned as output values. Pretty cool isn't it!

Let's start understanding with a basic example of how we add 2 numbers in Python and print the result?

print(1 + 2)

Now how to add two numbers in Clojure

 (+ 1 2)

Notice the syntax, it is (Function argument1 argument2 ).
Everything in Clojure is surrounded by parentheses. Clojure uses the prefix notation.

Also note, here ‘+’ is not a mathematical operator but in fact a Clojure function.
The evaluation scheme in Clojure treats this as a list where the first value is a function and the rest all are the arguments.

Fact #1

Lists are an important and special data structure in Clojure.
The first point of our discussion is why is it so. Let's try to understand this.

For instance

'("Clojure" "Python" "Java" )

This is treated as a list because of the presence of a single quote at the beginning. The reason for having this quote is because in LISP the first element will be the function and anything after that is considered as data for that function.

In absence of the quote at the beginning, Clojure will throw an error that It was trying to find a function but instead it found a string.

Another example

'(+ 2 3) ;=> (+ 2 3)

Here even though we were trying to perform addition which ideally should have returned us the value 5 it returned (+ 2 3) itself because of the presence of the single quote.

This leads us to the point that in Clojure ‘Code is Data’. We were trying to write the code for addition but because of a presence of a single quote in this case it was treated the same as data.

Now say, we want to use the result of this addition later at some point in time, can I store this in a variable?

https://medium.com/media/a42bf2956d32d9c987443140381f3d53/href

You can’t store anything in a variable, in fact, there is no concept of variables. All the core data structures in Clojure are Immutable. In Java, strings are immutable similarly the value 42 is immutable. Clojure has extended this ideology for all its core data structures.

To understand this point of immutability lets consider an example of a tree shown below

Suppose you need to add a new node called ‘s’ as a left child of node ‘f’.
Think for a moment about how is it going to add when I just said that the data structures in Clojure are immutable.

Now look at the image below

Fact #2

A brand new object is created and the new node ‘s’ is present in it.
Clojure does something called ‘Structural sharing’ behind the scenes where it ensures that a minimum amount of copying is done.

To add node ‘s’ its parent ‘f’ was needed similarly node ‘a’ and copy of node ‘c’ had to be made. Rest all the nodes are being shared. The left part of node ‘a’ is something we weren't touching to hence the new object created called ‘tree_2’ will simply point node a’s left to already existing node b.
This ensures immutability.

Let's take our discussion to another interesting fact which is about Truthiness. Assume a use case where you want to be inside the loop until the vector is empty.

(Note: I am taking an example in Python and referring to the sample code in it as vectors only for the sake of simplicity. In Python, although it is called lists.)

In Python or any OOPs languages, you could have simply answered this by having the while condition as follow

v = []
while v:
print(v)

You already know that whenever the vector will be empty the while condition will turn to be false and the loop will break.

In Clojure however, if you first try to run the below two examples in a REPL you'll notice that when the vector is empty in that case as well Clojure evaluates this to be true.

(if (count [1 2 3 4]) :truthy :falsey) ;=> :truthy
(if (count []) :truthy :falsey) ;=> :truthy

Fact #3

Every value except FALSE and NIL is treated as true.
An empty string, vectors of length 0, number 0 are all treated as true.

So what will you do to check if a collection like a vector or a list is empty?

Answer: use seq.
seq returns the sequence view of a collection and returns nil if the collection is empty.

(seq []) ;==> nil

These were the 3 things that I wanted to talk about in this post.

We started our discussion with how Lists are a key data structure in Clojure
Further, we discussed immutability where even if you try to make an update to an existing object Clojure will create a new object itself. This new object however will persist the previous version so only the new update which is needed will be done. Lastly, we learned that every value except nil and false is treated as true.

In the next post, I'll talk about Java interop and also how can we take up cases where we actually need to mutate something in Clojure and how it happens.

Thanks for reading till the end.

(str "Thank you !!")

3 Facts about Clojure every beginner must know was originally published in helpshift-engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Permalink

Meta Reduce 2021.3: Back in Action

This is going to be a pretty short status update, as very little has happened since my last “Meta Reduce” post. I was mostly enjoying the last weeks of my sabbatical and I didn’t do almost any work on my OSS projects. That being said, now I’m finally back home and I’m ready for some action.

OSS Projects

My biggest short-term goals are new releases of nREPL and CIDER. Both are almost ready and I expect them to go live in the next couple of weeks. A big update of clj-refactor.el is also on the horizon. I also have to submit CIDER and clj-refactor.el to the NonGNU ELPA package repository.

I don’t expect any significant developments on my other projects, but I’d be happy to be surprised there.

Meta

I’ve spent the final weeks of my vacation mostly traveling in France and reading.1 I’m really glad that I finally found the time to read “Only the Paranoid Survive” by Andy Grove. I’ve been meaning to read this book for years and it didn’t disappoint! I also enjoyed reading the autobiography of Walmart’s founder “Sam Walton: Made in America”.

In the mean time I also finished re-watching Daniel Craig’s James Bond movies and I’m finally ready to see “No Time to Die”. And the new “Dune” movie! I haven’t been to the cinema in ages, but I’ll make an exception for those two movies. I also finished watching “Sex Education” season 3, and I have to say that this was the worst season of the show so far. Yesterday I wrapped up “Squid Game” as well - a decent show, with plenty of flaws, that didn’t live up to the hype. Still, at least now I understand all the “Squid Game” memes out there.

I was very excited the read about the partnership between Fastmail and 1Password, that allows 1Password to generate unique e-mail aliases for each service that you sign up for. That’s a great step for privacy, and some very fortunate timing for me, as I switched to both services just a few months ago. This also convinced me that for the time being 1Password fits better my needs than Bitwarden. I was really on the fence about which service did I like more before the Fastmail news.

I continued to play with HEY World and I wrote a couple of short articles there:

I guess it’s safe to say I’m quite excited about the freshly announced MacBook Pros and quite upset about the COVID-19 situation in Bulgaria. I like HEY World and I’ll probably keep using for sharing short musings, that don’t fit into a tweet or two. I continue to favor the use of Jekyll (and Emacs) for most of my blogging.

Last, but not least - I’ve kept refining my StarCraf II game and I have the feeling I’m doing slightly better as of late. With infinite choice of strategies that you and your opponents can go for, that’s one of the toughest and most mentally rewarding computer games I’ve ever played. And one massive time sink!

Epilogue

And that’s a wrap. Now that I’m full of energy after my long vacation I hope to make some good progress on my OSS projects. By the way - I can heartily recommend doing a long vacation to everyone, especially given how tough the last couple of years have been on all of us.

  1. And eating! French food is great and now I don’t fit into half my pants. 

Permalink

Experiments with “AI” and Diet Logging

I recently fell into a spell of P.G Wodehouse novels. His style inspired me 1, and I scribbled this essay to experiment with it. It’s an eccentric mix of a “true” story (heavily embellished for comic effect), with a sprinkle of some ideas that may actually be useful. I hope you enjoy it!


Sometimes, you have a period in your life where all the cards are just right and every wish seems to conjure up before you, as if the universe decided to give you special treatment. A “personal renaissance”, if you will. I had once such period a few years ago, and it precipitated a multi-year quest. I wanted to relate the tale to you.

The Sauna

Our tale starts in 2018, when I worked at Facebook. I was hacking away at a product launch for F8 (our yearly developer conference), and we were a few weeks away from the big day. The inevitable crunch led me to a kind-of-dizzying-yet-thrilling quantity of work, and I found myself ubering back from the office every night at 9PM.

On one such an occasion, an indescribable urge came over me…I absolutely needed to go to the Sauna. A stranded dog would miss their owner less than I missed the vivifying steam of the Sauna. This may sound unusual, but at least in Georgia (the country I’m from) a good hour in heat is a staple for stress relief and general grooming (even Stalin was a fan), and boy was I in need.

Now, I was hazily aware that Archimedes Banya was nearby, but the idea of calling them and making a reservation plunged my uber ride in an atmosphere of dread.

If you asked any of my friends how I felt about making appointments, even the most charitable would say that unless I had a surge of some exceptional iron will, I’d have to be dragged kicking and screaming to the task, and unless vigilantly watched, I’d figure out some way to noodle out of it.

Well, there was no iron will left that day, and threats from a KGB drill sergeant couldn’t have forced me to make a phone call. So, temporarily defeated, gazing out of the window into the twilight, I made a quiet promise to myself. Once I finished with this launch, I said, I would solve this problem once and for all. I would set things up so I could plunge into a Sauna whenever I darn well pleased — appointments and phone calls be damned.

Fast forward a few weeks, and I signed the dotted line at the local Equinox. It was an ingenious hack if I say so myself: cheaper than going to a spa, available late into the evening, and I didn’t need to make an appointment. They even had two Saunas at the market street location: a lemon-scented steam room and the traditional wooden cabin room.

Boy did I revel in this victory. Once every few days, I’d rocket into the Equinox and shoot towards the change room. Determined and with towel in hand, I’d slow down my pace to a saunter, and eye the steam room. A leisurely hour of heat and ritual of cold plunges later, I’d strut out a newer, calmer man.

The ritual complete, I’d meander to the nearby Katsu Curry Place, with the air I imagine Napolean took on when he went on an evening stroll in Versailles. I’d finish the day with the satisfaction of a man who knew he had spent an evening he wouldn’t regret.

The Habit

Now, if any person heads into the change room of a gym every day for a month, and notices that the other folks seem to be gung-ho about what happens after the change room, they’re bound to get curious. One day, the urge to explore overtook me, and I delayed my usual leg towards the curry place, and took a gander at the facilities.

Well…it wasn’t so bad! A few bicep curls later, I thought it wouldn’t hurt to delay the restaurant and fit in a workout once in a while. Some weeks of this, and I began to noticed physical and emotional changes.

The progress inspired me to work out more, which produced more progress, and the kind of habit-forming loop that James Clear 2 himself would be proud of emerged. Victory upon victory, I felt. Now I had daily access to two Saunas, and workouts had become a routine.

Discovering my Trainer

Well, the gift kept on giving. One day at the Equinox a friend and I were taking a rest after a taxing set of squats, when he whispered, “Hey, take a look at the guy behind you”.

I rotated my gaze, to see a gentleman performing a superset of pushups with the exactness of a surgeon suturing a 0.3 millimeter blood vessel.

His form was so impeccable that even the uninitiated would stagger in their track to see what was happening, much in the same way I guess a lay carpenter would have if he had seen Michelangelo sculpting away. Whatever this gentleman was doing, you could tell that he was serous about it, and that it worked.

It turned out he was a trainer at Equinox, and a person with a heart of gold too. I started working with him a few weeks later, and experienced a change in dimension for the workouts. Progress was ever-abundant.

The Problem

A few months of this, and health was going swimmingly. Well, sort of. My trainer told me that to get to the next level, I had to re-examine this Katsu-Curry et al habit of mine.

He suggested that I use MyFitnessPal to log my diet. I went ahead and installed the app.

From the login screen on I had a grim sense of foreboding. This did not look like an app where the designers pained away their nights, agonizing over user experience.

My fears were well founded. I was first plunged into ads. I realized to my horror, that I had to tap at least 6 times 3 to log anything! A few days of half-hearted effort and I dropped the app. Well, I had a problem on my hands, but, I now had a weapon to solve it. I surged straight to the sauna.

As I mulled the problem over, an idea arose. Instead of this tedious task of tapping 6 times to log food, what if I started a new company, built an app were you could just take a picture of your food, and it would handle the rest?

Thrilling.

“AI”

It would be the latest and greatest in AI. In some sense this would get dangerously close to a level of complexity that no app previously had touched, but I thought — heck, why not try?

So, I did what every founder who wants to solve an AI problem verging on general intelligence would do — I started with a “human-augmented” solution. I tabbed over to Upwork, and wrote a job posting:

image

The Job posting. Admittedly the response times were a bit too optimistic

Within a day, I got a storm of responses, filtered the ones who didn’t read the “When applying” section, and gave them all a test: I asked them to log into MyFitnessPal, and show me what they saw.

This task turned out to be insurmountable for the majority, and filtered about 90% of the applicants. Some more sleuthing, and I landed on a skilled assistant.

The Initial Experience

So, following the footsteps of the great scientists, I started the next day and sent my first photo across the ether.

At the outset I was faced with a shock: I had to tap at least 5 times to send a photo on Messenger! 4. It was frustrating, but all was not lost. I consoled myself, that at least sending a voice message was 4 taps, and in the case where I had to log some custom food, I’d be in the black with convenience. So, I trekked forwards.

A day or two in, a second shock sprung forth. I noticed that with most food, I would have preferred to have done a quick search myself, or to have picked from a “recently used” component, rather than to have gone through the “AI” process.

I co-discovered, alongside a many an AI enthusiast, that sometimes a good UX can do wonders. Here, I consoled myself again. I reasoned that I could *use the “re-send” option on Messenger, to achieve a workable UX. Plus, if this was to become a business, I could* build a companion app that had this too. So, I marched on.

A more painful shock came, when I discovered myself painstakingly crafting 30 second videos to log food, putting in much the same effort that photographers do when working with Victoria’s Secret models. I made sure every detail my assistant needed to log accurately was in the frame, and this toil was inching me closer to the red when it came to convenience.

The Last Straw

Now, these changes I invented for myself where painful, but pain is much easier to bear when you inflict it on yourself, so I bit through all and trucked along for a few weeks more.

To my infinite chagrin, there was bombshell that I just couldn’t ignore.

None of it worked! My nutrition wasn’t changing.

Whether I sent photos or not, I’d discover myself at the Whole Foods in the middle of the day, contentedly inhaling a few bags of cherries. Yes, that’s right, bags (In my defense, I felt like during cherry season it’s unreasonable, every store flaunting them around so much it’s perilous for an eye to wander).

The “AI-Alternative”

So, I came back to the drawing board. After some grave reflection, one idea struck me.

Maybe I was “forcing” this AI-type-of-solution — A “hammer looking for a nail”, if you will. What if I looked at the problem with fresh eyes? Thinking “from first principles”, as Elon Musk would say.

So, I reflected on this, and a rough product spec began to materialize. I thought to myself: “Hey, maybe I don’t really need to log calories at all. What if I just took photos, and made it shareable with my trainer?”

This way, he could look over it once every few weeks, and we could make adjustments as needed.

Clojure to the Rescue

So, I shared this idea with a friend of mine, and we jumped into hacking up a solution in Clojure.

This time, we would use SMS instead of Messenger. This had the marked benefit, that it would reduce the number of taps, and open up the option of IOS shortcuts down the road. I would simply send photos, and it would log it in the database. My trainer could then open a URL, and see everything that was logged by date.

image

The AI hacker’s NLP: string/lower-case, string/trim, string/includes?

I felt like I had come across the first-principled solution alright.

I contentedly took pictures of my food, and week over week, my trainer kept telling me to lay off some variety of cherries, mandarins, plums, apricots, or Katsu Curry. Fruits was such a common pattern in the diet, that we affectionately called our project “Pluot” — a hybrid of plums and apricots (this is a real fruit).

…The Second Defeat

Well, after a month of logging, the first-principled solution also fell dead into the water.

The program itself was fine — apart from a light desire to “edit” the timestamp on photos — it worked well enough. The issue was, that this just didn’t lead to any change in my nutrition.

No matter the feedback, I found myself myself in Facebook’s “Micro Kitchen” 5, polishing off the office’s supply of mandarins. At this point, I had tried the AI solution, and I had tried the first-principles solution. I was at a loss for what to do.

So, I did what I guess every gentleman does after such a sound defeat. I put the past behind me, and joined some military excursion in Africa.

Okay, I wish — unfortunately I was a few generations too late for this kind of thing, so instead I did the millennial analogue. I plunged myself into hacking on products, went to Machu Pichu and Africa (Admittedly, with more convenience than what similar spirited youths experienced in the past. On our hike in Machu Pichu, porters were somehow able to bake a cake in the wilderness)

image

A modern byronic adventure

The Haze

In the same way the military man back from Africa is no longer the diffident young English lord he once was at the outset, I think my friends would all agree, that this period had changed me: I had “bloomed”, if you will.

All the travel and the hacking had its effect, but, perhaps even more importantly, I had continued my reading habit, and it evolved to unnatural proportions. I think if Amazon had been a local store, they would have been suspicious about me being a reseller of sorts.

You see, the time a gentleman consumes waiting in line for coffee (especially in San Francisco coffee shops) and airport delays adds up. Combine that with a predisposition to lounge around on Sundays, and you have a person with a formidable library.

A Lebanese Light

Over those years, Nassim Taleb was one of the two authors who I found myself addictively re-reading. His work is the kind that protests at being read while waiting in line at a coffee shop, and forces you to find a chair to sit and nurse an Americano too in rapt attention.

The experience is similar to visiting an eccentric grandfather at his luxurious country house, to discover him lounging in the veranda, a half-finished bottle of self-made wine on the table, ready to impart you with wisdom and wit.

Now, I had already read Nassim’s work in the outset of the story, but as I said, his work changes with you, and I had undergone some change. Plus, you’re bound to understand a work differently after the sixth re-read.

On one such a Sunday, I was perusing over his ideas on Phenomenology and Theory, when I had a breakthrough. He says that Phenomenology (the study of “what works”) is more robust than Theory (the study of “why” things work).

This plays out intuitively. In finance for example, you can be sure that there are going to be market cycles and sharp swings in price. But why? Well, the answer changes every year. You’ll find many an economist with strong opinions, but without consensus in sight.

The nub is, we love theories, to the point where we can’t help but be fooled by them. You pick some theory about why foreign bonds should cost what they cost for example, and proceed to plunge down a highway to bankruptcy, while you tell yourself how the “market is irrational” along each setback 6.

We never get apprised of this lesson. Each time, a new economist integrate the “data point” that got away (a euphemism for bankruptcy in this case), and comes up with a new theory. Moving forward, this new theory seems indestructible and accurately predicts the past. Of course, the future is another thing.

Phenomenology of Nutrition

From there I grasped the errors of my youthful adventure. When I started on my quest, I had never successfully changed my nutrition before….yet, there I was, confidently picking theories, hiring assistants, and plunging straight towards defeat.

With my new wordly wisdom under my belt, I reconsidered the problem. What was the phenomenology behind good nutrition?

Instead of starting off with theory, I started by looking at what my friends who had succeeded in losing weight had actually done. I went back to the advice my trainer gave me. My best friend and co-founder Joe had achieved success here, and I took a closer look at what he did.

Before, when I saw him log his diet for example, I was certain that he had either an inordinate amount of self-discipline, or a Mother Theresa-like patience when it came to poorly-designed iPhone apps.

But that wasn’t the case. He simply logged differently. When in a crowded place for example, he’d take pictures so he could log at a later time. If it would be particularly tedious to log a specific food (say he was stranded in the middle of Mexico, and got pollo con idunno), he’d find something analogous (Chipotle) and log that.

A series of these optimizations had led him to log consistently, with enough accuracy to diagnose issues as they came along. To steal the story from Nassim, it made me feel I like the way the first weary traveler lugging his luggage must have felt, when he discovered the invention of the wheeled suitcase.

The Intangible

At a glance, the ideas seemed simple and theorizable. For example: maybe it’s all about the balance of accuracy and consistency in your logging. But, before we move too fast: what about the adjacent skills that it created? As I logged more for example, in the same way a horse-whisperer learns to communicate with the animal, I felt like I began to “sense” the nutritional makeup of food. The schelps I underwent attuned me to how certain foods made me feel, and how it would affect the rest of my day. Where do we include this in the theory?

The most judicious path, I learned, was to rely on experimentation. Start with what works, and tweak from there. Joe and I continued to experiment, ended up hacking up a business in the process.

Group 1 (5)

That realization in one sense completed the quest, but in another, kicked off a whole new one. There’s so much more to observe and learn.

1

Here’s one to start with: Leave it to Psmith.

2

Check out Atomic Habits

3

Tap MFP → Tap (+) → Tap Food → Tap time period → Tap item → Tap checkmark

4

Tap Messenger → Tap Conversation → Tap Camera → Take Photo → Send

5

A small kitchen stocked with goodies of all kinds, situated a thirty second saunter away from any desk.

6

An instructive story.


Are you thinking seriously about your health and fitness? Want to take it to the next level? Joe and I are hacking on consistent.fit. The current cohort is closed, but sign up for the waitlist, if you’d like to be notified when we finish this one!


Thanks to Joe Averbukh, Daniel Woelfel, Ian Sinnott, Jacky Wang, Mark Shlick, for reviewing drafts of this essay

Permalink

Using Discord for Open-Source Projects

You might have noticed that this year I’ve been promoting Discord as the main chat for my bigger OSS projects like RuboCop and CIDER. You may have been surprised by this move, as Discord originated in the gaming community and is still mostly associated with it. That’s why I decided to explain my reasoning in this short article.

I discovered Discord at the beginning of the year, and I like most people I assumed it was just some chat for gamers.1 However, after playing with it for a few months I figured out it’s a great fit for the needs of my projects for several reasons:

  • You can create new Discord servers2 for each project, where you can create a nice channels structure like #support, #hacking, #general, #feature-requests, etc.3
  • The UI is simple and clean, and the client application is not a resource hog (unlike Slack). The UI assessment is obviously subjective, but a lot of people I’ve spoken with on the subject voiced the same opinion.
  • Even on a free plan you have unlimited chat history retention. That’s one of my main issues with Slack and the famous Clojurians Slack, often the history there disappears before I’ve had the chance to read something.
  • Discord’s voice channels are a nice way to quickly speak with someone on some topic. Admitted I use those mostly while gaming, but I definitely see the potential for doing something like “office hours” with your community or whatever.
  • I like Discord’s version of threads a lot more that Slack’s. Slack threads are just too easy to miss.
  • Discord has very nice moderation tools.
  • It’s a cross-platform tool. Although I guess most of the famous chat apps are cross-platform apps these days.
  • It’s not Slack.

Discord reminds me a lot of the early days of Slack, where it was clear the target audience where hackers (compared to big enterprises today) and the whole UX was optimized for this. Compare this to all the buttons and menus that have replaced the old days of markup and text commands.

Obviously Discord is not perfect (which tool is?), but I like it a lot and I’ve decided to place my bets on it. I’m well aware that it’s a proprietary tool, and that there were some privacy concerns about it in the past, but that’s fine with me, given the lack of good (in my subjective perspective) truly free alternatives. I never forget that perfect is the enemy of done.

There’s also the meta matter that chat never really took off for any of my projects, expect CIDER’s Slack channel on the Clojurians Slack (#cider). I guess the majority of people simply prefer to use only GitHub for project-related conversations and that’s perfectly fine for me. If I weren’t using Discord for other purposes, I doubt I would have bothered installing it just to chat about my projects. Same with Slack, really - the only reason I’m the Clojurians Slack is the fact that I use Slack for work. I guess that’s also the reason why Gitter never took off, even if it was designed to be a chat for OSS projects - nobody was using outside of OSS, which really limited its appeal.

Anyways, that’s all from me for today. I just wanted to encourage people looking to add some chat to their OSS projects to consider Discord. I’m also curious to hear what chats are you using to collaborate on OSS projects if any. Feel free to share your thoughts in the comments.

  1. Truth be told, I kept confusing Discord with Discourse. 

  2. You can think of Discord servers as an instance of Discord or an isolated namespace. They are basically the same as workspaces in Slack. 

  3. I wrote a bit more about the structure of CIDER’s Discord here

Permalink

Sending notifications in Clojure web app — part 2— multi-agent using Duct

Sending notifications in Clojure web app — part 2— multi-agent using Duct

These meerkats stay alert. Notifications can send alerts too. See? This image is relevant to the content.

In the previous chapter we learned how to manage sending notifications to users via websockets. Most probably we’ll be hitting this whenever something interesting happens in the application that’s worthwhile of other users’ attention:

(defn submit-comment!
[comment]
(actually-persist-the-comment)
; and now notification side-effect:
(let [users-to-notify (a-function-to-get-subscribed-users)]
(websocket/notify-clients!
{:channels (websocket/users->channels users-to-notify)
:message {:category :new-comment
:payload
{:post-id 999
:comment comment}}})))

But what if we want to add another notification engines like email, text message, etc? Without going into implementation details of those agents themselves we can already tell that the approach above is hardly scalable:

(defn submit-comment!
[comment]
(actually-persist-the-comment)
; and now notification side-effect:
(let [users-to-notify (a-function-to-get-subscribed-users)]
(notification.websocket/notify-clients!
{:channels (notification.websocket/users->channels
[{:user-id 111
:name "Bob"}
{:user-id 222
:name "Jill"}])
:message {:category :new-comment
:payload
{:post-id 999
:comment comment}}})
(notification.email/notify-recipients!
{:recipients users-to-notify
:message {:category :new-comment
:payload
{:post-id 999
:comment comment}}})
(notification.sms/notify-recipients!
{:recipients users-to-notify
:message {:category :new-comment
:payload
{:post-id 999
:comment comment}}})))

Of course we can abstract it away behind a simple reusable function:

(ns foo.service.notification.core
(:require [foo.service.notification.email :as email]
[foo.service.notification.sms :as sms]
[foo.service.notification.websocket :as ws]))

(defn notify-recipients!
[recipients message]
(ws/notify-clients!
{:channels (ws/users->channels recipients)
:message message})
(email/notify-recipients!
{:recipients recipients
:message message})
(sms/notify-recipients!
{:recipients users-to-notify
:message message}))

;; And then wherever needed:

(defn submit-comment!
[comment]
(actually-persist-the-comment)
; and now notification side-effect:
(let [users-to-notify (a-function-to-get-subscribed-users)]
(notification.core/notify-recipients!
users-to-notify
{:category :new-comment
:payload
comment})))

But at Magnet we like to go one step farther and abstract everything that goes outside the main app behind boundaries:

https://medium.com/media/11523c3ba834d062e83d7469dd3a5a4e/href

With this setup, publishing to all agents is as simple as this:

(notification.core/publish!
config

{:message {:category :offer-accepted
:payload
{:buyer :user-999
:bid
1337}}})

There is one more question that remains unanswered in the gist above: how do you get recipients for a given message? Well, here’s how.

DSL for recipients handling

Let’s go back to our example where we want to send a notification about a new comment to everyone who might be interested in that event. Who is everyone then? It could be like this:

  • the author of the post
  • everyone that has ever commented that post

Great, let’s code that little function then:

(defn post-observers
[{:keys [post-id] :as comment}]
(let [post-author (post/get-author post-id)
post-commenters (post/get-commenters post-id)]
(conj post-commenters post-author)))

And let’s apply it:

(defn submit-comment!
[config comment]
(actually-persist-the-comment)
; and now notification side-effect:
(let [users-to-notify (post-observers comment)]
(notification.core/publish!
config
{:message {:category :new-comment
:payload
comment}})))

Do you see what might go wrong with this? There is a high possibility, that comment namespace will be required by post namespace. If we implement this function in comment namespace there we’ll run into a cyclic dependency.

In my opinion, a convenient and clean way to combat such cases is by using a multimethod. post-observers implementation naturally belongs to post namespace, so we should keep it there. We just need to define a multimethod in a safe place that will not require anything and thus everything could require it:

https://medium.com/media/6769072a5aede4eb4b821e03c8ebde75/href

And that’s it! Now you can easily compose different rulesets for various events where implementing new ones is just as easy as implementing new defmethod. Here’s an example unit test of composing different rules:

(testing "Register new rule"
(defmethod sut/handle-recipients-rule :the-chosen-one
[_ _]
[test-utils/user-1337-id])

(is (= #{user-1337}
(sut/handle-recipients
(test-config)
[[:the-chosen-one]])))

(is (= #{user-1 user-2 user-1337}
(sut/handle-recipients
(test-config)
[[:direct [test-utils/user-1-id
test-utils/user-2-id]]
[:the-chosen-one]]))))

Bonus: custom notification settings

None of us likes to get swamped in notifications. That’s why we get all those notification settings where we can choose which events we want to be notified about via emails and which via text message etc.

That’s why we have this reduce function in notification-recipients function. Let me extend it one last time:

(defn- filter-subscribers
[potential-recipients {:keys [agent category]}]
(filter (fn [user]
(get-in user
[:notification-settings agent category]
; Uncomment the `true` below if you want to treat unset settings as a consent to notify via this agent
; true
))
potential-recipients))

(defn notification-recipients
"Given configuration of notification agents and the notification itself,
returns a map where keys are the notification agents and values are sets
of users that will be notified using a given agent."
[{:keys [notification-agents] :as config} notification]
(let [{:keys [recipients-ruleset message]} notification
{:keys [category]} message
potential-recipients (recipients/handle-recipients config recipients-ruleset)]
(reduce (fn [acc [agent _]]
(let [recipients (set (filter-subscribers
potential-recipients
{:agent agent
:category category}))]
(if (seq recipients)
(assoc acc agent recipients)
acc)))
{}
notification-agents)))

Now, if your recipient’s data will contain a notification settings map (example will be shown in tests below) then it will filter recipients for each agent respecting the privacy settings:

https://medium.com/media/5942be04cbbc954376df76dd1469c028/href

As you can see, now it returns different sets of users for each agent so only them will get those notifications. Simple, isn’t it? :)

And that’s a wrap for this small serie. I hope it’s been insightful for you. If you have any questions about this notifications system or the environment around it don’t hesitate to drop a comment or email us at info@magnet.coop


Sending notifications in Clojure web app — part 2— multi-agent using Duct was originally published in magnet.coop on Medium, where people are continuing the conversation by highlighting and responding to this story.

Permalink

102: Transducers, servers, and comments

Thanks to everybody who has contacted me about jobs or sent through tips on people that are hiring. I’ve had a lot of great conversations with people, one thing that’s been neat to see is how widely Clojure is being used across so many different industries.

-main

Reactive Clojure: You don’t need a web framework, you need a web language

Dustin Getz wrote an article describing the Hyperfiddle stack. Being able to seamlessly colocate client and server code will be extremely powerful, I’m looking forward to Photon being open sourced.

notion.site


Michiel Borkent on the ClojureScript Podcast

Jacek Schae interviewed Michiel Borkent about his recent move to working full-time on open source development.

clojurescriptpodcast.com


Fast and Elegant Clojure

Ben Sless refactored a slow Clojure implementation to various versions which are still idiomatic without sacrificing performance, and then finally a loop/recur form for when you need maximum performance. There’s also some neat custom transducers.

github.io


Grokking Clojure transducers

If you need a refresher on transducers before reading Ben Sless’s versions, this is a good article from Eero Helenius to develop an intuition for them.

solita.fi


Stressed Servers

Ben Sless also did some benchmarking on Clojure web servers, JVM garbage collectors, and routing frameworks. It’s also a good overview also of how to *think* about scaling web servers, and building a mental model of how they fail. You can pair it with Zach Tellman’s talk Everything Must Flow from a few years ago.

github.io


Speed up your ClojureScript Webapp

And on the frontend, Lucio D’Alessandro shows how to benchmark ClojureScript webapps and a bunch of optimisations you can use at different levels of your application.

juxt.pro


Asami: Turn your JSON into a Graph in 2 Lines

Paula Gearon spoke at the recent Strange Loop conference about Asami, an open source Datomic-like graph database that she’s built.

youtube.com


Clojurists Together September 2021 Monthly Update

Read the latest updates from a bunch of Clojurists Together funded projects including clojure-lsp, polylith, Holy Lambda, and Typed Clojure.

clojuriststogether.org


An Architect’s View: deps.edn and monorepos VII (Polylith)

Sean Corfield continues his series on monorepos, polylith, and developer workflow using Portal for VSCode to navigate through data.

corfield.org

Libraries & Books.

coffi

coffi is a new library for doing FFI (Foreign Function Interface) calls from Clojure to native code. Very cool and looks like it should be easier than other existing methods.

github.com


Rich Comment Forms

Rich Comment Forms are a new tool from the Hyperfiddle team for writing tests inline with your Clojure/ClojureScript source. They’ve put a lot of work on the developer experience for the tool, it looks useful for pairing and REPL driven development.

github.com


Lacinia reaches 1.0

Congratulations to Walmart Lab’s Lacinia reaching the 1.0 milestone!

github.com


salutem

salutem is a new health check library for sync / async health checks. This has clearly had a lot of thought and documentation put into it. The async health-checking in particular looks very useful.

github.com


Clojure src, test, and meta

Daniel Gregoire makes the case for having a meta folder along with src and test, and introduces metazoa, a library for inspecting and querying metadata. There’s also a video which discusses the motivations and use of the library more.

danielgregoire.dev

Foundations.

Amazon Corretto, A Journey into Latency Reduction

The Amazon Corretto JDK team gave a talk about why they built Corretto and talked more about the Shenandoah GC.

youtube.com

Tools.

clojure-lsp releases

clojure-lsp has had a bunch of releases recently with improvements. I’m happy to see all the progress, it’s going to have a big impact on lots of Clojure users.

github.com


babashka/neil

Neil is a new CLI to add common aliases and features to your deps.edn-based projects.

github.com


A preview of Clerk

Martin Kavalar recently released a preview and rationale for Clerk, a tool for converting a Clojure namespace into a notebook style interface. Avoiding out-of-order execution in particular looks like a very useful feature.

twitter.com


clojuby

clojuby is an interesting experimental project letting you use JRuby to talk to Ruby functions in a Clojure syntax, for example creating a Sinatra app from Clojure.

gitlab.com


Hiding whitespace is now remembered for each pull request | GitHub Changelog

GitHub isn’t directly a Clojure tool, but I find myself often enabling “Hide white space” when reviewing Clojure PRs. You can now set this permanently, rather than having to enable it on each PR you viewed. One downside to watch out for is that you might miss unintentional white space diffs.

github.blog


Clojure REPLs Deep Dive - YouTube

Nikita Prokopov explored the existing Clojure REPLs and Sublime Plugins for Clojure, how they work, and their pros and cons.

youtube.com


Meta Reduce 2021.2: Autumn Begins

Bozhidar Batsov has an update on the projects he’s been working on lately including the CIDER family.

metaredux.com

Recent Developments.

[CLJ-2664] Provide fast idiomatic access to java.lang.Math

There’s work underway to create a new clojure.java.math namespace which is easier to access from Clojure than java.lang.Math but with the same level of performance. The patch is an interesting read to see heavy use of inlining.

atlassian.net


CLJS-3330: Flag for legacy loading of goog.object & goog.array

Previously you could assume goog.object and goog.array were always available for use in your ClojureScript files without requiring them. However, if I’m reading the patch and discussion correctly, the new approach will be to always require any goog.*namespace that you’re using.

No changes needed yet, but I’d start to look out for this. It sounds like clj-kondo will be able to lint for it which will make updating your code a lot easier.

github.com

Learning.

October 2020 Plans & Hopes for Clojure Data Science

Daniel Slutsky has this month’s scicloj post where people talk about what they’re working on in the Clojure data science community. I’ve been impressed by scicloj and their consistent efforts to improve the Clojure data science ecosystem. They’re also looking to run workshops for the upcoming re:Clojure conference.

clojureverse.org


Dmitri Sotnikov - Engineering for Scale in Clojure

Dmitri Sotnikov was recently on the RH podcast talking his work introducing Clojure at SIM University Health Network.

anchor.fm


Finding my inner Wes Anderson with Babashka

Tim Zöller uses Babashka and AppleScript to query the Apple Photos photo database.

javahippie.net

I’m Daniel Compton. I maintain public Maven repositories at Clojars, private ones at Deps, and help fund OSS Clojure projects (along with tons of generous members like LatacoraRoamWhimsicalStylitics PitchNubankCiscoAppsFlyerJUXTMetosinSolitaAdgojiNextjournalFlexianaToyokumoGriffinParkside, and Doctronic) at Clojurists Together. If you’ve enjoyed reading this, tell your friends to sign up at therepl.net, or post a link in your company chatroom. If you’ve seen (or published) a blog post, library, or anything else Clojure/JVM related please reply to this to let me know about it.

If you’d like to support the work that I’m doing, consider signing up for a trial of Deps, a private, hosted, Maven Repository service that I run.

Thanks!

Permalink

Sending notifications in Clojure web app — part 1 — Websockets

Sending notifications in Clojure web app — part 1 — Websockets

The story starts simple — a user leaves a comment under a post and you want send a websocket notification to every other user that might be interested in that update.

Disclaimer: a lot of this code is heavily inspired on the docs written by Dmitri Sotnikov.

First we need to make our BE able to keep a websockets registry and send transit messages to them:

https://medium.com/media/c6b194161070e974ae8cdb87d4d37414/href

Then we want clients to connect to that registry. Since a websocket handshake happens through an HTTP call, let’s define an API route:

https://medium.com/media/eb339d6a31a568533143739a15ef4443/href

Please note that we add `user-id` to each channel registry. I believe it’s important when we want to select only some channels to send a message to. Also note that one user can have many channels open (e.g. one because of a phone connected to the webapp and one for a laptop).

Once we have the api in place, let’s make clients capable of connecting to the back end:

https://medium.com/media/b9d87f08b1671342070b79470f313e79/href

Now all that’s left to do is actually connecting. You can decide when to run :make-websocket effect whenever it works best for your app. I think that a good place to do it is once your application acknowledges user identity:

(rf/reg-event-fx
::set-user-data
(fn [{:keys [db]} [_ user]]
{:db (assoc db :user user)
:make-websocket {:user user}}))

With that setup we are ready to send messages:

(notify-clients!
{:channels [x y z]
:message "Hello"})

How are you supposed to know what are the channels you ask? You don’t have to know them! Did you notice that in the first snippet there are some utility functions for translating users into channels? Let’s use them:

(notify-clients!
{:channels (users->channels [{:user-id 111
:name "Bob"}
{:user-id 222
:name "Jill"}])
:message {:topic "Hello"}})

And that’s all for this part. Tune in for the next post where I’ll cover how other notification agents (like email or text messages) could be plugged in without making the code complex.


Sending notifications in Clojure web app — part 1 — Websockets was originally published in magnet.coop on Medium, where people are continuing the conversation by highlighting and responding to this story.

Permalink

Clojure Deref (Oct 22, 2021)

Welcome to the Clojure Deref! This is a weekly link/news roundup for the Clojure ecosystem. (@ClojureDeref RSS)

Highlights

The Clojure community has a long and friendly relationship with our cousins in the Racket/Scheme community and we’ve often had speakers at Clojure conference from that world. I wanted to call your attention to the recent Strange Loop keynote from Will Byrd, "Strange Dreams of Stranger Loops" which takes his prior investigations into generating quines (programs that produce themselves) with miniKanren into further territories of twines (twin quines) and quine relays and many other interesting digressions.

Near the end of the talk, Will announced two "competitions", each for $1000, the QuiPS Prize for the best "small" strange loop along the order of quines and quine relays, self-reproducing behavior, and the Mant Prize - for the best "large" strange loop, and you’ll need to watch the video to get the full sense of the intent here. I’ve linked his pages for these prizes, which are as yet unpublished. Feel free to pester Will to fill those out more completely. :)

I’m highlighting these here both to give them some more circulation and because I think there are many smart and curious minds in the Clojure community that would enjoy working on things like this to spur the Quine Industrial Complex.

In the core

Lots of things in mid-flight right now, some getting near the "done" end. CLJ-2664 is a new Clojure namespace clojure.java.math wrapping java.lang.Math. There are a couple related pieces of work for adding numeric parsing functions and uuid constructors. All of those are coming out of a survey of which JDK static methods were in common use and trying to cover a bit more of that surface area in the Clojure API directly. All of that is 1.11 scope. Additionally, I think we’re near the end of the spec updates for trailing map support, which is being done in CLJ-2606 and that will end up in new versions of spec (both old and new). You can also find a variety of other tickets ready for Rich’s review in the Screened list (see workflow page for more details).

In addition to all that, we’ve been doing a lot of thinking and planning for future directions of the Clojure team, nothing I can share on that right now but it’s been fun to talk about ways we can grow.

Videos and podcasts

Libraries and Tools

New releases and tools this week:

  • coffi 0.2.259 - A Foreign Function Interface in Clojure for JDK 17

  • salutem - A health check library for sync / async health checks

  • clj-kondo 2021.10.19 - A linter for Clojure code that sparks joy

  • clj-media - View or create videos and gifs with clojure

  • heroicons-clojure - heroicons for Fulcro and Reagent

  • xtdb-inspector - Web UI for inspecting XTDB database

  • shh 2021.10.20 - A CLI password manager designed for efficiency

  • clojure-lsp 20211020-164947 - A Language Server for Clojure(script)

  • nbb v0.0.107 - Ad-hoc CLJS scripting on Node.js

  • clj-jq 1.1.3 - Clojure wrapper for jackson-jq

  • Calva 2.0.220 - Calva is an integrated REPL powered environment for enjoyable and productive Clojure and ClojureScript development in Visual Studio Code

  • pathom3 2021.07.10-alpha - A library for navigating data

  • luna 0.1.0-SNAPSHOT - A Domain Specific Language (DSL) that translates to regex.Pattern

  • aws-api - AWS, data driven

  • java.data 1.0.92 - Functions for recursively converting Java beans to Clojure and vice versa

  • re-graph 0.1.16 - A graphql client for clojurescript and clojure

  • glimt 0.2.1 - HTTP FSM for re-frame

Permalink

Senior Clojure Engineer at RentPath

Senior Clojure Engineer at RentPath


Looking to join a company in the midst of a digital transformation where the consumer is king and talent, technology and data are our greatest resources??  Keep reading to see if this opportunity is of interest to you!!

RentPath is looking for an experienced Senior Software Engineer (Clojure) to join our team.  Engage in an encouraging  company culture and become a part of an Interesting and charismatic group of professionals who love what they do!   We offer, even encourage the opportunity to contribute to open source!

Reporting to the Services Engineering Manager, and in primary service to RentPath’s search engine optimization (SEO), internal data, and analytics teams, the Clojure Software Engineer is responsible for the delivery of the best in class technical solutions to meet the needs of our customers.

A Day In the Life:

  • Clojure Software Engineer holds a reputation for being coachable, receptive to mentorship, and is showing continuous improvement in technical ability.
  • Highly available, rapid and highly responsive data/business logic is consistently available to our stakeholders.
  • RentPath’s products are kept current and modernized; growing and adapting to new trends and technologies to ensure we are on the forefront of curve.
  • Supervisor and appropriate stakeholders are kept consistently informed regarding architectural diagrams, and flaws in their creation are minimal and caught/addressed swiftly; end-user is enabled to utilize codes correctly and as intended.
  • Workflow transparency exists to such a degree that the right work is successfully prioritized, commercialization is well coordinated and execution delivers high quality output the first time, without requiring multiple iterations and rework.

 **What we need from YOU**

  • Professional software development experience utilizing Clojure in production, and with a heavy focus on functional programming
  • Experience working with stream processing platforms and message brokers, such as Apache Kafka and RabbitMQ 
  • Demonstrated usage and understanding of cloud-based technologies
  • Familiarity with containerization and configuration tools such as Docker, Kubernetes, and Terraform

A Few of our Perks and Benefits

  • Immediate 401k match
  • Minimal Travel
  • Generous PTO and Paid Holidays
  • Paid volunteering days
  • Medical, dental, vision plans
  • Paid maternity and parental leave
  • Flexible spending accounts
  • Employee discounts
  • Covered parking or Marta stipend (for Atlanta HQ - based employees)
  • On-site gym (for Atlanta HQ - based employees)

Still interested? Upload your resume and a Recruiter will check it out as soon as possible.
 

All applications for positions with RentPath are subject to our Privacy Policy.

Permalink

5 best programming languages for AI advancement

Artificial intelligence (man-made brainpower) opens up a universe of opportunities for application designers. By exploiting AI or profound learning, you could create far superior client profiles, personalization, and suggestions, or consolidate more brilliant hunts, a voice interface, or wise help, or work on your application in quite a few alternate ways. You could even form applications that see, hear, and respond to circumstances you wont ever expect.

Which programming language would it be advisable for you to figure out how to plumb the profundities of AI? Youll need a language with numerous great AI and profound learning libraries, obviously. It ought to likewise include great runtime execution, great instrument support, an enormous local area of developers, and a sound biological system of supporting bundles. That is an extensive rundown of necessities, yet there are still a lot of good alternatives.

Here are my picks for the six best programming dialects for AI advancement, alongside two noteworthy notices. A portion of these dialects are on the ascent, while others are slipping. Still others you possibly need to think about in case youre keen on recorded profound learning designs and applications. How about we perceive how they all stack up.

Python

At number one, its still Python. How is it possible that it would be something else, truly? While there are incensing things about Python, in case youre accomplishing AI work, you more likely than not will utilize Python sooner or later. Whats more, a portion of the unpleasant spots have smoothed a bit.

As we head into 2020, the issue of Python 2.x versus Python 3.x is becoming unsettled as pretty much every significant library upholds Python 3.x and is dropping Python 2.x help in a hurry. All in all, you can at last exploit all the new language includes decisively.

And keeping in mind that Pythons bundling bad dreams where each unique arrangement is broken in a somewhat unique manner are as yet present, you can utilize Anaconda about 95% of the time and not stress over things to an extreme. In any case, it would be pleasant if the Python world could fix this long-standing issue for the last time.

All things considered, the math and details libraries accessible in Python are practically unmatched in different dialects. NumPy has become so universal it is just about a standard API for tensor tasks, and Pandas brings Rs incredible and adaptable dataframes to Python. For regular language handling (NLP), you have the revered NLTK and the blazingly-quick SpaCy. For AI, there is the fight with Scikit-learn. Whats more, with regards to profound learning, the entirety of the current libraries (TensorFlow, PyTorch, Chainer, Apache MXNet, Theano, and so forth) are adequately Python-first undertakings.

Assuming youre perusing state of the art profound learning research on arXiv, you will discover most of the studies that offer source code do as such in Python. Then, at that point there are different pieces of the Python environment. While IPython has become Jupyter Notebook, and less Python-driven, you will in any case track down that most Jupyter Notebook clients, and the vast majority of the scratch pad shared on the web, use Python. With respect to sending models, the approach of microservice designs and advances, for example, Seldon Core imply that its exceptionally simple to convey Python models underway nowadays.

Theres no way to avoid it. Python is the language at the cutting edge of AI research, the one youll track down the most AI and profound learning structures for, and the one that nearly everyone in the AI world talks about. Therefore, Python is first among AI programming dialects, in spite of the way that your creator reveals the whitespace issues essentially one time each day.

C++

C++ is probably not going to be your best option when fostering an AI application, yet when you need to wring each and every digit of execution from the framework a situation that turns out to be more normal as profound learning goes to the edge and you need to run your models on asset obliged frameworks its an ideal opportunity to venture once again into the unnerving universe of pointers again.

Fortunately, current C++ can be lovely to compose (genuine!). You have a selection of approaches. You can either make a plunge at the lower part of the stack, utilizing libraries like Nvidias CUDA to compose your own code that runs straightforwardly on your GPU, or you can utilize TensorFlow or PyTorch to get admittance to adaptable significant level APIs. Both PyTorch and TensorFlow permit you to stack models created in Python (or PyTorchs TorchScript subset of Python) and run them straight in a C++ runtime, drawing you nearer to the uncovered metal for creation while saving adaptability being developed.

So, C++ turns into a basic piece of the tool stash as AI applications multiply across all gadgets from the littlest implanted framework to colossal groups. Artificial intelligence at the edge implies its difficult enough to be precise any longer; you should be acceptable and quick.

Java and other JVM dialects

The JVM group of dialects (Java, Scala, Kotlin, Clojure, and so on) keeps on being an incredible decision for AI application advancement. You have an abundance of libraries accessible for all pieces of the pipeline, regardless of whether its regular language handling (CoreNLP), tensor activities (ND4J), or a full GPU-sped up profound learning stack (DL4J). Besides you get simple admittance to enormous information stages like Apache Spark and Apache Hadoop.

Java is the most widely used language of most ventures, and with the new language builds accessible in Java 8 and later forms, composing Java code isnt the scornful experience a considerable lot of us recollect. Composing an AI application in Java might feel a touch exhausting, however it can take care of business and you can utilize all your current Java framework for advancement, organization, and observation.

JavaScript

Youre probably not going to learn JavaScript exclusively for composing AI applications, yet Googles TensorFlow.js proceeds to improve and offer a fascinating method of conveying your Keras and TensorFlow models to your program or through Node.js utilizing WebGL for GPU-sped up computations.

Notwithstanding, one thing we havent actually seen since the dispatch of TensorFlow.js is a tremendous deluge of JavaScript engineers flooding into the AI space. I feel that may be because of the encompassing JavaScript biological system not having the profundity of accessible libraries in contrast with dialects like Python.

Further, on the worker side, theres not actually quite a bit of a benefit to conveying models with Node.js rather than one of the Python alternatives, so we might see JavaScript-based AI applications remain principally program situated soon. However, that actually sets out a lot of intriguing open doors for entertainment only like the Emoji Scavenger Hunt.

Swift

Swift For TensorFlow. A completely composed, sans cruft restricting of the best in class highlights of TensorFlow, and dim wizardry that permits you to import Python libraries as though you were utilizing Python in any case.

The Fastai group is chipping away at a Swift form of their well known library, and were guaranteed bunches of additional advancements in producing and running models with moving a great deal of tensor smarts into the LLVM compiler. Is it creation prepared at the present time? Not actually, however it might for sure direct the way toward the up and coming age of profound learning improvement, so you ought to examine whats new with Swift.

R language

R comes in at the lower part of our rundown, and its declining. R is the language that information researchers love. Nonetheless, different software engineers regularly discover R somewhat aggravating, because of its dataframe-driven methodology. On the off chance that you have a devoted gathering of R engineers, it can bode well to utilize the incorporations with TensorFlow, Keras, or H2O for research, prototyping, and experimentation, however I wonder whether or not to suggest R for creation use or for greenfield improvement, because of execution and functional concerns. While you can compose performant R code that can be conveyed on creation workers, it will more likely than not be simpler to take that R model and recode it in Java or Python.

Originally posted at: https://en.everypage.site/technology/5-best-programming-languages-for-ai-advancement/

Permalink

Middle Data Engineer

We are looking for a Middle Data Engineer to join our team. The product is an AWS hosted multi-module payment and analytical platform for the healthcare services, written in Clojure/Golang/Python language stack. The product encompasses a few applications for customer journeys (web, mobile), data science/data analytics platform, multiple integrations with federal and governmental resources, and complex micro-service architecture. The product’s domain is Healthcare/Fintech, hence all the compliances and accent on security and high-performance. 

Responsibilities: 

  • API, lambda development  
  • Working with big data, data lakes, ML  
  • Participate in solution design and deliver high-quality code  
  • Regularly communicate with the team members at the client’s side, participate in status meetings, design sessions, and brainstorming  
  • Provide estimation and reporting of assigned tasks 

Requirements:    

  • 3+ years of professional experience 
  • Desired development experience building and maintaining ETL pipelines  
  • Desired experience working with database technologies and data development such as Python 
  • Experience with Postgres, DynamoDB, or Mongo experience  
  • English-upper-intermediate level  

Would be a plus:

  • ML experience 
  • AWS lambdas experience 
  • Cassandra experience 
  • Micro-service architecture experience 

We offer friendly working conditions with competitive compensation and benefits including:

  • Comfortable working environment
  • Friendly team and management
  • Competitive salary 
  • Free English classes 
  • Regular performance-based compensation review 
  • Flexible working hours 
  • 100% paid vacation, 4 weeks per year 
  • 100% paid sick-leaves
  • Corporate and team building events 

Apply Now 

The post Middle Data Engineer first appeared on Agiliway.

Permalink

Книга в ДМК-Пресс

Моя книга про Кложу выходит в издательстве ДМК-Пресс. Ожидаемая дата — середина ноября. Твердый переплет, формат B5 (165x235 мм).

Внимание: на время предзаказа цена 799 рублей (вместо 999 рублей). С промокодом

Grishaev_Clojure20

она станет еще меньше. Если вы бываете в книжных магазинах, то знаете, что компьютерных книг дешевле тысячи почти не бывает. А тут такое предложение! Надо брать.

Книга почти ничем не отличается от второго издания, что вышло в Ридеро. Исправил опечатки и неточности в коде, на которые указали читатели. Особая благодарность Алексею Иванцову, который не просто нашел ошибки, но и оформил их в виде PR к репозиторию книги. Мне оставалось только нажать кнопку, чтобы их принять.

Кроме того, собрал и залил исправленные PDF на Gumroad. Если вы уже покупали там, зайдите и скачайте новые файлы. Если нет — самое время купить, цена всего пять долларов.

Для меня выход книги в ДМК-Пресс — это событие. Во-первых, издание вышло на меня само. Когда заинтересован не ты, а в тебе, это меняет дело: значит, книга действительно кому-то нужна.

Во-вторых, приятно, что ДМК-Пресс — издание с долгой историей и богатым списком выпущенных книг. Именно в ДМК девять лет назад вышло “Программирование на Clojure” — тот самый альманах на 800 страниц. Он кстати, почти не устарел, советую прочитать. Вдвойне приятно, что моя книга займет место рядом с ним.

Что это значит для читателя? Главное — книгу в твердой обложке теперь можно купить не только у меня, а на сайте ДМК или в магазине. Это главная проблема Ридеро: у них нет массовой печати в твердой обложке, только разовый (и потому дорогой) тираж. Так что теперь не обязательно слать мне Яндекс.Деньги: заказывайте на сайте ДМК. Однако я всегда рад подписать книжку и отправить в любую страну.

Permalink

Finding my inner Wes Anderson with Babashka

I have found a new toy, and its name is babashka. Babashka is a “Native, fast starting Clojure interpreter for scripting” by Michiel Borkent. This scripting environment packages a lot of Clojure goodness, is precompiled with GraalVM to a native binary, resulting in incredibly fast startup time. It enables developers to execute Clojure code immediately. So of course I am currently not only in the process of migrating some parts of our build pipeline to babashka, I am also writing some nonsensical scripts for fun – be it a script to publish my current track playing in Apple Music to Slack, a small script to set my Desktop wallpaper to NASAs picture of the day, or other small stuff that makes me happy. This blog post is about me finding my prettiest pictures with babashka.

The goal

Some time ago I found an article about the database of Apples “Photos” App, Using SQL to find my best photo of a pelican according to Apple Photos. The app is storing all of its data in an SQLite database, to which we can connect to. Inside, there are not only information and metadata about the photos themselves, it also contains the results of Apples machine learning algorithms which continuously analyze the photos. It is located at ~/Pictures/Photos Library.photoslibrary/database/Photos.sqlite This sounded like a great opportunity for me to play with babashka a little more. Here are some of the columns from the table ZCOMPUTEDASSETATTRIBUTES:

  • ZBEHAVIORALSCORE
  • ZFAILURESCORE
  • ZHARMONIOUSCOLORSCORE
  • ZIMMERSIVENESSSCORE
  • ZINTERACTIONSCORE
  • ZINTERESTINGSUBJECTSCORE
  • ZPLEASANTSYMMETRYSCORE

Pleaseant symmetry sounds nice to me. So, let’s go looking for my most Anderson-esque picture with Clojure!

The SQL-Part

Each column has a reference to the ZASSET table, which stores all of the images for the Photos app. The higher the value in a column, the better the picture matches the column description. First, we needed an SQL script to give us the UUID for the best photo in a category:

select asset.ZUUID
from ZCOMPUTEDASSETATTRIBUTES attr
left join ZASSET asset
on attr.ZASSET = asset.Z_PK
order by ZINTERESTINGSUBJECTSCORE desc
limit 1

The UUID belongs to this picture, which means my Macbook thinks the most interesting picture I have ever taken is a flock of seagulls. Good start!

Seagulls in flight

The Apple Script part

Unfortunately, not all of my photos are on my Mac, some are in the Apple Cloud. Although the original photos stored on my drive are renamed with their UUID and stored neatly in a folder, most of the photos were not there, and I could not simply access the filesystem in order to extract them. Luckily, it turns out you can “remote control” Apple Photos with AppleScript! After some trial and error, I came up with the following script to extract a photo by its UUID:

tell application "Photos"
    set theitem to media item id "E8555C22-BE76-4153-AE11-C535C070C952"
    set thePath to POSIX path of "/tmp"
    export {theitem} to thePath
end tell

The the library docs for AppleScript are okay, but I cannot say that I really like the language. It’s no fun, it’s frustrating to get the syntax right, and just somehow counterintuitive. AppleScript can be run from the commandline with the command osascript -e <THESCRIPT>, so calling this script that way will export the photo with the given UUID to /tmp with its original format: JPEG, HEIC or RAW.

The babashka part

Before we dive into the code: Developing simple scripts in a REPL is a lot of fun. Instead of “change, save, run”, we an build and test the single functions in our script in isolation and stick them together, once everything works. As I use Emacs and Cider, I started a REPL with the command bb --nrepl-server in the shell were my script was present, and then connected Cider to that nrepl server.

This style of development matches my brain really well, as I am easily distracted and sometimes can have a hard time concentrating. Short feedbackcycles for the win! So, Having solved the prerequirements, let’s start hacking!

SQL

First we want to connect to the database. Luckily, there is a Pod for that. Babashka pods are small applications which can be used as libraries in babashka. The don’t have to be written in Clojure, though: The SQLite pod is written in Go, for example. Pods need to be referenced in the script, and can then be required:

(require '[babashka.pods :as pods])
(pods/load-pod 'org.babashka/go-sqlite3 "0.0.1")
(require '[pod.babashka.go-sqlite3 :as sqlite])

(def sqlite-path "/users/zoeller/Pictures/Photos Library.photoslibrary/database/Photos.sqlite")

(def query ["select asset.ZUUID
from ZCOMPUTEDASSETATTRIBUTES attr
left join ZASSET asset
on attr.ZASSET = asset.Z_PK
order by ZINTERESTINGSUBJECTSCORE desc
limit 1"])

(sqlite/query sqlite-path query)
;;=>  [{:ZUUID "E8555C22-BE76-4153-AE11-C535C070C952"}]

That’s all about SQL, that’s the whole code. No other setup needed. If executed in a script from the command line (bb ./my-file.clj), this returns almost immediately. No JVM startup, no Clojure init time.

AppleScript

Of course, I could simply use Selmer (which is supported by babashka) as a templating engine, store the AppleScript code as a String and replace the UUID I want. But where is the fun in that? I wondered, if I could write some kind of primitive DSL for that, so I did not need another dependency. And I could – in a way:

(def applescript '[tell application :photos \newline
                     set theitem to media item id :uuid \newline
                     set thePath to POSIX path of :path \newline
                     export \{theitem\} to thePath \newline
                   end tell])

(defn as-applescript [script params]
  (reduce #(str %1 " " %2)
          (for [token script]
            (cond
              (keyword? token) (str "\"" (get params token) "\"")
              (char? token) token
              :else (name token)))))

This is rather dirty, but it works for my limited usecase: The escaped vector contains symbols which resemble the symbols from applescript. Unfortunately, AppleScript uses syntactic whitespace, the line breaks are mandatory. Also, it uses curly braces in its syntax, and the Clojure Reader is not a fan of curly braces with just one element in them, even in escaped vectors. So I added the newline and the braces as characters in the vector. The keywords work as parameters, and can be passed to the script creator as a map:

(as-applescript applescript
                {:photos "Photos"
                 :uuid "E8555C22-BE76-4153-AE11-C535C070C952"
                 :path "/tmp"})
                       
;;=> "tell application \"Photos\" \n set theitem to media item id \"E8555C22-BE76-4153-AE11-C535C070C952\" \n set thePath to POSIX path of \"/tmp\" \n export { theitem } to thePath \n end tell"

For now I am happy with that. While doing this, I also remembered that I always wanted to look up how to properly design small domain specific languages, so I’ll add that to my “To Do” list, too.

Tying it all together

Here is the final result in 42 lines of code:

#!/usr/bin/env bb

(require '[babashka.pods :as pods])
(pods/load-pod 'org.babashka/go-sqlite3 "0.0.1")
(require '[pod.babashka.go-sqlite3 :as sqlite])
(require '[clojure.java.shell :refer [sh]])

(defn query-db []
  (let [sqlite-path "/users/zoeller/Pictures/Photos Library.photoslibrary/database/Photos.sqlite"
        query "select asset.ZUUID
from ZCOMPUTEDASSETATTRIBUTES attr
left join ZASSET asset
on attr.ZASSET = asset.Z_PK
order by ZPLEASANTSYMMETRYSCORE desc
limit 1"]
    (sqlite/query sqlite-path [query])))

(def applescript '[tell application :photos \newline
                     set theitem to media item id :uuid \newline
                     set thePath to POSIX path of :path \newline
                     export \{theitem\} to thePath \newline
                   end tell])

(defn as-applescript [script params]
  (reduce #(str %1 " " %2)
          (for [token script]
            (cond
              (keyword? token) (str "\"" (get params token) "\"")
              (char? token) token
              :else (name token)))))

(defn export-photo [uuid path]
  (sh "osascript" "-e"
      (as-applescript applescript
                      {:photos "Photos"
                       :uuid uuid
                       :path path})))

(for [{:keys [ZUUID]} (query-db)]
  (if (= 0 (:exit (export-photo ZUUID "/Users/zoeller/Desktop/test")))
    (str "Exported " ZUUID)
    (str "Did not export " ZUUID)))

The shebang in the beginning tells my OS that this is a babashka script. I can now just execute it with ./epp.clj on the command line (Export Pretty Pictures). It works exactly like I imagined. So… what is my picture with the most pleasant symmetry? It’s this one:

Zinnengasse in Zürich

Well… at least the journey was fun.

What else to do with this

Aside from the (underwhelming) categorizations like “Most pleasant symmetry”, there is a lot of interesting stuff in that database. There is a whole table dedicated to facial recognition, storing if people smiled, how much they smiled, if their left or right eye was closed and what type of facial hair they had. So we could build queries to find the most beautifully lit foto of a smiling man with a beard having one eye closed. I will certainly have some more fun with this, but I won’t show any pictures of other people here, so I will keep those to me ;)

Permalink

Copyright © 2009, Planet Clojure. No rights reserved.
Planet Clojure is maintained by Baishamapayan Ghose.
Clojure and the Clojure logo are Copyright © 2008-2009, Rich Hickey.
Theme by Brajeshwar.