Clojure Gazette 1.98

Clojure Gazette 1.98

Dinner, Opportunity, and Live Reloading

Clojure Gazette

Issue 1.98 October 19, 2014


Editorial

Hi Clojurists,

Clojure/conj is coming up. It's a month away. I've been lucky enough to find a way to go, including arranging with my wife and daughter, who are very supportive. But my luck is even greater, because by writing the Pre-Conj Prep emails, I've been able to digitally meet many of the speakers.

It has also forced me to get organized about the conference. Instead of just jumping on the plane and attending, I really want to make the most of my time. I'm working on a schedule, but I should tell you what I know now in case you are interested in planning around it.

Saturday, November 22 (the day the conference ends), I am organizing a dinner at a restaurant within walking distance of the hotel and venue. You should come! I would also like to use the afternoon to go to a Smithsonian museum, probably Air & Space to see the Wright Brothers exhibit. I'm sure I'll have the plans firm next week, so I'll let you know.

Rock on!
Eric Normand <eric@lispcast.com> @ericnormand

PS Learn more about the Clojure Gazette and subscribe. Learn about advertising in the Gazette.

Sponsor: Clojure/conj

 
Have you ever wanted to be in a room of 500 Clojurists, all pondering parens, talking shop, and learning what's happening with Clojure? Clojure/conj is coming up and it's not too late to buy tickets. Come share the single-track experience. Check out previous conference videos.
 
Regular priced tickets are sold out, but Cognitect has offered some regular priced tickets just for Gazette readers. They're first-come-first-serve. Just use coupon code ClojureGazette when you buy them.

Clojure/conj Opportunity Grants


Strange Loop doubled the percentage of women who attended this year over last year. One of the big drivers was their Diversity Scholarship program. Clojure/conj is doing something very similar. They want to increase the number of people from under-represented groups, so they're issuing grants. But the number of applicants was higher than expected. It must be working! They need our help to fly out as many applicants as possible to the conference.

This is momentous for so many reasons. There is something stirring in the air. People are becoming more and more aware of the lack of diversity, the harrassment, and the systematic oppression. And people are taking action. I can't claim that I'm doing that much myself. In fact, I'm probably just another part of the problem. But there's a wave cresting, and if you ever considered giving some money to this cause, now is the time to see it have the biggest impact your dollars ever might.

Frankly, women, transexuals, homosexuals, blacks, and countless others are heroes for facing the abuse they receive for doing nothing more than exercising their freedoms. If a little money can help bring an equal footing, even at a fringe language conference, then count me in. They are accepting individual donations, or if you have the possibility to persuade your company, there are larger sponsorships available.

FP Days


If you can't make it to Clojure/conj this year, but you can make it to London, FP Days is for you. It's happening at the same time. It's capped at 150 attendees to ensure a personal feel. Get your tickets because the price keeps going up.

Arcadia 0.1a Launched


The Clojure in Unity 3D project has been named Arcadia. It's now available. Watch the Strange Loop talk about it to see how it works.

Rich Hickey Q&A


An older interview conducted by Michael Fogus asking Rich Hickey questions about Clojure. Great answers to some great questions.

My Tribute to Steve Ballmer


Not really about Steve Ballmer, this article goes through some image processing steps to convert images to ascii renditions, all in Clojure.

Introducing Elm Reactor


Elm's Time Traveling Debugger turned some heads when it first came out. Well, they've been working on it ever since and now it has a really nice feature set. They've worked out the UI for it and, well, there's never been a better time to get into Elm.

Chestnut


A Leiningen template for getting live reloading of ClojureScript code set up from the get go. Start with this template and you can start coding ClojureScript right away.

Why We're Learning Clojure

A nice take on why a company that is heavily invested in Rails might use Clojure for some projects.

Zero downtime Clojure Deployments


uSwitch uses a micro-service architecture, so there are lots of small servers running all over the place. Combine this with fast, iterative development and the JVM's long startup time means that any time a new service is deployed, there is a gap between the last request handled by the old service and the first request handled by the new service. This article talks about a way to switch over without losing any availability.

Chatting cats use DataScript for fun


Datascript is a simple implementation of the Datomic data model in ClojureScript. It lets you store data in a flat log and query it to extract out a tree of your choosing. This long article goes through the creation of a chat application using Datascript, with some good lessons learned at the end.
Copyright © 2014 LispCast, All rights reserved.


unsubscribe from this list    change email address    advertising information

Permalink

Map transformation in clojure 1/2

Here it’s another chapter of the series about clojure map.

For those who missed the previous chapter:

Let’s keep going in our journey about clojure map.

In the next chapters I am going to talk about how to modify clojure maps.

As you may know in Clojure we usually don’t “modify” anything but we rather return a new value.

Let’s explore what function clojure.core gives us to modify maps.

The easiest function to start with is definitely

assoc

Given a map, a key and a value assoc(ciate) add the key and the respective value to the map.

user> (assoc {:a 1 :b 2} :c 3)
{:c 3, :b 2, :a 1}

As you may guess assoc can take more than just a couple of :keys value.

user> (assoc {:a 1 :b 2} :c 3 :d 4)
{:d 4, :c 3, :b 2, :a 1}
user> (assoc {:a 1 :b 2} :c 3 :d 4 :e 5)
{:e 5, :d 4, :c 3, :b 2, :a 1}

It is important is that you don’t forget to provide an even number of argument after the map.

user> (assoc {:a 1 :b 2} :c 3 :d)
IllegalArgumentException assoc expects even number of arguments after map/vector, found odd number  clojure.core/assoc (core.clj:192)

You may also wonder what happens if you try to assoc to an already existing key.

user> (assoc {:a 1 :b 2} :a "new-value")
{:b 2, :a "new-value"}

As we can see assoc will reset (update) the old value to the new one.

assoc is a perfectly legitimate way to update old values in a clojure map.

If assoc will make our maps bigger

dissoc

will make them smaller.

As you may have already guessed dissoc(ciate) will remove a map entry from our maps.

It works exactly like assoc.

user> (dissoc {:a 1 :b 2 :c 3} :a)
{:c 3, :b 2}

If you pass more arguments it will remove more map entries

user> (dissoc {:a 1 :b 2 :c 3} :a :b)
{:c 3}

Also note that dissoc a not existing key is a valid behavior, dissoc will eliminate the map entry, which does not exist and will return the (not) updated map.

user> (dissoc {:a 1 :b 2 :c 3} :d :e :f)
{:c 3, :b 2, :a 1}

merge

Another very useful function is merge which, as you may have guessed, merge two or more different maps into one.

user> (merge {:a 1 :c 3} {:b 2 :d 4})
{:d 4, :b 2, :c 3, :a 1}
user> (merge {:a 1 :c 3} {:b 2 :d 4} {:foo "foo" :bar :bar})
{:foo "foo", :bar :bar, :d 4, :b 2, :c 3, :a 1}

merge is very simple to reason about, but there is a little detail to keep in mind.

What happens if there is a key collision (if we try to merge two maps with some keys in common) ?

Fortunately this case is documented and made clear in the spec.

In case of key collision the resulting element will be the later to appear.

user> (merge {:a 1} {:a 2})
{:a 2}
user> (merge {:a 1 :b 1} {:a 2 :c 2} {:c 3 :b 3})
{:c 3, :b 3, :a 2}

end

In the next chapter we are going to explore some function a little more complex, function such as update-in and merge-with.

As always stay tuned and for any question please don’t hesitate to write me or to comment bellow.

Permalink

The Trifecta of ClojureScript, Om and core.async

This post is an adaptation of my lighting talk given at Austin JS and Austin Clojure Meetup.

It's too easy to build a front-end house of straw. Somehow the JavaScript and DOM manipulations we write turn into a fragile state machine glued together with cryptic callbacks. We start with good intentions and write modular, understandable code with clean lines of separation between our views and models. But as our app becomes more complex, and as more people touch the code, the lines often begin to blur.

Our application state becomes difficult to reason about; they are barraged by asynchronous modifications, passed to and fro between different models, and are irresponsibly stored in our views as CSS classes. Our DOM crumbles, dependent on programmer incantation to wrangle it to the shape we want. And our code grows uncontrollably complex like wildfire, oblivious to the scorched state machine it leaves in its wake.

But who is to blame? Naturally, ourselves. Laziness, ignorance, hurriedness, insipidness. With enough dedication, skill and discipline we can learn to build graceful systems. But I don’t think the fault is completely ours. Given the complexity of asynchronous user interfaces and the need to orchestrate state between many components, it may be that tools we have are unfit for the job. Perhaps we need better tools.

Is there something better? Can we take back control of our state, erase the fragility of DOM manipulation and restrain ad-hoc, asynchronous messages?

I think we can, through these means:

  • Prevent wanton state changes.
  • Give up control of DOM manipulation.
  • Remove state management in callbacks.

With ClojureScript, Om and core.async, we can do all three.

I am not prophetic enough to declare that what I’m presenting here is the solution; it is never that easy. But I do think that the ideas brought by Clojure, Om (via React) and core.async are game changers. Their ideas are so good that it would be a shame for anyone to miss out on it.

Taking back control of your state

If you're not familiar with Clojure syntax, you may want to read this quick primer.

Clojure (and ClojureScript) data structures are immutable. We prefer, for example, that the number 0 always stay the number 0. This is because 0 is considered a value, and values cannot change. Clojure extends this idea to data structures like maps. You cannot, for example, modify a Clojure map by adding a new element to it; you must make a “copy”.

To understand why this is important, let's see how mutable JavaScript data structures behave. There is no good sense of object equality:

o1 = {foo: 0};
o2 = {foo: 0};
o1 == o2;
// false

And if you pass your object to some function, you have no idea what that other function may potentially do to your object.

o = {foo: 0};
g(o);  // Your co-worker wrote this

// What is o?

You can try cloning your object before passing it to g, but now you have to decide if you should do a shallow clone or a deep clone. And just how deep do we need to go? One level? Two? All the way until you hit primitives?

Clojure's immutable data structures, on the other hand, gives us meaningful equality:

(= {:foo 0 :bar {:car "far"}}
   {:foo 0 :bar {:car "far"}})
;; ⇒ true

;; Corresponding JavaScript object:
;; {foo: 0, bar: {car: "far"}}

And there is no need to clone:

(def o {:foo 0})
(g o) ; Corresponding JavaScript: g(o)

;; o is still o
(= o {:foo 0})
;; ⇒ true

Immutability seems restrictive at first, but just as an artist thrives off the constraints of a canvas, you'll find that immutability frees you.

Immutability frees us to treat objects like values, because they are. We all prefer that 0 == 0 and "far" == "far". So why do we put up with {foo: 0} != {foo: 0}?

Immutability also encourages modularity. When you can't modify your data and override your local variables, you unwittingly focus less on state manipulation and more on writing smaller, more manageable functions.

And lastly, immutability prevents you from building a spaghetti state monster. The Clojure map {:name "Ayo", :email "ayo@example.com"} is not a stateful object that anyone can add to, remove from or modify. That map is just a value, and the value is {:name "Ayo", :email "ayo@example.com"}.

Mutable objects, on the other hand, allow undiscerning programmers to modify state at inopportune times. Consider, for example, the User model below:

var User = function (name, email) {
  this.name = name;
  this.email = email;

  this.isValid = function () {
    if (this.name.length > 100 ||
        this.email.indexOf("@") === -1) {
      return false;
    }
    return true;
  };

  this.resetEmail = function () {
    this.email = "";
  };

  this.magic = function () {
    // lots of code here…
    this.resetEmail();
    // lots of code here…
  };

  this.save = function () { /* … */ };

  return this;
};

And now let's use it:

var u = User("Ayo", "ayo@example.com")
if (u.isValid()) {
  // …
  if (somePredicate) { u.magic(); }
  // …
  u.save(); // Oops, saved invalid email.
}

Obviously this is a contrived example, but in a large code base with many collaborators, mistakes like this are common.

And yes, we can do a number of things to mitigate disasters like the one above. We can emphasize good coding styles. We can enforce code reads that will hopefully catch such errors. These things are possible, but hard to do. For example, what if the commit was adding the single expression u.magic()? Is the code reader expected to know what magic() does through and through? But with immutable data structures, the data structures and compiler enforces all of this for you, for free.

The conclusion is simple:

  1. Immutability prevents you from doing stupid things.
  2. Immutability frees you to do the correct thing.

Om and React

Given a page number of a book, you know that the contents will be the same every single time. Books are immutable. If you take the page number as your input, and your fingers flipping to the page as your function, you get:

$$\mathit{fingers}(\mathit{page}) \Rightarrow \mathit{contents}$$

Can we do the same with our programs? Can we pass in a state and get the UI at that given state?

With Om, you can.

Om is a ClojureScript library for React. While there are a lot of good articles and videos that explain Om and React, I like to think of an Om application as a function $f$ that takes your global application state $s$ and returns a $DOM$.

$$f(s) \Rightarrow DOM$$

Your application state $s$ is one giant, immutable map that contains all the necessary data for your UI to render. But naturally our UI needs to transition state at some point (e.g. when a user clicks something, or when an AJAX request comes back). To do this, we enclose our immutable map in a Clojure atom, like this:

(def my-state (atom {:my-data {} :users [] :foo 0}))

An atom is like a box. You can replace the thing inside the box, but you can't ever modify the content itself (because it's immutable). To get the value inside of the box, you must deference it using @. To “modify” an atom, use swap! or reset!. Like this:

@my-state
;; ⇒ {:my-data {} :users [] :foo 0}

(reset! my-state {:bar 1})
@my-state
;; ⇒ {:bar 1}

(swap! my-state merge {:far 2})
@my-state
;; ⇒ {:bar 1 :far 2}

Virtual DOM

In Om (and React), instead of writing raw HTML, you build virtual DOM elements. For example:

;; Renders "Hello `name`", where `name` is retrieved
;; from the global state.
(dom/div
  #js {:className "greeting"}
  (str "Hello" (:name my-state)))

In React JSX, this may look like this:

<div className="greeting">Hello {this.props.name}</div>;

// Compiled JS:
// React.DOM.div({className: "greeting"}, "Hello ", this.props.name);

The virtual DOM is just a bunch of JavaScript objects—objects controlled by the React framework. At every predefined tick, React will take your virtual DOM, check which elements have changed, and render only those elements. React's Reconciliation page describes in detail the optimizations made under the hood to make these renders fast. In short, React tries its best to only render changed components.

But Om has one distinct advantage against React: ClojureScript data structures are immutable. The rocket fuel, as David Nolen, creator of Om, calls it.

Let's see how this rocket fuel works. Say you have a custom component that renders a list of users and their names. Our state may look something like this:

@my-state
;; ⇒ {:users [{:name "Ayo"} {:name "Zoey"}]}

The virtual DOM may look something like this (simplified for brevity):

(ul
  (li "Ayo")
  (li "Zoey"))

At the next render phase, React will ask our component, “should I re-render you?” If our state were mutable objects, we would have to self-reconcile our state between two render phases, via the usual prescription of a recursive equality check or some hand-tuned mechanism. But because our state is immutable, we have only have to do a reference equality check, via Om cursors.

David Nolen's article goes in depth.

State over time

As your program's state changes over time, Om automatically re-renders the components that depend on the changed state, producing the DOM represented by your state. We can think of it like this:

$$f(s_0) \Rightarrow \mathit{DOM}_0$$

$$f(s_1) \Rightarrow \mathit{DOM}_1$$

$$f(s_2) \Rightarrow \mathit{DOM}_2$$

What a refreshing way of thinking!

Other than a marvelous elegance, you also get features like snapshots for free. The famous example is this: say we save a snapshot of our states over time, $[s_0, s_1, s_2, …]$. Then choose any state, pass that into your program, and bam your UI is rendered as expected.

This makes time travel (e.g. undo/redo) easy. Planjure, Om TodoMVC and Goya are three example apps that implement undo/redo, all with just a few lines of code.

With Om, the complexity of state management, writing error-prone code to make sure that component A knows that component B knows that component C got a mouse click and is waiting for an AJAX response—all of that goes away. You find yourself astonished, really, at how simple everything can be.

I don't mean, of course, that your programs will be bug or frustration free. But I do think that Om obliterates the state monster and leads you towards the better path.

How to communicate

JavaScript environments are asynchronous—we use callbacks to handle events. Let's say we're building a document editor and we want to save the document either when the user clicks [Save] or [Exit]. Saving a document may take several seconds, so we should ignore new save requests if the current one is in-flight (e.g. the user clicks [Save] and [Exit] consecutively).

If we were writing this in Backbone.js, we may have components that look like this

var SaveButtonView = Backbone.View.Extend({
  events: {
    'click .save-button': 'save'
  },
  save: function () { this.model.save(); }
});

var ExitButtonView = Backbone.View.Extend({
  events: {
    'click .exit-button': 'save'
  },
  save: function () { this.model.save(); }
});

var SaveButtonModel = Backbone.Model.Extend({
  save: function () { this.document.save(); }
});

var ExitButtonModel = Backbone.Model.Extend({
  save: function () { this.document.save(); }
});

var DocumentModel = Backbone.Model.Extend({
  save: function () { /* complex logic */ }
});

But now let's say we want two GET requests to succeed before we print Done!. Furthermore, print Failed! on failure if one of the requests fail. The naive approach is

$.get(opts1, function () {
    $.get(opts2, function () {
    console.log("Done!");
  }, function () {
    console.log("Failed!");
  });
}, function () {
  console.log("Failed!");
});

Even from the code above, and from many of our experiences, we realize that callbacks become unwieldy and cumbersome as our logic increases in complexity. Callback hell, it's often called. Many go to promises to ease the pain. But as the name implies, given a promise, you must know how to fulfill your promise. Why must this be?

Your boss may promise to give you a raise, but she may not actually know how to put more money in your bank account. Most likely, she will send an email to someone else who knows how to put the money into your bank account. Notice that email is a communication channel.

Our code works in the same way—if some module A gets a promise, why should we assume that module A knows how handle this? And let’s not just pass the promise to some module B that knows how to handle it. Why should A even know about B? Your boss does not walk over to accounting, interrupt someone, tell them to change your salary, and watch them type out the number on the screen. No, your boss just sends an asynchronous email and continues with her work.

In the document app example, we said that our app should save the document whenever we click [Save] or [Exit]. Let’s introduce a new method, Ctrl+S. Why should both the keyboard handler and the mouse handler have to hold a reference to the save module which does some complicated logic?

core.async erases this problem by introducing channels. Your app now holds a save channel. Any handler or module that wants to save the document simply puts a signal into the channel (any value would work). The save module, then, saves the document whenever a new value is found in the channel. Any module, including not-yet-written modules, may produce and consume from the channel. Just like email.

In essence, core.async simplifies cross-module communication by putting channels between them. Rich and Timothy goes in depth.

Moving forward

In the beginning, I asserted that we could improve our applications by removing complex state modifications and by abstracting away the DOM. We saw the power of immutable data structures and of thinking about the DOM as an output to a function, instead of a stateful object to be modified. ClojureScript, Om and core.async are a trifecta of tools that help us realize these goals.

But If you’re not ready to take the plunge into Clojure, try out Facebook’s immutable-js and React instead.

At the end, we know that UI development is messy, and often times broken. We want to solve this, but it will require us to try out new ideas and new ways of thought. Both the JavaScript and Clojure communities, it seems, have benefited and learned from each other. So if the ideas presented here seem too foreign, don't balk. If they seem too radical, don't dismiss it. Let these ideas simmer. And if you want, try it out; play with it. But I think you’ll find these tools to be brilliant, powerful and, frankly, a lot of fun.

References

David Nolen: The Final Frontier

David Nolen: The future of JavaScript MVCs

David Nolen: Time Travel

Facebook: Immutable Data Structures

Facebook: React's Reconciliation

Prismatic: Om sweet Om

Rich Hickey: Clojure Concurrency (video)

Rich Hickey: Clojure core.async (video)

Timothy Baldridge: Core.Async (video)

Permalink

A Classic Puzzle


;; A Classic Puzzle

;; This problem, my sources inform me, can be solved by pre-school
;; children in 5-10 minutes, by programmers - in 1 hour, and by
;; mathematicians ... well, check it yourself! :)

(def input (quote ("8809" "7111" "2172" "6666" "1111" "3213" "7662" "9313" "0000" "2222" "3333" "5555" "8193" "8096" "7777" "9999" "7756" "6855" "9881" "5531")))

(map classic input) ;-> (6 0 0 4 0 0 2 1 4 0 0 0 3 5 0 4 1 3 5 0)

;; What is:
(classic "2581")

;; What is classic ?

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Just to make it a bit easier to read

(map (juxt identity classic) input) ;->

["8809" 6]
["7111" 0]
["2172" 0]
["6666" 4]
["1111" 0]
["3213" 0]
["7662" 2]
["9313" 1]
["0000" 4]
["2222" 0]
["3333" 0]
["5555" 0]
["8193" 3]
["8096" 5]
["7777" 0]
["9999" 4]
["7756" 1]
["6855" 3]
["9881" 5]
["5531" 0]



Permalink

Trying Out Reference Cursors

On Thursday David Nolen teased that "the next Om release is gonna to be a doozy”.  On Saturday, he revealed “Reference Cursors”. As he explains and demonstrates in this tutorial reference cursors let your ui components and your data have different hierarchies.

As an example of where this might be useful, imagine your data looks like this:

(defonce app-state (atom {:messages [{:sender "Paul" :text "Let It Be"}]
                          :members [{:name "John" :instrument :guitar}
                                    {:name "George" :instrument :guitar}
                                    {:name "Paul" :instrument :bass}
                                    {:name "Ringo" :instrument :drums}
                                    {:name "Clarence" :instrument :saxophone}]}))

And your UI is structured like this:
 
(defn main-panel [app owner]
  (reify
    om/IRender
    (render [this]
      (dom/div nil
               (dom/div #js {:className "row"}
                        (om/build msgs-panel (:messages app)))
               (dom/div #js {:className "row"}
                        (om/build user-panel (nth (:members app) 0))
                        (om/build user-panel (nth (:members app) 1))
                        (om/build user-panel (nth (:members app) 2))
                        (om/build user-panel (nth (:members app) 3)))))))

The messages panel knows about the messages, and each user panel knows only about the user it represents.

What happens if you want to display each user’s messages in the user panels? You could pass the entire app-state to each panel, but then you have to find another way to indicate which particular user each panel represents.

Reference cursors allow exposing a data hierarchy independently from the ui structure. In this application, we expose the messages:

(defn messages []
  (om/ref-cursor (:messages (om/root-cursor app-state))))

We can then observe this cursor in the user panels, as in the let binding here:

(defn user-panel [app owner]
  (reify
    om/IRender
    (render [_]
      (let [xs (->> (om/observe owner (messages))
                    (filter #(= (:sender %) (:name app)))
                    reverse)]
        (dom/div #js {:className "col-xs-3 panel panel-warning msg-log"}
                 (dom/h4 #js {:className "panel-heading"}
                         (str (:name app) " " (:instrument app)))
                 (dom/div #js {:className "panel-body"}
                          (apply dom/ul #js {:className "list-group"}
                                 (map #(dom/li #js {:className "list-group-item"}
                                               (:text %))
                                      (take 3 xs)))))))))

Even in this small application, reference cursors make for a much cleaner solution. For larger applications, they are going to make a huge difference.

You can find the code for this application on github and you can see the application running here.

Permalink

Networked Bodies, Watermans

out-440x220

On November 8th I’m doing visuals and sound for two unrelated dance performances: Hacking Choreography 2.0 (with Kate Sicchio and Tara Baker) and Ring the Changes+ (with Chisato Minamimura and body>data>space), both as part of the Networked Bodies weekend at Watermans. Hacking was shown earlier this year in Brussels as a project supported by ICT & Art Connect; Changes was premiered at the Southbank Centre in September. The software platforms are completely different (Field vs. Quil – although both performances are Clojure-based), as are the projector rigs, so the setup/rehearsal schedule and changeover are going to be interesting. To add a bit more excitement, Kate and I will be simultaneously live-coding the same running system for Hacking during the performance (me on-stage, Kate NREPL’d in from New York).

Tickets available (for the performances, and for other events in the Networked Bodies weekend) here.

(Photos: Dann Emmons (L), Roswitha Chesher (R).)

Permalink

Copyright © 2009, Planet Clojure. No rights reserved.
Planet Clojure is maintained by Baishamapayan Ghose.
Clojure and the Clojure logo are Copyright © 2008-2009, Rich Hickey.
Theme by Brajeshwar.