Planet Clojure

Escape Hatches: Principles vs. Guarantees

Often, as well-meaning programming language enthusiasts, we might find ourselves encouraging newcomers with simplified descriptions of language features. There’s sometimes a tricky balance between explaining things in an approachable way and explaining them correctly and precisely. Let me give some examples of what I’m talking about, and then we can zoom out to think about the implications.

Ruby

Ruby, like many OO languages, has visibility methods that let you ensure nobody will depend on the code you’re using! This helps you know what’s safe to modify!

Sure enough, the basics work like we'd expect:

class ActivateUser
  def call(activation_code)
    user = fetch_user(activation_code)
    raise ArgumentError, "Activation code '#{activation_code}' is invalid" if !user
    user.update!(:activation_code => nil, :active => true)
    user
  end

  private
  def fetch_user(activation_code)
    User.where(:activation_code => activation_code).first
  end
end

class TestActivateUser < Minitest::Test
  def setup
    User.delete_all
    User.create(:email => "jen@example.com", :activation_code => "GOOD_CODE")
    @activator = ActivateUser.new
  end

  def test_a_bad_activation_code_cant_be_fetched
    assert_raises(ArgumentError) { @activator.call("BAD_CODE") }
  end

  def test_a_good_activation_code_updates_and_returns_the_user
    user = @activator.call("GOOD_CODE")
    assert_equal user, User.where(:email => user.email).first
    assert_equal "jen@example.com", user.email
    assert_nil user.activation_code
    assert_equal true, user.active
  end

  def test_a_good_activation_code_updates_the_user
    user = @activator.call("GOOD_CODE")
    assert_equal "jen@example.com", user.email
    assert_nil user.activation_code
    assert_equal true, user.active
  end

  def test_private_method_access_prevented
    assert_raises(NoMethodError, /private method `fetch_user` called for/) {
      @activator.fetch_user("GOOD_CODE")
    }
  end
end

But we can bypass private methods in at least a few ways:

def test_private_method_access_allowed_with_send
  user = @activator.send(:fetch_user, "GOOD_CODE")
  assert_equal "jen@example.com", user.email
end

def test_private_methods_can_be_made_public
  # We could send this message to `ActivateUser` instead, but then we'd
  # need `Minitest::Test.i_suck_and_my_tests_are_order_dependent!` to keep
  # our tests deterministic, and nobody wants that.
  @activator.singleton_class.send(:public, :fetch_user)

  user = @activator.fetch_user("GOOD_CODE")
  assert_equal "jen@example.com", user.email
end

Ah, Ruby. private is a principle, not a guarantee.

Clojure

Use Clojure - you don’t have to worry about anyone mutating your collections out from under you! Immutability by default!

Not so fast. This test suite passes too:

(test/deftest test-immutability
  (test/testing "the awesomeness of immutability"
    (let [xs [0 1 2 3 4 5 6 7 8 9]
          ys (map (fn [x] (* x x)) xs)]
      (test/is (= (range 10) xs))))

  (test/testing "the pleasing nature of... wait what??"
    (let [xs [0 1 2 3 4 5 6 7 8 9]]
      (do-something xs)
      (test/is (= (repeat 10 :boom) xs)))))

In Clojure, you can use Java interop to mutate PersistentVector. Similar things are possible for other persistent data structures.

(defn- replace-array [f arr]
  (dotimes [i (alength arr)]
    (let [current (aget arr i)]
      (aset arr i (f current)))))

(defn- replace-node [f root]
  (let [node-arr (->> root .array)
        first-node (first node-arr)]
    (if (and first-node (instance? clojure.lang.PersistentVector$Node first-node))
      (doseq [node (filter identity node-arr)]
        (replace-node f node))
      (replace-array f node-arr))))

(defn mutate-vector [f xs]
  (replace-node f (.root xs))
  (replace-array f (.tail xs)))

(defn do-something [xs]
  (mutate-vector (constantly :boom) xs))

Ah, Clojure. "Immutability" is a principle, not a guarantee.

Haskell

You should use Haskell, where the type system enforces immutability and even side effects! You can tell just by looking at the type signature!

You can probably sense where this is going: this isn’t really true either. unsafePerformIO can take you to some pretty gnarly places:

count :: String -> Int
-- definition redacted for now

data StringData = StringData {
  text :: String,
  reversed :: String,
  size :: Int
} deriving (Show)

stats :: String -> StringData
stats text = StringData {
  text = text,
  reversed = reverse text,
  size = count text
}

main :: IO ()
main = print (size (stats "Hello, World!"))

Since count returns an Int, we should see main printing an Int. But that's not what happens!

It prints:

"OHH NOOOOOO"
13

Despite the type signature clearly indicating purity, we’re able to do whatever impure side-effecting operations we want:

import System.IO.Unsafe (unsafePerformIO)

count :: String -> Int
count text =
  unsafePerformIO $ do
    print "OHH NOOOOOO"
    return $ length text

Ah, Haskell. Purity is a principle, not a guarantee.

Zooming out

If you're a developer and/or advocate for one of these technologies, I’m sure you have your counter-arguments ready about why the example in your preferred language isn’t such a big deal. It might surprise you, but I don’t disagree with the counterpoints—far from it!

My point is not that everything is on fire and nothing is sacred. Affordances to help developers do the right things can still be very useful, even when they’re not guarantees. You can break a law, but laws are still useful. And programming principles with escape hatches can be, too.

I don't want to litigate the necessity of these escape hatches for the language designers, because I don't think it's actually that important. Whatever the reasons, these facilities give us the power to do things that seem to conflict with key principles in these languages.

And when we overstate our opinions and frame principles as guarantees, to put it charitably we confuse people about what they can and should believe. We don’t have to give everyone all the details of edge cases like these. But some acknowledgement of where the lines exist, between principle and guarantee, can help us to avoid those misunderstandings and potential distrust.

Being precise with language can be a huge pain sometimes. Such a huge pain that I'm confident there are at least a handful of imprecise and even incorrect statements in this post. But by striving for precision as a principle, I think we're a lot more likely to build trust with the folks we're communicating with.

If you're interested in playing with these examples, check out the Github repo.

Permalink

Love letter to Clojure (part 1)

Gene Kim shared part 1 of a “love letter to Clojure” inspired by Bryan Cantrell’s amazing “I’m falling in love with Rust” blog post in September 2018

In this blog post, I will explain how learning the Clojure programming language three years ago changed my life. It led to a series of revelations about all the invisible structures that are required to enable developers to be productive.
…
Without doubt, Clojure was one of the most difficult things I’ve learned professionally, but it has also been one of the most rewarding. It brought the joy of programming back into my life. For the first time in my career, as I’m nearing fifty years old, I’m finally able to write programs that do what I want them to do, and am able to build upon them for years without them collapsing like a house of cards, as has been my normal experience.

Discuss on Changelog News

Permalink

Why do I prefer Clojure to Haskell?

I prefer Clojure to Haskell. It’s probably mostly due to accidents of history: getting in Lisp at an earlier age, spending more time with Lisp, and only having one Haskell job. But I do have some philosophical differences with the Haskell philosophy as it relates to Haskell as an engineering tool. In this episode, I go over four of them, and try to relate them with anecdotes.

Why do I prefer Clojure to Haskell?

I prefer Clojure to Haskell. It's probably mostly due to accidents of history: getting in Lisp at an earlier age, spending more time with Lisp, and only having one Haskell job. But I do have some philosophical differences with the Haskell philosophy as it relates to Haskell as an engineering tool. I

The post Why do I prefer Clojure to Haskell? appeared first on LispCast.

Permalink

PurelyFunctional.tv Newsletter 348: Tip: cleanup in finally

(let [resource (create-resource)]
  (use-resource resource)
  (cleanup-resource resource))

(let [resource (create-resource)]
  (try
    (use-resource resource)
    (cleanup-resource resource)
    (catch Exception e
      (handle-exception e))))

(let [resource (create-resource)]
  (try
    (use-resource resource)
    (cleanup-resource resource)
    (catch Exception e
      (handle-exception e)
      (cleanup-resource resource))))

(let [resource (create-resource)]
  (try
    (use-resource resource)
    (catch Exception e
      (handle-exception e))
    (finally
      (cleanup-resource resource))))

(defn fetch-user [userid]
  (let [db (db/connect)]
    (try
      (db/fetch db "user" userid)
      (finally
        (db/disconnect db)))))

(let [writer (io/writer "my-file.txt")]
  (try
    (doseq [x (range 100)]
      (.write writer x)
      (.write writer "\n"))
    (finally
      (.close writer))))

(with-open [writer (io/writer "my-file.txt")]
  (doseq [x (range 100)]
    (.write writer x)
    (.write writer "\n")))

(run! println (range 100))

(defn runp! [f coll]

The post PurelyFunctional.tv Newsletter 348: Tip: cleanup in finally appeared first on PurelyFunctional.tv.

Permalink

Grokking Simplicity – HTML All The Things [Podcast]

In this episode, I talked about my new book “Grokking Simplicity – Taming complex software with functional thinking”.

HTML All The Things

Transcript

Matt: Hello everybody and welcome to the HTML All The Things podcast, Episode 63. Grokking Simplicity with Eric Normand, I’m here with [inaudible 0:14] Lawrence and I’m joined again by my co-host, Mike Koran.

If you’ve been enjoying the podcast so far, and you want to support us, there are a couple of ways that you could do that. You can review us on the apple podcast or on the podcast platform that you are listening to this on.

You can check us out on Patreon, we only have a couple of tiers right now but that three-dollar tier will give you a shout out in the podcast and we’ll even share a link to your website in our show notes. Probably the most important one is just to tell your friends that we’re here and we’re ready to be listened to. Get this out there. Just share.

We really need a slogan for that, “Just Share,” or something like that. It almost sounds like “Shazam.” We also have a discord server, so if you’re ready to do more than just listen to us, if you want to come and hang out, you can come and join us in our discord server, which now has over 200 members.

I’ll do my best impression of a party noise maker, [screams] , but anyway, [inaudible 1:08] come, join us in that discord server to come chat about off topic things, about development things. We got people in there talking about WordPress, HTML, CSS, JavaScript, jQuery, BHP, everything under the sun.

There’s probably something that we’re not talking about in there, but anyway we have a whole bunch of stuff we’re talking about. If you want to come and hang out with a cool group of developers from all walks of the industry, from the experience through the non-experience, feel free to do so.

Link to that discord server will be in the description almost said, what is this, YouTube? It might actually be, but anyway, we’ll be in the show notes. Now, in today’s episode, we will be discussing the book, “Grokking Simplicity: Taming complex software with functional thinking” with none other than the author himself, Eric Normand.

He started with functional programming in the year 2000 and has since watched the industry slowly take notice. Eric has spoken at conferences around the world covering topics like functional programming, Clojure, ClojureScript, property-based testing and concurrent programming.

On top of that, he also offers consulting and training services having worked with the likes of the federal government through non-profit companies. If you’d like to pick up Eric’s book, we have a promo code available which is podhtmlall19.

We’ll be including a link to the place where you can actually purchase the book as well as that promo codes. You can easily read it, just copy and paste it. That’ll be in the full show notes on the HTML all the things website. We also have actually five copies of the book to give away for free.

We’ll be putting two of those codes in the show notes on htmlallthethings.com. Now, full disclosure, these are single use codes, so first come, first served. If you’re lucky enough to grab one of these codes, now if you copy them, they aren’t gonna disappear. You better copy that code and immediately try to redeem it.

Also, one thing to just note in case there’s a delay of some sort, these codes are only valid for two months from September 12th, 2019. Make sure you claim them in time before they time out. Now, I did mention that we have five codes that we can give away.

The other three, we will be giving away in our discord server. I will be tagging the @everyone tag for anyone that’s in the server. I’ll be explaining how we’re going to give those away and they’ll be given away within a day or two as of this recording, which is, let me just read the date to make sure this is official, which is October 8th, 2019.

Make sure that you go and check that out if you’re listening to this as a fresh episode. If you’re in the future, I’m sorry, but you can use that discount code if you’d like to get a nice discount.

Now, without further ado, let’s cut to that, let me say juicy, but I don’t know what I meant by juicy like am I just thinking of the keg, the juicy steak. Anyway, let’s cut to that interview with Eric and Mike, stay tuned.

All right everyone, we have Eric on the line, and as is the tradition around here, Eric, please take it away. Let us know what you’re doing, what you’re up to, and a little bit of your back story.

Eric Normand: Hi. I’m Eric. I am currently writing a book about functional programming for beginners to functional programming, but people who have had some experience with at least one language. It’s called Grokking Simplicity, and it’s published through Manning.

Basically, it’s supposed to be a fun trip through the paradigm shift of functional programming. That’s one thing I’m doing. I’m also a teacher online. I record videos, courses on Clojure and functional programming. You can find those @purelyfunctional.tv. That’s about all I do.

Matt: Great, thank you for being on the show. I’m really looking forward to this episode because we haven’t really done…I don’t know if we’ve done any talks. Maybe Michael will have a comment on that, done any talks at all on functional programming on the show.

This should be a really interesting one, especially for a lot of our listeners because they are usually either hobbyists or beginners or some veterans in their course, but this will be a really interesting sort of a topic to cover since it’s so new to us.

I’ll jump right in here. These are going to be some preemptive questions regarding the book in the segment Writing a Book in our show notes there. “Grokking Simplicity” isn’t complete yet, but rather has been released as an early access title. Why did you choose to go the early access route?

Eric: The idea behind early access to get more feedback faster. Writing a book is a long process. It’s good to know how the book is going to do [laughs] before you finish the book. If it doesn’t do well, you could try to figure out why. Get some feedback from some of the early readers and pivot. Maybe they’ll ask you good questions that directs the new chapters that you’re still writing.

I’ve talked to the people at Manning. They say that the reason they started it was because technology changes so quickly that often a new version — let’s say you’re writing a book about a database — of the database could come out before you finish writing your book.

It’s a shame because the first three chapters have been ready for a year. [laughs] If someone could have been using them and gaining some value out of it before the final chapters are done. I look at it like that.

It’s a good way to get it out there. It’s very motivational when people buy it. I’m not done. Oh, some people are buying it. OK, I better keep working on it. I better finish it up. I do that with my courses as well. I knew what I was getting into, and I really think that it helps a lot.

Mike: How do you choose when to stop and release the early access portion, or if that’s not enough, or it’s much? When do you cut yourself off and be like, OK, this is enough for an early access bit?

Eric: Good question. Manning has their own process behind it that I’m not really in control of. I don’t really have that much introspection into how it works. But this is what they’ve told me that they want to release something when it has significant value and it’s not embarrassing.

Something where you’ve put some good quality content in there and it’s not too rough that people would be like, “What is this?” Then the idea is publish one new chapter or a significant update to the book every month just to keep the momentum going, keep people who have bought engaged with the book. Don’t let it go too long without new content so that they’d forget what the book is about and why they were reading it.

Mike: Another thing. Matt and I read the chapters that we got. You mentioned in those early chapters future chapters. Like in Chapter 17, we’ll cover this and that, and stuff like that. Before you even begin, you have a full outline of what you’re going to be covering chapter by chapter.

I’m assuming you have to send that to the publisher, get that approved. There’s probably a big process where your book is planned, it’s just those other chapters aren’t refined yet, right? Is that how it works?

Eric: Yeah, that’s right. After we signed the contract and everything, the first thing they ask me for…This is the publication contract. The first thing ask me for was a table of contents. I was like, “Are you sure? I could make one up, but I don’t really know [laughs] what I’m going to have.

They were like, “Yeah. That’s what we want.” We want to know what are the topics you think you’re going to cover. It can change. It’s not a commitment, but that’s where we work. At first, I was just very hesitant to do it, very reluctant but the more I thought about it, the more I was like, “OK, I have these ideas in my head. They don’t know what I’m going to write about. They have no idea. They need something. [laughs] They can’t just wait for me to write them.”

I sat down and I made up the table of contents. They send it out for review to other functional programming experts and asked them questions like, “Are there any topics missing? Are there any things that you think are important?”

Sort of grade the thing on one to five. Is it good quality? Do you think that this would be worth reading? Questions like that. Everyone said, “Yeah. Those things sound good.” At the same time the people who weren’t functional programming experts like my editor and other people at the end manning, they were like, “I don’t know what this is about.”

[laughter]

Eric: What is that chapter about? One of my chapters is, “sets.”

[laughter]

Eric: They are like, “Sets? What does that have to do with functional programming?” I’m like, “Well, you wanted me to write the topic with the chapters. This is it. This is the topics of the chapters.” Since then, the table of contents has refined.

I’ve gotten better about understanding how to craft the title of a chapter and what I actually want to say. Actually, I think you read…Yes. You read the first release the V1 of the MIP.

In the V2 which is coming out really soon, I can’t say when it’s out of my hands but it’s close. There’s a totally different table of contents because I reorganized the material in a much better way. It’s evolving. It starts somewhere and it evolves.

Matt: Sorry. What I was going to ask there was, how open are you and maybe the publisher are to actually changing the maybe not the addition number but the previous chapters? Do you guys get stuck on, “OK, I’ve written parts.” Let’s say parts one through four. You keep refining and refining that.

Then struggle to get on to part five in this case? Or do you keep writing ahead and then just keep going back when you receive that feedback and do like little changes?

Eric: Yeah. This is another interesting question. Just a little background. I’ve been blogging for over 10 years. I’m much more comfortable like, “Let me just write a first draft and hit publish.” [laughs] Then my publisher is there.

They want me to go over and like, “Oh, this could be better.” I’m like, “Yeah, but it’s good enough for the MIP, right? I’ll come back later.” They’re like, “Well, while you’re here just finish it.” I’m like, “Just send it out.”

There’s a little tension there because I really do have this tendency of just like, “Get the idea out, see what people think. Get some comments before regretting sending it out [laughs] and definitely change it.”

I would much rather finish a sketch of all the chapters and then come back through because by the time you get to the end, you’ve learned so much that when you do the second pass you can incorporate that learning.

My editor is much more like, “No, first impressions matter.” Like, “We know how to improve this. Let’s just keep improving this until we can’t.” Really in practice what happens is we do that until I feel stuck.

I feel I could improve this, but I’m tired of it. I’m burned out on this chapter. Can we just put it down and move on? That’s what happens. It’s more like, “I just don’t want to work on it anymore.”

[laughter]

Matt: I guess that’s when you know you’ve put enough work into it, right?

Eric: Yeah. I guess so. I don’t know. I’ve never seen him stop. [laughs] Maybe we can keep going and going and going. It’s really maybe that’s what he’s doing. He’s pushing me until I can’t go anymore. I don’t know.

Matt: The next question here, so when would you say is a good time to start writing a book? Couple of things to compare to. Like, how much expertise do you need to have? Do you need to perceive things a certain way? Do you need to have a level of mastery in some way?

Eric: That’s a good question. I think it depends a lot on your goals. What are you trying to get out of the book? One of the things is that you don’t want to…You don’t want to fake expertise, fake mastery, fake more experience than you have. You don’t want to do that. People can see through that really easily.

That being said, I think that if you’re a good writer you can make a compelling story or a compelling read out of your beginner experience. If you could turn that into something, I think it’s there. However, depending on your goals, you might…

What I’m afraid is someone who doesn’t have a lot of expertise will waste their time. It takes a long time to write a book, a lot of effort. If your goal is something like write this book and make a lot of money or sell a lot of copies or even gain some notoriety in the community, you’re much better off earlier on publishing smaller things.

A blog for instance, articles in the magazine. Smaller things that you can finish and see how well you’re doing you’re developing the craft of writing, you’re developing your communication style and the message that you want to have.

You might find that, “OK, I have written 10 magazine articles. I think that I’ve got a book idea now.” “This is too big for a magazine article.” I think that it should come naturally and organically.

For instance, my book, my goal is to spread functional programming. I think that is one of my goals. I have several goals for the book. One of them is to make functional programming more accessible, more popular.

I just had this idea, this little core and I had to BS the rest of the table of contents. I knew that there was a core there. As I developed it, it turned into a well fleshed out table of contents. It’s big enough for a book. I knew that.

I only knew that because I’ve written a lot on my blog. I’ve given conference talks. I’ve published these videos. I knew that it was a major thing. I couldn’t do it justice in a blog post.

Matt: That’s interesting. I like how you broke it down. For me, when I think about writing a book, it scares me right away. I think it’s because I don’t have that body of work that you’re talking about. I don’t have something that I couldn’t say in a blog post. I think that’s my issue.

Whereas what you’re saying is, when you get so many blog posts, and you’re like, “OK, I’ve planned out seven or eight different blog posts on the same topic.” Maybe that’s the point when you’re like, “OK, maybe I can expand this, because I have so much to say about it, into a book.”

I really like how you broke that down. I think that’ll help the audience understand when it is you should start thinking about writing a book, rather than, “Do I have the knowledge? Do I have the expertise?” Stuff like that.

You’re right. That’s the wrong way to think about it, because it’s impossible to tell.

Eric: Right. I didn’t know I was going to write a book until I found that idea. It wasn’t like, “OK, I need an idea for a book, let me brainstorm ideas.” No, it was the opposite. I had this thing…

I wrote a blog post and it was about different paradigms like, “How do you define imperative programming? What’s object-oriented programming? Then, “What’s functional programming?”

In doing that I realized I came up with a pretty unique formulation for what it is. This is from years and years of writing about functional programming and talking to people, going to conferences, practicing functional programming. Then I was invited to speak at a conference, and they said, “You pick the topic.”

I said, “I think I want to talk about this idea about what functional programming is.” I gave the talk. Then I realized, “This is a big idea. The talk couldn’t contain it. I need to turn it into something bigger.”

I started podcasting about it, and it’s just like streaming out of me, all these topics on functional programming. No preparation, I just start recording and start talking about a topic.

Then, I would do that multiple times a day [laughs] like so far ahead, I was like I should publish two a week as I am never going to finish my recorded episodes. Then, I decided, “OK, it is going to be a book.”

Matt: That’s cool. It’s a cool little insight into an author’s mind, I guess. [laughs] One question and this is more of an organizational project planning kind of question, how do you break up your day? How much time do you spend writing your book?

Is it like a daily thing where like this day is just meant for the book or do you break up your day, be like four hours book time, four hours not book time? What is your process?

Eric: My process is really disorganized.

Matt: [laughs] Like all of us.

Eric: Yeah. I’m trying to think if there is some like, “Oh, this is my ideal,” like I have done it for a week and it was awesome. I’ll tell you, what I try to do is have a day for the book where it’s like, “Tuesday, I’m going to sit down and work on chapter six.”

That lets me get into the book, forget other things. I close my email, I close all that, just work on the book. I have two kids and they disrupt my schedule a lot. One of them is a baby, so it is very disruptive.

The book also is not very text-heavy. When I write blog posts, I’m like, “OK, I want to write a lot of blog posts. I have a big post to write.” I have a schedule like, OK, I’ll wake up early in the morning, and I just get my keyboard and a coffee and I just start writing. I just type.

This book isn’t like that. I really need to be able to work in different modes. It is also such a big topic. I have to take notes. I have note cards, I am like saying, “This is later. This goes in that section.”

It’s also a lot of mouse work, like laying pictures out, doing codes, snippets, and things in the book. It’s not quite as like single-minded focus as I remember from writing a lot of blogs. That’s why I like to have the whole day because I don’t know how long it’s going to take.

I want to be able to just spend all day with no cards like shuffling them around, maybe I don’t produce any pages that day.

Matt: How vital would you say is — I guess this is more toward blog posts — hitting those targets that a lot of engagement specialists and a lot of marketing folks will say, “You should do a blog a day or a blog every other day.”

How much pressure do you put on yourself to actually reach targets like that like, “Oh, I need to post something every Wednesday,” let’s say, or something like that.

Eric: Right now, the only schedule I really try to keep…I guess I have two schedules. I have the podcast. I feel like that should be regular, I don’t know why but I feel like I want a podcast to go out every Monday and Thursday.

I’ve got my newsletter that goes out every Monday morning. I want to do at least an hour of video per week — video courses. Other than that, I feel like I cannot really sustain anymore.

I have also, in my blogging, gone more toward a longer form and more comprehensive guide approach because I think at the point where I am at, in terms of you SEO and authority building and stuff, I really want…

I’m working in Clojure, that’s what I’m teaching. I want there to be a comprehensive Clojure learning site. These little one off 500 to 1,000 worded blog posts are not going to ever get there. I need to say, “OK, I’m going to look at every single data structure,” and just list them out.

For each one, I am going to talk about when to use it, when not to use it, and that turns into an 80,000-word thing. That’s just the link that has to be. I think that works better for me where I am at.

I do think that there is something to having a schedule because I am not typing so much right now like typing words for writing stuff, I find that I hesitate a lot more like “Oh, maybe I shouldn’t type that,” whereas when I have a schedule like, “OK, every day, I have to post a blog post.”

I don’t think. I’m like, “I’ve got to get this thing out,” because [laughs] this is a lot of words. You know I had tricks. Here’s a trick that I used to use. I was for, I don’t know, a few months posting a blog a day.

What I realized was, when someone would ask me a question in email, I wouldn’t hesitate, I would just open up the email, hit reply, and just start typing an answer. I thought why does blogging have to be harder than that.

What I would do was the night before, I would email myself a question. I would phrase it like, “Hey, Eric. I’m a big fan. I have a question about Clojure,” and [laughs] then I would write out the question like I was really asking it.

Then, I’d go to sleep and then in the morning I’d wake up with my coffee and I’d open up the mail client like, “Oh, an email from a fan,” and [laughs] I’d open it up. Just like, “Oh, nice question, Eric.”

I would always start like I was actually responding like, “Hey, Eric, nice question, thanks for reaching out, enter.” Then, I’d just start typing the answer and then that will be my blog post. It worked, it was like a psychological trick to get me to write more without hesitating and second-guessing, “Is this worth it?” or whatever. No.

It wasn’t like these comprehensive guides. It was a specific question, a specific answer and it worked. It’s really great.

Matt: It’s like setting your clock ahead by 10 minutes. You’re just tricking yourself every day.

Eric: [laughs] Yeah, exactly. By telling your friend, “Pick end between 10 and 20” and set my clock ahead so I won’t know how far ahead it is.

Matt: [laughs] Yeah, exactly. It works, it’s been proven to work. That’s a cool little trick there, that’s awesome. In your blog post would you actually keep the header parts?

Eric: That’s a good question. What I would do is say, “Hey, that’s a great question. Let me rephrase the question for you just to see if I understand it and then I’m going to answer it.” I would start the paragraph like, “Someone just asked me, duh, duh, duh, duh.” [laughs] and then I’d answer it. Yeah.

Matt: Giving away all your secrets to your fans. Everyone thought you were just getting asked all these questions but…

[laughter]

Eric: Sometimes they were directed at me, but usually it took me a month to really come up with the answer. Now that I sent it to myself…

Matt: Now it’s in writing and it’s there in front of you. That’s cool.

Eric: Exactly.

Matt: With that all being said, there was a segment about, “How to write a book” and stuff like that. I want to move on to functional programming because I think it is an important topic. I learned about it a little bit when I was studying in university. We learned about all the different programming paradigms.

When I sat down to read your book, I didn’t know what to expect obviously because you and I had never talked and I had never read anything from the Grokking series. Grokking is a series from what I understand it right that you’re just adding on to as an author?

Eric: Yeah. Series is a book term. They’re not related like you got to read this one, then the next one, and then the next one like a television series. You have to watch them in order. They do have a similar style and they’re supposed to have a similar scope and feel to them.

Matt: Yeah, exactly. When I sat down to read it, I was actually in a hotel room and I popped out my iPad and went through it. It was actually a really interesting and engaging read. Even for someone that maybe wasn’t expecting it and wasn’t looking for, “OK, I need to change all my stuff to functional programming.”

I thought it was good in the sense that it taught me what I was doing that was already functional and how to think a little bit differently as well. Which is valuable to anyone at any point in their programming life. If you’re just starting out even, I would say this is a valuable thing to know about. To know about a thing called functional programming.

Because not only will you know about, “OK, this is how to think functionally,” but you’ll know that what you’ve been doing, there’s certain things you’ve been doing that are already functional. It’s cool to label what you’ve been doing because it’s tough.

When I was first starting out, you just sit down and write code. Your code just looks however it is — spaghetti code, functional code it doesn’t matter. It just works. It’s tough to talk about it to other industry professionals. It’s tough to label it, it’s tough to organize it in your head as you go forward.

It was definitely a helpful reminder of functional programming and a very much a “scope-broadener.” It definitely broadened my interests in it which was cool. I like the way book was laid out like you were saying, it wasn’t text-heavy. It’s very much diagram-heavy and picture-heavy which is a really cool way of doing a programming book.

Because I read a few of them and a lot of the ones that I have read have been text-heavy and they’ve been a little bit harder to sit down and read in an afternoon. I sit down and read when I need to like, if I pick up a book usually about programming, it’s usually because I need something from it period.

It’s not an entertainment thing for me. Whereas a Grokking-type style like this style of book was almost an entertainment thing. I sat down with a coffee like you were saying, you were writing it, I sat down the same way reading it and I was like, “Wow, this is actually really interesting.” Props to you on creating that atmosphere in your book and…

Eric: Thank you.

Matt: Yeah, no problem. I just want you to give a quick overview to our audience because our audience is usually very new to programming. Usually new to web development in general. They’re looking at it from a very beginner point-of-view, so if you can give a nice overview, doesn’t have to be too in-depth obviously of what functional programming is.

Eric: Sure. I do want to say thank you for all the nice things you just said about my book. I feel like you hit the nail on the head or I hit the nail on the head. [laughs] That was my goal. What you said is my goal. It’s to make it feel like, “Oh, I know some functional programming.”

I do want to also thank y’all two because there’s a lot of beginners out there and you’re making great content for beginners and really making it very accessible for them. Anything to help people along, any content out there, thank you so much.

Matt: Thank you.

Mike Koran: Thank you very much, yeah.

Eric: OK, an overview of functional programming. This is not going to be the Wikipedia definition of functional programming, because I think the Wikipedia definition of functional programming does a disservice by being too academic and obtuse.

There’s a lot of baked-in assumptions about what you need to know to understand that definition. This book is trying to take people who don’t have those assumptions already and give them a perspective for how to see their code. So that that definition might one day make sense to them.

Functional programming is a paradigm. You might have heard of imperative programming or object-oriented programming. Functional programming is one of those. Those are three different paradigms. It’s a set of perspectives and skills that you approach problems with.

You have some programming problems, how do you look at it? What are the first things you do? What are the things you look for? How do you look at the problem, and then what kinds of programming skills are you going to bring to the table to solve it?

The way I define functional programming is the first perspective, the first distinction that a functional programmer makes is to divide everything into actions, calculations, and data. Actions are things that have an effect on the world or if your system is affected by the world.

If you get an event like a mouse click, your program is being affected by the outside world. Or you could send an AJAX request that’s having an effect on the world. Because you’re posting some data to the server, now the server is different.

Calculations are pure computations. That means that it’s something like addition. You know four plus four doesn’t have any effect on the world. It just calculates eight and it’s going to calculate eight every time. Doesn’t matter when you run it. It’s timeless. It’s just always going to do that. You can write your functions to do the same.

If you wrote a function that would just return four plus four. That function now, every time you run, is going to give you eight. It’s going to give you the same answer. You know you can make a more complex function of course that has arguments.

If, for the same arguments, it returns the same value and it doesn’t send Ajax requests or do any actions, then it’s a calculation. It’s something that you don’t have to worry about when it runs, whereas for instance, if your system sends an email, you do have to worry about when that runs.

I want to send the email, or at least I have to worry about how many times it runs. Sending the email zero times is totally different from sending it one time. There might be things that you have to do in a certain order like I want to set the button to spinner, like get the Ajax request, and then that response is going to…Now, I’m going to put the value into the button.

These things have to happen in a certain order and it’s hard to have order in a Async environment like JavaScript. It turns out other systems, other languages that don’t have Async stuff, they still have the same problem.

Then data. This is the third thing. Data is just what you would call data. Numbers, strings, data structures. What’s interesting about them is they are totally inert. They don’t do anything. They just sit there. You can send them across the wire. You can save them to a disk. You can do whatever you want to them. They can be interpreted at all ends of that sending and receiving.

Because they don’t do anything, they need to be interpreted by something. They need to be interpreted to give them meaning, because the number four doesn’t mean anything unless you know how to interpret it.

The same with a huge Jason blob. What do I do with this? Oh, I’m looking for the user ID, which is down in this object deeper in here. OK, that’s where the user ID is stored. You’re interpreting that object in a certain way.

Data is the safest, because it can never crash. It can never have any problems you can have bad formatted data, of course, but it’s data, it’s still data. It’s not going to cause any problems.

Calculations are good because they’re pure. They can be easily tested. One thing about testing is you’re probably going to want to run the same function multiple times. You don’t have to set up any environment for a pure function, for a calculation because it doesn’t depend on anything on the outside except the arguments, and you pass those in and you can choose exactly what to pass in.

You can run it on a different machine. You can run your test on your billed server. That test is going to be valid for someone’s browser, when the JavaScript is running in the browser.

Actions are the most problematic but they’re necessary, because you need to send that email or you’re not doing what your software is meant to do. We need them, but we also know that they’re most dangerous. They’re the most difficult to work with.

A lot of functional programming, starting with these divisions, splitting things up and identifying these are my actions, these are the calculations, these are the data. Once you split them up, you can start to focus more attention on the actions because that is where the problem is. You just pay more attention to them.

You can move code from action into calculation. You refactor stuff. Pull it out, “Oh, look, here’s this little calculation I’m doing inside of this big action.” I can now make that into a function, which is a calculation. You’re moving code out, making it more testable, more reusable.

This is a large part of what the first part of the book is about. It’s about how to identify them, how to refactor stuff out, how to work with data so that it’s immutable, meaning you never change it.

Then, the chapter wraps up with a little bit of architecture, a little design called stratified design, where it’s about how you can design these layers of meaning so that the layers of code align themselves with how fast they change.

You’d imagine the fastest changing code is on top and the more stable code is on the bottom. That’s really nice because the stuff on top is built on the stuff below it. You don’t have to change the stable stuff. If you had to change the stable stuff, [laughs] everything on top would topple over.

Because it changes less frequently, you’ve sort of reduced your maintenance to the minimum.

Matt: Yeah, that’s a really good, succinct functional programming overview. Thank you for that, that’s for sure.

Hopefully, with that, people can kind of form something in their minds of what functional programming is because as we’re going to move forward, we’ll talk a little bit about functional programming as well as when to use it and stuff like that. Just keep what Eric had said in mind.

Next question here would be, most developers, especially ones listening to the podcast, wouldn’t label themselves in one of these paradigms because they’re just starting out like they’re just doing JavaScript.

JavaScript can be one of many things [laughs] when you’re writing it. That’s one of its things. How important is it, two-part question here, to know about all the different approaches that are out there like all the different paradigms out there right away as you’re starting out developing?

How important is it to use one and then strictly follow it, like really dive into only functional programming and be an expert in that?

Eric: I have two answers to that [laughs] . I don’t think it’s so important to be strict on any one paradigm when you’re programming. I look at them as sets of skills. In my book, I’m really breaking up functional programming as a set of skills that happen to be used by functional programmers.

These are things that come from that perspective. They’re not exclusive to functional programmers, but I would say that they’re more common, functional programming team, but they’re useful. They need to pay the skills and the approaches, need to pay for themselves right away.

Meaning you should not think if I’m not a 100 percent functional, it’s not worth doing at all. You should be able to see, “Oh, if I just made this one thing immutable, then I would solve the problem that I have.”

You don’t have to make everything immutable, just this one thing or this function is getting really long, let me factor out all of the little calculations inside. Maybe that’s one little thing you do in one place in your code and it just makes it a little bit better.

That would make me very happy if people learned to do this kind of piecemeal functional programming. That said, in terms of learning it, I think there’s something to be said for immersion.

If you’re really into learning object-oriented programming, you might be better off choosing a language that makes it really easy and makes everything else hard. Then you’re going to have to figure out how to solve problems with those skills and those perspectives rather than some other one that you might already know.

Likewise, with functional programming, I think it is easier to learn if you are in a functional language so one that makes functional programming the default.

That’s not to say that you can’t learn it and you can’t learn the skills, but it’s just like learning a language. You can learn to speak a few words and even some grammar and make sentences, but everyone knows the best thing is to just live in the country where you’re forced to use it.

Matt: Absolutely. Just to follow up with that, what would you say are the functional languages then like? Do you have a few in mind?

Eric: Yeah. Clojure, Haskell, Erlang, Elixir, PureScript, TypeScript. Not TypeScript. I wouldn’t put TypeScript in that. Elm on the front end. Rust is pretty functional. Swift is functional.

Matt: Swift is functional. Where would you label something like JavaScript?

Eric: JavaScript is multiparadigm. I would say that you can do functional programming in it but it’s not…It’s like a city where you can speak both German and Spanish. If you speak Spanish, you’re not going to learn German because you’re just going to speak Spanish every time.

If you know imperative programming in JavaScript, it’s going to take a lot of will power and discipline to stick with the functional stuff which might be good. If you’re working at work and you just need to get features done, sometimes you’re just going to do it the familiar way.

Then sometimes you’re going to say, “OK I’ve more time, let me see how I would do this in a functional way.” That’s a perfectly fine way to learn. Do it again. Once the way you’re familiar with and then the second time try to approach it with different set of constraints. That’s a great way to learn it fact.

Matt: That makes perfect sense. I think with the web developing crowd in general it’s kind of these concepts are always a little bit harder because of that multi-paradigm approach of JavaScript.

When you’re talking to someone and you’re like, “Just use the object-oriented approach” They are like, “Why? I’m just writing this code.” Like, “Why are you trying to get me down in certain specific path?”

Eric: I’m just getting it to work. [laughs]

Matt: Yeah, I’m just getting it to work even though if they were using something like Java, they would be forced to use the object-oriented and it’s a lot easier.

Eric: You see the opposite which is someone who overuse its OO. That’s great because they’re exploring the class keyword, the new class keyword in JavaScript. They want to do everything with that instead of the simple way which would just be use the function. Like, “Why are you making a class for that?”

That’s great except if you’re doing it on production code like that’s not your time, you’re spending. [laughs] You’re spending company time on your learning. Maybe you should be doing that on your own time to explore that thing.

See, I think doing these extreme things which is like, “I want to use the class everywhere, I want to find the limits, right?” That’s great. I’ve grew up on that kind of stuff like, “I’m never going to call. I’m never going to write a function. I’m going to only do methods.”

Sure, do that. Then that might not be the actual most professional way to write it. You have to recognize that.

Matt: That makes perfect sense. One thing I bring you back to the book a little bit, in the book you actually chose to use JavaScript as the language to explain functional programming. I think you do a good job explaining why.

I just want you to point out to everyone. It’s also important for our audience to know that when you’re going to be reading the book, most of you are familiar with JavaScript so this book resonates with you more than someone that’s not familiar with JavaScript.

Eric: Good question. I feel the need to emphasize that it isn’t a functional programming in JavaScript book. This a very common question that I get because they’re like, “Oh, I read this other book which is all about functional programming in JavaScript.”

I’m like, “Yeah, but it’s not what this is about.” It is a functional programming in general book language neutral, but then I needed a language to write code samples in and because I did want it to be in the code seeing refactoring exercises in code.

I had to choose a language and I chose JavaScript because it like a lingua franca. Even if you’re…Front-end programmers sure, very familiar with it. Even back-end programmers they sometimes have to go in into JavaScript. It’s got a similar enough syntax to Java and C# and C that you could read it if you needed to.

We don’t go into any super detail about like some weird feature in JavaScript. I tried to write all of the code samples not in pure idiomatic JavaScript. People comment on that all the time. They are like, “That’s not how I would write it.” I’m like, “No, that’s not the point.”

I’m using a very clear style for that’s like even a Java programmer can understand this JavaScript. As much as possible I’m trying to do that. It’s written for clarity of code. It’s not trying to teach you like, “This is the best way to do functional programming in JavaScript.”

Matt: How useful do you think functional programming is in sort of a…if you take it right into web development into like a rapid-fire web development agency that is working with a whole bunch of libraries, a whole bunch of different new technologies? They have some older clients that are on older technologies.

They’re all over the place because they have different servers. They are using Apache and Jinx, the whole bit. They’re all over the place. How useful or not useful do you think functional programming is for people that are using so many different technologies across maybe a team or two?

Eric: I think that there’s a lot that functional programming can add to help manage the complexity of all of those libraries. That’s not to say that you’re not going to have to like cheat and use some just straight up imperative code somewhere.

That first idea of identifying, “This is safe to call at any time. I don’t have to worry about side effects of this code. I can call it a hundred times and it won’t matter. I don’t have to set anything up. I don’t have to set up any global variables. This is a safe function to call.”

Versus this other function, “I have to be very careful when I call it. I have to set up systems of making sure that it doesn’t get called twice, or making sure that it gets called only after this other thing gets called.”

That first distinction that you make between code that’s safe to run at any time versus code that you have to be careful with. That I think it’s just universally applicable. It doesn’t matter if your agency is using some specific library that’s not functional legacy code. I think that is just something to be aware of all the time.

Matt: Yeah, that’s a good way to approach starting using functional programming, I think. Another thing is like if you’re working in a small team of people and you can just differentiate those two things like code that is just a calculation isn’t immutable and stuff like that versus the code that will perform changes.

If you could just differentiate those two things inside of your team, it’ll make communication a lot clearer when they take up your pay, when they take up your code and for all their Git workflow and stuff like that.

I think even there is a good place to start, if you’re not going to fully delve into a functional programming approach.

With that being said, I kind of want to point out, one of the things that I do a lot of and I’m one of those people that don’t define which approach I use at this point although that might change when I start working on larger projects with more of a team.

Right now, what I do is I use a very data first approach to programming where I spend a significant amount of time structuring my JSON responses, structuring my APIs, structuring the data that’s going to be used in the application before I start structuring my functions and my calculations and how I’m going to be using that data essentially.

How well would this approach work with something like functional programming?

Eric: Yeah, I think really well. I think that that’s a really great thing to do is spend a lot of time on your data because that’s like the new interface. You have some web endpoints that say send me a Jason like this and I’ll do stuff for you.

There’s two parts to it. There’s the URL, which is the name of the endpoint. It’s its public name, how you access it. Then there’s the JSON that gets sent to it. That’s public. You can’t change it once it goes out there. You don’t want to break backwards compatibility with any clients that are using it.

I really think that spending a significant amount of time designing your data, imagining how it’s going to be used, that’ll pay off a lot.

Matt: Cool. Regardless of the approach, right? It does work with functional programming.

Eric: Yeah, regardless and with functional programming. The whole third part is about data modeling. We’re in JavaScript, so we’re using arrays and objects basically. How do you use those to model whatever domain you’re trying to model? It’s something that I think was lost in the OO years.

When everyone was so gung-ho about OO, it was all about making an interface and putting some methods on a class and having a class hierarchy. That was what modeling was. A dog is an animal, and a cat is an animal. [laughs] Things like that.

Silly exercises that don’t really help you solve the problems of your code, but you would sit there and you’d draw these diagrams, and then deal with all the problems and if statements. I feel like the straightforward data modeling approach of saying, “What data do we need to capture?”

I didn’t define data. Data, if you look it up in the dictionary, it says it is facts about events. An even being something that happens. Just an informal event. A fact is some piece of information. You have all these events that happen.

Web requests come in, and the fact is this was the body of the request. That probably represents some other event. The user clicked this button, and it was a buy button. We’re capturing that that was clicked and that they intended to buy.

That’s all encoded in the data. How do you do that? How do you encode that in the data because this JSON blob is going to leave the browser, be sent across the world to another server, and now it’s going to have to communicate all of that to the server?

I’m trying to show how profound this is, that somehow you can encode the user’s intention in a string [laughs] basically that then has an effect and sends them a fidget spinner or whatever they buy on their site. That’s amazing that this Jason can do that, but you have to design it well. It has to be well crafted to be able to do that.

Matt: When you start thinking about it like that, it’s crazy. Bringing it back to functional programming, it’s all asynchronous as well. You don’t know how many times it’s going happen. You don’t know when it’s going to happen.

Eric: Are there retries?

Matt: You have to account for that, somehow. [laughs] It’s definitely an interesting paradigm. Technically, we don’t usually think of it in that sense. We usually think of it as, “OK, I need to wait for this to happen and then do something.”

I don’t know. It’s tough to explain in this kind of context, but it’s cool that we bring it down to earth about how crazy these systems are. With that, I want to do a precursor to what I’m about to ask. I understand that this is bad practice, what I’m going to ask right now.

I know from experience and from talking to a lot of other developers and programmers that this happens a lot where you want to invest time into doing project planning and design, gathering requirements, doing your UML diagrams. What’s the functional diagram that you were pointing out in the book?

Eric: A timeline diagram?

Matt: Timeline diagram, yup. Doing timeline diagrams for projects. A lot of the time, your management or your team will just be like, “Just write the code.” As much as you want to fight back, you don’t really have much of an arsenal to fight back when you’re just starting out as a junior developer.

What are your one or two points that you would throw back at them for getting the paid time to actually do those diagrams and plan out your code?

Eric: There’s a t-shirt that I saw recently that says, “Weeks of coding can save you hours of planning.”

[laughter]

Matt: I like that. Now I want that t-shirt.

Eric: I know. I wanted it too. It’s so true. I don’t know how many times I spent so much time coding and then realize, “Oh, if I just sat down and thought about this, I would know that it was a dumb idea.” [sighs]

I don’t know how to convince someone that thinking is actually work in our line of work, that that is the most valuable thing you can do. I do think that there’s something to be said for prototyping, like let me do some experiments with some code.

Too often we don’t really have a good, clear statement of what problem we’re trying to solve, so we don’t even know where we’re going to wind up [laughs] if we start solving it. We don’t know what we’re doing. We simply don’t spend enough time thinking about why we’re doing it. Is there a better way to do it? Let me think of a few ways. I don’t know.

I know the feeling of being rushed and how, “Oh, I could solve that,” and just type it up. I do. I get it out the door and it feels good. It’s the thing that I often later regret. Now, the thing is also, there are a lot of obvious problems where I’ve solved this before. I’ve solved this 100 times. I know the way to solve it. Those come with time.

Also, in a startup, you often make those tradeoffs like I’d rather have the wrong feature out today, then the right feature out in two weeks. In that perspective, thinking is indistinguishable from procrastination.

The trouble with recommending that someone thinks more is you’re not giving them a concrete process to go through. You do learn that over time. I do think we need more step-by-step processes to follow in our industry to arrive at solutions.

Matt: I guess it’d be more presenting it to the management staff. If you’re trying to convince your management, “Hey, I want more time to plan. I want more time to think about this rather than just jump right in,” are there any tools that can easily… ?

Maybe they be organizational tools like a Trello or something that where you can more easily line out, “This is why I want to think about this,” and hand that to maybe a non-technical staff member, or someone who’s maybe a little less tech-savvy to try to convince them, “Please give me the extra three hours so I can think about this”?

Eric: I wouldn’t ask for permission.

Matt: [laughs] Fair enough.

Eric: It’s not because they’re not going to give it to you and you should just do it anyway. It’s more because as a professional, you have more of an understanding of the process and what needs to be done.

When you ask for permission like that, often what I find is they say no, but they’re not saying don’t do your job correctly. What they’re saying is, “No, I’m not going to do your job of deciding to do that or not, to think or not.”

What I would do is if it’s something big, like a bigger project, I would write it up as a proposal. I would shop it around in the team, ask for comments on it, turned it into an artifact that can be shared, write it up in English, and spread it around.

I’ve often found that when a manager, the non-technical person who’s asking for a feature, let’s say, when they give me the feature, they often think this should be really easy, “I’ve thought a lot about it. It should be easy.” Then ask questions. Show them that it’s not easy. It’s not complete.

What is the error message when this times out? What happens if they type a long email address that’s more than 20 characters like 10 percent of our users have? How does that get displayed in this tiny box that you’ve drawn on this picture?

Really push at it because I think that they don’t see the complexity. You’re going to do it. You’re going to do your best to do what they asked for, and it’s going to be wrong according to what their idea was. I really think that asking questions is a good tool. Something that’s maybe more internal, I would shop it around in a written form.

Matt: Sometimes you have to take charge of your own project and make sure that you do your own job rather than throwing that to management being like, “Can I please…?” [laughs] “May I please do this to do this correctly?”

Eric: I feel the same about code clean-up, people talking about, “Oh, can we have like a sprint to clean up our code?” No, don’t ask. Just do the cleanup. If they say, “Oh, can we get this feature done?” It’s like, “We’ll put it on the backlog,” just like you would say about anything if you are busy with something else.

In some sense, it’s not their decision to make of what operations in the code you’re doing are. They shouldn’t have to be managing that part of the situation.

Matt: That’s a good way to put it. I think that that’s a good way. A lot of employees, these days especially, will be more timid, if you will, especially if they are junior employees or the brand new, they’ll be more timid.

You just have to make sure you do your job properly. At the end of the day, if you hand whatever it is in and it’s not done correctly, whether it be assignment or an actual production thing that’s not going to be just for school or for learning, it has to be done correctly. You have to do your own due diligence.

With that being said, I’m going to transition nicely into the next question here, which is, in the first pages of the book, you lay out a scenario where a developer ask a manager if they can use functional programming on an email app, the manager looks up the definition and sees that “avoids side effects,” and that sending an email would be a side effect.

With that being said, the manager then says that functional programming is too risky to use on the project. If a programmer needed to use a variable, a function, or even recursion on performing part of their duties, they wouldn’t have to ask the manager for each step of the way. They would just implement the solution their way.

This leads into a couple of questions here. With that being said, what is so different about functional programming that would warrant asking permission? Second one here, is this a situation where the entire team and not just one programmer must adopt functional programming for it to be effective?

Eric: That’s a really good question. To give some context for that part of the book, my intention — and whether I did a good job with my intention, it’s another matter — is simply to have a little narrative for why there’s a miscommunication with management and that functional programming definition you find on Wikipedia.

It wasn’t really like this is a common scenario where someone’s asking permission to these functional programming. That said, I do think that in certain companies, they might have a code style guideline. They would have some rule in their style guide that says we use classes not functions. Something like that. I’m just making it up.

They would have some rule where it’s harder to do functional programming. You are going up against the team. Maybe you need someone higher up to allow that exception to the rule. That’s what I was thinking.

Does that mean that you need the whole team to buy in? I think they do need to buy in, but I’m going to preface that by saying that, like I was saying before, these are principles that work in any paradigm.

They’re skills that are used by functional programmers, but it doesn’t mean you have to be like all in on functional programming. They should see the benefit like you should be able to say, “Look, if we just make this a pure function, it would be much easier to test.”

If you know your team needs to test stuff like, “Oh, yeah, you’re right. That is easier to test.” It should be as clear as that. Buy in, yes, because we work in teams. You can’t just be a renegade and just do whatever you want in the code without regard to the style of the code and how you practice coding on a team.

You do need buy in, but like I said before, it doesn’t mean like everything has to be functional from now on and you have to go 100 percent on everything.

Matt: Yeah. That makes some good sense because I would be worried as a junior guy even, if you come in and you’re all about functional programming. That’s what you’ve been learning and then some of the senior guys are just kind of ripping through their own code.

You’re just like, “Ah, I don’t really want to go and bug all them,” and be like, “Hey guys, let’s change this whole project.” That’s like…

[crosstalk]

Eric: Yeah. A monad would be really great right about here.

[laughter]

Eric: Just to be clear, there are a lot of, let me call them, advanced functional techniques that we’re not even going to get into in the book. Functional programming is a huge field. We can’t do it all in one book.

A lot of times they would be very unfamiliar. If you just saw it right in the middle of the code, you’d be like, “What is that thing doing?” I’m not suggesting doing that. I’m suggesting these more fundamental ideas, like the ones we go over in the book or just good ideas in general. They should be easy to convince people of.

Matt: Next question here then, it’s mentioned that functional programming is more useful in more complicated applications, like in something like a distributed system. Is there any benefit for a beginner programmer in trying to use functional programming in the simple applications that they’re writing or learning on?

Eric: Is there any benefit? Yes. These are principles and skills that they’re design and architecture ideas that really pay off more at scale. The benefit is there, but it’s very small. If you got a small program, any old style, any old discipline, no discipline is fine. Just get it working.

As the program grows, you’re going to find that you’re slowing down, it’s turning into a mess, you’re getting lost, like, “What does this do anymore?” You’re starting to get bugs that you can’t identify.

That’s where disciplines help. There’s object-oriented disciplines that could help, there’s functional programming disciplines that could help. That’s my answer. There’s a benefit, but it’s small.

Matt: I think we’ve touched on this a bit already as well, but would you say that someone who’s maybe they’ve mastered the stage of program that they’re at, so when they go and approach, so they’re not just learning the syntax.

They’re at the stage of when they go and approach another problem, and it’s still a simple problem, they’re able to just do it. They’re able to think it up in their head. They know what’s going on, generally.

Would you say that at that stage, even though it’s simple, would you say that that’s when they should maybe start learning some of these paradigms like functional programming?

Or, should that still be left to where they’re given a huge project and they’re like, “Wow, I don’t know how to do this now I got to look up these paradigms to sort of organize this big project”? Or, is that too much all at one time?

Eric: I would say that bringing experience of getting into a mess [laughs] would help. It definitely helps because then you’re like, “I wish I had some skill to get me through this or to avoid it.” I think that it doesn’t take long before an undisciplined approach…

I’m talking about a few hundred lines of code you start to get lost, and like, “Oh man, that function’s really big. Maybe I need to break it up a little bit, but I don’t know how.” It really starts there. To give advice for someone who’s still struggling to just get something working, don’t worry about a paradigm. Just code.

When you start to feel like, “Wow, there must be a better way. Is there some discipline that I could have?” That’s when you should start looking into other code styles, paradigms, things like that. Little rules, disciplines that help you make sense of what’s going on.

There’s a lot going on there. People have come up with quite a lot of good guidance and, I want to call it wisdom, over time of, like the three categories. Actions, calculations, and data. Just identifying those and saying, “Wow, I’ve got quite a lot of code and actions. Maybe I should start removing some of that code.”

If you’re in an object-oriented approach, and like, “Oh, this would make a really nice…I’ve got a lot of conditionals in here. Maybe I should encapsulate the differences and use polymorphism.” You have the terms that come from the paradigms. They can start helping you solve these problems.

Mike: I’m just thinking, I want to bring it all together for the audience. This is an impromptu suggestion. You could shoot me down, both of you. What I’m thinking is maybe we should go through an example functional programming, just a quick functional programming example, and maybe do that breakdown that you were just mentioning with the actions, data, and calculations.

One of the examples that I was thinking of off the top of my head was one that you used in the book, which would be really simple to explain, which is the pizza ordering, the online pizza ordering application. What do you guys think of that right now?

Matt: Sure.

Eric: Yeah, sure. Let’s do it.

Mike: Let’s do it. For that, I’m just going to break down what an online pizza ordering application would do, then we can classify the steps, just quickly. We don’t have to do every single step, obviously, but classify the steps that would be considered actions, would be considered calculations, would be considered data. Which parts of the steps would be considered each?

A pizza ordering application would take an order from a user. That user could enter his information, and he would order the pizza that he wants, depending on which toppings he wants and stuff like that. He would then place that order, having already sent all the information in.

Then the process of actually creating the pizza will start happening. You would have many different steps in that process for putting the cheese on, spreading the sauce, putting all the toppings that the user defined. Then putting it in the oven, waiting for it to cook in the oven, taking it out, putting it in a box, putting it in a delivery capsule or whatever, and sending it to that user’s house.

That’s just a one user example. There’s billing in there and stuff like that, but I’m thinking at the start, the user entering his data. How would we break that down?

Eric: Let’s say they have a web page that they can enter this stuff on. The web page is loaded and there’s form fields that they’re filling out. All of that entering, the typing, the moussing around, that’s generating events. That’s going on to the event queue, which goes into the event loop. This is all actions.

However, the little piece of data that’s sitting on the queue, that’s just data. It’s already happened. It’s in the past already. By the time it gets processed, that mouse click is old. It just happens so fast. We don’t think about that, but it’s old. It’s already happened. That’s just some inert data that’s sitting there.

You might do some validation. That validation on the data in the form before you submit to the server, that could be a pure function. That could be a calculation. It’s just like true, false, is this good? Do they complete the order correctly? It could maybe return a string, the error message or something if it’s not good. That’s certainly a good candidate for a pure function for a calculation.

Sending it to the server, that’s an action, because it’s having an effect on the world. Presumably, if you didn’t send it, you wouldn’t get the pizza. The zero or one time that it gets sent is important.

That’s a rule of thumb for an action if it depends on when it’s run or how many times it’s run. When it’s run, it’s often about the order. It’s relative to other things happening. We’ll probably hit more examples of that.

Then the server receives this request. That’s an action as well, but the request, itself, is data. It’s just some JSON that’s traveling over the wire. Then probably some validation. Again, that could probably be a calculation. This gets sent to the pizza kitchen, I guess is the next step. Is that right? Next step is pizza kitchen?

Mike: Yup.

Eric: Pizza kitchen is getting this data, represents the order, one pepperoni pizza. There’s some steps involved. There is a recipe that they’re going to follow. The recipe is data. Maybe there’s two pepperoni pizzas, so they have to multiply all the ingredients by two. That’s a calculation.

Another rule of thumb that I like to say about calculations is they’re often very hidden because you don’t see them happening. Because they’re not having an effect on the world, you couldn’t see them. It’s like what you’re thinking in your head like, “I’m planning. Oh, two pizzas. I’m going to need twice as much pepperoni.”

You’re doing all the calculation in your head. It’s not having an effect on the world. Until you actually go get the pepperoni, you’re not having an effect. If you think of calculations as all the hidden planning and thinking that goes on between actions, that’s a good way to think about it.

Then there’s all the steps of rolling out the dough and putting the toppings on, putting it in the oven. That’s all actions. What was the next step? The oven?

Mike: The oven.

Eric: You put it in the oven. You just wait 10 minutes or however long, an action. You don’t want to wait 10 minutes at the wrong time because you’ll burn your pizza. That means it’s an action. Take it out, put it in a box, ship it over to the person.

Probably, there’s going to be a ticket on the box that has their address, data again. There’s got to be calculations in there. For instance, you’re taking that address and turning it into a GPS location with directions to get there.

That could probably be a calculation, although it is more complicated because it’s usually a cloud thing. You’re communicating with another server that’s doing that calculation. For the sake of argument, let’s say they don’t have a cloud GPS thing. They’re just getting directions on their Garmin or whatever.

Mike: Even in a JavaScript sense, it could be a calculation, because when you’re communicating with the cloud, that can be done in a separate calculation function that would always return the same thing every time no matter what with the same input. It still would be a calculation even though it’s communicating with a cloud.

Eric: I would tend to agree. The only complication is sometimes it could time out.

Mike: Oh, because it’s not your server.

Eric: Right. Whenever you communicate across the network, there’s a possibility of having a 500 error or something that you would never get from multiplying two numbers together.

Mike: Cool.

Eric: Yes. Other than that, it is cacheable. Like you were saying, it’s going to give the same thing every time. This is getting deep into the functional programming stuff, but because it’s cacheable, what you’re saying is that it doesn’t so much have an effect on the world as it’s bringing in information. It’s going to bring in the same information every time.

That means zero request is different from one request, but then that second request, I don’t need to do because I cached it. It’s item potent. Item potent means zero or one time makes a difference. Classic example of item potent is the elevator button. You press up to call the elevator. It lights up. If you press it again, it’s not going to have any effect.

The difference between not pressing and pressing it is important, but that second press, the third press, those don’t matter. It’s the same with calling to the mapping service. The first one matters, but then after that, it doesn’t matter.

This is the kind of reasoning you can start to do with these concepts of time and whether it matters or not. Then they arrive at the house. They give the guy the pizza. Eating it is definitely an action.

Mike: [laughs] It’s the best action.

Eric: Yeah. [laughs]

Mike: I like this example. I wanted to bring it into play because I just wanted the people out there to understand a little bit about functional programming. Even though it might be a little bit over their head or whatever, but it’s important to know that there is something like this out there, a structure to what they’re doing.

It’s OK if it’s over their head. They can, A, either get your book and read it or, B, just look at it yourself, go and watch some videos and study it, something you would probably do in a university class. We have some high school people listening to our podcast. It’s important for them to know that this is a future thing that they might be looking at.

There’s definitely a lot of little side questions. As you were talking about that, I was going to interject and try to get some clarification on little, very intricate things in functional programming which is cool to me.

Again, I didn’t want to bog down the users, but maybe in the future, if people are interested, let us know. We can have you back on, and we can have a more in-depth functional programming discussion…

Eric: Yeah, I’d love that.

Mike: …where we talk about the little intricacies of functional programming…

Eric: If you have questions, bring them in.

Mike: …which would be cool for me.

Matt: Moving on to the question there, it flows nicely because we’re talking about a complex application now. If you’re already in some app development team, or something like that, and you have an existing complex application that was not at all developed with functional programming in mind, is there any way that it can transition to functional programming moving forward?

Would it be better if you maybe rewrote the whole application under the guideline of functional programming?

Eric: My big hope is that the book shows that you don’t have to do a big rewrite, that you can get benefits in an incremental way because I can’t recommend that someone rewrites their application.

Matt: Right. Yeah.

[laughter]

Eric: It’s too much work to do all at once. It’s probably going to kill the business to do that. In fact, the chapter I’m working on now is all about immutability and one of the ways to implement immutability is called copy-on-read.

It’s a discipline that if you call a function, and you get a value out — let’s say it’s on JSON, it’s some object — you instantly, right away, immediately make a deep copy of that. This is your copy because you don’t know what this code that you called is going to do with that object that it gave you.

It’s one of the things in JavaScript and a lot of other languages where you have these immutable things. It’s not clear if you own the thing, if it’s a newly created thing that only you have, or they’re going to pass the same object to some other code. You just don’t know what’s happening to this object.

It’s like if someone gives you a message on a piece of paper, or you write down a message to yourself, you put it in your pocket. Then later in the day, you pull it out of your pocket. You don’t expect that it has changed. You expect that it is the same message, that nothing touched it.

In the computer world, we have this thing where you can share references to the same object, and you can’t expect that it’s the same when you look at it the second time. To mitigate that you do copy-on-read. “I’m going to make my own copy. You can keep that or just let it get garbage collected, or whoever else has a reference can keep it. I’m going to make my own copy.”

When you do that, that lets you interact with all your legacy code, because they don’t have to be functional. All that code that I’m talking to, it can do whatever it wants. It can mutate as much as it wants. It can change the memory, change the values. I have my copy and I can now treat it as an immutable thing and be functional.

Yes, it’s a thing. I think about it a lot. How do you slowly, gradually introduce these functional ideas without going whole hog on an existing codebase? It’s really important.

Matt: I think that’s a really great wrap up to that question because it’s one of those things where a manager or management team would ask. “You want to introduce functional programming now? We’re two years in, we’ve been updating this whatever app for this many years. Now you want to change the paradigm we’re using? What’s that going to mean? How risky is that?”

Eric: Right. What I hope is, like you were saying, you wouldn’t even have to bring it up to the manager. Let’s say you’re pairing on this code and someone’s like, “Oh, man, they’re passing me these objects, but I know they’re going to reuse this object and change it and pass it to someone else.”

You just know right away. “Yeah, copy-on-read. That’s how we do it.” And then it’s not a problem anymore. It’s a little local solution to the problem that you’ve got in that code. You don’t have the budget to go rewrite that whole thing, to make sure it doesn’t make changes to that object. No, just a local solution.

I feel like the functional programming skills I’m teaching in the book should be able to be used that way, like we have this one problem, we’re just going to use it right here.

Matt: With that being said too, even as early in the book, again, if we take our example and your example combined with the pizza timeline diagram, within a couple of pages of each other from the beginning of the example through the end, there are already variations where the person that owns the pizzeria, whatever she does, she edits what’s going on there.

That’s an attestment to she didn’t have to tear down her own pizzeria and rebuild the whole thing. She can just go in and be like, “No, no, no, let’s put a dotted line here. Let’s move these around. Let’s take this, take these procedures and make them a different order,” etc., etc., etc.

That’s an attestment to how functional, [laughs] if you will, functional programming can be in an already thriving production environment.

Eric: Yeah, I hope so. I hope that people look at them like refactorings. It’s just ways to reorganize little pieces of your code locally. You don’t have to change everything, just like, “Oh, this code is getting messy,” or, “We’re losing control over what it’s doing. Let’s apply a little functional programming to it and make it better.”

Matt: Awesome. The final question in the segment here is, how much time is spent planning when using functional programming versus nonfunctional programming in your experience?

Eric: I don’t think that there’s any real difference. I think that planning is important for everything. For all coding, it’s equally important, let’s say, between all paradigms. It’s very important.

Matt: I think that would be one thing that our users or listeners, especially as beginners, would, if they start thinking in less terms of, “Oh, I have to do this application. I’m just going to start coding it.”

If they start thinking in these paradigms, that’s one of the questions they’re going to think up, like, “Whoa, I gotta spend a day planning what’s going on here.” In reality when you have a complex application, whether you do object-oriented or whatever, or functional, you do need to do a fair bit of planning ahead of time. that’s important for beginners to acknowledge,

Eric: Sure. How do I put this? As an experienced programmer, I’ve been programming…I’m 38 years old. I’ve been programming a while. I don’t want to count how many years [laughs] I’ve been programming.

Matt: [laughs]

Eric: I sit down and I see code. I can be like, “That’s a code smell.” If we’re looking for a bug, let’s say, I’m like, “We should look there. We should look there. We should look there.” It takes me a minute to see it.

A junior programmer sitting next to me, it will take me hours to explain why. I’ve done this before where I’m like, “Well, you see, what’s happening here is where let’s trace through the code and you see how this thing is calling this and then it modifies them and something else is modifying it at the same…” You see this instantly.

What I’m trying to get at is I don’t think any amount of planning can get you to see it faster. Do you know what I mean? You can’t just say I’m going to plan away all the problems.

Matt: [laughs] Right.

Eric: As a junior, you’re going to make code that does those things. There’s code I still make problems too. I have a bag of tricks that I can use to plan. What I’m saying is they don’t have the bag of tricks. Maybe planning is not as valuable when you’re starting out.

Matt: That’s a really good way to put it. As you were explaining it there, the first thing that popped into my mind was I’m from an IT background. One of the first things popped to my mind was when you bring the trainee on board, the beginner, you teach them, “This is how this is how we format hard drives. This is how we install graphics cards. This is how we do whatever.”

As you get more experience as you go on, when you read a ticket, just reading the ticket, sight and seeing computer is, you haven’t looked at that person’s laptop or their desktop. You’re just like, “It’s probably the hard drive. It’s probably the CD drive.” To them, they’re like, “How would you know that?” Then you have to explain, “Well, it’s doing a ticking.”

Eric: You smell and hear it. [laughs]

Matt: Yeah, exactly.

[laughter]

Matt: It’s one of those things where it’s almost like a muscle memory you develop with that bag of tricks like you were saying.

Mike, I know you were mentioning something…We usually do the web news here, but you were mentioning something how we think we’ve cover more or less the questions in here.

Mike: We’re approaching an hour, 40 at this point. I’m thinking the web news might be a little bit extra. We’ve had a lot of good discussions. I don’t know, Eric, how you’re feeling. If you’re up for the task or not, but I’m feeling like it’s probably already pretty late where you are. I don’t want to put any pressure on you to do any extra talking…

[crosstalk]

Eric: If I could step out that would be good.

Matt: OK, cool.

Eric: I feel like…

Matt: Well…Sorry…

[crosstalk]

Eric: I feel like I’ve laid down some good stuff.

Mike: Oh, yeah.

Matt: We want to thank you again for being on the show. We’d like to invite you at this time to share what you’re up to. We know we were talking about your book, but anything you’re up to. Any links or anything you’d like us to include in the show notes. You have the floor. Please, take it away.

Eric: I have a podcast if you’re interested in this kind of topic. It’s called “Thoughts on Functional Programming.” It started as me exploring the ideas for the book before I started writing it. It was really just me with my phone driving or walking around. Little moments between driving somewhere to pick up my kid from school. I would just turn on the camera and talk.

It had very informal style. Now I do a little bit more planning and better audio than you can get in the car, let’s say. It’s at lispcast.com/podcast. I got, man, 140 something episodes now. If you’re interested in this, go check it out.

If you’re interested in the book, you can go to lispcast.com/gs, for grokking simplicity. That will take you to the page on the Manning site. It’s in early access now. First three chapters are available. Soon to be the fourth. Maybe even by the time this is published, it’ll be out.

When you do early access, you get the eBook right away, but then you can…If you get the print version, it’ll be delivered to you when it’s done. You can still get the print version with the [inaudible 101:47] . That’s it.

Matt: Awesome. All right. Thank you very much once again for being on the show. It was a really informational episode. We hope you enjoyed it. Thank you for listening. Make sure you don’t miss an episode by subscribing on the platform of your choice.

You could follow us on the socials via @htmlallthethings, that’s on Facebook and Instagram. You could follow us on Twitter @htmleverything. We are on Medium and we’re on GitHub. Remember we’re on Patreon. That’s patreon.com/htmlallthethings. Check out the tiers and give that a go.

Many thanks to our three-dollar-tier patrons. Sean from RabbitWerks JavaScript. You could find him at YouTube.com/rabbitwerksjavascript. Also, Garrick from Local Path Computing & Web Design. You could find him at localpathcomputing.com. Craig aka Cosworth.

Ryan Gatchel from Blue Black Digital. Find him at blueblackdigital.com. Chris from Self Made Web Designer. You could find him at selfmadewebdesigner.com. And as well Tim from The Web Hacker. You could find him at thewebhacker.com.

Remember we have that promo code or the discount code, and two full book promo codes. You can get a free book, single use. They do expire as I mentioned in the beginning.

Make sure you check out the full show notes which are usually posted shortly after, a few hours after the actual episode goes live. They’re published on the HTML All The Things website. That’s htmlallthethings.com.

[background music]

Matt: Feel free to leave a comment or a review on the platform that you are listening to this on. We are signing off.

The post Grokking Simplicity – HTML All The Things [Podcast] appeared first on LispCast.

Permalink

ClojureScript Macros

ClojureScript Macros are a hurdle for most CLJS beginners and wrapping your head around how they work can be quite confusing. I’ll try to cover the basics you need to know to start writing your own macros.

First of all - ClojureScript macros are written in Clojure and run during the ClojureScript compilation on the JVM. A macro is just a function that takes code (as simple data) and generates other code (again simple data) out of it. The CLJS compiler will then turn the result into JS. This is done in Clojure so that the generated JS does not need a full ClojureScript compiler at runtime.

Self-hosted CLJS is capable of compiling CLJS at runtime and does support macros but requires at least 2MB of JS so it isn’t practical for most build targets. This post will not cover anything self-host related, as that is a topic for advanced users.

Step By Step

1. Create a CLJ namespace, eg. my.util in src/main/my/util.clj (assuming src/main is one of the :source-paths). Note that this is a .clj file to create a Clojure namespace, not ClojureScript. Define the foo macro as you’d normally do in Clojure.

    (ns my.util)
   
    (defmacro foo [& body]
      ...)
   

2. Create a CLJS namespace of the same name, so src/main/my/util.cljs and add a :require-macros for itself in the ns form.

    (ns my.util
      (:require-macros [my.util]))
   

3. Use it.

    (ns my.app
      (:require [my.util :as util]))
   
    (util/foo 1 2 3)
    ;; :refer (foo) and (foo 1 2 3) would also work
   

Done.

Steps Explained

To explain why we do those steps we best go in reverse.

• When the CLJS compiler processes the (util/foo 1 2 3) call it expands the util alias to its fully qualified (my.util/foo 1 2 3) form

• The compiler looks up my.util/foo. Typically the compiler would only look for that var in the ClojureScript environment (so what was defined in the .cljs files). The my.util namespace however had the :require-macros for itself which tells the compiler to also look for macros of the same name

• The my.util/foo CLJ macro is found and the compiler will expand the form using that macro

• The CLJS compiler continues with the expanded form

The Old Way

Before CLJS-948 macros required a bit more ceremony. Thus you still see these patterns in older code. Nowadays all macros should be written as described above.

In the above example we are using the macro self-require directive (ie. :require-macros) to inform the CLJS compiler about macros that supplement a given CLJS namespace. This requires that there actually is a matching CLJS namespace, but in the past the self-require trick didn’t exist so macros sort of existed on their own.

It was common to create dedicated .macros namespaces (eg. as seen in cljs.core.async.macros). core.async provides a go macro should you can nowadays access in two ways.

The old way

(ns my.app
  (:require-macros [cljs.core.async.macros :refer (go)])
  (:require [cljs.core.async :as async :refer (chan)]))

(go :foo)

or the modern way

(ns my.app
  (:require [cljs.core.async :as async :refer (chan go)]))

(go :foo)

The problem with the old way was that the consumer of a library had to have special knowledge about macros and how to consume them. In the example above the user had to know that chan was a regular var and that go was a macro. In the modern way the compiler can figure this out on its own and all it took was a matching namespace with the :require-macros self-require trick. If a regular CLJS “var” has a defmacro of the same name in CLJ it will expand the macro first if applicable.

Macro Limitations

Macros can pretty much do everything that regular CLJ macros can do but since you are basically dealing with 2 separate versions of the same namespace there are some things to watch out for.

Gotcha #1: Namespace Aliases and Dependencies

Since the macros run in CLJ and not CLJS the namespace aliases you configured in CLJS will not work in the macro. It is recommended to use fully qualified names if you need to access code from other namespaces. Defining the :require in CLJS ensures that the clojure.string code will actually be available at runtime. If only the CLJ variant had that :require the CLJS compiler might not provide that namespace.

;; CLJS
(ns my.app
  (:require [my.util :refer (foo)]))

(foo :hello "world")

;; CLJS 
(ns my.util
  (:require-macros [my.util])
  (:require [clojure.string :as str]))

;; CLJ
(ns my.util)

;; this would fail since the CLJ namespace doesn't know about the str alias
(defmacro foo [key value]
  `{:key ~key
    :value (str/upper-case ~value)})

;; so instead use the fully qualified name
(defmacro foo [key value]
  `{:key ~key
    :value (clojure.string/upper-case ~value)})

There may be cases where there are actually matching CLJ namespace that you could require in the CLJ variant but if you want to be safe just use the fully qualified name.

Gotcha #2: Caching and :parallel-build

The ClojureScript compiler might be caching the result of the compilation so you should try to avoid side-effects in macros at all cost. The compiler might also be using multiple threads for compilation so if you use side-effects that rely on specific ordering things might get messy.

If you cannot avoid side-effects make sure to turn off caching and probably :parallel-build.

Gotcha #3: Macro-Support code

The CLJ “macro” namespace is just a regular Clojure namespace. You can use all the code you want in it during macro expansion but if your macro expands to code that needs to be called at runtime that code must be defined in the CLJS variant instead.

;; CLJS
(ns my.util
  (:require-macros [my.util])
  (:require [clojure.string :as str]))

(defn my-helper [m] ...)

;; CLJ
(ns my.util)

(defmacro foo [key value]
  `(my-helper {:key ~key
               :value (clojure.string/upper-case value)}))

This will expand to (my.util/my-helper ...) which the CLJS compiler will find properly. CLJS code cannot access a defn defined in the CLJ namespace so make sure everything is where it needs to be. You can use a fully qualified symbol for my-helper but the syntax quote (ie. the ` backtick) automatically applies the current namespace (ie. my.util) to all unqualified symbols so it can be omitted here.

Gotcha #4: CLJ Macros

So far this post assumed that the macro only had to generate CLJS code and didn’t need to work in regular CLJ code. Some libraries however will want to work on both platforms so they need a way to detect whether they are supposed to expand to CLJS code or CLJ code. This can be done by inspecting the special &env variable during macro expansion. Note that you cannot do this with reader conditionals when using .cljc files.

(ns my.util)

(defmacro foo [key value]
  (if (:ns &env) ;; :ns only exists in CLJS
   `(cljs-variant ...)
   `(clj-variant ...))

Gotcha #5: CLJC

CLJC can be difficult to get right since you are defining two completely independent namespaces (CLJ+CLJS) at the same time in one file.

I strongly recommend writing your macros in 2 separate files until you feel comfortable with that. I still do it for 100% of my macros.

A lot of the macros in the wild out there written in CLJC do it wrong in some way. Most of the time this isn’t a big problem since the Closure Compiler gets rid of the code that “leaked” in the CLJS side of things. Nevertheless if you must use CLJC files you should make sure that all CLJ related code is properly behind #?(:clj ...) conditionals.

(ns my.util
  #?(:cljs (:require-macros [my.util])))

(defn my-helper [m] ...)

(defn macro-helper [k v] ...)

(defmacro foo [key value]
  `(my-helper ~(macro-helper key value))

It is somewhat easy to overlook that in this example macro-helper is actually called during macro expansion and not at runtime. The macro-helper defn however was not behind a reader conditional so it will be compiled as part of the CLJS variant and actually exist as a regular function at runtime. That doesn’t necessarily hurt anyone but depending on how much code is “leaked” that way may make compilation slower or may lead to actual additional bytes in a release build if the Closure Compiler was not able to eliminate all of it (eg. if you used defmethod for CLJ multi-method).

Summary

• ClojureScript macros are written in Clojure and run on the JVM
• Define macros by creating a CLJ and CLJS namespace of the same name, where the CLJS variant has a :require-macros directive in its ns
• CLJC is harder to get right, avoid it if you can
• Writing and using macros isn’t all that difficult ;)

Permalink

Integrating Deep Learning With clojure.spec

clojure.spec allows you to write specifications for data and use them for validation. It also provides a generative aspect that allows for robust testing as well as an additional way to understand your data through manual inspection. The dual nature of validation and generation is a natural fit for deep learning models that consist of paired discriminator/generator models.

TLDR: In this post we show that you can leverage the dual nature of clojure.spec’s validator/generator to incorporate a deep learning model’s classifier/generator.

A common use of clojure.spec is at the boundaries to validate that incoming data is indeed in the expected form. Again, this is boundary is a fitting place to integrate models for the deep learning paradigm and our traditional software code.

Before we get into the deep learning side of things, let’s take a quick refresher on how to use clojure.spec.

quick view of clojure.spec

To create a simple spec for keywords that are cat sounds, we can use s/def.

1

(s/def ::cat-sounds #{:meow :purr :hiss})

To do the validation, you can use the s/valid? function.

1
2

(s/valid? ::cat-sounds :meow) ;=> true
(s/valid? ::cat-sounds :bark) ;=> false

For the generation side of things, we can turn the spec into generator and sample it.

1
2

(gen/sample (s/gen ::cat-sounds))
;=>(:hiss :hiss :hiss :meow :meow :purr :hiss :meow :meow :meow)

There is the ability to compose specs by adding them together with s/and.

1
2
3

(s/def ::even-number (s/and int? even?))
(gen/sample (s/gen ::even-number))
;=> (0 0 -2 2 0 10 -4 8 6 8)

We can also control the generation by creating a custom generator using s/with-gen. In the following the spec is only that the data be a general string, but using the custom generator, we can restrict the output to only be a certain set of example cat names.

1
2
3
4
5
6
7
8
9

(s/def ::cat-name
  (s/with-gen
    string?
    #(s/gen #{"Suki" "Bill" "Patches" "Sunshine"})))

(s/valid? ::cat-name "Peaches") ;=> true
(gen/sample (s/gen ::cat-name))
;; ("Patches" "Sunshine" "Sunshine" "Suki" "Suki" "Sunshine"
;;  "Suki" "Patches" "Sunshine" "Suki")

For further information on clojure.spec, I whole-heartedly recommend the spec Guide. But, now with a basic overview of spec, we can move on to creating specs for our Deep Learning models.

Creating specs for Deep Learning Models

In previous posts, we covered making simple autoencoders for handwritten digits.

Then, we made models that would:

• Take an image of a digit and give you back the string value (ex: “2”) – post
• Take a string number value and give you back a digit image. – post

We will use both of the models to make a spec with a custom generator.

Note: For the sake of simplicity, some of the supporting code is left out. But if you want to see the whole code, it is on github)

With the help of the trained discriminator model, we can make a function that takes in an image and returns the number string value.

1
2
3
4
5
6
7
8

(defn discriminate [image]
  (-> (m/forward discriminator-model {:data [image]})
      (m/outputs)
      (ffirst)
      (ndarray/argmax-channel)
      (ndarray/->vec)
      (first)
      (int)))

Let’s test it out with a test-image:

1

(discriminate my-test-image) ;=> 6

Likewise, with the trained generator model, we can make a function that takes a string number and returns the corresponding image.

1
2
3
4

(defn generate [label]
  (-> (m/forward generator-model {:data [(ndarray/array [label] [batch-size])]})
      (m/outputs)
      (ffirst)))

Giving it a test drive as well:

1
2
3
4

(def generated-test-image (generate 3))
(viz/im-sav {:title "generated-image"
             :output-path "results/"
           :x (ndarray/reshape generated-test-image [batch-size 1 28 28])})

Great! Let’s go ahead and start writing specs. First let’s make a quick spec to describe a MNIST number – which is a single digit between 0 and 9.

1
2
3
4
5

(s/def ::mnist-number (s/and int? #(<= 0 % 9)))
(s/valid? ::mnist-number 3) ;=> true
(s/valid? ::mnist-number 11) ;=> false
(gen/sample (s/gen ::mnist-number))
;=> (0 1 0 3 5 3 7 5 0 1)

We now have both parts to validate and generate and can create a spec for it.

1
2
3
4
5
6

(s/def ::mnist-image
    (s/with-gen
      #(s/valid? ::mnist-number (discriminate %))
      #(gen/fmap (fn [n]
                   (do (ndarray/copy (generate n))))
                 (s/gen ::mnist-number))))

The ::mnist-number spec is used for the validation after the discriminate model is used. On the generator side, we use the generator for the ::mnist-number spec and feed that into the deep learning generator model to get sample images.

We have a test function that will help us test out this new spec, called test-model-spec. It will return a map with the following form:

1
2
3
4

{:spec name-of-the-spec
 :valid? whether or not the `s/valid?` called on the test value is true or not
 :sample-values This calls the discriminator model on the generated values
 }

It will also write an image of all the sample images to a file named sample-spec-name

Let’s try it on our test image:

1
2
3
4
5
6
7

(s/valid? ::mnist-image my-test-image) ;=> true


(test-model-spec ::mnist-image my-test-image)
;; {:spec "mnist-image"
;;  :valid? true
;;  :sample-values [0 0 0 1 3 1 0 2 7 3]}

Pretty cool!

Let’s do some more specs. But first, our spec is going to be a bit repetitive, so we’ll make a quick macro to make things easier.

1
2
3
4
5
6
7

(defmacro def-model-spec [spec-key spec discriminate-fn generate-fn]
    `(s/def ~spec-key
       (s/with-gen
         #(s/valid? ~spec (~discriminate-fn %))
         #(gen/fmap (fn [n#]
                      (do (ndarray/copy (~generate-fn n#))))
                    (s/gen ~spec)))))

More Specs – More Fun

This time let’s define an even mnist image spec

1
2
3
4
5
6
7
8
9
10

 (def-model-spec ::even-mnist-image
    (s/and ::mnist-number even?)
    discriminate
    generate)

  (test-model-spec ::even-mnist-image my-test-image)

  ;; {:spec "even-mnist-image"
  ;;  :valid? true
  ;;  :sample-values [0 0 2 0 8 2 2 2 0 0]}

And Odds

1
2
3
4
5
6
7
8
9
10

  (def-model-spec ::odd-mnist-image
    (s/and ::mnist-number odd?)
    discriminate
    generate)

  (test-model-spec ::odd-mnist-image my-test-image)

  ;; {:spec "odd-mnist-image"
  ;;  :valid? false
  ;;  :sample-values [5 1 5 1 3 3 3 1 1 1]}

Finally, let’s do Odds that are over 2!

1
2
3
4
5
6
7
8
9
10

  (def-model-spec ::odd-over-2-mnist-image
    (s/and ::mnist-number odd? #(> % 2))
    discriminate
    generate)

  (test-model-spec ::odd-over-2-mnist-image my-test-image)

  ;; {:spec "odd-over-2-mnist-image"
  ;;  :valid? false
  ;;  :sample-values [3 3 3 5 3 5 7 7 7 3]}

Conclusion

We have shown some of the potential of integrating deep learning models with Clojure. clojure.spec is a powerful tool and it can be leveraged in new and interesting ways for both deep learning and AI more generally.

I hope that more people are intrigued to experiment and take a further look into what we can do in this area.

Permalink

Ep 050: Parentheses! Parentheses! Parentheses!

Each week, we discuss a different topic about Clojure and functional programming.

If you have a question you’d like us to discuss, tweet @clojuredesign, send an email to feedback@clojuredesign.club, or join the #clojuredesign-podcast channel on the Clojurians Slack.

This week, our topic is: “Parentheses! Parentheses! Parentheses!” We defend the lowly parentheses, and discuss the benefits of having this stalwart shepherd dutifully organizing our code.

Selected quotes:

• “I’m converting my visual cues into audio information.”
• “Parentheses are like dust bunnies that have stacked up over time.”
• “Pounds of parentheses.”
• “The parenthesis is greatly misunderstood.”
• “Parentheses make the grammar of the language incredibly simple.”
• “I’ve never seen that in math, at least not in normal math.”
• “If you have 15 close parentheses, you might have to do some refactoring.”
• “Parentheses are the hugs your program needs.”
• “Peace, Love, Parentheses.”

Permalink

Love Letter To Clojure (Part 1)

In this blog post, I will explain how learning the Clojure programming language three years ago changed my life. It led to a series of revelations about all the invisible structures that are required to enable developers to be productive. These concepts show up all over The Unicorn Project, but most prominently in the First Ideal of Locality and Simplicity, and how it can lead to the Second Ideal of Focus, Flow, and Joy.

Without doubt, Clojure was one of the most difficult things I’ve learned professionally, but it has also been one of the most rewarding. It brought the joy of programming back into my life. For the first time in my career, as I’m nearing fifty years old, I’m finally able to write programs that do what I want them to do, and am able to build upon them for years without them collapsing like a house of cards, as has been my normal experience.

The famous French philosopher Claude Lévi-Strauss would say of certain tools, “Is it good to think with?” For reasons that I will try to explain in this post, Clojure embraces a set of design principles and sensibilities that were new to me: functional programming, immutability, an astonishingly strong sense of conservative minimalism (e.g., hardly any breaking changes in ten years!), and much more…

Clojure introduced to me a far better set of tools to think with and to also build with. It’s also led to a set of aha moments that explain why for decades my code would eventually fall apart, becoming more and more difficult to change, as if collapsing under its own weight. Learning Clojure taught me how to prevent myself from constantly self-sabotaging my code in this way.

For nearly three years, I’ve been gushing to whoever will listen about how great Clojure is. I was dazzled by Bryan Cantrell’s amazing “I’m falling in love with Rust” blog post in September 2018—after reading it, I committed myself to eventually write my love letter to Clojure (but only after I first wrote a love letter to my wife, @notoriousMVK, of course!).

Organizing, writing, and finishing this post has been surprisingly difficult. I’ve wanted to write this article for years, having written portions of it over the last twelve months, trying to figure out how to share my experience and epiphanies without being insipid, pretentious, or boring. But after three days of constant writing and second-guessing myself, and now exceeding 8,500 words, in the interest of finally getting something posted, I’m committed to post this today.

The stuff that doesn’t make it in will go into “My Love Letter to Clojure #2”.

I will discuss the following:

• What the Second Ideal of Focus, Flow and Joy Means to Me
• Relearning Coding: How Coding Mostly Left My Daily Life in 2008, But Came Back in 2016 (or If I Can Learn Clojure, Anyone Can)
• Why I Love LISPs Now
• Functional Programming and Immutability (and John Carmack)
• A Mistake I’ve Been Making Since Grad School, and How I Fixed It: Composability
• The Epiphany I Had Reading “An Introduction to React in 2019 (For People Who Know Just Enough jQuery To Get By)”
• What Rich Hickey Says About Simplicity: Where The First Ideal Of Locality and Simplicity Comes From
• Solving Business Problems, Not Solving Puzzles—Why I Detest Doing Infrastructure Work These Days
• Lastly, the REPL…the Ultimate in Fast and Fun Feedback Loops!
• The Amazing Clojure Community, Parting Thoughts, and What I’d Like To Write About in the Future
  

What hopefully comes across is my gratitude to Clojure, its inventor Rich Hickey, and the entire Clojure community.

What the Second Ideal of Focus, Flow and Joy Means to Me

• The First Ideal—Locality and Simplicity
• The Second Ideal—Focus, Flow, and Joy
• The Third Ideal—Improvement of Daily Work
• The Fourth Ideal—Psychological Safety
• The Fifth Ideal—Customer Focus
  

In The Unicorn Project (download the excerpts), the Second Ideal of Focus, Flow, and Joy is intended to describe the optimal mental state of creative flow that we all seek, so beautifully described by the famous psychologist Dr. Mihaly Csikszentmihalyi (pronounced cheek-sent-MEE-hai). He gave one of the most amazing TED talks ever (2004), and wrote the book Flow: The Psychology of Optimal Experience (2008).¹

For many of us in technology, we often associate those amazing moments of productivity and creative flow with coding. It’s the feeling that you get when you are getting so much productive work done: you lose track of time, and maybe even your sense of self, feeling the triumph of achieving amazing feats, brilliantly solving the problem you set out to solve. For me, it often felt like conquering the Level 30 quest when you’re only a Level 5 Paladin. (Insert your D&D character of choice here.)

For most of the twenty years of my professional career, I have self-identified primarily as an Ops person. This is despite getting my masters degree in computer science in 1995—the courses I loved the most were high-speed compilers (main lesson: don’t read files more than once) and high-speed networking (main lesson: avoid making memory copies entirely, if you can).

But as much as I loved computer science, I always gravitated toward Ops, because I observed that that’s where the excitement was, and that’s where the saves were made—it if weren’t for the heroic work of Ops, all the mistakes and failures of Dev and Infosec would affect our customers.

But something changed. For three years, I now self-identify as a developer! Without a doubt, it’s because I learned the Clojure programming language. I mentioned above that it was one of the most difficult things I’ve learned professionally—before I wrote my first line of working code, I must have spent forty hours reading books and blog posts, as well as watching videos, before I was able to write my first real non-trivial program.

A big reason is that it’s a LISP programming language, which I had never used before. Another reason is that Clojure is a functional programming language that embraces and encourages immutability, which means that you’re not allowed to mutate variables, which is another thing I’ve never been exposed to before.

I’m incredibly grateful for the help I got from Mike Nygard (author of the amazing book Release It!, and incidentally, the person who first showed me Clojure and the Datomic database in 2012, which baffled me so much, I mostly forgot about it until five years later) and so many other people in the Clojure community. (More on the incredible community around Clojure later.)

Now, solving problems with Clojure has become one of my favorite things to do.

Coding in Clojure is fun, in a way that I’ve never experienced before. For years, in my ideal month, I spend 50% of my time writing and 50% of my time hanging out with the best in the game. These days, that’s still the case, but I want to be spending 20% of my time coding to solve problems I want to solve as well.

Here’s how much fun I’m having: over the last three years, I’ve been able to quickly solve problems, such as quickly exporting and transforming data from one tool to another, building word clouds, analyzing git repo histories…. But I’ve built some tools that I use nearly every day, such as a program to manage cards on my Trello boards, grab screenshots from Google Photos, track information from various book e-commerce sites. It’s blown me away how I can solve problems with a level of ease and joy I’ve never experienced before.

My first significant Clojure project revealed something stunning to me. I’m fortunate to have participated in helping write and rewrite an application called TweetScriber three times. It’s a program that a bunch of us wrote in 2012 to allow us to take notes and tweet at the same time on an iPad—it turns out that this is a very useful thing to do when you’re writing a book, as it forces you to take notes, write things succinctly enough so that you can tweet it out (which forces a certain pithy and terse style), and get feedback by observing what tweets get people’s attention. (The tweets that got the most retweets inevitably found their way into a presentation or a book.)

• In 2012, Flynn and Raechel Little wrote the first version as an iPad app in Objctive-C—it was about 3,000 lines of code. (They did such an awesome job!! It worked splendidly until iOS 7 or so, when something broke terribly and it wouldn’t even start up anymore.)
• In 2015, I rewrote it as a JavaScript/React application, and it was about 1,500 lines of code.
• In 2017, I rewrote it again as a ClojureScript application, and it was only 500 lines of code! Holy cow!!!
  

First off, I was actually amazed that I could even rewrite this application in ClojureScript as a novice, and how surprisingly straightforward it was. (Clojure runs on the JVM or on the CLR for all you .NET people, while ClojureScript gets transpiled into JavaScript so it can run on the browser, inside node.js, etc. In all cases, they can leverage the massive library and tooling ecosystem that these universes provide.) But what was so remarkable to me was how superior my ClojureScript implementation was, in so many dimensions.

It was shorter. It forced a separation of concerns between mutable state, control flow logic, UI elements, etc. that I’ve long read about, but finally was able to put into practice. (I used the re-frame framework, which co-evolved at the same time as the JavaScript Redux framework.)

What has been the most astonishing to me is that I’ve been able to keep adding functionality to TweetScriber, year after year. Right before the last DevOps Enterprise Summit, I added a way to add photos to tweets without leaving the app; I experimented with saving the tweet ids in a database instead of just text. It was easy to experiment with adding these features without blowing everything else up.

It’s now nearly 5,000 lines of code, and I’m still able to understand it, and add things I want to it without it falling over like a house of cards.

The ability to do this is new to me. There’s something very different about the way I code now, and I attribute it to Clojure and what it’s taught me. I will make the maybe astonishing claim that 90% of the errors I used to make have simply disappeared—as in, an entire category of errors I used to make have been eliminated.

(I intend to open source the TweetScriber application, along with three other apps I’ve written. Just as soon as I figure out how to take all the secrets out of my GitHub repo. If anyone wants to help, let me know. I’ll take any help I can get. :).

You can run TweetScriber, albeit with non-existent documentation here. You can read all the notes I’ve posted here. (OMG, I’m thrilled that someone else is using the program!) You can read all saved scribed that people have posted over the years here (that portion was written by awesome Jeff Weiss and William Hertling).

In short, Clojure brought the joy of programming back into my life!

Relearning Coding: How Coding Mostly Left My Daily Life in 2008, But Came Back in 2016

If I can learn Clojure, anyone can.

Before I got into my specific aha moments, I thought it would be worthwhile to describe to what extent coding stopped being part of my daily life. I’m going to make the case that me learning Clojure is sort of like Encino Man learning to drive—you know, the movie about the caveman who was frozen in ice and wakes up in the completely changed, modern world.

I think this is relevant to many people in technology—I’ve observed that many people in the technology field started their careers coding, but then as their career progressed, they started coding less—sometimes it’s to go into people management, project management, information security, or product management, or into the business functions.

But I believe now more than ever that coding is a proficiency that will be needed in every profession, regardless of what specific domain you specialize in or what role you have within an organization. As the brilliant Mark Schwartz (former CIO of US Citizenship and Immigration Services and author of many amazing books, including Seat at the Table: IT Leadership in the Age of Agility) stated in an address to the government system integrator community, “we need technical people, because last I heard, this is still a technical field.”

(I mean, OMG, right? The response to any remark that “I’m not technical” seems more and more jarring to me. Maybe I’ll write more on that at at later date.)

Here are some concrete statistics on what happened to me, mostly to motivate my claim that I had become “non-technical” for almost a decade. Here’s a chart that shows my programming background, showing year by year how many lines of code I wrote (it’s a wild guess).

I created this graph after Dr. Mik Kersten mentioned to me that he’s written over one million lines of Java in his career, and Rod Johnson (inventor of Spring) said that he’s written millions of lines of Java. Until then, I had never thought about how many lines of code I’ve written in my career…

What I discovered was pretty surprising…

• Perl: thousands of LoC
• C/C++: hundreds of thousands of LoC
• Ruby: tens of thousands of LoC
• Java: hundreds of LoC
  

What really caught my attention: I had only written “hundreds of lines of Java” — in fact, I had written tons more Perl than Java! In contrast, most of my generation had likely written 10K, 100K, or even millions of lines of Java! I had somehow gotten stuck in the C and C++ cul-de-sac, and due to a fluke of timing, I missed the entire Java revolution, which was undeniably the center of gravity for both industry and academic research—multiple generations of PhDs were created around almost every aspect of Java and the JVM.

Maybe amusingly, at the time, I was happy to sit on the Java sidelines, because…stupid reasons…. When I got my undergraduate degree in computer science (1990–1993), the predominant pedagogic language was C. By the time I went to graduate school (1993–1995), C++ was just coming out, and it seemed laughable at the time. One of my professors, the famous Dr. Todd Proebsting, joked, “How do you know if a program was written in C++? It takes 10 seconds for the program to exit()!” (The long delay before the program actually quit was from all the class destructors being called.)

Around 1995, Java was just starting to be talked about—and we mostly laughed at that, too, because of the absurdly long startup times of the JVM. At that time, Perl and Tcl were dismissed as mere “scripting languages,” not suitable for real programmers.

By the time the Java revolution was in full swing in the early 2000s, coding had virtually vanished from my daily life, and it wasn’t until the late 2000s that I got back into doing substantial hobby coding projects. (Around then, I had great fun trying to compete in the Netflix recommendation challenge with William Hertling, which we did in Ruby.)

In contrast, Java revolutionized programming, and a whole ecosystem flourished around it, such as Maven (Maven is to Java as npm is to JavaScript, ruby gems is to Ruby, pip is to Python, etc.), the JVM, Eclipse, etc…

I somehow not only missed out on the Java revolution, but I never got exposed to LISP programming (or even Emacs), which made my entry into Clojure much more difficult.

And yet, now I love the JVM, which Clojure runs on. And I love LISP, which Clojure is a dialect of.

Again, if I can do it, anyone can!

Why I love LISPs Now

Let’s talk about Clojure being a LISP first. As noted in the chart above, LISP is a very ancient language, dating back to 1958. And it’s very distinctive. Even before I went to college, I knew that people made fun of LISP, joking that it stood for “lots of stupid parentheses.”

But I’m never going back. Like almost everything about Clojure, as a LISP, its syntax is astonishingly simple. Everything is inside parentheses, the verb is always at the front and arguments are next.

Yes, almost everyone who hasn’t used a LISP initially finds it alien—my first reaction was, “holy cow, this doesn’t even look like code.” But it’s actually a far more uniform and simpler syntax.

Contrast LISP to the complicated order of precedence operations you find in almost every other programming language, as well as their huge grammars and syntax. That takes a lot of brain space.

I was talking to a JavaScript programming lead at a bar (yes, this is what I love talking about in bars), and he asked what a Clojure program would look like that would parse an integer in a JSON file, and increment that integer by two. Here’s what it looks like—I think it’s quite beautiful:

; input is JSON string: "{foo: 2}"

(defn xform [s]
  (-> (js/JSON.parse s)
      js->clj
      (update "foo" + 2)
      clj->js
      js/JSON.stringify))

Here’s a slightly more annotated version that describes more fully what is going on:

(defn xform-annotated [s]
  ; I love the -> function called threading
  ; it basically chains the functions together -- composition
  ; of functions, passing results of previous function as first argument
  (-> 
    ; calls out to node.js
	; note that first is name of function, and "s" is argument
    (js/JSON.parse s)
    ; convert it from native JS object to Clojure map
    (js->clj)
    ; update key "foo" by applying function (fn (+ 2))
    ;     note that no mutation happens -- it returns a new map
    (update "foo" + 2)
    ; convert back to native JSON object
    (clj->js)
    ; call out to node function to convert back into string
    (js/JSON.stringify)))

A wonderful (and free primer) on LISP syntax is here: https://www.braveclojure.com/do-things/—this is just one portion of Daniel Higgenbottom’s fantastic book Clojure for the Brave and True: Learn the Ultimate Language and Become a Better Programmer. (More book recommendations later.)

To end this section, like I said, I’m never going back to a non-LISP. Life is too short to learn fussy language grammars with tons of special cases. But I’ve become fussy in other ways, too. (More on that later.)

Functional Programming and Immutability (and John Carmack)

Another thing that initially flummoxed me about Clojure was the notion of immutability. After all, how are you supposed to write a program when you’re no longer allowed to change the value of variable?

Over the years, I’ve read many articles about functional program extolling the value of pure functions and immutability—these are two of the characteristics often associated with functional programming languages, such as Haskell, OCaml, Erlang, Elm, Elixir, Scala, PureScript, F#, and of course, Clojure.

Some of the promised benefits include programs that are easier to reason about, the ability to trivially parallelize them, and so forth. One of the first books I read to learn Clojure was Clojure Programming: Practical LISP For The Java World—a book I highly recommend, because it is targeted at programmers who are used to Python, Ruby, and Java.

I was still struggling to get my head around core principles and its idioms. And then I read a code sample, with the following warning: “In Ruby, even strings, often faithfully immutable in other languages, are mutable. This can be the source of all sorts of trouble.”

When I read the following code sample, I literally bolted upright in bed, panicked by what I just read:

# Ruby code 
s1 = "hello"
s2 = "world"

s3 = s1 + s2	# s1 is still "hello"
s4 = s1 << s2 	# s1 is now "hello world" 
				#    !!!! Mutation of s1 is side-effect!

>> s = "hello" 
=> "hello" 
>> s << "*" 
=> "hello*" 	# s value was mutated!!!!
>> s == "hello" 
=> false
😱😱😱

When I say that I “bolted upright in bed,” I mean it quite literally. I was actually panicked from the terror that I felt, wondering how many times I might have accidentally written code that changed a variable I didn’t mean to change. In fact, you can see this tweet I wrote the next morning in that state of existential horror. That was when I realized that mutation was more dangerous than I had thought.

I love Ruby. Last night, while reading, I learned I didn’t understand something very fundamental about Ruby strings: shame on me! pic.twitter.com/Mk26jH1z35

— Gene Kim (@RealGeneKim) October 18, 2016

(Incidentally, people have reported similar feelings of sheer terror after reading Brian Goetz’s book Java Concurrency In Practice, realizing all the things that can go wrong in concurrent programming in code they had written, often leading one to embrace languages like Clojure. More on Brian Goetz later.)

Over the next year, it genuinely dawned on me how disallowing variable mutation made programs simpler and easier to reason about, even for single-threaded programs. It made me realize that “passing variables by reference,” something that I thought was such a time-saver, was actually one of the things that caused me tremendous problems, because it caused variables to be changed by distant parts of the code.

Here’s a brief video I did that described my aha moment of how potentially dangerous passing variables by reference is to the unsuspecting.

Horrors! After programming in Clojure, it seemed so dangerous (and vulgar!) that you could call a function, and it could change the object you passed it! How can you reason about a program that allowed uncontrolled mutation like this?

(By the way, that talk from Scott Havens is amazing: Until very recently, he was Director of Software Engineering, Jet.com/Walmart Labs, where he was responsible for rebuilding the entire inventory management systems that support Walmart, the world’s largest company, written in F#—another amazing functional programming language. He talked about how he used functional programming principles to massively simplify the vast architecture that supported the inventory management systems, making it simpler, more reliable, easier to maintain, and cheaper to run. This is functional programming not in the small, but in one of the largest and most complex business processes in the world—and this is what the last 20% of The Unicorn Project (download the excerpts) is inspired by.)

Just as Clojure showed me how a simpler LISP syntax frees your brain to think more about the problem you want to solve, the world without mutation made me realize just how difficult it is to keep track of how mutations are happening in your programs.

But don’t take it from me—take it from John Carmack, who influenced so many of us, as one of the founders of id Software (DOOM, Quake, etc.), now CTO of Oculus VR. He wrote an amazing article in 2013 in Gamasutra magazine about the power of using functional programming concepts in C++.

My pragmatic summary: A large fraction of the flaws in software development are due to programmers not fully understanding all the possible states their code may execute in. In a multithreaded environment, the lack of understanding and the resulting problems are greatly amplified, almost to the point of panic if you are paying attention. Programming in a functional style makes the state presented to your code explicit, which makes it much easier to reason about, and, in a completely pure system, makes thread race conditions impossible.

No matter what language you work in, programming in a functional style provides benefits. You should do it whenever it is convenient, and you should think hard about the decision when it isn’t convenient. You can learn about lambdas, monads, currying, composing lazily evaluated functions on infinite sets, and all the other aspects of explicitly functionally oriented languages later if you choose.

C++ doesn’t encourage functional programming, but it doesn’t prevent you from doing it…

Pure functions are trivial to test; the tests look like something right out of a textbook, where you build some inputs and look at the output. Whenever I come across a finicky looking bit of code now, I split it out into a separate pure function and write tests for it. Frighteningly, I often find something wrong in these cases, which means I’m probably not casting a wide enough net.

I remember reading this article in 2013, but it wasn’t until I re-read it in 2016 that I realized how universal this problem is—it’s not just for games written in C++ that need to run at 60 frames per second, it’s for any programmer.

In his 2013 QuakeCon keynote, he describes his experiments rewriting Castle Wolfenstein 3D using functional programming techniques in Haskell, to explore to what extent one could use a programming language that disallowed mutation, which was absolutely fascinating.

(In this talk, Carmack also talks about his explorations of LISP, doing the exercises from the famous MIT SICP book, why he’s growing ever more convinced on the value of strong, statically typed languages for large codebases that need to be maintained for a long time, and so many other fascinating observations from one of the programmer legends… I will be writing more on this in Part 2.)

A Mistake I’ve Been Making Since Grad School: Composability

After gaining more proficiency in Clojure, many other things become apparent. One thing that I learned is the benefits of pushing side-effects (i.e., any impure functions, such as anything that involves input or output) to the edges of the program and making sure that everything else in between is a pure function.

To recap, pure functions are those that the output is strictly a function of the inputs. One of the benefits of pure functions is that the function can be tested completely in isolation, as John Carmack noted above.

When I started making a point of doing this, the benefits started becoming quite apparent. In fact, I started to get the suspicion that not practicing this was one of the main reasons the code I wrote would become increasingly untestable and difficult to change—which led to the feeling that my code was collapsing under its own weight.

This is a problem that I’ve been having for a very long time…. In fact, I still have a very vivid memory of when this problem killed a project I was working on. It was during my high speed compilers class in graduate school in 1994, where we built a Modula-2 compiler in C++ that generated SPARC assembly code, which was then compiled into a Solaris executable.

You first wrote a lexical analyzer to tokenize the input (using lex ), then the compiler that would turn the tokens into an abstract syntax tree (using yacc), turn it into some intermediate format, and then emit assembly code. And then use as to compile.

It started off great, especially since I had written lexers and parsers before. But at each later stage, I remember the feeling of tucking the previous compilation phase into the next phase, thinking, “Okay, here goes nothing…. I’m putting my code somewhere where I can no longer directly access it, and it feels like throwing it into a deep, dark well… I sure hope it works.”

In other words, I had written the compiler phases in a way where each phase was no longer independently runnable, let alone testable. I was good enough where it was not really a problem until the very last phase, where we were executing our SPARC executable program.

I remember my compiler working for all the test cases, until we had to implement recursion. Then it started blowing up after a certain number of recursive calls—after several all-nighters, I finally figured out that I was incorrectly computing the memory locations of variables on the stack, probably incrementing the stack pointer the wrong way. After a certain number of recursive function calls, the generated program would just segfault and die.

Even thought I figured out conceptually what was going wrong, I ran out of time, and couldn’t fix the mess I had created before submitting my fatally flawed compiler. I probably got one of the lowest grades in the class.

What I had learned was that burying my code so that it could only be executed inside other functions violated the principle of composition—the ability to run and test it, independent and isolated from the other modules. (My thanks to Dr. Stephen Magill for taking the time to explain this to me, who also helped me write my first Haskell program!)

Here’s some pseudo-code of the good way vs. bad way…

; 
; good way: steps are composed together, which each compiler phase
; indepdendently executable and testable
; 
(-> (tokenize-source-files!)
    (generate-abstract-syntax-tree)
    (generate-intermediate-representation)
    (generate-assembly-instructions)
    (write-assembly-output-files!))

; 
; bad way
;    all the intermediate steps buried inside other functions, no longer reachable or inspectable
; 
tokenize-source-files-and-generate-ir-and-generate-assembly();

In Clojure, I fell in love with the convention of marking impure functions with a !, to make it obvious when side-effects could happen. This includes any function that reads from disk or a database, let alone writes to one—after all, even external inputs make the output impure. These effectful functions should be pushed to the beginning or the end of your program, which makes everything in the middle independently runnable and testable.

It’s embarrassing that it’s taken thirty years after I started coding professionally to finally discover this mistake, which has plagued almost every large program that I’ve worked on.

And of course, many people learned this twenty years ago, either by intuition, experience, or by reading great books such as Refactoring: Improving the Design of Existing Code by Martin Fowler. But knuckle-headed me only learned after being forced to confront this in Clojure, which seems to bring these problems to the front and center.

(What’s also interesting to me is that by pushing side-effects to the edges, almost everything can be rewritten as pure functions and be tested with unit tests. Previously, most of my tests often tested my code and the I/O, API calls, etc… In the ideal, you just want to test the code you wrote. You don’t need to test whether the file system works in every test, or whether Trello API works, etc…. By pushing I/O to the edges, the need for brittle mocks and stubs almost disappears entirely.)

The Epiphany I Had Reading “React for jQuery Programmers To Get By”

Something else happened during my first rewrite of TweetScriber that primed me for another Clojure aha moment. Knowing almost nothing about writing a JavaScript app, I was initially doing some research and stumbled upon this amazing article, An Introduction to React in 2019 (For People Who Know Just Enough jQuery To Get By) which was originally written by Shu Uesugi (@chibicode) in 2015 and revised by Julien Benchetrit (@julienbenc).

I had virtually no experience with working in the browser DOM, had never used jQuery (but had heard about how revolutionary it was years back), and I’ve certainly never used one of these more modern frameworks like React.

The article seemed especially timely, because at first glance, the exercises seemed to cover 50% of the TweetScriber functionality I needed! But reading through the exercises, it seemed to discuss something very, very important, with some profound lessons in how program state leads to incredible complexity—more things that have caused my programs to eventually collapse in on itself.

• Step 1. Write a Tweetbox, which lets you type into an textarea window, with a Tweet button.
  
  Cool. I barely knew enough HTML to write this, so copying this code would be awesome. Easy enough.
  
• Step 2. Change the Tweetbox so that if the contents exceed 140 characters (the original article was written in 2015), then disable the Tweet button.
  
  Great. We now have some state associated with the program, but it isn’t too bad, because it’s just keeping track of the number of characters in the textarea and adjusting whether the button is disabled or not.
  
• Step 7. Change the Tweetbox so that it displays characters remaining, and a display message that indicates by how many characters you’ve exceeded the limit.
  
  Okay, this is getting a little more complicated, because now these different components need to be updated differently based on the state…
  
• Step 9. Add an Add Photo button, which displays “photo added” if a photo has already been added, and also subtract 23 characters from the number of characters remaining (because links to photos consume characters).
  
  Okay, the point of these exercises was becoming very evident, because now state management is becoming a real issue, and using the jQuery style of callbacks was becoming a real mess—even keeping track of which components needed to know of each other, and what their responsibilities are to each other, seemed impossible to remember.
  
  And this is for a small toy application!!! In a real application, the complexity is orders of magnitude worse!

In other words, in the jQuery implementation, the flow of information from one component to another becomes entangled. The diagram below shows it magnificently.

SIMPLE! Source: Shu Uesugi and Julien Benchetrit

In contrast, consider the the same functionality implemented with something like React, in a more functional style, where state is mutated in one place, and all the UI elements are rendered from the state. State is changed in a very uniform way, and the UI is rendered as pure functions, never mutating state.

The diagram below show how much simpler the data flow is…

COMPLECTED! Source: Shu Uesugi and Julien Benchetrit

I found this to be a shockingly convincing argument about how code should be written. And I was pretty happy with my first version that I wrote in TypeScript and React.

(In my ClojureScript rewrite using re-frame, it addressed one of the problems I found in using React, which was state is scattered across all the components. Re-frame and Redux take a different approach, which is to put all mutable state into one place, which I’ve found works magnificently. This has been so successful for me that I will never write a client web app in any other type of framework.)

I think this aha moment prepared me for the discovery of Rich Hickey and Clojure…

What Rich Hickey Says About Simplicity: Where The First Ideal Of Locality and Simplicity Comes From

Rich Hickey is the inventor of Clojure, and his talks and Clojure have so much affected how I think about software—in fact, this entire blog post attempts to frame some of these aha moments.

There was one particular Rich Hickey talk that hit me like a ton of bricks, which was his famous “Simple Made Easy” talk that he gave at Strange Loop 2011—if the jQuery/React epiphany was an aha moment, this talk was a chorus of aha moments. ²

The talk is on the InfoQ site here (which shows the slides associated with the soundtrack), and you can find a wonderful transcript of the talk here. You can also find a version of the talk he did for the Ruby and Ruby on Rails community on YouTube here.

This talk is so impactful to me that I’m almost tempted to say, “Go ahead and take an hour to listen to the talk. I’ll wait here.”

Among many other things, he talks about how limited the human brain is in its ability to reason about things, to keep track of things, to grapple with complexity (like the jQuery example above). He stated that the difference in cognitive capability between your average programmer and your average programmer is not the vaunted 10x difference. He said (emphasis mine):

Then we have this other part though, which is the mental capability part. And that’s the part that’s always hard to talk about, the mental capability part because, the fact is, we can learn more things. We actually can’t get much smarter. We’re not going to move; we’re not going to move our brain closer to the complexity. We have to make things near by simplifying them.

But the truth here is not that they’re these super, bright people who can do these amazing things and everybody else is stuck because the juggling analogy is pretty close. Right? The average juggler can do three balls. The most amazing juggler in the world can do, like, 9 balls or 12 or something like that. They can’t do 20 or 100. We’re all very limited. Compared to the complexity we can create, we’re all statistically at the same point in our ability to understand it, which is not very good. So we’re going to have to bring things towards us.

He talks about the need for simplicity in the software we write, and the stuff that we write our software in. The conditions for simplicity include having components that are completely decoupled from each other with no knowledge each other. Just like in the jQuery example, things quickly become a complete mess when all the different controls have to know about each other—it’s difficult to get it to work correctly, to reason about what happens when state changes, and it’s difficult to write additional functionality to it.

Hickey goes through the some of the core concepts in programming languages, and describes the extremes of which represent the ideal of simplicity and which ones represent complexity that eventually leads to the inability to understand and safely change our code, as well as misery and catastrophe.

Source: Rich Hickey, Simple Made Easy (2011)

Source: Rich Hickey, Simple Made Easy (2011)

As I mentioned in the beginning of this article, The Unicorn Project (download the excerpts) is about the invisible structures that enable developers to be productive. So much of this comes from the concepts that Rich Hickey has espoused over the years, and which have become infused into Clojure and the Clojure community:

Erik answer, “‘Complect’ is an archaic word, resurrected by Sensei Rich Hickey. It is a verb that means to turn something simple into something complex.

“In tightly coupled and complected systems, it’s nearly impossible to change anything, because you can’t just change one area of the code, you must change one hundred, or even a thousand, areas of the code. And even the smallest changes can cause wildly unpredictable effects in distant parts of the system, maybe in something you’ve never even heard of.

“Sensei Hickey would say, ‘think of four strands of yarn that hang independently—that’s a simple system. Now take those same four strands of yarn and braid them together. Now you’ve complected them.’ Both configurations of yarn could fulfill the same engineering goal, but one is dramatically easier to change than the other. In the simple system, you can change one string independently without having to touch the others. Which is very good.”

Erik laughs, “However, in the complected system, when you want to make a change to one strand of yarn, you are forced to change the other three strands, too. In fact, for many things you may want to do, you simply cannot, because everything is so knotted together!

“And when that happens,” he continues, “you’ve trapped yourself in a system of work where you can no longer solve real business problems easily anymore—instead, you’re forced to merely solve puzzles all day, trying to figure out how to make your small change, obstructed by your complected system every step of the way. You must schedule meetings with other teams, try to convince them to change something for you, escalate it to their managers, maybe all the way up the chain.

“Everything you do becomes increasingly distant from the real business problem you’re trying to solve,” he says. “And that, Dwayne, is what everyone discovered when they switched out the routers in those manufacturing plants. Before, you had three independent strands, with team able to work independently but at the cost of having to maintain three networking switches.

“When you put them all on one switch, you complected their value streams, all now having dependencies on each other that didn’t exist before! They must constantly communicate, coordinate, schedule, marshal, sequence, and deconflict their work. They now have an extremely high cost of coordination, which lengthened lead times, decreased quality, and in your story, led to a week-long catastrophe that significantly impaired the business, going all the way up to Steve!” Erik says with glee.

You can find many collections of “Rich Hickey’s Greatest Hits” on the Internet. Here are some:

• https://github.com/tallesl/Rich-Hickey-fanclub
• https://www.youtube.com/playlist?list=PLXsqD83He-e5oUh_DFrHbO3MoNj3tG8Vh
  

(In The Unicorn Project (download the excerpts), the joke about “how many people do you need to take out to lunch in order to get a feature done” was taken from his talk that he did at 2015 Java One conference: https://www.youtube.com/watch?v=VSdnJDO-xdg. In fact, in some ways, coupling as one of the key themes of the book came from watching this talk, where he compared modern REST interfaces with the bad old days of CORBA and Sun RPC, where changing anything needed everybody to cooperate.)

What I find so interesting is that functional programming also eliminates iteration and looping, which is so prone to off-by-one errors. In The Unicorn Project, when Maxine pairs with some middle schoolers in Python, she spots their “off by one” error when they’re iterating through an array. This was inspired by Cornelia Davis, describing when she worked with her college-aged son on a genomic sequencing program in Python, showing the contrast in imperative vs. functional programming style: https://youtu.be/R1RDhUf1Go4?t=492

By the way, here’s a whole brilliant talk about showing how to convert an imperative program into a more functional style in JavaScript: “Solving Problems The Clojure Way,” by Rafal Dittwald.

Solving Business Problems, Not Solving Puzzles—Why I Detest Infrastructure These Days…

So, I feel like I need to explain another consequence of now recognizing and wanting to avoid complexity. I mentioned that I now self-identify as a developer…. What I didn’t tell you was that I now self-identify as one of those very fussy, parochial developers that Ops people hate.

In my ideal, I just want to work on solving the problem I set out to solve, working within my pure functional application bubble. And I don’t want to deal with anything outside of that bubble.

Here’s all the things I used to love doing, but now I detest doing…

• dealing with anything outside of my application
• connecting to anything to anything (including databases)
• SQL databases and figuring out why my queries are so slow
• updating dependencies (because, so hard…)
• secrets management (ditto)
• Bash
• YAML
• patching (so inconvenient…)
• building kubernetes deployment files (mostly by Googling)
• trying to understand why my cloud costs are so high
  

This is not to say that these things aren’t important. On the contrary, they’re absolutely critical. Which is why I think the brightest days of Infrastructure and Operations and Infosec are still ahead of us, not behind us. I believe the job of infrastructure and security people is to create the platforms that developers can use to take care of all these non-functional requirements, without having to Google and Stack Overflow all day.

Lastly, the REPL…the Ultimate In Fast Feedback Loops!

Okay, it’s about time to wrap up this first article, which is pushing 8K words, and there’s so many other things I want to write about because I’m so excited by them, but it’s time to post something…otherwise, I’ll never post anything.

But there’s one thing that I need to write about, which is something that seems pretty unique to Clojure, or LISPs, which is the amazingly interactive nature of development. One reason I think I’ll be using Clojure for years (or decades) to come is because of the REPL.

REPL stands for Read-Eval-Print-Loop, which many languages have. But I don’t know of any other modern language that can be integrated into an editor, where you can modify the running program, inspect variables, run functions…from inside the program!

In the spirit of Focus, Flow, and Joy, I think the REPL experience is absolutely amazing and sublime. It feels like you’re building your program from the inside, writing and testing expressions, saving I/O to variables and transforming the output using pure functions. I’ve done this with Clojure programs running on the JVM, or ClojureScript programs running in a browser…I’ve done it to analyze and visualize large data sets, save to and query databases…

Going back to programming where you have to save and reload your program, and then maybe even navigate to get back to a certain program state, just seems intolerable now.

This is definitely one of those things that you have to see a master at work to fully appreciate. Here’s Bruce Hauman talking in 2015 about the now famous figwheel, a ClojureScript build tool that enables live code reloading (and so much more). Watch as he shows something he wrote on the plane ride out to the conference, as he changes the solar system simulator and the JavaScript app gets hot-reloaded in real time.

(Yes, you can do this in JavaScript, but trust me, due to how dicey state mutation is, it’s very flakey. On the other hand, the way Clojure enforces mutation to be done in a very specific way makes it almost perfect for this style of coding.)

Here’s a 2019 video of Sean Corfield (famous for his work on the mostly widely used Clojure JDBC libraries) showing how he uses the REPL to debug a problem reported in the core.memoize function, using the Atom editor and the Chlorine Clojure plug-in. He walks through the steps required to reproduce the problem, form by form, and the output of each form as it’s evaluated is displayed in the editor. In the failing case, we watch as he inspects the internal state of the function, hypothesizes what is going wrong, confirms it, and fixes it. (We then watch as he creates an additional test to the Clojure test suite and runs in across 10 years of Clojure versions.)

I think Sean makes debugging almost look fun, and it shows just how powerful working within a REPL can be.

(I use the Cursive plug-in for IntelliJ by Colin Fleming and team, which I find to be absolutely terrific. I use Visual Studio Code for almost all non-Clojure files, so I have high hopes for the Calva plug-in…. But I love the Cursive features so much I can’t imagine switching anytime in the near future.)

Here’s a great video of Mark Seeman implementing the “fizzbuzz” program in three languages: C#, Haskell, and then Clojure. Note that he’s using a REPL to do the Clojure implementation, and it sure seems like he’s having more fun doing it than in the other languages! (He’s using a mostly abandoned editor called LightTable, which was groundbreaking when it was released.)

And here’s a video of the aforementioned and amazing Mike Nygard showing how to do TDD using Clojure and the Datomic database, from inside Emacs. (Many people equate Clojure with Emacs — but I think only 50% of people in the Clojure community use Emacs. I put this video last to demonstrate how many options viable Clojure editor options there are!)

The Amazing Clojure Community, Parting Thoughts, and What I’d Like To Write About In The Future

I’ve found the Clojure community to be incredibly amazing and supportive. There’s a Clojurian Slack Channel that is incredible. The Clojure/conj and other conferences tend to attract very experienced developers who share similar sensibilities. All these conferences tend to invite some of the most accomplished researchers and contributors in the programming domain.

An example: one of most amazing talks I’ve seen was from the 2016 Clojure/conj where Brian Goetz, architect for the Java language, talked about his stewardship of the Java ecosystem. What’s unmistakable and so admirable is his sense of responsibility to not break code that nine million developers have written.

Here’s a talk from Tiago Luchini on writing declarative domain models—after you watch this, you’ll never want to deal with a SQL or noSQL database again…and which is why I switched to using the Datomic database, from the same organization that brings us Clojure.

Some great learning aids for Clojure:

• Eric Normand has an amazing set of tutorials on Functional TV: https://purelyfunctional.tv/browse/?taxonomy=language—if you don’t have a lot of experience with Java (like me), I’d especially recommend his modules on Java, JVM, etc.
• Arne Brasseur has another set of amazing tutorials on Lambda Island: https://lambdaisland.com/
• https://feierabendprojekte.wordpress.com/2018/10/27/high-level-vs-low-level/amp/
• The famous Bob Martin of Clean Code fame on “Why Clojure?”: https://blog.cleancoder.com/uncle-bob/2019/08/22/WhyClojure.html
  

I recommend the following books to get up the speed on Clojure:

• My opinion that the best book to learn Clojure is Russ Olsen’s Getting Clojure: Build Your Functional Skills One Idea at a Time (2018)—this wasn’t published when I learned Clojure. Oddly enough, I learned Ruby by reading one of his books, too!
• Next, I would read: Clojure Programming: Practical LISP For The Java World by Chas Emerick, Brian Carper, and Christophe Grand. It has great examples of how to solve common problems, like connecting to a database, building a web server, etc.
• I also liked Living Clojure: An Introduction and Training Plan for Developers by Carin Meier, as it has a fantastic “coding kata” teaching style.
  

(I’ve bought almost every book on Clojure that I could find—I like every one one of them.)

• In the spirit of learning from people more experienced than you, I’d recommend this amazing podcast: Functional Design in Clojure by Christoph Neumann and Nate Jones: https://overcast.fm/itunes1443852833/functional-design-in-clojure
  
• David Nolen videos:
  • ClojuTRE 2016: “The Next Five Years of ClojureScript”: the amazing story of how David Nolen got involved with Clojure and ClojureScript while he was working at the NYTimes, how he became one of the top contributors to ClojureScript, his view on its importance, and what may likely come: https://www.youtube.com/watch?v=mty0RwkPmE8
  • CraftConf 2019: “Rethinking Identity with Clojure”: a great talk about the core concepts that Clojure rethinks, and more brilliant stuff: https://www.youtube.com/watch?v=77b47P8EpfA&t=1s
    

(I also want to thank Mike Fikes for all his help. I’ll never forget how, in a moment of impulse a couple of years ago, I asked whether he’d be available to chat, to teach me about the difference between the different ClojureScript runtimes, and how to connect them to IntelliJ. You’re amazing, and keep up the great work! Sorry, and thank you!!!)

The work within the Clojure community is so interesting, I subscribe to the Planet Clojure RSS feed, which aggregates articles about Clojure and functional programming from many sources: http://planet.clojure.in/.

I have a lot of enthusiasm to write in the future about the following, because it illuminates the First Ideal of Simplicity and Locality, and the Second Ideal of Focus, Flow, and Joy.

• The Clojure programs I wrote to analyze writing patterns throughout development of The Unicorn Project (download the excerpts).
• My top twenty videos from the Clojure community that blew my mind.
• My adventures in trying to learn static functional programming languages, like Haskell, and learning category theory—that’s been unexpectedly rewarding. This is the world of monoids, functors, applicative functors, and the monad.
• Doing a screencast of analyzing the Stack Overflow 2019 Survey data to better understand the demographics of Clojure developers, and why they rank as the highest paid population.

Footnotes

1. Thank you to Parker Matthew from Pivotal for cementing this connection for me in one of our many amazing conversations. It’s interesting to note the amazing tension and mutually supportive nature between Dr. Csikszentmilhali’s work, and the work of Dr. Anders Ericsson about deliberate practice, described in “Peak: Secrets from the New Science of Expertise”.
   The dueling dynamic of “practice as a transcendental experience” vs. “practice as hard work and perseverance” is magnificently described in Dr. Sally Ducksworth book “Grit: The Power of Passion and Perseverance”
2. The person who created the Strange Loop conference is Alex Miller, who is also a very prominent figure in the Clojure community. In another strange twist of history, Adrian Cockcroft (@adrianco) told me that this was his favorite conference in 2014. I filed it away at the time.
   Years later in 2017, I found myself devouring video after video from this conference — it is a magnificent program, a confluence of Papers We Love, static typers and category theorists, hackers working on amazing projects, and even DevOpsy things. I’ve been wanting to attend for the last three years, but had to cancel at the last minute each time.
   2020 will be the year, I’m sure!!!

The post Love Letter To Clojure (Part 1) appeared first on IT Revolution.

Permalink

The joys of coverage

Tracking the code coverage is great (except that Clojure has some problems)

Permalink