Extending Prismatic Schema to Higher Order Functions

Over the past year I have used Prismatic Schema extensively on a large Clojure project. We really like the aid that if offers in understanding, debugging, and using our code. However, we regularly feel a bit of a let down when an argument to a function is another function. Prismatic Schema doesn’t allow you to say much in these cases beyond: this arg is a function.

To address this we extended Prismatic Schema to allow us to add type annotations to the function arguments in higher-order functions (in addition to several other small extensions). This is done by calling s/fn which expects [output-schema & input-schemas]

Usage examples and details of how it works are here

The type checking for the most part is simply checking that the type of the input function exactly matches the declared type. So the checker largely ignores issues of type hierarchies, covariance, contravariance, etc. As chouser pointed out to me the basic issue this feature raises is that instead of comparing parameter instance objects to declared argument types, this feature requires comparing declared types to declared types. This is one way in which the semantics of this feature are different than “normal” Prismatic schema.

I am torn about the code. On the one hand I am dissatisfied that it is not a full, general solution. On the other hand I can shrug and recognize that for a class of functions and usage patterns it works just fine. So while from a type theoretic perspective it is quite limited, we have started using it and getting benefit from it.

Permalink

Survey, etc

Just a few quick notes on recent Immutant happenings...

Survey

Yesterday we published a short survey to help us gauge how folks have been using Immutant. Please take a few moments to complete it if you haven't already.

Luminus

The Luminus web toolkit now includes an Immutant profile in its Leiningen project template, so you can now do this:

$ lein new luminus yourapp +immutant
      $ cd yourapp
      $ lein run -dev
      

That -dev option is triggering the use of immutant.web/run-dmc instead of immutant.web/run so it should plop you in your browser with code-reloading enabled. You can pass most of the other run options on the command line as well, e.g.

$ lein run port 3000
      

Beta2 bugs

In our last release, 2.0.0-beta2, we updated our dependency on the excellent potemkin library to version 0.3.11. Unfortunately, that exposed a bug whenever clojure.core/find was used on our Ring request map. Fortunately, it was already fixed in potemkin's HEAD, and Zach was kind enough to release 0.3.12. We've bumped up to that in our incrementals and hence our next release.

We've also fixed a thing or two to improve async support when running inside WildFly.

Plans

We're still hoping to release 2.0.0-Final within a month or so. Now would be a great time to kick the tires on beta2 or the latest incremental to ensure it's solid when we do!

Permalink

Immutant 2 (The Deuce) Beta2 Released

We're just bananas to announce The Deuce's second beta: Immutant 2.0.0-beta2. At this point, we feel pretty good about the stability of the API, the performance, and the compatibility with both WildFly 8 and the forthcoming WildFly 9.

We expect a final release before spring (in the Northern Hemisphere). We would appreciate all interested parties to try out this release and submit whatever issues you find. And again, big thanks to all our early adopters who provided invaluable feedback on the alpha, beta, and incremental releases.

What is Immutant?

Immutant is an integrated suite of Clojure libraries backed by Undertow for web, HornetQ for messaging, Infinispan for caching, Quartz for scheduling, and Narayana for transactions. Applications built with Immutant can optionally be deployed to a WildFly cluster for enhanced features. Its fundamental goal is to reduce the inherent incidental complexity in real world applications.

What's changed in this release?

The biggest change in this release is a new API for communicating with web clients asynchronously, either via an HTTP stream, over a WebSocket, or using Server-Sent Events. As part of this change, the immutant.web.websocket namespace has been removed, but wrap-websocket still exists, and has been moved to immutant.web.middleware. For more details, see the web guide.

In conjunction with this new API, we've submitted changes to Sente that will allow you to use its next release with Immutant.

For a full list of changes, see the issue list below.

How to try it

If you're already familiar with Immutant 1.x, you should take a look at our migration guide. It's our attempt at keeping track of what we changed in the Clojure namespaces.

The guides are another good source of information, along with the rest of the apidoc.

For a working example, check out our Feature Demo application!

Get It

There is no longer any "installation" step as there was in 1.x. Simply add the relevant dependency to your project as shown on Clojars. See the installation guide for more details.

Get In Touch

If you have any questions, issues, or other feedback about Immutant, you can always find us on #immutant on freenode or our mailing lists.

Issues resolved in 2.0.0-beta2

  • [IMMUTANT-439] - Provide SSE support in web
  • [IMMUTANT-515] - Add :servlet-name to the options for run to give the servlet a meaningful name
  • [IMMUTANT-517] - Allow undertow-specific options to be passed directly to web/run
  • [IMMUTANT-518] - Error logged for every websocket/send!
  • [IMMUTANT-520] - WunderBoss Options don't load properly under clojure 1.7.0
  • [IMMUTANT-521] - Add API for async channels
  • [IMMUTANT-524] - immutant.web/run no longer accepts a Var as the handler
  • [IMMUTANT-526] - Improve the docs for messaging/subscribe to clarify subscription-name

Permalink

Clojure Reactive Programming has been published

I'm extremely happy to let everyone know my book, Clojure Reactive Programming, has finally been published!

You can get it at the publisher's website or on Amazon. I had a great time writing it and I truly hope you find it useful!

I've met a few authors here and there and I heard more than once that a book is never really finished. I now know what they mean.

The book doesn't cover everything I wanted to write about due to time and space limitations. Having said that, now that the book is out I do plan to expand on a few things using this blog.

Stay tuned!

Thanks to everyone who gave me feedback on early drafts of the book! :)

Permalink

Student applications due today

The student application deadline is coming up at 19;00 UTC, less than seven hours from now.  What does this mean for you?
If you are a student…
 
You must have your application submitted to Melange by 19:00 UTC.  This is a hard deadline, and we have no control over that.  On a case by case basis, during review, we may allow you to revise your application, but please be sure that the application is ready when you submit it.  Please adhere to the Student Application Deadlines.  Also, be sure that the mentor you have been working with is signed up as a mentor in Melange.
If you are considering being a mentor…
 
Please be sure to sign into Melange and request to connect with Clojure as a mentor.  Be sure to write something about who you are and what projects you are interested in mentoring when you apply.
What happens next?
 
Once the student application deadline closes, all of the Clojure mentors and admins will review all of the student proposals.  During this period, we will assess the number of good mentor/student combinations available. By the 13th of April, we have let Google know how many students we would like to have.  Over the next couple of weeks, Google will allocate student slots to all of the organisations and we will deduplicate in any cases where the same student is accepted by more than one organisation.  Finally, on the 27th of April, Google will announce the students who have been selected.  Until then, we cannot comment on who is or is not accepted.
Thanks again to everyone who is volunteering to make this effort a success.  Clojure/GSoC could not succeed without all of your efforts.  I am looking forward to a wonderful summer of code.

Permalink

Taming those pesky datetimes in a clojure stack

Have you ever faced frustrating issues when using dates in your clojure stack ? If I mention java.util.Date, java.sql.Date/java.sql.Timestamp clj-time, json/ISO-8601 and UTC/Timezones, does your bloodpressure rise slightly ?

This is the blog post I wished I had several weeks back to save me from some of the date pains my current project has been through.

Introduction

A little while back date handling started to become a nightmare in my current project. We have a stack with a ClojureScript frontend, a clojure WebApp and a couple of clojure microservices apps using Oracle as a data store.

We decided pretty early on to use clj-time. It’s a really quite nice wrapper on top of joda-time. But we didn’t pay much attention to how dates should be read/written to Oracle or how we should transmit dates across process boundaries. Timezones is another issue we didn’t worry to much about either.

You will probably not regret using an UTC timezone for your Servers and Database. This post puts it succinctly. Your webclient(s) though is out of your control !

I’m sure some of the measures we have taken can be solved more elegantly, but hopefully you might find some of them useful.

Reading from/writing to the database

We use clojure/java.jdbc for our database integration. Here’s how we managed to simplify reading and writing dates/datetimes.

(ns acme.helpers.db
  (:import [java.sql PreparedStatement])
  (:require [acme.util.date :as du]
            [clj-time.coerce :as c]
            [clojure.java.jdbc :as jdbc]))


(extend-protocol jdbc/IResultSetReadColumn                                (1)
  java.sql.Date
  (result-set-read-column [v _ _] (c/from-sql-date v))                    (2)

  java.sql.Timestamp
  (result-set-read-column [v _ _] (c/from-sql-time v)))

(extend-type org.joda.time.DateTime                                       (3)
  jdbc/ISQLParameter
  (set-parameter [v ^PreparedStatement stmt idx]
    (.setTimestamp stmt idx (c/to-sql-time v))))                          (4)
1 We extend the protocol for reading objects from the java.sql.ResultSet. In our case we chose to treat java.sql.Date and java.sql.Timestamp in the same manner
2 clj-time provides some nifty coercion functions including the facility to coerce from sql dates/times to DateTime
3 We extend the DateTime class (which is final btw!) with the ISQLParameter protocol. This is a protocol for setting SQL parameters in statement objects.
4 We explicitly call setTimestamp on the prepared statement with a DateTime coerced to a java.sqlTimestamp as our value

Now we can interact with oracle without being bothered with java.sql.Date and java.sql.Timestamp malarkey.

It’s vital that you require the namespace you have the above incantations, before doing any db interactions. Might be evident, but it’s worth emphasizing.
Clojure protocols are pretty powerful stuff. It’s deffo on my list of clojure things I need to dig deeper into.

Dates across process boundaries

Our services unsurpringly uses JSON as the data exchange format. I suppose the defacto standard date format is ISO-8601, it makes sence to use that. It so happens this is the standard format for DateTime when you stringify it.

You might want to look into transit. It would probably have been very useful for us :)

Outbound dates

(ns acme.core
  (:require [clojure.data.json :as json]
            [clj-time.coerce :as c]))


(extend-type org.joda.time.DateTime           (1)
  json/JSONWriter
  (-write [date out]
    (json/-write (c/to-string date) out)))    (2)
1 Another extend of DateTime, this time with the JSONWriter protocol.
2 When serializing DateTime to json we coerce it to string. clj-time.coerce luckily uses the ISO-8601 format as default

Inbound dates

(ns acme.util.date
  (:require [clj-time.core :as t]
            [clj-time.format :as f]
            [clj-time.coerce :as c]))


(def iso-date-pattern (re-pattern "^\\d{4}-\\d{2}-\\d{2}.*"))


(defn date? [date-str]                                                         (1)
  (when (and date-str (string? date-str))
    (re-matches iso-date-pattern date-str)))


(defn json->datetime [json-str]
  (when (date? json-str)
    (if-let [res (c/from-string json-str)]                                     (2)
      res
      nil))) ;; you should probably throw an exception or something here !

(defn datetimeify [m]
  (let [f (fn [[k v]]
            (if (date? v)
              [k (json->datetime v)]                                           (3)
              [k v]))]
    (clojure.walk/postwalk (fn [x] (if (map? x) (into {} (map f x)) x)) m)))
1 A crude helper function to check if a given value is a date. There is a lot that passes through as valid ISO-8601 we settled for atleast a minimum of YYYY-MM-DD
2 Coerces a string to a DateTime, the coercion will return nil if it can’t be coerced, that’s probably worth an exception
3 Traverse a arbitrary nested map and coerce values that (most likely) are dates
Hook up middleware

(defn wrap-date [handler]                                     (1)
  (fn [req]
    (handler (update-in req [:params] (datetimeify %)))))


def app (-> routes
            auth/wrap-auth
            wrap-date                                         (2)
            wrap-keyword-params
            wrap-json-params
            wrap-datasource
            wrap-params
            wrap-config))
1 Middleware that calls our helper function to coerce dates with the request map as input
2 Hook up the middleware

Handling dates in the webclient

We have a ClojureScript based client so it made sense for us to use cljs-time. It’s very much inspired by clj-time, but there are some differences. The most obvious one is that there is no jodatime, so Google Closure goog.date is used behind the scenes.

So how do we convert to and from the iSO-8601 string based format in our client ?

Surprisingly similar to how we do it on the server side as it happens !

;; require similar to the ones on the server side. cljs-time. rather than clj-time.


(defn datetimes->json [m]                                                       (1)
  (let [f (fn [[k v]]
            (if (instance? goog.date.Date v)                                    (2)
              [k (c/to-string v)]
              [k v]))]
    (clojure.walk/postwalk (fn [x] (if (map? x) (into {} (map f x)) x)) m)))


;; AJAX/HTTP Utils

(defn resp->view [resp]                                                         (3)
  (-> resp
      (update-in [:headers] #(keywordize-keys %))
      (assoc-in [:body] (-> resp datetimeify :body))))                          (4)

(defn view->req [params]                                                        (5)
  (-> params
      datetimes->json))                                                         (6)
1 Function that traverses a nested map and converts from DateTime to ISO-8601
2 Almost an instanceOf check to decide if the value is eligible for coercion
3 Handy function to transform an ajax response to something appropriate for use in our client side logic
4 datetimeify is identical to our server side impl
5 Handy function to take a map, typically request params, and transform to something appropriate for communication with a backend server. If you are using something like cljs-http it might be appropriate to hook it in as a middleware.
6 Coerce any DateTime values to ISO-8601 date strings
What about timezones on the client ? The default for the datetime constructor in cljs-time is to use UTC. So when displaying time and/or accepting date with time input from the client you need to convert to/from the appropriate timezone.
(ns acme.client
  (:require [cljs-time.format :as f]
            [cljs-time.core :as t]))


(def sample (t/now)) ;; lets say 2015-03-27T00:53:38.950Z


(->> sample
     t/to-default-time-zone                          ; UTC+1 for me
     (f/unparse (f/formatter "dd.MM.yyyy hh:mm")))   ; => 27.03.2015 01:53

Summary

Using clojure protocols we managed to simplify reading and writing date(times) to the database. Protocols also helped us serialize date(times) to json. For reading json we had to hack it a little bit. By using fairly similar libs for dates on both the client and our server apps we managed to reuse quite a bit. In addition We have reasonable control of where we need to compensate for timezones. Most importantly though, our server-side and client-side logic can work consistently with a sensible and powerful date implementation.

Permalink

Trying Clojure again?

EDIT: Indeed, as people on Reddit pointed out, installing Lein is simply downloading a script and running it. Installing CIDER from MELPA was also easy. The two, however, aren’t completely compatible at the moment because CIDER from MELPA wants nREPL 0.2.7 at least and Lein only pulls in 0.2.6 (even though, I believe the current is 0.2.10).

It has been five years since I last tried Clojure. I feel like I should try it again.

I don’t want to beat my head against Leiningen for even ten minutes this time. Is there some way to reasonably use Emacs + Clojure without have to install and configure CLASSPATHS and Mavens and Ants and JDKs?

It seems SWANK-CLOJURE has been deprecated in favor of CIDER. The CIDER doc says how to configure Leiningen or Boot for use with CIDER. Is there some way that I can avoid Leingingen and Boot? Or some way that I can click one ‘Install’ button and have Leiningen and Boot work?

Permalink

Clojure Weekly, Mar 26th, 2015

Welcome to another issue of Clojure Weekly! Here I collect a few links, normally 4/5 urls, pointing at articles, docs, screencasts, podcasts and anything else that attracts my attention in the clojure-sphere. I add a small comment so you can decide if you want to look at the whole thing or not. I also maintain a similar Haskell Weekly if you’re so inclined. That’s it, enjoy!

Tail Recursion In Clojure | 8th Light

Interesting write-up. Loop-recur is implemented in Java inside Clojure, this blog post is a nice step-by-step guide about how to obtain similar results in macro land. It’s making use of sophisticated techniques like walking the AST of the body of the passed in expression (with Riddley). Similar to what happens with core.async, yet another example of the power of macros to hide the state manipulation necessary to achieve a stack-free recursion.

Hindley-Milner in Clojure | LispCast

That’s an awesome exercise! Some typed languages offer type inference, a way for the compiler to infer the type of an expression just by looking at type declaration in other places. Languages like Haskell or Scala has that. In this small project (on top of big concepts) Eric Normand is implementing the widespread algorithm W for type inference on top of a small typed language (a Clojure DSL) all written in Clojure. Why that’s useful? Because it helps understand problems that typed language are trying to solve bridging the gap between dynamically and statically typed languages.

Stop Treading Water: Learning to Learn by Edward Kmett

Enjoyable keynote by Edward Kmett of Lenses fame (but not only). Kmett is a prominent Haskell expert, well known for his hundred libraries on github, but this talk is not about Haskell. Main take away is to not run away in front of complexity, but just find a way to effectively learn difficult topics. The danger is to spend your life solving problems with the wrong tool. He’s also warning us about jargon when used to create closed circles and always try to unroll complex terminology for people who don’t necessarily know. This is true for some fundamental functional programming aspects especially when they are close to mathematics.

Sisal Lives!

Exactly like in startups, it’s unlikely you’ll find new ideas in programming languages any time soon. Sisal (Streams and Iteration in a Single Assignment Language) is cited on some articles as a peculiar example of applicative language that is not functional (no high order functions). More importantly it compiles down to a dataflow representation that is implicitly parallel. Grab the tutorial (http://www2.cmp.uea.ac.uk/~jrwg/Sisal/00.Contents.html) if you are so inclined. But not sure the implementation linked here is still working.

take-while - clojure.core | ClojureDocs

And related cousin drop-while are just what the name says, a walk through a collection until some condition is met. It answers the typical procedural view of the problem about how to map a sequence and “return” at some point. There is no return in Clojure in the sense of jumping away from the current execution flow. That is typical of imperative languages and with functional languages you can get away with lazyness.

Permalink

Copyright © 2009, Planet Clojure. No rights reserved.
Planet Clojure is maintained by Baishamapayan Ghose.
Clojure and the Clojure logo are Copyright © 2008-2009, Rich Hickey.
Theme by Brajeshwar.