Mnemonics-based keymap for IntelliJ IDEA

Emacs is great for editing text, IntelliJ IDEA is great for navigation.
Let’s take best from both worlds into one editor.

Rationale

If you can configure it, you will configure it
- Anonymous, about Linux

Emacs is keyboard-driven editor, meaning all actions you can do without mouse or touchpad. It is a good part. The bad part is it lacks rich navigation capabilities, integration with external tools, code aware completion, etc. Well, you still can try hundred of plugins and configure exactly what you need, or try tens of prebuild emacs-based distributions with batteries included, but IntelliJ IDEA just works out of the box.

IDEA has most things that missing in Emacs and still considered among many as the best IDE for Java.
Even more, Cursive extension for Clojure works really well.

The most annoying things in IDEA is an editor and interaction with tool windows, I mean you need mouse. Even using emacs-based keymap, you will click mouse/touchpad extensively. Your fingers will be screaming. Especially, if you work on the latest MacBook Pro.

This is an attempt to develop mnemonics-based keymap (a.k.a Emacs style) for IntelliJ, to reduce mouse interaction to minimum, while providing a simple way to remember the hotkeys.

Preparation

I’m a happy user of MacBook Pro 13’’ (2017). Really, I find MacOS still the best system for people and devs, but this new keyboard drive me crazy.

Ironically, Escape button escaped to virtual world, so you can’t get tactile feedback from it. Apple provided a discount, so you got four arrows in the price place of three. Greedy backspace, which steal all your home row attention if you want just delete a char. Small vertical enter.

I can continue ranting about non-consistent sound from different keypresses, but thanks to this blog I remapped most of this keys, and happy so far.

Shortly, CapsLock remapped to Control (do you know people who use CapsLock?), then Control-IJKL become normal arrows, Control-H as a backspace (hello Emacs users), Control-M and Control-N work in pair as a horizontal Enter, and the same way Control-[ and Control-] work like Escape. Also, I don’t use right Command, so it also became Control, now you can use arrows with one hand while… eating or so.

This may sound strange configuration, but for me it works like a charm.
It took approximately two weeks to get used to new keyboard mapping.
Below, I list all hotkeys.

Basic hotkeys (C- prefix)

Basic hotkeys triggered with Control prefix. (C-a means press “A” while holding Control key)
This category exposes basic keybindings for navigating and editing text.

C-a - move cursor to the start of line

“A” is always beginning of something, line in this case.

C-b - move cursor one word backward

“B” for backward. For long sentences it’s faster than arrows.

C-c - prefix, reserved for Code mode

C-d - delete character (forward)

“D” for delete. Because, my MacBook keyboard doesn’t have delete button, only backspace.

C-e - move cursor to the end of line

“E” for end. Opposite to “A”.

C-f - move cursor one word forward

“F” for forward. Opposite to “B”.

C-g - IDEA Escape

IDEA Escape used in different context, mostly meaning stop input or action.
Stop selection, stop search, close popups, etc.
Mnemonic is stop looking for “G” spot.

C-h - delete character (backward, a.k.a Backspace)

It was a hotkey for backspace in emacs, don’t know why…
In our case it’s close to J key, which represent left arrow, so instead of moving cursor left, you slide index finger to the left, meaning, move cursor and delete the char.

C-i/C-j/C-k/C-l - arrows up/left/down/right

Look at the keyboard, how nicely ijkl combination represent the arrows, like wasd for gamers.
And almost like hjkl for vimers.

C-m/C-n - enter or return

In home row position your right index finger on J button, just slide a little down and you will get to M. Or N. Works for both cases.

C-r - replace in current file

“R” for replace. Also note, that while replace top dialog is available you have additional contextual hotkeys

  • C-m - (enter) replace occurence and go to next
  • C-k - (arrow down) skip and go to next occurence
  • C-i - (arrow up) skip and go to previous occurence
  • C-g - (escape) exit from replace dialog

C-s - search in current file

“S” for Search (and Solr). Similar to replace dialog, some contextual hotkeys available

  • C-m or C-k - (enter or arrow down) go to next occurence
  • C-i - (arrow up) skip and go to previous occurence
  • C-g - (escape) exit from replace dialog

C-t - prefix for Tool Mode

C-w - cut

Like C-x for normal people. I don’t how what mnemonic here, just used it since emacs times.
Maybe, “W” looks like a saw. Or two pairs of scissors.

C-x - prefix for Execute mode

C-y - paste

Like C-v for normal people. “Y” stands for “yanking” in emacs. Don’t know what it means.

C-z - undo

Good old C-z. It’s like time machine, if everything is fucked up, you press it to go back in time.

C-= - extend selection

= located on the same key, where + is. Plus means to enlarge something.
Smart selection is my #1 used feature in IDEA.

C-- - shrink selection

- is an opposite to +, so if there is a way to enlarge something, should be an option to shrink it.
Not as popular as extend selection, but if you did one extra selection, there is a way to go back, rather than starting from beginning.

C-] / C-[ - escape

IDEA has it’s own escape C-g but it doesn’t work for all cases.
You still need real global escape. And better if you can touch it.

C-. - completion

For me as for java-programmer-in-the-past, . has a special meaning - access members of object. IDEA often in this case automatically trigger code completion. So yes, . is for completion.

C-SPACE - start/end sticky selection

Alternative Keys (C-M-)

This category slightly changes the functionality of basic hotkeys.
Hold both Control and Meta key (Command for MacOS) and you have new set of keybindings.
Semantic meaning of such hotkeys stays the same, but adds some features or works in a different context.

C-M-a - move cursor to the start of the file

C-a move cursor to start of the line, with meta it moves to the start of the file.

C-M-b - move cursor one function backward

C-b move cursor one word backward, with meta it moves one function backward.

C-M-d - delete line from cursor to the end of line

C-d delete one character backward, with meta it deletes the rest of the line in the same direction.
Note: for most programs (including terminal) this functionality available with hotkey C-k, but C-k is global arrow down for us. We resolved this conflict by global remapping C-M-d to C-k. You can use C-M-d across all applications which support C-k.

C-M-e - move cursor to the end of the file

C-e move cursor to end of the line, with meta it moves to the end of the file.

C-M-f - move cursor one function forward

C-f move cursor one word forward, with meta it moves one function forward.

C-M-h - delete line from the cursor to the start of line

C-h delete one character forward, with meta it deletes the rest of the line in the same direction.
Never used.

C-M-i/C-M-j/C-M-k/C-M-l - text movement

C-ijkl move cursor in four direction, with meta it moves line or selection in four directions. Left/right directions works like a Tab/Shift-Tab (indent/unindent)

C-M-m/C-M-n - alternative enter

C-m/C-n powered by Karabiner and replace default enter fucntionality.

Alternative enter available in Idea in various context. For example, pressing enter on project view file, open this file in editor, but doesn’t switch to it. Alternative enter switch to the file. Also, pressing enter on unfinished input in REPL, adds newline. If you press alternative enter instead, whole form will be sent for evaluation.

C-M-r - replace globally

C-r replace occurrences in current file, with meta it replace across whole project/module. While in this dialog use arrows to navigate through occurences and press Opt-R ro replace. (Don’t like Opt-R, but there is no way to override it in Idea)

C-M-s - search globally

C-s search for occurrence in current file, with meta it search across whole project/module.

C-M-z - redo

C-z undo the action, with meta you can redo it it.

C-M-[/C-M-] - escape from tool window and close it

One usage of escape button is to switch from the tool window back to editor. Tool window stil remains visible. If you want to hide tool window press extra meta key (will be covered more in Tool Mode section)

Code Mode (C-c prefix)

Code Mode provide hotkeys (prefixed with C-c) to work with the code.

C-c a - git annotate

When you find shitty piece of code, you want to know who added it. Blame him.
Idea call it A - Annotate.

C-c b - move cursor to matching brace

B for Brace. Navigating to matching brace helps you to add something before or after form.
Also, nice to see your parens are balanced.

C-c c - comment

Most of the time is simpler to comment by adding ; in front of the line, or #_ if you want to comment whole form. But when cursor is somewhere in the middle of line, this hotkey, comment the line and jumps to the next one. Handy. C obviously for comment.

C-c d - duplicate line

DRY principle stands for Don’t Repeat Yourself. I doubt it is true for developers, copy/paste it’s all we have.
D for Duplicate or Do Repeat Yourself.

C-c e - jump to next error

Programming without errors is not a programming. Jump to next error.

C-c C-e - error description

After you jumped to error, you need to get description of what’s wrong. Usually people do this by hovering mouse over the error, but we have a keyboard.

C-c f - find current file in project view

To see context of directory or package you are working in, reveal the file in project view.

C-c h - inline help

Call inline documentation of the function, if you forgot the order of arguments or want to make sure it does exactly what you need. H for Help. Escape closes popup.

C-c i - indent code

Copypasted code from somewhere else and got broken indentation? Press “I” to indent.

C-c j - ace jump

Jump. It’s a special keybinding prefix powered by acejump plugin. Type this prefix, type some char and acejump will highlight all occurences of this char in the visible editor. Use hints to move to specific place.
Poor man’s replacement for the mouse “click here”.

C-c k - kill whole line

Kill current line, no matter where you at.

C-c l - go to line

Move to the line. Generally used if you got an exception with extra on what line error occurs.

C-c m - list methods

Correct name for this command is “show file structure”, but you mostly working with the code, so file structure are methods and variables. M methods.

C-c C-m/C-c C-n - IDEA show intentions

Ideal way to perform imports, requires and other contextual suggestions.
Mnemonic is “Code (C-c) Enter (C-m, C-n)”.

C-c r - refactor menu

R for refactor, very powerful IDEA feature, but limited in clojure. Hopefully, could be improved to contain more refactoring items.

C-c s - go to source

Exciting code navigating feature. Press “S” (source), to go to the function definition, either current project or external library.

C-c t - transpose forms

If you misplaced the arguments to the function, press “T” (transpose) to swap them. Works on forms as well, current form will be swapped with the left.

C-c u - find usages

U for usages. Prior to changing function or its implementation you want to know what pieces it affects. Must have feature for any editor.

C-c w - wrap selection

Select piece of code, and wrap it to something. try-catch block is the most popular choice, but you can add your own snippets.

C-c - - collapse

Long function can be collapsed into oneliner. Minus sign means reduce visible part of text.

C-c = - expand

Collapsed function, can be expanded to normal view again. As already mentioned, = sign located on the button with +, which means enlarge.

C-c C-- - collapse all

Alternatively to collapse, there is posibility to collapse all functions. Hide all, open one and focus on it. But if your namespace overloaded with a lot of functions think about better way to focus on one function.

C-c C-= - expand all

Similarly, there is expand all keyboard, bring everything to normal.

Execute mode (C-x prefix)

Execute mode provide hotkeys to interact with REPL and editor window.

C-x b - set bookmark

Save frequently-visited location in the code to bookmark.

C-x C-b - show bookmarks

Show all saved bookmarks and navigate/edit them.

C-x d - show debug menu

D for Debug. But remember, best debug are print statements.

C-x e - evaluate form in REPL

Eval is most important hotkey for REPL driven development.
So important so C-x C-e does the same.

C-x f - go to file

Old school emacs hotkey to open file, C-x C-f does the same.

C-x g - show git dialog

Opens git dialog. Not keyboard friendly and not as powerful as magit, but just in case you use IntelliJ git client. I am totally fine with console tig tool, though simple magit port for IDEA would be good.

C-x C-h - Clear REPL output

C-h is backspace in our world, and backspace means to clear.

C-x i - interrupt

If function runs too long, press I to interrupt.

C-x k - kill window

Kill editor window, when it’s not needed anymore.

C-x l - load file in REPL

Load file in REPL. Sync all forms between file and REPL environment.
Very important file of the flow: change file -> load to REPL -> repeat

C-x m - start/stop macro recording

Not as powerful macro capabilities as in emacs, but still there is a way to record routine actions.
The same hotkeys start and stops macro recording. After macro is recorded, you can provide a name for it, to make it persistent.
M-m plays the last macro.

C-x n - set current file to namespace in REPL

Working in user namespace is often error-prone, when loaded function names mixed in one environment.
Use namespaces for REPL instead. The same as (in-ns)

C-x o - go to next split window

Emacs allows to open and arrange several buffers in one screen. Similarly, IDEA allows to view multiple editors at once, C-x o cycles to next visible editor. I always working in one editor, sometimes open two to copy chunks or compare files. Therefore, O means other.

C-x C-o - Open file…

IDEA is project-based editor, but there is a way to open external files for quick edit. Don’t know why.

C-x q - stop REPL

Sometimes, you will need to restart REPL to refresh dependenies, cleanup environment, etc.
Just quit current REPL and start new one using C-x x.

C-x s - search REPL history

Just search.

C-x t - Run test under cursor

TDD adepts will love that.

C-x C-t - Run all tests in current namespace

When feature is done, run all tests for it.

C-x C-j - navigate backward

Go through history of files and places which has been edited.
C-j is a left arrow, remember?

C-x C-l - navigate forward

Similarly, go forward through history of edited files.

C-x x - show run menu

X for execute. Double X for Execute program using execute mode.

C-x 0 - close current split editor

The same as C-x k, good to have consistency there. 0 means nothing.

C-x 1 - close all split editors except current

Leave only current editor. 1 and only one.

C-x 2 - split editor verically

A way to add more editors to the view.

C-x 3 - split editor horizontally

Another way to add more editors to the view.

C-x C-= - Increase font size

Again, = located on the same button as +.

C-x C-- - Decrease font size

Decrease font size. - opposite to +.

C-x C-0 - Set font size to default

Good to have default one.

Tool mode (C-t prefix)

Tool mode allows to manage IDEA tool windows.

C-t c - open TERMINAL tool window

C for console. Generally not used, but when you switch often between terminal and edited file, could be handy.

C-t d - open DEBUG tool window

IDEA Debug has a lot of features, when you debugging Java code, which could be another topic.
I use sometimes Opt-I (step in), Opt-O (step-out), Opt-N (next statement) and Opt-E (valuate expression). Because keys like F8 are completely meaningless (and not available in MacBook Pro)

C-t e - open EVENT LOG tool window

Event log holds some information about IDEA system exceptions and compilation.
Not really useful when you have REPL.

C-t f - maximize tool window

F for Fullscreen.

C-t g - open GIT tool window

Another place for Git. Quick look at what has been edited.

C-t h - Hide all windows (except editor)

H for Hide. Also C-h is backspace which has meaning delete, tool windows in this context.

C-t C-i/C-j/C-k/C-l - Adjust Size of tool window

Arrows naturally shows the direction in which tool window will be extended.

C-t l - open LEININGEN tool window

L is lein. Shows dependency tree and tasks available. Also lein tasks could be run from this view, what some people find handy.
I still prefer using lein from terminal.

C-t m - open MAVEN tool window

The same for Maven.

C-t o - open REPL OUTPUT tool window

Jump to the REPL output.

C-t p - open PROJECT tool window

P. Open project view.

C-t q - Quit tool window

Tool windows which are not needed anymore should be closed to free space.

C-t r - open REPL tool window

All you need is REPL. REPL is all you need.

C-t s - open SEARCH tool window

Back to search view, where find usages C-c u was submitted.

C-t t - open TODO tool window

Who else uses TODO list to organize their work?

C-t x - open RUN tool window

Mostly used in Java. When you executed program, shows it’s output.

Popular keybindings (M-)

Popular hotkeys has no specific topic, but due to its frequent usage they belong to one group. M- does not require prefix, which means, they will be even faster than other hotkeys.

M-a - select all

Select all is a frequent action, for example copy configuration from one file to another.

M-b - Paredit Barf

Structural editing for Clojure, moves last form out of the current form.

M-c - Copy

Just plain copy.

M-d - Delete whitespaces

Delete whitespaces (including newlines). Like C-d but applied continuosly to the first non-whitespace symbol.

M-e - Recently edited files

Shows menu with recently edited files and tool windows.
Poor replacement for emacs ibuffer.

M-f - Move forward into SEXP on the same level

Structural movement on the forms at one level. Kinda useful.

M-i - Interrupt evaluation

Already discussed, available at hotkey C-x i. I so often evaluate lazy sequence in REPL, so hotkey promoted to the popular ones.

M-j - Jump

The same, available in code mode C-c j, but if used extensively extra prefix makes it slower.

M-k - open symbol

If you remember name of the function or var, but forgot namespace it belongs to, use this. K has no special meaning here, just available no prefix hotkey.

M-l - Load file into REPL

Promoted from C-x l

M-m - Play last macro

We have already seen how you can record a macro using C-x m. Playback just applies it. Unfortunately, there is no C-u modifier like in emacs to repeat macro n times. But you can press M-m n times.

M-n - open namespace

Used mostly when working with clojure code, otherwise could be superceded by open file C-x C-f.

M-o - REPL output

Promoted from C-t o

M-p - project view

The same as C-t p

M-r - REPL

No questions why REPL is so popular, promoted C-t r.

M-s - Paredit Slurp

Structural editing for Clojure. Awesome feature.

M-t - Run Tests

Promoted from C-x C-t

M-u - Uber menu

Open context menu. If some action can not be achieved using hotkey, you still have the context menu with all options available.

M-v/M-w - Paste

One from MacOS, one from emacs, so I added them both.

M-x - Enter action

Honestly, there is no need to have a hotkey for every action. It makes sense to add hotkey if it has mnemonic, so you do not spend extra cost remebering it, or its really popular. Otherwise all IDEA actions available to you under M-x, execute action.

M-, - IDEA Settings

Open IDEA settings. I doubt you can do something there without a mouse.

Conclusion

I guess everyone has its own way of coding, favorite editor and set of plugins. That is fine, and I am not going to convince you this is better way of doing that yours. All I wanted to share, that despite many years of clojure in emacs, I found IDEA with Cursive smart and very promising development environment for clojurists. (maybe, not so extensible)

I still use emacs and cider sometimes for simple clojure projects, org-mode, markdown editor, notes, default terminal editor and ocassional development on other languages.

And lastly, in case if you interested in the same settings, idea_keys.jar

Permalink

PurelyFunctional.tv Newsletter 293: Design, deps.edn, Type Inference

Issue 293 – September 24, 2018 · Archives · Subscribe

Hi Clojurers,

I have an important announcement. Though it pains me to say it, there will be no Clojure SYNC 2019. There are more details here.

Rock on!
Eric Normand <eric@purelyfunctional.tv>

PS Want to get this in your email? Subscribe!

Hire Eric to train your team

I have a couple of spots open for client work. I’m currently offering two specialized services. These are open for remote or on-site:

Clojure Kickstart is training to get your team of programmers productive in Clojure. Get the skills, the tooling, and the workflow you need to make the most of Clojure.

After the Prototype is where I and your team workshop a powerful core abstraction and come up with a plan for refactoring it into your codebase.

If you or someone you know are interested, please reply to this email.

The emperor’s old clothes Paper

Tony Hoare’s Turing Award Lecture from 1980.

Select quotes:

I have regarded it as the highest goal of programming language design to enable good ideas to be elegantly expressed.

Even in those days and even with such a simple language (ALGOL 60), we recognized that a subset could be an improvement on the original.

But I was not satisfied. I did not see why the design and implementation of an operating system should be so much more difficult than that of a compiler.

If our basic tool, the language in which we design and code our programs, is also complicated, the language itself becomes part of the problem rather than part of its solution.

The price of reliability is the pursuit of the utmost simplicity. It is a price which the very rich find most hard to pay.

Announcement: No Clojure SYNC in 2019

I’m sorry to say it, but it won’t be happening. This post talks about why, and the future of Clojure SYNC.

Embracing Simpler Tools YouTube

David Nolen talks about the relatively new Clojure command line tooling and deps.edn in the context of using simple tools.

Why are actions hard to test by definition? Podcast

As part of writing my book, I was coming up with a good definition of actions (also known as effects in some circles). Actions are notoriously harder to test than calculations (aka, pure functions), and I realized that the definition could explain why.

Cult member echo chamber FTW Podcast

Dmitri Sotnikov on the defn podcast. Definitely a great discussion.

ClojureScript type inference, Graal, and Clojurists Together Podcast

Mike Fikes interviewed on The REPL podcast. Great discussion.

Practical Object-Oriented Design: An Agile Primer Using Ruby Book

While I’ve never programmed in Ruby, I have always been a fan of Sandi Metz’s talks. After seeing her speak at OSCON this year, I dove into her book.

I’ve always been skeptical of the term “design” applied to code. For instance, as it is applied in “Design Patterns”. The first chapter of this book addresses the question in a way that I’ve never seen before. Design, in this context, is all about developmental concerns (as opposed to functional or operational concerns). It’s not about correctness. It’s about how easy it is to make your code correct, especially as requirements change and you learn more about the domain.

The book shows lots of manipulations you can make to existing code to make responding to changing requirements easier. It all looks great. Then it ends with testing, which seems a little scarier since it shows tests that are correct and passing, but that could easily become incorrect and still pass. It really makes it difficult to believe in the magic of TDD after reading that. Even with a full test suite, you still need to spend time designing your tests.

It was a good book, well worth a read. Sandi Metz is one of a handful of Object-Oriented aficionados who were forged in the days of Smalltalk. That experience gave her a much different perspective than we often see from those who grew up with Java, C#, or C++.

In the end, I was left wondering: can incremental refactorings, guided by simple design principles, make serious changes in your code? Sure, it can clean it up, but would you ever be guided to change a core abstraction into a superior one? For example, if my calculator software were written using roman numerals internally, would these design principles and refactorings lead me to discover decimal numbers?

If so, this is an amazing advancement for human knowledge. Write software, keep adding features, and keep refactoring it incrementally to make adding those features easier, until you’ve found a better abstraction. We could apply this in any field of human knowledge. There’s actually some evidence for this. For example, Gerald Sussman has shown how programming notation can clarify physics notation. Or a program can reveal corner cases in a business process. These anecdotes are in the right direction, although these aren’t completely new abstractions, merely better representations of them.

The possibility remains: maybe you can’t follow design principles to come up with better abstractions. In that case, there must be something parallel to manipulating the code. We can, of course, refactor code to replace an abstraction, but we must already have that new abstraction in mind. Without the new abstraction, refactoring is just moving code around. Design principles can tell you where to put the code for calculating epicycles. But can it put the sun at the center of the system? Imagine how “clean” your code would be after getting rid of those epicycle methods!

Like I said, it’s a good book. It made me think. And I’d love to have a similar book in the functional programming community.

Clojure Collections Currently recording

I’m back to recording this course that is all about the heart of the Clojure language: Clojure’s persistent collections.

They’re more than just fast implementations of useful data structures. They also capture essential access patterns in terms of Java Interfaces (or, in the case of ClojureScript, Clojure protocols). I’ve finished recording all of those access patterns. Learning these patterns is essential to coding in Clojure, and I wish I had known these when I first started.

I’ve also started in on recording the usage patterns. These are the common ways that the collections are used in everyday Clojure code. These are actually harder to identify. I’ve found and named a bunch, but I’m sure there are more.

New lessons this week

The post PurelyFunctional.tv Newsletter 293: Design, deps.edn, Type Inference appeared first on PurelyFunctional.tv.

Permalink

How to create a habit of reuse

In OO languages like Java, people tend to make new classes more than they reuse existing ones. In Clojure and other FP languages, we tend more on the side of reuse. How do we develop that habit?

Transcript

Eric Normand: How do you create a habit of reuse instead of making new stuff? Hi, my name is Eric Normand and these are my thoughts on functional programming.

In functional programming, at least the way I’m used to seeing it, we tend to reuse pieces a lot more than in typical object-oriented programming. Why is that? I don’t think it has anything to do with the paradigm or the language. I think it has a lot more to do with the habits, habits that the programmers have in those various communities.

I’ll give an example. Actually, Rich Hickey gives a really good talk where he’s talking about how, in the Java world, there’s a thing called servlets. The servlet defines how to process an HTTP request. There’s a class called HttpRequest, something like that. It defines all the stuff that you would expect in an HTTP request.

He puts the slide up and the slide has all the methods in the class, just their signatures, and it’s pretty clear what they do. They’re well named, but he says, what are the hashmaps? Where are all the hashmaps? It’s kind of a weird question. You’re not used to asking that, but sometimes things are pretty clearly hashmapping.

There was a method called getHeader. It took a string and it returned a string. It’s kind of like getting out of a hashmap. It’s got a key and a value. Then there was also getProperty, and you would pass in a string and get a string back. If you could do getPropertyNames, it would give you a list of all of the property names that were defined on that request.

You could see, yeah, these are hashmaps. Why did they define their own custom little protocol for that? Then he’s like, “But look, there’s actually deeper hashmaps that you don’t think about.” Because all these other things that are just defined methods on the class, they’re just getters, so they’re just stuff like getPort, and getHost, and getIP.

Why not just make those key? Those are just the keys, the IP, the host, the port. The whole thing should be a hashmap and these are the keys. What he was getting at was that in Clojure, we have this thing called Ring that defines the hashmap format for representing an HTTP request.

That’s really cool because it means that you don’t need a new type. You just need this new spec for what goes into the hashmap. Everyone already knows how to use a hashmap.

I mean, if you’re a Clojure programmer using hashmaps, and there’s all sorts of functions for dealing with them already, and they can be serialized. Basically, we’re getting all this reuse out of hashmaps. When in Java, they have hashmaps. They could have done that.

Instead, they chose to define a new class that required more documentation, required re-implementing a lot of this functionality. Maybe it even uses the hashmap internally to represent those headers, they might, it might. The whole point was that in Clojure, this is what we do. We reuse. It looks like a hashmap, just use a hashmap. Why would you create a new type just for that?

In Java, they don’t do that. To me, it’s just a habit. Now, how do you develop that habit? That was the question I started with. The habit is really about understanding the two parts. You have to understand, number one, what you’ve got already. What are the things you’ve got that you could reuse? You’ve got to understand your data types.

You got to know you got vectors, you got hashmaps, you got sets, you got your sequence abstractions. You got all sorts of things that you got to learn and have indexed. You have to have these data types indexed by their access patterns.

If you look at a thing and it says it’s a getProperty, and it takes a string and it returns a string, now you got to be thinking, “Well, that’s kind of a key value thing. That’s probably a hashmap.” You have to have that automatically.

The second thing that you have to be doing, just constantly be accessing that index. Instead of thinking, “What new thing do I need to make?” You should be thinking, “What thing can I be reusing?” Then you start to look at the thing in terms of these access patterns.

The access patterns are going to depend on the language. Clojure defines a number of them. They’re standard interfaces that come with the language. They include stuff like the sequence abstraction, which is how you access items one at a time, whether you’re going to be remembering duplicates. That’s an access pattern.

There’s accessing a value, given the key. It’s adding stuff to a collection. Where are you going to add it? You can add to the front, to the back, if it maintains order. These are all the types of things you have to be thinking about your data structures.

You can’t just think of…Unfortunately, I was taught this way. Like a list — that’s just an ordered collection, but it’s not quite. To think of it that way is missing a little bit of the important information. I was taught Java and you have a list interface and it has these methods on it.

One of the methods is like get and you give it an index, an integer, and it’ll give you the item at that index, which sounds reasonable. The problem is that when you implement that interface, get might change algorithmic complexity.

If you have an array list, which is another type in Java, it can quickly give you the value based on the index because it’s implemented with a big array. Arrays can do that random access into the array.

If you use the linked list, which is another type in Java, another class, then you can’t do that. You have to actually iterate through the list, counting how many you’ve seen and then return the last one, the one at that index, so it’s linear as the list gets bigger.

To say that they are both implementing the same interface, it’s kind of a lie because your algorithm could go from constant time to linear, or worse, it could go from linear to quadratic. It’s accidentally quadratic just because you didn’t realize that that was going to be accessed.

What Clojure has done is to a large extent — it’s not perfect at this because you can still get an element at an index — it just uses a different function from get. If you’re in a linked list you can do nth, but nth is known to be potentially linear, whereas get on a list doesn’t work, but get on a vector does work because it’s constant time.

These operations, part of the contract is that they maintain algorithmic complexity. When you implement those interfaces, part of the contract is you shouldn’t be implementing this if you cannot be constant time.

Like I said, to a large extent, it’s like 90 percent, this is true. There are some exceptions. They’re unfortunate but to a large extent, I believe that Clojure does it right. These are the things you need to be thinking about.

If you’re a programmer, you’re doing anything at any kind of reasonable scale, you have to start thinking about the algorithmic complexity, and it should be part of the interface. Those are the things that need to be the first choice you make. How am I going to access this stuff? Then, of course, that leads to this kind of reuse.

Why re-implement the hashmap or even wrap it up if I’m going to have to define all these new methods? Why not just use what’s already there?

My name is Eric Normand. You can reach me on Twitter. I’m @ericnormand with a D. You can also email me eric@lispcast.com. I hope to hear from you because I love getting into discussions with people. Awesome. Rock on.

The post How to create a habit of reuse appeared first on LispCast.

Permalink

Just Juxt #41: Tricky card games (4clojure #141)

Cards

In trick-taking card games such as bridge, spades, or hearts, cards are played in groups known as "tricks" - each player plays a single card, in order; the first player is said to "lead" to the trick. After all players have played, one card is said to have "won" the trick. How the winner is determined will vary by game, but generally the winner is the highest card played in the suit that was led. Sometimes (again varying by game), a particular suit will be designated "trump", meaning that its cards are more powerful than any others: if there is a trump suit, and any trumps are played, then the highest trump wins regardless of what was led.

Your goal is to devise a function that can determine which of a number of cards has won a trick. You should accept a trump suit, and return a function winner. Winner will be called on a sequence of cards, and should return the one which wins the trick. Cards will be represented in the format returned by Problem 128, Recognize Playing Cards: a hash-map of :suit and a numeric :rank. Cards with a larger rank are stronger.

(ns live.test
  (:require [cljs.test :refer-macros [deftest is run-tests]]))
  
(defn trick [t]
  (fn [[l :as c]]
    (last (sort-by (juxt #(= (:suit %) t)
                         #(= (:suit %) (:suit l))
                         :rank) c))))

(deftest trick-test
  (is (= {:suit :club :rank 9}  ((trick nil) [{:suit :club :rank 4}
                                           {:suit :club :rank 9}])))
  (is (= {:suit :spade :rank 2} ((trick nil) [{:suit :spade :rank 2}
                                           {:suit :club :rank 10}])))
  (is (= {:suit :club :rank 10} ((trick :club) [{:suit :spade :rank 2}
                                       {:suit :club :rank 10}])))
  (is (= {:suit :heart :rank 8}
   ((trick :heart) [{:suit :heart :rank 6} {:suit :heart :rank 8}
                 {:suit :diamond :rank 10} {:suit :heart :rank 4}]))))
         
(run-tests)

Permalink

Announcement: No Clojure SYNC in 2019

Well, it’s sad but I have to announce it: I won’t be running Clojure SYNC in 2019.

Clojure SYNC in 2018 was the first one. It was a success. I got lots of good reviews about it. And it was mildly profitable, which I hear is difficult in a conference’s first year. We had an epic-level speaker, Gerald Sussman, and an important new feature announcement from David Nolen, cljs.main.

I was excited to make next year’s even better. However, I don’t think I can. I’ve got too many other responsibilities and projects going on. I had to cut something, and Clojure SYNC makes the most sense to cut.

It’s not easy to cut it. It takes years to get a conference off the ground. You’re trying to ramp up awareness and interest, while also stabilizing the format and processes to make it easier each time. Cutting Clojure SYNC feels like throwing away the momentum I’ve built up after a first successful show. I also realize that the spot on the calendar that I claimed could easily be taken by some other conference. If it is, I hope it is in service of the community. If you want to run a conference, please be my guest!

But it does make sense to cut it. Although it was profitable, it was not as lucrative as I had hoped, and there are more lucrative things I should be spending my time on considering how much work it was. I also don’t think it will be possible to commit the time it will take to make it a super valuable conference to spend time and money to attend, and I wouldn’t want any less for my attendees.

So what’s the future?

After the first Clojure SYNC, I wanted to turn it into something that could get the community more organized. I wanted it to be a place you had to go because that’s where things got started. Instead of just talks, it would include a day of workshopping and discussion, where people with common interests could meet and organize around projects. The workshop would let them do the important hashing out of ideas that’s best done in person. They could then continue the projects back home. I still think it’s a valuable vision.

Will there be a Clojure SYNC 2020?

Well, it’s hard to look so far in the future, and I’m hesitant to commit now after promising and now reneging on 2019. But some advisors have told me that many events do better when they occur every two years. There’s a lot more anticipating and less competition for limited travel budgets. That gives me some hope that skipping a year is not a waste.

So, I can’t promise anything about 2020. But I do promise to let you know far in advance if Clojure SYNC 2020 will (or will not) happen.

Rock on!
Eric

The post Announcement: No Clojure SYNC in 2019 appeared first on Clojure SYNC.

Permalink

Just Juxt #40: Intervals (4clojure #171)

Intervals

Write a function that takes a sequence of integers and returns a sequence of "intervals". Each interval is a a vector of two integers, start and end, such that all integers between start and end (inclusive) are contained in the input sequence. 

(ns live.test
  (:require [cljs.test :refer-macros [deftest is run-tests]]))
  
(defn intervals [c]
  (let [s (set c)]
    (map (juxt first last)
      (take-nth 2 (partition-by #(contains? s %)
                     (and (seq s)
                       (range (apply min s) (inc (apply max s)))))))))

(deftest intervals-test
  (is (= (intervals [1 2 3]) [[1 3]]))
  (is (= (intervals [10 9 8 1 2 3]) [[1 3] [8 10]]))
  (is (= (intervals [1 1 1 1 1 1 1]) [[1 1]]))
  (is (= (intervals []) []))
  (is (= (intervals [19 4 17 1 3 10 2 13 13 2 16 4 2 15 13 9 6 14 2 11])
         [[1 4] [6 6] [9 11] [13 17] [19 19]])))
         
(run-tests)

Permalink

Senior Developer - Clojure / AI / NLP

Senior Developer - Clojure / AI / NLP

All Street | London, UK
An AI research assistant on every analyst's desktop for better, faster research.
£50000 - £80000

Background

New fintech company based in London, using Clojure and Clojurescript to create tools for financial analysts. We are developing AI-based automation tools to provide the equivalent of an assistant researcher at every analysts desktop.

We are growing the number of developers in London, and are looking for people with strong functional programming skills, especially Clojure. The ideal candidate will have been using Clojure commercially, although we will consider others with less Clojure experience, if you have other strong functional programming skills, and skills relevant to our wider technology stack and domain of interest.

Experience of any of the following would be an asset::

  • Clojure
  • ClojureScript
  • C#/.Net
  • Javascript
  • HTML5
  • Python
  • Natural Language Processing
  • Semantic Analysis of Text
  • Concept Extraction
  • Natural Language Generation
  • Machine Learning
  • Behavioural Analytics
  • Content Production Systems
  • Document Formatting

The Job

We are looking to hire a Senior Developer to work in our London office, and work within a small team of engineers, which includes some remote team-members. Our aim is to create a vibrant, productive working environment. We apply agile principles to our product development, and have adopted Scrum, Continuous Deployment and AWS-based Devops, with a view to delivering user-centric software which is iterated quickly to provide continuously increasing value to our customers.

The work is highly collaborative in nature, so the environment requires a high level of interpersonal trust and confident communication between team members.

While the core product is a web-based system written in Clojure, you will be exposed to advanced natural language processing and machine learning techniques.

The ideal candidate, will be deployment focused, with a high degree of curiosity, and with the ability to solve challenging problems and ask thoughtful questions, while designing and delivering simple and elegant solutions using functional programming.

You must be able to envision a variety of solutions and to clearly articulate them to colleagues, including non-technical stakeholders, and explain the trade-offs involved in choosing between options.

You must love working with others and sharing knowledge, to get the best result for the end-user and ultimately for the business.

Requirements

Ideally, the successful candidate will have:

  • At least five years experience of delivering software in a commercial environment
  • Demonstrable commercial experience of functional programming
  • Familiarity with Scrum
  • Interest in natural language processing and machine learning

Location

We are located in the City of London, where many of our target customers are based. This is also adjacent to Shoreditch and Silicon Roundabout, the home of Google's Campus London and Microsoft Reactor, which make it a vibrant area, full of like-minded developers and entrepreneurs.

Permalink

The easiest way to make your existing code more functional

What is the easiest way to make an existing functions more functional? It’s quite a simple technique and you can apply it today.

Transcript

Eric Normand: What is the easiest way to get started in functional programming and to make your existing code just a little bit more functional today?

Hi, my name is Eric Normand. These are my thoughts on functional programming. I think the easiest refactoring you can do right now is the following. Find a function that reads in global mutable variables.

You probably have a few and it’s being called somewhere. Make sure that this function doesn’t do any other actions. Remember, reading a global mutable variable is an action because it depends on when you run it, you’re going to get a different answer depending on when you read that value because it can change over time.

What you want to do is take that function. Make sure there’s no other actions in it besides reading global variables. Go through and find each time you read a global variable. Make that an argument to your function. Instead of reading from the global variable, you’re just going to read from the argument. You can even use the same name as the global variable. That’s fine.

You do that for all your reads of global variables, then you have to go to the functions that call that one. Instead of calling it with the old arguments, they are going to read the value from the global variables and pass those in. You’ve moved the responsibility of reading the global variables up one level of the calls. What you had before was an action that called an action.

You had some function that called this function that you just changed. Both of those were actions. The function that you just added all this arguments to, that one was an action because it read from global mutable state. The function that called that one was an action, because it called that one. Now, you’ve moved all the actions out of this function. Now, it’s a calculation.

Now, it is a pure function from arguments to return value. Therefore, you’ve made your total code base incrementally a little bit more functional. You have more code that’s in a calculation and less in actions. That means this function is now more testable because you can call it as many times as you want. You don’t have to setup these global variables in their environment.

Often, one of the reasons why these kinds of functions are so hard to test is because we don’t realize all the possible states that could exist in those global variables. There’s always code that could write to that anytime. We just don’t think of all the values. Now, you can test it a lot more thoroughly because it’s all self-contained.

I hope that helps you make your code a little bit more functional. You can always get in touch with me if you have any questions or any suggestions. I’m on Twitter @ericnormand. You can also email me anytime. I love getting into discussions. My email address is eric@lispcast.com. Thank you very much.

The post The easiest way to make your existing code more functional appeared first on LispCast.

Permalink

The Road to Typed Clojure 1.0: Part 1

What set of features would deserve a 1.0 release for Typed Clojure?

We’ve learnt valuable lessons from real-world developers about the pain points of using Typed Clojure, and after several years of mulling it over, we have a much better idea of how we might improve it. This series of posts will outline our proposed solutions, and give an impression of what we hope for Typed Clojure 1.0.

CircleCI has a great blog post outlining numerous issues with Typed Clojure in practice.

They boil down to:

  1. Type checking is too cumbersome to be interactive
  2. Hard to distinguish user-errors from type system shortcomings
  3. Not enough community support for library annotations
  4. Poor inference of higher-order function usage

I’ve already written about the limitations of the local type inference Typed Clojure uses. In this post, we’ll look at some interesting new directions in local type inference that will help type check more programs, with less local annotations, and more helpful error messages.

Improved Local Inference

When we call higher-order functions as programmers, we’re implicitly reasoning about the flow of values through and between functions. Can we automatically recover this information to help the type checker perform similar reasoning?

Let’s use map as an example. The control flow of the following expression

(map (fn [x] x) [1 2 3])

can be described in two steps.

  1. Call (fn [x] x) on elements of [1 2 3],
  2. return a sequence of those results.

In a picture:

             2.
             /---------------v
(map (fn [x] x) [1 2 3]) ;= [1 2 3]
          ^------/
                 1.

If we look at the type of map, could we recover the same information?

(ann map (All [a b]
           [[a -> b] (Seqable a) -> (Seqable b)]))

Yes. We notice the first argument has type [a -> b], and so requires a value of type a before it produces a value of type b. The second argument is of type (Seqable a), and is thus our only source of a’s. Let’s draw this first step.

(ann map (All [a b]
           [[a -> b] (Seqable a) -> (Seqable b)]))
             ^----------------/
                              1.

The second step involves collecting the b’s the function returns and returning a sequence of them. Since the only way to get a b is from the function argument, we can draw another arrow.

(ann map (All [a b]
                  2.
                  /--------------------------v
           [[a -> b] (Seqable a) -> (Seqable b)]))
             ^----------------/
                              1.

There are many potential ways this kind of control-flow information can help more type check Clojure code, and provide more informative error messages. Here are some ideas.

Infer local fn's

Take the following code, where f and g are unannotated anonymous functions like (fn [x] ...).

(let [h (comp f g)]
  (map h c))
;=> r

We can delay checking f and g until h is called, and wait for more type information from the map expression.

              v-\ 3.
(let [h (comp f g)]
              | ^
     2./--------/
       |    4.\--v
  (map h c)) ;=> r
       ^-/ 1.

There are 4 control flow edges above.

  1. Arguments flow from c to h.
  2. The same argument is forwarded to g.
  3. g’s result is passed to f.
  4. A sequence of f’s results returned as r.

All this can be justified just from the types of map and comp. Here’s how to derive the information from comp. The control flow of an expression ((comp f g) v) looks like:

       v----\ 2.
((comp f    g)    v) ;=> v'
       |    ^-----/ 1.   ^
     3.\-----------------/

Notice that without v, there is nothing to call g with, so it’s safe to assume (comp f g) by itself can never call g, and, by transitivity, f.

The type of comp has this exact information: the output of comp must be called before either comp argument is called.

(ann comp (All [a b c]
              v-------------\ 2.
            [[b -> c] [a -> b] -> [a -> c]]))
              |        ^-----------/ 1. ^
            3.\-------------------------/

In words, once a is provided:

  1. a flows to the second argument.
  2. b flows to the first argument.
  3. c flows to the return.

We can justify a delayed check of comp’s arguments because providing a “kicks it all off”.

Improved Error Messages

CircleCI’s blog post rightly complained that error messages often don’t provide enough guidance for a user to identify the cause(s) of an error.

By using the same control flow information as above, we can do a better job of explaining why some applications fail. For example, take

(comp inc boolean)

Typed Clojure currently reports an error which includes the types of comp, inc, and boolean, and leaves the user to diagnose the issue.

Instead, we can incorporate the control flow information into the error message like so:

Type Error:
Input to first argument of comp must accept the output of
the second argument of comp.

In the following diagram, `b` was inferred as Boolean from
the result of `boolean`, but it flowed to the input of `inc`,
which only accepts Number.


 [[b -> c] [a -> b] -> [a -> c]]
   ^             |
   \-------------/
in: (comp inc boolean)

Type check transducers

Transducers in Clojure are a frequent source of anonymous functions. Transducers “compose left-to-right”, so the following code first increments elements of c, then decrements.

(into []
      (comp (map (fn [x] (inc x)))
            (map (fn [y] (dec y))))
      c)
;=> r

We could build a similar control flow graph from the types of into, comp and map that allow us to propagate the type of c to the formal parameter x.

(into []
    /-----------------v
    | (comp (map (fn [x] (inc x)))
    \       (map (fn [y] (dec y))))
     \
      c)
;=> r

Then, having a type for x, would trigger the checking of the subsequent composed transducers.

Higher-order operations

Clojure has a host of handy map operations like get-in, assoc-in, and select-keys. We can “reify them at the type level” and then use control flow analysis to type check operations “in order” (of evaluation).

For example, update updates a map at a given entry with a function.

(update {:a 1} :a inc) ;=> {:a 2}

By reifying get and assoc at the type level, we could annotate update like so:

(ann update (All [m k v]
              [m k [(Get m k) -> v] -> (Assoc m k v)]))

Here’s a useful “order” to check the type variables, so we avoid any hard questions about running get or assoc “backwards.

(ann update (All [m k v]
                               2./----------------v
              [m k [(Get m k) -> v] -> (Assoc m k v)]))
            1. \ |       ^ ^                  ^ ^
                \--------/ /------------------/ /

In words, first we infer the value of the map and key from the first two arguments to update. Then we have enough information to “call” the third argument, which gets propagated to Assoc, which itself now has “ground” types so is easy to calculate.

This is enough information to help check anonymous fn’s passed to update, like:

(update {:a 1} :a #(+ 42 %)) ;=> {:a 2}

Notice that the control flow happens to go left-to-right in update, but as we’ve seen in other examples like comp, it could just as well be reversed and we could derive the same information.

Conservative checking

If for some reason we can’t infer the type of a local fn, instead of making a last-ditch effort to check the fn with argument types Any, we can simply ask the user to annotate the fn.

For example, checking the following function at type [Any -> ?].

(fn [x] (inc x))

Gives a confusing error message that inc does not accept type Any. Now, since we could lean on control improved inference to check most fn’s, we can reasonably be more conservative here.

I’m very excited at the possibilities here. It’s still early days for this research, but surely, if you can intuitively reconstruct the information for a given application based on local information, there should be an algorithm to recover it, right? :)

Next post

In the next few posts, we’ll describe our approach to improving feedback time from the type checker, a framework for providing custom type rules for macros, and explain how automatic library annotations can help mitigate the lack of community-provided annotations.

Permalink

The Functional Style (Part 2)

Introduction

In the previous article, we introduced functional programming and its basic principles. It was a lot of verbiages and no practice. The idea of programming without side effects is all well and good, but we need to know how to actually use it. So, let’s explore functional programming further by looking at example code. The Roman numeral kata is a good exercise that we can use to illustrate these ideas.

Roman Numerals

Briefly explained, Roman numerals constitute a number system that represents numbers using letters from the Latin alphabet: the number one is represented by I, five by V, ten by X, fifty by L, one hundred by C, five hundred by D, and one thousand by M. The rules are:

Permalink

Copyright © 2009, Planet Clojure. No rights reserved.
Planet Clojure is maintained by Baishamapayan Ghose.
Clojure and the Clojure logo are Copyright © 2008-2009, Rich Hickey.
Theme by Brajeshwar.