Advent 2019 part 8, Everything is (not) a pipe

This post is part of Advent of Parens 2019, my attempt to publish one blog post a day during the 24 days of the advent.

I’ve always been a big UNIX fan. I can hold my own in a shell script, and I really like the philosophy of simple tools working on a uniform IO abstraction. Uniform abstractions are a huge enabler in heterogenous systems. Just think of Uniform Resource Locators and Identifier (URLs/URIs), one of the cornerstones of the web as we know it.

Unfortunately since coming to Clojure I feel like I’ve lost of some of that power. I’m usually developing against a Clojure process running inside (or at least connected to) my trusty editor, and the terminal plays second fiddle. How do I pipe things into or out of that?

On top of that consider that most UNIX tools are line based (grep, awk, sed, cut, …), how do I marry that with Clojure’s EDN view of the world (another fabulous uniform abstraction).

I already covered nc, rep, and jet in a previous post. Each of these is just a single focused tool, but together they start to form a kind of Swiss army knife. Today I’ll look at a few more of these: HTTPie, Pup, kafkacat and websocat.

HTTPie is a tasty alternative to curl. It’s a bit more intuitive when it comes to passing along custom headers or parameters, and it can pretty print and format JSON. I have to thank Tommi from Metosin for indirectly introducing me to this one, I came across it in one of the Metosin READMEs (can’t remember which one).

After adoping HTTPie (the command is simply http) life was great, until I tried to use it to connect to a websocket. Websockets are kind of HTTP so I thought it might work, but they’re also kind of their own thing so I don’t blame the devs for leaving this out. Focused tools, right? Fortunately there’s websocat to do just that.

See how there’s a common theme here? We have various mechanisms for shuttling data from one place to another, either as a stream, or a single shot transfer, and we’re simply hooking these to UNIX pipes. (Macbook owners can think of dongles as an apt metaphor).

The final tool I want to mention is kafkacat, you can probably guess by now what it does. I’ve been poking around a bit with Kafka lately, and having a simple command line client for both sending and receiving was super helpful to debug and better understand things.

So this is your set of dongles:

  • TCP streams: nc
  • Websocket streams: websocat
  • HTTP requests: http
  • kafka topics: kafkacat
  • nREPL connections: jet

But just plugging things together isn’t always enough, they also need to speak the same format. In the Clojure world we like things like Transit and EDN, but the rest of the world speaks JSON, or XML/HTML, or CSV, and UNIX tools like to have simple lines of text, so we need to convert between these formats.

I already mentioned jet, and jq is helpful for extracting things from JSON. The final one I want to introduce in this series is pup, which is like jq for HTML.

For instance:

get-latest-datomic-version () {                   
    http https://my.datomic.com/downloads/free | pup .latest attr{href} | sed 's^/downloads/free/^^'
}

Or say so you are putting Transit on a Kafka topic, but you want to see that as pretty printed EDN to inspect what’s going over the wire? easy peasy.

kafkacat -b 127.0.0.1:9092 -t topic-1 | jet --from transit --pretty
  • jet convert/query EDN/Transit/JSON ; pretty-print EDN
  • jq convert/query/pretty-print JSON
  • pup extract from HTML/XML

And of course all the classics are still there, sed, grep, awk, cut, head, tail, tee and many more.

Comment on ClojureVerse

Permalink

Recording Screencasts with Emacs

This article is part of the “Meta Advent 2019” series. I’ve committed to writing a new blog post here every day until Christmas.

Yesterday I was reminded of one of my favourite aspects of blogging - you share some knowledge, but you also gain a lot of knowledge in return. To be more specific - several readers commented on my article Dead Simple Emacs Screencasts, that they actually thought the article would cover how to record screencasts with Emacs itself. I have to admit this caught me by surprise, as even though I know well enough that Emacs can do anything, I still never thought of trying to record screencasts with it. I’ve been using tools like Apple Quicktime for conventional screencasts and LICEcap or Gifox for simple animated gifs. Let’s see if we can do better and record some awesome screencasts straight from Emacs!

Well, it turns out that there’s a mode for that!1 It’s named gif-screencast and seems to be a fairly new addition to the Emacs ecosystem, as its first commit is from February 2018. The package basically creates a single frame for each action you take by making a screenshot. This simple approach has the nice benefit of optimizing the size of the resulting gif, as it elimates awkward pauses when you’re not doing anything. As a bonus, it’s possible to easily edit the end result! Here’s a bit of the project’s rationale straight from its README:

Most of the time, one user action equals one visual change. By exploiting this fact, this package tailors GIF-screencasting for Emacs and captures one frame per user action. It’s much more efficient than capturing frames even at a frequency as low as 10 per second. You could roughly expect 3 to 10 times smaller files compared to videos of the same quality.

Internally the mode shells out to several command-line tools that do the heavy lifting:

  • a screenshot taking tool (by default that’s scrot on Linux and screencapture on macOS)
  • an image cropping tool (ImageMagick’s mogrify by default)
  • a tool to convert those screenshots into a gif (ImageMagick’s convert by default).
  • a tool to optimize the final gif (gifsicle by default). Alternative you can just set gif-screencast-want-optimized to nil.

All of those are configurable via defcustoms. You can check them out by doing M-x customize-group gif-screencast.

I guess it’s clear at this point it’s a good idea to install ImageMagick to have gif-screencast work well. If you’re using Linux you don’t do much besides getting the external tools installed, but macOS users will need a bit of extra setup. Put something like this in your Emacs config:

(with-eval-after-load 'gif-screencast
  (setq gif-screencast-args '("-x")) ;; To shut up the shutter sound of `screencapture' (see `gif-screencast-command').
  (setq gif-screencast-cropping-program "mogrify") ;; Optional: Used to crop the capture to the Emacs frame.
  (setq gif-screencast-capture-format "ppm") ;; Optional: Required to crop captured images.

If you’re on a HiDPI display you’ll also need to patch your gif-screencast installation a bit:

(advice-add
 #'gif-screencast--cropping-region
 :around
 (lambda (oldfun &rest r)
   (apply #'format "%dx%d+%d+%d"
          (mapcar
           (lambda (x) (* 2 (string-to-number x)))
           (split-string (apply oldfun r) "[+x]")))))

That’s quite the setup, right? Don’t worry, we’re done with it! Now you can finally do M-x gif-screencast and create your first screencast! Let’s play a bit with CIDER:

gif_screencast_demo.gif

When you’re done with your screencast you have to do M-x gif-screencast-stop. You can also pause a recording with M-x gif-screencast-pause. On macOS by default the screencasts get saved to ~/Videos/emacs. On Linux XDG_VIDEO_DIR would be used (typically it’s ~/Video).

A couple of notes:

  • For me at least Emacs became a bit slow during the screencast recording, but this wasn’t a major issue.
  • I’ve noticed that gif-screencast doesn’t handle properly scrolling (e.g. in a REPL buffer).
  • Cropping is broken on HiDPI displays. See this issue for details.
  • It’s hard to show overlays that are immediately cleared on the next command. This is clearly visible in the example I showed earlier.
  • Even with optimization enabled the resulting gif for me was much bigger than what I’d get from Gifox. Perhaps I did something wrong.
  • It’s a good idea to bind the commands for starting, pausing and stopping screencasts to some keybindings:
(with-eval-after-load 'gif-screencast
  (global-set-key (kbd "<f8>") #'gif-screencast-toggle-pause)
  (global-set-key (kbd "<f9>") #'gif-screencast-stop))

Interestingly emacs-gif-screencast defines gif-screencast-mode, but doesn’t bind any keybindings there by default. That certainly seems like an opportunity for improvement to me.

So, all in all gif-screencast is a pretty cool tool that proves once again how versatile Emacs is. Unfortunately, it’s also a bit rough around the edges, so I’ll probably stick with Gifox for the time being. Still, I’m pretty excited about the possibilities created by gif-screencast and I’ll certainly keep an eye on it. That’s all I have for you today! Emacs forever!

  1. What a shocker, right? 

Permalink

You have to know about persistent data structures

The first thing I look for when checking out a new programming language is immutable persistent data structures. This is an injury I've gotten from Clojure, which has them out of the box. I tend to structure my whole app around these kinds of data structures. They are smart, fast, efficient, immutable and amazing.

In this post, I'll try to make you feel the same way about them.

Axiom: immutability is good

I'm not going to try to sell this one. Because it's a tough sell.

As in, I know a few people that don't like immutable values, and I have no idea how to sell it to them.

Personally, I like to use something that fundamentally eliminates a whole category of bugs from my code.

Sorry. Couldn't resist being snarky. I don't want to trash talk those who don't like immutability. But I just don't get it.

The problem with immutable values is that they're immutable

Let's say you have an immutable hash map or something. It's implemented using some kind of tree structure under the hood, like a lot of data structures are.

The problem happens when we want to add a key to this map. The data structure is immutable, so we can't change it!

So, we have to create a whole new copy of everything, an add our new stuff to it.

(Red = new stuff)

A new node was added at the bottom right of the image. And we had to copy all the data in the tree structure, since we can't change the immutable original.

So that's the end of the story, then? Immutability sucks, because you have to copy everything all the time you want to change something?

Well, yes.

But, also, no.

Copy on write - nah

I mean, I'm not gonna diss copy on write completely. There does exist some mildly useful software that leverages it, such as the Linux Kernel.

The whole central and important concept of forking a process in Linux/Unix, relies on copy on write (CoW). When you have a process that uses 11GB of RAM, and you fork it, you have a new process, identical to the original one, with its own copy of the 11GB of RAM. To make the process of forking not horribly tear-inducingly slow, the kernel relies on CoW. The action of forking doesn't actually do anything to those 11GBs of RAM. It's only when you start writing to memory from the fork, that the kernel starts copying and then writing - copy on write - the memory of the original process.

CoW is technically speaking a type of persistent data structure.

However, CoW is at its most useful when the data is mutable. When your data is immutable to the core, you have another option.

That other option is...

Structural sharing - yass

Now this is my kind of persistent data structure.

So.

We have this huge data structure. We know that it is immutable. We want to change one little piece of it. What optimization can we run on this process?

Do you see it yet?

The old data structure is immutable. We can just use that!

(Red = new stuff)

See what we did? Why copy immutable data?

We just need to create a new root node, a new 2nd node, and a new data node, and then just point to the old stuff! The old stuff is immutable, after all. The whole idea is that it will never change.

You just understood the Clojure state model. The values themselves are immutable. You can have functions that takes an immutable value, that return an updated immutable value. Creating new immutable values is cheap - because you can share almost all of the data with the old immutable value.

Yay!

Clojure does another really cool thing

In the real world, it turns out that if you just have a small map with just a handful of keys, operating on a fancy tree structure can be kind of expensive.

Under the hood, Clojure will do a very cool optimization: if your map has fewer than 8 (I think) items in it, it will just be represented as a normal mutable list under the hood.

When you add an item, Clojure will make a full copy of that list, containing the old items and the newly added item.

Why is that cool?

Well, it turns out that it's actually faster to do it that way, when you only have a small amount of data. Checking a list of 8 items for whether or not your key exists in it, is much faster than having to maintain and balance a binary tree or a b-tree.

Think of it like garbage collection generations. Under the hood, your stuff will be in the "eden" space or "tenured" space. Which is fancy speak for: stuff that is just created and immediately discarded is garbage collected differently than long lived stuff. This happens completely under the hood, as a performance optimization, without you having to do anything special to make it happen.

An other another cool thing!

Let's say you have this function in a tight loop that adds 100s of items to a map, based on a data stream of some kind. Maybe a CSV or a database or something like that.

By default, all maps are immutable, so that means you have to do 100s of potentially expensive operations on the persistent data structures.

However! Clojure has transients.

Transients are a special version of immutable data structures. You first "unlock" it and create a "transient" map (or list or set or ...). Then you work on it as you would with a normal immutable data structure, using map or reduce or whatever. Then when you're done, you "lock" it and you get a normal immutable map back.

This optimization allows Clojure to know that in your tight loop, nobody else will actually use the data structure, so it can be "less fancy" in how it builds it up. And much faster, skipping the entire potential overhead of managing a persistent data structure.

Clojure has pretty cool data structures.

Permalink

Advent 2019 part 7, Do that doto

This post is part of Advent of Parens 2019, my attempt to publish one blog post a day during the 24 days of the advent.

doto is a bit of an oddball in the Clojure repertoire, because Clojure is a functional language that emphasizes pure functions and immutabilty, and doto only makes sense when dealing with side effects.

To recap, doto takes a value and a number of function or method call forms. It executes each form, passing the value in as the first argument. At the end of the ride it returns the original value.

The typical example of using doto is for dealing with mutable Java objects

(doto (java.util.HashMap.)
  (.put "hello" "world")
  (.put "hi" "earth"))
;; => {"hi" "earth", "hello" "world"}

And it really cleans up invoking the Builder pattern.

(.build
 (doto (FooBuilder.)
   (.setBar 123)
   (.toggleQux)
   (.setName "hello")))

But what I want to show is that there are a few places where doto is useful when you’re not doing Java interop.

The first is with prn or println. These functions return nil, so if you want to quickly add a prn to check what a function is returning you have to work around that, for instance by adding a let.

;; say you have something like this
(foo (bar (baz x)))

;; what's coming back from `baz`?
;; this does not work:
(foo (prn (bar (baz x))))

;; you could do this, but that's a mouthful
(let [x (bar (baz x))]
  (prn x)
  (foo x))

;; instead throw in a `(doto ... prn)`
(foo (doto (bar (baz x)) prn))

The cool thing is this also works in thread-first forms (not in thread-last though), just throw that (doto prn) in between any two forms.

(-> x
    baz
    (doto prn)
    bar
    foo)

Or with some markers to make the output a bit more obvious:

(-> x
    baz
    (doto (prn '<-baz))
    bar
    (doto (prn '<-bar))
    foo)

You can also throw in a (doto ... assert) if you’re getting paranoid that there’s a nil somewhere where you don’t expect it.

And if you’re using Datomic from a REPL or from tests, and you want to be able to quickly recreate an in-memory database, you can do this.

(let [uri (doto "datomic:mem:test1" d/delete-database d/create-database)
      conn (d/connect uri)]
  ,,,)

Comment on ClojureVerse

Permalink

Ep 058: Reducing It Down

Each week, we discuss a different topic about Clojure and functional programming.

If you have a question or topic you’d like us to discuss, tweet @clojuredesign, send an email to feedback@clojuredesign.club, or join the #clojuredesign-podcast channel on the Clojurians Slack.

This week, the topic is: “Reduce and reducing functions” We take a long hard look at reduce and find the first of many generally useful nuggets inside.

Selected quotes:

  • “Parentheses are a joy, they are hugs around your code.”
  • “You want to boil it down into a result, so you use reduce.”
  • “We’ve taken ahold of the cooking metaphors and are just running with it!”
  • “Reducing functions are a backbone of functional programming, because we don’t have mutation.”
  • “Understanding reduce is really important for understanding functional programming.”

Related episodes:

Permalink

MECCA NSF

MECCA NSF

This could plausibly stand for:

  1. MECCA, Not Safe For [anyone, anywhere]
  2. Nintendo Sound Format

We'll ignore the first interpretation for now and go with the second.

NSF files contain the music from Nintendo games. While working on the main music sequencer part of this project, I recalled the main part of my motivation - my discontent with other music software, in particular their inability to play well together, even talk to each other. I realized that I consider absolutely vital in my music application the ability to interface with as many other formats as possible.

A brief naive tour of some music encoding formats

This is simply a sketch of the formats I deal with in this project. For a better treatment and modern solutions, check out Alda!

  1. MusicXML - This is actually the most common format for scorewriters, such as Sibelius, Finale and Musescore. It is also the only one I'm mentioning here that is human readable, as it is simply sheet music described in XML. I'm working on a MusicXML parser which will likely be covered in a future post, but it is important to note that this is a markup language. Its primary focus is on describing the layout of notes on the page.
  2. MIDI - In contrast, MIDI files contain just pure note data, as a stream of hexadecimal bytes representing on/off events and other commands. This is the scope of MECCA MIDI, and will likely be covered in a future post. Suffice to say here that this is the standard format for digital music storage and distribution. (We are talking about note data, not audio)
  3. NSF files - Like MIDI, also a binary format, but here we have something completely different... the music is literally written in 6502 machine code!

So it turns out, a parsing project has turned into a CPU emulation project. That is cool, because I'm up for the challenge. However, our emulator does not need to operate in real time, so I like to think of it as more of a static analyser in that all we need to do is make our program aware of just enough instructions to recreate the music score. Let's see how far we get! We'll start with the .nsf file of Super Mario Bros. but feel free to upload one of your own (you can find them here).

(ns mecca.klipse
  (:require [reagent.core :as r]
            [goog.object :as o]
            [goog.crypt :as crypt]))
(def file-atom (r/atom "4E45534D1A011201C4BD34BED0F25375706572204D6172696F2042726F732E0000000000000000000000000000004B6F6A69204B6F6E646F0000000000000000000000000000000000000000000031393835204E696E74656E646F000000000000000000000000000000000000001A410000000000000000000000000000000048A200BD00BE9500E8D0F8BD00BF9D0007E8D0F768A007C90D900938E90DA000A24086FCC9079004C838E907AAA901E000F0050ACA4CF3BD99F40060004FD0010028200005EB000000000800000000000000000000000000000000000000000000000000000000000000000000000001AABD44BEA2004CC4BD0000000000000000020103040C0A07080B05090D0F0E101112131400000000000000000000000000000000000000000000010101010100000000000000000000000000000000000000286060606060000000000000000000000000000000000000000000000000000000000000000000000000000000000001010000000000000000000000000000000000000000000000B0E0000000000000000000000000000000000000000000000090A6019F1300000000180000000801FA11273E0000000000000001300000000000000028000700000002000000000100000000000000FFFF002490010000010801000001000A000802FFFF000000000000000301000C00000000000200000000000000008000000105C2000000012800000000020000010000000200000100000000000000000000000100030000000000101E00000000000A000000000000000900180000000000000000000000000000000000000000000000000000000000A1185A6ADE0BB70000BD01000905030E0309050E000000000030B800000000001200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003080700000000A56900959FC5023010BD3304C9809009A502959FA9009D330468F02BA50249FFA8C88407BD330438E5019D3304B59FE900959FC5071010BD3304C980B009A507959FA9FF9D330460B50F480AB01268F0034C82C8AD1F072907C907F00E4CCCC068290FA8B90F00D002950F600303060606060606070707050904050608090A060B1040B0B08040408040F0F0F0A56D38E904856DAD250738E9048D2507AD1A0738E9048D1A07AD1B0738E9048D1B07AD2A0738E9048D2A07A9008D3B078D2B078D39078D3A07B9F89B8D2C0760AD4507F05EAD2607D059A00B883054AD5F07D96BC0D0F5AD2507D976C0D0EDA5CED981C0D023A51DC900D01DAD5F07C906D023EED906EEDA06ADDA06C903D01EADD906C903F00FD007AD5F07C906F0E6208CC02071D0A9008DDA068DD906A9008D4507ADCD06F0109516A901950FA900951E8DCD064C26C2AC3907B1E9C9FFD0034C16C2290FC90EF00EE005900AC8B1E9293FC92EF00160AD1D0718693029F08507AD1B0769008506AC3907C8B1E90A900BAD3B07D006EE3B07EE3A0788B1E9290FC90FD019AD3B07D014C8B1E9293F8D3A07EE3907EE3907EE3B074CCCC0AD3A07956EB1E929F09587CD1D07B56EED1B07B00BB1E9290FC90EF0694C50C2A507D587A506F56E9041A90195B6B1E90A0A0A0A95CFC9E0F04CC8B1E92940F005ADCC06F06DB1E9293FC9379004C93F9031C906D007AC6A07F002A9029516A901950F2026C2B50FD04960ADCB06D009AD9803C901D00BA92F9516A900951E206CC2604C1BC7C8C8B1E94A4A4A4A4ACD5F07D00E88B1E98D5007C8B1E9291F8D51074C5BC2AC3907B1E9290FC90ED003EE3907EE3907EE3907A9008D3B07A60860B516C915B00DA8B5CF690895CFA9019DD8039820048E0EC30EC30EC31EC3F0C228C3F1C242C36BC3F0C275C375C3F7C287C7D1C74AC33DC385C3A0C7F0C2A0C7A0C7A0C7A0C7B8C7F0C2F0C25CC45CC45CC45CC459C4F0C2F0C2F0C2F0C2DFC712C83FC845C80BC803C80BC84BC857C849C560BC1EB9F0C2F0C2F0C2F0C2F0C207C381C860200EC34C46C3A90295B695CF4A9D96074A951E4C46C3A9B895CF60F8F4A001AD6A07D00188B90CC395584C5AC3200EC3A901951E608050A9009DA2039558ACCC06B926C39D9607A90B4C5CC3A9004C19C3A9009558A909D012A030B5CF9D01041002A0E09875CF9558A9039D9A04A9029546A90095A09D340460A9029546A9099D9A04602046C3BDA70729109558B5CF9D340460ADCB06D00BA9008DD106203DC34CD9C74C98C9262C32382022242613141516AD8F07D03CE005B038A9808D8F07A004B91600C911F02B8810F6EED106ADD106C907901DA204B50FF005CA10F93010A900951EA9119516208AC3A92020D8C5A60860A5CEC92C90F9B91E00D0F4B96E00956EB987009587A90195B6B9CF0038E90895CFBDA7072903A8A202B998C39501C8C8C8C8CA10F4A608206CCFA457C008B00EA8BDA8072903F0059849FFA8C8982046C3A0029558C9003001889446A9FD95A0A901950FA905951E60283828382800001010002075C5A9009558B51638E91BA8B94FC49D8803B954C49534B5CF18690495CFB5871869049587B56E6900956E4CD9C78030408030505070204080A0704090680E05060E1C20100C1E22181410602048AD8F07D0A12046C3BDA8072903A8B9A4C48D8F07A003ADCC06F001C88400E400B084BDA707290385008501A9FB95A0A900A457F007A904C01990010A481865008500BDA8072903F007BDA907290F850068186501A8B998C49558A9019546A557D012A400982902F00BB55849FF1869019558F646982902F00FA586187988C49587A56D69004C3CC5A58638F988C49587A56DE900956EA901950F95B6A9F895CF602075C58E6803A9008D63038D6903B5878D6603A9DF8D90079546A9208D64039D8A07A9058D83044A8D650360A0FFC8B90F00D0FA8CCF068A0980990F00B56E996E00B587998700A901950F99B600B5CF99CF006090807090FF01AD8F07D0F49D3404A5FD090285FDAC6803B91600C92DF03120D9D1186920ACCC06F00338E9108D8F07BDA70729039D1704A8B99DC595CFAD1D071869209587AD1B076900956E4C1FC6B9870038E90E9587B96E00956EB9CF0018690895CFBDA70729039D1704A8B99DC5A000D5CF9001C8B9A1C59D3404A9008DCB06A9089D9A04A90195B6950F4A9D0104951E60003060600020604070406030AD8F07D047A9208D8F07CED706A00688B91600C931D0F8B9870038E93048B96E00E9008500ADD70618791E00A868187931C69587A5006900956EB937C695CFA90195B6950F4A9558A90895A0600102040810204080403090502060A0700A0BAD8F07D06FAD4E07D057E003B066A000BDA707C9AA9001C8AD5F07C901F001C8982901A8B99AC69516ADDD06C9FFD005A9008DDD06BDA7072907A8B98AC62CDD06F007C89829074CD6C60DDD068DDD06B992C620D8C59D1704A9208D8F074C6CC2A0FFC8C005B00DB90F00F0F6B91600C908D0EF60A5FE090885FEA908D0A8A00038E93748C904B00B48A006AD6A07F002A002688401A0B02902F002A0708400AD1B078502AD1D078503A002684A9001C88CD306A2FFE8E005B02DB50FD0F7A5019516A502956EA50395871869188503A50269008502A50095CFA90195B6950F206CC2CED306D0CC4C5EC2A90195584A951E95A0B5CF9D340438E9189D1704A9094CDBC7B5168DCB0638E91220048EA4C3B7C7A8C4A3C53DC69CC660A005B91600C911D005A901991E008810F1A9008DCB06950F60A9029546A9F89558A9039D9A0460D6CFD6CFACCC06D005A0022071C8A0FFADA003951E10028AA88CA003A9009546A82071C8A9FF9DA2034C28C8A90095584C28C8A040B5CF100749FF186901A0C09D0104981875CF95582063C3A905AC4E07C003F007ACCC06D002A9069D9A0460204BC84C48C82057C84C2BC8A9109D3404A9FF95A04C60C8A9F09D3404A90095A0A0012071C8A9049D9A0460080CF80000FFB58718796BC89587B56E796EC8956E6060A608A900B416C015900398E91420048EE0C835C995D2D6C8D6C8D6C8D6C847C947C947C947C947C947C947C947C9D6C865C965C965C965C965C965C965C94DC94DC965D085BC4BB9D6C8D9D2BAB8D6C8A4B7D7C86020AFF12052F14C7DE8A9009DC50320AFF12052F1207DE82043E220C1DF2033DA2053D8AC4707D0032005C94C7AD6B51620048E77CA77CA77CA77CA77CAD8C977CA89CB36CC34C94ACC4ACCB0C9B0D3F9CAFFCA25CB28CF77CA34C9DFCE6020EBD120AFF12052F12043E22053D84C7AD6203CCD4C7AD620AFF12052F1204CE2207BDB2052F12066ED2055D64C7AD620AFF12052F12073E22045DBAD4707D0032082C92052F120C8E54C7AD6B51638E92420048E32D4D3D54FD64FD607D631D63DD6A900950F9516951E9D10019D96079D25019DC5039D8A0760BD9607D01620F7C2BDA80709809D3404290F09069D9607A9F995A04C92BF301C00E8001808F80CF4B51E2920F0034CE5CAB53CF02DD63CADD103290CD06ABDA203D017ACCC06B9CEC99DA2032094BA9009B51E0908951E4C58CADEA2034C58CA2037B51E2907C901F03EA9008500A0FAB5CF3013A0FDC970E600900BC600BDA8072901D002A0FA94A0B51E0901951EA5003DA907A8ADCC06D001A8B910CA9D8A07BDA80709C0953CA0FCA5092940D002A0049458A0012043E1300AC8BD9607D004A9F895589446A000B51E2940D019B51E0AB030B51E2920D05BB51E2907F024C905F004C903B0302063BFA000B51EC902F00C2940F00DB516C92EF007D0034C02BFA001B558481002C8C81879D0C995582002BF68955860BD9607D01E951EA5092901A8C8944688AD6A07F002C8C8B9D4C99558602063BF4C02BFC90ED009B516C906D0032098C9602092BF4C02BFB5A01D3404D0139D1704B5CFDD0104B009A5092907D002F6CF60B5CFD55890034C75BF4C70BF2045CB2066CBA001A5092903D011A5092940D002A0FF8400B5CF18650095CF60A9138501A5092903D00DB458B5A04AB00AC401F003F65860F6A06098F0FAD65860B55848A001B5A02902D00BB55849FF1869019558A00294462002BF8500689558603F03B51E2920D04DACCC06BDA8073987CBD0128A4A9004A445B008A0022043E1100188944620DFCBB5CF38FD3404C920900295CFB44688D00EB5871875589587B56E6900956E60B58738F5589587B56EE900956E604C8CBFB5A02902D037A509290748B5A04AB01568D011BD34041869019D34049558C902D002F6A06068D014BD340438E9019D34049558D007F6A0A9029D960760BD9607F008A5094AB002F6CF60B5CF6910C5CE90F0A90095A060B51E2920F0034C92BFA9E895584C02BF40800404B51E2920F0034C8CBF8503B51638E90AA8B946CC8502BD010438E5029D0104B587E9009587B56EE900956EA9208502E0029049B558C9109016BD17041865029D1704B5CF650395CFB5B669004CACCCBD170438E5029D1704B5CFE50395CFB5B6E90095B6A000B5CF38FD34041007A01049FF186901C90F900398955860000103040506070708000306090B0D0E0F100004090D101316171800060C12161A1D1F2000070F161C212527280009121B21272C2F30000B151F272E333738000C18242D353B3E40000E1B28323B424648000F1F2D38424A4E50001122313E49515658010302000009121B242D363F48515A630C1820AFF1ADD1032908D074AD4707D00ABD88032010D4291F95A0B5A0B416C01F900DC908F004C918D00518690195A085EF2052F1208ECEBCE506ADB9039900028507ADAE039903028506A90185002008CEA005B516C91F9002A00B84EDA9008500A5EF208ECE20BBCDA500C904D008ACCF06B9E5068506E600A500C5ED90E260A5038505A406A5014605B00449FF6901186DAE039903028506CDAE03B009ADAE0338E5064CE6CD38EDAE03C9599004A9F8D015ADB903C9F8F00EA5024605B00449FF6901186DB903990002850720EDEC9848AD9F070D4707D0708505A4B588D069A4CEAD5407D005AD1407F009E605E60598186918A89838E507100549FF186901C908B01CA506C9F0B016AD0702186904850438E506100549FF186901C9089013A505C902F023A405A5CE18793ACDE6054C32CEA201A504C506B001E88646A200A50048202CD9688500681869048506A6086048290FC9099005490F1869018501A400B92ECD186501A8B9C7CC85016848186908290FC9099005490F1869018502A400B92ECD186502A8B9C7CC8502684A4A4AA8B92ACD850360F8A070BD002020200000B51E2920F008A9009DC5034C92BF2002BFA00DA9052096BFBD34044A4A4A4AA8B5CF38F9D5CE100549FF186901C908B00EBD34041869109D34044A4A4A4AA8B9DACE9DC50360153040B51E2920F0034C63BFB51EF00BA90095A08DCB06A910D013A9128DCB06A002B925CF9901008810F7206CCF9558A001B5A02901D00AB55849FF1869019558C894464C02BFA0002043E1100AC8A50049FF1869018500A500C93C901CA93C8500B516C911D01298D5A0F00DB5A0F006D658B558D0409895A0A500293C4A4A8500A000A557F024AD7507F01FC8A557C9199008AD7507C9029001C8B516C912D004A557D006B5A0D002A000B90100A40038E9018810FA601A5898969492908E8C8A8886848280AE6803B516C92DD0108608B51EF01A2940F006B5CFC9E0900AA98085FCEE72074C71D0208CBF4C7BD1CE6403D044A9048D6403AD630349018D6303A9228505AC6903B9DDCF8504AC0003C8A20C20CD8AA608208F8AA90885FEA90185FDEE6903AD6903C90FD00B2063C3A940951EA98085FE4C7BD121411131B51E2920F014B5CFC9E0909EA2042098C9CA10FA8DCB06A60860A9008DCB06AD4707F0034C39D1AD630310034C0FD1CE6403D00DA9208D6403AD630349018D6303A509290FD004A9029546BD8A07F01C2043E11017A9019546A9028D6503A9209D8A078D9007B587C9C8B03EA5092903D038B587CD6603D00CBDA7072903A8B961D08DDC06B587186D65039587B446C001F017A0FF38ED6603100749FF186901A001CDDC0690038C6503BD8A07D028208CBFAD5F07C9059009A5092903D0032094BAB5CFC980901CBDA7072903A8B961D09D8A074C49D1C901D009D6CF2063C3A9FE95A0AD5F07C907F004C905B027AD9007D022A9208D9007AD630349808D630330E120D9D1ACCC06F00338E9108D9007A9158DCB0620BCD1A010B5464A9002A0F098187587ACCF06998700B5CF18690899CF00B51E991E00B546994600A50848AECF068608A92D951620BCD1688508AAA9008D6A0360EE6A0320D7C8B51ED0F5A90A9D9A042043E24C53D8BF40BFBFBF4040BFAC6703EE6703AD670329078D6703B9D1D160AD4707D030A940ACCC06F002A9608500BD010438E5009D0104B587E9019587B56EE900956EBC1704B5CFD99DC5F006187D340495CF2052F1B51ED0C3A9518500A002A5092902F002A0828401BCE506A200ADB903990002A500990102E600A501990202ADAE039903021869088DAE03C8C8C8C8E8E00390D9A60820AFF1BCE506ADD1034A489005A9F8990C02684A489005A9F8990802684A489005A9F8990402684A9005A9F899000260D6A0D00CA90895A0F658B558C903B0182052F1ADB9038DBA03ADAE038DAF03BCE506B5582017ED60A900950FA90885FEA9058D38014C36D3000008080008000854555657A9008DCB06AD4607C905B02C20048E11D3F2D212D34ED3A2D3A005ADFA07C901F00EA003C903F008A000C906F002A9FF8DD706941EEE460760ADF8070DF9070DFA07F0F1A5092904F004A91085FEA023A9FF8D3901205F8FA9058D3901A00BAD5307F002A011205F8FAD53070A0A0A0A09044C36BCB5CFC9729005D6CF4C65D3ADD706F0383036A9168DCB062052F1BCE506A203ADB903187DCDD2990002BDD5D2990102A922990202ADAE03187DD1D2990302C8C8C8C8CA10DAA608602065D3A9069D9607EE4607602065D3BD9607D005ADB107F0EF60B51ED056BD8A07D051B5A0D023B55830142043E11009A50049FF1869018500A500C9219035B55849FF1869019558F6A0BD3404B4581003BD17048500A5094A9019AD4707D014B5CF18755895CFC500D009A90095A0A9409D8A07A9209DC503608507B534D00EA018B5581865079558B5A0690060A008B55838E5079558B5A0E90060B5B6C903D0034C98C9B51E100160A8BDA2038500B546F0034CBBD5A92DD5CF900FC400F00818690295CF4CB1D54C98D5D9CF00900DE400F0F418690299CF004CB1D5B5CF48BDA2031018BD34041869058500B5A06900301AD00CA500C90B900CB004C508F00C20B7BF4CA7D420B1D54CA7D420B4BFB41E6838F5CF1879CF0099CF00BDA2033004AA2021DCA408B9A000193404F077AE0003E020B070B9A00048482041D5A5019D0103A5009D0203A9029D0303B9A000300DA9A29D0403A9A39D05034CFFD4A9249D04039D0503B91E00A86849FF2041D5A5019D0603A5009D0703A9029D080368100DA9A29D0903A9A39D0A034C30D5A9249D09039D0A03A9009D0B03AD000318690A8D0003A6086048B98700186908AECC06D00318691048B96E00690085026829F04A4A4A8500B6CF6810058A186908AA8AAE00030A2A482A290309208501A50229010A0A050185016829E01865008500B9CF00C9E89006A50029BF85006098AA20AFF1A9062011DAADAD039D1701A5CE9D1E01A90195462063C399A000993404609848206BBF68AA206BBFA608BDA2033004AA2021DCA60860B5A01D3404D0159D1704B5CFDD0104B00BA5092907D002F6CF4CFED5B5CFD558900620B7BF4CFED520B4BFBDA20330032021DC60A90E2047CB2066CBBDA203301CA5861865008586A56DA400300569004C28D6E900856D8CA1032021DC60BDA20330062088BF2021DC602002BF8500BDA2033007A91095582014D660205BD64CFED5205BD64C71D6AD4707D019BD1704187D34049D1704B5CF75A095CF60BDA203F0032019DC60B516C914F055AD1C07B416C005F004C00DD0026938E9488501AD1A07E9008500AD1D0769488503AD1B0769008502B587C501B56EE5003020B587C503B56EE5023019B51EC905F013C00DF00FC030F00BC031F007C032F0032098C960FFFFFFB524F0560AB053A5094AB04E8A0A0A18691CA8A20486019848B51E2920D034B50FF030B516C9249004C92B9026C906D006B51EC902B01CBDD803D0178A0A0A186904AA2027E3A6089009A9809524A601203ED768A8A601CA10BBA60860060002121107052D2052F1A601B50F100B290FAAB516C92DF00CA601B516C902F06BC92DD02DCE8304D0622063C395588DCB06A9FE95A0AC5F07B936D79516A920C003B0020903951EA98085FEA601A909D033C908F036C90CF032C915B02EB516C90DD006B5CF691895CF201BE0B51E291F0920951EA902B416C005D002A906C006D002A9012011DAA90885FF60A5094A9036AD47070DD603D02E8A0A0A186924A82025E3A608901BBDBE06D01BA9019DBE06B56449FF1869019564AD9F07D0084C2CD9A9009DBE06602098C9A9062011DAA92085FEA539C902900EC903F024A9238D9F07A94085FB60AD5607F01BC901D023A608A9028D560720F185A608A90C4C47D8A90B9D100160A9018D5607A909A0002048D96018E830D008F8A5094AB0F42041DCB023BDD803D01EA50EC908D018B51E2920D0122052DC2025E3A608B009BD910429FE9D910460B416C02ED0034C00D8AD9F07F0064C95D70A0604BD910429011DD803D059A9011D91049D9104C012F04EC00DF07DC00CF079C033F042C015B071AD4E07F06CB51E0AB034B51E2907C902902CB516C906F025A90885FFB51E0980951E2005DAB94FD89558A903186D8404BC9607C003B003B992D82011DA60A59F3002D06AB516C9079009A5CE18690CD5CF905BAD9107D056AD9E07D03DADAD03CDAE0390034CF6D9B546C901D0034CFFD9AD9E07D024AE5607F0228D5607A9088D9E070A85FF20F185A90AA001850E841DA0FF8C4707C88C7507A608608657E886FCA9FC859FA90BD0E102060506B516C912F0BDA90485FFB516A000C914F01BC908F017C933F013C90CF00FC8C905F00AC8C911F005C8C907D01DB965D92011DAB54648202FE0689546A920951E2063C39558A9FD859F60C909901D29019516A000941EA9032011DA2063C32005DAB951D895584CF1D9100BA904951EEE8404AD8404186D91072011DAEE9107AC6A07B9D2D99D9607A9FC859F60B546C901D0034C2CD9201CDB4C2CD9A0012043E11001C8944688609D1001A9309D2C01B5CF9D1E01ADAE039D170160804020100804027FBFDFEFF7FBFDA5094A90ECAD4E07F0E7B516C915B06EC911F06AC90DF066BDD803D0612052DCCA305B86019848B50FF04CB516C915B046C911F042C90DF03EBDD803D0398A0A0A186904AA2027E3A608A4019020B51E191E002980D011B991043D25DAD018B991041D25DA99910420B4DA4CAADAB991043D2CDA99910468A8A601CA10A5A60860B91E00151E2920D033B51EC906902EB516C905F027B91E000A900AA9062011DA2095D7A40198AA2095D7A608BD2501186904A6012011DAA608FE250160B91E00C906901DB91600C905F0F12095D7A401B92501186904A6082011DAA601FE25016098AA201CDBA608B516C90DF022C911F01EC905F01AC912F008C90EF004C907B00EB55849FFA8C89458B5464903954660A9FF9DA203AD4707D029B51E3025B516C924D006B51EAA205FDB2041DCB0148A2054DCB5CF85008A482025E368AA900320BCDBA60860AD4707D0379DA2032041DCB02FA9028500A6082052DC2902D022B9AD04C92090052025E3B019B9AD0418698099AD04B9AF0418698099AF04C600D0D5A60860A608B9AF0438EDAD04C904B008A59F1004A901859FADAF0438F9AD04C906B01BA59F3017A500B416C02BF005C02CF0018AA6089DA203A900851D60A9018500ADAE0438F9AC04C908900DE600B9AE0418EDAC04C909B003204BDFA608608000A8B5CF187916DC2CB5CFA40EC00BF017B4B6C001D01138E92085CE98E90085B5A900859F8D330460ADD003C9F0B009A4B588D004A5CEC9D060A5080A0A186904A8ADD103290FC90F602010AD1607D02EA50EC90BF028C9049024A901AC0407D00AA51DF004C903D004A902851DA5B5C901D00BA9FF8D9004A5CEC9CF900160A002AD1407D00CAD5407D00788AD0407D00188B9ADE385EBA8AE5407AD1407F001E8A5CEDD62DC903520E9E3F03020A1DFB04FA49F1027A404C0049021208FDFB010AC4E07F013AC8407D00E20EDBC4CF6DCC926F004A90285FFA901859FA4EBA5CEC9CFB06020E8E320A1DFB0144820E8E38500688501D00CA500F04920A1DF90034C05DE209ADFB03CA49F3038C9C5D0034C0EDE20BDDEF02CAC0E07D023A404C0059007A54585004C4BDF20C4DEA9F025CE85CE20E8DEA900859F8D33048D8404A900851DA4EBC8C8A9028500C884EBA5CEC9209016C9E4B02820ECE3F00DC91CF009C96BF005209ADF9017A4EBC8A5CEC908900DC9D0B00920ECE3D005C600D0CB6020BDDEF061209ADF90034C2EDE20A1DFB05720DDDE9008AD0E07D04A4CFFDDA41DC000D03EA43388D039C96CF004C91FD031ADC403D004A01084FF09208DC403A586290FF00EA000AD1A07F001C8B903DE8DDE06A50EC907F00CC908D008A902850E60204BDF60A034201CDEEE48074CFEBBA9008D7207A9028D7007A9188557A402A90091064C4D8AF907FF001822506890A404C0069004C00A900160C924F004C925D039A50EC905F041A9018533EE2307A50EC904F01FA933201697A98085FC4A8D1307A204A5CE8D0F07DD29DEB003CAD0F88E0F01A904850E4C88DEC926D00AA5CEC920B004A901850EA903851DA90085578D0507A58638ED1C07C910B004A9028533A433A5060A0A0A0A187924DE8586A506D009AD1B07187926DE856D60C95FF002C9606020DDDE9013A9708D0907A9F98DDB06A9038D86074A8D0E0760C967F005C96818D0013860A50B2904F05CA500C911D056A501C910D050A9308DDE06A903850EA91085FFA9208DC403ADD606F03929030A0AAAA586C9609006E8C9A09001E8BCF287888C5F07BEB49CBDBC9C8D5007A98085FCA9008D51078D60078D5C078D5207EE5D07EE570760A900A457A600CAD00AE8C0003028A9FF4C66DFA202C001101DA901A0108C8507A0008457C90010018884001865868586A56D6500856D8A49FF2D90048D900460106188C420B0DFDD8BDF60246D8AC620B0DFDD96DF60C9C2F006C9C3F0021860A90185FE60A829C00A2A2AAA986001010202020510F0B51E2920D0F1205BE190ECB416C012D006B5CFC92590E0C00ED0034C63E1C005D0034C85E1C012F008C02EF004C007B07420AEE1D0034CE2E020B5E1F0F8C923D064A402A9009106B516C915B00CC906D003208EE1A9012011DAC9099010C911B00CC90A9004C90D900429019516B51E29F00902951ED6CFD6CFB516C907F007A9FDAC4E07D002A9FF95A0A0012043E11001C8B516C933F006C908F002944688B9BFDF955860A50438E908C905B072B51E2940D057B51E0A90034CFEE0B51EF0F9C905F01FC903B01AB51EC902D015A910B416C012D002A9009D9607A903951E204FE160B516C906F022C912D00EA9019546A9089558A5092907F010A0012043E11001C898D546D0032024E1204FE1B51E2980D005A900951E60B51E29BF951E60B516C903D004B51EF038B51EA80A9007B51E09404CFCE0B9B9DF951EB5CFC920901FA016A90285EBA5EBD546D00CA9012088E3F00520B5E1D008C6EBC8C01890E760E005F009B51E0A9004A90285FFB516C905D009A9008500A0FA4C37CA4C36DBB58738E5868500B56EE56D602063C3B5CF29F0090895CF60B5CF18693EC94460205BE1901AB5A0186902C903901120AEE1F00C20B5E1F007204FE1A9FD95A04CFEE020AEE1F01DC923D0082095D7A9FC95A060BD8A07D00CB51E2988951E204FE14CFEE0B51E0901951E60A900A0154C88E3C926F00EC9C2F00AC9C3F006C95FF002C96060B5D5C9189021209CE3F01C20B5E1F017B5A63018B53AD014A9FD95A6A901953AB5D529F895D560A900953A60A9809524A90285FF6002080E2003140D2002140E2002090E15000018060000200D0000300D0000080806040A08030E0D140002101504040C1C8A186907AAA002D0078A186909AAA006209CE24CDEE2A0488400A0444C52E2A0088400A004B58738ED1C078501B56EED1A0730060501F002A400982DD1039DD803D0194C7CE2E820F6F1CAC9FEB00D8A186901AAA001209CE24CDEE28A0A0AA8A9FF99B00499B10499B20499B304608600B9B8038502B9AD0385018A0A0A48A8BD99040A0AAAA501187DFDE199AC04A501187DFFE199AE04E8C8A502187DFDE199AC04A502187DFFE199AE0468A8A60060AD1C071869808502AD1A0769008501B586C502B56DE5019015B9AE04300DA9FFBEAC04300399AC0499AE04A60860B9AC041011C9A0900DA900BEAE04100399AE0499AC04A60860A2008406A9018507B9AC04DDAC04B02ADDAE049012F042B9AE04D9AC04903ADDAC04B035A40660BDAE04DDAC04902AB9AE04DDAC04B022A40660DDAC04F01ADDAE049015F013D9AE04900AF008B9AE04DDAC04B00418A40660E8C8C60710A938A40660488A186901AA684CA5E38A18690DAAA01B4CA3E3A01A8A186907AAA90020F0E3A608C9006000070E08030C02020D0D08030C02020D0D08030C02020D0D0800100414040404202008180818022020081808181220201818181818141406060810C8A9002CA901A200488404B9B0E31875868505B56D690029014A05056A4A4A4A20E19BA404B5CE1879CCE329F038E9208502A8B1068503A40468D005B5CE4C2BE4B586290F8504A50360FF00308400ADB903187933E4BE9A03BCE506840220AEE4ADAE03990302990B02991302186906990702990F02991702A921990202990A029912020940990602990E02991602A205A9E1990102C8C8C8C8CA10F4A402A500D005A9E0990102A200AD9D0338F90002C9649005A9F8990002C8C8C8C8E8E006D0E7A40060A206990002186908C8C8C8C8CAD0F3A402600400040000040004000800080800080080828183818380820303C3C3BCF306AD4707D008B52A297FC901F004A200F007A5094A4A2903AAADBE03187DC4E4990002187DCCE4990402ADB303187DC0E4990302187DC8E4990702BDD0E4990102BDD4E4990502BDD8E4990202990602A608ADD60329FCF009A900952AA9F820C1E560F950F750FAFBF8FBF6FBBCE506ADAE03990302186908990702990B0218690C8505B5CF20C1E56908990802AD0D018502A90185038504990202990602990A02A97E990102990902A97F990502AD0F07F0159818690CA8AD0F010AAABD41E58500BD42E520B2EBA608BCE506ADD103290EF014A9F8991402991002990C0299080299040299000260BCE5068402C8C8C8ADAE0320AEE4A608B5CF20BBE5AC4E07C003F005ACCC06F002A9F8BCE506991002991402A95BAE4307F002A975A608C820B5E5A902C820B5E5E820F6F1CABCE5060A489005A9F8990002680A489005A9F8990402680A489005A9F8990802680A489005A9F8990C02680A489005A9F8991002680A9005A9F8991402ADD1030A900320B3E560A5094AB002D6DBB5DB20C1E5ADB303990302186908990702A902990202990602A9F7990102A9FB9905024CBDE660616263BCF306B52AC902B0C6B5DB990002186908990402ADB303990302990702A5094A2903AABD82E6C820C1E588A902990202A982990602A6086076777879D6D6D9D98D8DE4E47677787902010201ACEA06ADB9031869088502ADAE038505A639BDCEE60DCA0385048A480A0AAAA90185078503BDBEE68500BDBFE620B2EBC60710F1ACEA0668F02FC903F02B8500A5094A29030DCA03990202990602A600CAF006990A02990E02B906020940990602B90E020940990E024C64EBFCFCAAABACADFCFCAEAFB0B1FCA5A6A7A8A9FCA0A1A2A3A469A56AA7A8A96BA06CA2A3A4FCFC96979899FCFC9A9B9C9DFCFC8F8E8E8FFCFC95949495FCFCDCDCDFDFDCDCDDDDDEDEFCFCB2B3B4B5FCFCB6B3B7B5FCFC70717273FCFC6E6E6F6FFCFC6D6D6F6FFCFC6F6F6E6EFCFC6F6F6D6DFCFCF4F4F5F5FCFCF4F4F5F5FCFCF5F5F4F4FCFCF5F5F4F4FCFCFCFCEFEFB9B8BBBABCBCFCFCBDBDBCBC7A7BDADBD8D8CDCDCECECFCF7D7CD18CD3D27D7C89888B8AD5D4E3E2D3D2D5D4E3E28B8AE5E5E6E6EBEBECECEDEDEEEEFCFCD0D0D7D7BFBEC1C0C2FCC4C3C6C5C8C7BFBECAC9C2FCC4C3C6C5CCCBFCFCE8E7EAE9F2F2F3F3F2F2F1F1F1F1FCFCF0F0FCFCFCFC0C0C000C0CA8543CEA184848CCC01818189024FF489CD2D8F0F6FC01020302010103030301010202210102010102FF02020101020202081818191A1918B5CF8502ADAE038505BCE50684EBA9008D0901B5468503BDC5038504B516C90DD00AB4583006BC8A07F00160B51E85ED291FA8B516C935D008A000A9018503A915C933D013C602A903BC8A07F00209208504A00084EDA908C932D008A003AE0E07BD78E885EF84ECA608C90CD007B5A03003EE0901AD6A03F009A016C901F001C884EFA4EFC006D01DB51EC9029004A20486EC29200D4707D00CA5092908D006A50349038503B95BE805048504B940E8AAA4ECAD6A03F030C901D013AD63031002A2DEA5ED2920F0038E09014C4BEAAD63032901F002A2E4A5ED2920F0EEA50238E91085024C46E9E024D011C005D00AA230A9028503A90585EC4CCAE9E090D012A5ED2920D009AD8F07C910B002A2964C37EAA5EFC904B010C002900CA25AA4EFC002D004A27EE602A5ECC904D01EA272E602A4EFC002F004A266E602C006D00CA254A5ED2920D004A28AC602A408A5EFC905D00CA5EDF0242908F05DA2B4D01CE048F018B99607C905B04EE03CD00DC901F046E602E602E6024C29EAA5EFC906F037C908F033C90CF02FC918B02BA000C915D010C8AD5F07C907B01DA2A2A90385ECD015A5093976E8D00EA5ED29A00D4707D0058A186906AAA5ED2920F00EA5EFC9049008A0018C09018884ECA4EB20AAEB20AAEB20AAEBA608BCE506A5EFC908D0034C64EBAD0901F03DB902020980C8C820B5E5888898AAA5EFC905F00DC911F009C915B0058A186908AABD010248BD050248B911029D0102B915029D05026899150268991102AD6A03D0B6A5EFA6ECC905D0034C64EBC907F01DC90DF019C90CF015C912D004E005D048C915D005A942991602E002903BAD6A03D036B9020229A3990202990A029912020940E005D0020980990602990E02991602E004D013B90A020980990A029912020940990E02991602A5EFC911D036AD0901D021B912022981991202B916020941991602AE8F07E010B030990E022981990A029026B902022981990202B906020941990602A5EFC9189010A982990A029912020940990E02991602A608ADD1034A4A4A489005A90420C1EB684A489005A90020C1EB684A4A489005A91020B7EB684A489005A90820B7EB684A901220B7EBB516C90CF009B5B6C902D0032098C960BD3EE78500BD3FE785014C82F2187DE506A8A9F84CC1E5187DE506A8204AEC9910026085858686ADBC038502ADB1038505A90385044A8503BCEC06A200BDCDEB8500BDCEEB20B2EBE004D0F1A608BCEC06AD4E07C901F008A986990102990502BDE803C9C4D024A987C820BBE588A903AE4E07CAF0014AA60899020209409906020980990E022983990A02ADD403482904F008A9F8990402990C02682908F008A9F899000299080260A9028500A975A40EC005F006A9038500A984BCEC06C820BBE5A5090A0A0A0A29C00500C820BBE58888ADBC0320C1E5ADB103990302BDF10338ED1C07850038EDB10365006906990702ADBD03990802990C02ADB203990B02A50038EDB20365006906990F02ADD4032046ECADD4030A9005A9F820C1E5A5001010B90302D907029008A9F8990402990C0260BCF106ADBA03990002ADAF03990302A5094A4A4829014964990102684A4AA902900209C099020260686766BCEC06B524F6244A2907C903B04AAABD06EDC820BBE588A608ADBA0338E904990002990802186908990402990C02ADAF0338E904990302990702186908990B02990F02A902990202A982990602A942990A02A9C2990E0260A900952460BCE506A95BC820B5E5C8A90220B5E58888ADAE03990302990F02186908990702991302186908990B02991702B5CFAA48E020B002A9F820BEE568186980AAE020B002A9F8990C02991002991402ADD103482908F008A9F8990002990C0268482904F008A9F8990402991002682902F008A9F8990802991402A60860A4B588D020ADD3032908D019BCEE06ADB003990302ADBB03990002A974990102A902990202602028C818004050588088B87860A0B0B8000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F2021222324252627080928292A2B2C2D08090A0B0C302C2D08090A0B2E2F2C2D080928292A2B5C5D08090A0B0C0D5E5FFCFC080958595A5A080928292A2B0E0FFCFCFCFC32333435FCFCFCFC36373839FCFCFCFC3A373B3CFCFCFCFC3D3E3F40FCFCFCFC32414243FCFCFCFC32334445FCFCFCFC32334447FCFCFCFC32334849FCFCFCFC32339091FCFCFCFC3A379293FCFCFCFC9E9E9F9FFCFCFCFC3A374F4FFCFC00014C4D4E4E00014C4D4A4A4B4B3146AD9E07F005A5094AB040A50EC90BF047AD0B07D03CAC0407F031A51DC900F02B2034EFA5092904D021AAACE406A5334AB004C8C8C8C8AD5407F009B91902CDB5EEF007E8BDE7EE9919026020ECEF4C45EF20B0F04C45EFA00EB907EE8DD506A90420BEEF20E9F0AD1107F025A000AD8107CD11078C1107B0188D1107A007B907EE8DD506A004A557050CF001889820BEEFADD0034A4A4A4A8500A203ADE406186918A8A9F84600900320C1E59838E908A8CA10EF6058010060FF04A205BD9EEF9502CA10F8A2B8A00420DCEFAD260209408D2202608507ADAD038D55078505ADB8038502A5338503ADC4038504AED506ACE406BD17EE8500BD18EE20B2EBC607D0F160A51DC903F052C902F03EC901D011AD0407D051A006AD1407D022A0004C28F0A006AD1407D016A002A557050CF00EAD0007C909901BA5452533D015C82091F0A9008D0D07B907EE60A0042091F04C62F0A0042091F04C68F0A005A59FF0DE2091F04C6DF0A0012091F0AD82070D0D07D00BA50A0AB006AD0D074CD0F0A9034C6FF0A90285002062F048AD8107D015AD0C078D8107AD0D07186901C5009002A9008D0D076860AD5407F00598186908A8600001000100010200010202000200020002000200AC0D07A5092903D00DC8C00A9005A0008C0B078C0D07AD5407D00CB99CF0A00F0A0A0A7907EE609818690AAAA009BD9CF0D002A001B907EE60ACE406A50EC90BF013ADD506C950F01EC9B8F01AC9C0F016C9C8D024B91202293F991202B91602293F0940991602B91A02293F991A02B91E02293F0940991E0260A200A0004C42F1A00120A8F1A0034C42F1A00020A8F1A0022071F1A60860A00220A8F1A0064C42F1A901A0014C65F1A909A0042065F1E8E8A909C88600186500AA2071F1A60860B5CE99B803B58638ED1C0799AD0360A200A0004CC0F1A00020A8F1A0024CC0F1A00120A8F1A0034CC0F1A00220A8F1A0064CC0F107160D8A1879A5F1AA60A901A0014CBAF1A909A0048600186500AA984820D7F10A0A0A0A0500850068A8A50099D003A6086020F6F14A4A4A4A85004C39F27F3F1F0F0703010080C0E0F0F8FCFEFF070F078604A001B91C0738F5868507B91A07F56DBEF3F1C9003010BEF4F1C9011009A9388506A908206DF2BDE3F1A604C900D0038810D06000080C0E0F07030100040004FF008604A001B937F238F5CE8507A901F5B5BE34F2C9003010BE35F2C9011009A9208506A904206DF2BD2BF2A604C900D0038810D1608505A507C506B00C4A4A4A2907C001B0026505AA60A5034A4AA500900C990502A501990102A940D00A990102A501990502A9000504990202990602A502990002990402A505990302186908990702A502186908850298186908A8E8E86078EECBF2FFFFAD7007D0048D154060A9FF8D1740A90F8D1540ADC607D006A5FAC901D05DADB207D023A5FAF0668DB2078DC607A9008D154085F185F285F3A90F8D1540A92A8DBB07A944D011ADBB07C924F008C91EF0F1C918D009A964A284A07F2088F3CEBB07D02AA9008D1540ADB207C902D005A9008DC607A9008DB207F012201BF4207CF52067F62094F6A90085FB85FCA90085FF85FE85FD85FAACC007A5F42903F007EEC007C030900698F003CEC0078C1140608C01408E0040602081F3A200A8B901FFF00B9D0240B900FF09089D0340608E04408C054060209FF3A204D0E0A208D0DC9F9B989695949290909A97959392A9408DBB07A962208BF3A299D025A926D002A918A282A0A72088F3A9288DBB07ADBB07C925D006A25FA0F6D008C920D029A248A0BC2081F3D020A905A099D004A90AA093A29E8DBB07A90C2088F3ADBB07C906D005A9BB8D0140D060A4FFF02084F130AA46FFB0AA46FFB0D446FFB02C46FFB04A46FFB07F46FFB0BE46FFB080A5F1F017309A4AB0974AB0C24AB01B4AB03C4AB0674AB0B64AB04860A90E8DBB07A09CA29EA9262088F3ACBB07B9B0F38D0040C006D005A99E8D0240D025A90EA0CBA29F8DBB07A9282088F3D015ACBB07C008D009A9A08D0240A99FD002A9908D0040CEBB07D00EA20086F1A20E8E1540A20F8E154060A92F8DBB07ADBB074AB0104AB00D2902F009A091A29AA9442088F34CA2F4580254564E444C524C483E363E3630284A504A643C323C322C243A643A342C222C221C141404222416042426180426281A04282A1C042A2C1E042C2E20042E3022043032A935A28DD004A906A2988DBD07A07FA94220A6F3ADBD07C930D005A9548D0640D02EA9208DBD07A094A95ED00BADBD07C918D01CA093A918D07FA9368DBD07ADBD074AB00BA8B9D9F4A25DA07F20A6F3CEBD07D00EA20086F2A20D8E1540A20F8E154060A5F22940D065A4FEF02084F2303E46FEB08A46FEB06A46FEB06A46FEB0A046FEB08046FEB0B046FEB03CA5F2F01730274AB0134AB05D4AB05A4AB08D4AB0074AB0994AB026604C2CF54C68F5A9388DBD07A0C4A918D00BADBD07C908D08EA0A4A95AA29FD083A9308DBD07ADBD07A2034AB0D6CAD0FAA8B9D3F4A282A07FD0E4A910D002A9208DBD07A97F8D0540A9008DBE07EEBE07ADBE074AA8CCBD07F00CA99D8D0440B9F8F420A9F3604C6DF5010E0E0D0B060C0F0A09030D080D060CA9208DBF07ADBF074A9012A8BE2BF6B9EAFF8D0C408E0E40A9188D0F40CEBF07D009A9F08D0C40A90085F360A4FDF00A84F346FDB0CA46FDB00BA5F3F0064AB0C44AB00660A9408DBF07ADBF074AA8A20FB9C9FFD0BC4C3AF7A5FCD00CA5FBD02CADB10705F4D0EE608DB107C901D00620A7F42071F5A6F48EC507A0008CC40784F4C940D030A2088EC407D029C904D00320A7F4A0108CC707A0008CB10785F4C901D00EEEC707ACC707C032D00CA011D0E4A00884F7C84A90FCB90CF9A8B90DF985F0B90EF985F5B90FF985F6B910F985F9B911F985F8B912F98DB0078DC107A9018DB4078DB6078DB9078DBA07A90085F78DCA07A90B8D1540A90F8D1540CEB407D05FA4F7E6F7B1F5F004103DD02FADB107C940D005ADC507D01D2904D01CA5F4295FD013A90085F48DB1078D0840A9908D00408D0440604CD4F64CA4F620CBF88DB307A4F7E6F7B1F5A6F2D00E20A9F3F00320D8F88DB507209FF3ADB3078DB407A5F2D01AADB1072991D013ACB507F003CEB50720F4F88D0440A27F8E0540A4F8F05ACEB607D032A4F8E6F8B1F5D00FA9838D0040A9948D01408DCA07D0E920C5F88DB607A4F1D0348A293E208BF3F00320D8F88DB7072081F3A5F1D01FADB1072991D00EACB707F003CEB70720F4F88D0040ADCA07D002A97F8D0140A5F9CEB907D04CA4F9E6F9B1F5F041101320CBF88DB807A91F8D0840A4F9E6F9B1F5F02C20ADF3AEB8078EB907ADB107296ED006A5F4290AF0198AC912B00FADB1072908F004A90FD006A91FD002A9FF8D0840A5F429F3F051CEBA07D04CACB007EEB007B1F5D008ADC1078DB007D0EE20C5F88DBA078A293EF024C930F018C920F00C2910F018A91CA203A018D012A91CA20CA018D00AA91CA203A058D002A9108D0C408E0E408C0F4060AA6A8A2A2A2A29071865F06DC407A8B966FF60ADB1072908F004A904D00CA5F4297DF004A908D002A928A282A07F60ADB1072908F004B996FF60A5F4297DF004B99AFF60B9A2FF60A5595464593C314B695E464F368D364B8D69696F756F7B6F756F7B8187818D69699399939F9399939F8187818D9399939F0872FC271820B8F92E1A4020B0FC3D2120C4FC3F1D1811FD0000081CFA0000A4FB936210C8FE24141845FC1E140852FDA070680851FE4C241801FA2D1CB81849FA2012701875FA1B1044189DFA110A1C18C2FA2D105818DBFA140D3F18F9FA150D211825FB18107A184BFB190F541874FB1E122B1872FB1E0F2D842C2C2C82042C04852C842C2C2A2A2A82042A04852A842A2A001F1F1F981F1F989E981F1D1D1D941D1D949C941D861885263084042630861485222C8404222C21D0C4D031D0C4D000852C221C84262A822826048722343A82400436843A34822C30852A005D554D15199615D5E3EB2DA62B279C9E5985221C14841E2282201E04871C2C348236043034042C04262A85228404823A38363204340424262C04262C300005B4B2B02BAC849C9EA284949C9E851422842C851E822C842C1E8404823A38363204340464046486640005B4B2B02BAC8437B6B64585141C8222842C4E824E844E22840485328530862C040005A4059E059D8584148524282C822284221421D0C4D031D0C4D000822C842C2C822C3004342C0426862200A42525A429A21D9C95822C2C042C042C308534040400A42525A4A86304850E1A8424852214840C8234843434822C8434863A0400A02121A0212B05A3821884181882181804863A2231903190317131909090008234842C852284248226360436862600AC275D1D9E2DAC9F8514822084222C1E1E822C2C1E04872A4040403A3682342C0426862200E3F7F7F7F5F1AC279E9D8518821E84222A2222822C2C22048604822A36043687363430862C040000686A6C45A231B0F1EDEBA21D9C958604852282228722262A842C22861451903111008022282226222422262228222A2228222622282226222422262228222A222822262026202420262028202620282026202420262024202620282026202820262024283028322830282E2830282E282C282E283028322830282E2830282E282C282E0004706E6C6E7072706E706E6C6E7072706E6E6C6E706E706E6C6E6C6E706E706E6C76787674767472747678767476747274841A831820841E831C28261C1A1C822C0404220404841C87262A26842428248022009C0594050D9F1E9C989D822204041C04048414861E80168014811C30043030041E3204323204203404343404360484360046A464A448A666A64AA868A86A442B812A42044242042C64046464042E4604464604220484228704060C141C22862C228704600E141A24862C2487040810181E28863030806400CDD5DDE3EDF5BBB5CFD5DBE5EDF3BDB3D1D9DFE9F1F7BFFFFFFF3400860487141C228634842C04040487141A248632842C04860487181E28863687303030802C82142C62261028800482142C62261028800482081E5E18601A800482081E5E18601A8604831A181684141A180E0C168314201E1C282687241A1210620E8004040082181C20222628812A2A2A042A04832A822286343234810422262A2C3086348332823684348504812286302E30810422262A2C2E863083228236843485048122863A3A3A823A81408204813A863636368236813A82048136863482262A368134348534812A862C008490B0845050B000989694929496585858445C449FA3A1A385A3E0A623C49F9D9F859FD2A623C4B5B1AF85B1AFAD85959EA2AA6A6A6B5E9D840404822286228214222C12222A14222C1C222C14222C12222A14222C1C222C18222A16202818222A12222A18222A12222A14222C0C222C14223412223010222E1622341826361626361426361222365C22340C2222811E1E851E81128614812C221C2C221C852C04812E241E2E241E852E0481322822322822853287363636843A005C544C5C544C5C1C1C5C5C5C5C5E564E5E564E5E1E1E5E5E5E5E625A50625A5062222262E7E7E72B8614811480141481141414148616811680161681161616168128221A28221A288028288128872C2C2C84308304840C8362108412831C221E2226181E041C00E3E1E31DDEE023EC7574F0F4F6EA312D83121404181A1C1426221E1C181E220C14FFFFFF0088002F000002A60280025C023A021A01DF01C401AB0193017C016701530140012E011D010D00FE00EF00E200D500C900BE00B300A900A00097008E00860077007E007100540064005F0059005000470043003B0035002A00230475035702F902CF01FC006A050A1428501E3C02040810204018300C03060C183012240836030906121B240C240206040C1218081201030206090C0498999A9B90949495959697989091929293939394949494949495959595959596969696969696969696969696969696969595949315161617171819191A1A1C1D1D1E1E1F1F1F1F1E1D1C1E1F1F1E1D1C1A18161415161617171819191A1A1C1D1D1E1E1F6B6F64653534"))

(defn file-upload []
  [:div
   [:h1 "File upload"]
   [:input#input
    {:type      "file"
     :on-change 
     (fn [e]
       (let [dom    (o/get e "target")
             file   (o/getValueByKeys dom #js ["files" 0])
             reader (js/FileReader.)]
         (.readAsArrayBuffer reader file)
         (set! (.-onload reader)
               #(reset! file-atom (-> % .-target .-result
                                            (js/Uint8Array.)
                                            crypt/byteArrayToHex
                                            .toUpperCase)))))}]])

[file-upload]

We have read the file in as an array buffer and stored it in file-atom as one long string:

@file-atom

If you have uploaded your own file, reload the snippet by deleting the last character and reentering it.

These bytes are ripped straight from the game, and prepended with a header describing the music data to follow:

(defn hex-bytes
  ([file n] (hex-bytes file n (inc n)))
  ([file from to]
   (map #(apply str %)
        (partition 2 (take (- (* 2 to) (* 2 from))
                           (drop (* 2 from) file))))))

(defn hex->ascii [bytes]
  (apply str (map #(js/String.fromCharCode (str "0x" %)) bytes)))

(defn word [file offset]
  (str "0x" (first (hex-bytes file (inc offset)))
       (first (hex-bytes file offset))))

(defn header [file]
       [:div
        [:p (str "Version number: " (first (hex-bytes file 0x05)))]
        [:p (str "Total songs: " (js/parseInt (str "0x" (first (hex-bytes file 0x06)))))]
        [:p (str "Starting song: " (js/parseInt (str "0x" (first (hex-bytes file 0x07)))))]
        [:p (str "Load address: " (word file 0x08))]
        [:p (str "Init address: " (word file 0x0a))]
        [:p (str "Play address: " (word file 0x0c))]
        [:p (str "Song name: " (hex->ascii (hex-bytes file 0x0e 0x2e)))]
        [:p (str "Artist: " (hex->ascii (hex-bytes file 0x2e 0x4e)))]
        [:p (str "Copyright: " (hex->ascii (hex-bytes file 0x4e 0x6e)))]
        [:p (str "Play speed (NTSC): " (js/parseInt (word file 0x6e)) " (in 1/1000000th sec ticks)")]
        [:p (str "Bankswitch init values: " (hex-bytes file 0x70 0x78))]
        [:p (str "Play speed (PAL): " (js/parseInt (word file 0x78)) " (in 1/1000000th sec ticks)")]
        [:p (str "PAL/NTSC: " (first (hex-bytes file 0x7a)))]
        [:p (str "Extra Sound Chip Support: "
                 (let [byte (first (hex-bytes file 0x7b))]
                   (case byte
                     "01" "This song uses VRC6 audio"
                     "02" "This song uses VRC7 audio"
                     "04" "This song uses FDS audio"
                     "08" "This song uses MMC5 audio"
                     "10" "This song uses Namco 163 audio"
                     "20" "This song uses Sunsoft 5B audio"
                     "none")))]])

[header @file-atom]

Now we can load the music data into the RAM banks so that offset $080 in the file lines up with the proper load address (ignoring bankswitching for now):

(defn format-hex [s]
  (let [length (count s)]
    (str "$"
         (apply str (repeat (- 4 length) "0"))
         s)))

(defn bank-selector [file]
  (let [bank (r/atom 0)
        hovered (r/atom nil)]
    (fn []
      [:div
       [:h4 (str "Select 4KB bank: " @bank)]
       [:span
        [:svg {:width     30
               :view-box  "0 -0.5 10 11"
               :transform (str "translate(0,10)" (when-not (= @hovered :left) "scale(0.7)"))
               :cursor    "pointer"
               :on-mouse-over #(reset! hovered :left)
               :on-mouse-out #(reset! hovered nil)
               :on-click  #(if (= 0 @bank)
                             (reset! bank 7)
                             (swap! bank dec))}
         [:path {:stroke "#000000"
                 :d      "M4 0h1M3 1h2M2 2h1M4 2h3M1 3h1M0 4h1M1 5h1M2 6h1M4 6h3M3 7h2M4 8h1"}]
         [:path {:stroke "#f8f800"
                 :d      "M3 2h1M2 3h5M1 4h6M2 5h5M3 6h1"}]]
        (str (apply str (interpose " - " (map #(-> % (.toString 16) format-hex) (take 2 (drop @bank (iterate #(+ 4096 %) (js/parseInt (word @file-atom 0x08)))))))))
        [:svg {:width         30
               :view-box      "0 -0.5 10 11"
               :transform     (str "translate (0,5),rotate (180)" (when-not (= @hovered :right) "scale(0.7)"))
               :cursor        "pointer"
               :on-mouse-over #(reset! hovered :right)
               :on-mouse-out  #(reset! hovered nil)
               :on-click      #(if (= 7 @bank)
                                 (reset! bank 0)
                                 (swap! bank inc))}
         [:path {:stroke "#000000"
                 :d      "M4 0h1M3 1h2M2 2h1M4 2h3M1 3h1M0 4h1M1 5h1M2 6h1M4 6h3M3 7h2M4 8h1"}]
         [:path {:stroke "#f8f800"
                 :d      "M3 2h1M2 3h5M1 4h6M2 5h5M3 6h1"}]]]
       [:br]
       [:div
        [:textarea
         {:rows      30
          :cols      74
          :value   (let [offsets (take 2 (drop @bank (iterate #(+ 4096 %) 0)))]
                     (apply str (interpose " " (hex-bytes file (first offsets) (last offsets)))))
          :read-only true}]]])))

[bank-selector (drop (* 2 0x80) @file-atom)]

Permalink

Ruby, Where do We Go Now?

This article is part of the “Meta Advent 2019” series. I’ve committed to writing a new blog post here every day until Christmas.

A while ago I wrote an article, where I made the case Ruby seems to have lost its way. I thought I had exhausted the topic, but a few of the recent developments around the Ruby 2.7’s development lead me to revisit it.

Ruby 2.7 has become infamous at this point for several highly controversial changes that were either quickly revised or removed. A classic tell-tale of good project management and a clear vision for the future, right?

Let’s examine a few of those ill-fated changes in no particular order…

Flip-flops Forever

Remember how the flip-flop operator (..) was finally deprecated in Ruby 2.6 after years of discussions? Well, it just got (quietly) reinstated.

I find it really amusing how this went down - 3 (!!!) people complained about the deprecation and Matz quickly responded with:

I hear the negative feedback from the community. OK, I give up. The warning should be removed.

– Matz

There are numerous other instances where he ignored the complaints of way more people. Go figure. As usual, there was no real discussion on the ticket - just a decision. Not to mention that no one even thought to try to solicit more community feedback on the subject. I learned about the removal of the deprecation when someone posted this on Twitter…

None of this is new, in Ruby-land it’s business as usual. It’s just becoming progressively more frustrating for me.

Numbered Parameters Revisited

Remember the highly controversial numbered block parameter syntax?

# now
[1, 2, 3].each { |i| puts i }
(1..10).map { |i| i * 3 }
(1..9).each_slice(3).map { |x, y, z| x + y + z }

# Ruby 2.7
[1, 2, 3].each { puts @1 }
(1..10).map { @1 * 3 }
(1..9).each_slice(3).map { @1 + @2 + @3 }

After quite the backlash this syntax was revised to something more reasonable:

[1, 2, 3].each { puts _1 }
(1..10).map { _1 * 3 }
(1..9).each_slice(3).map { _1 + _2 + _3 }

A great success, right? Matz made a couple of interesting comments on this mega-thread. My favourite one was:

Yes, I admit { @1[@2] = “Go Fish: #{@2}” } is cryptic. But {@1 * @2} is not. So use numbered parameters with care (just like other features in Ruby). The possibility to make code cryptic itself should not be the reason to withdraw a feature.

The problem is introducing it or this could break existing code. That is the problem. Numbered parameters are a compromise to simplify block expression without breaking compatibility.

FYI, I am not fully satisfied with the beauty of the code with numbered parameters, so I call it a compromise.

– Matz

So, he accepted something that he wasn’t perfectly happy about. OK, I can understand the need for compromises from time to time. But I totally hate this line of reasoning “The possibility to make code cryptic itself should not be the reason to withdraw a feature.”. I think that adding features for the sake of adding them is not a reason to add something either.

I also think that we should be careful with the impact of features on tools and with all the accidental complexity they bring (e.g. now Ruby parsers are more complex). Not to mention the fact that no one could make a strong case for introducing this feature in the first place. I’m still not sure what’s the grand problem it solves…

The Totally Useless Pipeline Operator

This was probably the most ridiculous part of Ruby 2.7’s development cycle. At some point the pipeline operator landed in master:

# now
foo()
  .bar(1, 2)
  .display

# Ruby 2.7
foo()
  |> bar 1, 2
  |> display

Everyone who has actually used the pipeline operator in Elixir (the source of inspiration for this change), however, knows that it behaves nothing like this. Here Ruby’s team basically came up with different syntax for . (method invocation) with a lower precedence that allowed you to do questionable things like:

(a..b).each {}

a..b |> each {}

I won’t go into much details about this misguided operator. After all, a lot of great articles exist about it already.1 For me the important things here are:

  • This feature was not well thought-out
  • It generated a massive backlash
  • It was quickly reverted

Sounds familiar, doesn’t it?

The Rise and Fall of the Method Reference Operator

Ruby 2.7 was supposed to introduce a method reference operator:

[2, 4, 8, 16, 32].map { |n| Math.log2(n) }

[2, 4, 8, 16, 32].map(&Math.:log2)

Well, not anymore. This feature was recently reverted. There were some good reasons for it to be reverted, which serves to underline my point that a lot of changes to Ruby recently were made without thinking much about their implications. Matz, made one particularly interesting comment on this ticket:

In recent years, we have added/discussed a lot of features inspired by functional programming. But those features are not designed with a grand design. Those are rather ad hoc additions. We felt the future, more functional Ruby should be designed with the big picture of functional Ruby programming, which we lack currently.

Besides that, I don’t like the specific operator (.:) which looks like braille.

So you don’t have to consider this revert as a rejection of this whole idea, but an invitation to the next try.

– Matz

I was really glad to see this, as it seems that Matz also understands that without a solid vision in the end we’ll just get one big mess. I’m a bit sad about the .: operator being reverted, as this was a feature I could see myself using, but that’s a small price to pay for a better future of Ruby.

Parameter Forwarding Weirdness

Just when I had some hope about the future I came across another “brilliant” addition to Ruby 2.7 - parameter forwarding. It’s best explained with an example:

def foo(...)
  bar(...)
end

This defines a method foo, that can have any parameters whatsoever and simply forwards all of them to bar. I don’t know about you, but I always wanted something like this!2

There are some plans to make it a bit more generic and support forwarding a subset of all parameters:

def foo(a, b, ...)
  if a.even?
    bar(b, ...)
  end
end

Still, I think this is another random addition that doesn’t solve real problems and just adds mental and technical complexity.

A RuboCop Perspective

You might be wondering I am so grumpy about a bit of chaos during the development cycle of Ruby? After all it seems that most problematic features tend to get improved/removed.

I’m the author of RuboCop, a popular static code analyzer and formatter for Ruby, that’s why my perspective on all of this is probably different from that of many other people. Together with RuboCop and Parser’s team we actually have to support all of those language changes. Imagine how fun it is to add support for something, change this a couple of times, while tracking upstream changes, and then just delete it at the end. Or to have to do complex changes to support some new syntax that probably is going to be used by a handful of people… Trust me, that’s get frustrating after a while, and we’d definitely appreciate less turbulence in the design of the language.

You might say now - why don’t you just wait for the stable release of Ruby before implementing support for it? Have you ever seen users who don’t want support for the latest and greatest feature at day one? Yeah, I haven’t seen many of those either. :-) Not to mention that when you’re in the business of parsing code unknown syntax doesn’t result in a very pleasant end user experience.

Epilogue

So, we made it to the end of this somewhat bitter and rantish article. So, what’s the takeaway from all of this, if any?

It’s good that Ruby’s team listens to user feedback, but it might also be nice for them to actually start asking about feedback in some more open/structured manner prior to making language changes. Especially given their recent track record with the Ruby community… You can’t really expect everyone to be monitoring all tickets in Ruby’s issue tracker, right?3 I’ve often dreamt of a state where Ruby’s team actually announces in advance the focus for the next release (a.k.a. a roadmap), so interested parties can actually go and help with the tasks at hand. That should not require that much extra work - just a bit of good old planning. Some voting system on tickets and annual surveys might be a good idea as well - other programming communities are making good use of those.

Many people are excited about Ruby 3.0, but I’m not. It’s a major release in theory, but in practice it will add relatively little to what we have today. It’s not even clear if the new concurrent programming model will make it there or not at this point.

Ruby 2.7 has showed better than ever the chaos in Ruby’s development and the lack of a clear direction/destination. There are some signs that Matz understands this as well, so I’m still optimistic about the future.

I think Matz should take a page from Rich Hickey’s4 book and take a bit of hammock-time to figure out where he wants to take Ruby next. So, Ruby, where do we go now?

P.S. The article generated an interesting discussion on reddit. I encourage you to check it out.

  1. That’s my favourite one https://dev.to/baweaver/ruby-2-7-the-pipeline-operator-1b2d

  2. You can’t here my voice, but that’s sarcasm here. 

  3. Especially given the fact that the vast majority of ideas for language changes are just ignored and stay dormant for ages. 

  4. The author of Clojure. 

Permalink

Copyright © 2009, Planet Clojure. No rights reserved.
Planet Clojure is maintained by Baishamapayan Ghose.
Clojure and the Clojure logo are Copyright © 2008-2009, Rich Hickey.
Theme by Brajeshwar.