I am sorry, but everyone is getting syntax highlighting wrong

Translations: Russian

Syntax highlighting is a tool. It can help you read code faster. Find things quicker. Orient yourself in a large file.

Like any tool, it can be used correctly or incorrectly. Let’s see how to use syntax highlighting to help you work.

Christmas Lights Diarrhea

Most color themes have a unique bright color for literally everything: one for variables, another for language keywords, constants, punctuation, functions, classes, calls, comments, etc.

Sometimes it gets so bad one can’t see the base text color: everything is highlighted. What’s the base text color here?

The problem with that is, if everything is highlighted, nothing stands out. Your eye adapts and considers it a new norm: everything is bright and shiny, and instead of getting separated, it all blends together.

Here’s a quick test. Try to find the function definition here:

and here:

See what I mean?

So yeah, unfortunately, you can’t just highlight everything. You have to make decisions: what is more important, what is less. What should stand out, what shouldn’t.

Highlighting everything is like assigning “top priority” to every task in Linear. It only works if most of the tasks have lesser priorities.

If everything is highlighted, nothing is highlighted.

Enough colors to remember

There are two main use-cases you want your color theme to address:

  1. Look at something and tell what it is by its color (you can tell by reading text, yes, but why do you need syntax highlighting then?)
  2. Search for something. You want to know what to look for (which color).

1 is a direct index lookup: color → type of thing.

2 is a reverse lookup: type of thing → color.

Truth is, most people don’t do these lookups at all. They might think they do, but in reality, they don’t.

Let me illustrate. Before:

After:

Can you see it? I misspelled return for retunr and its color switched from red to purple.

I can’t.

Here’s another test. Close your eyes (not yet! Finish this sentence first) and try to remember what color your color theme uses for class names?

Can you?

If the answer for both questions is “no”, then your color theme is not functional. It might give you comfort (as in—I feel safe. If it’s highlighted, it’s probably code) but you can’t use it as a tool. It doesn’t help you.

What’s the solution? Have an absolute minimum of colors. So little that they all fit in your head at once. For example, my color theme, Alabaster, only uses four:

  • Green for strings
  • Purple for constants
  • Yellow for comments
  • Light blue for top-level definitions

That’s it! And I was able to type it all from memory, too. This minimalism allows me to actually do lookups: if I’m looking for a string, I know it will be green. If I’m looking at something yellow, I know it’s a comment.

Limit the number of different colors to what you can remember.

If you swap green and purple in my editor, it’ll be a catastrophe. If somebody swapped colors in yours, would you even notice?

What should you highlight?

Something there isn’t a lot of. Remember—we want highlights to stand out. That’s why I don’t highlight variables or function calls—they are everywhere, your code is probably 75% variable names and function calls.

I do highlight constants (numbers, strings). These are usually used more sparingly and often are reference points—a lot of logic paths start from constants.

Top-level definitions are another good idea. They give you an idea of a structure quickly.

Punctuation: it helps to separate names from syntax a little bit, and you care about names first, especially when quickly scanning code.

Please, please don’t highlight language keywords. class, function, if, elsestuff like this. You rarely look for them: “where’s that if” is a valid question, but you will be looking not at the if the keyword, but at the condition after it. The condition is the important, distinguishing part. The keyword is not.

Highlight names and constants. Grey out punctuation. Don’t highlight language keywords.

Comments are important

The tradition of using grey for comments comes from the times when people were paid by line. If you have something like

of course you would want to grey it out! This is bullshit text that doesn’t add anything and was written to be ignored.

But for good comments, the situation is opposite. Good comments ADD to the code. They explain something that couldn’t be expressed directly. They are important.

So here’s another controversial idea:

Comments should be highlighted, not hidden away.

Use bold colors, draw attention to them. Don’t shy away. If somebody took the time to tell you something, then you want to read it.

Two types of comments

Another secret nobody is talking about is that there are two types of comments:

  1. Explanations
  2. Disabled code

Most languages don’t distinguish between those, so there’s not much you can do syntax-wise. Sometimes there’s a convention (e.g. -- vs /* */ in SQL), then use it!

Here’s a real example from Clojure codebase that makes perfect use of two types of comments:

Disabled code is gray, explanation is bright yellow

Light or dark?

Per statistics, 70% of developers prefer dark themes. Being in the other 30%, that question always puzzled me. Why?

And I think I have an answer. Here’s a typical dark theme:

and here’s a light one:

On the latter one, colors are way less vibrant. Here, I picked them out for you:

Notice how many colors there are. No one can remember that many.

This is because dark colors are in general less distinguishable and more muddy. Look at Hue scale as we move brightness down:

Basically, in the dark part of the spectrum, you just get fewer colors to play with. There’s no “dark yellow” or good-looking “dark teal”.

Nothing can be done here. There are no magic colors hiding somewhere that have both good contrast on a white background and look good at the same time. By choosing a light theme, you are dooming yourself to a very limited, bad-looking, barely distinguishable set of dark colors.

So it makes sense. Dark themes do look better. Or rather: light ones can’t look good. Science ¯\_(ツ)_/¯

But!

But.

There is one trick you can do, that I don’t see a lot of. Use background colors! Compare:

The first one has nice colors, but the contrast is too low: letters become hard to read.

The second one has good contrast, but you can barely see colors.

The last one has both: high contrast and clean, vibrant colors. Lighter colors are readable even on a white background since they fill a lot more area. Text is the same brightness as in the second example, yet it gives the impression of clearer color. It’s all upside, really.

UI designers know about this trick for a while, but I rarely see it applied in code editors:

If your editor supports choosing background color, give it a try. It might open light themes for you.

Bold and italics

Don’t use. This goes into the same category as too many colors. It’s just another way to highlight something, and you don’t need too many, because you can’t highlight everything.

In theory, you might try to replace colors with typography. Would that work? I don’t know. I haven’t seen any examples.

Using italics and bold instead of colors

Myth of number-based perfection

Some themes pay too much attention to be scientifically uniform. Like, all colors have the same exact lightness, and hues are distributed evenly on a circle.

This could be nice (to know if you have OCD), but in practice, it doesn’t work as well as it sounds:

OkLab l=0.7473 c=0.1253 h=0, 45, 90, 135, 180, 225, 270, 315

The idea of highlighting is to make things stand out. If you make all colors the same lightness and chroma, they will look very similar to each other, and it’ll be hard to tell them apart.

Our eyes are way more sensitive to differences in lightness than in color, and we should use it, not try to negate it.

Let’s design a color theme together

Let’s apply these principles step by step and see where it leads us. We start with the theme from the start of this post:

First, let’s remove highlighting from language keywords and re-introduce base text color:

Next, we remove color from variable usage:

and from function/method invocation:

The thinking is that your code is mostly references to variables and method invocation. If we highlight those, we’ll have to highlight more than 75% of your code.

Notice that we’ve kept variable declarations. These are not as ubiquitous and help you quickly answer a common question: where does thing thing come from?

Next, let’s tone down punctuation:

I prefer to dim it a little bit because it helps names stand out more. Names alone can give you the general idea of what’s going on, and the exact configuration of brackets is rarely equally important.

But you might roll with base color punctuation, too:

Okay, getting close. Let’s highlight comments:

We don’t use red here because you usually need it for squiggly lines and errors.

This is still one color too many, so I unify numbers and strings to both use green:

Finally, let’s rotate colors a bit. We want to respect nesting logic, so function declarations should be brighter (yellow) than variable declarations (blue).

Compare with what we started:

In my opinion, we got a much more workable color theme: it’s easier on the eyes and helps you find stuff faster.

Shameless plug time

I’ve been applying these principles for about 8 years now.

I call this theme Alabaster and I’ve built it a couple of times for the editors I used:

It’s also been ported to many other editors and terminals; the most complete list is probably here. If your editor is not on the list, try searching for it by name—it might be built-in already! I always wondered where these color themes come from, and now I became an author of one (and I still don’t know).

Feel free to use Alabaster as is or build your own theme using the principles outlined in the article—either is fine by me.

As for the principles themselves, they worked out fantastically for me. I’ve never wanted to go back, and just one look at any “traditional” color theme gives me a scare now.

I suspect that the only reason we don’t see more restrained color themes is that people never really thought about it. Well, this is your wake-up call. I hope this will inspire people to use color more deliberately and to change the default way we build and use color themes.

Permalink

10 Developer Habits That Separate Good Programmers From Great Ones

10 Developer Habits That Separate Good Programmers From Great Ones

There's a moment in every developer's career when they realize that writing code that works isn't enough. It happens differently for everyone. Maybe you're staring at a pull request you submitted six months ago, horrified by the decisions your past self made. Maybe you're debugging a production issue at 2 AM, surrounded by energy drink cans, wondering how something so simple could have gone so catastrophically wrong. Or maybe you're pair programming with someone who makes everything look effortless—solving in minutes what would have taken you hours—and you're left wondering what separates you from them.

I've been writing code professionally for over a decade, and I can tell you with certainty: the difference between good programmers and great ones has very little to do with knowing more algorithms or memorizing syntax. It's not about graduating from a prestigious university or working at a FAANG company. The real separation happens in the invisible places—in the daily habits, the tiny decisions made a thousand times over, the discipline to do the unglamorous work that nobody sees.

This isn't about natural talent. I've watched brilliant developers flame out because they relied solely on raw intelligence. I've also watched average programmers transform into exceptional ones through deliberate practice and habit formation. The great ones aren't born—they're built, one habit at a time.

What follows isn't a collection of productivity hacks or keyboard shortcuts. These are the deep, fundamental habits that compound over time, the practices that will still matter whether you're writing Python microservices today or quantum computing algorithms fifteen years from now. Some of these habits will challenge you. Some will feel counterintuitive. All of them will require effort to develop.

But if you commit to them, you won't just become a better programmer. You'll become the kind of developer others want on their team, the one who gets pulled into the hardest problems, the one who shapes how engineering gets done.

Let's begin.

Habit 1: They Read Code Far More Than They Write It

When I mentor junior developers, I often ask them: "How much time do you spend reading other people's code compared to writing your own?" The answer is almost always the same: not much. Maybe they glance at documentation or skim through a library's source when something breaks, but intentional, deep code reading? Rarely.

This is the first habit that separates the good from the great: great developers are voracious code readers.

Think about it this way. If you wanted to become a great novelist, you wouldn't just write all day. You'd read—extensively, critically, analytically. You'd study how Hemingway constructs sentences, how Ursula K. Le Guin builds worlds, how Toni Morrison uses language to evoke emotion. Programming is no different. The craft of software engineering is learned as much through observation as through practice.

But here's what makes this habit so powerful: reading code teaches you things that writing code alone never will. When you write, you're trapped in your own mental models, your own patterns, your own biases. You'll naturally reach for the solutions you already know. Reading other people's code exposes you to different ways of thinking, different approaches to problems, different levels of abstraction.

I remember the first time I read through the source code of Redux, the popular state management library. I was intermediate-level at the time, comfortable with JavaScript but not what I'd call advanced. What struck me wasn't just how the code worked—it was how simple it was. The core Redux implementation is just a few hundred lines. The creators had taken a complex problem (managing application state) and distilled it down to its essence. Reading that code changed how I thought about software design. I realized that complexity isn't a badge of honor; simplicity is.

Great developers make code reading a regular practice. They don't wait for a reason to dive into a codebase. They do it because they're curious, because they want to learn, because they know that buried in those files are lessons that took someone years to learn.

Here's how to develop this habit practically:

Set aside dedicated reading time. Just like you might schedule time for coding side projects, schedule time for reading code. Start with 30 minutes twice a week. Pick a library or framework you use regularly and read through its source. Don't skim—actually read, line by line. When you encounter something you don't understand, resist the urge to skip over it. Pause. Research. Figure it out.

Read with purpose. Don't just read passively. Ask questions as you go: Why did they structure it this way? What problem were they solving with this abstraction? What would I have done differently? Are there patterns I can adopt? What makes this code easy or hard to understand?

Read code from different domains and languages. If you're a web developer, read embedded systems code. If you work in Python, read Rust. The patterns and principles often transcend the specific technology. I've applied lessons from reading Erlang's OTP framework to architecting Node.js microservices, even though the languages are wildly different. The underlying principles of fault tolerance and supervision trees were universally applicable.

Join the reading club movement. Some development teams have started "code reading clubs" where developers meet regularly to read through and discuss interesting codebases together. If your team doesn't have one, start it. Pick a well-regarded open-source project and work through it together. The discussions that emerge from these sessions are gold—you'll hear how different people interpret the same code, what they notice, what they value.

Study the masters. There are certain programmers whose code is worth studying specifically. John Carmack's game engine code. Rich Hickey's Clojure. Linus Torvalds' Git. DHH's Rails. These aren't perfect (nothing is), but they represent thousands of hours of refinement and deep thinking. Reading their work is like studying under a master craftsperson.

The transformation this habit creates is subtle but profound. You'll start to develop intuition about code quality. You'll recognize patterns more quickly. You'll build a mental library of solutions that you can draw from. When you encounter a new problem, instead of Googling immediately, you'll remember: "Oh, this is similar to how React handles reconciliation" or "This is the strategy pattern I saw in that Python library."

I've interviewed hundreds of developers, and I can usually tell within the first few technical questions whether someone is a serious code reader. They reference implementations they've studied. They compare approaches across different libraries. They have opinions informed by actual examination of alternatives, not just Stack Overflow answers.

Reading code won't make you a great developer by itself. But it's the foundation. Everything else builds on this. Because you can't write great code if you haven't seen what great code looks like.

Habit 2: They Invest Deeply in Understanding the 'Why' Behind Every Decision

Good programmers implement features. Great programmers understand the business context, user needs, and systemic implications of what they're building.

This might sound obvious, but it's one of the most commonly neglected habits, especially among developers who pride themselves on their technical skills. I've worked with brilliant engineers who could implement any algorithm, optimize any query, architect any system—but who treated requirements like gospel, never questioning whether what they were asked to build was actually the right solution.

Here's a story that illustrates this perfectly. A few years ago, I was working on a fintech platform, and we received a feature request to add "pending transaction" functionality. The product manager wanted users to see transactions that were authorized but not yet settled. Straightforward enough.

A good developer would have taken that requirement and implemented it. Created a new status field in the database, added some UI components, written the business logic. Done. Ship it.

But one of our senior engineers did something different. She scheduled a meeting with the PM and asked: "Why do users need to see pending transactions? What problem are they trying to solve?"

It turned out users were complaining that their account balances seemed wrong—they'd make a purchase, but their balance wouldn't reflect it immediately. They weren't actually asking to see pending transactions; they were confused about their available balance. The real solution wasn't to show pending transactions at all—it was to display two balances: current balance and available balance, accounting for pending authorizations.

This might seem like a small distinction, but it completely changed the implementation. Instead of building a whole new UI section for pending transactions (which would have added cognitive load), we refined the existing balance display. The solution was simpler, better aligned with user needs, and took half the time to implement.

This is what investing in the "why" looks like in practice.

Great developers treat every feature request, every bug report, every technical decision as an opportunity to understand the deeper context. They don't just ask "What needs to be built?" They ask:

  • What problem is this solving? Not the technical problem—the human problem. Who is affected? What pain are they experiencing?
  • What are the constraints? Is this urgent because of a regulatory deadline? Because of competitive pressure? Because a major client threatened to leave? Understanding urgency helps you make better tradeoff decisions.
  • What are the second-order effects? How will this change user behavior? How will it affect the system's complexity? What maintenance burden are we taking on?
  • Is this the right solution? Sometimes the best code is no code. Could we solve this problem through better UX? Through configuration instead of programming? Through fixing the root cause instead of treating symptoms?

I once spent three hours in a technical design review for a caching layer that would have solved our performance problems. The engineer who proposed it had done excellent work—detailed benchmarks, solid architecture, clear migration plan. But then someone asked: "Why are we having these performance problems in the first place?"

We dug deeper. Turned out a poorly optimized query was the root cause, making millions of unnecessary database calls. We'd been about to build a caching system to work around a problem that could be fixed with a two-line SQL optimization. Understanding the "why" saved us from weeks of unnecessary work.

This habit requires courage, especially when you're early in your career. It feels risky to question requirements, to push back on product managers or senior engineers, to suggest that maybe the planned approach isn't optimal. But here's what I've learned: people respect developers who think critically about what they're building. They want collaborators who catch problems early, who contribute to product thinking, who treat software development as problem-solving rather than ticket-closing.

How to develop this habit:

Make "Why?" your default question. Before starting any significant piece of work, ensure you can articulate why it matters. If you can't, you don't understand the problem well enough yet. Schedule time with whoever requested the work—product managers, other engineers, customer support—and ask questions until the context is clear.

Study the domain you're working in. If you're building healthcare software, learn about healthcare. Read about HIPAA. Understand how hospitals operate. Talk to doctors if you can. The more you understand the domain, the better you'll be at evaluating whether technical solutions actually solve real problems. I've seen developers who treated the domain as background noise, and their code showed it—technically proficient but misaligned with how the business actually worked.

Participate in user research. Watch user testing sessions. Read support tickets. Join customer calls. There's no substitute for seeing real people struggle with your software. It fundamentally changes how you think about what you're building. After watching just one user testing session, you'll never write a cryptic error message again.

Practice systems thinking. Every change you make ripples through the system. That innocent feature addition might increase database load, complicate the deployment process, or create a new edge case that breaks existing functionality. Great developers mentally model these ripples before writing code. They think in systems, not in isolated features.

Document the why, not just the what. When you write code comments, don't explain what the code does (that should be obvious from reading it). Explain why it exists. Why this approach instead of alternatives? What constraint or requirement drove this decision? Future you—and future maintainers—will be grateful.

I'll be honest: this habit can be exhausting. It's mentally easier to just implement what you're told. But here's the thing—great developers aren't great because they chose the easy path. They're great because they took responsibility for outcomes, not just outputs. They understood that their job wasn't to write code; it was to solve problems. And you can't solve problems you don't understand.

The developers who cultivate this habit become trusted advisors. They get invited to planning meetings. They influence product direction. They become force multipliers for their teams because they catch misalignments early, before they turn into wasted sprints and disappointed users.

Understanding the "why" transforms you from a code writer into an engineer. And that transformation is everything.

Habit 3: They Treat Debugging as a Science, Not a Guessing Game

It's 11 PM. Your production system is down. Customers are angry. Your manager is asking for updates every ten minutes. The pressure is overwhelming, and your first instinct is to start changing things—restart the server, roll back the last deploy, tweak some configuration values—anything to make the problem go away.

This is where good developers and great developers diverge most dramatically.

Good developers guess. They rely on intuition, past experience, and hope. They make changes without fully understanding the problem, treating debugging like a game of whack-a-mole. Sometimes they get lucky and stumble on a solution. Often they don't, and hours vanish into frustration.

Great developers treat debugging as a rigorous scientific process. They form hypotheses, gather data, run experiments, and systematically eliminate possibilities until they isolate the root cause. They're patient when patience feels impossible. They're methodical when chaos reigns.

Let me tell you about the worst production bug I ever encountered. Our e-commerce platform started randomly dropping orders—not all orders, just some of them. Maybe 2-3% of transactions would complete on the payment side but never create an order record in our database. Revenue was bleeding. Every hour the bug remained unfixed cost the company thousands of dollars.

The pressure to "just fix it" was immense. The easy move would have been to start deploying patches based on gut feelings. Instead, our lead engineer did something counterintuitive: she made everyone step back and follow a structured debugging process.

First, reproduce the problem. Seems obvious, but many developers skip this step, especially under pressure. She set up a staging environment and hammered it with test transactions until we could reliably reproduce the order drops. This single step was crucial—it meant we could test theories without experimenting on production.

Second, gather data. What do these dropped orders have in common? We pulled logs, traced requests through every system component, analyzed timing, examined user agents, scrutinized payment gateway responses. We weren't looking for the answer yet—we were building a complete picture of the problem.

Third, form hypotheses. Based on the data, we generated a list of possible causes, ranked by likelihood: database connection timeout, race condition in order creation logic, payment gateway webhook failure, API rate limiting, network partition, corrupted state in Redis cache.

Fourth, test systematically. We tested each hypothesis one at a time, starting with the most likely. For each test, we clearly defined what result would prove or disprove the theory. No guessing. No "let's try this and see what happens." Every experiment was deliberate.

It took four hours of methodical investigation, but we found it: a race condition where concurrent payment webhooks could create a state where the payment was marked successful, but the order creation transaction was rolled back. The bug only manifested under high load with specific timing conditions—hence the intermittent nature.

Here's the key insight: we could have easily spent twenty hours flailing around, making random changes, creating new bugs while trying to fix old ones. Instead, systematic debugging found the root cause in a quarter of the time. More importantly, we fixed it correctly, with confidence that it was actually resolved.

This habit—treating debugging as a disciplined practice rather than chaotic troubleshooting—is perhaps the most underestimated skill in software engineering.

How great developers debug:

They resist the urge to jump to solutions. When you see an error, your brain immediately wants to fix it. Fight this instinct. Spend time understanding the problem first. I have a personal rule: spend at least twice as much time understanding a bug as you expect to spend fixing it. This ratio has saved me countless hours of chasing symptoms instead of causes.

They use the scientific method explicitly. Write down your hypothesis. Write down what evidence would confirm or refute it. Run the experiment. Document the results. Move to the next hypothesis if needed. I literally keep a debugging journal where I log this process for complex bugs. It keeps me honest and prevents me from testing the same theory multiple times because I forgot I already tried it.

They make problems smaller. Great debuggers are masters of binary search in debugging. If a bug exists somewhere in 1,000 lines of code, they'll comment out 500 lines and see if the bug persists. Then 250 lines. Then 125. They systematically isolate the problem space until the bug has nowhere to hide.

They understand their tools deeply. Debuggers, profilers, log analyzers, network inspectors, database query analyzers—great developers invest time in mastering these tools. They can set conditional breakpoints, analyze memory dumps, trace system calls, interpret flame graphs. These tools multiply their effectiveness exponentially. I've seen senior developers debug issues in minutes that stumped others for days, simply because they knew how to use a profiler effectively.

They build debugging into their code. Great developers write code that's easy to debug. They add meaningful log statements at key decision points. They build observability into their systems from the start—metrics, traces, structured logs. They know that 80% of a bug's lifetime is spent trying to understand what's happening; making that easier is time well invested.

They reproduce, then fix, then verify. Never fix a bug you can't reproduce—you're just guessing. Once you can reproduce it, fix it. Then verify the fix actually works under the conditions where the bug originally occurred. Too many developers skip this verification step and end up shipping fixes that don't actually fix anything.

They dig for root causes. When you find a bug, ask "Why did this happen?" five times. Each answer leads you deeper. "The server crashed." Why? "Out of memory." Why? "Memory leak." Why? "Objects not being garbage collected." Why? "Event listeners not removed." Why? "No cleanup in component unmount." Now you've found the root cause, not just the symptom.

I've worked with developers who seemed to have an almost supernatural ability to find bugs. Early in my career, I thought they were just smarter or more experienced. Now I know the truth: they had simply internalized a systematic approach. They trusted the process, not their intuition.

This habit has a profound psychological benefit too. Debugging stops being stressful and starts being intellectually engaging. Instead of feeling helpless when bugs occur, you feel confident—you have a process, a methodology, a way forward. The bug might be complex, but you know how to approach complexity.

There's a reason the best developers don't panic during incidents. They've trained themselves to treat every bug as a puzzle with a solution, not a crisis. They know that systematic investigation always wins in the end. That confidence is built through this habit.

And here's something beautiful: when you approach debugging scientifically, you don't just fix bugs faster—you learn more from each one. Every bug becomes a lesson about the system, about edge cases, about your own mental models. Debuggers who just guess and check learn nothing. Scientific debuggers accumulate deep system knowledge with every issue they resolve.

The next time you encounter a bug, resist the temptation to immediately start changing code. Take a breath. Open a notebook. Write down what you know. Form a hypothesis. Test it. Let the scientific method be your guide.

You'll be amazed how much more effective you become.

Habit 4: They Write for Humans First, Machines Second

Here's an uncomfortable truth: most of your career as a developer won't be spent writing new code. It'll be spent reading, understanding, and modifying existing code—code written by other people, or by past versions of yourself who might as well be other people.

Yet when I review code from good developers, I consistently see the same mistake: they optimize for cleverness or brevity instead of clarity. They write code that impresses other developers with its sophistication, but which requires intense concentration to understand. They treat the compiler or interpreter as their primary audience.

Great developers flip this priority. They write code for humans first, machines second.

This might sound like a platitude, but it represents a fundamental shift in mindset that affects every line of code you write. Let me show you what I mean.

Here's a code snippet I found in a production codebase:

def p(x): return sum(1 for i in range(2, int(x**0.5)+1) if x%i==0)==0 and x>1

Can you tell what this function does? If you're experienced with algorithms, you might recognize it as a prime number checker. It works perfectly. The machine executes it just fine. But for a human reading this code? It's a puzzle that needs solving.

Now here's how a great developer would write the same function:

def is_prime(number):
    """
    Returns True if the number is prime, False otherwise.

    A prime number is only divisible by 1 and itself.
    We only need to check divisibility up to the square root of the number
    because if n = a*b, one of those factors must be <= sqrt(n).
    """
    if number <= 1:
        return False

    if number == 2:
        return True

    # Check if number is divisible by any integer from 2 to sqrt(number)
    for potential_divisor in range(2, int(number ** 0.5) + 1):
        if number % potential_divisor == 0:
            return False

    return True

The second version is longer. It's more verbose. The machine doesn't care—both run in O(√n) time. But the human difference is night and day. The second version is self-documenting. A junior developer can understand it. You can understand it six months from now when you've forgotten you wrote it. The intent is crystal clear.

This habit—writing for human comprehension—manifests in many ways:

Naming that reveals intent. Variable names like temp, data, obj, result tell you nothing. Great developers choose names that encode meaning: unprocessed_orders, customer_email_address, successfully_authenticated_user. Yes, these names are longer. That's fine. The extra few characters are worth it. You type code once but read it dozens of times.

I remember reviewing code where someone had named a variable x2. I had to trace through 50 lines of logic to figure out it represented "XML to JSON converter". They'd saved themselves typing 18 characters and cost every future reader minutes of cognitive load. That's a terrible trade.

Functions and methods that do one thing. When a function is trying to do multiple things, it becomes hard to name, hard to test, and hard to understand. Great developers extract functionality into well-named functions even when it feels like "overkill." They understand that a sequence of well-named function calls often communicates intent better than the raw implementation.

Strategic comments. Here's a nuance many developers miss: great developers don't comment what the code does—they comment why it does it. If your code needs comments to explain what it does, the code itself isn't clear enough. But comments explaining why certain decisions were made? Those are gold.

"Why" comments might explain:

  • "We're using algorithm X instead of the obvious approach Y because Y has O(n²) complexity with our data patterns"
  • "This weird timeout value came from extensive testing with the external API—smaller values cause intermittent failures"
  • "We're intentionally not handling edge case X because it's impossible given the database constraints enforced by migration Y"

These comments preserve context that would otherwise be lost. They prevent future developers from "optimizing" your carefully chosen approach or removing code they think is unnecessary.

Code structure that mirrors mental models. Great developers organize code the way humans naturally think about the domain. If you're building an e-commerce system, your code structure should reflect concepts like orders, customers, payments, and inventory—not generic abstractions like managers, handlers, and processors.

I once worked on a codebase that had a DataManager, DataHandler, DataProcessor, and DataController. None of these names conveyed what they actually did. When we refactored to OrderValidator, PaymentProcessor, and InventoryTracker, suddenly the codebase became navigable. New team members could find things. The code structure matched their mental model of the business.

Consistent patterns. Humans are pattern-matching machines. When your codebase follows consistent patterns, developers can transfer knowledge from one part to another. When every module does things differently, every context switch requires re-learning. Great developers value consistency even when they might personally prefer a different approach.

Appropriate abstraction levels. This is subtle but crucial. Great developers are careful about mixing abstraction levels in the same function. If you're writing high-level business logic, you shouldn't suddenly drop down to low-level string manipulation details. Extract that into a well-named helper function. Keep each layer of code at a consistent conceptual level.

Here's an example of mixed abstraction levels:

function processOrder(order) {
  // High-level business logic
  validateOrder(order);

  // Suddenly low-level string manipulation
  const cleanEmail = order.email.trim().toLowerCase().replace(/\s+/g, '');

  // Back to high-level
  chargeCustomer(order);
  sendConfirmation(order);
}

Better:

function processOrder(order) {
  validateOrder(order);
  const normalizedOrder = normalizeOrderData(order);
  chargeCustomer(normalizedOrder);
  sendConfirmation(normalizedOrder);
}

Now the function reads like a sequence of business steps, not a mix of business logic and implementation details.

This habit requires discipline because writing for machines is often easier than writing for humans. The machine is forgiving—it doesn't care if your variable name is x or customer_lifetime_value_in_cents. But humans care deeply.

I've seen talented developers handicap themselves with this habit. They write impressively compact code, demonstrating their mastery of language features. But then they spend hours in code reviews explaining what their code does because nobody else can figure it out. They've optimized for the wrong thing.

There's a famous quote often attributed to various programming luminaries: "Any fool can write code that a computer can understand. Good programmers write code that humans can understand." The wisdom in this statement becomes more apparent with every year of experience.

When you cultivate the habit of writing for humans first, something remarkable happens: your code becomes maintainable. Teams move faster because understanding is easy. Onboarding new developers takes days instead of weeks. Bugs decrease because the code's intent is clear. Technical debt accumulates more slowly because future modifications don't require archaeological expeditions through cryptic logic.

I can always identify great developers in code reviews by one characteristic: I rarely have to ask "What does this code do?" The code itself tells me. I might ask about trade-offs, about performance implications, about alternative approaches—but I never struggle with basic comprehension.

Write code as if the person maintaining it is a violence-prone psychopath who knows where you live. The person maintaining your code will be you in six months, and you'll thank yourself for the clarity.

Habit 5: They Embrace Constraints as Creative Catalysts

When I was a junior developer, I viewed constraints as problems to be overcome or worked around. Limited time? Frustrating. Legacy system compatibility? Annoying. Memory restrictions? Limiting. I saw my job as defeating these constraints to implement the "proper" solution.

Great developers think about constraints completely differently. They embrace them. They lean into them. They recognize that constraints don't limit creativity—they focus it, channel it, and often produce better solutions than unlimited resources would allow.

This is one of the most counterintuitive habits that separates good from great, and it takes years to internalize.

Let me share a story that crystallized this for me. I was working at a startup building a mobile app for emerging markets. Our target users were on low-end Android devices with spotty 2G connections and limited data plans. Our initial instinct was to treat these constraints as handicaps—we'd build a "lite" version of our real product, stripped down and compromised.

Then our tech lead said something that changed my perspective: "These aren't limitations. These are our design parameters. They're telling us what excellence looks like in this context."

We completely shifted our approach. Instead of asking "How do we cram our features into this constrained environment?", we asked "What's the best possible experience we can create given these parameters?"

We designed offline-first from the ground up. We compressed images aggressively and used SVGs where possible. We implemented delta updates so the app could update itself over flaky connections. We cached intelligently and prefetched predictively. We made every byte count.

The result? An app that felt snappy and responsive even on terrible connections. An experience that was actually better than many apps designed for high-end markets, because we'd been forced to think deeply about performance and efficiency. Our Western competitors who designed for high-bandwidth, powerful devices couldn't compete in that market. Their apps were bloated, slow, and data-hungry.

The constraints didn't handicap us. They made us better.

This principle extends far beyond technical constraints. Consider time constraints. Good developers see tight deadlines as stress. Great developers see them as clarity. When you have unlimited time, you can explore every possible solution, refactor endlessly, polish indefinitely. Sounds great, right? But unlimited time often produces worse results because nothing forces you to prioritize, to identify what really matters, to make hard trade-off decisions.

I've watched projects with loose deadlines drift aimlessly for months, adding feature after feature, refactoring the refactorings, never quite shipping. Then I've seen teams given two weeks to ship an MVP who produced focused, well-scoped products that actually solved user problems. The time constraint forced clarity about what was essential.

Or consider team constraints. Maybe you're the only backend developer on a small team. Good developers see this as overwhelming—too much responsibility, too much to maintain. Great developers see it as an opportunity to shape the entire backend architecture, to make consistent decisions, to build deep expertise. The constraint of being alone forces you to write extremely maintainable code because you'll be the one maintaining it.

Or legacy system constraints. You're integrating with a 15-year-old SOAP API with terrible documentation. Good developers complain about it. Great developers recognize it as an opportunity to build a clean abstraction layer that isolates the rest of the codebase from that complexity. The constraint of the legacy system forces you to think carefully about boundaries and interfaces.

Here's how to cultivate this habit:

Reframe the language. Stop saying "We can't do X because of constraint Y." Start saying "Given constraint Y, what's the best solution we can design?" The linguistic shift creates a mental shift. You move from problem-focused to solution-focused thinking.

Study historical examples. Twitter's original 140-character limit wasn't a bug—it was a constraint that defined the platform's character. Game developers creating for the Super Nintendo worked with 32 kilobytes of RAM and produced masterpieces. They didn't have unlimited resources, but the constraints forced incredible creativity and efficiency. The Apollo Guidance Computer had less computing power than a modern calculator, but it got humans to the moon. Study how constraints drove innovation in these cases.

Impose artificial constraints. This sounds crazy, but it works. If you're building a web app, challenge yourself: what if it had to work without JavaScript? What if the bundle size had to be under 50KB? What if it had to run on a $30 Android phone? These artificial constraints force you to question assumptions and explore different approaches. You might not ship with these constraints, but the exercise makes you a better developer.

Embrace the "worse is better" philosophy. Sometimes a simpler solution that doesn't handle every edge case is better than a complex solution that handles everything. Constraints force you to make this trade-off explicitly. The UNIX philosophy—small programs that do one thing well—emerged from extreme memory and storage constraints. Those constraints produced better design principles than unlimited resources would have.

Look for the constraint's gift. Every constraint is trying to tell you something. Memory constraints tell you to think about efficiency. Time constraints tell you to focus on impact. Legacy constraints tell you to design clean interfaces. Budget constraints tell you to use proven technologies instead of chasing novelty. What is the constraint teaching you?

I've seen developers waste enormous energy fighting constraints instead of working with them. They'll spend weeks architecting a way to bypass a database query limitation instead of restructuring their data model to work within it. They'll add layers of complexity to work around a framework's design instead of embracing the framework's philosophy.

Great developers pick their battles. Sometimes constraints truly are wrong and should be challenged. But more often, constraints represent real trade-offs in a complex system, and working within them produces better results than fighting them.

This habit also builds character. Embracing constraints requires humility—accepting that you can't have everything, that trade-offs are real, that perfection isn't achievable. It requires creativity—finding elegant solutions within boundaries. It requires focus—distinguishing between what's essential and what's merely nice to have.

The modern development world often feels like it's about having more: more tools, more frameworks, more libraries, more features, more scalability. But some of the most impactful software ever created was built with severe constraints. Redis started as a solution to a specific problem with strict performance requirements. Unix was designed for machines with tiny memory footprints. The web itself was designed to work over unreliable networks with minimal assumptions about client capabilities.

When you embrace constraints, you stop fighting reality and start working with it. You become a pragmatic problem-solver instead of an idealistic perfectionist. You ship solutions instead of endlessly pursuing optimal ones.

And here's the beautiful paradox: by accepting limitations, you often transcend them. The discipline and creativity that constraints force upon you produce solutions that work better, not worse. The app optimized for 2G connections also screams on 5G. The code designed for maintainability by a solo developer remains maintainable as the team grows. The feature set focused by time constraints turns out to be exactly what users needed.

Constraints aren't your enemy. They're your teacher, your focus, your catalyst for creative solutions. Learn to love them.

Habit 6: They Cultivate Deep Focus in an Age of Distraction

The modern developer's environment is a carefully engineered distraction machine. Slack pings, email notifications, endless meetings, "quick questions," and the siren song of social media and news feeds—all conspiring to fragment your attention into a thousand tiny pieces.

Good developers work in these conditions. They context-switch constantly, juggling multiple threads, believing that responsiveness is a virtue. They wear their busyness as a badge of honor.

Great developers build fortresses of focus. They understand that their most valuable asset isn't their knowledge of frameworks or algorithms—it's their ability to concentrate deeply on complex problems for extended periods. They treat uninterrupted time as a non-negotiable resource, more precious than any cloud computing credit.

This isn't just a preference; it's a necessity grounded in the nature of our work. Programming isn't a mechanical task of typing lines of code. It's an act of construction and problem-solving that happens largely in your mind. You build intricate mental models of systems, data flows, and logic. These models are fragile. A single interruption can shatter hours of mental assembly, forcing you to rebuild from scratch.

I learned this the hard way early in my career. I prided myself on being "always on." I had eight different communication apps open, responded to messages within seconds, and hopped between coding, code reviews, and support tickets all day. I was exhausted by 3 PM, yet my output was mediocre. I was putting in the time but not producing my best work.

Everything changed when I paired with a senior engineer named David for a week. David worked in mysterious two-hour blocks. During these blocks, he'd turn off all notifications, close every application except his IDE and terminal, and put on headphones. At first, I thought he was being antisocial. But then I saw his output. In one two-hour focus block, he'd often complete what would take me an entire distracted day. The quality was superior—fewer bugs, cleaner designs, more thoughtful edge-case handling. He wasn't just faster; he was operating at a different level of quality.

David taught me that focus is a skill to be developed, not a trait you're born with. And it's perhaps the highest-leverage skill you can cultivate.

Here's how great developers protect and cultivate deep focus:

They schedule focus time religiously. They don't leave it to chance. They block out multi-hour chunks in their calendar and treat these appointments with themselves as seriously as meetings with the CEO. During this time, they are effectively offline. Some companies even formalize this with policies like "no-meeting Wednesdays" or "focus mornings," but great developers implement these guardrails for themselves regardless of company policy.

They master their tools, but don't fetishize them. Great developers use tools like "Do Not Disturb" modes, website blockers, and full-screen IDEs not as productivity hacks, but as deliberate barriers against interruption. The goal isn't to find the perfect app; it's to create an environment where deep work can occur. They understand that the tool is secondary to the intent.

They practice single-tasking. Multitasking is a myth, especially in programming. What we call multitasking is actually rapid context-switching, and each switch carries a cognitive cost. Great developers train themselves to work on one thing until it reaches a natural stopping point. They might keep a "distraction list" nearby—a notepad to jot down random thoughts or to-dos that pop up—so they can acknowledge the thought without derailing their current task.

They defend their focus courageously. This is the hardest part. It requires saying "no" to well-meaning colleagues, setting boundaries with managers, and resisting the cultural pressure to be constantly available. Great developers learn to communicate these boundaries clearly and politely: "I'm in the middle of a deep work session right now, but I can help you at 3 PM." Most reasonable people will respect this if it's communicated consistently.

They recognize the cost of context switching. Every interruption doesn't just cost the time of the interruption itself; it costs the time to re-immerse yourself in the original problem. A 30-second Slack question can easily derail 15 minutes of productive flow. Great developers make this cost visible to their teams, helping create a culture that respects deep work.

They structure their day around energy levels. Focus is a finite resource. Great developers know when they are at their cognitive best—for many, it's the morning—and guard that time fiercely for their most demanding work. Meetings, administrative tasks, and code reviews are relegated to lower-energy periods. They don't squander their peak mental hours on low-value, shallow work.

They embrace boredom. This sounds strange, but it's critical. In moments of frustration or mental block, the immediate impulse is to reach for your phone—to seek a dopamine hit from Twitter or email. Great developers resist this. They stay with the problem, staring out the window if necessary, allowing their subconscious to work on the problem. Some of the most elegant solutions emerge not in frantic typing, but in quiet contemplation.

The benefits of this habit extend far beyond increased productivity. Deep focus is where mastery lives. It's in these uninterrupted stretches that you encounter the truly hard problems, the ones that force you to grow. You develop the patience to debug systematically, the clarity to see elegant architectures, and the persistence to push through complexity that would overwhelm a distracted mind.

Furthermore, focus begets more focus. Like a muscle, your ability to concentrate strengthens with practice. What starts as a struggle to focus for 30 minutes can, over time, become a reliable two-hour deep work session.

In a world that values shallow responsiveness, choosing deep focus feels countercultural. But it's precisely this choice that separates competent developers from exceptional ones. The developers who can enter a state of flow regularly are the ones who ship complex features, solve the hardest bugs, and produce work that feels almost magical in its quality.

Your most valuable code will be written in focus. Protect that state with your life.

Habit 7: They Practice Strategic Laziness

If "laziness" sounds like a vice rather than a virtue, you're thinking about it wrong. Good developers are often hardworking—they'll pour hours into manual testing, repetitive configuration, and brute-force solutions. They equate effort with value.

Great developers practice strategic laziness. They will happily spend an hour automating a task that takes five minutes to do manually, not because it saves time immediately, but because they hate repetition so much they're willing to invest upfront to eliminate it forever. They are constantly looking for the lever, the shortcut, the abstraction that maximizes output for minimum ongoing effort.

This principle, often attributed to Larry Wall, the creator of Perl, is one of the three great virtues of a programmer (the others being impatience and hubris). Strategic laziness isn't about avoiding work; it's about being profoundly efficient by automating the boring stuff so you can focus your energy on the hard, interesting problems.

I saw a perfect example of this with a DevOps engineer I worked with. Our deployment process involved a 15-step checklist that took about 30 minutes and required intense concentration. A mistake at any step could take down production. Most of us treated it as a necessary, if tedious, part of the job.

She, however, found it intolerable. Over two days, she built a set of scripts that automated the entire process. The initial investment was significant—probably 16 hours of work. But after that, deployments took 2 minutes and were error-free. In a month, she had recouped the time investment for the entire team. In a year, she had saved hundreds of hours and eliminated countless potential outages. That's strategic laziness.

This habit manifests in several key ways:

They automate relentlessly. If they have to do something more than twice, they write a script. Environment setup, database migrations, build processes, testing routines—all are prime candidates for automation. They don't just think about the time saved; they think about the cognitive load eliminated and the errors prevented.

They build tools and abstractions. Great developers don't just solve the immediate problem; they solve the class of problems. When they notice a repetitive pattern in their code, they don't copy-paste with minor modifications—they extract a function, create a library, or build a framework. They'd rather spend time designing a clean API than writing the same boilerplate for the tenth time.

They are masters of delegation—to the computer. They constantly ask: "What part of this can the computer handle?" Linting, formatting, dependency updates, performance monitoring—tasks that good developers do manually are delegated to automated systems by great developers. This frees their mental RAM for tasks that genuinely require human intelligence.

They optimize for long-term simplicity, not short-term speed. The strategically lazy developer knows that the easiest code to write is often the hardest to maintain. So they invest a little more time upfront to create a simple, clear design that will be easy to modify later. They're lazy about future work, so they do the hard thinking now.

They leverage existing solutions. The strategically lazy developer doesn't build a custom authentication system when Auth0 exists. They don't write a custom logging framework when structured logging libraries are available. They have a healthy bias for using battle-tested solutions rather than reinventing the wheel. Their goal is to solve the business problem, not to write every line of code themselves.

How to cultivate strategic laziness:

Develop an allergy to repetition. Pay attention to tasks you find yourself doing repeatedly. Does it feel tedious? That's your signal to automate. Start small—a shell script to set up your project, a macro to generate boilerplate code. The satisfaction of eliminating a recurring annoyance is addictive and will fuel further automation.

Ask the lazy question. Before starting any task, ask: "Is there an easier way to do this?" "Will I have to do this again?" "Can I get the computer to do the boring parts?" This simple metacognition separates the habitually lazy from the strategically lazy.

Invest in your toolchain. Time spent learning your IDE's shortcuts, mastering your shell, or configuring your linters isn't wasted—it's compounded interest. A few hours learning Vim motions or VS Code multi-cursor editing can save you days of typing over a year.

Build, then leverage. When you build an automation or abstraction, think about how to make it reusable. A script that's only useful for one project is good; a tool that can be used across multiple projects is great. Write documentation for your tools—future you will thank you.

The beauty of strategic laziness is that it benefits everyone, not just you. The deployment script you write helps the whole team. The well-designed abstraction makes the codebase easier for everyone to work with. The automated test suite prevents bugs for all future developers.

This habit transforms you from a code monkey into a force multiplier. You stop being just a producer of code and become a builder of systems that produce value with less effort. You become the developer who, instead of just working hard, makes the entire team's work easier and more effective.

And in the end, that's the kind of laziness worth cultivating.

Habit 8: They Maintain a Feedback Loop with the Production Environment

Good developers write code, run tests, and push to production. They trust that if the tests pass and the build is green, their job is done. They view production as a distant, somewhat scary place that operations teams worry about.

Great developers have an intimate, ongoing relationship with production. They don't just ship code and forget it; they watch it walk out the door and follow it into the world. They treat the production environment not as a final destination, but as the ultimate source of truth about their code's behavior, performance, and value.

This habit is the difference between theoretical correctness and practical reality. Your code can pass every test, satisfy every requirement, and look beautiful in review, but none of that matters if it fails in production. Great developers understand that production is where their assumptions meet reality, and reality always wins.

I learned this lesson from a catastrophic performance regression early in my career. We had built a new feature with a complex database query. It was elegant, used all the right JOINs, and passed all our unit and integration tests. Our test database had a few hundred rows of synthetic data, and the query was instant.

We shipped it on a Friday afternoon. By Saturday morning, the database was on fire. In production, with millions of rows of real-world data, that "elegant" query was doing full table scans. It timed out, locked tables, and brought the entire application to its knees. We spent our weekend in panic mode, rolling back and writing a fix.

A great developer on our team, Maria, took this personally. Not because she wrote the bad query (she hadn't), but because she saw it as a systemic failure. "We can't just test if our code works," she said. "We have to test if it works under real conditions."

From that day on, she became the guardian of our production feedback loops.

Here's what maintaining a tight production feedback loop looks like in practice:

They instrument everything. Great developers don't just log errors; they measure everything that matters. Response times, throughput, error rates, business metrics, cache hit ratios, database query performance. They bake observability—metrics, logs, and traces—into their code from the very beginning. They know that you can't fix what you can't see.

They watch deployments like hawks. When their code ships, they don't just move on to the next ticket. They watch the deployment metrics. They monitor error rates. They check performance dashboards. They might even watch real-user sessions for a few minutes to see how the feature is actually being used. This immediate feedback allows them to catch regressions that slip past tests.

They practice "you build it, you run it." This Amazon-originated philosophy means developers are responsible for their code in production. They are on call for their features. They get paged when things break. This might sound punishing, but it's the most powerful feedback loop imaginable. Nothing motivates you to write robust, fault-tolerant code like knowing your phone will ring at 3 AM if you don't.

They use feature flags religiously. Great developers don't deploy big bang releases. They wrap new features in flags and roll them out gradually—to internal users first, then to 1% of customers, then to 10%, and so on. This allows them to get real-world feedback with minimal blast radius. If something goes wrong, they can turn the feature off with a single click instead of a full rollback.

They analyze production data to make decisions. Should we optimize this query? A good developer might guess. A great developer will look at production metrics to see how often it's called, what its average and p95 latencies are, and what impact it's having on user experience. They let data from production guide their prioritization.

They embrace and learn from incidents. When production breaks, great developers don't play the blame game. They lead and participate in blameless post-mortems. They dig deep to find the root cause, not just the symptom. More importantly, they focus on systemic fixes that prevent the entire class of problem from recurring, rather than just patching the immediate issue.

How to develop this habit:

Make your application observable from day one. Before you write business logic, set up structured logging, metrics collection, and distributed tracing. It's much harder to add this later. Start simple—even just logging key business events and performance boundaries is a huge step forward.

Create a personal dashboard. Build a dashboard that shows the health of the features you own. Make it the first thing you look at in the morning and the last thing you check before a deployment. This habit builds a sense of ownership and connection to your code's real-world behavior.

Volunteer for on-call rotation. If your team has one, join it. If it doesn't, propose it. The experience of being woken up by a pager for code you wrote is transformative. It will change how you think about error handling, logging, and system design forever.

Practice "production debugging." The next time there's a production issue, even if it's not in your code, ask if you can shadow the person debugging it. Watch how they use logs, metrics, and traces to pinpoint the problem. This is a skill that can only be learned by doing.

Ship small, ship often. The more frequently you deploy, the smaller each change is, and the easier it is to correlate changes in the system with changes in its behavior. Frequent deployments reduce the fear of production and turn it into a familiar, manageable place.

Maintaining this feedback loop does more than just prevent bugs—it closes the circle of learning. You write code based on assumptions, and production tells you which of those assumptions were wrong. Maybe users are using your feature in a way you never anticipated. Maybe that "edge case" is actually quite common. Maybe the performance characteristic you assumed is completely different under real load.

This continuous learning is what turns a good coder into a great engineer. You stop designing systems in a vacuum and start building them with a deep, intuitive understanding of how they will actually behave in the wild.

Production is the most demanding and honest code reviewer you will ever have. Listen to it.

Habit 9: They Prioritize Learning Deliberately, Not Accidentally

The technology landscape is a raging river of change. New frameworks, languages, tools, and paradigms emerge, gain fervent adoption, and often fade into obscurity, all within a few years. A good developer swims frantically in this river, trying to keep their head above water. They learn reactively—picking up a new JavaScript framework because their job requires it, skimming a blog post that pops up on Hacker News, watching a tutorial when they're stuck on a specific problem. Their learning is ad-hoc, driven by immediate necessity and the loudest voices in the ecosystem.

A great developer doesn't just swim in the river; they build a boat and chart a course. They understand that in a field where specific technologies have a half-life of mere years, the only sustainable advantage is the ability to learn deeply and efficiently. They don't learn reactively; they learn deliberately. Their learning is a systematic, ongoing investment, guided by a clear understanding of first principles and their long-term goals, not by the whims of tech trends.

This is arguably the most important habit of all, because it's the meta-habit that enables all the others. It's the engine of growth.

I witnessed the power of this habit in two developers who joined my team at the same time, both with similar backgrounds and talent. Let's call them Alex and Ben.

Alex was a classic reactive learner. He was bright and capable. When the team decided to adopt a new state management library, he dove in. He learned just enough to get his tasks done. He Googled specific error messages, copied patterns from existing code, and became functionally proficient. His knowledge was a mile wide and an inch deep—a collection of solutions to specific problems without a unifying mental model.

Ben took a different approach. When faced with the same new library, he didn't just read the "Getting Started" guide. He spent a weekend building a throwaway project with it. Then, he read the official documentation cover-to-cover. He watched a talk by the creator to understand the philosophy behind the library—what problems it was truly designed to solve, and what trade-offs it made. He didn't just learn how to use it; he learned why it was built that way, and when it was the right or wrong tool for the job.

Within six months, the difference was staggering. Alex could complete tasks using the library, but he often wrote code that fought against its core principles, leading to subtle bugs and performance issues. When he encountered a novel problem, he was often stuck.

Ben, on the other hand, had become the team's go-to expert. He could anticipate problems before they occurred. He designed elegant solutions that leveraged the library's strengths. He could explain its concepts to juniors in a way that made them stick. He wasn't just a user of the tool; he was a master of it.

Alex had learned accidentally. Ben had learned deliberately.

Here’s how great developers structure their deliberate learning:

They Learn Fundamentals, Not Just Flavors. The great developer knows that while JavaScript frameworks come and go (Remember jQuery? AngularJS? Backbone.js?), the underlying fundamentals of the web—the DOM, the event loop, HTTP, browser rendering—endure. They invest their time in understanding computer science fundamentals: data structures, algorithms, networking, operating systems, and design patterns. These are the timeless principles that allow them to evaluate and learn any new "flavor" of technology quickly and deeply. Learning React is easy when you already understand the principles of declarative UI, the virtual DOM concept, and one-way data flow. You're not memorizing an API; you're understanding a manifestation of deeper ideas.

They Maintain a "Learning Backlog." Just as they have a backlog of features to build, they maintain a personal backlog of concepts to learn, technologies to explore, and books to read. This isn't a vague "I should learn Go someday." It's a concrete list: "Read 'Designing Data-Intensive Applications,'" "Build a simple Rust CLI tool to understand memory safety," "Complete the 'Networking for Developers' course on Coursera." This transforms learning from an abstract intention into a manageable, actionable project.

They Allocate "Learning Time" and Protect It Ferociously. They don't leave learning to the scraps of time left over after a exhausting day of meetings and coding. They schedule it. Many great developers I know block out one afternoon per week, or a few hours every morning before work, for deliberate learning. This time is sacred. It's not for checking emails or putting out fires. It's for deep, uninterrupted study and practice.

They Learn by Doing, Not Just Consuming. Passive consumption—reading, watching videos—is only the first step. Great developers internalize knowledge by applying it. They don't just read about a new database; they install it, import a dataset, and run queries. They don't just watch a tutorial on a new architecture; they build a toy project that implements it. This practice builds strong, durable neural pathways that theory alone cannot. They understand that true mastery lives in the fingertips as much as in the brain.

They Go to the Source. When a new tool emerges, the reactive learner reads a "10-minute introduction" blog post. The deliberate learner goes straight to the primary source: the official documentation, the original research paper (if one exists), or a talk by the creator. They understand that secondary sources are often simplified, opinionated, or outdated. The truth, in all its nuanced complexity, is usually found at the source. Reading the React documentation or Dan Abramov's blog posts is a different league of learning than reading a list of "React tips and tricks" on a random blog.

They Teach What They Learn. The deliberate learner knows that the ultimate test of understanding is the ability to explain a concept to someone else. They write blog posts, give brown bag lunches to their team, contribute to documentation, or simply explain what they've learned to a colleague. The act of organizing their thoughts for teaching forces them to confront gaps in their own understanding and solidify the knowledge. It's the final step in the learning cycle.

They Curate Their Inputs Wisely. The digital world is a firehose of low-quality, repetitive, and often incorrect information. Great developers are ruthless curators of their information diet. They don't try to read everything. They identify a handful of trusted, high-signal sources—specific blogs, journals, podcasts, or people—and ignore the rest. They favor depth over breadth, quality over quantity.

How to cultivate this habit:

Conduct a quarterly "skills audit." Every three months, take an honest inventory of your skills. What's getting stronger? What's becoming obsolete? What emerging trend do you need to understand? Based on this audit, update your learning backlog. This transforms your career development from a passive process into an active one you control.

Follow the "20% rule." Dedicate a fixed percentage of your time—even if it's just 5% to start—to learning things that aren't immediately relevant to your current tasks. This is how you avoid technological obsolescence. It's how you serendipitously discover better ways of working and new opportunities.

Build a "personal syllabus." If you wanted to become an expert in distributed systems, what would you need to learn? In what order? A deliberate learner creates a syllabus for themselves, just like a university course. They might start with a textbook, then move to seminal papers, then build a project. This structured approach is infinitely more effective than random exploration.

Find a learning cohort. Learning alone is hard. Find one or two colleagues who share your growth mindset. Start a book club, a study group, or a "tech deep dive" session. The social commitment will keep you accountable, and the discussions will deepen your understanding.

The payoff for this habit is immeasurable. It's the difference between a developer whose value peaks five years into their career and one who becomes more valuable with each passing year. It's the difference between being at the mercy of the job market and being the one that companies fight over.

Deliberate learning is the ultimate career capital. In a world of constant change, the ability to learn how to learn, and to do it with purpose and strategy, isn't just a nice-to-have. It's the single greatest predictor of long-term success. It is the quiet, persistent engine that transforms a good programmer into a great one, and a great one into a true master of the craft.

Habit 10: They Build and Nurture Their Engineering Judgment

You can master every technical skill. You can write pristine code, debug with scientific precision, and architect systems of elegant simplicity. You can have an encyclopedic knowledge of algorithms and an intimate relationship with production. But without the final, most elusive habit, you will never cross the chasm from being a great technician to being a truly great engineer.

That final habit is the cultivation of engineering judgment.

Engineering judgment is the silent, invisible partner to every technical decision you make. It’s the internal compass that guides you when the map—the requirements, the documentation, the best practices—runs out. It’s the accumulated wisdom that tells you when to apply a rule, and, more importantly, when to break it. It’s what separates a technically correct solution from a genuinely wise one.

A good developer, when faced with a problem, asks: "What is the technically optimal solution?" They will find the most efficient algorithm, the most scalable architecture, the most pristine code structure. They are in pursuit of technical perfection.

A great developer asks a more complex set of questions: "What is the right solution for this team, for this business context, for this moment in time?" They weigh technical ideals against a messy reality of deadlines, team skills, business goals, and long-term maintenance. They understand that the best technical solution can be the worst engineering decision.

I learned this not from a success, but from a failure that still haunts me. Early in my career, I was tasked with building a new reporting feature. The existing system was a tangled mess of SQL queries embedded in PHP. It was slow, unmaintainable, and a nightmare to modify.

I saw my chance to shine. I designed a beautiful, event-sourced architecture with a CQRS pattern. It was technically brilliant. It would be infinitely scalable, provide perfect audit trails, and allow for complex historical queries. It was the kind of system you read about in software architecture books. I was immensely proud of it.

It was also a catastrophic failure.

The project took three times longer than estimated. The complexity was so high that only I could understand the codebase. When I eventually left the company, the team struggled for months to maintain it, eventually rewriting the entire feature in a much simpler, cruder way. My "technically optimal" solution was an engineering disaster. It was the wrong solution for the team's skill level, the wrong solution for the business's need for speed, and the wrong solution for the long-term health of the codebase.

I had technical skill, but I had failed the test of engineering judgment.

Engineering judgment is the synthesis of all the other habits into a form of professional wisdom. Here’s how it manifests:

They Understand the Spectrum of "Good Enough." Great developers know that not every piece of the system needs to be a masterpiece. The prototype for a one-off marketing campaign does not need the same level of robustness as the core authentication service. The internal admin tool can tolerate more technical debt than the customer-facing API. They make conscious, deliberate trade-offs. They ask: "What is the minimum level of quality required for this to successfully solve the problem without creating unacceptable future risk?" This isn't laziness; it's strategic allocation of effort.

They See Around Corners. A developer with strong judgment can anticipate the second- and third-order consequences of a decision. They don't just see the immediate feature implementation; they see how it will constrain future changes, what new categories of bugs it might introduce, and how it will affect the system's conceptual integrity. When they choose a library, they don't just evaluate its features; they evaluate its maintenance status, its upgrade path, its community health, and its architectural philosophy. They are playing a long game that others don't even see.

They Balance Idealism with Pragmatism. They hold strong opinions about code quality, but they hold them loosely. They can passionately argue for a clean architecture in a planning meeting, but if the business context demands a quicker, dirtier solution, they can pivot and implement the pragmatic choice without resentment. They document the trade-offs made and the technical debt incurred, creating a ticket to address it later, and then they move on. They understand that software exists to serve a business, not the other way around.

They Make Decisions Under Uncertainty. Requirements are ambiguous. Timelines are tight. Information is incomplete. This is the reality of software development. A good developer freezes, demanding more certainty, more specifications, more time. A great developer uses their judgment to make the best possible decision with the information available. They identify the core risks, make reasonable assumptions, and chart a course. They know that delaying a decision is often more costly than making a slightly wrong one.

They Distinguish Between Symptoms and Diseases. A junior developer treats the symptom: "The page is loading slowly, let's add a cache." A good developer finds the disease: "The page is loading slowly because of an N+1 query problem, let's fix the query." A great developer with sound judgment asks if the disease itself is a symptom: "Why are we making so many queries on this page? Is our data model wrong? Is this feature trying to do too much? Should we be pre-computing this data entirely?" They operate at a higher level of abstraction, solving classes of problems instead of individual instances.

How to Cultivate Engineering Judgment (Because It Can't Be Taught, Only Grown)

Judgment isn't a skill you can learn from a book. It's a form of tacit knowledge, built slowly through experience, reflection, and a specific kind of practice.

Seek Diverse Experiences. Judgment is pattern-matching on a grand scale. The more patterns you have seen, the better your judgment will be. Work at a startup where speed is everything. Work at an enterprise where stability is paramount. Work on front-end, back-end, and infrastructure. Each context teaches you a different set of values and trade-offs. The developer who has only ever worked in one environment has a dangerously narrow basis for judgment.

Conduct Retrospectives on Your Own Decisions. This is the single most powerful practice. Don't just move on after a project finishes or a decision is made. Schedule a solo retrospective. Take out a notebook and ask yourself:

· "What were the key technical decisions I made?"
· "What was my reasoning at the time?"
· "How did those decisions play out? Better or worse than expected?"
· "What did I miss? What would I do differently with the benefit of hindsight?"
This ritual of self-reflection is how you convert experience into wisdom.

Find a Yoda. Identify a senior engineer whose judgment you respect—someone who seems to have a preternatural ability to make the right call. Study them. When they make a decision that seems counterintuitive, ask them to explain their reasoning. Not just the technical reason, but the contextual, human, and business reasons. The nuances they share are the building blocks of judgment.

Practice Articulating the "Why." When you make a recommendation, force yourself to explain not just what you think should be done, but why. Lay out the trade-offs you considered. Explain the alternatives you rejected and why. The act of articulating your reasoning forces you to examine its validity and exposes flaws in your logic. It also invites others into your thought process, allowing them to challenge and refine your judgment.

Embrace the "Reversibility" Heuristic. When faced with a difficult decision, ask: "How reversible is this?" Adopting a new programming language is largely irreversible for a codebase. Adding a complex microservice architecture is hard to undo. Choosing a cloud provider creates lock-in. These are high-judgment decisions. On the other hand, refactoring a module, changing an API endpoint, or trying a new library are often easily reversible. Great developers apply more rigor and demand more certainty for irreversible decisions, and they move more quickly on reversible ones.

Develop a Sense of Proportion. This is perhaps the most subtle aspect of judgment. It’s knowing that spending two days optimizing a function that runs once a day is a waste, but spending two days optimizing a function called ten thousand times per second is critical. It’s knowing that a 10% performance degradation in the checkout flow is an emergency, while a 10% degradation in the "about us" page is not. This sense of proportion allows them to focus their energy where it truly matters.

The Compounding Effect of the Ten Habits

Individually, each of these habits will make you a better developer. But their true power is not additive; it's multiplicative. They compound.

Reading code widely (Habit 1) builds the mental library that informs your engineering judgment (Habit 10). Understanding the "why" (Habit 2) allows you to make the pragmatic trade-offs required by strategic laziness (Habit 5) and sound judgment (Habit 10). Cultivating deep focus (Habit 6) is what enables the deliberate learning (Habit 9) that prevents you from making naive decisions. Treating debugging as a science (Habit 3) and maintaining a feedback loop with production (Habit 8) provide the raw data that your judgment synthesizes into wisdom.

This is not a checklist to be completed. It is a system to be grown, a identity to be adopted. You will not master these in a week, or a year. This is the work of a career.

Start with one. Pick the habit that resonates most with you right now, the one that feels both necessary and just out of reach. Practice it deliberately for a month. Then add another.

The path from a good programmer to a great one is not a straight line. It's a spiral. You will circle back to these habits again and again throughout your career, each time understanding them more deeply, each time integrating them more fully into your practice.

The destination is not a job title or a salary. The destination is becoming the kind of developer who doesn't just write code, but who solves problems. The kind of developer who doesn't just build features, but who builds systems that are robust, maintainable, and a genuine pleasure to work with. The kind of developer who leaves every codebase, every team, and every organization better than they found it.

That is the work. That is the craft. And it begins with the decision, right now, to not just be good, but to begin the deliberate, lifelong practice of becoming great.

Permalink

Toronto, meet Nu: Building the purple future from Canada

When Nubank was born in 2013, the mission was simple but ambitious: to fight complexity and empower people through simple, transparent, and accessible financial products.

A lot has changed since then. Today, Nubank is one of the largest digital financial services platforms in the world, with more than 127 million customers across Brazil, Mexico, and Colombia.

We are the third largest financial institution in Brazil, with the lowest complaint rate among the country’s top 15 banks. And we continue to grow with the purpose of building technology that gives people control over their financial lives.

We’ve become a global company listed on the NYSE (NU), recognized by Time, Fast Company, and Forbes as one of the most innovative and fastest-growing companies in the world.

But even after a decade, one thing hasn’t changed: we believe the best way to build the future of finance is with technology made by people, for people.

Why Toronto

From now on, Nubank is seeking talent in Toronto to accelerate its global journey of technology and innovation.

Our goal is not to open a physical operation in Canada, but to connect local professionals with our global engineering ecosystem, a network that already spans Brazil, Mexico, Colombia, the United States, Germany, Uruguay, and Argentina.

Choosing Toronto to represent this new moment was an easy decision. After all, the city is one of the world’s leading tech hubs, with a vibrant community of engineers and startups. It’s a mature, diverse, and multicultural market that values autonomy, collaboration, and purpose.

And these values, of course, are deeply aligned with Nubank’s culture.

“Nubank’s journey started with a world-class team building a digital-first stack that has enabled us to serve over 127 million customers with unmatched scale and resilience. We are now looking for the next generation of world-class engineers – individuals passionate about fighting complexity and building a truly customer-centric future of finance. Tapping into Toronto’s incredible talent pool will help us leverage our technical advantages, from our robust cloud infrastructure to advances in AI, and redefine financial services for hundreds of millions of people across the globe,” said Eric Young, CTO, Nubank. 

This expansion is not only geographic—it’s cultural and strategic. It’s an invitation for engineers in Canada to join a global project that’s redefining the future of finance in Latin America and impacting the lives of hundreds of millions of people.

Engineering at the heart of everything

At Nubank, technology is at the core of everything we do. Our ecosystem grows sustainably, combining efficiency, scalability, and impact.

Our stack is cloud-native, distributed, and immutable, designed for high performance, security, and resilience. It can be understood through four main pillars:

Cloud infrastructure

As a 100% digital product, Nubank relies on a strong partnership with the AWS ecosystem, which allows us to deliver products efficiently and at scale. This includes components such as storage, routing, session management, and Lambda functions.

Security is an essential part of this structure. With multiple processes and continuous alignment to financial regulations, we ensure full compliance in every operation.

Backend

We use Clojure and Go in our backend layer. Clojure, a functional language strategically chosen by Nubank, offers key advantages: immutable concurrency, immutable data structures, function purity, and expressiveness, making the code easier to understand, test, and maintain, without sacrificing power or conciseness.

Today, we operate with more than 3,000 microservices, all designed to support the scale and speed our customers expect.

Mobile

We adopt an approach called Backend-Driven Content (BDC), which allows us to design screens directly on the server side, without needing to code them in Flutter or native Android/iOS languages.

This drastically accelerates deployment time and reduces dependencies on app store reviews or release cycles.

Database

We use Datomic as our transactional database, a technology built around immutability, an essential differentiator in the financial sector, where maintaining permanent and reliable records of every transaction is critical.

In addition, our microservices-based and backend-driven architecture runs on more than 85 Kubernetes clusters, with 355,000 active pods and 1 petabyte of logs ingested daily.

A new chapter in our global journey

Our talent expansion in Toronto reinforces our commitment to attracting professionals who share our mission and help expand our global presence.

It’s also a bet on the talent and diversity that drive the tech ecosystem in Canada.

It’s also a reminder of what brought us here: the belief that technology can transform realities, build bridges, and simplify what once seemed impossible.

Technology companies have a set of competitive advantages that allow them to gain ground in markets traditionally dominated by large institutions, and Nubank is living proof of that.

We’re building the purple future. Now, also from Toronto.

The post Toronto, meet Nu: Building the purple future from Canada appeared first on Building Nubank.

Permalink

Native Apps with ClojureScript, React and Static Hermes

What you see in the demo below is ClojureScript UIx app driving native window with ImGui UI via custom React reconciler. Both hot-reloading and REPL-driven development are supported, as you'd expect it from a typical ClojureScript project.

The JavaScript side of the app runs in Hermes engine, which exposes ImGui to JS env. However, a release build of this exact app is a fully native 8MB executable. You can try it yourself on macOS or Linux.

While Hermes engine is a JavaScript VM, it is also an AOT compiler for JavaScript that emits either Hermes byte code or C.

Compiling this sample JavaScript program with Static Hermes: shermes -emit-c index.js

print(1 + globalThis.value);

outputs the following C program or 50KB executable, when compiled straight into a binary shermes -Os index.js

locals.t0 = _sh_ljs_get_global_object(shr);
frame[3] = _sh_ljs_try_get_by_id_rjs(shr,&locals.t0, get_symbols(shUnit)[1] /*print*/, get_read_prop_cache(shUnit) + 0);
locals.t0 = _sh_ljs_try_get_by_id_rjs(shr,&locals.t0, get_symbols(shUnit)[2] /*globalThis*/, get_read_prop_cache(shUnit) + 1);
locals.t0 = _sh_ljs_get_by_id_rjs(shr,&locals.t0,get_symbols(shUnit)[3] /*value*/, get_read_prop_cache(shUnit) + 2);
np0 = _sh_ljs_double(1);
frame[1] = _sh_ljs_add_rjs(shr, &np0, &locals.t0); // 1 + globalThis.value
np0 = _sh_ljs_undefined();
frame[4] = _sh_ljs_undefined();
frame[2] = _sh_ljs_undefined();
locals.t0 = _sh_ljs_call(shr, frame, 1);
_sh_leave(shr, &locals.head, frame);
return np0;

The above demo is 8MB executable on macOS, where 3MB is Hermes runtime, 1.8MB is React the JavaScript library compiled to C and the rest 3.2MB goes to native libraries ImGui and libwebsockets, and ClojureScript code compiled to C.

Here's a rough diagram of how things are bundled in the executable.

You may noticed there's untyped and typed JavaScript. When AOT compiling typed JavaScript, which can be a subset of either TypeScript or Flow, Hermes emits a more optimal C making it suitable for performance sensitive code, such as bindings to native libraries runnning in a hot loop.

As an example, compiling the following typed program with -typed flag: shermes -typed -emit-c index.js

function add(a: number): number {
  return a + globalThis.value;
}
print(add(1));

emits C that:

  • doesn’t create an add function/closure, instead the arithmetic is emitted inline
  • adds type assertion for unkown value globalThis.value
  • includes fewer symbols & caches

You can check both typed and untyped C here, here's also a diff to make it more obvious.

As I said earlier, the app runs raw JavaScript in dev to provide fast feedback loop and support interactive development. This is possible because Hermes has multiple tiers of code optimization:

  1. During developent JavaScript VM loads the code and emits byte code (JIT) at runtime
  2. You can also compile JavaScript to bytecode (.hbc file) ahead of time. This use case is quite popular in React Native, where application code is compiled into Hermes bytecode at build time to improve startup time of mobile apps, especially on Android.
  3. Finally, with Static Hermes you can compile JavaScript to native object file. This way you get statically linked machine code, skipping parsing and JIT.

Now the best part about this setup is actually React itself. I didn't have to change anything in UIx to make it work with original imgui-react-runtime demo. The real power and a curse of React is that it is a platform independent abstraction. It's up to the host platform to provide specific realization of the contract. If you are Clojure developer, this might sound familiar to you.

The abstraction in question is React's reconciler. By implementing its interface you can create custom render targets, such as React Native, PixiJS React, react-three-fiber, you can even create Terminal UIs and PDFs with it. Heck, if you take a step back and think about the reconciler as a generic abstraction for turning declarative description of something into a set of imperative operations, you can even run hardware with it.

Ok, so how fast is AOT compiled JavaScript with Static Hermes? Here's a quick benchmark:

(simple-benchmark []
    (->> (range)
         (map inc)
         (filter odd?)
         (map #(zipmap (repeatedly % Math/random) (repeatedly % Math/random)))
         (take 1e2)
         (reduce #(apply + %1 (mapcat identity %2)) 0))
    1e2)

Optimized executable runs for ~6450 ms. Same code executed as JavaScript in Node and Bun runs for ~1100 ms. That's 6x slower, ouch. On the bright side, the executable is just a few MBs, while embedding JS VM will get you at least up to 60MB in the case of Bun. If you ask me, I'll always prefer 6x faster app, doesn't matter how big it is.

However, the case with React is quite specific. React being a declarative abstraction on top of imperative APIs almost always means that hot code paths are executed in underlying layers, whether it's browser's DOM, mobile views, WebGL or ImGui. And often times you have an option to use that lower level, more performant API directly, when needed.

Hermes was originally developed for React Native and mobile apps, with a focus on improving startup performance. Even generated native code can't compete with JITs in modern JavaScript engines.

Bonus point. Similarly to how shadow-cljs displays compiler warnings and errors in a browser, the demo ImGui app also captures and displays warnings and errors, as well as includes an error boundary component that catches runtime exceptions thrown during render phase.

Checkout the code of this project at roman01la/cljs-static-hermes, which is based off the work done by a member of Hermes team, at tmikov/imgui-react-runtime. If you are interested in driving native window from runtimes like Node or Bun, checkout this project showcasing use of Bun's FFI to interface with GLFW and OpenGL: roman01la/hra.

Permalink

Eight Queens in core.logic

Eight Queens in core.logic

Eight Queens in core.logic

Welcome to my blog.

Here I report my core.logic solution to the classic Eight queens puzzle.

(ns eight-queens
  (:refer-clojure :exclude [==])
  (:require
   [clojure.string :as str]
   [clojure.core.logic :refer :all :as l]
   [clojure.core.logic.fd :as fd]))

;; Classic AI problem:
;;
;;
;; Find a chess board configuration, where (n=8) queens are on the board;
;; And no pairs attack each other.
;;

;;
;; representation:
;;
;; permutation <0...7>,
;; 1 number per row,
;; so that [0,1,2,3,4,5,6,7] is placing all queens on a diagonal (everybody attacks each other).
;;
;; - constrains the configuration:
;; - all queens are on different rows.
;; - all queens are on different columns.
;;
;; This is fine, because those configurations are not in the set of solutions.
;;
;;

  (defn queens-logic
    [n]
    ;; make (n=8) logic variables, for each row
    (let [colvars (map (fn [i] (l/lvar (str "col-" i))) (range n))]
      (l/run* [q]
        ;; 1. assign the domain 0-8
        (everyg (fn [lv] (fd/in lv (fd/interval (dec n)))) colvars)
        ;; 2. 'row must be different' constraint // permutation
        (fd/distinct colvars)
        ;; 3. diagonal constraint
        ;; for each queen, say that the other queens are not attacking diagonally
        (and* (for [i (range n)
                    j (range (inc i) n)
                    :let [row-diff (- j i)
                          ci (nth colvars i)
                          cj (nth colvars j)]]
                (fresh []
                  ;; handle south-east and north-east cases
                  (fd/eq (!= cj (+ ci row-diff)))
                  ;; '-' relation didn't work somehow
                  (fd/eq (!= ci (+ cj row-diff))))))
        (l/== q colvars))))

  (take 1 (queens-logic 8))
  '((1 3 5 7 2 0 6 4))

In relational programming, the code constructs are logic variables and goals. We write a program that sets up the constraints of the variables, then hand it to the logic engine with run.

After deciding on the clever representation, a permutation of column positions, we can program the constraints we need:

  1. Each queen is a number between 0 and n = 8 for each row, the number says which column it is on (or vise-versa).
  2. Each queen is a different number from the others - it is on a different row. (1+2 are the 'permutation constraint')
  3. The queens don't attack each other diagonally.

Verifying the correctness:

(comment

  ;; reference is definend below
  (def refence-outcome (find-all-solutions 8))

  (def outcome (queens-logic 8))

  [(= (into #{} refence-outcome) (into #{} outcome))
   (every? zero? (map quality refence-outcome))
   (every? zero? (map quality outcome)) (count outcome)]

  ;; =>
  [true true true 92])


ai generated back-track and a hill-climber solution:

;; ============================
;; Helpers and non-relational solutions

(defn quality
  "Count the number of queens attacking each other in the given board configuration.
   board-config is a vector of column positions, one per row.
   Returns the number of pairs of queens that attack each other."
  [board-config]
  (let [n (count board-config)]
    (loop [row1 0
           conflicts 0]
      (if (>= row1 n)
        conflicts
        (let [col1 (nth board-config row1)
              new-conflicts (loop [row2 (inc row1)
                                   acc 0]
                              (if (>= row2 n)
                                acc
                                (let [col2 (nth board-config row2)
                                      ;; Check diagonal attacks
                                      diag-attack? (= (Math/abs (- row1 row2))
                                                      (Math/abs (- col1 col2)))]
                                  (recur (inc row2)
                                         (if diag-attack? (inc acc) acc)))))]
          (recur (inc row1) (+ conflicts new-conflicts)))))))

(defn valid-solution?
  "Returns true if the board configuration has no conflicts."
  [board-config]
  (zero? (quality board-config)))

(defn print-board
  "Prints a visual representation of the board."
  [board-config]
  (let [n (count board-config)]
    (doseq [row (range n)]
      (let [col (nth board-config row)]
        (println (apply str (for [c (range n)]
                              (if (= c col) "Q " ". ")))))))
  (println))

(defn solve-backtrack
  "Solves the N-Queens problem using backtracking.
   Returns the first valid solution found, or nil if none exists."
  [n]
  (letfn [(safe? [board row col]
            (let [board-vec (vec board)]
              ;; Check if placing a queen at [row col] is safe
              (not (some (fn [r]
                           (let [c (nth board-vec r)]
                             (or (= c col)
                                 (= (Math/abs (- r row))
                                    (Math/abs (- c col))))))
                         (range row)))))

          (place-queens [board row]
            (if (= row n)
              board ;; Solution found
              (some (fn [col]
                      (when (safe? board row col)
                        (place-queens (conj board col) (inc row))))
                    (range n))))]

    (place-queens [] 0)))

(defn find-all-solutions
  "Finds all solutions to the N-Queens problem.
   Returns a sequence of all valid board configurations."
  [n]
  (letfn [(safe? [board row col]
            (let [board-vec (vec board)]
              (not (some (fn [r]
                           (let [c (nth board-vec r)]
                             (or (= c col)
                                 (= (Math/abs (- r row))
                                    (Math/abs (- c col))))))
                         (range row)))))

          (place-queens [board row]
            (if (= row n)
              [board] ;; Return solution in a vector
              (mapcat (fn [col]
                        (when (safe? board row col)
                          (place-queens (conj board col) (inc row))))
                      (range n))))]

    (place-queens [] 0)))

(defn random-config
  "Generates a random board configuration of size n."
  [n]
  (vec (shuffle (range n))))


(defn solve-hill-climbing
  "Solves the N-Queens problem using hill climbing with random restarts.
   max-restarts: maximum number of random restarts to attempt.
   max-steps: maximum steps per climb attempt."
  [n & {:keys [max-restarts max-steps]
        :or {max-restarts 100 max-steps 1000}}]
  (letfn [(swap-positions [config i j]
            (assoc config i (nth config j) j (nth config i)))

          (get-neighbors [config]
            (for [i (range n)
                  j (range (inc i) n)]
              (swap-positions config i j)))

          (climb [config steps]
            (if (or (zero? steps) (valid-solution? config))
              config
              (let [current-quality (quality config)
                    neighbors (get-neighbors config)
                    better-neighbors (filter #(< (quality %) current-quality) neighbors)]
                (if (empty? better-neighbors)
                  config ;; Local minimum reached
                  (recur (first (sort-by quality better-neighbors))
                         (dec steps))))))]

    (loop [restarts 0]
      (if (>= restarts max-restarts)
        nil ;; Failed to find solution
        (let [start-config (random-config n)
              result (climb start-config max-steps)]
          (if (valid-solution? result)
            result
            (recur (inc restarts))))))))


(comment
  ;; Example usage:

  ;; Test the quality function
  (quality [0 1 2 3 4 5 6 7]) ;; All on diagonal - many conflicts
  ;; => 28

  (quality [0 4 7 5 2 6 1 3]) ;; A valid solution
  ;; => 0

  ;; Solve for 8 queens using backtracking
  (def solution (solve-backtrack 8))
  solution
  ;; => [0 4 7 5 2 6 1 3]

  (print-board solution)
  ;; Q . . . . . . .
  ;; . . . . Q . . .
  ;; . . . . . . . Q
  ;; . . . . . Q . .
  ;; . . Q . . . . .
  ;; . . . . . . Q .
  ;; . Q . . . . . .
  ;; . . . Q . . . .

  ;; Find all solutions
  (def all-sols (find-all-solutions 8))
  (count all-sols)
  ;; => 92 (there are 92 distinct solutions for 8 queens)

  ;; Solve using hill climbing
  (def hc-solution (solve-hill-climbing 8))
  (print-board hc-solution)

  ;; Test quality on various board sizes
  (quality [0 1]) ;; => 0 (2 queens, no conflict)
  (quality [0 2 1]) ;; => 0 (3 queens, valid)
  (quality [1 3 0 2]) ;; => 0 (4 queens, valid)
  )

I could print all solutions; Why not do it with html; So it renders on this blog.

ai generated

(require '[hiccup2.core :as html])

(defn board-to-hiccup
  "Converts a board configuration to hiccup format with checkerboard pattern."
  [board-config]
  (let [n (count board-config)]
    [:div {:style {:display "inline-block"
                   :border "2px solid #333"}}
     (for [row (range n)]
       (let [col (nth board-config row)]
         [:div {:style {:display "flex"}}
          (for [c (range n)]
            (let [is-dark? (odd? (+ row c))
                  has-queen? (= c col)]
              [:div {:style {:width "60px"
                             :height "60px"
                             :background-color (if is-dark? "#769656" "#eeeed2")
                             :display "flex"
                             :align-items "center"
                             :justify-content "center"
                             :font-size "40px"
                             :font-weight "bold"
                             :color "#000"}}
               (when has-queen? "♕")]))]))]))

(spit
 "board.html"
 (html/html
     [:div
      {:style
       {:display "flex"
        :padding "8px"
        :gap "8px"
        :flex-wrap "wrap"}}
      (doall (map board-to-hiccup (queens-logic 8)))]))

All solutions printed because why not

Permalink

The “Jankiest” way of writing Ruby gems

Recently I wrote about a Jank, the programming language that is able to bring Cojure to the native world using LLVM. I also mentioned that Jank is not yet ready for production usage – and that is just true; it still have many bugs that prevent us to use it for more complex problems, and maybe even some simple ones.

That doesn’t mean we can’t try it, and see how bright the future might be.

At my last post I ended up showing in an example of a Ruby code. Later, after many problems trying to make something more complex, I finally was able to hack up some solution that bypasses some of the bugs that Jank still have.

And let me tell you, even in the pre-alpha stage that the language is, I can already see some very interesting things – the most important one being the “interactive development” of the native extension – or if you want to use the Ruby terms, monkey patching native methods.

Let’s start with some very simple code: a Ruby method that’s supposed to be written in a native language. I’m going to start using C++ because it’s the main way of doing that. For now, don’t worry about the details – it’s just a class creation with a method that just prints “hello word” in the screen:

#include &lt;iostream&gt;
#include &lt;ruby.h&gt;

static VALUE hello(VALUE self) {
  std::cout &lt;&lt; &quot;Hello, world!\n&quot;;
  return Qnil;
}

void define_methods() {
  VALUE a_class = rb_define_class(&quot;Jank&quot;, rb_cObject);
  rb_define_method(a_class, &quot;hello&quot;, RUBY_METHOD_FUNC(hello), 0);
}

extern &quot;C&quot; void Init_jank_impl() {
  define_methods();
}

We should be able to port this directly to Jank by just translating from C++ to a Clojure-like syntax – should being the keyword here, because we can’t. There are a bunch of different bugs that prevents us from doing that right now:

  1. We can’t define the hello funcion, because we have no way to add type signatures to Jank functions. To bypass that, we have to define the method in C, using cpp/raw, which brings us to
  2. Using cpp/raw to define C++ functions doesn’t work – the code generated will somehow duplicate the method definition, so things won’t compile. We can solve that by using the same technique that C headers use – we use #ifndef and #define to avoid duplications
  3. Unfortunately, Jank doesn’t actually understand the callback method signature. It expects something that matches exactly the method signature, but for Ruby callbacks (of native functions) the C API sometimes use “type aliases” or “C preprocessors/macros”. We can also solve that, but we need to “cast” the C function with the “concrete” signature to the one with the “abstract” one so that Jank will be happier.
  4. Finally, Jank doesn’t understand C macros/preprocessors. So in some cases (for example, converting a Ruby string to a C one) we’ll also need to generate an “intermediate function” to solve the issue.

First experiments

With all that out of the way, we can actually do some experiments:

(ns ruby-ext)

(cpp/raw &quot;
#ifndef JANK_HELLO_FN
#define JANK_HELLO_FN
#include &lt;jank/c_api.h&gt;
#include &lt;ruby.h&gt;

static VALUE jank_hello(VALUE self) {
  std::cout &lt;&lt; \&quot;Hello, world\\n\&quot;;
  return Qnil;
}

static auto convert_ruby_fun(VALUE (*fun)(VALUE)) {
  return RUBY_METHOD_FUNC(fun);
}

#endif
&quot;)

(defn init-ext []
  (let [class (cpp/rb_define_class &quot;Jank&quot; cpp/rb_cObject)]
    (cpp/rb_define_method class &quot;hello&quot; (cpp/convert_ruby_fun cpp/jank_hello) 0)))

And this works. But obviously, that’s not what we want. We want a Jank function to be called that will be used as the Ruby implementation. To do that, we can actually use the C API to callback Jank, so let’s just add a function and change our jank_hello code to call this new function:

(defn hello []
  (println &quot;Hello, from JANK!&quot;))

(cpp/raw &quot;
...
static VALUE jank_hello(VALUE self) {
  rb_ext_ractor_safe(true);
  auto const the_function(jank_var_intern_c(\&quot;ruby-ext\&quot;, \&quot;hello\&quot;));
  jank_call0(jank_deref(the_function));
  return Qnil;
}
...
&quot;)

Compiling the Jank code

This actually works, and we get a Hello, from JANK! appearing in the console! But to make that work, we need to compile the Jank code, and then link that together with our C++ “glue code” before we can use that in Ruby. But to compile for Jank, we actually need to include the Ruby headers and the ruby library directory, otherwise it won’t work – but knowing which flags to use can be a challenge. Luckily Ruby can help with that:

ruby -e &quot;require &#039;rbconfig&#039;; puts &#039;-I&#039; + RbConfig::CONFIG[&#039;rubyhdrdir&#039;] + &#039; -I&#039; + RbConfig::CONFIG[&#039;rubyarchhdrdir&#039;] + &#039; -L&#039; + RbConfig::CONFIG[&#039;libdir&#039;] + &#039; -l&#039; + RbConfig::CONFIG[&#039;RUBY_SO_NAME&#039;]&quot;

That will return the flags you need. Then you save your file as ruby_ext.jank, and compile it with jank <flags-from-last-cmd> --module-path . compile-module ruby_ext. Unfortunately, the output (the ruby_ext.o file) goes to a different directory depending on lots of different things (at least on my machine) – so my build script first deletes the target directory, then use a wildcard in the extconf.rb file (the file that’s used to prepare a Ruby code) so that we can actually get anything under target directory. Then, finally, we can build a final Ruby gem:

#extconf.rb
require &#039;mkmf&#039;

dir_config(&#039;jank&#039;, [&#039;.&#039;])
$CPPFLAGS += &quot; -I/usr/local/lib/jank/0.1/include&quot;
RbConfig::MAKEFILE_CONFIG[&#039;CC&#039;] = &#039;clang++&#039;
RbConfig::MAKEFILE_CONFIG[&#039;CXX&#039;] = &#039;clang++&#039;
RbConfig::MAKEFILE_CONFIG[&#039;LDSHARED&#039;] = &#039;clang++ -shared&#039;

with_ldflags(&quot;
  -L/usr/local/lib/jank/0.1/lib/
  target/*/ruby_ext.o
  -lclang-cpp
  -lLLVM
  -lz
  -lzip
  -lcrypto
  -l jank-standalone
&quot;.gsub(&quot;\n&quot;, &quot; &quot;)) do
  create_makefile(&#039;jank_impl&#039;)
end

#jank_impl.cpp
#include &lt;ruby.h&gt;
#include &lt;jank/c_api.h&gt;

using jank_object_ref = void*;
extern &quot;C&quot; jank_object_ref jank_load_clojure_core_native();
extern &quot;C&quot; jank_object_ref jank_load_clojure_core();
extern &quot;C&quot; jank_object_ref jank_var_intern_c(char const *, char const *);
extern &quot;C&quot; jank_object_ref jank_deref(jank_object_ref);
extern &quot;C&quot; void jank_load_ruby_ext();

extern &quot;C&quot; void Init_jank_impl() {
  auto const fn{ [](int const argc, char const **argv) {
    jank_load_clojure_core_native();
    jank_load_clojure_core();

    jank_load_ruby_ext();
    auto const the_function(jank_var_intern_c(&quot;ruby-ext&quot;, &quot;init-ext&quot;));
    jank_call0(jank_deref(the_function));

    return 0;
  } };

  jank_init(0, NULL, true, fn);
}

And compile with rm target -Rf && jank <flags> --module-path . compile-module ruby_ext && ruby extconf.rb && make. This surprisingly works! I say “surprisingly” because, remember, this way of using the language is not officially supported!

Refactoring the glue code away

Now, this whole number of “glue code” is a problem – it’s quite tedious to remember to make a C code, then convert that C code to be usable via Ruby. But LISPs have macros, and Jank is no exception – so let’s define a defrubymethod that accepts a parameter+type, and will generate all boilerplate for us:

(cpp/raw &quot;
#ifndef JANK_CONVERSIONS
#define JANK_CONVERSIONS
static jank_object_ref convert_from_value(VALUE value) {
  return jank_box(\&quot;unsigned long *\&quot;, (void*) &amp;value);
}
#endif
&quot;)

(defmacro defrubymethod [name params &amp; body]
  (let [rb-fun-name (replace-substring (str name) &quot;-&quot; &quot;_&quot;)
        cpp-code (str &quot;#ifndef &quot; rb-fun-name &quot;_dedup\n#define &quot; rb-fun-name &quot;_dedup\n&quot;
                      &quot;static VALUE &quot; rb-fun-name &quot;_cpp(&quot;
                      (-&gt;&gt; params (map (fn [[param type]] (str type &quot; &quot; param)))
                           (str/join &quot;, &quot;))
                      &quot;) {\n&quot;
                      &quot;  auto const the_function(jank_var_intern_c(\&quot;&quot; *ns* &quot;\&quot;, \&quot;&quot; name &quot;\&quot;));\n&quot;
                      &quot;  jank_call&quot; (count params) &quot;(jank_deref(the_function), &quot;
                      (-&gt;&gt; params (map (fn [[param]] (str &quot;convert_from_value(&quot; param &quot;)&quot;))) (str/join &quot;, &quot;))
                      &quot;);\n&quot;
                      &quot;  return Qnil;\n}\n#endif&quot;)]
    `(do
       (cpp/raw ~cpp-code)
       (defn ~name ~(mapv first params) ~@body))))

We need to implement replace-substring too, because str/replace is still not available in Jank. Anyway, now we can create Ruby methods like this:

(defrubymethod hello-world [[self VALUE]]
  (println &quot;HELLO, WORLD!&quot;))

(defn init-ext []
  (let [class (cpp/rb_define_class &quot;Jank&quot; cpp/rb_cObject)]
    (cpp/rb_define_method class &quot;hello_world&quot; (cpp/convert_ruby_fun cpp/hello_world_cpp) 0)))

Way easier, and no boilerplate. But now, it’s “superpower” time – because Jank is dynamic, we can actually reimplement hello-world while the code is running and Ruby will use the new implementation – it doesn’t matter that the implementation is native!

A REPL (kinda – more like a REP)

Of course, we also have a problem with this approach: Jank doesn’t actually have a REPL API (yet), so the easiest way to solve this is to add another method in our Ruby class that will evaluate something in Jank. This might sound confusing, and it kinda is – we’re actually registering, in Jank, a method that will be called by Ruby; this method will call a C++ code, that will delegate to Jank, to evaluate some Jank code. The trick here is to be able to pass Ruby parameters do Jank too, and to make this a reality what I decided to do is:

  1. Ruby will pass a string containing Jank code; that string can refer some pre-defined variables p0, p1, etc – each is a “parameter” that we can pass
  2. To be able to “bind” these variables, we’ll create a Jank function called __eval-code that will contain these parameters. We then will call this __eval-code and we’ll “box” the Ruby parameters (“boxing” is a way for Jank to be able to receive any arbitrary C++ value)
  3. To be able to inspect these values, we’ll also create a rb-inspect function that will call inspect in the “unboxed” Ruby value
(defn define-eval-code [code num-of-args]
  (let [params (-&gt;&gt; num-of-args
                    range
                    (mapv #(str &quot;p&quot; %)))
        code (str &quot;(defn __eval-code [&quot; params &quot;] &quot; code &quot;\n)&quot;)]
    (eval (read-string code))))

(cpp/raw &quot;
...
static const char * inspect(VALUE obj) {
  ID method = rb_intern(\&quot;inspect\&quot;);
  VALUE ret = rb_funcall(obj, method, 0);
  return StringValueCStr(ret);
}

static VALUE eval_jank(int argc, VALUE *argv, VALUE self) {
  try {
    const char *code = StringValueCStr(argv[0]);
    auto const the_function(jank_var_intern_c(\&quot;ruby-ext\&quot;, \&quot;define-eval-code\&quot;));
    jank_call2(jank_deref(the_function), jank_string_create(code), jank_integer_create(argc));

    auto const the_function2(jank_var_intern_c(\&quot;ruby-ext\&quot;, \&quot;__eval-code\&quot;));
    std::vector&lt;jank_object_ref&gt; arguments;
    for(int i = 0; i &lt; argc; i++) {
      arguments.push_back(jank_box(\&quot;unsigned long *\&quot;, (void*) &amp;argv[i]));
    }

    jank_call1(jank_deref(the_function2),
      jank_vector_create(argc,
        arguments[0],
        arguments[1],
        arguments[2],
        arguments[3],
        arguments[4],
        arguments[5],
        arguments[6],
        arguments[7],
        arguments[8],
        arguments[9],
        arguments[10],
        arguments[11],
        arguments[12],
        arguments[13]
    ));
  } catch(jtl::ref&lt;jank::error::base&gt; e) {
    std::cerr &lt;&lt; \&quot;ERROR! \&quot; &lt;&lt; *e &lt;&lt; \&quot;\\n\&quot;;
  }
  return Qnil;
}

static auto convert_ruby_fun_var1(VALUE (*fun)(int, VALUE*, VALUE)) {
  return RUBY_METHOD_FUNC(fun);
}
...&quot;)

(defn rb-inspect [boxed]
  (let [unboxed (cpp/* (cpp/unbox cpp/VALUE* boxed))]
    (cpp/inspect unboxed)))

(defn init-ext []
  (let [class (cpp/rb_define_class &quot;Jank&quot; cpp/rb_cObject)]
    (cpp/rb_define_method class &quot;hello_world&quot; (cpp/convert_ruby_fun cpp/hello_world_cpp) 0)
    (cpp/rb_define_method class &quot;eval_jank&quot; (cpp/convert_ruby_fun_var1 cpp/eval_jank) -1)))

This… was a lot of code. But here’s what it is: define-eval-code will basically concatenate (defn __eval-code [[p0 p1 p2 p3]] ... ) for example, if we pass 3 parameters (self will always be p0). Then we have the implementation, in C++, for eval_jank – this will just get the first parameter, convert to a const char*, and pass that to define-eval-code that we created previously; this will be evaluated, and we’ll create (or patch!) the __eval-code function. Now, the problem is how to pass parameters – I hacked a bit the solution by creating a C++ vector, “boxing” everything, and then arbitrarily creating a Jank vector with 14 elements; and that’s it.

Notice also that jank_box is passing the type as unsigned long * instead of VALUE *. This is actually because, again, Jank doesn’t support type aliases for now – so VALUE * gets resolved in Jank to unsigned long *, but don’t get resolved in C++. Also, be aware of the space between long and * – this is something that Jank also needs, otherwise you won’t be able to unbox the variable.

Now that we have everything in order, we can finally test some superpowers!

Ruby console

Ruby have a REPL, and that’s what we’ll use. After compiling the whole code, run irb and then you can test the code:

irb(main):001&gt; jank = Jank.new
=&gt; #&lt;Jank:0x00007324fec7da08&gt;
irb(main):002&gt; jank.hello_world
HELLO, WORLD!
=&gt; nil
irb(main):003&gt; jank.eval_jank &#039;(prn (rb-inspect p1))&#039;, [1, 2, 3]
&quot;[1, 2, 3]&quot;
=&gt; nil
irb(main):004&gt; jank.eval_jank &#039;(defn hello-world [_] (println &quot;Another Implementation!&quot;) )&#039;
=&gt; nil
irb(main):005&gt; jank.hello_world
Another Implementation!
=&gt; nil

Yes – in line 004, we reimplemented hello-world, and now Ruby happily uses the new version! Supposedly, this could be used to implement the whole code interactively – but without proper nREPL support, we can’t – some stuff simply doesn’t work (yet, probably) like cpp/VALUE – it depends on Ruby headers being present, the compilation flags, etc. Maybe in the future we can have better support for shared libraries, who knows?

And here is the very interesting part – this is not limited to Ruby. Any language that you can extend via some C API can be used via Jank, which is basically almost any language – Python, Node.JS, and maybe even WASM in the future. The future is very bright, and with some interesting and clever macros, we might have an amazing new choice to make extensions to languages!

And… another thing

Most of the time, when doing extensions, one of the worst things is the need to convert between the native types and the language ones. In Ruby, everything is a VALUE – in Jank, everything is a jank_object_ref. But again, Jank have macros – can we use that to transparently convert these types? And in such a way that’s fast, relies on no reflection, etc? Turns out we can – we can change the defrubymethod to receive a parameter before our body that will be the “return type”. We then will implement some Jank->Ruby conversions, and vice-versa, and transparently convert, box and unbox stuff, etc. So as a “teaser” here’s one of the first versions of this conversion:

(defn ruby-&gt;jank [type var-name]
  (case type
    Keyword (str &quot;to_jank_keyword(&quot; var-name &quot;)&quot;)
    String (str &quot;to_jank_str(&quot; var-name &quot;)&quot;)
    Integer (str &quot;jank_integer_create(rb_num2long_inline( &quot; var-name &quot;))&quot;)
    Double (str &quot;jank_real_create(NUM2DBL(&quot; var-name &quot;))&quot;)
    Boolean (str &quot;((&quot; var-name &quot; == Qtrue) ? jank_const_true() : jank_const_false())&quot;)
    VALUE (str &quot;jank_box(\&quot;unsigned long *\&quot;, (void*) &amp;&quot; var-name &quot;)&quot;)))

(def jank-&gt;ruby
  &#039;{String &quot;  auto str_obj = reinterpret_cast&lt;jank::runtime::obj::persistent_string*&gt;(ret);
    return rb_str_new2(str_obj-&gt;data.c_str());&quot;
    Integer &quot;  auto int_obj = reinterpret_cast&lt;jank::runtime::obj::integer*&gt;(ret);
    return LONG2NUM(int_obj-&gt;data);&quot;
    Double &quot;  auto real_obj = reinterpret_cast&lt;jank::runtime::obj::real*&gt;(ret);
    return DBL2NUM(real_obj-&gt;data);&quot;
    Boolean &quot;  auto bool_obj = reinterpret_cast&lt;jank::runtime::obj::boolean*&gt;(ret);
    return bool_obj-&gt;data ? Qtrue : Qfalse;&quot;
    Keyword &quot;  auto kw_obj = reinterpret_cast&lt;jank::runtime::obj::keyword*&gt;(ret);
    auto kw_str = kw_obj-&gt;to_string();
    auto kw_name = kw_str.substr(1);
    return ID2SYM(rb_intern(kw_name.c_str()));&quot;
    Nil &quot;  return Qnil;&quot;})

You might be asking: why does it return a string, all the time? And that’s because we’ll generate a C++ string on our macro – this string, which originally was just a way to help define a C++ function with the right signature, pass control do Jank, and return Qnil from Ruby, will now also convert parameters before sending them to Jank, which means we’ll have the correct type information on Jank’s side.

With all this, we can actually create some Jank code without any explicit conversion, that behaves like Jank, and will seamlessly work in Ruby too:

(defrubymethod sum-two-and-convert-to-str [[self VALUE] [p1 Integer] [p2 Integer]] String
  (str (+ p1 p2)))

(defn init-ext []
  (let [class (cpp/rb_define_class &quot;Jank&quot; cpp/rb_cObject)]
    ...
    (cpp/rb_define_method class &quot;sum_and_to_s&quot; (cpp/convert_ruby_fun2 cpp/sum_two_and_convert_to_str_cpp) 2)
    ...))

And then you can run with this:

Jank.new.sum_and_to_s(12, 15)
# =&gt; &quot;27&quot;

As soon as Jank stabilizes, this might actually be the best way to create language extensions! If you want to check out the work so far, see this repository with all the code in this post (and probably more in the future). Just install Jank, run ./build.sh, and cross your fingers 🙂

Permalink

Pindakaas party, plywood problems, and CNC daydreams

Peanut butter tasting (!!!)

I recently re-read Gwern On Having Enough Socks and, uh, now I’m hosting a blind-tasting of a dozen peanut butters this Saturday, November 22 in Amsterdam. If you’re in town, email me for the event details. Together we will find the best 100% pindakaas met stukjes.

A shoe tower

The weather has turned chilly, and my girlfriend is highly encouraging me to build some bedroom storage so our coats and sweaters are more accessible.

Since making a standard box wardrobe out of plywood would be more expensive and time consuming yet aesthetically indistinguishable from Ikea, I decided to explore a lightweight frame using the hardwood dowels left over from my floating castle desk.

In particular, I wanted to cut a groove in the dowels to hold (and hide) the raw edge of thin plywood shelves.

I 3D-printed a custom shoe for my lil’ trim router:

but this yielded poor results, so I ended up buying a full-sized plunge router.

As it turns out, it takes a lot to hold round wood down securely to a table and route a groove exactly down the center. I eventually got there with enough scrap material, clamps, and track saw track as straight edge:

Since I was using decorative pan head screws, I needed both through holes (for the screw shafts) and counterbores (for the shiny heads) along the dowels. Free-handing with a drill worked about as well as you might expect — turns out it’s very obvious when a 15mm diameter counterbore isn’t exactly centered on a 20mm diameter dowel — so I ended up buying a benchtop drill press:

This Bosch PBD 40 has some awesome features like electronic speed control and digital depth readout. Unfortunately, it also came with noticeable wiggle in the chuck and about 1mm deviation between the base plate’s v-groove and the drill center, both of which make it impossible to throw material down and get a repeatable hole location — you have to carefully line everything up first.

As far as I can tell (please let me know if I’m wrong!) there’s not really anything better on the market until you’re willing to spend for the $1,400 Nova Viking benchtop drill press.

After much drilling, I decided there was absolutely no way I was going to enjoy building a shoe tower using dowels (much less an entire bedroom storage system), so I demoted this prototype down to “plant stand”, which ended up turning out OK:

(Just ignore all those extra holes.)

I used brass tube to hide the screw threads. Originally I thought I’d be able to easily cut brass tube with a hacksaw — turns out absolutely not; brass is way too gummy and it’s impossible (for me, anyway) to maintain a consistent cut line by hand. Luckily, I found one of these $10 things which, pleasingly, is extremely good at its one job:

(Notice the the right side piece was crushed in a drill chuck and has an all-over-the-place failed cut from me earlier attempting to cut it by spinning it against a clamped-down hacksaw.)

For my second shoe tower attempt, I wanted a lightweight plywood frame combined with fabric — something in the direction of Mieke Meijer’s Airframe cabinet (which looks cool as hell).

That build went much more smoothly, and the final product turned out better than I expected:

The only hiccup during the build was discovering that my dowel jig won’t work with material thinner than 15mm (my plywood was 12mm). I made a sort of jig/guide using my new drill press and a bit of scrap wood, but the final joints still turned out a bit wonky.

This is a typical woodworking experience for me — a seemingly simple task like “drill holes centered in a plywood edge” takes two hours longer than expected, and even then doesn’t come out quite right. ¯\_(ツ)_/¯

Have you tried rubbing some computer on it?

Back when I had a CNC router in my closet, I used to think it’d be so much faster to work in a more traditional workshop with proper saws and drill presses and stuff.

But now that I’m frustrated in my back garden shed, discovering that I’m missing the appropriate drill size / pattern bit / alignment jig and spending hours carefully drilling not-quite-centered holes, I’ve started daydreaming that, surely, a “conversational” CNC router workflow could be better: clamp stuff down to the machine bed, get a high resolution / magnified image of the work, and specify exactly where and what to cut by pointing at stuff in the computer.

Ideally you’d get an orthographic image — a top-down projection without any perspective distortion.

Since I’m not about to shell out a few grand for a telecentric lens, I started falling down a rabbit hole of building my own camera by using the machine gantry to move a large CCD line-sensor across the bed.

The TCD1304, for example, has 3648 pixels over a 29mm sensor line, runs about $10 on AliExpress, and is used by solid looking open-source hardware and firmware (they’re popular sensors for making your own spectrometer, apparently).

However, I realized that I know basically nothing about optics. Seeing the details of this gigapixel camera from flatbed scanner CCD disabused me of the notion that I’d have a snowball’s chance in hell of making a usable camera.

I spent some time with ChatGPT’s Deep Research exploring other ideas, but it didn’t come up with much besides reminding me that I’ve been spewing tokens on this bullshit for a while now:

Good ol’ YouTube came to the rescue, though — Paper Tools cleverly avoids the entire perspective distortion problem by only allowing you to pick coordinates from the center of the camera image.

I.e., rather than giving you a static orthographic image that you then CAD/CAM on top of, their workflow requires that you literally drive your CNC machine around to the exact point locations every time you select one.

Quite an obvious solution in hindsight, and I’m curious to watch how that project unfolds.

Next: Drawers

After a week of daydreaming about an alternative, fluid conversational CAD/CAM interfaces for the CNC router currently across the Atlantic Ocean from me, I came back to my senses and decided to keep prototyping furniture ideas sans computer.

I figured solid wood might be more satisfying than plywood. Of course, I don’t have a jointer, planer, nor experience working with actual hardwoods, but I do have some leftover 2x softwood I can start with.

My current idea is to lean heavily on bright colors — paint, 3d-printed plastic, and paracord.

I’m experimenting with cross-bracing using the latter two materials:

For the life of me I can’t find a stopper knot that I can slide or otherwise tie under tension — if you have any ideas or suggestions of how to get a tidy looking knot that’ll slide to the left, I’d love to hear them!

Misc. stuff

  • My powered air respirator now works great: I ended up going with a dual-fan design and dropped the belt clip in favor of an easier-to-don sling bag.

  • I love the engineering competence and details in this NTSB interview with a submersible expert. The names are all redacted, but you might be able to guess the interviewee from his first answer: “Well, I’m sure you’re familiar with my film Titanic”. I also particularly liked one of the safety discussions: “I think the most dangerous part of our whole operation was these young software engineers puking over the railing in a high sea.”

  • The Biochemical Beauty of Retatrutide: How GLP-1s Actually Work. I’ve never studied physiology, and this article was a perfect overview of one of the most exciting medical advances of the recent years. I was also delighted when the author connected biochemistry with familiar endurance running feelings.

  • Speaking of nicer visual / probing systems for CNCs, this now-on-Kickstarter Nestworks C500 has a bunch of nice looking quality of life details. I swear it was just yesterday I dropped $10k for a Shopbot Desktop with a sketchy USB connection. I’m so lucky to have such deflationary hobbies.

  • Your cheap furniture has a secret. A history of rubberwood.

  • How Did The World Get So Ugly?

  • Smells Like Teen Spirit (Synth Pop 80s)

  • I’m renting a place with gas cooktop, but I wanted induction so I bought the €40 Ikea single-burner and am extremely happy with it. Normally not a fan of touch controls, but it has extremely responsive ones. The protocol between the display and induction coil has also been reverse engineered in case you want to add your own control loop. (I don’t need another project, I don’t need another project..)

  • “They needed to connect the organ promptly to the patient manually, with a technique called anastomosis. The medical sutures of the time were too thick, and the needles too large, to avoid damaging delicate blood vessels. To overcome such challenges, the world needed a skilled seamstress.”

  • “Complexity is free in 3d printing, the limit of design geometry is mostly how much time you’re willing to spend in CAD. I wanted to print the most complicated art piece I could think of.”

  • I was curious how the support structures were generated, so I asked Claude to read the code and it gave me a great explanation for $0.04. What a world!

  • Fucking Goddamn Basics of Rationalist Discourse

  • SpreadSheesh! by Dennis Heihoff - Clojure Electric-based spreadsheet with live code evaluation and custom UI rendering

Permalink

Exception handling differences between Clojure map & pmap

With this post, I am in deeper waters than usual. What might sound like a recommendation in the following could be a potential disaster in disguise. Be warned.

Personally, I prefer not to know about implementation details about the function I’m calling. Although that was the situation I suddenly found myself in, when a function I call replaced map with pmap.

Here is how I approached the weirdness with exceptions tangled with pmap.

On the surface, map and pmap appear interchangeable, since they both return a lazy sequence. But the data contract breaks due to how exceptions are handled.

The following example showcases the behavior that caught me by surprise, because I had expected it to return {:error-code 42}:

(try
    (->> (range 1)
         (pmap (fn [_] (throw (ex-info "Oh noes" {:error-code 42}))))
         doall)
    (catch Exception e
      (ex-data e)))
; => nil

It did not. But using a normal map does:

(try
    (->> (range 1)
         (map (fn [_] (throw (ex-info "Oh noes" {:error-code 42}))))
         doall)
    (catch Exception e
      (ex-data e)))
; => {:error-code 42}

doall is necessary to ensure the exception is triggered while inside the try-catch block, instead of just returning an (unrealized) lazy sequence, which will cause havoc later.

As far as I know, pmap uses futures somewhere behind the scenes, which might be the reason why exceptions caused during mapping are wrapped in a java.util.concurrent.ExecutionException.

Since I am in control of the function replacing map with pmap, I decided to put the unwrapping where pmap is called, to hide the implementation detail from the caller:

(try
  (->> coll
       (pmap #(occationally throw-exception %))
       doall)) ; realize lazy seq to trigger exceptions
  (catch Exception e
    ; Unwrap potentially wrapped exception by `pmap`
    (throw (if (instance? java.util.concurrent.ExecutionException e)
             (ex-cause e)
             e)))))

The conditional unwrapping allows for a slightly more complex implementation in the try block that can throw exceptions outside pmap as well.

The above implementation assumes that an ExecutionException always has a cause, which might not be the case - I don’t know.

Use with caution.

Permalink

November 2025 Short-Term Q3 Project Updates

This is the second project update for three of our Q3 2025 Funded Projects. (Reports for the others are on a different schedule). A brief summary of each project is included to provide overall context. Thanks everyone for your awesome work!

Jank: Jeaye Wilkerson
This quarter, I’ll be building packages for Ubuntu, Arch, Homebrew, and Nix. I’ll be minimizing jank’s dependencies, automating builds, filling in test suites for module loading, AOT building, and the Clojure runtime. I’ll be working to get the final Clang and LLVM changes I have upstreamed into LLVM 22, adding a health check to jank to diagnose installation issues, and filling in some C++ interop functionality I couldn’t get to last quarter. Altogether, this quarter is going to be a hodgepodge of all of the various tasks needed to get jank shipped.

Malli: Ambrose Bonnaire-Sergeant
Malli’s core algorithms for transforming schemas into things like validators and generators are quite sensitive to the the input schema. Even seemingly equivalent schemas can have different performance characteristics and different generators.
I would like to create a schema analyzer that can simplify complex schemas, such that two different but equivalent schemas could have the same representation. Using this analyzer, we can build more efficient validators, more reliable generators, more helpful error messages, more succinct translations to other specification tools, and beyond.

Uncomplicate Clojure ML: Dragan Duric
My goal with this funding in Q3 2005 is to develop a new Uncomplicate library, ClojureML.

  • a Clojure developer-friendly API for AI/DL/ML models (in the first iteration based on ONNX Runtime, but later refined to be even more general).
  • Implement its first backend engine (based on ONNX Runtime).
  • support relevant operations as Clojure functions.
  • an extension infrastructure for various future backend implementations.
  • a clean low-level integration with Uncomplicate, Neanderthal, Deep Diamond and Clojure abstractions.
  • assorted improvements to Uncomplicate, Neanderthal, Deep Diamond, to support these additions.
  • develop examples for helping people getting started.
  • related bugfixes.
  • TESTS (of course!).

AND NOW FOR THE REPORTS!

Jank: Jeaye Wilkerson

Q3 2025 $9K, Report No. 2, Published October 31, 2025

Thank you so much for the sponsorship this quarter! This past month of jank development has been focused on stability of all existing features in preparation for the alpha release in December. Of note, I done the following:

  • Stabilized binary builds for macOS, Ubuntu, Arch, and Nix
  • Improved loop IR to support native values without boxing
  • Merged all pending LLVM changes upstream, so jank can build with LLVM 22 (releasing in January 2026)
  • The lein-jank plugin and template have been improved so that starting a new jank project is now just a couple of commands
  • libzip has been removed as a dependency, which enables Ubuntu 25.04 support
  • AOT building on all supported platforms has been stabilized
  • Added a daily scheduled CI binary package check for macOS, Ubuntu, and Arch
  • Added runtime type validation for cpp/unbox usages
  • Added C preprocessor value support, so cpp/FOO can refer to any #define
  • Improved IR gen and static type checking for if so that both branches must provide the same type

Normally I have an associated blog post, but as I’ll be leaving for Clojure Conj 2025 next week, I’m taking the time to polish my talk.


Malli: Ambrose Bonnaire-Sergeant

Q3 2025 $0K, Report No. 2, Published November 11, 2025

This month I fixed a bug in the work I merged last month: Robust :and parser, add :andn

The bugfix got me thinking about recursive refs again (following on from my work on recursive generators in malli.generator from a few years ago), which is a major part of the performance work I proposed for this Clojurists Together project (compiling efficient validators for recursive schemas).

I realized that we can use the cycle detection logic from malli.generator to detect cycles in validators! A major performance difference between Plumatic Schema/Spec and Malli is in recursive validators: Malli never “ties the knot” and will lazily compile and cache every layer of recursion. This means Malli uses linear space for recursive validators, while Schema/Spec use constant space, and Malli can never fully compile a recursive validator and must pay a time cost for compiling schemas at “runtime” (e.g., when your webserver is accepting requests).

I submitted a Malli “discussion” PR Sketch: tying the knot for ref validator proposing an approach to fixing this performance issue for recursive validators. See the PR if you’re interested in the exact details, as it contains a full explanation of the problem and (mostly) the solution, as well as a reference implementation that passes all the tests.

A neat part of this approach is that it is a completely transparent optimization from a user’s perspective, with no extra configuration or changes needed. Similar optimizations may be possible for other operations like m/explain and m/parse, but that’s future work.

For now, the maintainers showed interest in accepting this optimization and my next step will be to further test and validate the approach, and propose a final PR.


Uncomplicate Clojure ML: Dragan Duric

Q3 2025 $9K, Report No. 2, Published October 31, 2025

My goal with this funding in Q3 2005 is to develop a new Uncomplicate library, ClojureML (see above).

Progress during the second month:

In the first month, I have already implemented the first version, which I released to Clojars as org.uncomplicate/diamond-onnxrt version 0.5.0.

So, what has been done since then, in the second month?

We are at version 0.17.0 now!

As I already established solid foundations in the first month, in the second month, I focused on expanding the coverage of ONNX Runtime’s C api in Clojure, writing tests to get a feel how it works, working on hiding the difficult parts under a nicer Clojure API, fixing bugs. It wasn’t always a smooth sail, but with every storm I got a better understanding of the ways things work in ONNX Runtime, and I also covered most of the API available by ONNX Runtime version 1.22.2, at least the relevant part that can be used by Deep Diamond right now (which excludes sparse tensors and string tensors). The majority is covered and well tested.

The second area was using the newly added core API in higher-level integration with Deep Diamond. The result is that the public onnx function can now be integrated with the rest of DD Tensor machinery! And the public API that the user has to see is only one function, while everything related to ONNX is automatically wired under the hood! I am quite pleased with what I achieved on that front. Of course, I also implemented custom configurations that the user can provide as a Clojure map, and the underlying machinery will call the right onnx runtime function to set everything up.

The third big area is CUDA support. Since the whole setup up to this point was quite elegant (I’m sorry for having to praise my own work :), especially having lots of supporting functionality in Deep Diamond and Neanderthal, the Clojure code for this in diamond-onnx was not too demanding. However, the upstream library for this, namely ONNX Runtime’s backend for CUDA, and the build provided by javacpp-presets, was quite difficult to tame. I had to run many multi-hour C++ compilations, thaw through C++ compilation errors, dig to unearth causes, finding ways to fix, helping upstream, and this took lots and lots of time. But, that’s what it had to be done. That’s why I’ll have to wait for upgrade to the newest ONNX Runtime 1.23.2, but I managed to tame version 1.22.2.

During all this, I did numerous refinements to the codebase, so future library improvements will be easier. This is not a one-shot, I hope that this library will be a staple in the Clojure AI/ML community for the years to come, and that I’ll be able to further expand it and improve it as the requirements expand.

I probably forgot to mentioned some stuff that I worked on, too, but I hope I’ve mentioned the most important things.

Permalink

Explaining Financial Models by Deleting Transactions

Author: Denis Reis

How can you trust a model to support financial decisions if you can’t understand its decisions? For us, this wasn’t just an academic question. It was a core requirement for security, debugging, and responsible deployment. Here, at Nubank, we are leveraging the advent of transformer-based architectures, which have enabled the development of foundation models that automatically discover general features and learn representations directly from raw transaction data. By processing sequences of transactions (often by converting them into tokens like a natural language model would), these systems can efficiently summarize complex financial behavior.

In this post, we explore the explainability of such models, that is, the ability to examine how a single transaction, and its individual properties, influences the final prediction produced by a model. In this new scenario, where we have a transformer that processes a sequence of transactions, standard tools come with some limitations and requirements that make them difficult to maintain in a rapidly evolving environment like ours. However, we adopt a simpler approach, Leave One Transaction Out (LOTO), which can provide a good assessment of the impact of transactions (and their properties) on the final model prediction, while being easily compatible with virtually any model architecture.

Why Explainability is Crucial for Transaction Foundation Models

Understanding how the input influences the output of these models is paramount for responsible deployment, continuous improvement, and, critically, for monitoring and preventing exploitation.

  • Monitoring Model Behavior and Drift: Explainability allows us to track changes in the relative importance of different transaction types over time. Such changes can signal a shift in how customers interact with their finances, which may require updating our models and decision rules.
  • Debugging and Gaining Insights: It’s essential to understand how the different parts of the input contribute to a model’s output, both for individual predictions (local explainability) and across the entire dataset (global explainability). This insight is invaluable for debugging model anomalies, deepening our understanding of the financial behaviors, and guiding “feature selection” by highlighting which transaction types are most relevant.
  • Exploitability Monitoring and Prevention: This is a core concern. We define exploitability as a model vulnerability that malicious users could leverage to manipulate behavioral assessments, potentially leading to unwarranted benefits or other undesirable outcomes. In the case of these models, a vulnerability could be a transaction that is easily produced by a malicious actor and, when present, makes the model produce an output that leads to an unreasonably more favorable outcome for the actor. By monitoring changes in customer behavior, we can detect active exploits, and by examining how relevant transactions are to the model’s prediction before deploying the model, we can detect potential vulnerabilities.

How to Explain?

When addressing model explainability, a standard approach in both literature and industry is SHAP [1], a powerful framework for local explainability that has several beneficial properties for our use case.

For each data point, SHAP assigns a value, which we’ll henceforth refer to as importance, to each component of the input that represents how much that component is pushing the model’s prediction away from a baseline prediction (usually, the average prediction of a dataset that is provided as “background data”).  For example, if the SHAP value of an attribute is large for a particular data point, that feature is pushing the model’s prediction of that data point upwards. We can move from these local explainability assessments to a global one by aggregating the SHAP values, such as the average absolute value, across several data points.

In our context, our input is a sequence of transactions, which for transformers are represented as a sequence of large embeddings.  In this scenario, when assessing the relevance of transactions as a whole on the model’s prediction, SHAP values of the individual embedding dimensions aren’t relevant. Fortunately, in such cases, SHAP can group these smaller parts into transaction inputs and assign a single SHAP value to each transaction as a whole.

Another very positive aspect of SHAP is that it is model architecture-agnostic: we can compute SHAP values regardless of how the model operates internally. This property is especially relevant for us, since we are exploring several different architectures, including hybrid architectures that combine both neural networks and tree-based models.

Although SHAP satisfies our explainability requirements, unfortunately, it is computationally prohibitive for deep neural network use cases like ours. This has led to the exploration of more efficient, gradient-based approximations, such as Integrated Gradients [2] or  Layer-Wise Relevance Propagation (LRP) [3]. However, while more efficient, we lose some of the properties of SHAP that are valuable to us, which makes adopting these other approaches challenging.

The first challenge is that, unlike SHAP, these methods cannot group components together to obtain a single importance for the entire transaction; we would need to aggregate these individual values in some way. We investigated different aggregation schemes for gradient-based attributions, and the ones we considered to be the best tended to use absolute values. The problem, in this case, is that they end up disregarding directionality (i.e., whether a transaction pushed the prediction up or down).

Moreover, in our particular case, we couldn’t easily separate the importance that a transaction has due to its attributes (value, weekday, etc) from its position in the sequence: the model seemed to assign high importance to the first and last transactions in a sequence, regardless of their other characteristics. We hypothesize this is likely due to how the model “understands”  the sequence, that is, the structure of the input data. This bias is visualized in the figure below, where importance scores are heavily concentrated at both ends of the transaction sequence. Disclaimer: this figure, as well as the other figures displayed in this post, is based on synthesized data that illustrates the behavior observed on our real data.

A last challenge with these gradient-based methods is that they are not architecture-agnostic, imposing specific requirements for the types of models they can work with. First, the models need to be differentiable and provide access to their internal gradients, which was incompatible with some architectures we were examining. Second, some particular methods impose even more restrictive constraints regarding architectures or require intricate, domain-specific configurations. For instance, Layer-Wise Relevance Propagation (LRP) [5] demands specific propagation rules for each type of layer in the model. Even a more general method like Integrated Gradients (IG) introduces a critical implementation challenge: the non-trivial task of selecting a domain-specific baseline input vector that represents a “neutral” input. Keeping these restrictive, model-specific explainability tools in sync with our rapid pace of development was becoming too costly.

Lastly, another architecture-agnostic alternative we considered was LIME (Local Interpretable Model-agnostic Explanations) [4]. However, LIME introduces its own complexities: it requires generating a new dataset of perturbations and fitting a surrogate model for every individual explanation. This adds computational overhead, a layer of approximation, and requires defining a non-trivial perturbation strategy for our transactional data.These challenges led us to a simpler, more direct solution, Leave One Transaction Out (LOTO), that still fully satisfies our needs. It retains architecture-agnosticism, provides meaningful, directional values that associate the impact of transactions to predictions, and provides explainability at both local and global levels.

Leave One Transaction Out (LOTO)

A consistent characteristic across all the model architectures we examined is their ability to handle arbitrary sequences of transactions of varying length. This ability provides a very simple way to measure a transaction’s true contextual marginal impact: we just remove it and see what happens. Leave One Transaction Out (LOTO).

The method is straightforward: remove each transaction from a customer’s sequence one at a time and measure the difference between the new prediction and the original one. This directly reveals the impact of that specific transaction in that specific fixed context. It asks the simple question, “What was the direct contribution of this particular transaction’s presence to the prediction, considering the presence of all other transactions (the context)?” We can refer to this contribution as a LOTO value.

It’s important to clarify what “removing” means in this context. This is not like traditional feature ablation, which would be analogous to altering the feature space (e.g., modeling P(y | a=i, b=j) instead of the original P(y | a=i, b=j, c=k)).

Since our models are designed to process variable-length sets or sequences of transactions, “removing” one is simply a change in the input data for a single prediction, while the model’s architecture remains fixed. Conceptually, it’s like the difference between assessing P(y | … transaction_c=PRESENT) and P(y | … transaction_c=ABSENT). We are just feeding the model a sequence with one less element to see how its output changes.

The main drawback is that LOTO only tests transactions one by one. This means we miss out on interaction effects. For example, a payment to a utility company might be unimportant on its own, but when combined with a large, unexpected cash withdrawal, the two together might have a big impact. LOTO can’t capture this combined effect (what we call interaction effects). We accepted this trade-off for the vast gains in simplicity and computational efficiency, reserving more complex interaction analyses for future work.

We apply LOTO in two distinct modes: a global analysis to understand general trends by randomly choosing and removing one transaction from each customer in a large set, and a local analysis for deep dive debugging by removing every transaction, one at a time, from a single customer’s sequence. In either case, each removed transaction creates a new variant of the original input that the model must process. In the former case, since we remove only one transaction from each data point, the total number of variants is the same as the number of data points in the dataset. With hundreds of thousands to millions of these data points, we can build robust aggregations or even train simple meta-models (like a decision tree) on the LOTO results to automatically discover rules like, “Transactions of type X over amount Y consistently have a negative impact”. 

For example, as illustrated in the figure below, we could identify that the type of transaction has a clear bias on the impact of a certain binary target. Once again, the figure is based on synthesized data for illustrative purposes.

For a deep, local analysis of a single customer, we remove each of their transactions one by one, generating hundreds of variants for each customer. This is substantially more computationally expensive than gradient-based techniques, which generally require running the model only once. For this reason, we reserve local analyses for specific deep dives. However, in this regard, LOTO provides a clearer picture of a transaction’s importance, free from the positional bias that can mislead gradient-based methods. By directly measuring the impact of removing a transaction, LOTO helps distinguish whether a transaction is important because of its content or simply because of its location in the sequence. As the figure below illustrates, LOTO reveals a much more even distribution of high-impact transactions, helping us accurately diagnose the model’s behavior. 

Note that, in the figure above, it is still visible that more recent transactions are frequently more impactful. Another way of visualizing this in the global analysis is by simply checking the correlation between the age of a transaction and its impact on the prediction, as shown below.

Conclusion: Simplicity as a Feature

In our journey to build a reliable and interpretable financial AI system, we found that the simplest explainability method was also the most effective. By directly measuring the impact of removing each transaction, LOTO provides clear, actionable, and unbiased insights that are robust to our evolving model architecture.

References

[1] Lundberg, S. M., & Lee, S. I. (2017). A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems, 30. arXiv. https://arxiv.org/abs/1705.07874 

[2] Sundararajan, M., Taly, A., & Yan, Q. (2017). Axiomatic Attribution for Deep Networks. Proceedings of the 34th International Conference on Machine Learning, 70, 3319-3328. arXiv. https://arxiv.org/abs/1703.01365 

[3] Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K. R., & Samek, W. (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE, 10(7), e0130140. https://doi.org/10.1371/journal.pone.0130140 [4] Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135–1144. https://arxiv.org/abs/1602.04938

The post Explaining Financial Models by Deleting Transactions appeared first on Building Nubank.

Permalink

Call for Applications: 2026 Annual Funding

Greetings folks!

Thanks to our members' generous support, this is the 5th year we we will be awarding annual funding to 5 developers - paid in twelve $1,500 monthly stipends (for a total of $90,000 USD). In the past 4 years, we have seen that giving developers flexible, long-term funding gives them the space to do high-impact work. This might be continuing maintenance on existing projects, new feature development, or perhaps a brand-new project.

We’ve been excited with what they came up with in the last few years and are looking forward to seeing more great work in 2026! Thanks all and good luck!

PROCESS

Apply: Anyone interested in receiving annual funding submits the application outlining what they intend to work on and how that work will benefit the Clojure community. The deadline for applications is Nov. 25th, 2025 midnight Pacific Time. Please note that past grantees must re-apply each year.

Board Review: The Clojurists Together board will review the applications and select finalists to present to the members by Dec. 3rd.

Members Vote: The final ballot will go out to members using a Ranked Vote election to determine the final recipients. As always, your input and participation is important and helps make Clojurists Together effective by ensuring members’ voices inform the work undertaken. Deadline Dec. 12, 2025 midnight Pacific time.

Awards will be announced no later than Dec. 18, 2025.

Project Updates: Recipients are required to submit a report bi-monthly to the membership.

A special call-out to Latacora, Roam Research, Whimsical, Nubank, Cisco, JUXT, Metosin, Solita, Adgoji, Grammarly, Nextjournal, ClojureStream, Shortcut, Flexiana, Toyokumo, doctronic, 180° Seguros, bevuta IT GmbH, Jepsen, Xcoo, Sharetribe, Basil, Cognician, Biotz SL, Matrix Operations, Strategic Blue. Eric Normand, Nubizzi, Oiiku, and Singlewire Software. They have all contributed significant amounts to Clojurists Together which lets us award approximately $90,000 in long-term funding to Clojure developers.

Permalink

Clojure in your browser

There is a recent article on Clojure Civitas on using Scittle for browser native slides. Scittle is a Clojure interpreter that runs in the browser. It even defines a script tag that let’s you embed Clojure code in your HTML code. Here is an example evaluating the content of an HTML textarea:

HTML code

<script src="https://cdn.jsdelivr.net/npm/scittle@0.6.22/dist/scittle.js"></script>
<script type="application/x-scittle">
(defn run []
  (let [code (.-value (js/document.getElementById "code"))
        output-elem (js/document.getElementById "output")]
    (try
      (let [result (js/scittle.core.eval_string code)]
        (set! (.-textContent output-elem) (str result)))
      (catch :default e
        (set! (.-textContent output-elem)
              (str "Error: " (.-message e)))))))

(set! (.-run js/window) run)
</script>
<textarea id="code" rows="20" style="width:100%;">
(defn primes [i p]
  (if (some #(zero? (mod i %)) p)
    (recur (inc i) p)
    (cons i (lazy-seq (primes (inc i) (conj p i))))))

(take 100 (primes 2 []))
</textarea>
<br />
<button id="run-button" onclick="run()">Run</button>
<pre id="output"></pre>

Scittle in your browser


              

Permalink

Copyright © 2009, Planet Clojure. No rights reserved.
Planet Clojure is maintained by Baishamapayan Ghose.
Clojure and the Clojure logo are Copyright © 2008-2009, Rich Hickey.
Theme by Brajeshwar.