Free Yourself From the Tyranny of Null (Part 1): Why We’ve Banned Null in our Codebase

Posted By Douglas Rapp on Thursday, January 3rd, 2013

Here at StackMob we daily face the challenge of building powerful, reliable, and scalable services that will power countless mobile applications worldwide. We accomplish this with a variety of technologies and tools, but one thing that surprised me when I joined is that there’s an effective ban on using null in the codebase. I’ve come to realize that this is one of the most important things we do maintainability-wise, and the lessons learned could be of use to anyone working with complex systems.

So what’s wrong with null exactly? The inventor of null famously called it a Billion Dollar Mistake. The problem is that any object you try to access can just not be there! You probably take this as a given because you learned about programming that way, but if you stop to think about it, this is absolutely terrifying. The only thing that indicates what can be null is the code itself and, if you’re very lucky, comments that are actually up-to-date. Let’s illustrate this with an example. I’ll work in Java to start with since that’s what people are familiar with, but as you’ll see we’ll soon have reason to switch to Scala.

What makes the null problem so insidious is that when you’re working on a small project by yourself, it’s perfectly fine. You remember for the most part what needs to be null-checked and what doesn’t, and the problem of tracking down Null Pointer Exceptions (NPEs) is manageable. But then your project gets bigger: the codebase expands, the team grows, or simply time passes. Suddenly the code you’re looking at may have been written by some idiot (possibly you from a few months ago) who’s done something silly, and you have no idea what can be safely dereferenced. Best case scenario is you waste time looking through some code and comments to figure everything out. In the worst case you blithely don’t realize anything’s wrong and inject a subtle NPE that will crop up at the worst possible time. Changing code becomes dangerous: say you need to be able to pass null to a method that’s never accepted null before. Can you? What will break? There’s no way to tell without delving into every subroutine called by your method?

Additionally, you’ll note that we return -1 when the header can’t be found for some reason. Since primitive types can’t be null in most languages, you’re stuck with something like this where you pick a value from outside the expected range to indicate a failure, and the caller has remember to handle it specially. This may be even scarier than null. In Java we can wrap it up in an Integer, which can be null, which is marginally better but still has all the problems of null.

This sounds like exactly the sort of problem that static typing is made for. If values that can be null (let’s call them “nullable”) and reliable values have different types, you always know what you’re working with. If you need to make something nullable, simple make the change in type and the compiler will furnish you with a handy list of what else needs to change. This may sound annoying, but every time you’re forced to think about how to handle a null case, you’re fixing a NPE before it ever happens.

So why is null still so popular? Let’s look at what the alternative would be in Java. The logical thing to do would be to use generics to create a boxed type that represents nullability. Let’s call it Option

This is all we need right here. Let’s see what happens when we replace all the nullable types in our example with our new Option

We’ve now, strictly speaking, solved our problem; so long as we stick to this pattern, bare values are guaranteed to be non-null enforced by the type system. However, making the Option datatype was the easy part. The real challenge is making the pattern easy and natural to use, otherwise you’ll just get frustrated and go back to nulls.

The first problem you’ll see is type signature bloat. Replacing T with Option<T> everywhere just looks bad, and is a pain to type and keep track of. Nested datatypes that were a bit ungainly before, can become monstrous once Option is inserted at various levels. Some of this is inevitable, but much of it can be alleviated with thoughtful program design. Having nullable types presented explicitly as ‘Option’ forces you to think about what’s optional and why. In the little example we’ve been working with, why is the request parameter optional? Should there be a default value passed in instead? Is this in fact the wrong place to split off the method? Maybe request shouldn’t be an Option and we should force the caller to handle the case where there isn’t a request. This is a good exercise to do with any code, even if you switch back to nulls afterwards: I guarantee you’ll find some potentials NPEs.

The second problem is actually working with these new Option types. In my example, you’ll notice the pattern

If you’ve been paying attention, you may have noticed that calling the get() method is a lot like dereferencing a possibly null pointer, and isDefined() is a lot like a null check. Having the type in place is a good reminder to do the check, but it isn’t foolproof, and it’s still ugly with nested if statements.

Current versions of Java are simply not equipped to deal with these sorts of problems, and I honestly can’t blame anyone who gives up and goes back to null at this point. Null-free programming needs to be a delight if it’s going to succeed, and Scala has just the right tools to make that happen. In part two of this blog post we’ll see how using Option in Scala can not only be fun to write, but can even produce code that’s cleaner and easier to understand than the Java equivalent.

Read part two now.