Adam Victor Brandizzi

There’s an old Soviet saying:

If you think it, don’t say it.
If you say it, don’t write it.
If you write it, don’t be surprised.

It’s not a pleasant way to live.  The coiner of this quote was not celebrating his oppression.  The power of the Western brand has long been associated with its outright rejection of this sort of thought policing.

In the wake of a truly profound compromise of sensitive photographs of celebrities, those of us in Information Security find ourselves called upon to answer what this all means – to average citizens, to celebrities, to a global economy that has found itself transformed in ways not necessarily planned.  Let’s talk about some things we don’t normally discuss.

Victim Shaming Goes Exponential

Hey @itsjenIawrence. Maybe you shouldn't pose nude if you can't handle the public seeing it. #dumdum. And don't step on downed power lines!

—
Larry WACHS (@houseofwachs) September 01, 2014

Dumdum?  Really?

We shouldn’t entirely be surprised.  Victim shaming is par for the course in Infosec, and more than Infosec, for uncomfortably similar reasons.  When social norms are violated, a society that cannot punish the transgressor will punish the victim.  If the victim is not made a monster, their unpunished victimization today could become our unpunished victimization tomorrow.  And that’s such a dark and scary conclusion that it’s really quite tempting to say –

No, it’s OK.  Only these celebrities got hacked, not me, because they were so stupid they took sexy photos.  It attracted the hackers.

As if the hackers knew there had to be such photos in the first place, and only stole the photos.  As if we don’t all have private lives, with sensitive bits, that could be or already have been acquired by parties unknown.  We’ve all got something to lose.  And frankly, it may already be lost.

You Don’t Necessarily Know When You’ve Been Hit, Let Alone What’s Gone

There’s a peculiar property of much criminality in the real world:  You notice.  A burgled home is missing things, an assaulted body hurts.  These crimes still occur, but we can start responding to them immediately.  If there’s one thing to take away from this compromise, it’s that when it comes to information theft you might find out quickly, or you may never find out at all.  Consider this anonymous post, forwarded by @OneTrueDoxbin to Daniel Wolf (better known as the surprisingly insightful @SwiftOnSecurity):

It is a matter of undisputable fact that “darknet” information trading networks exist.  People collected stamps, after all, and there’s way rarer stuff floating around out there than postal artifacts.  This sort of very personal imagery is the tip of a very large and occasionally deeply creepy iceberg.  The most interesting aspects of Daniel Wolf’s research have centered on the exceptions – yes, he found, a significant majority of the files came from iPhones, implicating some aspect of the Apple iCloud infrastructure.  But not all – there’s some JVC camcorder data here, a Motorola RAZR EXIF extension there – and there’s a directory structure that no structured database might have but a disorganized human whose photo count doesn’t number in the billions would absolutely use.  The exceptions show more of a network and less of a lone operator.

The key element of a darknet is, of course, staying dark.  It’s hard to do that if you’re taunting your victims, and so generally they don’t.  Some of the images Daniel found in his research went back years.  A corollary of not discovering one attack is not detecting many, extending over many victims and coming from multiple attackers.

Of course, darknets have operational security risks, same as anyone, and eventually someone might come in to game the gamers.  From someone who claims to be the original leaker:

“People wanted shit for free. Sure, I got $120 with my bitcoin address, but when you consider how much time was put into acquiring this stuff (i’m not the hacker, just a collector), and the money (I paid a lot via bitcoin as well to get certain sets when this stuff was being privately traded Friday/Saturday) I really didn’t get close to what I was hoping.

Real?  Fake?  Can’t really know.  Pretty risky, trying to draw together a narrative less than a hundred hours since the initial compromise was detected.  It’s the Internet, people lie, even more so anonymously.  It fits with my personal theory that the person who acquired these images isn’t necessarily the person who’s distributing them (I figured hacker-on-hacker theft), but heh.  It’s amazingly easy to have your suspicions confirmed.

One reporter asked me how it was possible that J.P. Morgan could immediately diagnose and correct their extended infection, while Apple couldn’t instantaneously provide similar answers.  As I told them, J.P. Morgan knew without question they were hit, and had the luxury of deciding its disclosure schedule (with some constraints); this particular case simply showed up on 4Chan during Labor Day Weekend when presumably half the people who would investigate were digging their way back from Burning Man.  Internal discoveries and external surprises just follow different schedules.

I’ve personally been skeptical that an account brute forcing bug that happened to be discovered around now, was also used for this particular attack.  There’s only so many days in the year and sometimes multiple events happen close in time just randomly.  As it happens, Apple has confirmed at least some of these celebrity raids have come via account attacks, but whether brute forcing was required hasn’t been disclosed.  It does seem that this exploit has been used in the field since at least May, however, lending some credibility.

We have, at best, a partial explanation.  Much as we desperately would like this to be a single, isolated event, with a nice, well defined and well funded defender who can make sure this never happens again – that’s just not likely to be the case.  We’re going to learn a lot more about how this happened, and in response, there will be improvements.  But celebrity (and otherwise) photo exploitation will not be found to be an isolated attack and it won’t be addressed or ended with a spot fix to password brute forcing.

So there’s a long investigation ahead, quite a bit longer than a single press cycle.

Implications For Cloud Providers

Are we actually stuck right now at another password debate?  Passwords have failed us yet again, let’s have that tired conversation once more?  Sam Biddle, who otherwise did some pretty good research in this post, did have one somewhat amusing paragraph:

To fix this, Apple could have simply forced everyone to use two-factor verification for their accounts. It’s easy, and would have probably prevented all of this.

Probably the only time “simply”, “easy”, and “two-factor verification” have ever been seen in quite such proximity, outside of marketing materials anyway.  There’s a reason we use that broken old disco tech.

Still, we have to do better.  So-called “online brute-forcing” – where you don’t have a database of passwords you want to crack, but instead have to interact with a server that does – is a far slower, and far noisier affair.

But noise doesn’t matter if nobody is listening.  Authentication systems could probably do more to detect brute force attacks across large numbers of accounts.  And given the wide variety of systems that interface with backend password stores, it’s foolish to expect them all to implement rate limiting correctly.  Limits need to exist as close as possible to the actual store, independent of access method.

Sam’s particularly right about the need to get past personal entropy.  Security questions are time bombs in a way even passwords aren’t.  In a world of tradeoffs, I’m beginning to wonder if voice prints across sentences aren’t better than personal information widely shared.  Yes, I’m aware of the downsides, but look at the competition.

OK.  It’s time to ban Password1.  Many users like predictable passwords.  Few users like their data being compromised.  Which wins?  Presently, the former.  Perhaps this is the moment to shift that balance.  Service providers (cloud and otherwise) are burying their heads in the sand and going with password policies that can only be called inertial.   Defenders are using simple rules like “doesn’t have an uppercase letter” and “not enough punctuation” to block passwords while attackers are just straight up analyzing password dumps and figuring out the most likely passwords to attempt in any scenario.  Attackers are just way ahead.  That has to change.  Defenders have password dumps too now.  It’s time we start outright blocking passwords common enough that they can be online brute forced, and it’s time we admit we know what they are.

We’re not quite ready to start generating passwords for users, and post-password initiatives like Fido are still some of the hardest things we’re working on in all of computer engineering.  But there’s some low hanging fruit, and it’s probably time to start addressing it.

And we can’t avoid the elephant in the room any longer.

In Which The Nerd Has To Talk About Sex Because Everyone Else Won’t

It’s not all victim shaming. At least some of the reaction to this leak of celebrity nudity can only be described as bewilderment.  Why are people taking these photos in the first place?  Even with the occasional lack of judgment… there’s a sense of surprise.  Is everybody taking and sending sexy photos?

No.  Just everyone who went through puberty owning a cell phone.

I’m joking, of course.  There are also a number of people who grew up before cell phones who nonetheless discovered that a technology able to move audio, still images, and videos across the world in an instant could be a potent enabler of Flirting-At-A-Distance.  This tends to reduce distance, increasing…happiness.

Every generation thinks it invents sex, but to a remarkable degree generations don’t talk to each other about what they’ve created.  It’s rarely the same, and though older generations can (and do) try, there is nothing in all of creation humans take less advice about than mating rituals.

So, yeah.  People use communication technologies for sexy times.  Deal with it.

Interestingly, to a very limited extent, web browsers actually do.  You may have noticed that each browser has Porn Mode.  Oh, sure, that particular name never makes it through Corporate Branding.  It gets renamed “InPrivate Browsing” or “Incognito Mode” or “The I’m Busy I’ll Be Out In A Minute Window”.  The actual feature descriptions are generally hilarious parallel constructions about wanting to use a friend’s web browser to buy them a gift, but not having the nature of the gift show up in their browser cache.  But we know the score and so do browser developers, who know the market share they’d lose if they didn’t support safer consumption of pornography (at least in the sense that certain sites don’t show up on highly visible “popular tabs” pages during important presentations).

I say all this because of a tweet that is accurate, and needs to be understood outside the context of shaming the victim:

The risk to copy naked pics to iCloud, esp. as a celebrity, may just be too high to be acceptable. And saying that is not "blaming women".

—
JP Aumasson (@veorq) September 02, 2014

Technology can do great, wonderful, and repeatedly demanded things, and still have a dark side.  That’s not limited to sexy comms.  That applies to the cloud itself.

True story:  A friend of mine and I are at the airport in Abu Dhabi a few years back.  We get out of the taxi, she drops her cell phone straight into a twenty foot deep storm drain.  She starts panicking:  She can’t leave her phone.  She can’t lose her phone.  She’s got pictures of her kids on that phone that she just can’t lose.  “No problem, I get police” says the taxi driver, who proceeds to drive off, with all our stuff.

We’re ten thousand miles away from home, and our flight’s coming.  Five minutes go by. Ten.  Fifteen…and the taxi driver returns, with a police officer, who rounds up some airport workers who agree to help us retrieve the phone (which, despite being submerged for twenty minutes, continued to work just fine).

Probably the best customer service I’ve ever received while traveling, but let me tell you, I’d have rather just told my friend to pull the photos off the cloud.

The reality is that cell phone designers have heard for years what a painful experience it is to lose data, and have prioritized the seamless recovery of those bits best they can. It’s awful to lose your photos, your emails, your contacts.  No, really, major life disruption.  Lot of demand to fix that.  But in all things there are engineering tradeoffs, and data that is stored in more than one location can be stolen from more than one location.  98% of the time, that’s OK, you really don’t want to lose photos of your loved ones.  You don’t care if the pics are stolen, you’re just going to post them on Facebook anyway.

2% of the time, those pictures weren’t for Facebook.  2% of the time, no it’s cool, those can go away, you can always take more selfies.  Way better to lose the photos than see them all over the Internet.  Terrible choice to have to make, but not generally a hard decision.

So the game becomes, separate the 98% from the 2%.

So, actual concrete advice.  Just like browsers have porn mode for the personal consumption of private imagery, cell phones have applications that are significantly less likely to lead to anyone else but your special friends seeing your special bits.  I personally advise Wickr, an instant messaging firm that develops secure software for iPhone and Android.  What’s important about Wickr here isn’t just the deep crypto they’ve implemented, though it’s useful too.  What’s important in this context is that with this code there’s just a lot fewer places to steal your data from.  Photos and other content sent in Wickr don’t get backed up to your desktop, don’t get saved in any cloud, and by default get removed from your friend’s phone after an amount of time you control.  Wickr is of course not the only company supporting what’s called “ephemeral messaging”; SnapChat also dramatically reduces the exposure of your private imagery (with the caveat that with SnapChat, unlike Wickr, SnapChat itself gets an unencrypted copy of your imagery and messaging so you have to hope they’re not saving anything.  Better for national intelligence services, worse for you).

Sure, you can hunt down settings that reduce your exposure to specific cloud services.  You’ve still got to worry about desktop backups, and whatever your desktop is backing up to.  But really, if you can keep your 2% far away from your 98%, you’ll be better off.

In the long run, I think the standard photo applications will get a UI refresh, to allow “sensitive photographs” (presumably of surprise birthday parties in planning) to be locked to the device, allowed through SMS only in directly authenticated circumstances, and not backed up.  Something like a lock icon on the camera screen.  I don’t think the feature request could have happened before this huge leak.  But maybe now it’s obvious just how many people require it.

It sounds like a question that only a linguist could love. But neuroscientists have been trying to answer it using exotic brain-scanning technologies. Their findings have varied wildly, in some cases contradicting one another. If they make progress, the payoff will be big. Their findings will enrich a theory that aims to explain how wet masses of neurons can understand anything at all. And they may drive a stake into the widespread assumption that computers will inevitably become conscious in a humanlike way.

The hypothesis driving their work is that metaphor is central to language. Metaphor used to be thought of as merely poetic ornamentation, aesthetically pretty but otherwise irrelevant. "Love is a rose, but you better not pick it," sang Neil Young in 1977, riffing on the timeworn comparison between a sexual partner and a pollinating perennial. For centuries, metaphor was just the place where poets went to show off.

But in their 1980 book, Metaphors We Live By, the linguist George Lakoff (at the University of California at Berkeley) and the philosopher Mark Johnson (now at the University of Oregon) revolutionized linguistics by showing that metaphor is actually a fundamental constituent of language. For example, they showed that in the seemingly literal statement "He’s out of sight," the visual field is metaphorized as a container that holds things. The visual field isn’t really a container, of course; one simply sees objects or not. But the container metaphor is so ubiquitous that it wasn’t even recognized as a metaphor until Lakoff and Johnson pointed it out.

From such examples they argued that ordinary language is saturated with metaphors. Our eyes point to where we’re going, so we tend to speak of future time as being "ahead" of us. When things increase, they tend to go up relative to us, so we tend to speak of stocks "rising" instead of getting more expensive. "Our ordinary conceptual system is fundamentally metaphorical in nature," they wrote.

Metaphors do differ across languages, but that doesn’t affect the theory. For example, in Aymara, spoken in Bolivia and Chile, speakers refer to past experiences as being in front of them, on the theory that past events are "visible" and future ones are not. However, the difference between behind and ahead is relatively unimportant compared with the central fact that space is being used as a metaphor for time. Lakoff argues that it is impossible—not just difficult, but impossible—for humans to talk about time and many other fundamental aspects of life without using metaphors to do it.

Lakoff and Johnson’s program is as anti-Platonic as it’s possible to get. It undermines the argument that human minds can reveal transcendent truths about reality in transparent language. They argue instead that human cognition is embodied—that human concepts are shaped by the physical features of human brains and bodies. "Our physiology provides the concepts for our philosophy," Lakoff wrote in his introduction to Benjamin Bergen’s 2012 book, Louder Than Words: The New Science of How the Mind Makes Meaning. Marianna Bolognesi, a linguist at the International Center for Intercultural Exchange, in Siena, Italy, puts it this way: "The classical view of cognition is that language is an independent system made with abstract symbols that work independently from our bodies. This view has been challenged by the embodied account of cognition which states that language is tightly connected to our experience. Our bodily experience."

Modern brain-scanning technologies make it possible to test such claims empirically. "That would make a connection between the biology of our bodies on the one hand, and thinking and meaning on the other hand," says Gerard Steen, a professor of linguistics at VU University Amsterdam. Neuroscientists have been stuffing volunteers into fMRI scanners and having them read sentences that are literal, metaphorical, and idiomatic.

Neuroscientists agree on what happens with literal sentences like "The player kicked the ball." The brain reacts as if it were carrying out the described actions. This is called "simulation." Take the sentence "Harry picked up the glass." "If you can’t imagine picking up a glass or seeing someone picking up a glass," Lakoff wrote in a paper with Vittorio Gallese, a professor of human physiology at the University of Parma, in Italy, "then you can’t understand that sentence." Lakoff argues that the brain understands sentences not just by analyzing syntax and looking up neural dictionaries, but also by igniting its memories of kicking and picking up.

But what about metaphorical sentences like "The patient kicked the habit"? An addiction can’t literally be struck with a foot. Does the brain simulate the action of kicking anyway? Or does it somehow automatically substitute a more literal verb, such as "stopped"? This is where functional MRI can help, because it can watch to see if the brain’s motor cortex lights up in areas related to the leg and foot.

The evidence says it does. "When you read action-related metaphors," says Valentina Cuccio, a philosophy postdoc at the University of Palermo, in Italy, "you have activation of the motor area of the brain." In a 2011 paper in the Journal of Cognitive Neuroscience, Rutvik Desai, an associate professor of psychology at the University of South Carolina, and his colleagues presented fMRI evidence that brains do in fact simulate metaphorical sentences that use action verbs. When reading both literal and metaphorical sentences, their subjects’ brains activated areas associated with control of action. "The understanding of sensory-motor metaphors is not abstracted away from their sensory-motor origins," the researchers concluded.

Textural metaphors, too, appear to be simulated. That is, the brain processes "She’s had a rough time" by simulating the sensation of touching something rough. Krish Sathian, a professor of neurology, rehabilitation medicine, and psychology at Emory University, says, "For textural metaphor, you would predict on the Lakoff and Johnson account that it would recruit activity- and texture-selective somatosensory cortex, and that indeed is exactly what we found."

But idioms are a major sticking point. Idioms are usually thought of as dead metaphors, that is, as metaphors that are so familiar that they have become clichés. What does the brain do with "The villain kicked the bucket" ("The villain died")? What about "The students toed the line" ("The students conformed to the rules")? Does the brain simulate the verb phrases, or does it treat them as frozen blocks of abstract language? And if it simulates them, what actions does it imagine? If the brain understands language by simulating it, then it should do so even when sentences are not literal.

The findings so far have been contradictory. Lisa Aziz-Zadeh, of the University of Southern California, and her colleagues reported in 2006 that idioms such as "biting off more than you can chew" did not activate the motor cortex. So did Ana Raposo, then at the University of Cambridge, and her colleagues in 2009. On the other hand, Véronique Boulenger, of the Laboratoire Dynamique du Langage, in Lyon, France, reported in the same year that they did, at least for leg and arm verbs.

In 2013, Desai and his colleagues tried to settle the problem of idioms. They first hypothesized that the inconsistent results come from differences of methodology. "Imaging studies of embodiment in figurative language have not compared idioms and metaphors," they wrote in a report. "Some have mixed idioms and metaphors together, and in some cases, ‘idiom’ is used to refer to familiar metaphors." Lera Boroditsky, an associate professor of psychology at the University of California at San Diego, agrees. "The field is new. The methods need to stabilize," she says. "There are many different kinds of figurative language, and they may be importantly different from one another."

Not only that, the nitty-gritty differences of procedure may be important. "All of these studies are carried out with different kinds of linguistic stimuli with different procedures," Cuccio says. "So, for example, sometimes you have an experiment in which the person can read the full sentence on the screen. There are other experiments in which participants read the sentence just word by word, and this makes a difference."

To try to clear things up, Desai and his colleagues presented subjects inside fMRI machines with an assorted set of metaphors and idioms. They concluded that in a sense, everyone was right. The more idiomatic the metaphor was, the less the motor system got involved: "When metaphors are very highly conventionalized, as is the case for idioms, engagement of sensory-motor systems is minimized or very brief."

But George Lakoff thinks the problem of idioms can’t be settled so easily. The people who do fMRI studies are fine neuroscientists but not linguists, he says. "They don’t even know what the problem is most of the time. The people doing the experiments don’t know the linguistics."

That is to say, Lakoff explains, their papers assume that every brain processes a given idiom the same way. Not true. Take "kick the bucket." Lakoff offers a theory of what it means using a scene from Young Frankenstein. "Mel Brooks is there and they’ve got the patient dying," he says. "The bucket is a slop bucket at the edge of the bed, and as he dies, his foot goes out in rigor mortis and the slop bucket goes over and they all hold their nose. OK. But what’s interesting about this is that the bucket starts upright and it goes down. It winds up empty. This is a metaphor—that you’re full of life, and life is a fluid. You kick the bucket, and it goes over."

That’s a useful explanation of a rather obscure idiom. But it turns out that when linguists ask people what they think the metaphor means, they get different answers. "You say, ‘Do you have a mental image? Where is the bucket before it’s kicked?’ " Lakoff says. "Some people say it’s upright. Some people say upside down. Some people say you’re standing on it. Some people have nothing. You know! There isn’t a systematic connection across people for this. And if you’re averaging across subjects, you’re probably not going to get anything."

Similarly, Lakoff says, when linguists ask people to write down the idiom "toe the line," half of them write "tow the line." That yields a different mental simulation. And different mental simulations will activate different areas of the motor cortex—in this case, scrunching feet up to a line versus using arms to tow something heavy. Therefore, fMRI results could show different parts of different subjects’ motor cortexes lighting up to process "toe the line." In that case, averaging subjects together would be misleading.

Furthermore, Lakoff questions whether functional MRI can really see what’s going on with language at the neural level. "How many neurons are there in one pixel or one voxel?" he says. "About 125,000. They’re one point in the picture." MRI lacks the necessary temporal resolution, too. "What is the time course of that fMRI? It could be between one and five seconds. What is the time course of the firing of the neurons? A thousand times faster. So basically, you don’t know what’s going on inside of that voxel." What it comes down to is that language is a wretchedly complex thing and our tools aren’t yet up to the job.

Nonetheless, the work supports a radically new conception of how a bunch of pulsing cells can understand anything at all. In a 2012 paper, Lakoff offered an account of how metaphors arise out of the physiology of neural firing, based on the work of a student of his, Srini Narayanan, who is now a faculty member at Berkeley. As children grow up, they are repeatedly exposed to basic experiences such as temperature and affection simultaneously when, for example, they are cuddled. The neural structures that record temperature and affection are repeatedly co-activated, leading to an increasingly strong neural linkage between them.

However, since the brain is always computing temperature but not always computing affection, the relationship between those neural structures is asymmetric. When they form a linkage, Lakoff says, "the one that spikes first and most regularly is going to get strengthened in its direction, and the other one is going to get weakened." Lakoff thinks the asymmetry gives rise to a metaphor: Affection is Warmth. Because of the neural asymmetry, it doesn’t go the other way around: Warmth is not Affection. Feeling warm during a 100-degree day, for example, does not make one feel loved. The metaphor originates from the asymmetry of the neural firing. Lakoff is now working on a book on the neural theory of metaphor.

If cognition is embodied, that raises problems for artificial intelligence. Since computers don’t have bodies, let alone sensations, what are the implications of these findings for their becoming conscious—that is, achieving strong AI? Lakoff is uncompromising: "It kills it." Of Ray Kurzweil’s singularity thesis, he says, "I don’t believe it for a second." Computers can run models of neural processes, he says, but absent bodily experience, those models will never actually be conscious.

On the other hand, roboticists such as Rodney Brooks, an emeritus professor at the Massachusetts Institute of Technology, have suggested that computers could be provided with bodies. For example, they could be given control of robots stuffed with sensors and actuators. Brooks pondered Lakoff’s ideas in his 2002 book, Flesh and Machines, and supposed, "For anything to develop the same sorts of conceptual understanding of the world as we do, it will have to develop the same sorts of metaphors, rooted in a body, that we humans do."

But Lera Boroditsky wonders if giving computers humanlike bodies would only reproduce human limitations. "If you’re not bound by limitations of memory, if you’re not bound by limitations of physical presence, I think you could build a very different kind of intelligence system," she says. "I don’t know why we have to replicate our physical limitations in other systems."

What’s emerging from these studies isn’t just a theory of language or of metaphor. It’s a nascent theory of consciousness. Any algorithmic system faces the problem of bootstrapping itself from computing to knowing, from bit-shuffling to caring. Igniting previously stored memories of bodily experiences seems to be one way of getting there. And so may be the ability to create asymmetric neural linkages that say this is like (but not identical to) that. In an age of brain scanning as well as poetry, that’s where metaphor gets you.

Michael Chorost is the author of Rebuilt: How Becoming Part Computer Made Me More Human (Houghton Mifflin, 2005) and World Wide Mind: The Coming Integration of Humanity, Machines, and the Internet (Free Press, 2011).

Growing up, I learned there were two kinds of reviews I could seek out from my parents. One parent gave reviews in the form of a shower of praise. The other parent, the one with a degree from the Royal College of Art, would put me through a design crit. Today the reviews I seek are for my code, not my horse drawings, but it continues to be a process I both dread and crave.

Illustration:

Share This:

In this article, I’ll describe my battle-tested process for conducting code reviews, highlighting the questions you should ask during the review process as well as the necessary version control commands to download and review someone’s work. I’ll assume your team uses Git to store its code, but the process works much the same if you’re using any other source control system.

Completing a peer review is time-consuming. In the last project where I introduced mandatory peer reviews, the senior developer and I estimated that it doubled the time to complete each ticket. The reviews introduced more context-switching for the developers, and were a source of increased frustration when it came to keeping the branches up to date while waiting for a code review.

The benefits, however, were huge. Coders gained a greater understanding of the whole project through their reviews, reducing silos and making onboarding easier for new people. Senior developers had better opportunities to ask why decisions were being made in the codebase that could potentially affect future work. And by adopting an ongoing peer review process, we reduced the amount of time needed for human quality assurance testing at the end of each sprint.

Let’s walk through the process. Our first step is to figure out exactly what we’re looking for.

Determine the purpose of the proposed change

Our code review should always begin in a ticketing system, such as Jira or GitHub. It doesn’t matter if the proposed change is a new feature, a bug fix, a security fix, or a typo: every change should start with a description of why the change is necessary, and what the desired outcome will be once the change has been applied. This allows us to accurately assess when the proposed change is complete.

The ticketing system is where you’ll track the discussion about the changes that need to be made after reviewing the proposed work. From the ticketing system, you’ll determine which branch contains the proposed code. Let’s pretend the ticket we’re reviewing today is 61524—it was created to fix a broken link in our website. It could just as equally be a refactoring, or a new feature, but I’ve chosen a bug fix for the example. No matter what the nature of the proposed change is, having each ticket correspond to only one branch in the repository will make it easier to review, and close, tickets.

Set up your local environment and ensure that you can reproduce what is currently the live site—complete with the broken link that needs fixing. When you apply the new code locally, you want to catch any regressions or problems it might introduce. You can only do this if you know, for sure, the difference between what is old and what is new.

Review the proposed changes

At this point you’re ready to dive into the code. I’m going to assume you’re working with Git repositories, on a branch-per-issue setup, and that the proposed change is part of a remote team repository. Working directly from the command line is a good universal approach, and allows me to create copy-paste instructions for teams regardless of platform.

To begin, update your local list of branches.

git fetch

Then list all available branches.

git branch -a

A list of branches will be displayed to your terminal window. It may appear something like this:

* master
remotes/origin/master
remotes/origin/HEAD -> origin/master
remotes/origin/61524-broken-link

The * denotes the name of the branch you are currently viewing (or have “checked out”). Lines beginning with remotes/origin are references to branches we’ve downloaded. We are going to work with a new, local copy of branch 61524-broken-link.

When you clone your project, you’ll have a connection to the remote repository as a whole, but you won’t have a read-write relationship with each of the individual branches in the remote repository. You’ll make an explicit connection as you switch to the branch. This means if you need to run the command git push to upload your changes, Git will know which remote repository you want to publish your changes to.

git checkout --track origin/61524-broken-link

Ta-da! You now have your own copy of the branch for ticket 61524, which is connected (“tracked”) to the origin copy in the remote repository. You can now begin your review!

First, let’s take a look at the commit history for this branch with the command log.

git log master

Sample output:

Author: emmajane 
Date: Mon Jun 30 17:23:09 2014 -0400

Link to resources page was incorrectly spelled. Fixed.

Resolves #61524. 

This gives you the full log message of all the commits that are in the branch 61524-broken-link, but are not also in the master branch. Skim through the messages to get a sense of what’s happening.

Next, take a brief gander through the commit itself using the diff command. This command shows the difference between two snapshots in your repository. You want to compare the code on your checked-out branch to the branch you’ll be merging “to”—which conventionally is the master branch.

git diff master

How to read patch files

When you run the command to output the difference, the information will be presented as a patch file. Patch files are ugly to read. You’re looking for lines beginning with + or -. These are lines that have been added or removed, respectively. Scroll through the changes using the up and down arrows, and press q to quit when you’ve finished reviewing. If you need an even more concise comparison of what’s happened in the patch, consider modifying the diff command to list the changed files, and then look at the changed files one at a time:

git diff master --name-only
git diff master <filename>

Let’s take a look at the format of a patch file.

diff --git a/about.html b/about.html
index a3aa100..a660181 100644
	--- a/about.html
	+++ b/about.html
@@ -48,5 +48,5 @@
	(2004-05)

- A full list of <a href="emmajane.net/events">public 
+ A full list of <a href="http://emmajane.net/events">public 
presentations and workshops</a> Emma has given is available

I tend to skim past the metadata when reading patches and just focus on the lines that start with - or +. This means I start reading at the line immediate following @@. There are a few lines of context provided leading up to the changes. These lines are indented by one space each. The changed lines of code are then displayed with a preceding - (line removed), or + (line added).

Going beyond the command line

Using a Git repository browser, such as gitk, allows you to get a slightly better visual summary of the information we’ve looked at to date. The version of Git that Apple ships with does not include gitk—I used Homebrew to re-install Git and get this utility. Any repository browser will suffice, though, and there are many GUI clients available on the Git website.

gitk

When you run the command gitk, a graphical tool will launch from the command line. An example of the output is given in the following screenshot. Click on each of the commits to get more information about it. Many ticket systems will also allow you to look at the changes in a merge proposal side-by-side, so if you’re finding this cumbersome, click around in your ticketing system to find the comparison tools they might have—I know for sure GitHub offers this feature.

Now that you’ve had a good look at the code, jot down your answers to the following questions:

1. Does the code comply with your project’s identified coding standards?
2. Does the code limit itself to the scope identified in the ticket?
3. Does the code follow industry best practices in the most efficient way possible?
4. Has the code been implemented in the best possible way according to all of your internal specifications? It’s important to separate your preferences and stylistic differences from actual problems with the code.

Apply the proposed changes

Now is the time to start up your testing environment and view the proposed change in context. How does it look? Does your solution match what the coder thinks they’ve built? If it doesn’t look right, do you need to clear the cache, or perhaps rebuild the Sass output to update the CSS for the project?

Now is the time to also test the code against whatever test suite you use.

1. Does the code introduce any regressions?
2. Does the new code perform as well as the old code? Does it still fall within your project’s performance budget for download and page rendering times?
3. Are the words all spelled correctly, and do they follow any brand-specific guidelines you have?

Depending on the context for this particular code change, there may be other obvious questions you need to address as part of your code review.

Do your best to create the most comprehensive list of everything you can find wrong (and right) with the code. It’s annoying to get dribbles of feedback from someone as part of the review process, so we’ll try to avoid “just one more thing” wherever we can.

Prepare your feedback

Let’s assume you’ve now got a big juicy list of feedback. Maybe you have no feedback, but I doubt it. If you’ve made it this far in the article, it’s because you love to comb through code as much as I do. Let your freak flag fly and let’s get your review structured in a usable manner for your teammates.

For all the notes you’ve assembled to date, sort them into the following categories:

1. The code is broken. It doesn’t compile, introduces a regression, it doesn’t pass the testing suite, or in some way actually fails demonstrably. These are problems which absolutely must be fixed.
2. The code does not follow best practices. You have some conventions, the web industry has some guidelines. These fixes are pretty important to make, but they may have some nuances which the developer might not be aware of.
3. The code isn’t how you would have written it. You’re a developer with battle-tested opinions, and you know you’re right, you just haven’t had the chance to update the Wikipedia page yet to prove it.

Submit your evaluation

Based on this new categorization, you are ready to engage in passive-aggressive coding. If the problem is clearly a typo and falls into one of the first two categories, go ahead and fix it. Obvious typos don’t really need to go back to the original author, do they? Sure, your teammate will be a little embarrassed, but they’ll appreciate you having saved them a bit of time, and you’ll increase the efficiency of the team by reducing the number of round trips the code needs to take between the developer and the reviewer.

If the change you are itching to make falls into the third category: stop. Do not touch the code. Instead, go back to your colleague and get them to describe their approach. Asking “why” might lead to a really interesting conversation about the merits of the approach taken. It may also reveal limitations of the approach to the original developer. By starting the conversation, you open yourself to the possibility that just maybe your way of doing things isn’t the only viable solution.

If you needed to make any changes to the code, they should be absolutely tiny and minor. You should not be making substantive edits in a peer review process. Make the tiny edits, and then add the changes to your local repository as follows:

git add .
git commit -m "[#61524] Correcting <list problem> identified in peer review."

You can keep the message brief, as your changes should be minor. At this point you should push the reviewed code back up to the server for the original developer to double-check and review. Assuming you’ve set up the branch as a tracking branch, it should just be a matter of running the command as follows:

git push

Update the issue in your ticketing system as is appropriate for your review. Perhaps the code needs more work, or perhaps it was good as written and it is now time to close the issue queue.

Repeat the steps in this section until the proposed change is complete, and ready to be merged into the main branch.

Merge the approved change into the trunk

Up to this point you’ve been comparing a ticket branch to the master branch in the repository. This main branch is referred to as the “trunk” of your project. (It’s a tree thing, not an elephant thing.) The final step in the review process will be to merge the ticket branch into the trunk, and clean up the corresponding ticket branches.

Begin by updating your master branch to ensure you can publish your changes after the merge.

git checkout master
git pull origin master

Take a deep breath, and merge your ticket branch back into the main repository. As written, the following command will not create a new commit in your repository history. The commits will simply shuffle into line on the master branch, making git log −−graph appear as though a separate branch has never existed. If you would like to maintain the illusion of a past branch, simply add the parameter −−no-ff to the merge command, which will make it clear, via the graph history and a new commit message, that you have merged a branch at this point. Check with your team to see what’s preferred.

git merge 61524-broken-link

The merge will either fail, or it will succeed. If there are no merge errors, you are ready to share the revised master branch by uploading it to the central repository.

git push

If there are merge errors, the original coders are often better equipped to figure out how to fix them, so you may need to ask them to resolve the conflicts for you.

Once the new commits have been successfully integrated into the master branch, you can delete the old copies of the ticket branches both from your local repository and on the central repository. It’s just basic housekeeping at this point.

git branch -d 61524-broken-link
git push origin --delete 61524-broken-link

Conclusion

Also in Issue № 402

Git: The Safety Net for Your Projects

by Tobias Günther

This is the process that has worked for the teams I’ve been a part of. Without a peer review process, it can be difficult to address problems in a codebase without blame. With it, the code becomes much more collaborative; when a mistake gets in, it’s because we both missed it. And when a mistake is found before it’s committed, we both breathe a sigh of relief that it was found when it was.

Regardless of whether you’re using Git or another source control system, the peer review process can help your team. Peer-reviewed code might take more time to develop, but it contains fewer mistakes, and has a strong, more diverse team supporting it. And, yes, I’ve been known to learn the habits of my reviewers and choose the most appropriate review style for my work, just like I did as a kid.

My post The Islamic State Is Right About Some Things was a “success” as far these things go. It was noted in a column in The New York Times, and highlighted issues which you can see being emphasized in pieces in Slate and The Spectator. But obviously in a single post there is a lot of nuance which I had to elide for reasons of space. Though I may be a population genomicist by day, I do think that in certain domains outside of my bread & butter I bring insights which you can’t find elsewhere, so I try to inject it into the broader discussion. But I’m limited in what I can do in a single post. One of the things I noticed as my post was circulating is that many people asserted that I was suggesting you can understand the actions of the Islamic State by the nature of its theology. Long time readers (I’ve been writing for 12 years on these sorts of issues) might be surprised by this, as was I, because actually I think that is one of the major problems that people have when attempting to understand the nature of religious phenomena. Theology is an abstruse field which is the purview of religious professionals of a particular sort. The vast majority of humans today are marginally literate at best, and for most of human history have been illiterate. To put it succinctly and semi-accurately I think our interpretations of theology are actually effects of prior beliefs, which are due to non-theological parameters. For example, I suspect most Christians would assert that their theology is such that slavery is anathema to their moral system with a proper understanding of God (i.e., theology). Obviously this was not so for the whole of Christian history up until 1800. One conclusion I derive from these sorts of facts is that theology derives its content from the subjective preferences of its practitioners. It is not like mathematics, an objective sequence of inferences and derivations from axioms. Nor is it like the natural sciences, extending itself step by step along a scaffold defined the world around us. Rather, it starts from a presupposition, that God, with particular semantically distinct characteristics, exists, and then proceeds to enter into complex and subtle interpretations of that fact.

I have come to this state of affairs over time through reading. Though I was raised in a religious (Muslim) environment, it was not exceedingly devout or observant, and my personal beliefs were rather devoid of much interest or consideration of supernatural entities. For some people God is an intuitive and intoxicating concept, which draws them in a magnetic fashion. For me a lack of belief is, and was, the natural state. Atheism bubbled up naturally, unbidden, at the age of eight when I decided to look within. When I considered God’s existence seriously, I couldn’t but help reject it. This meant that my understanding of religion has always been as an outsider, and I tended to take religious people at their word when it came to what they believed and how they believed. Religious people of the sort I interacted with explained that their faith was revealed in a set of scriptures, and from those scriptures one could derive the nature of religion. Even religions, such as Roman Catholicism, where scripture is not emphasized generally accept that the foundational texts are necessary and essential in truly comprehending the faith in a deep way with mind (as opposed to just receiving sacraments through liturgy). This was congenial to my mind, as it rationalized religion, turning into a system of propositions from a set of axioms. My scientific bent meant that I naturally understood this sort of mentality.

Therefore, to understand something like Islamic violence, one only need to look at the foundational texts. But though this seems like a fruitful way to go I no longer believe it describes the structure of reality because on an individual level religious belief and practice does not seem rooted at all in texts. Though one can make broad correspondences and draw arrows of causality, with an understanding at a lower and more fine-grained scale this model has as much validity as Galenic medicine. It captures fragments of reality and presents it before us in a persuasive fashion, but at a deeper level of inspection it fails to explain the basic mechanics of religious belief. To understand how I came to this position one has to know that I have long been interested in evolutionary psychology, and therefore cognitive science. After 9/11 I decided to read books on religion besides the basic scriptures, and I stumbled upon the field of evolutionary cognitive anthropology, and in particular the scientific study of religion in the naturalistic paradigm. Two of the primary sources in this domain are Scott Atran’s In God’s We Trust and Pascal Boyer’s Religion Explained. In these dense works they illustrate the cognitive foundations of religious belief and practice, and exposed me to the reality that despite what many religious believers might tell you religious scripture is actually a sideshow to the richness of the phenomenon. Like the coffee table book that one proudly displays the value of scriptures is that is a visible marker and a common point of reference, as opposed to an instruction manual. In Theological Incorrectness the author explores the reality that religious people don’t even seem to believe what they say they believe on a deep level. For example, monotheists and polytheists seem to have the same internal model of the supernatural world, despite their explicit verbal scripts being very different. To put this in another context, many people who espouse views which deny the existence of the supernatural still get “spooked” in a dark cemetery. Why? They are sincere in their belief that there are no ghosts and demons in the dark, but in the deep recesses of their minds reflexive intuitions honed over evolutionary time remain at the ready, alert for any sign of danger in the darkness. Similarly, most religious people may believe sincerely in a glorious afterlife, but when there is a gun to their head they may soil themselves nonetheless.

Belief matters, but it seems likely that it matters at the margins. For whatever reason we humans tend to believe that we have explicit control over our beliefs and actions, and our decisions are due to conscious reflection. This is just often not so, and it has been scientifically validated to my satisfaction. On a personal level I think it is possible that in a different social milieu I would have “rediscovered” my faith in God at some point because of constant feedback from my peers. Though the United States is often depicted, correctly, as a particularly pious developed nation, it is not difficult to seal oneself in a secular bubble. Very few of my friends are religious, despite most Americans being religious. So my atheism is nicely insulated from countervailing pressures. My beliefs, my understanding of reality, is the outcome of a complex interaction between my dispositions and my social-cultural environment. So it is for us all.

But I don’t want to imply from this that if you understand the cognitive science of religion you understand religion. Rather, it is the basic general chemistry of the understanding of the religious phenomenon. In Darwin’s Cathedral David Sloan Wilson outlines a theory of religion which explains the patterns around us in functional terms; i.e., religions as forms of cultural adaptations. Though I’m sceptical of religious models predicated on rational choice theory, that also has its utility in particular contexts. Religion in a socially corporate context such as India is far different from that in the United States, where religion is understood in more individual terms (e.g., defection from a mainstream religion to another mainstream religion does not necessarily entail a massive rupture in your social ties to friends and family in the United States, so churn is common).

So where does this leave us in relation to the Islamic State? Does genocide history and scriptures of Islamic explain its atavistic savagery? I think not. Unlike most Muslim spokespersons I don’t think the behaviour of the Islamic State is “un-Islamic.” Religion is to my mind a made-up affair, and people can remake it in its own image however they want. And, as a point of fact the early Wahabbi movement in the 18th century exhibited many of the same ticks as the Islamic State, down to genocide treatment of those who avowed wrong belief. What I found particular interesting in a detached manner about the Islamic State is how well versed many of its proponents are in a particular streak of the history of Islam. Watching the Vice documentary of the Islamic State I can pick up terms and concepts from my rudimentary religious education, as well as references to “the Romans,” which in that case refers to the Byzantines under the Heraclian dynasty. Rather than theology I suspect history is a better guide as to what’s going on, and why, from the violent exclusive strain of Islam which periodically emerges from the Kharijites down to the Wahabbis, to early modern period and post-colonial conflicts, as well as the ethnography of political radicalism among small motivated groups such as the anarchists. Most proximately the Islamic State clearly draws energy and strength from Sunni resentment toward Alawite hegemony in Syria and Shia dominance in Iraq. Over time this may evolve into something else, as a generation grows up under the influence of the message of the Islamic State and its broader Weltanschauung. It is essential to keep in mind both the generalities (e.g., it is a Sunni movement) and particularities (e.g., it is global in its imagination and aspiration, at least notionally) when attempting to gauge the possible arcs of the future.

Addendum: And in the interest of frankness, I will also admit that though comments can be highly informative, I don’t listen closely when someone decides to lecture me on the nature of religion because it is rare than I encounter anyone with as much breadth of knowledge as me in this domain (i.e., I have read economic, sociobiological, cognitive, and historical models of religion). If I seem to dismiss your opinion, that’s probably because I don’t think much of your ideas because you likely know far less than I do.

Há uma perda de produtividade que reduziu o potencial de crescimento do Brasil.

É melhor ele ter um celular e pegar três serviços por dia do que empregar e atender dez clientes - é perda de produtividade na veia.

More discussion on Hacker News, and the Clojure, Haskell, Rust and Programming subs on Reddit.

Over the last few years of maintaining code old and new at REA, it has become abundantly clear that the neglect and misuse of type-systems has had a sharply negative impact on our codebases. Here we address the concrete causes and consequences, and propose concrete and achievable solutions.

Types at REA

REA’s codebases vary between statically typed languages such as Java and Scala on one hand, and “dynamic” languages such as Ruby, Javascript, and Perl on the other. I’ll mostly leave the dynamic ones alone — while the same arguments obviously apply, they are what they are. However, if we are paying for the conceptual overhead of a static type system, we need to be sure we are using it effectively; otherwise we are simply writing the same old Ruby play-dough but with long build-times and cumbersome syntax. Used properly, the tradeoff of static types is overwhelmingly beneficial.

What’s in a type?

A type is some logical proposition about a codebase, where an implementation is its proof. They are distinct from runtime tags, which are what people mostly mean when they talk about dynamic “types”. Terminology can vary, but I’ll stick to this usage. For example:

• Haskell has a spectacularly rich static type system, with no runtime-accessible metadata, apart from pattern matching on tagged unions.
• Dynamic languages such as Ruby are not really untyped, but rather unityped; statically, we know that everything inhabits a single message-receiving type. At runtime, of course, they employ tags to differentiate numbers from strings, arrays, maps, and user-defined structures.
• Java has both a type system applied by its compiler, and a tag system at runtime that allows reflection, casting, RTTI, and various dynamic features. Neither is a compelling specimen.

Types as design

Even if the language we use doesn’t have a notion of types, you’d better believe they exist — how can we even write code without statically reasoning about it first? A maintainer must construct a model in their heads: what keys do we assume are in this map? Might this thing be null? What are the allowed values of this symbol? Can this string be empty? What messages will this object respond to?

Many dynamic-typing advocates have a curiously limited view of what a “type” is – in fact, almost any proposition we can statically make about the code can be represented as a type. Language choice notwithstanding, there is an enormous amount of information that fits into this category; by proving it up-front we can drastically reduce the number of incorrect programs that are even expressible. As the saying goes, don’t write software that isn’t broken; write software that can’t be broken.

Types are a powerful tool for clarifying thoughts, and designing correct software, arguably far more so than popular test-driven methodologies.

Types done wrong

Essentially anything that can be cleanly and obviously known about the code up-front belongs in a type. The most egregious failures here often stem from using the most common everyday concepts — strings, exceptions, primitives, maps, nulls, typecasts and so forth. Many readers of this post will, like its author, find something to feel guilty about here. Let’s take a tour:

Nulls

The harm represented by nulls is hopefully widely understood by now, but bears repeating.

Any value that we know could be null cannot be directly accessed by correct software; we must either surround usages in an if-guard, or employ some kind of harmless Null Object that can hopefully respond in a sensible way. It is a bald-faced lie told by the type-system in Java, C# and Scala, and to the developer’s mental model in Ruby. If a variable claims to be a Banana, surely you can feel justified in peel()-ing it? If it is null, then it is no banana at all, but a ticking time bomb waiting to explode, potentially at an unrelated line of code far away. Well-written code cannot tolerate even the possibility.

The proliferation of duplicated defensive code at numerous locations is a further burden, which bloats both code and tests, while reducing quality.

Solution:

• Never permit nulls in code you control, and firmly regulate the contact points of systems and frameworks you don’t.
• If a type has a sensible “empty” or default value that can fulfil the contract of your type, then initialise variables to this, or employ a Null Object.
• Avoid mutable entities that need to be initialised piecemeal after creation. Write immutable objects that are immediately and fully initialised from constructor input.
• If a particular variable might or might not be present, then this should be encoded in the type system using an option type, like Scala’s Option, Function Java’s Option, Java 8′s Optional, or Haskell’s Maybe. This correctly represents the uncertainty in the type system, so that any access is safe, simply by the fact that it compiled.

Exceptions

Exceptions are the primary error-handling mechanism employed by many widely-used languages. They are also a side-effect that makes a liar of the type system, and makes local reasoning about code far more difficult. They represent an undeclared method result smuggled through a back-channel separate from its declared return type. Furthermore, they transitively become an undeclared result of anything that calls that method, and anything that calls that, and so on. Trying to reason about the correct behaviour of code becomes very difficult, since the return type can no longer give you enough information. Exceptions kill modularity and inhibit composition.

Java awkwardly attempts to mitigate this with checked exceptions; they become a fully-fledged, type-checked part of a method signature. While this is better from a type-safety point of view, they still use an exotic second channel for returning results totally incompatible with the first, require an insufferable amount of handling code, and have far poorer tools for abstraction and reuse. Checked exceptions are widely despised by Java programmers, and frequently ignored by library authors.

Solution:

• Don’t throw exceptions in code you control, except in the most irretrievably broken circumstances.
• When dealing with code you don’t control, catch their exceptions as soon as possible and lift the various results into your return type.  In Scala, the easiest way to do this is the Try type, which directly lifts the result into an ADT of Success(yourValue) or Failure(thrownException).
• Exclusively encode possible function results in the return type. Don’t throw that AuthenticationException for a totally plausible and normal outcome! Here’s some alternatives:
  • When there is a main result alongside a possible failure result, use an existing Either or Validation type.
  • Define your own Algebraic Data Type (ADT) that describes the possible alternatives. For instance, in Scala or Java, this can take the form of a closed mini-class hierarchy.

Primitives

Primitive values such as integers and strings are often the first tools we reach for, but are woefully unsuited to most use-cases they are press-ganged into. This is because they have an astronomical number of possible values, and most use-cases do not.

Integers

Consider this function:

def blah(b: Boolean): Boolean

A function A -> B has B^A possible implementations, by the number of inhabitants in A and B. So this function has 2² = 4 possible implementations. Perhaps we could write a test case for each one.

Now consider this function:

def compare(a: Int, b: Int): Int

This one has not only 2³² possible results, but an incalculable (2³²)²⁶⁴ possible implementations. Even if we are generous and only allow the inputs to be 0 or 1, we still have 2¹²⁸ implementations. If the only meaningful return values are -1, 0 and 1, indicating “less than”, “equal” or “greater than”, then why would we embed them like specks of dust in a big desert of undefined behaviour?

If we encode the return value as an ADT representing the 3 possible results, as Haskell does, then we have a positively civilised 3⁴ = 81 possible mappings from input to output. Even in such a simple case, a more precise return type pruned 340 undecillion incorrect programs from the sphere of existence.

Remember, that was an utterly trivial example. So what happens when you have complex/composite/nested data structures, exceptions and even side-effects? How many of the possibilities are even remotely meaningful in your domain? Without precise types, how many were you hoping to reach with your TDD and “100% code coverage”?

Strings

String are perhaps the most commonly used data type, due to their immense versatility; however, they are rarely appropriate. Strings consist of a sequence of characters. This is the perfect representation for unstructured free text, and nothing else. Any restriction, structure or constraint in the format of the string means you don’t have a string at all; you have a URL, a Date, a Name, an Email, a Document, an ID, a Warning, or whatever else that might tempt you to deploy this amazing swiss-army-type.

Not only does “stringly typed” code result in a catastrophic expansion in the number of expressible incorrect programs, it inevitably results in duplication, as the validation, destructuring and restructuring code is repeated in every spot where the string format is supposed to be in use.

Solution:

• If there are a finite number of possibilities, use an ADT to represent them. This internally prevents a vast number of incorrect implementations, and externally prevents a vast number of incorrect usages.
• Use a wrapper type to encapsulate the desired structure; make it impossible to create invalid instances. This will also cull incalculable absurdities inside and outside of the function. This costs one line of code in Scala.
• Wrapper types can sometimes normalise errant input in their constructors, seamlessly eliminating redundant or invalid states.
• As a last resort, throw exceptions inside constructors to prevent any remaining possibility of an invalid instance.

Records vs Domain modelling

A couple of years ago, we were keen to avoid over-specific domain modelling, and took care to build our services as dealing in the domain of “records” or “attribute-maps”, rather than specifically tie our logic to Listings, Agents, Dogs, Cats, Aeroplanes or what-have-you. This was to prevent the loss of generality, and to keep the application focused on web and infrastructural concerns. So following the famous Alan Perlis line “It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures”, we decided on a unityped representation of domain data.

In an absolute sense, I can’t say whether this is a good idea or not. It’s potentially a totally valid point of view — perhaps the detail and structure of the record data is of no relevance to the code that provisions, streams, encodes, stores, secures and displays it — as long as it stays that way. In practice, however, it has left our codebase with serious flaws.

Firstly, the approach of using unityped records is totally predicated on the application not needing to know anything about their structure, beyond the obvious tree-shape. When the application suddenly needs to, say, differentiate “floorplan” from “main” images, or know if a “logo” was included, then the concept is doomed. This knowledge, completely understood at compile time, must be expressed in clumsy string-based map retrieval, faith-based typecasting and a total absence of any way to reason about the correctness, or even the intended behaviour of the code.

Either business logic has to be strictly forbidden from this unityped pipeline, or we need the types of of the data to truly reflect the knowledge we have, and need, at compile time.

Secondly, even if the unityped record approach was correctly chosen, using, as we did, Map<String, Object> was a clear blunder. The verbosity and repetition are appalling; every single access, construction, iteration, de-reference has to be laboriously performed by hand at myriad locations around the codebase. Even if all we need to know is “it’s a map with certain behaviours and constraints”, then we should have encoded that in the types, and created some sort of Record class. In this case, it was also compounded by the use of Java (pre-8), which has extremely poor facilities for abstracting over collections and maps.

Solution:

• The static knowledge we have, or require about our domain objects should be captured in types.  If there is a clear case that we don’t need to differentiate between this-or-that domain object, then that should be captured in the types.
• Quickly translate interchange formats into data structures that reflect the knowledge we require, and can only hold meaningful and valid states. There are mapping tools in Java and Scala that can safely map between JSON and typed objects.

Names are overrated

Let’s look at a function:

def findAddress(userId: String): String

What does it do? Are you sure?

Now lets look at another:

def blah(foo: UserId): Address

Which one tells you more about its purpose — the one with the businessy names, or the one with the types?

Naming has role to play, but consider what it really does. It is a mnemonic, a reference that helps you uniquely recall a concept. While this is fine as far as it goes, names are totally useless for reasoning about software. For documentation, they are as poor as comments, or Word docs. Implementing a precise type signature proves that the software does what it says on the tin. If the types in question have been carefully designed to prevent invalid states, then often there will be only be a handful of possible implementations, or even one — not a number you’ve never heard before, ending in -illion.

Solution:

• Treat your types as the only real documentation.
• Constrain argument and return types to a named alternative, that limits possible states. This is all maintainers should need to know about your functions.
• Wrapper types start from one line of code in Scala.

The way forward

While I’ve listed a variety of different problems, you’ll notice a lot of repetition in the proposed solutions. In fact, we can solve these problems very simply, using only a few techniques — especially if we continue our adoption of Scala in place of Java and weakly-typed languages.

Algebraic Data Types (ADTs)

ADTs are a powerful tool for us here, because they allow us to encode limited possibilities in the type system, so that invalid combinations are inexpressible in a well-typed program.

They are called “algebraic” because they are sums and products of other types. (Sums are like OR, and products are like AND).

For instance, a List in Scala is defined as an ADT — it is a Cons of a value AND another List, OR an empty List. In Haskell, this could be written simply as:

data List a = Cons a (List a) | Empty

In Scala, at the cost of some more characters, we could encode this as a mini-class hierarchy:

sealed trait List[+A]
case class Cons[A](a: A, rest: List[A]) extends List[A]
case object Empty extends List[Nothing]

This is almost a straightforward Java-style class hierarchy, but notice the sealed keyword: unlike normal OO classes, List cannot be extended, except by the classes below it. Without this feature, the number of possible outcomes would still be totally unbounded. In Java, we can still benefit from using this style, but the code required to manually write accessors, constructors, threading through arguments, correct hashcode/equals implementations and unit tests is significant, and error prone.

OO lore has it that pattern matching is evil, and that subtype-polymorphism is the answer to all questions. This is false; there are complementary pros and cons to subtype polymorphism and pattern matching. Since there are only a few fixed cases, it is perfectly idiomatic and sensible to pattern match on ADTs; the Scala compiler will even complain if we haven’t matched every possible eventuality.

myList match {
  case Cons(a, rest) => println(s"the head is $a, the rest is $rest")
  case Empty => println("Nothing to see here")
}

ADTs are a handy weapon in our war against buggy code!

Wrapper types

Wrapper types are one of the best ways to avoid the buggy swamplands of code written with bare strings and primitives. In Scala, this starts at almost no effort:

case class Angle(radians: Double) extends AnyVal

“Case classes” in Scala are (mostly) like any other class, except that the compiler will generate useful functionality. Scala will automatically bind the constructor parameter to an immutable field exposed through accessor methods, generate correct equals and hashCode, a default toString, and a pattern matching extractor. Hugely useful.

Value Classes: eliminating runtime overhead

Explicitly saying extends AnyVal makes our class a Value Class. This class won’t even exist at runtime — it provides type safety in the compiler, then vanishes!

Normalising and validating input

We can get all the benefits of classes here – we can define our own operations, and normalise or validate the constructor input. Here are some examples of this technique applied to Angles and Percentages.  Note they are correct-by-construction; any instance of this type is guaranteed to represent a valid and normalised value.

We should make far wider use of wrapper classes:

case class Email(email: String)
case class Password(password: String)
case class SecurityToken(token: String)
case class ConsumerId(id: String)
case class AgentId(id: String)
case class Price(cents: BigInt)

Types are low hanging fruit

While there are no silver bullets, there’s an awful lot of low-hanging fruit just lying around. Let’s pluck it! We can make major improvements in our software quality, even with minor adjustments to our coding style. Code can be easier to reason about, with vastly less ways to fail, at a very low cost.

Types, kindly bestowed upon us by some languages, are a magnificent tool to improve quality. They prove desirable properties of our code; we should make it our business to put as much code in their reach as we can! There will always be a point where types have no more to say, and must pass the quality baton to tests. Consider though, how much less work tests must do, and how much less code they must expend, when entire universes of nonsense have been prohibited from existence.

In particular, by eschewing exceptions, using Algebraic Data Types to model the precise shape of our data, and wrapper types to constrain crude Strings and primitives, we can make immediate gains before we even get to more advanced abstractions like typeclasses and higher-kinded types.

In Java, much of this has been known for a long time, but the language’s lack of support for value-based classes, ADTs and pattern-matching has meant that good practices are often discarded as prohibitively cumbersome or expensive. Regrettably, despite the welcome addition of lambdas, Java 8 provides little respite.

In languages like Haskell and Scala, these methods are so cheap as to be no-brainers; in new projects you have no excuse for passing up these delights!

Either way, I hope that I’ve convinced you of the good news — there are plentiful green fields of easy code-improvement ahead, before we even get close to tough tradeoffs.