Ever since I first heard about Knack, the Palo Alto startup that makes computer games designed to measure whether you’re likely ever to amount to anything, I knew I had to play the games and find out whether I’m likely ever to amount to anything. But I was also scared, for two main reasons:
1. The games might get it wrong. Other than an occasional round of Mario Kart Wii, I never play video or computer games. As a result I’m no good at gaming. My ineptitude would, I feared, work against me when I played Knack’s games, preventing all my wonderful qualities from shining through. Knack’s CEO and founder, Guy Halfteck, assured me that this wouldn’t be the case. “In our data-analytic work, we’re actually factoring out your game-play experience,” he promised. Did I believe that? Well, mayyybe. But my really big fear was that:
2. The games might get it right. What if playing a computer game for 10 minutes really can deliver a spot-on assessment of one’s strengths and weaknesses? Along with identifying your natural leanings a la Myers-Briggs or StrengthsFinder, Knack’s games are supposed to measure plain old aptitude. And you can’t really study for them. Basically, they leave nowhere to hide.
I dwell on these fears because these games could end up mattering, a lot. In recent HBR.org posts, both Andy McAfee and Michael Staton suggested that Knack game results could supplant college degrees as signals of employability. In an article in the December Atlantic, Don Peck described Knack’s success in identifying successful idea generators at Royal Dutch Shell, and depicted such game-based assessments as part of a revolution in people analytics that would transform hiring over the next decade.
I first met Halfteck at a conference late in 2012, and got him to send me a link to one of his company’s games last spring so I could play it and have my performance assessed. Then I hemmed and I hawed, and hawed and hemmed. It wasn’t till December, spurred by a promise to a colleague to write something for HBR’s Talent and the New World of Hiring Insight Center (promise since broken, as the Insight Center ended a couple weeks ago) that I sat down with my laptop one evening and fired up Wasabi Waiter.
The game involves serving different meals to guests depending on the expressions on their faces. I initially couldn’t figure how to move my waiter (turns out you just have to click where you want him to go), and my 14-year-old son walked over to check what was going on. He helped me on the first round, then took over, to the great relief of the characters in the game, who finally started getting the meals they wanted.
It was easy to see some of the things the game was measuring. There’s lots of research relating the ability to recognize facial emotions and social skills. The hectic serving and disposing of plates seems to be about showing how you handle multiple tasks and juggle priorities. The various unannounced features allow a player to display resourcefulness and curiosity. And Halfteck claims that every little pause and click a player makes conveys some kind of information.
Since my son had assumed my Wasabi Waiter identity, I asked Halfteck to send me another invite. In the meantime, I played a Knack game designed for iPhones and iPads, Balloon Brigade. This time the player’s job is to fill up and lob water balloons — initially to water flowers, later to extinguish hordes of marauding, flaming imps. Here’s what it looks like:
The seeming straightforwardness of Balloon Brigade restored my confidence. My balloon-filling contraption never caught fire. I extinguished lots of flaming imps. There were things I never figured out — such as what the difference was between the two buttons at the bottom of the contraption (and as I look at the above image, I see a third button that I never noticed while playing). But I was starting to feel like less of a failure.
So I took on Wasabi Waiter again. It went okay. I did get flustered when my restaurant got crowded. But I was able to serve most of the people most of the time. My customer-satisfaction ratings improved. I did, however, completely misread a couple of customers’ faces.
A few days later, I got this report from Knack (HBR’s art department has moved the images around a bit to fit better in this space and switched them from green to blue to fit with our house style, but they are otherwise unchanged):
The scores, Knack said, were percentiles calculated with reference to a sample group representative of the “general adult population of employees and job-seekers in the United States.” It was grading on the curve, thus. I clearly wasn’t outstanding at anything, but I was pretty good at “task efficiency” and worst at “social intelligence.”
That sounded a little like I should be working on the assembly line at a factory where I’m the only assembly-line worker. When I asked Halfteck to elaborate, he emailed back that task efficiency actually “predicts your competency in reasoning logically, thinking critically, processing novel information quickly, and solving problems.” Okay, that sounds better — in a later conversation he termed it “cognitive throughput.” My low social intelligence score, though, “means that jobs that involve frequent interaction with clients, subordinates, or others will not be the best match.” And my “signature score” of 61, Halfteck wrote, means I’m “in the top 40% in terms of employability in professional/knowledge jobs.” Wow! The top 40%!
I then forwarded the report to my current boss, HBR editor-in-chief Adi Ignatius, and a former boss, recently retired Time Inc. editor in chief John Huey, to find out if they thought it rang true. I included Halfteck’s explanation of what “task efficiency” meant, but my high score there still was something of a red flag. Wrote Huey:
I would say “efficient” is among the last words I would choose to describe anything about you. Au contraire, your inefficiency seems to me the only thing that holds you back.
He then wrote a bunch of nice things about me that seem too self-congratulatory to repeat here, concluding, “Task efficiency? Bah humbug!”
Adi (sorry for the inconsistency, but in my world Huey has almost always been referred to by his last name and Adi always by his first) agreed that while I was good at reasoning and thinking and all that, I didn’t always do it quickly but instead “tend to get bogged down in process and in longer projects.” (This is undeniable. I wrote a book once. It was supposed to take me two years. It took more than five.) Adi also thought I was perfectly conscientious and gritty, whatever that means. But he did conclude with the warning:
Since you only got a 61 overall, we probably need to talk offline about HBR’s ROI on JF.
I think that’s a joke. But that 61 did bother me. Back in the days when I took standardized tests, my percentile ranking was a lot higher than that. So I went back to Halfteck, who by now must have been feeling a bit like he was playing guidance counselor to a very old high schooler, and asked what might explain the difference. It was a combination, he said, of
- Knack simply being better than SATs and such, “because what we measure is actually richer; we can look at other vectors of optimization.” (Of course he’d say that, but there are lots of questions about the predictive usefulness of current standardized tests.)
- The SAT and similar tests giving more weight to education than Knack’s games do.
- The usual cognitive decline that comes with aging.
- The fact that “we’re still working on making the measures more robust.” In other words, Knack could be wrong.
In other words, basically, I’m one out of four. And in general, I have to admit that my scores didn’t feel nearly as off to me as they did to Huey and Adi (and you’ll note that neither of them said anything to dispute my low “social intelligence” marks).
I spend most of my days reading and writing in near-solitude. That is, there may be others around at the office and home, and I do meet people for lunch and do interviews in person or over the phone. But most of the time it’s me and a computer or iPad or book. I used to have a job where I went to lots of meetings and had a few people who reported to me, but I begged to be relieved of that and, later, to move 200 miles away from my colleagues (I’m now the lone editorial employee in HBR’s advertising sales office in New York). I think I am pretty curious and open-minded, so that score makes sense. As for risk, in my career I have worked for four large and stable media companies (stable when I worked there at least, except for my last few years at Time Inc.) and America’s oldest and the world’s richest university. That sounds risk-intolerant to me. If lacking grit means I go out of my way to avoid trying, unpleasant situations, that sounds like me, too. I’m not sure what to make of those conscientiousness and stability rankings, but in general Knack’s assessment does seem to be directionally correct. And to come up with all of that by simply having me play games for 10 to 15 minutes is staggering.
Halfteck, a lawyer and economist with multiple Ivy League degrees, says he decided to found Knack after going through a months-long interview process for a job at a hedge fund, only to be turned down. ”The feedback they gave me didn’t make any sense to me,” he told me. “I was very frustrated because I thought I had the potential to be a leader at that company.” That kind of frustration with current hiring processes is pretty common, among both job seekers and those doing the hiring. Anything that could shift things in the direction of matching talent better with jobs, as Knack’s games probably could, ought therefore to be applauded.
Except that … what if these game assessments were really good — almost perfect at predicting job success. And what if every employer used them? Maybe Halfteck would have gotten that hedge fund job, and never started a company. Then where’d we be? Disappointment and failure make us better and stronger, right?
“There is value in serendipity, there is value in randomness,” Halfteck said when I ran that argument by him. “The fact that our systems are not perfect is a good thing.” But what about, he went on, the surely far larger numbers of people who didn’t start companies after failing to get jobs they would have done well in? What about the gains to society from those who know from an early age what they want to do and are contributing to their fields in a big way by their 30s (presumably before their cognitive throughput starts slowing down)? Shouldn’t we want more of that?
Yeah, we should. And all indications are that we will be getting more, thanks to Knack’s games and countless other coming improvements in talent analytics. I’m just unclear on whether I should be sad to encounter this revolution in how people are matched to their jobs so late in my career, or grateful to have been allowed to do so many cool things over the years despite my now-obvious mediocrity.
