Reaching the right person with the right message at the right time has long been the holy grail of marketing. The technology and data is tantalizingly close, and yet context and intent remain elusive. Personalized marketing is still wasted if it hits the wrong context and intent.
Here’s how Google framed the opportunity recently:
“Every day, three billion people around the world have dozens of moments that matter to them and their lives. These moments create billions of ‘signals’, which not only include context e.g. where someone is, what device they are using or the time of day, but also intent: what someone wants or needs at that moment.
“This combination of context and intent-driven signals is a goldmine for marketers, providing more opportunities to be relevant and connect with consumers in more meaningful ways than ever before.”
"If marketing kept a diary, this would be it."
- Ann Handley, Chief Content Officer of MarketingProfs
Google has been advocating the concept of winning the “moments that matter” for your brand. As they concluded in a piece of research a couple years ago:
“People are more loyal to their need in the moment than to any particular brand.”
That’s a sobering thought for brand builders. Those signals may be a goldmine for marketers, but we’re still in the awkward adolescent phase of data-driven marketing marketing. We still have some work to do.
Here are a couple related cartoons I’ve drawn over the years:
So if my historical sources are telling me the truth…
…and I’m synthesizing the history properly…
…then, in fact, the entire edifice of Western civilization – all the cultural, social, and philosophical structures that define the world in which we live today – can be traced back to a stupid loophole in Roman inheritance law.
NOTE: Everything here is taken either from Francis Fukuyama’s The Origins of Political Order or from a Livejournal post by the Infamous Brad that I am currently unable to find. I get credit for absolutely nothing, except noticing the connection between Section II and Section III.
Leonid Moroz has spent two decades trying to wrap his head around a mind-boggling idea: even as scientists start to look for alien life in other planets, there might already be aliens, with surprisingly different biology and brains, right here on Earth. Those aliens have hidden in plain sight for millennia. They have plenty to teach us about the nature of evolution, and what to expect when we finally discover life on other worlds.
Moroz, a neuroscientist, saw the first hint of his discovery back in the summer of 1995, not long after arriving in the United States from his native Russia. He spent that summer at the Friday Harbor marine laboratory in Washington. The lab sat amid an archipelago of forested islands in Puget Sound – a crossroads of opposing tides and currents that carried hundreds of animal species past the rocky shore: swarms of jellyfish, amphipod crustaceans, undulating sea lilies, nudibranch slugs, flatworms, and the larvae of fish, sea stars and countless other animals. These creatures represented not just the far reaches of Puget Sound, but also the farthest branches of the animal tree of life. Moroz spent hours out on the pier behind the lab, collecting animals so he could study their nerves. He had devoted years to studying nervous systems across the animal kingdom, in hopes of understanding the evolutionary origin of brains and intelligence. But he came to Friday Harbor to find one animal in particular.
He trained his eyes to recognise its bulbous, transparent body in the sunlit water: an iridescent glint and fleeting shards of rainbow light, scattered by the rhythmic beating of thousands of hair-like cilia, propelling it through the water. This type of animal, called a ctenophore (pronounced ‘ten-o-for’ or ‘teen-o-for’), was long considered just another kind of jellyfish. But that summer at Friday Harbor, Moroz made a startling discovery: beneath this animal’s humdrum exterior was a monumental case of mistaken identity. From his very first experiments, he could see that these animals were unrelated to jellyfish. In fact, they were profoundly different from any other animal on Earth.
Moroz reached this conclusion by testing the nerve cells of ctenophores for the neurotransmitters serotonin, dopamine and nitric oxide, chemical messengers considered the universal neural language of all animals. But try as he might, he could not find these molecules. The implications were profound.
The ctenophore was already known for having a relatively advanced nervous system; but these first experiments by Moroz showed that its nerves were constructed from a different set of molecular building blocks – different from any other animal – using ‘a different chemical language’, says Moroz: these animals are ‘aliens of the sea’.
If Moroz is right, then the ctenophore represents an evolutionary experiment of stunning proportions, one that has been running for more than half a billion years. This separate pathway of evolution – a sort of Evolution 2.0 – has invented neurons, muscles and other specialised tissues, independently from the rest of the animal kingdom, using different starting materials.
This animal, the ctenophore, provides clues to how evolution might have gone if not for the advent of vertebrates, mammals and humans, who came to dominate the ecosystems of Earth. It sheds light on a profound debate that has raged for decades: when it comes to the present-day face of life on Earth, how much of it happened by pure accident, and how much was inevitable from the start?
If evolution were re-run here on Earth, would intelligence arise a second time? And if it did, might it just as easily turn up in some other, far-flung branch of the animal tree? The ctenophore offers some tantalising hints by showing just how different from one another brains can be. Brains are the crowning case of convergent evolution – the process by which unrelated species evolve similar traits to navigate the same kind of world. Humans might have evolved an unprecedented intellect, but the ctenophore suggests that we might not be alone. The tendency of complex nervous systems to evolve is probably universal – not just on Earth, but also in other worlds.
Sign up for Aeon’s Newsletter
As major animal groups go, the ctenophore is poorly understood. Its body superficially resembles that of a jellyfish – gelatinous, oblong or spherical, with a circular mouth at one end. Ctenophores are abundant in the oceans, but long-neglected by scientists. Well into the 20th century, drawings in textbooks often showed the animal upside down, its mouth hanging toward the seafloor, in jellyfish fashion, whereas in real life, it drifts with its mouth pointed upward.
Unlike the jellyfish, which uses muscles to flap its body and swim, the ctenophore uses thousands of cilia to swim. And unlike the jellyfish with its stinging tentacles, the ctenophore hunts using two sticky tentacles that secrete glue, an adaptation with no parallel in the rest of the animal kingdom. The ctenophore is a voracious predator, known for its ambush tactics. It hunts by spreading its branched, sticky tentacles to form something like a spiderweb, and catches its prey meticulously, one by one.
When scientists began examining the ctenophore nervous system in the late 1800s, what they saw through their microscopes seemed ordinary. A thick tangle of neurons sat near the animal’s south pole, a diffuse network of nerves spread throughout its body, and a handful of thick nerve bundles extended to each tentacle and to each of its eight bands of cilia. Electron microscope studies in the 1960s showed what seemed to be synapses between these neurons, with bubble-like compartments poised to release neurotransmitters that would stimulate the neighbouring cell.
Scientists injected the neurons of living ctenophores with calcium – causing them to fire electric pulses, just as happens in the nerves of rats, worms, flies, snails and every other animal. By stimulating the right nerves, researchers could even prompt its cilia to rotate in different patterns – causing it to swim forward or back.
In short, the ctenophore’s nerves seemed to look and act just like those of any other animal. So biologists assumed that they were the same. This view of ctenophores played into a larger narrative on the evolution of all animals – one that would also turn out to be wrong.
The story of the sponge supported the convenient view that the nervous system had evolved gradually, toward greater complexity
By the 1990s, scientists had placed ctenophores low on the animal tree of life, on a branch next to cnidarians, the group that includes jellyfish, sea anemones and coral. Jellyfish and ctenophores both have muscles, and both have diffuse nervous systems that haven’t fully condensed into a brain. And, of course, both have bodies that are famously soft, jiggly and often transparent.
Below ctenophores and jellyfish on the evolutionary tree sat two other branches of animals that were clearly more primitive: placozoans and sea sponges, which both lacked nerve cells of any kind. The sponge in particular had seemed just barely on the cusp of animalhood: not until 1866 did the English biologist Henry James Clark demonstrate that the sponge was, indeed, an animal.
This helped to enshrine the sponge as our closest living link to an ancient, pre-animal world of single-celled protists, akin to modern-day amoeba and paramecium. Researchers reasoned that sponges had evolved when ancient protists gathered into high-rise colonies, with each cell using its flagella – threadlike structures akin to cilia – for feeding instead of swimming.
This narrative supported the convenient view that the nervous system had evolved gradually, toward greater complexity with each successive branch of the animal tree. All animals were sons and daughters of a single moment of evolutionary creation: the birth of the nerve cell. And only once, in subsequent evolution, had those neurons crossed a second momentous threshold – aggregating into a centralised brain. This view was bolstered by another line of evidence: striking similarities in the way that individual nerve cells were arranged in insects and humans, into neural circuits underlying episodic memory, spatial navigation and overall behaviour. In fact, scientists held, the first brain must have appeared quite early, before the ancestors of insects and vertebrates parted evolutionary ways. If this was true, then the 550 to 650 million years elapsing since that event would represent a single storyline, with multiple animal lineages elaborating on the same, basic brain blueprint up and down the chain.
This picture of brain evolution made sense, but observing the scene at Friday Harbor in 1995, Moroz began to suspect that it was profoundly wrong. To demonstrate his hunch, he collected several species of ctenophores. He sliced their neural tissue into thin slivers and treated them with chemical stains indicating the presence of dopamine, serotonin or nitric oxide – three neurotransmitters that were widespread across the animal kingdom. Again and again, he looked into the microscope and saw no trace of the yellow, red or green stains.
Once you repeat the experiments, says Moroz: ‘You start to realise it’s a really different animal.’ He surmised that the ctenophore was not just different from its supposed sister group, the jellyfish. It was also vastly different from any other nervous system on Earth.
The ctenophore seemed to follow an entirely different evolutionary pathway, but Moroz couldn’t be sure. And if he published his results now, after looking at just a few important molecules, people would utterly dismiss them. ‘Extraordinary claims require extraordinary evidence,’ says Moroz. And so he embarked on a long, slow road, one even longer than even he suspected at the time.
He applied for funding to study ctenophores using other techniques – for example, looking at their genes – but gave up after being turned down multiple times. He was still young at that point, had left the Soviet Union only a few years before, and had only just started publishing his work in English-language journals where it would generate broader interest. So Moroz put ctenophores on a back burner and returned to his primary work, studying neural signalling in snails, clams, octopuses and other molluscs. It was only by chance, 12 years later, that he returned to his passion project.
In 2007, he briefly visited Friday Harbor for a scientific conference. One evening, he strolled out onto the same docks where he had spent so much time in 1995. There, by chance, he glimpsed the iridescent sparkles of ctenophores drifting under the light of a lantern. Scientific tools had advanced by then, making it possible to sequence an entire genome in days rather than years. And Moroz was now established, with his own lab at the University of Florida. He could finally afford to dabble in curiosities.
So he fetched a net and fished a dozen or so ctenophores, a species called Pleurobrachia bachei, from the water. He froze them and shipped them to his lab in Florida. Within three weeks, he had a partial ‘transcriptome’ of the ctenophore – some 5,000 or 6,000 gene sequences that were actively turned on in the animal’s nerve cells. The results were startling.
First, they showed that Pleurobrachia lacked the genes and enzymes required to manufacture a long list of neurotransmitters widely seen in other animals. These missing neurotransmitters included not just the ones that Moroz had noted back in 1995 – serotonin, dopamine and nitric oxide – but also acetylcholine, octopamine, noradrenaline and others. The ctenophore also lacked genes for the receptors that allow a neuron to capture these neurotransmitters and respond to them.
This confirmed what Moroz had waited years to find out: that when he failed to find common neurotransmitters in ctenophore nerves back in 1995, it wasn’t simply that his tests weren’t working; rather, it was because the animal wasn’t using them in any way. This, says Moroz, was ‘a big surprise’.
‘We all use neurotransmitters,’ he says. ‘From jellyfish to worms, to molluscs, to humans, to sea urchins, you will see a very consistent set of signalling molecules.’ But, somehow, the ctenophore had evolved a nervous system in which these roles were filled by a different, as-yet unknown set of molecules.
The ctenophore had evolved from the ground up, using a different set of molecules and genes than any other animal known on Earth
Moroz’s transcriptome and genomic DNA sequences showed that the ctenophore also lacked many other genes, known from the rest of the animal kingdom, that are crucial for building and operating nervous systems. Pleurobrachia was missing many common proteins called ion channels that generate electric signals that travel rapidly down a nerve. It was missing genes that guide embryonic cells through the complex transformation into mature nerve cells. And it was missing well-known genes that orchestrate the stepwise connection of those neurons into mature, functioning circuits. ‘It was much more than just the presence or absence of just a few genes,’ he says. ‘It was really a grand design.’
It meant that the nervous system of the ctenophore had evolved from the ground up, using a different set of molecules and genes than any other animal known on Earth. It was a classic case of convergence: the lineage of ctenophores had evolved a nervous system using whatever genetic starting materials were available. In a sense, it was an alien nervous system – evolved separately from the rest of the animal kingdom.
But the surprises didn’t stop there. The ctenophore was turning out to be unique from other animals in far more than just its nervous system. The genes involved in development and function of its muscles were also entirely different. And the ctenophore lacked several classes of general body-patterning genes that were thought to be universal to all animals. These included so-called micro-RNA genes, which help to form specialised cell types in organs, and HOX genes, which divide bodies into separate parts, be it the segmented body of a worm or lobster, or the segmented spine and finger bones of a human. These gene classes were present in simpleton sponges and placozoa – yet absent in ctenophores.
All of this pointed to a stunning conclusion: despite being more complex than sponges and placozoans – which lacked nerve cells and muscles and virtually every other specialised cell type – ctenophores were actually the earliest, oldest branch on the animal tree of life. Somehow over the subsequent 550 to 750 million years, the ctenophore had managed to evolve a nervous system and muscles similar in complexity to those of jellyfish, anemones, sea stars and many types of worms and shellfish, cobbled together from an alternative set of genes.
Moroz tried to publish his results in 2009. The paper was rejected. And so he continued doing more experiments.
Even as Moroz firmed up his results through the late 2000s, other research teams were beginning to piece together bits of what he already knew – raising the worrying prospect that, after so many years, someone else might arrive at his conclusions before he had a chance to publish them himself.
First, a study in Nature in 2008 called into question the basic structure of the animal tree of life, undermining the long-held assumption that sponges were the first, most primitive branch. That study compared the DNA sequences of 150 genes in order to reconstruct the evolutionary relationships of 77 different animal species – including two species of ctenophores. For the first time ever, this paper publicly suggested that intricate ctenophores – and not simple sponges – might actually be the earliest branch. The mere suggestion of this created ‘a firestorm’ in the scientific community, says Steven Haddock, a biologist with the Monterey Bay Aquarium Research Institute who co-authored that study.
In December 2013, another team published the first-ever genome of a ctenophore – a species called Mnemiopsis leidyi, separate from the one that Moroz has studied the most. That paper, published in Science, also concluded that ctenophores, not sponges, were the evolutionary branch closest to the origin of all animals.
Despite being more complex than sponges, ctenophores seemed to be closest to the origin of all animals
Over the next few months, the deeply rooted narrative that sponges were the earliest animals continued to fall apart in other ways. In January 2014, Sally Leys, one of the world’s leading sponge biologists, based at the University of Alberta in Edmonton, called into question the 150-year-old narrative that sponges were more or less just a colonial version of single-celled organisms thought to be ancestors of all animals. Detailed studies showed that the sponge and the cells of a protist called a choanoflagellate used a different set of genes and proteins to build similar-looking structures. Therefore, sponges could not have evolved from anything resembling a choanoflagellate. Their similarity under a microscope was yet another deceptive example of convergent evolution: two unrelated organisms evolving similar structures to perform similar functions – but using different genes as starting materials.
These studies blew apart the circumstantial evidence that sponges were the earliest branch of the animal tree of life. What had seemed like a strong argument was simply a case of mistaken identity. Despite being far more complex than sponges, with nervous systems, muscles and other organs, ctenophores now seemed to be the earliest branch, closest to the origin of all animals.
But none of those studies had looked at nerve cells in any detail. So the broader world still didn’t know the core of Moroz’s discovery: the separately invented nervous system.
Moroz spent the intervening years filling the gaps in his evidence. His team slowly sequenced the last several percent of his own Pleurobrachia ctenophore genome, slogging through difficult stretches of DNA that gummed up even modern technologies. Moroz hired three dozen students to do detailed studies of what genes were expressed in the individual nerve cells of the ctenophores, and how these cells wired themselves into circuits as the animal developed from an embryo.
Moroz finally published his genome of the ctenophore Pleurobrachia in Nature, in June 2014. His work, seven years in the making, firmly established that the ctenophore’s nerve cells and nervous system had evolved separately from those of all other animals. To him, the ctenophore represented the closest thing to an alien brain, or mind, on Earth.
Ctenophores provide an extreme, striking example of what is probably a general pattern: just as eyes, wings and fins evolved many times over the course of animal evolution, so too have nerve cells. Moroz now counts nine to 12 independent evolutionary origins of the nervous system – including at least one in cnidaria (the group that includes jellyfish and anemones), three in echinoderms (the group that includes sea stars, sea lilies, urchins and sand dollars), one in arthropods (the group that includes insects, spiders and crustaceans), one in molluscs (the group that includes clams, snails, squid and octopuses), one in vertebrates – and now, at least one in ctenophores.
‘There is more than one way to make a neuron, more than one way to make a brain,’ says Moroz. In each of these evolutionary branches, a different subset of genes, proteins and molecules was blindly chosen, through random gene duplication and mutation, to take part in building a nervous system.
What’s fascinating is how these different pathways of evolution arrived at nervous systems that look so similar across the animal tree of life. Take for example the work of Nicholas Strausfeld, a neuro-anatomist at the University of Arizona in Tucson. He and others have found that the neural circuits underlying smell, episodic memory, spatial navigation, behaviour choice and vision in insects are nearly identical to those performing the same functions in mammals – despite the fact that different, though overlapping, sets of genes were harnessed to build each one.
These similarities reflect two key principles of evolution, factors that are probably important on any world where life has emerged. The first is convergence: these far-flung branches of the evolutionary tree arrived at common designs for a nervous system because they each had to solve the same fundamental problems. The second is shared history: the idea that all of these differently built nervous systems shared at least some element of common origin. On our world, they each evolved from molecular building blocks that were forged in the physical and chemical environments of early Earth.
In fact, much of the basic signalling machinery of all nervous systems might have evolved from a life-or-death adaptation that arose in the first cells on Earth, four billion years ago. Early cells probably inhabited aquatic environments, such as hot springs or brine pools, that contained a mixture of dissolved minerals including some, like calcium, that threatened life. (Important biological molecules such as DNA, RNA and ATP are known to coalesce into refractory goo when exposed to calcium – similar to the scum that forms in bathtubs.) So biologists surmise that early life must have evolved ways to keep all but the lowest levels of calcium outside its cells. This protective machinery might include proteins that pump calcium atoms out of a cell, and an alarm system that goes off when calcium levels rise. Evolution later harnessed this exquisite responsiveness to calcium to signal within and between cells – to control the beating of cilia and flagella that microbes use to move, or to control the contraction of muscle cells or trigger the electric firing of neurons in organisms such as ours. By the time nervous systems began to emerge, roughly half a billion years ago, many of the critical building blocks were already set.
If the history of Earth was rewound, evolution might not arrive at 2017 with the same animal groups we see today
These principles have huge implications for understanding evolution, and understanding the forms that life might take on Earth or in other worlds. They shed light on the relative importance of accident and destiny in shaping the trajectory of evolution over billions of years.
The late Harvard palaeontologist Stephen Jay Gould argued in his book Wonderful Life (1989) that accidents matter: that the evolutionary history of animals has been shaped by decimation as much as by innovation. He pointed out that the Cambrian world 570 million years ago contained more groups of animals, called phyla, than exist today. Those diverse branches in the early animal tree were steadily pruned by mass extinctions. Those extinctions fuelled evolution by opening ecological niches that surviving animal groups could diversify into – providing opportunity for innovation.
At the same time, Simon Conway Morris, a palaeontologist at the University of Cambridge, has stressed the importance of evolutionary convergence: that evolution tends to arrive at the same solutions over and over again, even in distant branches of the animal tree, and even when the proteins or genes used to build a similar structure are not themselves related.
Take these two ideas to their logical ends, and one arrives at a startling conclusion. If the history of Earth was rewound and played back, evolution might not arrive at this present year with the same assortment of animal groups that we see today. Mammals or birds, perhaps even all vertebrates, might be absent. But evolution might still arrive at most, or even all, of the same innovations that permitted the emergence of sophisticated brains: those innovations might simply emerge on other branches of the animal tree.
As scientists speculate what kind of life might exist on other worlds, a provocative idea is taking hold: that alien life, unlike anything we know, might already exist here on Earth. The idea is that life might have arisen two or more times on our planet – not just once, as long assumed. Our form of life came to dominate, while other forms receded into the corners. This ‘shadow biosphere’ would be difficult to detect, since it might not contain DNA, proteins or the other molecules that we rely on to detect life.
The phylum of ctenophores isn’t quite that exotic. It is based on the same basic chemistry that we share, but it still represents a shadow biology for animals. Ctenophores are a long-lost cousin that we didn’t even know we had.
Because the ctenophore invented brains and muscles using a set of proteins and genes so different from any other animal that has ever been studied, it provides a unique opportunity to explore some enormous questions: how divergent can nervous systems be? Do we truly understand how life senses its surroundings and behaves?
The ctenophore could even provide useful insights for predicting how nervous systems might evolve in other worlds, in more exotic life forms not based on DNA or proteins. Evolutionary biologists believe that even life based on exotic biochemistry will still tend to be built along similar lines of organisation. Nick Lane, a biochemist at University College London, has written that extraterrestrial life probably compartmentalises itself within some sort of cell membrane, and powers itself using electrochemical differences in the pH or ion concentrations from one side of the membrane to the other, just like cells on Earth. Chemicals extracted from ancient meteorites can readily form membranes – even if those membranes aren’t composed of the exact same molecules. And once cell membranes become fixed in the biology of another world, the process of evolving a nervous system will likely unfold in a manner similar to that seen on Earth.
Moroz is still trying to learn what he can from the ctenophores. These animals were neglected for so long by scientists, in part, because they were so fragile and difficult to keep alive in the lab. Moroz is circumventing this by outfitting a ship with modern research equipment for sequencing genomes, growing embryos, and stimulating neurons in living animals on site. He hopes that by teasing apart the neural circuits of the ctenophore, he can learn more about the design principles of brains in general – and test whether those principles really are universal, or not.
Just getting to this point has been a long process. In order to realise that ctenophores really were so alien, Moroz first had to reject much of what he had learned from researchers who came before. Because his ‘initial hypothesis was exactly what was in the textbooks’, he explains, moving to a new way of thinking took him 20 years.
Brains, beyond their signature achievements in thinking and problem solving, are paragons of energy efficiency. The human brain’s power consumption resembles that of a 20-watt incandescent lightbulb. In contrast, one of the world’s largest and fastest supercomputers, the K computer in Kobe, Japan, consumes as much as 9.89 megawatts of energy — an amount roughly equivalent to the power usage of 10,000 households. Yet in 2013, even with that much power, it took the machine 40 minutes to simulate just a single second’s worth of 1 percent of human brain activity.
Now engineering researchers at the California NanoSystems Institute at the University of California, Los Angeles, are hoping to match some of the brain’s computational and energy efficiency with systems that mirror the brain’s structure. They are building a device, perhaps the first one, that is “inspired by the brain to generate the properties that enable the brain to do what it does,” according to Adam Stieg, a research scientist and associate director of the institute, who leads the project with Jim Gimzewski, a professor of chemistry at UCLA.
The device is a far cry from conventional computers, which are based on minute wires imprinted on silicon chips in highly ordered patterns. The current pilot version is a 2-millimeter-by-2-millimeter mesh of silver nanowires connected by artificial synapses. Unlike silicon circuitry, with its geometric precision, this device is messy, like “a highly interconnected plate of noodles,” Stieg said. And instead of being designed, the fine structure of the UCLA device essentially organized itself out of random chemical and electrical processes.
Yet in its complexity, this silver mesh network resembles the brain. The mesh boasts 1 billion artificial synapses per square centimeter, which is within a couple of orders of magnitude of the real thing. The network’s electrical activity also displays a property unique to complex systems like the brain: “criticality,” a state between order and chaos indicative of maximum efficiency.
Moreover, preliminary experiments suggest that this neuromorphic (brainlike) silver wire mesh has great functional potential. It can already perform simple learning and logic operations. It can clean the unwanted noise from received signals, a capability that’s important for voice recognition and similar tasks that challenge conventional computers. And its existence proves the principle that it might be possible one day to build devices that can compute with an energy efficiency close to that of the brain.
These advantages look especially appealing as the limits of miniaturization and efficiency for silicon microprocessors now loom. “Moore’s law is dead, transistors are no longer getting smaller, and [people] are going, ‘Oh, my God, what do we do now?’” said Alex Nugent, CEO of the Santa Fe-based neuromorphic computing company Knowm, who was not involved in the UCLA project. “I’m very excited about the idea, the direction of their work,” Nugent said. “Traditional computing platforms are a billion times less efficient.”
Switches That Act Like Synapses
Energy efficiency wasn’t Gimzewski’s motivation when he started the silver wire project 10 years ago. Rather, it was boredom. After using scanning tunneling microscopes to look at electronics at the atomic scale for 20 years, he said, “I was tired of perfection and precise control [and] got a little bored with reductionism.”
In 2007, he accepted an invitation to study single atomic switches developed by a group that Masakazu Aono led at the International Center for Materials Nanoarchitectonics in Tsukuba, Japan. The switches contain the same ingredient that turns a silver spoon black when it touches an egg: silver sulfide, sandwiched between solid metallic silver.
Applying voltage to the devices pushes positively charged silver ions out of the silver sulfide and toward the silver cathode layer, where they are reduced to metallic silver. Atom-wide filaments of silver grow, eventually closing the gap between the metallic silver sides. As a result, the switch is on and current can flow. Reversing the current flow has the opposite effect: The silver bridges shrink, and the switch turns off.
Soon after developing the switch, however, Aono’s group started to see irregular behavior. The more often the switch was used, the more easily it would turn on. If it went unused for a while, it would slowly turn off by itself. In effect, the switch remembered its history. Aono and his colleagues also found that the switches seemed to interact with each other, such that turning on one switch would sometimes inhibit or turn off others nearby.
Most of Aono’s group wanted to engineer these odd properties out of the switches. But Gimzewski and Stieg (who had just finished his doctorate in Gimzewski’s group) were reminded of synapses, the switches between nerve cells in the human brain, which also change their responses with experience and interact with each other. During one of their many visits to Japan, they had an idea. “We thought: Why don’t we try to embed them in a structure reminiscent of the cortex in a mammalian brain [and study that]?” Stieg said.
Building such an intricate structure was a challenge, but Stieg and Audrius Avizienis, who had just joined the group as a graduate student, developed a protocol to do it. By pouring silver nitrate onto tiny copper spheres, they could induce a network of microscopically thin intersecting silver wires to grow. They could then expose the mesh to sulfur gas to create a silver sulfide layer between the silver wires, as in the Aono team’s original atomic switch.
Self-Organized Criticality
When Gimzewski and Stieg told others about their project, almost nobody thought it would work. Some said the device would show one type of static activity and then sit there, Stieg recalled. Others guessed the opposite: “They said the switching would cascade and the whole thing would just burn out,” Gimzewski said.
But the device did not melt. Rather, as Gimzewski and Stieg observed through an infrared camera, the input current kept changing the paths it followed through the device — proof that activity in the network was not localized but rather distributed, as it is in the brain.
Then, one fall day in 2010, while Avizienis and his fellow graduate student Henry Sillin were increasing the input voltage to the device, they suddenly saw the output voltage start to fluctuate, seemingly at random, as if the mesh of wires had come alive. “We just sat and watched it, fascinated,” Sillin said.
They knew they were on to something. When Avizienis analyzed several days’ worth of monitoring data, he found that the network stayed at the same activity level for short periods more often than for long periods. They later found that smaller areas of activity were more common than larger ones.
“That was really jaw-dropping,” Avizienis said, describing it as “the first [time] we pulled a power law out of this.” Power laws describe mathematical relationships in which one variable changes as a power of the other. They apply to systems in which larger scale, longer events are much less common than smaller scale, shorter ones — but are also still far more common than one would expect from a chance distribution. Per Bak, the Danish physicist who died in 2002, first proposed power laws as hallmarks of all kinds of complex dynamical systems that can organize over large timescales and long distances. Power-law behavior, he said, indicates that a complex system operates at a dynamical sweet spot between order and chaos, a state of “criticality” in which all parts are interacting and connected for maximum efficiency.
As Bak predicted, power-law behavior has been observed in the human brain: In 2003, Dietmar Plenz, a neuroscientist with the National Institutes of Health, observed that groups of nerve cells activated others, which in turn activated others, often forming systemwide activation cascades. Plenz found that the sizes of these cascades fell along a power-law distribution, and that the brain was indeed operating in a way that maximized activity propagation without risking runaway activity.
The fact that the UCLA device also shows power-law behavior is a big deal, Plenz said, because it suggests that, as in the brain, a delicate balance between activation and inhibition keeps all of its parts interacting with one another. The activity doesn’t overwhelm the network, but it also doesn’t die out.
Gimzewski and Stieg later found an additional similarity between the silver network and the brain: Just as a sleeping human brain shows fewer short activation cascades than a brain that’s awake, brief activation states in the silver network become less common at lower energy inputs. In a way, then, reducing the energy input into the device can generate a state that resembles the sleeping state of the human brain.
Training and Reservoir Computing
But even if the silver wire network has brainlike properties, can it solve computing tasks? Preliminary experiments suggest the answer is yes, although the device is far from resembling a traditional computer.
For one thing, there is no software. Instead, the researchers exploit the fact that the network can distort an input signal in many different ways, depending on where the output is measured. This suggests possible uses for voice or image recognition, because the device should be able to clean a noisy input signal.
But it also suggests that the device could be used for a process called reservoir computing. Because one input could in principle generate many, perhaps millions, of different outputs (the “reservoir”), users can choose or combine outputs in such a way that the result is a desired computation of the inputs. For example, if you stimulate the device at two different places at the same time, chances are that one of the millions of different outputs will represent the sum of the two inputs.
The challenge is to find the right outputs and decode them and to find out how best to encode information so that the network can understand it. The way to do this is by training the device: by running a task hundreds or perhaps thousands of times, first with one type of input and then with another, and comparing which output best solves a task. “We don’t program the device but we select the best way to encode the information such that the [network behaves] in an interesting and useful manner,” Gimzewski said.
In work that’s soon to be published, the researchers trained the wire network to execute simple logic operations. And in unpublished experiments, they trained the network to solve the equivalent of a simple memory task taught to lab rats called a T-maze test. In the test, a rat in a T-shaped maze is rewarded when it learns to make the correct turn in response to a light. With its own version of training, the network could make the correct response 94 percent of the time.
So far, these results aren’t much more than a proof of principle, Nugent said. “A little rat making a decision in a T-maze is nowhere close to what somebody in machine learning does to evaluate their systems” on a traditional computer, he said. He doubts the device will lead to a chip that does much that’s useful in the next few years.
But the potential, he emphasized, is huge. That’s because the network, like the brain, doesn’t separate processing and memory. Traditional computers need to shuttle information between different areas that handle the two functions. “All that extra communication adds up because it takes energy to charge wires,” Nugent said. With traditional machines, he said, “literally, you could run France on the electricity that it would take to simulate a full human brain at moderate resolution.” If devices like the silver wire network can eventually solve tasks as effectively as machine-learning algorithms running on traditional computers, they could do so using only one-billionth as much power. “As soon as they do that, they’re going to win in power efficiency, hands down,” Nugent said.
The UCLA findings also lend support to the view that under the right circumstances, intelligent systems can form by self-organization, without the need for any template or process to design them. The silver network “emerged spontaneously,” said Todd Hylton, the former manager of the Defense Advanced Research Projects Agency program that supported early stages of the project. “As energy flows through [it], it’s this big dance because every time one new structure forms, the energy doesn’t go somewhere else. People have built computer models of networks that achieve some critical state. But this one just sort of did it all by itself.”
Gimzewski believes that the silver wire network or devices like it might be better than traditional computers at making predictions about complex processes. Traditional computers model the world with equations that often only approximate complex phenomena. Neuromorphic atomic switch networks align their own innate structural complexity with that of the phenomenon they are modeling. They are also inherently fast — the state of the network can fluctuate at upward of tens of thousands of changes per second. “We are using a complex system to understand complex phenomena,” Gimzewski said.
Earlier this year at a meeting of the American Chemical Society in San Francisco, Gimzewski, Stieg and their colleagues presented the results of an experiment in which they fed the device the first three years of a six-year data set of car traffic in Los Angeles, in the form of a series of pulses that indicated the number of cars passing by per hour. After hundreds of training runs, the output eventually predicted the statistical trend of the second half of the data set quite well, even though the device had never seen it.
Perhaps one day, Gimzewski jokes, he might be able to use the network to predict the stock market. “I’d like that,” he said, adding that this was why he was trying to get his students to study atomic switch networks — “before they catch me making a fortune.”
I’m standing in what is soon to be the center of the world, or is perhaps just a very large room on the seventh floor of a gleaming tower in downtown Toronto. Showing me around is Jordan Jacobs, who cofounded this place: the nascent Vector Institute, which opens its doors this fall and which is aiming to become the global epicenter of artificial intelligence.
We’re in Toronto because Geoffrey Hinton is in Toronto, and Geoffrey Hinton is the father of “deep learning,” the technique behind the current excitement about AI. “In 30 years we’re going to look back and say Geoff is Einstein—of AI, deep learning, the thing that we’re calling AI,” Jacobs says. Of the researchers at the top of the field of deep learning, Hinton has more citations than the next three combined. His students and postdocs have gone on to run the AI labs at Apple, Facebook, and OpenAI; Hinton himself is a lead scientist on the Google Brain AI team. In fact, nearly every achievement in the last decade of AI—in translation, speech recognition, image recognition, and game playing—traces in some way back to Hinton’s work.
The Vector Institute, this monument to the ascent of Hinton’s ideas, is a research center where companies from around the U.S. and Canada—like Google, and Uber, and Nvidia—will sponsor efforts to commercialize AI technologies. Money has poured in faster than Jacobs could ask for it; two of his cofounders surveyed companies in the Toronto area, and the demand for AI experts ended up being 10 times what Canada produces every year. Vector is in a sense ground zero for the now-worldwide attempt to mobilize around deep learning: to cash in on the technique, to teach it, to refine and apply it. Data centers are being built, towers are being filled with startups, a whole generation of students is going into the field.
The impression you get standing on the Vector floor, bare and echoey and about to be filled, is that you’re at the beginning of something. But the peculiar thing about deep learning is just how old its key ideas are. Hinton’s breakthrough paper, with colleagues David Rumelhart and Ronald Williams, was published in 1986. The paper elaborated on a technique called backpropagation, or backprop for short. Backprop, in the words of Jon Cohen, a computational psychologist at Princeton, is “what all of deep learning is based on—literally everything.”
When you boil it down, AI today is deep learning, and deep learning is backprop—which is amazing, considering that backprop is more than 30 years old. It’s worth understanding how that happened—how a technique could lie in wait for so long and then cause such an explosion—because once you understand the story of backprop, you’ll start to understand the current moment in AI, and in particular the fact that maybe we’re not actually at the beginning of a revolution. Maybe we’re at the end of one.
Vindication
The walk from the Vector Institute to Hinton’s office at Google, where he spends most of his time (he is now an emeritus professor at the University of Toronto), is a kind of living advertisement for the city, at least in the summertime. You can understand why Hinton, who is originally from the U.K., moved here in the 1980s after working at Carnegie Mellon University in Pittsburgh.
When you step outside, even downtown near the financial district, you feel as though you’ve actually gone into nature. It’s the smell, I think: wet loam in the air. Toronto was built on top of forested ravines, and it’s said to be “a city within a park”; as it’s been urbanized, the local government has set strict restrictions to maintain the tree canopy. As you’re flying in, the outer parts of the city look almost cartoonishly lush.
Maybe we’re not actually at the beginning of a revolution.
Toronto is the fourth-largest city in North America (after Mexico City, New York, and L.A.), and its most diverse: more than half the population was born outside Canada. You can see that walking around. The crowd in the tech corridor looks less San Francisco—young white guys in hoodies—and more international. There’s free health care and good public schools, the people are friendly, and the political order is relatively left-leaning and stable; and this stuff draws people like Hinton, who says he left the U.S. because of the Iran-Contra affair. It’s one of the first things we talk about when I go to meet him, just before lunch.
“Most people at CMU thought it was perfectly reasonable for the U.S. to invade Nicaragua,” he says. “They somehow thought they owned it.” He tells me that he had a big breakthrough recently on a project: “getting a very good junior engineer who’s working with me,” a woman named Sara Sabour. Sabour is Iranian, and she was refused a visa to work in the United States. Google’s Toronto office scooped her up.
Hinton, who is 69 years old, has the kind, lean, English-looking face of the Big Friendly Giant, with a thin mouth, big ears, and a proud nose. He was born in Wimbledon, England, and sounds, when he talks, like the narrator of a children’s book about science: curious, engaging, eager to explain things. He’s funny, and a bit of a showman. He stands the whole time we talk, because, as it turns out, sitting is too painful. “I sat down in June of 2005 and it was a mistake,” he tells me, letting the bizarre line land before explaining that a disc in his back gives him trouble. It means he can’t fly, and earlier that day he’d had to bring a contraption that looked like a surfboard to the dentist’s office so he could lie on it while having a cracked tooth root examined.
In the 1980s Hinton was, as he is now, an expert on neural networks, a much-simplified model of the network of neurons and synapses in our brains. However, at that time it had been firmly decided that neural networks were a dead end in AI research. Although the earliest neural net, the Perceptron, which began to be developed in the 1950s, had been hailed as a first step toward human-level machine intelligence, a 1969 book by MIT’s Marvin Minsky and Seymour Papert, called Perceptrons, proved mathematically that such networks could perform only the most basic functions. These networks had just two layers of neurons, an input layer and an output layer. Nets with more layers between the input and output neurons could in theory solve a great variety of problems, but nobody knew how to train them, and so in practice they were useless. Except for a few holdouts like Hinton, Perceptrons caused most people to give up on neural nets entirely.
Hinton’s breakthrough, in 1986, was to show that backpropagation could train a deep neural net, meaning one with more than two or three layers. But it took another 26 years before increasing computational power made good on the discovery. A 2012 paper by Hinton and two of his Toronto students showed that deep neural nets, trained using backpropagation, beat state-of-the-art systems in image recognition. “Deep learning” took off. To the outside world, AI seemed to wake up overnight. For Hinton, it was a payoff long overdue.
Reality distortion field
A neural net is usually drawn like a club sandwich, with layers stacked one atop the other. The layers contain artificial neurons, which are dumb little computational units that get excited—the way a real neuron gets excited—and pass that excitement on to the other neurons they’re connected to. A neuron’s excitement is represented by a number, like 0.13 or 32.39, that says just how excited it is. And there’s another crucial number, on each of the connections between two neurons, that determines how much excitement should get passed from one to the other. That number is meant to model the strength of the synapses between neurons in the brain. When the number is higher, it means the connection is stronger, so more of the one’s excitement flows to the other.
A diagram from seminal work on “error propagation” by Hinton, David Rumelhart, and Ronald Williams.
One of the most successful applications of deep neural nets is in image recognition—as in the memorable scene in HBO’s Silicon Valley where the team builds a program that can tell whether there’s a hot dog in a picture. Programs like that actually exist, and they wouldn’t have been possible a decade ago. To get them to work, the first step is to get a picture. Let’s say, for simplicity, it’s a small black-and-white image that’s 100 pixels wide and 100 pixels tall. You feed this image to your neural net by setting the excitement of each simulated neuron in the input layer so that it’s equal to the brightness of each pixel. That’s the bottom layer of the club sandwich: 10,000 neurons (100x100) representing the brightness of every pixel in the image.
You then connect this big layer of neurons to another big layer of neurons above it, say a few thousand, and these in turn to another layer of another few thousand neurons, and so on for a few layers. Finally, in the topmost layer of the sandwich, the output layer, you have just two neurons—one representing “hot dog” and the other representing “not hot dog.” The idea is to teach the neural net to excite only the first of those neurons if there’s a hot dog in the picture, and only the second if there isn’t. Backpropagation—the technique that Hinton has built his career upon—is the method for doing this.
Backprop is remarkably simple, though it works best with huge amounts of data. That’s why big data is so important in AI—why Facebook and Google are so hungry for it, and why the Vector Institute decided to set up shop down the street from four of Canada’s largest hospitals and develop data partnerships with them.
In this case, the data takes the form of millions of pictures, some with hot dogs and some without; the trick is that these pictures are labeled as to which have hot dogs. When you first create your neural net, the connections between neurons might have random weights—random numbers that say how much excitement to pass along each connection. It’s as if the synapses of the brain haven’t been tuned yet. The goal of backprop is to change those weights so that they make the network work: so that when you pass in an image of a hot dog to the lowest layer, the topmost layer’s “hot dog” neuron ends up getting excited.
Suppose you take your first training image, and it’s a picture of a piano. You convert the pixel intensities of the 100x100 picture into 10,000 numbers, one for each neuron in the bottom layer of the network. As the excitement spreads up the network according to the connection strengths between neurons in adjacent layers, it’ll eventually end up in that last layer, the one with the two neurons that say whether there’s a hot dog in the picture. Since the picture is of a piano, ideally the “hot dog” neuron should have a zero on it, while the “not hot dog” neuron should have a high number. But let’s say it doesn’t work out that way. Let’s say the network is wrong about this picture. Backprop is a procedure for rejiggering the strength of every connection in the network so as to fix the error for a given training example.
The way it works is that you start with the last two neurons, and figure out just how wrong they were: how much of a difference is there between what the excitement numbers should have been and what they actually were? When that’s done, you take a look at each of the connections leading into those neurons—the ones in the next lower layer—and figure out their contribution to the error. You keep doing this until you’ve gone all the way to the first set of connections, at the very bottom of the network. At that point you know how much each individual connection contributed to the overall error, and in a final step, you change each of the weights in the direction that best reduces the error overall. The technique is called “backpropagation” because you are “propagating” errors back (or down) through the network, starting from the output.
The incredible thing is that when you do this with millions or billions of images, the network starts to get pretty good at saying whether an image has a hot dog in it. And what’s even more remarkable is that the individual layers of these image-recognition nets start being able to “see” images in sort of the same way our own visual system does. That is, the first layer might end up detecting edges, in the sense that its neurons get excited when there are edges and don’t get excited when there aren’t; the layer above that one might be able to detect sets of edges, like corners; the layer above that one might start to see shapes; and the layer above that one might start finding stuff like “open bun” or “closed bun,” in the sense of having neurons that respond to either case. The net organizes itself, in other words, into hierarchical layers without ever having been explicitly programmed that way.
A real intelligence doesn’t break when you slightly change the problem.
This is the thing that has everybody enthralled. It’s not just that neural nets are good at classifying pictures of hot dogs or whatever: they seem able to build representations of ideas. With text you can see this even more clearly. You can feed the text of Wikipedia, many billions of words long, into a simple neural net, training it to spit out, for each word, a big list of numbers that correspond to the excitement of each neuron in a layer. If you think of each of these numbers as a coordinate in a complex space, then essentially what you’re doing is finding a point, known in this context as a vector, for each word somewhere in that space. Now, train your network in such a way that words appearing near one another on Wikipedia pages end up with similar coordinates, and voilà, something crazy happens: words that have similar meanings start showing up near one another in the space. That is, “insane” and “unhinged” will have coordinates close to each other, as will “three” and “seven,” and so on. What’s more, so-called vector arithmetic makes it possible to, say, subtract the vector for “France” from the vector for “Paris,” add the vector for “Italy,” and end up in the neighborhood of “Rome.” It works without anyone telling the network explicitly that Rome is to Italy as Paris is to France.
“It’s amazing,” Hinton says. “It’s shocking.” Neural nets can be thought of as trying to take things—images, words, recordings of someone talking, medical data—and put them into what mathematicians call a high-dimensional vector space, where the closeness or distance of the things reflects some important feature of the actual world. Hinton believes this is what the brain itself does. “If you want to know what a thought is,” he says, “I can express it for you in a string of words. I can say ‘John thought, “Whoops.”’ But if you ask, ‘What is the thought? What does it mean for John to have that thought?’ It’s not that inside his head there’s an opening quote, and a ‘Whoops,’ and a closing quote, or even a cleaned-up version of that. Inside his head there’s some big pattern of neural activity.” Big patterns of neural activity, if you’re a mathematician, can be captured in a vector space, with each neuron’s activity corresponding to a number, and each number to a coordinate of a really big vector. In Hinton’s view, that’s what thought is: a dance of vectors.
Geoffrey Hinton
It is no coincidence that Toronto’s flagship AI institution was named for this fact. Hinton was the one who came up with the name Vector Institute.
There’s a sort of reality distortion field that Hinton creates, an air of certainty and enthusiasm, that gives you the feeling there’s nothing that vectors can’t do. After all, look at what they’ve been able to produce already: cars that drive themselves, computers that detect cancer, machines that instantly translate spoken language. And look at this charming British scientist talking about gradient descent in high-dimensional spaces!
It’s only when you leave the room that you remember: these “deep learning” systems are still pretty dumb, in spite of how smart they sometimes seem. A computer that sees a picture of a pile of doughnuts piled up on a table and captions it, automatically, as “a pile of doughnuts piled on a table” seems to understand the world; but when that same program sees a picture of a girl brushing her teeth and says “The boy is holding a baseball bat,” you realize how thin that understanding really is, if ever it was there at all.
Neural nets are just thoughtless fuzzy pattern recognizers, and as useful as fuzzy pattern recognizers can be—hence the rush to integrate them into just about every kind of software—they represent, at best, a limited brand of intelligence, one that is easily fooled. A deep neural net that recognizes images can be totally stymied when you change a single pixel, or add visual noise that’s imperceptible to a human. Indeed, almost as often as we’re finding new ways to apply deep learning, we’re finding more of its limits. Self-driving cars can fail to navigate conditions they’ve never seen before. Machines have trouble parsing sentences that demand common-sense understanding of how the world works.
Deep learning in some ways mimics what goes on in the human brain, but only in a shallow way—which perhaps explains why its intelligence can sometimes seem so shallow. Indeed, backprop wasn’t discovered by probing deep into the brain, decoding thought itself; it grew out of models of how animals learn by trial and error in old classical-conditioning experiments. And most of the big leaps that came about as it developed didn’t involve some new insight about neuroscience; they were technical improvements, reached by years of mathematics and engineering. What we know about intelligence is nothing against the vastness of what we still don’t know.
David Duvenaud, an assistant professor in the same department as Hinton at the University of Toronto, says deep learning has been somewhat like engineering before physics. “Someone writes a paper and says, ‘I made this bridge and it stood up!’ Another guy has a paper: ‘I made this bridge and it fell down—but then I added pillars, and then it stayed up.’ Then pillars are a hot new thing. Someone comes up with arches, and it’s like, ‘Arches are great!’” With physics, he says, “you can actually understand what’s going to work and why.” Only recently, he says, have we begun to move into that phase of actual understanding with artificial intelligence.
Hinton himself says, “Most conferences consist of making minor variations … as opposed to thinking hard and saying, ‘What is it about what we’re doing now that’s really deficient? What does it have difficulty with? Let’s focus on that.’”
It can be hard to appreciate this from the outside, when all you see is one great advance touted after another. But the latest sweep of progress in AI has been less science than engineering, even tinkering. And though we’ve started to get a better handle on what kinds of changes will improve deep-learning systems, we’re still largely in the dark about how those systems work, or whether they could ever add up to something as powerful as the human mind.
It’s worth asking whether we’ve wrung nearly all we can out of backprop. If so, that might mean a plateau for progress in artificial intelligence.
Patience
If you want to see the next big thing, something that could form the basis of machines with a much more flexible intelligence, you should probably check out research that resembles what you would’ve found had you encountered backprop in the ’80s: smart people plugging away on ideas that don’t really work yet.
A few months ago I went to the Center for Minds, Brains, and Machines, a multi-institutional effort headquartered at MIT, to watch a friend of mine, Eyal Dechter, defend his dissertation in cognitive science. Just before the talk started, his wife Amy, their dog Ruby, and their daughter Susannah were milling around, wishing him well. On the screen was a picture of Ruby, and next to it one of Susannah as a baby. When Dad asked Susannah to point herself out, she happily slapped a long retractable pointer against her own baby picture. On the way out of the room, she wheeled a toy stroller behind her mom and yelled “Good luck, Daddy!” over her shoulder. “Vámanos!” she said finally. She’s two.
“The fact that it doesn’t work is just a temporary annoyance.”
Eyal started his talk with a beguiling question: How is it that Susannah, after two years of experience, can learn to talk, to play, to follow stories? What is it about the human brain that makes it learn so well? Will a computer ever be able to learn so quickly and so fluidly?
We make sense of new phenomena in terms of things we already understand. We break a domain down into pieces and learn the pieces. Eyal is a mathematician and computer programmer, and he thinks about tasks—like making a soufflé—as really complex computer programs. But it’s not as if you learn to make a soufflé by learning every one of the program’s zillion micro-instructions, like “Rotate your elbow 30 degrees, then look down at the countertop, then extend your pointer finger, then …” If you had to do that for every new task, learning would be too hard, and you’d be stuck with what you already know. Instead, we cast the program in terms of high-level steps, like “Whip the egg whites,” which are themselves composed of subprograms, like “Crack the eggs” and “Separate out the yolks.”
Computers don’t do this, and that is a big part of the reason they’re dumb. To get a deep-learning system to recognize a hot dog, you might have to feed it 40 million pictures of hot dogs. To get Susannah to recognize a hot dog, you show her a hot dog. And before long she’ll have an understanding of language that goes deeper than recognizing that certain words often appear together. Unlike a computer, she’ll have a model in her mind about how the whole world works. “It’s sort of incredible to me that people are scared of computers taking jobs,” Eyal says. “It’s not that computers can’t replace lawyers because lawyers do really complicated things. It’s because lawyers read and talk to people. It’s not like we’re close. We’re so far.”
A real intelligence doesn’t break when you slightly change the requirements of the problem it’s trying to solve. And the key part of Eyal’s thesis was his demonstration, in principle, of how you might get a computer to work that way: to fluidly apply what it already knows to new tasks, to quickly bootstrap its way from knowing almost nothing about a new domain to being an expert.
Hinton made this sketch for his next big idea, to organize neural nets with "capsules."
Essentially, it is a procedure he calls the “exploration–compression” algorithm. It gets a computer to function somewhat like a programmer who builds up a library of reusable, modular components on the way to building more and more complex programs. Without being told anything about a new domain, the computer tries to structure knowledge about it just by playing around, consolidating what it’s found, and playing around some more, the way a human child does.
His advisor, Joshua Tenenbaum, is one of the most highly cited researchers in AI. Tenenbaum’s name came up in half the conversations I had with other scientists. Some of the key people at DeepMind—the team behind AlphaGo, which shocked computer scientists by beating a world champion player in the complex game of Go in 2016—had worked as his postdocs. He’s involved with a startup that’s trying to give self-driving cars some intuition about basic physics and other drivers’ intentions, so they can better anticipate what would happen in a situation they’ve never seen before, like when a truck jackknifes in front of them or when someone tries to merge very aggressively.
Eyal’s thesis doesn’t yet translate into those kinds of practical applications, let alone any programs that would make headlines for besting a human. The problems Eyal’s working on “are just really, really hard,” Tenenbaum said. “It’s gonna take many, many generations.”
Tenenbaum has long, curly, whitening hair, and when we sat down for coffee he had on a button-down shirt with black slacks. He told me he looks to the story of backprop for inspiration. For decades, backprop was cool math that didn’t really accomplish anything. As computers got faster and the engineering got more sophisticated, suddenly it did. He hopes the same thing might happen with his own work and that of his students, “but it might take another couple decades.”
As for Hinton, he is convinced that overcoming AI’s limitations involves building “a bridge between computer science and biology.” Backprop was, in this view, a triumph of biologically inspired computation; the idea initially came not from engineering but from psychology. So now Hinton is trying to pull off a similar trick.
Neural networks today are made of big flat layers, but in the human neocortex real neurons are arranged not just horizontally into layers but vertically into columns. Hinton thinks he knows what the columns are for—in vision, for instance, they’re crucial for our ability to recognize objects even as our viewpoint changes. So he’s building an artificial version—he calls them “capsules”—to test the theory. So far, it hasn’t panned out; the capsules haven’t dramatically improved his nets’ performance. But this was the same situation he’d been in with backprop for nearly 30 years.
“This thing just has to be right,” he says about the capsule theory, laughing at his own boldness. “And the fact that it doesn’t work is just a temporary annoyance.”
James Somers is a writer and programmer based in New York City. His previous article for MIT Technology Review was “Toolkits for the Mind” in May/June 2015, which showed how Internet startups are shaped by the programming languages they use.
Order Now
