This is old news now but I am still disturbed we have just created a cow-matrix
Moscow-area farmers strapped modified VR headsets to cows to see if it improved their mood – and, of course, their milk production. The project subjected cattle to a simulated summer field with colors tuned for the animals’ eyes, giving them a decidedly more pleasing landscape than a plain, confining farm. And yes, the headsets were adapted to the “structural features” of cows’ heads so that they could see properly.
It appears to have worked, at least on a basic level. The first test reduced the cows’ anxiety and boosted their overall sentiment. While it’s not certain how well this affects the quality or volume of milk, there are plans for a more “comprehensive” study to answer that question.
In my opinion, one of the best applications of neural networks is for generating Halloween costumes. Thanks to a dataset of over 7,100 costumes crowdsourced from readers of this blog, I’ve been able to generate Halloween costumes with progressively more powerful neural networks. In 2017, I used char-rnn, which learned to generate costumes starting from no knowledge of English (Statue of Pizza, the Fail Witch, Spartan Gandalf, and Professor Panda were some of its inventions). In 2018, I used textgen-rnn, also training from scratch, and teamed up with the New York Times to illustrate the costumes (some of my favorites were Sexy Wizard and Ruth Bader Hat Guy).
GPT-2 is good at costumes. Many of its inventions could easily have come from the training data. In fact, the neural net did tend to memorize the training data and repeat it back to me - technically this is what I asked for when I asked it to predict the training data. (The neural net is trying to give me exactly what I ask for, which isn’t necessarily exactly what I want.) I was using a handy script to filter out duplicates (thanks to John Tebbutt), and even so I had to check several of these to make sure they weren’t near copies of the training data. My previous Halloween costume generators would not have been smart enough to come up with things like “jackalope” or “Carl Sagan”, but GPT-2 has seen these words used online in similar contexts to things that ARE in the training data, and it makes the connection.
Gothy Terminator jackalope vampire cat Eye of Sauron incognito llama space cow Vampire Rock Scooby Gadget a raised eyebrow Battle worm Mastodon Swamp girl Carl Sagan A space squirrel walking carpet Frizzle the witch Cleopatra on vacation gothy giraffe Sexy Lego Batman skeleton
Oh yes, the sexy characters. The neural net definitely picked that up from the training data, and innovated admirably, bringing in words that it knew from the internet (barnacle, groundhog, and bunsen burner were not in the list of Halloween costumes), and adding a sexy twist. This is impressive (if somewhat horrifying) work. None of these were in its training data, but I wouldn’t be surprised if some of them exist.
Burlesque Horse Sexy Bulldog Sexy Egg Sexy Parsley Sexy Barnacle Sexy Walrus sexy locust Sexy Titmouse Sexy Hummingbird Sweet Potato Burlesque Sexy Groundhog Sexy bitcoin Sexy DNA Sexy Rubber Duck Sexy Bunsen burner Butt-Monks Sexy gingerbread man Sexy Flying Dutchman Sexy Chimneywatcher Sexy Flames Of War Sexy English Tea Party
And the neural net was pretty good at designing identifiable characters, even if they are a bit on the weird side.
A spangled Auroch manatee M. Bison the Clown Prince of Darkness Gingerbread Man guinea pig Skin Fairy sentient stone fast food bald eagle Fairy root vegetable Ghost in a packet of potato chips cozy coconut Kelpie the mage Crochet monster Walrus rider Star skunk Slytherin AI priest A skunk in a moose suit Semi-molten Kool Aid Man Time Lord Power Ranger The Power Dinosaur Space Oystermonger Deadly Snow Monkey An evil cupcake basic plumber’s equine Spooky mother hen The Bozo the Destroyer Eight Ball of Wrath Ursula, Queen of the Fart Science A poker player in possession of an onion
There are hints, though, that this is the work of an AI rather than the work of someone who understands what costumes are and how they work. These, for example, take somewhat ordinary costume concepts and then make them unnecessarily difficult.
Batman on egg Vampire in hot tub A Hidden Jesus Statue Zombie ice cream cone penguin as a Newt A wizard encased in a icicle Zombie fisherman on a quest Computer generated horse(?) telephone that accepts up to 4 numbers Third Eye Blind Photographed By Dorothy Zombie fisherman w/ lady diegrove tied around foot
And the following costumes are clearly the product of a glitchy AI:
Meat Belt Eyeballed Balloon Men Green beans in bun 10,000 Hands Favorite Caterpillar The Oatmeal Tree 102 SNOWBALLS in a basket Pie and Jell-O List of leg parts world´s nicest fart Pineapple wrapped sasquatch Is it a Snake, a Watermelon, or a Bush? Putting Turtles on Decor Fish tank ‘n chair ROBO-ACCIDENT pajamas made of wood and spiders Ssssssssssexy SSSssssssstinky Ssssssssssssexy ssssssssssssssssexy setup 9 × 11 party trick Smagma Monster Commentary couldn’t be heard over the squawking of clocks Poltergeist might be entertaining, but he’s harder to read in Hungarian Cereal Implanting Device blueberry sipping fizzy pop with eyes of ice blueberry sipping fizzy pop with fake blood on it A sarcastic, racist noble using progressively tinier body parts as a human shield
Bonus content! The above costumes are all from temperature 1.2; I also tried a higher temperature setting, but the generated costumes were at an expert level of chaos (I would like to see someone attempt to go as “hypnopotamus embroidered death”) Enter your email here to get them!
You can order my book You Look Like a Thing and I Love You! It’s out November 5 2019.
The Church of Scientology and companies run by its members spent $103 million over the past three years buying up vast sections of downtown Clearwater.
They now own most commercial property on every block within walking distance of the waterfront, putting the secretive church firmly in control of the area’s future.
Most of the sales have not previously been reported. The Tampa Bay Times discovered them by reviewing more than 1,000 deeds and business records, then interviewed more than 90 people to reconstruct the circumstances surrounding the transactions.
The lead-in scrollytelling through Clearwater is quite effective in laying the foundations of the story.
An interesting long read on the political and social dimensions around Duolingo’s decision to branch out into smaller languages like Irish and Hawaiian. Excerpt:
Of course, there’s a big difference between picking up a few words in Irish or Welsh to make you feel as if you’re connecting with your ancestors, and actually learning a language — particularly an endangered one that needs all the speakers it can get.
This is a tension that Duolingo has struggled with when it comes to its two endangered language courses, Navajo and Hawaiian. Those tongues are listed as vulnerable and critically endangered, respectively, by UNESCO.
Both languages were added to Duolingo this year to coincide with the United Nations International Year of the Indigenous Language. But they raised questions that weren’t necessarily an issue for courses such as French or Spanish, which aren’t expected to be used by native speakers of those languages.
“Who’s the audience for the Hawaiian course? Is it going to be tourists? Mostly? Because that would affect the content,” said Awodey. “Or is it going to be primarily built by and for indigenous speakers and people reconnecting with the language?”
In Hawaii, the team partnered with Kamehameha Schools, a network of private schools dedicated to teaching students of native Hawaiian heritage with a particular focus on preserving the Hawaiian language.
Despite this, however, the Duolingo Hawaiian course can sometimes risk speaking down to native Hawaiians, few of whom need teaching, for example, what a “lei” is.
“Everyone was super excited about it, but it’s totally tapered off because it’s not for natives, it’s too baby, it’s too simple,“ said Kū Kahakalau, executive director of Hawaiian language and culture NGO Kū-A-Kanaka.
Linguistic politics are often fraught with regard to majority tongues, let alone for endangered languages that have a long history of colonialism and disrespect. Scrutiny of such courses is always going to be tighter, and invisible red lines easier to cross.
"When you’re dealing with a heritage language, it does come with a bundle of stuff that we don’t have when teaching English,” said Duolingo learning scientist Hope Wilson. “There are lots of tricky issues to get into, very often there are divides within the communities where people don’t agree on, you know, issues of spelling or that kind of thing.”
I’ve done several experiments with a text-generating neural network called GPT-2. Trained at great expense by OpenAI (to the tune of tens of thousands of dollars worth of computing power), GPT-2 learned to imitate all kinds of text from the internet. I’ve interacted with the basic model, discovering its abilities to generate fan fiction, British snacks, or tea. I’ve also used a tool called gpt-2-simple that Max Woolf developed to make it easy to finetune GPT-2 on more specialized datasets - I’ve tried it on datasets like recipes or crochet. One of my favorite applications of GPT-2 and other text-generating neural nets is Dungeons and Dragons spells, creatures, character names, and characterbios.
Recently Max published a tutorial on how to use GPT-2 to make new apps that are more complicated than just printing out large reams of text. To my delight, people have used them to made D&D games. First, there was Nick Walton’s AI Dungeon, in which a finetuned GPT-2 generates a story and three very strange action options at each step.
My latest obsession is GPT-2 Adventure, designed by a cognitive neuroscience student named Nathan, which lets you type in your own actions. And folks, this game is SO WEIRD and SO FUN.
YOU FLOAT IN SPACE, LIKE A BALLOON, AND AS YOU DROP THE CUBE, THE X, IT VANISHES IN A CLOUD OF GREASY BLACK SMOKE
When I first launch the game, this is what I’m greeted with, a simple text prompt reminiscent of nostalgic dungeon crawling games. It sets the scene. It waits for me to describe my next move.
Even getting to this point is like a blast from the past. The game runs in a Google Colab notebook, a format where you’re looking at cells of code, with cryptic comments, vaguely alarming warning messages, and all. It’s not a sleek web app, but feels more akin to playing a game on an old DOS machine, or maybe on a graphing calculator. There are only two clicks to begin playing, but there’s a good two-minute wait while the game loads.
And it’s my turn to type something.
>> look up
It takes several seconds for the game to respond, as if I’m playing it on a very ancient computer. The computer is in fact a state-of-the-art NVIDIA GPU, likely a K80 or a T4. The T4 could eat my laptop for lunch. I’m just playing with a very very large neural net. Finally it answers.
YOU’RE IN LABYRINTH OF TWISTY LITTLE PASSAGES, ALL ALIKE
Almost immediately, the context seems to have changed. I’m not floating at all. I’m in a maze. I guess I’ll walk somewhere.
>> walk north
THE TROLL STEPS OUT FROM BENEATH THE BRIDGE AND BLOCKS YOUR WAY
YOU ARE ON THE SOUTH SIDE OF THE CHASM
A NOD IS GIVEN TO THE INFINITE WONDER THAT IS URBZIG
A SOLID RAINBOW SPANS THE CHASM
And suddenly things are happening. There is a troll. There is a chasm. There is a rainbow. There is… Urbzig? I play it safe.
>> praise urbzig
URBZIG IS THE PATRON SAINT OF ZORK
HE IS THE GODFATHER OF ZORK, THE PLACE TO GET NEWS OF EPIC PROPORTIONS
URBZIG IS A STRONGMAN,遊 ALL
If you’ve played classic dungeon crawling games like Zork, you may recognize some of these names and phrases. Checking Nathan’s original post, I see that the training data is indeed drawn from these games. But because it’s based on GPT-2, the training data also in some sense also includes much of the internet. This gives it a lot of latent flexibility to deal with situations that were never in the classic games.
>> ask Harry Potter to try walking across the rainbow
ASK HARRY POTTER TO TRY WALKING ACROSS THE RAINBOW HE CAN’T THE RAINBOW IS EXTREMELY WIDE ABOVE YOU IS A LARGE DOME UP AROUND ITS NECK IS A STRONGG PORE [UNFORTUNATELY, THE DIRECTION OF TRAVEL IS A LOT MORE INTERESTING THAN YOU THINK ][YOU ARE IN A LARGE ROOM, WITH A PASSAGE TO THE EAST, A PASSAGE TO THE WEST, AND A WALL OF BROKEN ROCK TO THE SOUTH
An interesting effect of its human-written training data is that, like Visual Chatbot, it isn’t prepared for humans that ask questions about objects that aren’t there. If I refer to Harry Potter, then Harry Potter must be present. If I type something about a giraffe, then it assumes the giraffe must be present.
>> ask the giraffe which direction to go next
THE GIRAFFE IS ALREADY OUTFITTED WITH A VISOR WHICH IS CLEARLY DEFINED
Although it’s not clear that this AI actually understands what a giraffe is.
>> look at the giraffes
THE GAZEBO HAS DISAPPEARED
>> put giraffe in backpack
Navigating this world is like navigating in a dream. My location suddenly changes without warning. I thought I was by a chasm with a rainbow bridge. Now apparently I’m in a building. Dreamlike, it is also really hard to read text.
>> look around
YOU ARE IN A LARGE ROOM, WITH A PASSAGE TO THE SOUTH, A PASSAGE TO THE EAST, AND A WALL OF BROKEN ROCK TO THE NORTH A SIGN POSTED ABOVE THE NORTH WALL READS, “STOP! PAY TROLL!” A SMALL WINDOW CAN BE SEEN IN EITHER WALL
>> read sign
THE SIGN IS VERY SIMPLE
>> yes but read sign
WE NOW HAVE ALL FIFTEEN TREASURES IT IS TIME TO GO BACK INTO THE CAVE FOR THE END GAME
>> that’s awesome. but i still want to know what the sign says. read sign.
YOU ARE IN A GAZEBO ROOM WHICH HAS DOORS ONLY TO THE EAST AND SOUTH
Another thing that’s surprisingly difficult to do is check inventory.
>> check inventory
>> check inventory
THERE IS AN INTERESTING LOOT HERE
>> what is the loot
WHAT IS THE LOOT?
>> check inventory
YOU ARE IN THE BAG CASE
Very occasionally I will succeed in opening a bag. It will usually be weird.
>> open bag
OPENING THE BAG REVEALS A TEA BAG
>> make tea
AY! A LOVELY SURPRISE FOR THE SURPRISE
More so than any other game I’ve played, I’m unsure of the rules here. Some weird things unexpectedly work.
IF YOU DON’T GET KILLED, YOU ARE TOAST
>> Eat the toast.
EAT THE TOAST IT TASTES LIKE A STEAK A STEAK IS PERFECTLY FINE
Other seemingly obvious things don’t work.
THERE IS A PATH TO THE NORTH, A PATH TO THE SOUTH, OR A STEEP PATHWAY CLIMBING SOUTHWEST ALONG THE EDGE OF A CHASM
>> go north
I DON’T KNOW HOW TO DO THAT
>> go south
I DON’T KNOW HOW TO DO THAT
>> go southwest
YOU WILL BE DESCENDING INTO A VALLEY THERE IS A NARROW BEACH TO THE NORTH AND A PATH LEADING INTO A CANYON TO THE SOUTHEAST YOU ARE ON THE CLIFF OVERLOOKING THE FAR SIDE OF THE ROOM A PATH LEADS SOUTH ALONG THE EDGE OF A CHASM, AND A NARROW PATH LEADING INTO A LOZENGE ROOM
Sometimes I’m wandering endlessly from room to room, from cave to cave, encountering the same chasm in slightly different variations, the same dried-up lake. Other times I’m embroiled in seemingly unescapable fights, usually with trolls. Often reality shifts entirely around me and I end up in a maze of twisty little passages, all alike, or in the hall of the mountain king. Everything seems weirdly familiar - even the room descriptions are echoes of classic dungeon crawling games. It is as if I’m dreaming about playing them again.
My actions also mold reality. I learn quickly to always check for diamonds, and never check for traps. If I find myself in trouble, I can always do something confusing and provoke a context change. Writing “In a large bowl, combine butter and sugars” is usually enough to teleport myself out of whatever situation I’m in. I can always ask a giraffe for advice, although the giraffe usually isn’t very helpful.
Without a particular mission to fulfill, the caves and chasms begin to blend into one another. I give myself missions, and that helps. Once I decide my goal is to put together a picnic (incessant “check for food” commands eventually yielded a sack with a lunch and a clove of garlic, belonging to a horse who agreed to join me on a picnic). Another time I demand a mission and am given some php code; i decide my mission is therefore to build a website, but am astounded when a unicorn steals my laptop. This is not the only time other content intrudes on the game’s fantasy landscape - once the game tries to tell me about finding cash online, and another time I am sadly informed that there is no wi-fi. I encounter dinosaurs and troikas, none of which were in the original training data. Occasionally things get creepy as hell.
>> check for loot
CHECK FOR LOOT HERE CHECK FOR GOLD HERE CHECK FOR TORCH HERE CHECK FOR BONE HERE DO CHEST COMPRESSIONS AGAIN DO CHEST ROTATE DO CRESCENT MOON APPEAR IN THE ROOM’S CENTER
>> do back away slowly
DO BACK AWAY SLOWLY DO NOT OVERDO IT THE CHEST WILL EXPLODE AND YOU WILL DIE OKAY, SO YOU SEE ME
You can play the game too! Just follow the colab notebook link in Nathan’s blog post. Tips: 1) It may be more fun if you set yourself an arbitrary goal to reach. 2) If you find things getting repetitive, try adjusting the temperature in the “Play the game” cell to 0.6 or 1.0.
Several years ago Ryo Yamazaki was brushing one of his three Scottish Fold cats. A lump of shed had accumulated on the floor and in that moment Yamazaki noticed that it kind of looked like a hat. For a chuckle he decided to mold it into a pointy hat and place it on one of […]
The emphatic “literally” is not a millennial invention; it goes back to the 1700s at least, though Smith gets it right that it’s English. John Dryden, a man who is best known as the founder of literary criticism and the prohibition against the terminal preposition, was an early user of the emphatic “literally.” Charlotte Brontë, Jane Austen, Mark Twain, Charles Dickens, William Thackeray, Vladimir Nabokov, and David Foster Wallace all used the emphatic “literally” in their works. Even Lindley Murray, 19th-century grammarian, uses the hyperbolic “literally” in his own grammar — and he was such a peever that he thought children, along with animals, shouldn’t be referred to with the pronoun “who,” as “who” conveys personhood, and only creatures with the ability to be rational are actually people.
We only began to take issue with the hyperbolic “literally” in the early 20th century. Ambrose Bierce called it “intolerable,” and usage maven H. W. Fowler said it should be “repudiated.”
Standing at 2,717 feet (828 m), the Burj Khalifa in Dubai, United Arab Emirates, is the tallest building in the world (and casts quite a long shadow!). The design of the 163-floor structure is an abstraction of the Hymenocallis, a type of desert flower that has long petals extending from its center. The building has been well received since it opened in 2010; however, its construction was controversial as it was built primarily by workers from South Asia and East Asia who earned low wages and were housed in abysmal conditions.
Hong Kong International Airport is located on Chek Lap Kok, an island in Hong Kong’s western waters. Today, for the second day in a row, airport check-ins were suspended and dozens of outgoing flights were cancelled amid clashes between riot police and protesters. Protests in Hong Kong have been ongoing since June, when a proposed bill threatened to have people extradited into mainland China’s justice system. The airport is the world’s 8th busiest by passenger traffic, handling 74.7 million travelers in 2018.
He defies convention — somehow, he is both a male and someone who doesn’t like rugby, polo shirts, Vespas, steroids, talking over people, or the color blue. (You don’t want to be a Tomgirl. Nobody will ever want to date or hire you. Everybody knows this.)
Daddy’s Little Prince
His parents’ goal is that he’ll settle down with a woman who can support him financially. The woman will not propose until after she has sowed every single one of her wild oats; meanwhile, he will wait, indulging only in self-bettering activities such as watching reruns of The West Wing, completing national park-themed jigsaw puzzles, and consuming Spirulina. His parents caution him not to live with the woman before marriage, because you know what they say about buying the cow when you can get the milk for free. (He is the cow.)
After entering into society as a Dude-utante, he was crowned Mr. Oregon three years in a row. He crushed every competition category, including Interview, Wetsuit, and Evening Flannel. He once burped the entire Bill of Rights and the cause closest to his heart is saving the turtles.
Everyone gossips about how he gives his body freely for women’s pleasure. Does he have no self-respect? If he and his partner discover they have gonorrhea, then it is definitely his fault because she has natural biological urges; he is just a slut.
He wants everything to be perfect on his special day (this IS the day he finally loses his virginity!), and as a result, he goes crazy. He is bossy. He is fussy. Why is he showing so much emotion? His virginity is the ultimate gift he can give his wife (again, it’s cool if she has had sex before; totally fine). He wears a white tux at his wedding to symbolize his purity, goodness, and his innocent desire for marshmallows. When the time comes, he leans in and kisses his new bride on the left nostril because he is so delightfully inexperienced.
Some say he’s a bit of a bimbo, a bit of a gold-digger, and a bit of a flirt. He spends all of his wife’s paycheck shopping for cufflinks that won’t clash with the caviar. He is high-maintenance, but he’s worth it. He is never described by his personality.
He expects equal pay, which is confusing because he is a man. (It is common knowledge that men should make less because they spend half of their working hours going on shopping sprees in their minds.) Also known as a Career Man, he is selfish and unnatural because he puts his career before starting a family. When he eventually feels professionally and spiritually stable enough to settle down and conceive, the doctor obsessively mentions his geriatric sperm. He is 35.
He chauffeurs their five children (“I want a whole basketball team!” she decided years ago) across the city for dental cleanings, Taekwondo classes, tuba lessons, and earwax removal appointments for the twins. But he’s a Stay-at-Home-Dad, so he does all of this without ever setting foot outside the home. He really misses the scent of pine, the feel of wind, and having friends outside of his online Fantasy Football league. He imagines that having friends in 3D would be delightful.
Since he is not married, everyone pities him and whispers that he must be growing penile cobwebs. Cat Man does not own a cat. (He’s actually highly allergic.)
He’s the archetypal Hero.
Male Scientist/ Athlete/ Engineer/ Mathematician/ Doctor/ Firefighter/ Boss/ CEO/ President
In the rare cases that a man is able to break the pink-tinted glass ceiling and therefore does not align with one of the aforementioned archetypes, his title must NEVER be mentioned without the word “male.”
TIL the dictionary isn’t as much an instruction guide to the English language, as it is a record of how people are using it. Words aren’t added because they’re OK to use, but because a lot of people have been using them.
July 26th, 2019: T-Rex really did make this bet, ten years ago this very night!! It's like the old saying goes: what even is the point of having a decade-plus-long art project if you're not going to have your past self toss balls for your future self to catch?