Shared posts

08 Nov 00:44

Elephants in the Room

by Scott Galloway

In 2017 I wrote a book about Amazon, Apple, Facebook, and Google called The Four. After working my ass off for 25 years, I became an overnight success: speaking gigs, appearances on cable news and talk shows, book and podcast deals. I spent a bunch of time with elected officials, and powerful people wanted to have lunch with me. People were fascinated — shocked, even — by this simple observation: Big Tech is powerful, maybe too powerful.

Much of the concern was a function of the ad-driven nature of platforms — algorithms that tapped into good/bad aspects of human nature to addict us. Most people knew how Facebook and Google made money, but not how they actually worked, how the ad revenue was fueled by the collection of data and the harvesting of attention. In fact, the phrase “Big Tech” was barely known back then. (Check the Wikipedia entry for Big Tech and see which NYU professor is credited with defining the category.) I just read the last sentence and realized I still crave other people’s affirmation. #Pathetic.

Anyway, things are different today. We know we’re being tracked, and we understand how digital platforms make money. We also know they are lucrative, as in, among the fastest growing, most profitable businesses in history. Since A Beautiful Mind won Best Picture in 2002, Google has grown its revenue 625-fold. Digital ads transformed the company from a garage project into a multinational corporation, and turned Meta from a college-campus website into the largest media business in the world. If you had to bet everything, it wouldn’t be a bad idea to go with whoever controls our attention. Meta or Google? Safe bets. Snap? Riskier, but the moppets love it. It’s fun to flirt with other sectors and firms, but these companies are the smart, safe bets.

Until now.

Sea Change

This is proving to be a historically bad year for tech. Few saw it coming. If you bought Facebook stock in 2015, you’ve lost money. If you purchased shares of General Motors, IBM, or Chevron, you’ve made more money than Meta shareholders. Seven years of gains, erased in 10 months. Meta’s meltdown is shocking, but not singular. Google is down 40% this year, Amazon 45%, and Snap 80%. These losses are unprecedented in the Big Tech era.

As with bankruptcy, the sell-off happened gradually, then suddenly. Last week was a turning point. Amazon, Google, Meta, and Snap all missed big on earnings. The common theme? Ads. Or lack thereof. We knew the Mad Men era had come to an end; we weren’t expecting the end of Ad Men. But let’s be honest, advertising sucks. Cable ads provide a glimpse into what it’s like to have restless leg syndrome, and digital ads, while more relevant, are carbon — the noxious byproduct of converting attention into shareholder value via algorithms that bring out the worst in the species. I’ll say it again, advertising sucks.

In a surprising turn of events, ads have become Big Tech’s Achilles’ heel. Google’s ad revenue grew just 3% this quarter, down from 43% growth a year ago. For the first time, YouTube ad revenue declined. Snap registered its slowest ad revenue growth ever. Meta’s ad sales, which make up more than 98% of the business, were a trainwreck. Meanwhile (and yes, this brings me joy) the company continues to incinerate $2 billion a month feeding Mark’s fever dream that he is a god of new worlds. BTW, it appears people are more likely to worship Tom: MySpace has more traffic than Meta’s Horizon Worlds.

Macro

What is/are the meteor(s) that have struck the Gengis & Khan of ads? Meta says the problem is “the uncertain and volatile macroeconomic landscape.” Google blames “the challenging macroclimate.” Snap, “macro headwinds.” Macro, as in, rising inflation, interest rates, and supply chain issues all conspiring to depress advertising demand. And it makes sense, because if people stop spending money on stuff, then the businesses that make that stuff have less money to spend on advertising.

But here’s where it gets scary for these companies’ shareholders: The “macro” culprit is a ghost, as people haven’t stopped spending. U.S. consumer spending beat expectations in September, rising 0.6% for the second month in a row. Meanwhile, the U.S. economy is doing, well, fine — U.S. GDP grew 2.6% last quarter. Of course, there’s still a lot of uncertainty. Growth has slowed, and we’re by no means in the clear. But the weakness we’re seeing in the macroeconomy is a drizzle compared to the Category 5 shitstorm we’re seeing in digital advertising.

The state of the economy is a distraction here. Something else is killing ads, and tech companies are reluctant to acknowledge it because, unlike the economy, it’s not cyclical but structural. Hint:

Elephant No. 1

The elephants in the room are Apple and TikTok. Apple is the larger of the two, figuratively and literally. Last week, the Cupertino giant cemented its position as the most enduring tech company in history: Facing the same macro headwinds as Google and Meta, Apple beat earnings expectations on both the top and bottom lines. At $2.4 trillion in market cap, the company is roughly 10 times more valuable than Meta. Three years ago, it was twice as valuable. It’s getting easier to understand why the Zuck wants out of this universe.

Executioner

Apple may be the last thing an ad-driven platform sees before everything goes black. A year ago, Apple’s iOS upgrade forced apps, including Facebook and Instagram, to ask users for permission to track their data. Meta relies on that data to serve personalized ads that garner greater clicks and sales. So if users opt out, the ads become less effective, and Meta, dramatically less valuable.

But few predicted the change would hurt Meta this much. Turns out when Apple’s privacy prompt pops up, only 16% of users agree to being tracked. And if data is the new oil, ad platforms are losing 84% of their Brent Crude. Apple has gone Putin on Meta’s Germany by cutting supply — though this decision was positioned more virtuously, in the name of privacy, vs. war.

Without data, the digital ad ecosystem doesn’t work. I saw this play out in real time. Here at Prof G Media, we collect data — specifically, whether or not you opened this email. It’s not powerful data, but it’s data. It helps us understand what resonates. However, several months ago our open rate fell off a cliff. Why? Apple’s privacy change: Every iPhone reader started showing up as a no-open. Overnight, our data had become useless.

This newsletter isn’t monetized, so the data blackbox isn’t a game changer. But for most online businesses, it’s dire. Small e-commerce companies across the country have seen customer acquisition costs skyrocket … by 10 times. Why? The ads are no longer shown to the right people at the right time. As a result, small businesses have had to shift spend. Since the iOS upgrade, almost half of e-commerce store owners have decreased their Facebook ad spending by 25% or more. This year, the average price of ads on Meta declined 20%. One year ago, before the privacy change, prices had risen 20%. Tim’s revenge turned Meta, overnight, from a spry, twentysomething growth stock into a Golden Girl, mature and complaining.

Elephant No. 2

In advertising, the pie stays the same. The industry has consistently accounted for roughly 1.3% of U.S. GDP. Which means a couple things: 1) There’s always demand, and 2) it’s a zero-sum game. As with foreign exchange, every increase is met with a commensurate decrease somewhere else.

So if the money isn’t going to Facebook, where is it going? The phantom menace here is TikTok. Last quarter (the same quarter Google, Meta, and Snap got crushed), TikTok was the highest-grossing app for the fourth consecutive quarter — meanwhile, the rest of the app market declined. It was also the most downloaded app on the App Store, and more than a quarter of Americans under 30 now get their news from it. TikTok’s global ad revenue will triple this year, to $12 billion, which would best the revenue of Snap and Twitter combined. That number doesn’t include Douyin, TikTok’s China-based counterpart. ByteDance, the parent company that houses these assets, was recently valued at $300 billion. That’s roughly equal to Meta, Snap, and Twitter (at Elon’s inflated price) combined.

TikTok’s growing influence is well documented. We’ve discussed it many times before. Less discussed is the extent to which Xi and Cook have formed an unspoken alliance to repel the advance of Meta. The company is in full-blown retreat. After Facebook and Instagram ads became less effective, ad buyers overwhelmingly turned to TikTok. Standard practice, according to one ad firm, was to move 10% to 15% of ad spend from Facebook to TikTok. The data is opaque (part and parcel of ByteDance being China-owned), but the trend is clear: TikTok is becoming the premier ad platform.

You’d think TikTok would be just as susceptible to Apple’s privacy change as Facebook. After all, TikTok’s ads are fueled by an algorithm, which is fed with your data. But research suggests TikTok is more insulated against Apple’s privacy change than others. According to cybersecurity experts, TikTok can circumvent Apple’s code audits and track activity without the user’s knowledge. Exactly how this works is (again) not so clear, and that’s the point — as one expert pointed out, “ByteDance has gone to monumental lengths to conceal the inner workings of the app.” What could go wrong?

Rangers

I’m on a plane as I write this, taking my sons to see the Glasgow Rangers play St. Johnstone in Perth, Scotland. We are football mad, and I want my sons to see my dad’s favorite team in a 10,000-seat stadium. I’m trying to re-create a memory my dad used to tell me about, when he and his father went to this same game — the Rangers were his team. He’d get emotional talking about that day, one of his few memories of doing something, alone, with his father.

About 20 years ago, I stopped dwelling on my dad’s shortcomings, put the bullshit aside, and decided to be the son I’d like to be … full stop. The catalyst for this was his sister telling me he’d been physically abused by his father. He had never mentioned this. The idea that the person you should trust most would beat you, and the damage that must do to a young soul, is unthinkable. When I told my dad about our trip, he didn’t remember who the Rangers were. He’s 92 and struggling. As his attention and memories are disappearing, I’m riddled with questions that will likely never be answered: What planes did he repair while serving in the Royal Navy? My mom, his second wife, told me his first wife tried to kill herself after he told her he was leaving her — is this true? Another difficult subject I never had the confidence to broach. All of a sudden, a ton of questions.

This weekend, I will take a bunch of videos of us rooting for the Rangers and send them to my father. I’ll also send them to my sons with a voiceover. I will tell them what I know of their grandfather’s affinity for the Rangers. I’ll also tell them that I have no interest in football, but love it because they love it, and it makes me feel closer to them. And that when they were born, for the first time, I knew I had purpose. That all “this” meant something.

Life is so rich,

P.S. I’ll go deeper on Big Tech and the strategies that do work for it in the Business Strategy Sprint, happening Dec 5-9. Join me. Sign up here.

The post Elephants in the Room appeared first on No Mercy / No Malice.

27 Oct 19:08

AutoNation's CEO Warns About Used-Car Bubble Popping

by Tyler Durden
AutoNation's CEO Warns About Used-Car Bubble Popping

After pointing out "Used Car Prices Record First Annual Drop In Two Years" and "Used-Car Prices Record Largest YoY Decline Since Financial Crisis" this month alone -- warning signs mount the used car market bubble deflates. 

The latest sign that wholesale used car prices are in free-fall is from the largest US chain of car dealerships, AutoNation, whose CEO, Mike Manley, warned soaring interest rates are curbing car demand, resulting in price drops.

"We're beginning to see used-car prices mitigate with faster depreciation" among mainstream and budget cars, Manley said in a Bloomberg interview. "We benefit from the mix of our portfolio being premium luxury."

Manley said AutoNation had been quickly turning over the portfolio of used cars, so none of its dealers are stuck with undesirable inventory selling for less than paid. 

He said new-vehicle inventory remains tight due to chip shortages and strong demand for vehicles over $30,000. 

"It's easing rather than becoming a glut," he added. 

On an earnings call with investors, Manley said new-vehicle inventory will be below pre-pandemic levels for 2023 as automakers preserve margins to pave the way for electric vehicle development and production. 

As for the used car market, he said it's just a matter of time before the wholesale prices for used cars, which are sliding, send retail prices lower. 

Last month, Vital Knowledge's newsletter said CarMax's profit from wholesale vehicles plunged 30% in its second quarter as buyers encountered "affordability challenges" due to rising interest rates. 

AutoNation's third quarter showed $6 in adjusted earnings per share, missing the average analyst mark of $6.27, according to FactSet. Revenue came in around $6.7 billion, a tad higher than analysts forecasted. 

The good news is that the used car market is cooling after skyrocketing during the pandemic due to stimulus checks and supply chain woes hampering new car production. The other piece of good news is that the Federal Reserve's most aggressive interest rate increases since the Volcker years of the early 1980s appear to be curbing demand. 

Separately, Hertz Global Holdings reported its third-quarter earnings that showed depreciation costs were rising due to its used car prices at auction fetching lower values. 

Readers may recall it was back in April when we asked one simple question: "Are Used Car Prices About To Peak For Real This Time?" 

... and with a little bit of time, we were right. We expect deals, especially in the used cars' luxury segment, to materialize in 2023. 

Tyler Durden Thu, 10/27/2022 - 14:40
31 Dec 23:38

Mass representative democracy

by Steve Randy Waldman

Among the kids, participatory direct democracy is often taken as the ideal to which democratic polities ought aspire. But at least in theory, the case for representative democracy is strong. Political decisions really matter. They should be made well. But they are hard. Whatever interests and values you hold dear, it takes a lot of work to inform and educate yourself enough to know what political choices would in fact best serve them. This work must be performed in the face of tsunamis of misinformation propounded by those serving interests and values that diverge from yours, but whose partisans are eager to co-opt you. Democracy-skeptical public choice theorists aren’t wrong when they say that most voters are (and ought to be) “rationally ignorant“.

The genius, in theory, of representative democracy is that voters hire specialists to do the information work for them. In a representative democracy, it is not really your job as an ordinary citizen to have a strong view about the details that actually get legislated, like how many tax brackets there should be and how the rates should be structured across them. Your job is one that you (and only you) are eminently well placed to perform — to know your own interests and values, and elect a person who reflects them. That person become a specialist who gets paid to do the work of translating those interests and values into political choices that give them real effect.

This is the standard case for representative democracy you probably learned in civics class. Yet at this point, most of us roll our eyes more than a little at it. I don’t feel remotely represented by my alleged representatives, to whom I have no personal connection and little affinity of values or interest. In the US House of Representatives, I am one of about 750,000 people that my “representative” allegedly represents. In my city, I elect a “supervisor” I have never met who allegedly represents the values and interests of 80,000 of my neighbors. It’s a bit ridiculous. More than a bit. A person who “represents” a population of tens of thousands of people whose only commonality is geography effectively represents no one at all. Elections impose constraints on politician behavior, sure. If you want to keep the gig, you can’t do whatever will offend some least-common-denominator id among your constituents. You must do what it takes to raise funds for the competitive advertising campaigns that “elections” become. The first constraint is “democratic”, but it constitutes so watered down a form of representation that it counts for very little. The second constraint is often antidemocratic, since global-dollar-weighted and local-population-weighted interests and values are in conflict quite often.

Within the guard rails set by these two constraints, what “representatives” do cannot straightforwardly be described as representation. Almost any choice a politician makes would give effect to the values and interests of some of their constituents but not others. Absent the universal information work representative democracy exists to absolve us of, most of us cannot even evaluate whether the choices our representatives make are likely to further or frustrate our interests. The social architecture of contemporary representative democracy is like a how-to manual for the so-called “iron law of oligarchy“. Our institutions immerse our “representatives” among a class of electeds, bureaucratic staff, and professional courtesans. The burdens of the job keep them segregated from the publics they purport to represent. In an organic, social sense, they become bound and accountable much more to their comrades-in-arms within the governing class than they could ever be to the amorphous, conflicted group they call constituents. An economist might describe all this as an “agency problem” but that’s not quite right. There’s not a coherent enough principal whose interests the agent can be said to betray.

Modern representative democracy is simply a system whose predictable result is governance by competing coalitions of insiders, who develop deep relationships and thick connections to one another, while the electorate they notionally serve becomes an inchoate, threatening demon that must be flattered and appeased. The values and interests insiders actually serve may be corrupt and self-serving, or they may be idealistic and selfless, but they cannot accurately be described as “representing” their constituents as a body. Constituents feel unrepresented, because they are. Popular pressure builds for flawed institutions of direct democracy — ballot initiatives, referenda — under which the information problems representative democracy exists to solve run riot, but at least it isn’t always the same fuckers calling all of the shots.

Direct democracy enfranchises the citizenry to decide upon matters of whose details and ramifications they are rationally ignorant, with predictably imperfect results. Contemporary representative democracy creates a corruptible class of specialist-coworkers, who develop their own values and interests and substitute them for the those of their constituents (whose actual values and interests are too diverse and conflicted to make their own strong claim). Is there anyway we could get something that combines the enfranchisement of direct democracy with the informed participation enabled by representation? Yes.

Imagine what an online direct democracy might look like. All of us would be the legislature. Obviously, we wouldn’t meet (or mostly pretend to meet) under a neo-Roman dome in some self-important provincial city. With a legislature of only a few hundred souls, attention must be very carefully allocated. In our current Congress, there’s a whole economy of scarce floor time. (“I yield the remainder of my time to my colleague”, you’ll hear them say.) If all of our legislators were permitted to speak as much as they would, deliberation would take too much time. In fact, most legislators never weigh in at all on most issues that in a broad sense come before the Congress. Congress organizes itself into committees (by arcane means, with corrupt effect), and most matters never make it through and past committees to consideration by the broader chambers. If we had a legislature of 250 million (roughly the voting-eligible population), obviously the vast majority of citizen-legislators’ proposals and bright ideas could not be put before all their citizen colleagues. If only 1% of citizen legislators were to make a proposal each year, we’d all have millions of proposals to evaluate. That’s untenable. So we’d have to design a kind of stochastic parliament, where people’s proposals would initially go to very tiny fractions of “the legislature”. These random samples would constitute ad hoc “committees”, and each citizen would be responsible for serious deliberation on the proposals that come before them in this way, but each participant would field only a modest number of such proposals. Following deliberation and potentially modification at this stage, these ad hoc committees would vote to promote or kill the proposal. If they promote, the same procedure would recur but with a larger sample, and less scope for deliberation and modification. The number of such proposals that could be promoted to higher levels of review would be limited and so competitively rationed: only those gathering the most support would gain scarce “slots” compelling the broad polity to review them. Finally, the tournament-winning, most promoted proposals would get plenary up or down votes, like a vote on the House floor.

You can imagine this kind of thing, but it would do little to address the problems we invented representative democracy to solve. To function well, our citizenry would have to be extraordinarily engaged and informed, and it would take up all of their time. It would be like permanent jury duty.

But what if we elected representatives to participate in this kind of mass-democracy framework? Instead of electing one per 800,000 or one per 80,000, what if we self-affiliated into groups of common interest of no more than, say, 1000 souls, for whom personal, physical “town meetings” could be regularly arranged? Obviously, not everyone would wish to attend all of these meetings, but everyone could if they wished. With no more than 1000 constituents, an elected could become at least acquainted with her full constituency. She could be accessible and available to them all. She could maintain direct relationships with a substantial fraction of the people she represents, and be motivated and held to account by those relationships, by gratitude and shame experienced personally rather than by abstract shifts in what some consultant claims the polls say.

Instead of a few hundred Congresspeople, we’d have 250,000 representatives whose full-time job it would be to stay and live among and interact with their constituents, and participate in the online legislature. There would be no Congressional offices in Washington, no risk of going native among colleagues who become much closer than constituents. At a municipal level, there would be no councilmen or supervisors at City Hall. In my San Francisco, there would be roughly 800 legislators and any of us who cared to would know our representative and interact with her as much or as little as we pleased.

This proposal recognizes that the hard part of being a representative, or at least what ought to be the hard part, is not fundraising, rising through committees, learning the personalities and peccadillos of influential colleagues so that you can “legislate effectively”. The hard part of being a representative is representing. The problem we should devote ourselves to is the challenge of making one person’s voice become a capable stand-in for many others’ necessarily absent. The legitimacy of our entire system of government depends upon this thin reed, the quality of the bond between elected and constituency. When that bond becomes as attenuated and deflected as it has under current institutions, “democracy” fails to confer very much legitimacy at all, or to be effective at serving the interests of the people on whose behalf it claims to rule.

This proposal recognizes also that human beings are best motivated and held accountable by direct relationships to other human beings. Pecuniary incentives are course-grained, and always susceptible to corruption. Career incentives — they’ll serve the people because they want to be reelected! — are wildly insufficient. The range of things a politician may do and still get reelected is wide, and can deviate a great deal from their constituents’ interests. Career incentives are easy for outsiders to game. If you serve constituencies that may not entirely be those you supposedly represent, there may be a gig on K Street or a place in the party bureaucracy for you. The foundational error of the neoliberal period was the conceit that aligning financial incentives to social goods was easy, so market success and contribution to social welfare could, to a first approximation, be equated. Market success, alas, in the market for political careers as much as cigarette sales, can be welfare destructive rather than socially valuable. The best work results from intrinsic devotion to excellence plus human relationships that help steer a person’s excellence towards ends that a community values. People need to be well paid not so financial incentives direct their work, but so that financial anxieties and ambitions don’t misdirect, eclipse, distort, occlude the fragile foundations of real human achievement. Our representatives should be paid well, and should serve and live among tangible human communities whose interests they know and experience through organic personal relationships.

So “expand the House” from 435 to, um, 250,000, and put it online. Obviously, this is an idea that can’t be put into immediate practice at a national level. We have a lot to learn before we’ll trust large-scale stochastic deliberative assemblies to resolve political questions with extraordinarily high stakes. However, it is a vision that we should be working towards. Whatever you think of “crypto”, one thing that proposals like this highlight is the need for extremely trustworthy networked computation infrastructures that are credibly neutral, that are not subject to the discretion of some party that owns or operates the machines. If you want to run a legislature over a network, there can’t be a company that manages the database that might potentially manipulate it. You need the system to produce very persuasive, public evidence of its integrity at all times. I don’t think public blockchains in anything like their contemporary forms will get us there, but they are working prototypes of this sort of trustworthy computation. They are also sites of experimentation in (rudimentary, badly flawed) online deliberative assemblies such as “DAOs“. There is plenty to hate about contemporary crypto, but in the midst of all the scam and speculation there are emerging fascinating “petrie dishes” for experimental democracy, to which it is worth paying some attention, and cheering useful innovation. Most of cryptoland is understandably but unfortunately cynical of representative models of democracy. But the usual alternative — “governance tokens” directly voted, like shares of stock in a traditional corporation — performs poorly. Token-voting is plutocratic by design, and outcomes tend to be dominated by insiders and activists while much larger “rationally ignorant” groups just “HODL” (hold) their tokens for speculative purposes without voting them. The interests of stakeholders who are not tokenholders get ignored entirey. (See Vitalik Buterin’s lengthy critique of “coin voting”.)

Legislation is, in computer lingo, a very stateful application. Online deliberative assemblies will need to keep precise track of large numbers of lengthy documents and particular revisions thereof, which cannot be done on contemporary blockchains at a reasonable cost and speed. But it would not be so difficult to repurpose some of the technologies that underlie contemporary crypto to build bespoke, city-scale legislatures that could be operated affordably and generate compelling evidence of their integrity. Cities should give “mass representative democracy” a try, soon. If you live in a city of any size, do you feel, today, like you are adequately represented in city government? If not, what hope do we have to make representative democracy work at a state or national scale? We are collectively, and correctly, coming to understand that we’ve never really had the kind of Our Democracy that talking heads on MSNBC are constantly telling us we must save. It’s time to roll up our sleeves and build institutions we’ll have reason to be less cynical about.

Update History:

  • 8-Jan-2022, 12:50 a.m. PST: “ballot initiatives, referenda — in under which the information problems representative democracy exists to solve run riot”; “whose actual values and interests are too diverse and conflicted”; “The range of things a politician can may do and still get reelected is quite wide , and can deviate a great deal from where their constituents’ interests might lie.”; “…in the market for political careers and as much as cigarette sales…”; “help steer a person’s excellence towards ends that a community values.”; “…to resolve political questions of with extraordinarily high stakes”; “…in (rudimentary, badly flawed) online deliberative assemblies such as ‘DAOs'”; “…it would not be so difficult to take repurpose some…”
25 Mar 23:37

Sovereign Writers and Substack

by Ben Thompson

There has been a bit of controversy around Substack over the last week; I’m not going to get into the specifics of various accusations made about various individuals, or their responses; however, I do think that there are a few fundamental issues about the Substack model specifically, and the shift to sovereign writers generally, that are being misunderstood.

I’m going to anchor on this piece from Peter Kafka at Recode:

Substack’s main business model is straightforward. It lets newsletter writers sell subscriptions to their work, and it takes 10 percent of any revenue the writers generate (writers also have to fork over another 3 percent to Stripe, the digital payments company)…

The money that Substack and its writers are generating — and how that money is split up and distributed — is of intense interest to media makers and observers, for obvious reasons. But the general thrust isn’t any different from other digital media platforms we’ve seen over the last 15 years or so.

From YouTube to Facebook to Snapchat to TikTok, big tech companies have long been trying to figure out ways they can convince people and publishers to make content for them without having to hire them as full-time content creators. That often involves cash: YouTube set the template in 2007, when it set up a system that let some video makers keep 55 percent of any ad revenue their clips generated…Like Substack, YouTube and the other big consumer tech sites fundamentally think of themselves as platforms: software set up to let users distribute their own content to as many people as possible, with as little friction as possible.

I’m not sure the connection to Facebook and YouTube hold (even with Substack Pro, which I’ll get to in a moment); As Kafka notes, Substack “lets newsletter writers sell subscriptions to their work”; that, by definition, means that Substack is not “figur[ing] out ways they can convince people and publishers to make content for them without having to hire them as full-time content creators”. Just look at my credit card statement, where I happened to find charges for Casey Newton’s Platformer and Bill Bishop’s Sinocism next to each other:

Credit card charges from Platformer and Sinocism (not Substack)

Notice that the names of the merchant, the phone number of the merchant, and the location are different — that’s because they are different merchants. Substack is a tool for Newton and Bishop to run their own business, no different than, say, mine; Kafka writes:

To be clear: You don’t need to work with a company like Substack or Ghost to create and sell your own newsletter. Ben Thompson, the business and technology writer whose successful newsletter served as the inspiration for Substack, built his own infrastructure cobbling together several services; my former colleague Dan Frommer does the same thing for his New Consumer newsletter. And Jessica Lessin, the CEO of the Information, told me on the Recode Media podcast that she’d consider letting writers use the paid newsletter tech her company has built for free.

Here’s what you see on your credit card statement for Stratechery:

Credit card statement from Stratechery

My particular flavor of membership management software is Memberful, but Memberful is not Stratechery’s publisher; I am. Memberful is a tool I happen to use to run my business, but it has no ownership of or responsibility for what I write. Moreover, Memberful — like Substack — doesn’t hold my customer’s billing data; Stripe does, and that charge is from my Stripe account, just as the first two charges are from Newton and Bishop’s respective Stripe accounts.

This is what makes “the intent interest of media makers and observers” so baffling. There is a very easy and obvious answer to “how that money is split up and distributed”: subscriber money goes to the person or publication the subscriber subscribes to. That’s it! Substack is a tool for the sovereign writer; the sovereign writer is not a Substack employee, creator, contractor, nothing. Users quite literally pay writers directly, who pass on 10% to Substack; Substack doesn’t get any other say in “how that money is split up and distributed.”

But what about Substack Pro?

Substack Pro

Back in 2017 I wrote a post called Books and Blogs that explained why subscriptions were a much better model for writers than books:

A book, at least a successful one, has a great business model: spend a lot of time and effort writing, editing, and revising it up front, and then make money selling as many identical copies as you can. The more you sell the more you profit, because the work has already been done. Of course if you are successful, the pressure is immense to write another; the payoff, though, is usually greater as well: it is much easier to sell to customers you have already sold to before than it is to find customers for the very first time…

Since then it has been an incredible journey, especially intellectually: instead of writing with a final goal in mind — a manuscript that can be printed at scale — Stratechery has become in many respects a journal of my own attempts to understand technology specifically and the way in which it is changing every aspect of society broadly. And, it turns, out, the business model is even better: instead of taking on the risk of writing a book with the hope of one-time payment from customers at the end, Stratechery subscribers fund that intellectual exploration directly and on an ongoing basis; all they ask is that I send them my journals of said exploration every day in email form.

Recurring revenue is much better than selling a book once; however, just as you have to spend time to write a book before you can sell it, you need time to build up a subscriber base that supports a full-time subscription. I accomplished this by writing Stratechery on nights and weekends while working at Microsoft and Automattic, and then, when I started charging, basically jumping off of the deep end, but working writers can’t always do the former (I would bet that publications start being stricter about this going forward). This is where Substack Pro comes in; from Kafka:

But in some cases, Substack has also shelled out one-off payments to help convince some writers to become Substack writers, and in some cases those deals are significant. Yglesias says that when it lured him to the platform last fall, Substack agreed to pay him $250,000 along with 15 percent of any subscription revenue he generates; after a year, Yglesias’s take will increase to 90 percent of his revenue, but he won’t get any additional payouts from Substack.

As Yglesias told me via Slack (he stopped working as a Vox writer last fall but still contributes to Vox’s Weeds podcast), the deal he took from Substack is actually costing him money, for now. Yglesias says he has around 9,800 paying subscribers, which might generate around $860,000 a year. Had he not taken the Substack payment, he would keep 90 percent of that, or $775,000, but under the current deal, where he’ll keep the $250,000 plus 15 percent of the gross subscription revenue, his take will be closer to $380,000.

Substack has been experimenting with this kind of offer for some time, but last week, it began officially describing them as “Substack Pro” deals.

In short, the best analogy to Substack Pro are book advances, which are definitely something that publishers do. In that case publishers give an author a negotiated amount of money in advance of writing a book for reasons that can vary; in the case of famous people the advance represents the outcome of a bidding war for what is likely to be a hit, while for a new or unknown author an advance provides for the author’s livelihood while they actually write the book. The publisher then keeps all of the proceeds of the book until the advance is paid back, and splits additional proceeds with the author, usually in an 85/15 split (sound familiar?); of course we don’t know the exact details of book deals, because they are not disclosed.1

At the same time, Substack Pro isn’t like a book advance at all in a way that is much more advantageous to the writer. Book publishers own the publishing rights and control the royalties as long as it is in print; writers in the Substack Pro program still own their customers and all of the revenue past the first year, of which they can give 10% to Substack for continued use of their tool, or switch to another tool. This is where the comparison to YouTube et al falls flat: YouTube wants to be permanently in the middle of the creator-viewer relationship, while Substack remains to the side; from this perspective Substack Pro is more akin to an unsecured bank loan — success or failure is still determined by the readers.

The Real Scam

Now granted, there may be some number of Substack Pro participants who end up earning less than their advance, particularly if Substack sees Substack Pro as more of a marketing tool to shape who uses Substack; if Substack actually runs Substack Pro like a business, though, I would expect lots of deals like the Yglesias one, which has turned out to be quite profitable for Substack. As Yglesias himself noted:

Substack Pro made it possible for Yglesias to launch Slow Boring without worrying about paying the bills, and is making a profit as a reward for bearing the risk of Yglesias not succeeding or succeeding more slowly than he needed. As for Yglesias, he may end up missing out on several hundred thousand dollars this year, but given he’s not selling a book but rather a subscription he can look forward to a huge increase in revenue next year.

This, needless to say, is not a scam, which is what Annalee Newitz argued:

For all we know, every single one of Substack’s top newsletters is supported by money from Substack. Until Substack reveals who exactly is on its payroll, its promises that anyone can make money on a newsletter are tainted. We don’t have enough data to judge whether to invest our creative energies in Substack because the company is putting its thumb on the scale by, in Hamish’s own words, giving a secret group of “financially constrained writers the ability to start building a sustainable enterprise.” We are, not to put too fine a point on it, being scammed.

It is, for the reasons I laid out above, easier to get started with a subscription business if you have an advance. No question. But this take is otherwise completely nonsensical: Substack’s top newsletters are at the top because they have the most subscribers paying the authors directly. For example, look at the “Politics” leaderboard, where Yglesias is seventh:

Substack's 'Politics' leaderboard

We already know that “Thousands of Subscribers” to Slow Boring is 9,800; given that 9,800 * $8/month = $78,400/month, we can surmise that The Weekly Dish has at least 15,680 subscribers ($78,400/month ÷ $5/month). Those are real people paying real dollars of their own volition, not because Substack is somehow magically making them do it.

Frankly, it’s hard to shake the sense that Newitz and other Substack critics simply find it hard to believe that there is actually a market for independent writers, which I can understand: I had lots of folks tell me Stratechery was a stupid idea that would never work, but the beautiful thing about being my own boss is that they don’t determine my success; my subscribers do, just as they do every author on Substack.

Still, that doesn’t change the fact there is a real unfair deal in publishing, and it has nothing to do with Substack. Go back to Yglesias: while I don’t know what he was paid by Vox (it turns out that Substack, thanks to their leaderboards, is actually far more transparent about writers’ income than nearly anywhere else), I’m guessing it was a lot less than the $380,000 he is on pace for, much less the $775,000 he would earn had he forgone an advance.2 It was Vox, in other words, that was taking advantage of Yglesias.

This overstates things, to be sure; while Yglesias built his following on his own, starting with his own blog in 2002, Vox, which Yglesias co-founded, was a team effort, including capital from Vox Media. Still, if we accept the fact that Yglesias charging readers directly is the best measurement of the value those readers ascribe to his writing, then by definition he was severely under-compensated by Vox. The same story applies to Andrew Sullivan, the author of the aforementioned The Weekly Dish; Ben Smith reported:

But Mr. Sullivan is, as his friend Johann Hari once wrote, “happiest at war with his own side,” and in the Trump era, he increasingly used the weekly column he began writing in New York magazine in 2016 to dial up criticism of the American left. When the magazine politely showed him the door last month, Mr. Sullivan left legacy media entirely and began charging his fans directly to read his column through the newsletter platform Substack, increasingly a home for star writers who strike out on their own. He was not, he emphasizes, “canceled.” In fact, he said, his income has risen from less than $200,000 to around $500,000 as a result of the move.

Make that nearly $1 million, i.e. $800,000 of surplus value that New York Magazine showed the door.

Substack Realities

Of course things aren’t so simple; Sullivan, like several of the other names on that leaderboard, are, to put it gently, controversial. That he along with other lightning-rod writers ended up on Substack is more a matter of where else would they go? Again, the entire point is that Sullivan’s readers are paying Sullivan, which means Substack was an attractive option precisely because they don’t decide who gets paid what — or if they get paid at all.

Just because Sullivan was forced to be a sovereign writer, though, doesn’t change the fact that writers who can command a paying audience have heretofore been significantly underpaid. That points to the real reason why the media has reason to fear Substack: it’s not that Substack will compete with existing publications for their best writers, but rather that Substack makes it easy for the best writers to discover their actual market value.

This is where Substack really is comparable to Facebook and the other big tech companies; the media’s revenue problems are a function of the Internet unbundling editorial and advertising. The fact that Google and Facebook now make a lot of money from advertising is unrelated. Similarly, media’s impending cost problem — as in they will no longer be able to afford writers that can command a paying audience — is a function of the Internet making it possible to go direct; that Substack is one of many tools competing to make this easier will be similarly unrelated.

This explains three other Substack realities:

  • First, Substack is going to have a serious problem retaining its most profitable writers unless it substantially reduces its 10% take.
  • Second, Substack is less threatened by Twitter and Facebook than many think; the problem with the social networks is that they want to own the reader, but the entire point of the sovereign writer is that they own their audience. Substack’s real threat will be lower-priced competitors.
  • Third, it would be suicidal for Substack to kick any successful writers off of its platform for anything other than gross violations of the law or its terms of service. That would be a signal for every successful writer to seek out a platform that is not just cheaper, but also freer (i.e. open-source).

This is also why Substack Pro is a good idea. To be honest, I was a tad bit generous above: signing up someone like Yglesias is closer to the “popular author bidding war” side of the spectrum, and may not be worth the trouble; what would be truly valuable is helping the next great writer build a business, perhaps in exchange for more lock-in or the rights to bundle their work. Ideally these writers would be the sort of folks who would have never gotten a shot in traditional media, because they don’t fit the profile of a typical person in media, and/or want to cover a niche that no one has ever thought to cover (these are almost certainly related).

The Sovereign Writer

I am by no means an impartial observer here; obviously I believe in the viability of the sovereign writer. I would also like to believe that Stratechery is an example of how this model can make for a better world: I went the independent publishing route because I had no other choice (believe me, I tried).

At the same time, I suspect we have only begun to appreciate how destructive this new reality will be for many media organizations. Sovereign writers, particularly those focused on analysis and opinion, depend on journalists actually reporting the news. This second unbundling, though, will divert more and more revenue to the former at the expense of the latter. Maybe one day Substack, if it succeeds, might be the steward of a Substack Journalism program that offers a way for opinion writers and analysts to support those that undergird their work.3

What is important to understand, though, is that Substack is not in control of this process. The sovereign writer is another product of the Internet, and Substack will succeed to the extent it serves their interests, and be discarded if it does not.

I wrote a follow-up to this Article in this Daily Update.

  1. Substack reportedly pays 15% of all revenue, not just revenue above-and-beyond the advance.
  2. There is an anonymous Google Doc with self-reported media salaries; only three individuals make more than $380,000, and none more than $775,000
  3. I am still bullish on the Local News Business Model.
07 Mar 22:56

A Word on NFTs

Artists and authors are issuing non fungible tokens (NFTs) for their work. The market price of these has exploded and bids are currently at $2 million for Jack Dorsey’s first tweet. A lot of people seem to think that it is completely insane to spend money for something that is freely copyable. Why would anyone pay if you can simply go to Twitter and see the tweet there? Screenshot if you want to?

The underlying misconception here is to think that in the digital world copies are indistinguishable from originals. In a trivial sense this is true. Let’s say you copy a digital artwork, you will now have exactly the same bit sequence as the original. But in a much more profound sense it is not. To understand that it helps to follow a thought experiment.

Put yourself 100 years ahead. We have made further progress in sensors and in 3D printing. You visit the Louvre, hold up your phone and take depth scan of the Mona Lisa. It captures not just the visible light at the top but also every layer underneath it. Back home you feed this into your advanced multi materials printer and it produces a perfect replica of the Mona Lisa. Admittedly, it would have to be quite advanced because it needs to deposit atoms at a time including specific isotopes (used for dating objects), but there’s nothing in principle standing in the way of this.

image

Do you now own the Mona Lisa? No, you have a copy of it. Even though your copy is now identical to the original. Imagine for a moment trying to sell it.

You: I am selling the Mona Lisa

Buyer: You mean the one that’s hanging in the Louvre?

You: Well, …

As long as a buyer can easily call up the Louvre and ask them if they still have the Mona Lisa and they can credibly assert that they do, your copy will be worth very little. Basically in the world I am describing it will be worth the cost of the materials and the printing process.

Now note something even weirder. In the middle of the night, the curators from the Louvre break into your place and swap their Mona Lisa for yours. So you now have the one that Leonardo painted, but which courtesy of advanced technology described above, is indistinguishable from the one you reproduced. Would your chances of selling it for more money be any better than before?

You: I am selling the real Mona Lisa this time!

Buyer: You mean the one that’s hanging in the Louvre?

You: Well, …

This is what NFTs do for digital content. They let someone assert “I am the Louvre” (for that piece of content).

This is not a fad. It is a fundamental and profound innovation. And one that I and others had envisaged a long time ago.

To be clear though, there are still important problems to be solved, in particular ones around people asserting authorship to works they didn’t create. This too isn’t a new problem but one that is now more important than ever. 

06 Jul 09:11

Hamilton Lessons

by Bob Lefsetz

1

No one is an accidental success.

You’ve got to want it, you’ve got to pursue it like you’re running out of time.

Did you watch “Hamilton” on Disney+?

“Hamilton” is the biggest musical event of the past five years, far eclipsing Ariana Grande or Kanye or anybody on the hit parade. Because it wasn’t built by the hit parade, it wasn’t forced down anybody’s throat, its spread was the essence of the internet, pull, not push, people wanted it.

And now they’ve got it.

Very few people have actually seen “Hamilton” on stage, compared with the number of people in America. But those who have testified. And those who haven’t…listened to the Original Broadway Cast album.

We know about the success of the musical, what is not emphasized is the success of the soundtrack.

Maybe you needed children in the house. Word of mouth is more important than ever, but in the pre-internet era the spread was limited to the confines of your high school, today there are no boundaries, kids have more friends than ever, they consume more information than ever, adults rail against screen time but it has already transformed the younger generations in ways adults not only could not foresee but still don’t understand. There’s a network. Which is always on. And when something is deemed important, all youngsters know it.

You might be overwhelmed by apps, by comments, by Reddit, by YouTube and its influencers, but these young digital natives live on their devices and not only know the ropes of how to use them, but what is being said.

Which brings us to the Black Lives Matter demonstrations in the wake of the death of George Floyd. Without the internet, there is no spontaneous combustion. And if you look, it was the youngsters out in the streets. MTV killed homophobia and racism amongst its viewers. Not completely, but there’s no gay marriage without MTV. And the younger you are, the less racist you are. It’s the old people who are most racist, and they’re fading away and their beliefs will not radiate.

In “Hamilton” you have a story of white people portrayed by people of color. There might have been some head-scratching at first, but that’s long gone. Then again, it’s interesting that people of color can tell the white man’s story, but white people can’t tell the stories of people of color. I get it, it’s kind of like racist jokes, you can tell them if you’re a member of the tribe, you can use the “n” word if you’re an African-American, but you don’t want anybody else employing these expressions. Then again, we must move towards harmony as opposed to exclusiveness, but that’s a debate for another day.

So, if you have someone in the house who is still in school, you know “Hamilton” by heart, you’ve heard it over and over and over again. And unlike today’s hit parade, the music was not here and then gone, it sustains. In today’s world it’s harder to break through, but it’s just as hard to stay on top. First and foremost, much of what makes it just isn’t that good, secondly, there are always people vying for pole position. But slow and steady wins the race. Even slower than terrestrial radio. And in America today, oftentimes the news media does not reflect the hearts and minds of citizens. We used to live in a top-down culture, now it’s bottom-up.

So, Apple should have paid beaucoup bucks for “Hamilton.” Then again, at first the filming of the stage play was to be released theatrically. But Disney pivoted in the Covid-19 era and put it up on their television pay service. And now everybody can see it. Over and over and over again.

My point about Apple is “Hamilton” would have driven subscriptions. And it will do so for Disney. Even more, it will continue subscriptions. If you’re young, you’re not one and done, you need to be able to see “Hamilton” on demand, whenever. And old people too.

So there was tons of hype in advance of yesterday’s launch, but I’ve yet to read the article about adoption, how many people watched, but those will come. And the numbers will be stratospheric, but unlike the Super Bowl, there will be repeatability.

So, if you have not watched “Hamilton,” if you are not planning to watch “Hamilton” this weekend, you are in the minority. And you want to watch it because it’s so damn good, but also because you want to be part of the discussion, that’s what makes you feel human, when you can converse with others about something. That’s what we’re all looking for in this lockdown era.

Just one note… If you’re new to the musical, watch it with the subtitles on. You’ll glean so much more.

And unlike with Quibi, “Hamilton” is a very big bite, two hours and forty minutes, but you’ll make it through, because you’ll enjoy it and want to know how it turns out, marveling that you did not learn all this in school.

2.

“How does a bastard, orphan, son of a whore
And a Scotsman, dropped in the middle of a forgotten spot
In the Caribbean by providence impoverished
In squalor, grow up to be a hero and a scholar?”

Why is it the nobodies from nowhere break into the entertainment scene?

Because they want it. And there are no pre-requirements.

You need to put in a decade of schooling to become an MD. There are hurdles to almost every profession in America today. And the disadvantaged have problems even getting to the starting line, now more than ever the disadvantaged are just that, they to go substandard schools and are not made aware of scholarship opportunities, meanwhile their parents are denigrated as are their schools themselves. That’s America, if you’re not successful it’s your fault. But those people looking down on you were beneficiaries of elite educations, or they inherited their wealth, or they made money in business and they don’t want you taking any of it.

But in entertainment…all doors are open. Which is why it’s so hard to make it, everybody wants to play. And despite all the talk of controlling labels, the story of the last decade is democracy, individuals concocting hits all by themselves in their own little burgs.

Just as important is the influence of said work. It’s confounding how whites are racist while they listen to hip-hop and adopt the lifestyle. But that’s just how powerful entertainment and its stars are.

3

Desire and hard work. It doesn’t matter what socioeconomic stratum you’re from, you’re not going to become an elite influencer if you’re not willing to sacrifice everything to make it. And I mean EVERYTHING!

If you want a normal life, get a job, hopefully it will pay the bills.

But if you want to have influence, stop saving for a house, don’t get married, don’t have children, don’t buy an expensive car, just WORK!

But you’ve got to establish a goal.

Alexander Hamilton wanted influence, power, he wanted to leave a dent in the universe. Don’t equate this with getting rich. Sure, money can gain you a modicum of power, but nowhere near as much power as that of entertainers, those in the arts. Life is empty without the arts, despite all the put-down of art history majors. It’s the ethereal creations of the artists that we’re drawn to, that we want to spend time with, that we want to talk about, just like “Hamilton” itself. So keep your eye on the prize, but know what the prize is.

4

Don’t undersell yourself. Hamilton wants to fight, he wants to be on the frontlines, he doesn’t want to be in the background. This is no different from an NBA star wanting the ball as the clock ticks down. Hamilton believes in himself, he doesn’t shy away from a challenge, he embraces it. And he’s always looking for more responsibility.

But the problem with this is you make mistakes. And those are anathema in today’s culture. Don’t let this dissuade you, if you’re not making mistakes, you’re not taking enough risks.

5

“Talk less, smile more”

Aaron Burr wants to be a friend to everybody, not go on the record for fear of alienating someone. This is the philosophy of too many entertainers today. They stand for nothing. They want to touch everybody, but as a result they barely go beyond skin-deep. You don’t make a fan by being bland, you make fans by having edges, by believing the message is more important than the fallout.

And speaking of Burr, actions have consequences. Don’t be a hothead, try to work it out. What I mean is you might get caught up in the moment and regret it. You can never erase your brain. It’s one thing to get into an argument, it’s another to snuff out someone’s life. There are lines you do not want to cross. But shy of this limit, which rarely comes into play, do not sacrifice your truth to be a member of the group, because the group does not need you, it’s only a matter of time before you’re ejected.

6

Men are ruled by the little head.

They want to go downtown and hang with the ladies.

They do things they regret.

In other words, the world revolves around sex. And it’s no different today, when the world’s richest man throws his wife overboard in pursuit of a good friend’s spouse.

7

Loyalty is everything. I do not mean you should stay loyal to someone who committed a crime, what I’m saying is associate with those with character. Marry someone who will take their vows seriously, who won’t abandon you when you least expect it, throwing your life into chaos. There are very few people you can trust, you realize this as you get older. And oftentimes the ones you can trust most have obvious flaws, but the truth is those you think are your buddies have deeper issues that they’re hiding, which will ultimately surface, to your detriment.

8

Own your mistakes. The truth will set you free.

That’s the essence of the Reynolds Pamphlet. Your enemies have no ammunition if you own your faux pas. And today, especially, what happened yesterday is forgotten tomorrow.

9

You may be paranoid, but that does not mean people aren’t out to get you.

Jefferson, Madison, Burr…

10

“Hamilton” is the story of the abused. Americans who could not cotton to British rule. This is what is happening today. It’s not only the Vichy Republicans, but the Vichy Democrats too, who want slow, marginal change, just like the Grammy organization. They say they’re going to do something, and then do nothing as the time passes.

You try to work it out, but sometimes you become overloaded and rebel.

And people don’t like it when you rebel. Wasn’t that the essence of Trump’s speech at Mt. Rushmore last night? We’re going to restore law and order, keep the peace, a peace that is not working for most people.

11

Many more lessons will pop out as you watch “Hamilton.” But the musical is not heavy-handed, the story is paramount, the aphorisms baked in. Immigrants do get the job done. Yesterday and today.

But the most impressive thing about “Hamilton” is that Lin-Manuel Miranda created it.

But it did not come out of thin air. Miranda was a fan of musicals, and of rap, from way back. “In the Heights” did not get as good reviews. He was working in darkness, with very little attention. When the spotlight shines, it’s harder to maintain that level and quality of production.

But Lin-Manuel Miranda needed this. He needed to leave his mark, to tell his story. And he did not write a verse without a chorus, the quality sustains throughout, which is even more difficult because most of the action takes place in the first act. The second act is more cerebral. The aftermath. But the big wheel keeps on turning, life goes on, every moment counts, the big game only lasts one day, what are you gonna do, what are you gonna think about after that?

“Hamilton” is subversive, even though it’s hiding in plain sight. When something is so good, and so truthful, it cannot be denied. And if you want to change people’s behaviors, it’s best to do this through art. Because when you grab people by their hearts and minds they’ll follow you anywhere.

So, can you feel the success of “Hamilton” on television?

Maybe not. But if Lin-Manuel asked you to come together in concerted action, you’d probably follow his lead.

This is today’s world. No one has complete mindshare. Trump is even sacrificing some of his, by going off the rails, by denying the effects of Covid-19. You can sacrifice belief, your heroes can betray you, it’s not easy to let go, but at some point you do. Just like you stopped buying the albums of classic rock heroes after they released two or three duds in a row.

“Hamilton” will cement the footprint of Disney+. It’s a masterstroke. To air it on TV for everybody over the holiday, instead of trying to hoover up money in the theatre and then drip it out on the flat screen as the buzz evaporates. This is a gift. This is the story of July 4th, 2020.

At first it’s kind of stilted. Watching the filming of a play.

But since it’s the original actors, the ones who sang on the soundtrack, it’s very fulfilling, like seeing Freddie Mercury instead of Adam Lambert.

Are you still going to want to see it live?

Interesting question. I’ve been lucky to be up close and personal three times. Just a few rows back in August 2015, dead center in October of 2016.

And I also saw it in Los Angeles, but I was thrown off by the cast, the originals and their voices were burned into my brain.

It’s been five years already. And the “Hamilton” flame still burns. People will still go see the stage play, but I think the TV show will hurt attendance a bit. But it won’t be long before it will be a staple of high schools and summer camps, even summer stock. You see, “Hamilton” and its legend will not die. We were waiting for the Beatles to return, little did we know they would come back as a rap-based musical.

There were no duds on Beatle albums.

Just like there are no duds in “Hamilton.”

Alexander Hamilton achieved his goal. He made a difference, and he will be remembered.

Same deal with Lin-Manuel Miranda.

If you plan to fly in this rarefied atmosphere know that success will come later rather than sooner, you’ve got to hoover up knowledge, you’ve got to make mistakes, you’ve got to find your way.

And it will take all your time.

And you still might not succeed.

But it isn’t about promotion, who you know is important, but not definitive, it’s about the WORK!

Is the work most important to you?

Then you’ve got a chance.

Put your head down. We are not

 

~~~

Visit the archive: http://lefsetz.com/wordpress/

@Lefsetz http://www.twitter.com/lefsetz

If you would like to subscribe to the LefsetzLetter

 

The post Hamilton Lessons appeared first on The Big Picture.

25 Apr 07:41

The Amish Health Care System

by Scott Alexander

I.

Amish people spend only a fifth as much as you do on health care, and their health is fine. What can we learn from them?

A reminder: the Amish are a German religious sect who immigrated to colonial America. Most of them live apart from ordinary Americans (who they call “the English”) in rural communities in Pennsylvania and Ohio. They’re famous for their low-tech way of life, generally avoiding anything invented after the 1700s. But this isn’t absolute; they are willing to accept technology they see as a net positive. Modern medicine is in this category. When the Amish get seriously ill, they will go to modern doctors and accept modern treatments.

The Muslims claim Mohammed was the last of the prophets, and that after his death God stopped advising earthly religions. But sometimes modern faiths will make a decision so inspired that it could only have come from divine revelation. This is how I feel about the Amish belief that health insurance companies are evil, and that good Christians must have no traffic with them.

And Deists believe that God is like a watchmaker, an artisan who built the world but does not act upon it. But by some miracle, the US government played along and granted the Amish exemptions from all the usual health care laws. They don’t have to pay Medicare taxes or social security. They aren’t included in the Obamacare mandate. They can share health care costs the way they want, ignoring any regulations to the contrary. They are genuinely on their own.

They’ve ended up with a simple system based on church aid. Everyone pays tithes to their congregation (though they don’t call it that). The churches meet in houses and have volunteer leaders, so expenses minimal. Most of the money goes to “alms” which the bishop distributes to members in need. This replaces the social safety net, including health insurance. Most Amish go their entire life without needing anything else.

About a third of Amish are part of a more formal insurance-like institution called Amish Hospital Aid. Individuals and families pay a fixed fee to the organization, which is not-for-profit and run by an unpaid board of all-male elders. If they need hospital care, AHA will pay for it. How does this interact with the church-based system? Rohrer and Dundes, my source for most of this post, say that it’s mostly better-off Amish who use AHA. Their wealth is tied up in their farmland, so it’s not like they can use it to pay hospital bills. But they would feel guilty asking their church to give them alms meant for the poor. AHA helps protect their dignity and keep church funds for those who need them most.

How well does this system work?

The Amish outperform the English on every measured health outcome. 65% of Amish rate their health as excellent or very good, compared to 58% of English. Diabetes rates are 2% vs. 8%, heart attack rates are 1% vs. 6%, high blood pressure is 11% vs. 31%. Amish people go to the hospital about a quarter as often as English people, and this difference is consistent across various categories of illness (the big exception is pregnancy-related issues – most Amish women have five to ten children). This is noticeable enough that lots of health magazines have articles on The Health Secrets of the Amish and Amish Secrets That Will Add Years To Your Life. As far as I can tell, most of the secret is spending your whole life outside doing strenuous agricultural labor, plus being at a tech level two centuries too early for fast food.

But Amish people also die earlier. Lots of old studies say the opposite – for example, this one finds Amish people live longer than matched Framingham Heart Study participants. But things have changed since Framingham. The Amish have had a life expectancy in the low 70s since colonial times, when the rest of us were dying at 40 or 50. Since then, Amish life expectancy has stayed the same, and English life expectancy has improved to the high 70s. The most recent Amish estimates I have still say low seventies, so I think we are beating them now.

If they’re healthier, why is their life expectancy lower? Possibly they are less interested in prolonging life than we are. R&D write:

Amish people are more willing to stop interventions earlier and resist invasive therapies than the general population because, while they long for healing, they also have a profound respect for God’s will. This means taking modest steps toward healing sick bodies, giving preference to natural remedies, setting common-sense limits, and believing that in the end their bodies are in God’s hands.

The Amish health care system has an easier job than ours does. It has to take care of people who are generally healthy and less interested in extreme end-of-life care. It also supports a younger population – because Amish families have five to ten children, the demographics are weighted to younger people. All of these make its job a little bit simpler, and we should keep that in mind for the following sections.

How much do the Amish pay for health care? This is easy to answer for Amish Hospital Aid, much harder for the church system.

Amish Hospital Aid charges $125 monthly per individual or $250 monthly per family (remember, Amish families can easily be ten people). Average US health insurance costs $411 monthly per individual (Obamacare policies) or $558 monthly per individual (employer sponsored plan; employers pay most of this). I’m not going to bother comparing family plans because the definition of “family” matters a lot here. On the surface, it looks like the English spend about 4x as much as the Amish do.

But US plans include many more services than AHA, which covers catastrophic hospital admissions only. The government bans most Americans from buying plans like this; they believe it’s not enough to count as real coverage. The cheapest legal US health plan varies by age and location, but when I take my real age and pretend that I live near Amish country, the government offers me a $219/month policy on Obamacare. This is only a little higher than what the Amish get, and probably includes more services. So here it seems like the Amish don’t have much of an efficiency advantage. They just make a different tradeoff. It’s probably the right tradeoff for them, given their healthier lifestyle.

But remember, only a third of Amish use AHA. The rest use a church-based system? How does that come out?

It’s hard to tell. Nobody agrees on how much Amish tithe their churches, maybe because different Amish churches have different practices. R&D suggest families tithe 10% of income, this article on church-based insurances says a flat $100/month fee, and this “Ask The Amish” column says that churches have twice-yearly occasions where they ask for donations in secret and nobody is obligated to give any particular amount (“often husbands and wives won’t even know how much the other is giving.”) So it’s a mess, and even knowing the exact per-Amish donation wouldn’t help, because church alms cover not just health insurance but the entire social safety net; the amount that goes to health care probably varies by congregation and circumstance.

A few people try to estimate Amish health spending directly. This ABC story says $5 million total for all 30,000 Amish in Lancaster County, but they give no source, and it’s absurdly low. This QZ story quotes Amish health elder Marvin Wengerd as saying $20 – $30 million total for Lancaster County, which would suggest health spending of between $600-$1000 per person. This sounds potentially in keeping with some of the other estimates. A $100 per month tithe would be $1200 per year – if half of that goes to non-health social services, that implies $600 for health. The average Amish family earns about $50K (the same as the average English family, somehow!) so a 10% tithe would be $5000 per year, but since the average Amish family size is seven children, that comes out to about $600 per person again. So several estimates seem to agree on between $600 and $1000 per person.

One possible issue with this number: does Wengerd know how much Amish spend out of pocket? Or does his number just represent the amount that the official communal Amish health system spends? I’m not sure, but taking his words literally it’s total Amish spending, so I am going to assume it’s the intended meaning. And since the Amish rarely see doctors for minor things, probably their communal spending is a big chunk of their total.

[Update: an SSC reader is able to contact his brother, a Mennonite deacon, for better numbers. He says that their church spends an average of $2000 per person (including out of pocket).]

How does this compare to the US as a whole? The National Center For Health Statistics says that the average American spends $11,000 on health care. This suggests that the average American spends between five and ten times more on health care than the average Amish person.

How do the Amish keep costs so low? R&D (plus a few other sources) identify some key strategies.

First, the Amish community bargains collectively with providers to keep prices low. This isn’t unusual – your insurance company does the same – but it nets them better prices than you would get if you tried to pay out of pocket at your local hospital. This article gives some examples of Amish getting sticker prices discounted from between 50% to 66% with this tactic alone; Medicare gets about the same.

Second, the Amish are honorable customers. This separates them from insurance companies, who are constantly trying to scam providers however they can. Much of the increase in health care costs is “administrative expenses”, and much of these administrative expenses is hiring an army of lawyers, clerks, and billing professionals to thwart insurance companies’ attempts to cheat their way out of paying. If you are an honorable Amish person and the hospital knows you will pay your bill on time with zero fuss, they can waive all this.

But can this really be the reason Amish healthcare is cheaper? When insurance companies negotiate with providers, patients are on the side of the insurances; when insurance companies get good deals (eg a deal of zero dollars because the insurance has scammed the hospital), the patient’s care is cheaper, and the insurance company can pass some of those savings down as lower prices. If occasionally scamming providers meant insurance companies had to pay more money total, then they would stop doing it. My impression is that the real losers here are uninsured patients; absent any pressure to do otherwise, hospitals will charge them the sticker price, which includes the dealing-with-insurance-scams fee. The Amish successfully pressure them to waive that fee, which gets them better prices than the average uninsured patient, but still doesn’t land them ahead of insured people.

Third, Amish don’t go to the doctor for little things. They either use folk medicine or chiropractors. Some of the folk medicine probably works. The chiropractors probably don’t, but they play a helpful role reassuring people and giving them the appropriate obvious advice while telling the really serious cases to seek outside care. With this help, Amish people mostly avoid primary care doctors. Holmes County health statistics find that only 16% of Amish have seen a doctor in the past year, compared to 54% of English.

Fourth, the Amish never sue doctors. Doctors around Amish country know this, and give them the medically indicated level of care instead of practicing “defensive medicine”. If Amish people ask their doctors to be financially considerate – for example, let them leave the hospital a little early – their doctors will usually say yes, whereas your doctor would say no because you could sue them if anything went wrong. In some cases, Amish elders formally promise that no member of their congregation will ever launch a malpractice lawsuit.

Fifth, the Amish don’t make a profit. Church aid is dispensed by ministers and bishops. Even Amish Hospital Aid is run by a volunteer board. None of these people draw a salary or take a cut. I don’t want to overemphasize this one – people constantly obsess over insurance company profits and attribute all health care pathologies to them, whereas in fact they’re a low single-digit percent of costs (did you know Kaiser Permanente is a nonprofit? Hard to tell, isn’t it?) But every little bit adds up, and this is one bit.

Sixth, the Amish don’t have administrative expenses. Since the minister knows and trusts everyone in his congregation, the “approval process” is just telling your minister what the problem is, and the minister agreeing that’s a problem and giving you money to solve it. This sidesteps a lot of horrible algorithms and review boards and appeal boards and lawyers. I don’t want to overemphasize this one either – insurance companies are legally required to keep administrative expenses low, and most of them succeed. But again, it all adds up.

Seventh, the Amish feel pressure to avoid taking risks with their health. If you live in a tiny community with the people who are your health insurance support system, you’re going to feel awkward smoking or drinking too much. Realistically this probably blends into a general insistence on godly living, but the health insurance aspect doesn’t hurt. And I’m talking like this is just informal pressure, but occasionally it can get very real. R&D discuss the case of some Amish teens who get injured riding a snowmobile – forbidden technology. Their church decided this was not the sort of problem that godly people would have gotten themselves into, and refused to help – their families were on the hook for the whole bill.

Eighth, for the same reason, Amish try not to overspend on health care. I realize this sounds insulting – other Americans aren’t trying? I think this is harsh but true. Lots of Americans get an insurance plan from their employer, and then consume health services in a price-insensitive way, knowing very well that their insurance will pay for it. Sometimes they will briefly be limited by deductibles or out-of-pocket charges, but after these are used up, they’ll go crazy. You wouldn’t believe how many patients I see who say things like “I’ve covered my deductible for the year, so you might as well give me the most expensive thing you’ve got”, or “I’m actually feeling fine, but let’s have another appointment next week because I like talking to you and my out-of-pocket charges are low.”

But it’s not just avoiding the obvious failure modes. Careful price-shopping can look very different from regular medical consumption. Several of the articles I read talked about Amish families traveling from Pennsylvania to Tijuana for medical treatment. One writer describes Tijuana clinics sending salespeople up to Amish Country to advertise their latest prices and services. For people who rarely leave their hometown and avoid modern technology, a train trip to Mexico must be a scary experience. But prices in Mexico are cheap enough to make it worthwhile.

Meanwhile, back in the modern world, I’ve written before about how a pharma company took clonidine, a workhorse older drug that costs $4.84 a month, transformed it into Lucemyra, a basically identical drug that costs $1,974.78 a month, then created a rebate plan so that patients wouldn’t have to pay any extra out-of-pocket. Then they told patients to ask their doctors for Lucemyra because it was newer and cooler. Patients sometimes went along with this, being indifferent between spending $4 of someone else’s money or $2000 of someone else’s money. Everything in the US health system is like this, and the Amish avoid all of it. They have a normal free market in medical care where people pay for a product with their own money (or their community’s money) and have incentives to check how much it costs before they buy it. I do want to over-emphasize this one, and honestly I am surprised Amish health care costs are only ten times cheaper than ours are.

I don’t know how important each of these factors is, or how they compare to more structural factors like younger populations, healthier lifestyles, and less end-of-life care. But taken together, they make it possible for the Amish to get health care without undue financial burden or government support.

II.

Why look into the Amish health system?

I’m fascinated by how many of today’s biggest economic problems just mysteriously failed to exist in the past. Our grandparents easily paid for college with summer jobs, raised three or four kids on a single income, and bought houses in their 20s or 30s and never worried about rent or eviction again. And yes, they got medical care without health insurance, and avoided the kind of medical bankruptcies we see too frequently today. How did this work so well? Are there ways to make it work today? The Amish are an extreme example of people who try to make traditional systems work in the modern world, which makes them a natural laboratory for this kind of question.

The Amish system seems to work well for the Amish. It’s hard to say this with confidence because of all the uncertainties. The Amish skew much younger than the “English”, and live much healthier lifestyles. Although a few vague estimates suggest health care spending far below the English average, they could be missing lots of under-the-table transactions. And again, I don’t want to ignore the fact that the Amish do live a little bit shorter lives. You could tell a story where all of these add up to explain 100% of the difference, and the Amish aren’t any more efficient in their spending at all. I don’t think this is right. I think the apparent 5x advantage, or something like it, is real. But right now this is just a guess, not a hard number.

What if it is? It’s hard to figure out exactly what it would take to apply the same principles to English society. Only about a quarter of Americans attend church regularly, so church-based aid is out. In theory, health insurance companies ought to fill the same niche, with maybe a 10% cost increase for profits and overhead. Instead we have a 1000% cost increase. Why?

Above, I said that the most important factor is that the Amish comparison shop. Everyone needs to use other people’s money to afford expensive procedures. But for the Amish, those other people are their fellow church members and they feel an obligation to spend it wisely. For the English, the “other people” are faceless insurance companies, and we treat people who don’t extract as much money as possible from them as insufficiently savvy. But there’s no easy way to solve this in an atomized system. If you don’t have a set of thirty close friends you can turn to for financial help, then the only institutions with enough coordination power to make risk pooling work are companies and the government. And they have no way of keeping you honest except the with byzantine rules about “prior authorizations” and “preferred alternatives” we’ve become all too familiar with.

(and as bad as these are, there’s something to be said for a faceless but impartial bureaucracy, compared to having all your neighbors judging your lifestyle all the time.)

This is a neat story, but I have two concerns about it.

First, when I think in terms of individual people I know who have had trouble paying for health care, it’s hard for me to imagine the Amish system working very well for them. Many have chronic diseases. Some have mysterious pain that they couldn’t identify for years before finally getting diagnosed with something obscure. Amish Hospital Aid’s catastrophic policy would be useless for this, and I feel like your fellow church members would get tired of you pretty quickly. I’m not sure how the Amish cope with this kind of thing, and maybe their system relies on a very low rate of mental illness and chronic disease. A lot of the original “hygiene hypothesis” work was done on the Amish, their autoimmune disease rates are amazing, and when you take out the stresses of modern life maybe a lot of the ailments the American system was set up to deal with just stop being problems. I guess my point is that the numbers seem to work out, and the Amish apparently remain alive, but when I imagine trying to apply the Amish system to real people, even assuming those real people have cooperative churches and all the other elements I’ve talked about, I can’t imagine it doing anything other than crashing and burning.

Second, I don’t think this is actually how our grandparents did things. I asked my literal grandmother, a 95 year old former nurse, how health care worked in her day. She said it just wasn’t a problem. Hospitals were supported by wealthy philanthropists and religious organizations. Poor people got treated for free. Middle class people paid as much as they could afford, which was often the whole bill, because bills were cheap. Rich people paid extra for fancy hospital suites and helped subsidize everyone else. Although most people went to church or synagogue, there wasn’t the same kind of Amish-style risk pooling.

This makes me think that the Amish method, even though it works, isn’t the method that worked for past generations. It’s an innovation intended to cover for health care prices being higher than anything that traditional societies had to deal with.

Why did health care prices start rising? I’ve wondered about this a lot before – see here, here, and here. Looking into this issue, I noticed glimpses of a different possibility. The increase started around the same time that health insurance began to spread. In one sense, this is unsurprising – of course health insurance would become a thing around the time care became unaffordable. But I’ve never seen someone really try to tease out causality here. Might the two trends have been mutually self-reinforcing? The price of care rises due to some original shock. Someone invents health insurance, which seems like a good idea. But this creates a series of perverse incentives, which other actors figure out how to exploit (eg the Lucemyra example above). Insurance-based-health-care becomes less efficient, but hospitals can’t or don’t internalize this to the insured patients – they just raise the price for everyone, insurance or no. That makes even more people need health insurance, and the cycle repeats as prices grow higher and higher and insurance becomes more and more necessary. This syncs well with some explanations I’ve heard of rising college prices, where once the government made easy loans and subsidies available to everyone, prices rose until they consumed all the resources available.

I have no idea if this is true or not. If it is, the Amish succeed partly by successfully forcing providers to internalize the costs of insurance to insurance patients. Sometimes they do this by literally asking hospitals for better prices because they are not insured (eg the “honest customer” example above). Other times they flee the country entirely to reach a medical system that doesn’t deal with insured patients (eg Tijuana). This seems to work well for them. But their reliance on church alms and Amish Hospital Aid suggests that their care is still more expensive and burdensome for them than past generations’ care was for them. They’ve just learned ways to manage the expense successfully.

04 Jun 19:05

[REPOST] Epistemic Learned Helplessness

by Scott Alexander

[This is a slightly edited repost of an essay from my old LiveJournal]

A friend recently complained about how many people lack the basic skill of believing arguments. That is, if you have a valid argument for something, then you should accept the conclusion. Even if the conclusion is unpopular, or inconvenient, or you don’t like it. He envisioned an art of rationality that would make people believe something after it had been proven to them.

And I nodded my head, because it sounded reasonable enough, and it wasn’t until a few hours later that I thought about it again and went “Wait, no, that would be a terrible idea.”

I don’t think I’m overselling myself too much to expect that I could argue circles around the average uneducated person. Like I mean that on most topics, I could demolish their position and make them look like an idiot. Reduce them to some form of “Look, everything you say fits together and I can’t explain why you’re wrong, I just know you are!” Or, more plausibly, “Shut up I don’t want to talk about this!”

And there are people who can argue circles around me. Maybe not on every topic, but on topics where they are experts and have spent their whole lives honing their arguments. When I was young I used to read pseudohistory books; Immanuel Velikovsky’s Ages in Chaos is a good example of the best this genre has to offer. I read it and it seemed so obviously correct, so perfect, that I could barely bring myself to bother to search out rebuttals.

And then I read the rebuttals, and they were so obviously correct, so devastating, that I couldn’t believe I had ever been so dumb as to believe Velikovsky.

And then I read the rebuttals to the rebuttals, and they were so obviously correct that I felt silly for ever doubting.

And so on for several more iterations, until the labyrinth of doubt seemed inescapable. What finally broke me out wasn’t so much the lucidity of the consensus view so much as starting to sample different crackpots. Some were almost as bright and rhetorically gifted as Velikovsky, all presented insurmountable evidence for their theories, and all had mutually exclusive ideas. After all, Noah’s Flood couldn’t have been a cultural memory both of the fall of Atlantis and of a change in the Earth’s orbit, let alone of a lost Ice Age civilization or of megatsunamis from a meteor strike. So given that at least some of those arguments are wrong and all seemed practically proven, I am obviously just gullible in the field of ancient history. Given a total lack of independent intellectual steering power and no desire to spend thirty years building an independent knowledge base of Near Eastern history, I choose to just accept the ideas of the prestigious people with professorships in Archaeology, rather than those of the universally reviled crackpots who write books about Venus being a comet.

You could consider this a form of epistemic learned helplessness, where I know any attempt to evaluate the arguments is just going to be a bad idea so I don’t even try. If you have a good argument that the Early Bronze Age worked completely differently from the way mainstream historians believe, I just don’t want to hear about it. If you insist on telling me anyway, I will nod, say that your argument makes complete sense, and then totally refuse to change my mind or admit even the slightest possibility that you might be right.

(This is the correct Bayesian action: if I know that a false argument sounds just as convincing as a true argument, argument convincingness provides no evidence either way. I should ignore it and stick with my prior.)

I consider myself lucky in that my epistemic learned helplessness is circumscribed; there are still cases where I’ll trust the evidence of my own reason. In fact, I trust it in most cases other than infamously deceptive arguments in fields I know little about. But I think the average uneducated person doesn’t and shouldn’t. Anyone anywhere – politicians, scammy businessmen, smooth-talking romantic partners – would be able to argue them into anything. And so they take the obvious and correct defensive maneuver – they will never let anyone convince them of any belief that sounds “weird”.

(and remember that, if you grow up in the right circles, beliefs along the lines of “astrology doesn’t work” sound “weird”.)

This is starting to resemble ideas like compartmentalization and taking ideas seriously. The only difference between their presentation and mine is that I’m saying that for 99% of people, 99% of the time, taking ideas seriously is the wrong strategy. Or, at the very least, it should be the last skill you learn, after you’ve learned every other skill that allows you to know which ideas are or are not correct.

The people I know who are best at taking ideas seriously are those who are smartest and most rational. I think people are working off a model where these co-occur because you need to be very clever to resist your natural and detrimental tendency not to take ideas seriously. But I think they might instead co-occur because you have to be really smart in order for taking ideas seriously not to be immediately disastrous. You have to be really smart not to have been talked into enough terrible arguments to develop epistemic learned helplessness.

Even the smartest people I know have a commendable tendency not to take certain ideas seriously. Bostrom’s simulation argument, the anthropic doomsday argument, Pascal’s Mugging – I’ve never heard anyone give a coherent argument against any of these, but I’ve also never met anyone who fully accepts them and lives life according to their implications.

A friend tells me of a guy who once accepted fundamentalist religion because of Pascal’s Wager. I will provisionally admit that this person “takes ideas seriously”. Everyone else gets partial credit, at best.

Which isn’t to say that some people don’t do better than others. Terrorists seem pretty good in this respect. People used to talk about how terrorists must be very poor and uneducated to fall for militant Islam, and then someone did a study and found that they were disproportionately well-off, college educated people (many were engineers). I’ve heard a few good arguments in this direction before, things like how engineering trains you to have a very black-and-white right-or-wrong view of the world based on a few simple formulae, and this meshes with fundamentalism better than it meshes with subtle liberal religious messages.

But to these I’d add that a sufficiently smart engineer has never been burned by arguments above his skill level before, has never had any reason to develop epistemic learned helplessness. If Osama comes up to him with a really good argument for terrorism, he thinks “Oh, there’s a good argument for terrorism. I guess I should become a terrorist,” as opposed to “Arguments? You can prove anything with arguments. I’ll just stay right here and not blow myself up.”

Responsible doctors are at the other end of the spectrum from terrorists here. I once heard someone rail against how doctors totally ignored all the latest and most exciting medical studies. The same person, practically in the same breath, then railed against how 50% to 90% of medical studies are wrong. These two observations are not unrelated. Not only are there so many terrible studies, but pseudomedicine (not the stupid homeopathy type, but the type that links everything to some obscure chemical on an out-of-the-way metabolic pathway) has, for me, proven much like pseudohistory – unless I am an expert in that particular subsubfield of medicine, it can sound very convincing even when it’s very wrong.

The medical establishment offers a shiny tempting solution. First, a total unwillingness to trust anything, no matter how plausible it sounds, until it’s gone through an endless cycle of studies and meta-analyses. Second, a bunch of Institutes and Collaborations dedicated to filtering through all these studies and analyses and telling you what lessons you should draw from them.

I’m glad that some people never develop epistemic learned helplessness, or develop only a limited amount of it, or only in certain domains. It seems to me that although these people are more likely to become terrorists or Velikovskians or homeopaths, they’re also the only people who can figure out if something basic and unquestionable is wrong, and make this possibility well-known enough that normal people start becoming willing to consider it.

But I’m also glad epistemic learned helplessness exists. It seems like a pretty useful social safety valve most of the time.

22 Apr 09:46

The Thing I Love Most About Uber

by bgurley

In spite of all the ink that journalists, analysts, and pundits have spilled on Uber over the years, no mainstream article has focused on what I consider to be the most elegant feature of this now ubiquitous, high growth global service — no driver-partner is ever told where or when to work. This is quite remarkable — an entire global network miraculously “level loads” on its own. Driver-partners unilaterally decide when they want to work and where they want to work. The flip side is also true — they have unlimited freedom to choose when they do NOT want to work. Despite the complete lack of a “driver-partner schedule” this system delivers pick-up times that are less than 5 minutes (in most US cities (with populations over 25K) and in 412 cities in 55 other countries. The Uber network, along with Mr. Smith’s invisible hand, is able to elegantly match supply and demand, without the “schedules” and “shifts” that are the norm in most every other industry.

Some have raised questions and concerns about the “gig” economy and the rise of these new independent and autonomous work types. Detractors frequently highlight that these work types lack some of the structured benefits that are frequently attached to traditional full time job offerings. However, what they fail to consider is that there is one critical and fundamental feature of the “gig” economy that is completely absent from traditional job types. That feature — worker autonomy of both time and place — simply does not exist in other industries. One cannot show up for work at Starbucks on a Monday and then decide not to work at all on Tuesday, and for only 2 hours on Wednesday. Oh yeah, and then on Thursday let’s just “play it by ear.” One cannot get a job at Walmart or McDonalds or ironically even as a taxi cab driver without agreeing to some sort of shift or schedule. It is unheard of for an employee to say “I want to work 3 hours this week, 45 the next, and then take 2 weeks off.” This autonomy and freedom of the “gig” work type, which is highly valued by millions and millions of people, would be impossible to implement for the overwhelming majority of companies.

In November of 2014, the Morgan Stanley sell-side research team that focuses on the auto industry, headed by Adam Jonas, made a trip to Detroit to visit the big three automakers. In their own words, “the highlight of the trip, however, was three Uber trips we took between meetings.” They chronicled these three trips in a report they published titled, Confessions of an Uber Driver: Rollin in the ‘D. Interestingly, they encountered three different driver-partners that epitomize why the “where you want, when you want” autonomy of Uber is so fundamentally important. Each of these individuals has a life situation that is supplemented and improved as a result of this super unique flexibility. Included herein is a summary of each driver-partner profile. You will notice that a traditional 9-5 job would have been completely unhelpful to any of the three.

  • The Veteran — She’s a retired US Army Veteran (recently stationed in Germany) and a mother of 3 daughters, the youngest of which is still in middle school. Our driver wanted a job that offered flexibility so she could take her daughter to and from school without relying on the area’s bus system. All of the other jobs she considered made it impossible to be there for her daughter when she needed to be. Uber provides enough flexibility so that she can take jobs when and where she wants while providing substantial income to help make ends meet while her husband, an active member of the US Military, is on tour.
  • The Student — Our driver was a 30-something Jordanian-born student at Henry Ford College studying computer science with an emphasis on internet security and encryption. He’s supporting a family and wanted a job with flexible hours that could accommodate his class schedule and his familial responsibilities.
  • The Dean — Here was a dean of students at a charter school in the area who ran into a cash flow deficit for many months while undertaking extensive construction/renovations to his residence. He began ‘Ubering’ last June to make extra cash and has since developed a steady level of business, making around $600 to $700 per week with flexible hours that worked around his time at the school.

In January of 2015, Uber partnered with Alan Krueger, a professor at Princeton University, to conduct the first comprehensive analysis of Uber’s driver-partners, based on both survey data and anonymized, aggregated administrative data. The results from this survey mirrored many of the points that Jonas uncovered and that McKinsey would later uncover. Here are a few key highlights:

  • 85% say they partner with Uber “to have more flexibility in my schedule and balance my work with my life and family”
  • 55% of drivers work less than 15 hours a week, highlighting that a majority of drivers use the service for supplemental income
  • Driver-partners do not turn to Uber out of desperation, only eight percent were unemployed just before they started working with the Uber
  • Two-thirds of these individuals reported that they had a full-time job
  • Reasons for partnering with Uber:
    • “to earn more income to better support myself or my family” (91%)
    • “to be my own boss and set my own schedule” (87%)
    • “to have more flexibility in my schedule and balance my work with my life and family” (85%)
    • “to help maintain a steady income because other sources of income are unstable/unpredictable” (74%)
  • When asked directly, “If both were available to you, at this point in your life, would you rather have a steady 9 to 5 job with some benefits and a set salary or a job where you choose your own schedule and be your own boss?” 73 percent chose the latter.

In October of 2016, McKinsey and Company (working with Uber) published a detailed research report titled, Independent Work: Choice, Necessity and the Gig Economy. The complete work, which is quite detailed and interesting, is publicly available. As the main report is quite lengthy at 138 pages, some readers may prefer the 18-page executive summary. Unsurprisingly, their findings were quite consistent with points already raised above — people value freedom and autonomy. Here is a subset of the relevant findings:

  • The size of the independent workforce is quite large — “up to 162mm individuals, engage in independent work.”
  • McKinsey identified the key feature mentioned above — “A high degree of autonomy: Independent workers have a high degree of control and flexibility in determining their workload and work portfolio.”
  • Supplemental income is a key driver — “More than half of them use independent work to supplement their income rather than earning the primary living from it.”
  • Independent by choice — “Most independent workers have actively chosen their working style and report high levels of satisfaction with it,” “Approximately 70 to 75 percent of independent earners are independent as a matter of preference.”
  • High levels of satisfaction —“Free agents report higher satisfaction than those who choose traditional jobs on 12 of the 14 dimensions we measured, and they are just as satisfied on the remaining two dimensions. Free agents cite higher satisfaction than traditional workers across issues ranging from the creativity they can express to opportunities for learning and recognition. They are happier with their overall level of income and are just as satisfied as traditional workers on income security and benefits. These observations hold regardless of gender, age, education level, or household income.”

Last year, on a trip to New Orleans, I met another driver in a similar situation to those profiled in the Morgan Stanley report. She was a single mother who worked during the week as a nurse. On Friday and Saturday nights, she would drive with Uber until she acquired $100 in earnings, then she would head home. This effort earned her over $800 a month in extra income that helped her support her family. There are no other supplemental job types that are as simple and consistent as Uber is for this single mother. And the impact to her life is real and meaningful.

Another reason Uber is such a great supplemental work type is that peaks in usage elegantly overlap with time windows that are convenient for traditional 9-5pm, Monday-Friday full-time workers. Friday and Saturday nights are simultaneously the consistent weekly peaks of (a) demand on the Uber system, and (b) spare time that is available for people with standard full-time jobs that want to pick up some incremental income (the chart to the right highlights this). The same thing happens with holidays and festivals. The need for rides (and therefore drivers) at music festivals or seasonal events or in a vacation town like Tahoe are bursty. That said, these same holiday weekends are when people searching for supplemental income are free from the primary occupation and can make the voluntary decision to earn more money. I have met drivers in Tahoe that came to town with their family (on vacation) and are earning while others are hiking or skiing. The matching of this excess supply with excess demand is both elegant and fortunate.

There is another incredible driver-partner benefit of the Uber system that is radically different from traditional work types. Uber pays the driver their money immediately when earned. While other employers have experimented with ways to do this from time to time, or once a month — Uber allows this up to 5 times a day. Normal employers are nowhere close on this dimension (most pay 2-3 weeks in arrears). Imagine how this can be helpful to someone who is living paycheck to paycheck in their primary occupation. Not only are the extra earnings in and of themselves useful, but the speed of delivery of the actual cash could mean avoiding nasty traps like usurious payday loans. In fact, based on an analysis of Federal Reserve data, 47 percent of Americans “can’t pay for an unexpected $400 expense through savings or credit cards, without selling something or borrowing money.” Now they have a much better option.

There are many difficult situations in modern life where having a simple, flexible, and consistent form of supplemental income is quite beneficial:

  • A bridge while looking for a full-time job — In the above survey 32 percent of driver-partners indicated a major reason they partner with Uber is “to earn money while looking for a steady, full-time job.”
  • Extra income for a stay at home parent — Many parents are in positions where having a full-time 40-hour week job is incompatible with their duties and responsibilities to their children. Driving with Uber means they are able to have extra income without missing their children’s pick-ups, drop-offs, baseball games or theatre performances. And they can be home in the afternoons to help with homework.
  • Using it to fund their way through college — I have met many driver-partners who are attending college. College is expensive and student debt is extremely high. Students cannot work two days at McDonalds and then skip three days to study for your final — this is “not a thing” with a traditional scheduled job type. Moreover, studying and test schedules can be sporadic and unpredictable. Interestingly — the “elegant balance” characteristic applies here as well. Guess where there are lots of people that should be riding in Ubers instead of driving? College towns. Guess where there are lots of people with extra time that would love to have extra money?
  • Aiding in skills transition/retraining — The notion of skills displacement and digital disruption of certain jobs due to automation or robotics is a hot-button issue. It even comes up with regards to ride sharing as a result of the excitement generated around autonomous vehicles. If people are in need of learning new skills, they would be materially aided by the presence of a flexible and autonomous supplemental income opportunity as they retrain. Nursing school or a vocational training school have all the same issues as going to college. They cost money and demand time — which is super hard to do while maintaining a traditional 40-hour/week full time job.
  • Work your way out of debt — Many Americans are unfortunately saddled with debt — credit card debt and student loan debt. If you are simultaneously living paycheck to paycheck you have no way to “catch up,” and as a result the interest payments chew up the marginal income you would use to pay down the debt. It’s a real trap. Supplemental income — working extra hours while you need to in order to get past a problem like this can be very powerful.
  • Cover an unexpected expense — Sometimes life gives you lemons. You wreck your car. Your refrigerator or washing machine dies. You have an unforeseen medical expense. These are the exact moments where being an Uber driver-partner can get you over the hump. Not forever — just for a few weeks to cover the extra expense.
  • Do what you love — Many of life’s most interesting career pursuits can be the hardest from an earnings perspective. We all know the notions of a “struggling actor” or “struggling artist” or “struggling musician.” Over the years I have met many of Uber’s driver-partners who use the extra earnings power so that they can pursue their dreams. One musician I met would even drive while he was touring on the road — in every city along the way.

The McKinsey study also uncovered these broader societal benefits that come from scalable “independent work” earnings structures:

“Independent work could have benefits for the economy, cushioning unemployment, improving labor force participation, stimulating demand, and raising productivity. Consumers and organizations could benefit from the greater availability of services and improved matching that better fulfills their needs. Workers who choose to be independent value the autonomy and flexibility.”

One thing to note about most of the scenarios above is that they are “temporary.” There is not a desire or intention on the part of the driver-partner to do this as a lifelong career pursuit. Rather, they recognize that it is an amazingly convenient way to solve a temporary need or to help bridge through to another station in life. Some labor lobbyists argue we should turn ride-sharing driving into a scheduled, full-time affair, but in doing so, you would eliminate the key reasons that most people take to the road in the first place. You would also potentially eliminate the world’s premier supplemental work offering.

In just a few short years, over 3 million driver-partners have joined the Uber platform. To put that in perspective, Walmart has grown to 2.3 million employees over 55 years. I think it’s safe to say that over the past five years, no industry has created more new jobs and new income opportunities than ride-sharing. And keep in mind that approximately three-fourths of the industry revenue goes straight to the labor provider — which is higher than almost any other industry on the planet. As a result, in just a few short years, global ride-sharing driver-entrepreneurs have taken in approximately $75+ billion dollars (with industry lifetime revenues north of $100 billion dollars). And keep in mind that ride-sharing only represents around 1% of the miles driven in the United States. As more and more people reduce car usage and abandon car ownership — this number will most certainly go higher and higher.

One interesting thing to note about Uber’s 3 million driver-partners — they all “volunteered” to start driving with Uber. This articulation may sound unusual, but some detractors want you to believe that driving with Uber is equivalent to working in the steel mill in a small mid-western town, where it is the only opportunity for the individual. That is not the case — people are “choosing” to be driver-partners, and they are doing so in record numbers. You have to ignore over 200 years of microeconomic research to be able to contort your brain into believing that all of these people are voluntarily making poor life decisions for themselves.

In all the discussion about why independent work is different than a traditional full time occupation, all of the focus has been on the features and benefits that are absent relative to the historic and perhaps idyllic notion of “work type.” What is missing from the conversation is why this job type is so special and unique to so many millions of people. There is simply no way for the vast majority of employers in the world to offer a completely independent and autonomous work-schedule. They are unlikely to enable “instant payment” either. Yet these are the EXACT same features that show up over and over again in the research as to why people chose independent work in the first place. Independent work is undisputedly “different” from a traditional job type — which is exactly why it is so valuable to so many people.

Driving with Uber reverses the way we have been trained to think about labor. Instead of making labor conform to management’s notion of a ‘job,’ Uber hands control to the worker. You do not have to make your life fit the needs of your job; you can make the job fit the needs of your life. Just how revolutionary this notion is has not, in my opinion, been adequately understood.

The post The Thing I Love Most About Uber appeared first on Above the Crowd.

26 Dec 17:14

Deep Empathy

by Barry Ritholtz

Source: MIT

The post Deep Empathy appeared first on The Big Picture.

03 Oct 06:31

Some comments on the New York Times story about Donald Trump's tax returns

by John Hempton
Decades ago - before I was a fund manager - I was the resident expert on tax avoidance working for the Australian Treasury. That was where I started to hone the accounting skills sometimes shown on this blog.

I very rarely do anything in tax - but now I think it is time.

The New York Times has published a story (including extracts) about Donald Trump's tax returns over two decades ago. The money-quote is this:

Donald J. Trump declared a $916 million loss on his 1995 income tax returns, a tax deduction so substantial it could have allowed him to legally avoid paying any federal income taxes for up to 18 years...

According to the New York Times the losses came
... through mismanagement of three Atlantic City casinos, his ill-fated foray into the airline business and his ill-timed purchase of the Plaza Hotel in Manhattan.
There is an issue here.

Donald Trump did not repay all the debt associated with those investments.

Either


  • the loss is a real loss and the Donald was really was out of pocket by $916 million (in which case he has legitimate NOLs)
  • or the loss was passed on to someone else by The Donald defaulting on debt - in which case Donald Trump should be assessed for income from debt forgiveness.


After all if the debt is forgiven it is not Donald Trump's loss. The loss is borne by the person who lent Donald money and did not get it back.

That - clearly stated by example - is why most income tax systems assess debt forgiveness as income.

--

Okay - I do not know whether Donald Trump had the wherewithal in 1995 to bear $916 million of losses personally. But I doubt it. (If he did his financial career is different from what is popularly accepted.)

So the alternative is the debt was forgiven in some way. But then the story the New York Times is running is wrong - because the $916 million of losses would not have survived the debt forgiveness and hence would have wiped out his NOLs and thus he would not be allowed to shelter his income for the next 18 years.

Unless that is there is an avoidance scheme the New York Times has not worked out. Those schemes go by the name of "debt parking".

Debt parking

Here is how debt parking works. Suppose the debtor (in this case The Donald) is going to get his debt cancelled for (say) 1c in the dollar. When he gets the debt wiped out the debtor (ie The Donald) will have to report assessable income equal to the debt wiped out (in this case 99 percent of $916 million).

The alternative though is for the debtor to set up a dummy party. The dummy party might be his wife or children or some company or trust set up by them or more likely some completely opaque offshore trust.

And that dummy party goes and buys the debt for say 1.1 cents in the dollar. Then they just sit there.

They don't force the debtor (ie The Donald) to repay. They don't make a profit or loss on the debt. And because the debtor never has his debt forgiven he never gets the assessment on debt forgiveness and he gets to keep his NOLs even though the losses did not come out of his pocket.

Every tax system worth its salt has some rules on "effective debt forgiveness" to prevent debt parking. And - from my experience which is now over twenty years old - none of them work entirely.

Now if Donald really has all those tax losses its pretty clear that the debt must be parked somewhere.

There is a vehicle out there (say an offshore trust or other undisclosed related party effectively controlled by Donald Trump) - which owns over $900 million in debt and is not bothering to collect it.

I do not have the time or energy to find that vehicle. But it is there. Now that this blog has gone public journalists are going to look for it.

There is a Pulitzer prize for whoever finds it. Just give me a nod at the acceptance ceremony.





John

14 Jun 20:37

Why startups should forget vitamins and painkillers, and focus on the Job To Be Done

by David Jackson

Edited excerpt from Does your startup solve a problem? Vitamin or painkiller? by Don Dodge:

One of the questions I ask entrepreneurs when evaluating start-ups is “Is your product a vitamin (nice to have) or a painkiller (got to have it)? Of course everyone wants to think their product is a “must have” painkiller, but very few are.

Many products fall into the vitamin category. Things like productivity tools, content aggregators, mashups, utilities, collaboration applications, measurement and monitoring tools, in fact anything that is a tool, development or otherwise, is by definition a vitamin.

Painkiller products are products that solve for a specific pain point. Sometimes the pain is measurable in terms of ROI, winning sales that could not be won before, or satisfying a regulatory requirement.

There is another set of products that are vitamins (nice to have) until you feel the pain. Then they become painkillers (got to have it). There are actually lots of products that fall into this category.

Notes:
(1) On vitamins versus painkillers, see also Building a valuable product — a checklist of questions to answer and How to sell your product if it’s a “nice to have”, not a “must have”.
(2) My personal viewpoint: the categorization of products into vitamins and painkillers is flawed and unhelpful. Note that (i) great businesses have been built from “vitamins” (think media and entertainment), (ii) helping a company to increase its revenues is, absurdly, a vitamin, and (iii) the categories are unstable — products can transition from vitamin to painkiller depending on the context and user.
(3) For me, the Job To Be Done framework is far more helpful. The right questions to ask are: What “job” is the user / customer looking to get done (user need)? How valuable is getting that job done for the person (willingness to pay)? How many people have that job (market size)? How well does your product help the person get the job done relative to the alternatives (competitive advantage)?
(4) See: A brief summary of Job To Be Done, with 3 takeaways for product managers.

24 May 04:00

Forty Percent of the Buildings in Manhattan Could Not Be Built Today

by Barry Ritholtz
15 Apr 12:08

Meanwhile, in a parallel universe…

by CommitStrip

Strip-Test-Turing-inverse-(650-final)(english)

13 Oct 08:05

Hard Choices: Growth versus Profitability

Quite a few companies are finding themselves in a situation roughly as follows: they have raised a lot of money at high valuations on a fast growth story, have real customers and the beginning of revenues, but growth has slowed down; they have money in the bank for 12-24 months and now need to decide between a strategy of rekindling growth or trying to push to profitability.

This is a complicated decision to make and I see a lot of teams struggling with it. There is no easy answer and only one thing is clear: you have to make a conscious decision between the two paths and make significant changes to your organization now in *either* case.

Picking Growth

The bull case here is that if you restart growth then you can raise more money and maybe even achieve an up-round in valuation. The bear case is that if you fail to do so you will be out of business. Put differently, picking growth is the “high alpha” path, meaning it increases risk while also having the possibility for a higher payoff. Many venture investors will like this since they have a portfolio and with power law returns (which now dominate many industries) a single big hit will far offset many losses.

Teams may also at first blush find this easier because it is what they have been working on. But therein lurks a danger. In all likelihood it will take you a lot longer than you think to rekindle growth. And getting back to growth is in many ways harder, the bigger your organization is. Why? Because there will be more people who are defending existing ways the product works which makes it harder to try something different. Serious experimentation is exactly what you need though.

So if you want to pursue the growth path and have had several “fat years” you must trim down your team and product. Only with a lean approach are you giving yourself the room — both in terms of time and ability to experiment — to have a good chance at getting back to high growth.

Picking Profitability

The bull case here is that if you are at or near profitability then you can be the master of the fate of the company and no longer depend on new investors (existing investors tend to put up the money that may be required to close a small gap). The bear case is that you may be “stuck” in a company that’s not very valuable and barely squeaking by. This is the “safer” option but of course some risk remains that you won’t make it at all.

Choosing the profitability path generally requires a shift away from product and engineering resources towards sales and marketing. Again it will take you longer than you think to get there and there is a lead time for new sales people to become productive and new marketing initiatives to produce leads. Committing to that requires a meaningful reorientation of senior leadership priorities and often some changes to the senior team.

This path doesn’t sound like fun for a lot of founders. And there’s a fear that you might not make it after all, or if you do make it, not have something that’s all that valuable. There is a view that you “should go big or go home” or that you should “fail fast.” The idea is that this should be true not just for VCs (see above) but also for founders and employees because they can then join another startup which once again might have a shot at high growth.

The counter though is that if you are working in a domain that you love and believe in the mission of the company, then you should be willing to stick with it for a long time. Becoming profitable has massive optionality of its own: the ability to go on for many years with new experiments any one of which could then ignite growth. I wish more people would take that view but it requires patience, discipline and conviction.

Of course life would be a lot easier if you didn’t have to make this choice — and, if you can raise more money, you may not have to. But increasingly companies will have to make this choice and make it in a pronounced and clear fashion. And the ones that do will have a good shot at either growth or profitability. The ones that will continue to go down both paths at once, however, are likely to run out of money before achieving either.

10 Jan 07:58

The God Login

by Jeff Atwood

I graduated with a Computer Science minor from the University of Virginia in 1992. The reason it's a minor and not a major is because to major in CS at UVa you had to go through the Engineering School, and I was absolutely not cut out for that kind of hardcore math and physics, to put it mildly. The beauty of a minor was that I could cherry pick all the cool CS classes and skip everything else.

One of my favorite classes, the one I remember the most, was Algorithms. I always told people my Algorithms class was the one part of my college education that influenced me most as a programmer. I wasn't sure exactly why, but a few years ago I had a hunch so I looked up a certain CV and realized that Randy Pausch – yes, the Last Lecture Randy Pausch – taught that class. The timing is perfect: University of Virginia, Fall 1991, CS461 Analysis of Algorithms, 50 students.

I was one of them.

No wonder I was so impressed. Pausch was an incredible, charismatic teacher, a testament to the old adage that your should choose your teacher first and the class material second, if you bother to at all. It's so true.

In this case, the combination of great teacher and great topic was extra potent, as algorithms are central to what programmers do. Not that we invent new algorithms, but we need to understand the code that's out there, grok why it tends to be fast or slow due to the tradeoffs chosen, and choose the correct algorithms for what we're doing. That's essential.

And one of the coolest things Mr. Pausch ever taught me was to ask this question:

What's the God algorithm for this?

Well, when sorting a list, obviously God wouldn't bother with a stupid Bubble Sort or Quick Sort or Shell Sort like us mere mortals, God would just immediately place the items in the correct order. Bam. One step. The ultimate lower bound on computation, O(1). Not just fixed time, either, but literally one instantaneous step, because you're freakin' God.

This kind of blew my mind at the time.

I always suspected that programmers became programmers because they got to play God with the little universe boxes on their desks. Randy Pausch took that conceit and turned it into a really useful way of setting boundaries and asking yourself hard questions about what you're doing and why.

So when we set out to build a login dialog for Discourse, I went back to what I learned in my Algorithms class and asked myself:

How would God build this login dialog?

And the answer is, of course, God wouldn't bother to build a login dialog at all. Every user would already be logged into GodApp the second they loaded the page because God knows who they are. Authoritatively, even.

This is obviously impossible for us, because God isn't one of our investors.

But.. how close can we get to the perfect godlike login experience in Discourse? That's a noble and worthy goal.

Wasn't it Bill Gates who once asked why the hell every programmer was writing the same File Open dialogs over and over? It sure feels that way for login dialogs. I've been saying for a long time that the best login is no login at all and I'm a staunch supporter of logging in with your Internet Driver's license whenever possible. So we absolutely support that, if you've configured it.

But today I want to focus on the core, basic login experience: user and password. That's the default until you configure up the other methods of login.

A login form with two fields, two buttons, and a link on it seems simple, right? Bog standard. It is, until you consider all the ways the simple act of logging in with those two fields can go wrong for the user. Let's think.

Let the user enter an email to log in

The critical fault of OpenID, as much as I liked it as an early login solution, was its assumption that users could accept an URL as their "identity". This is flat out crazy, and in the long run this central flawed assumption in OpenID broke it as a future standard.

User identity is always email, plain and simple. What happens when you forget your password? You get an email, right? Thus, email is your identity. Some people even propose using email as the only login method.

It's fine to have a username, of course, but always let users log in with either their username or their email address. Because I can tell you with 100% certainty that when those users forget their password, and they will, all the time, they'll need that email anyway to get a password reset. Email and password are strongly related concepts and they belong together. Always!

(And a fie upon services that don't allow me to use my email as a username or login. I'm looking at you, Comixology.)

Tell the user when their email doesn't exist

OK, so we know that email is de-facto identity for most people, and this is a logical and necessary state of affairs. But which of my 10 email addresses did I use to log into your site?

This was the source of a long discussion at Discourse about whether it made sense to reveal to the user, when they enter an email address in the "forgot password" form, whether we have that email address on file. On many websites, here's the sort of message you'll see after entering an email address in the forgot password form:

If an account matches name@example.com, you should receive an email with instructions on how to reset your password shortly.

Note the coy "if" there, which is a hedge against all the security implications of revealing whether a given email address exists on the site just by typing it into the forgot password form.

We're deadly serious about picking safe defaults for Discourse, so out of the box you won't get exploited or abused or overrun with spammers. But after experiencing the real world "which email did we use here again?" login state on dozens of Discourse instances ourselves, we realized that, in this specific case, being user friendly is way more important than being secure.

The new default is to let people know when they've entered an email we don't recognize in the forgot password form. This will save their sanity, and yours. You can turn on the extra security of being coy about this, if you need it, via a site setting.

Let the user switch between Log In and Sign Up any time

Many websites have started to show login and signup buttons side by side. This perplexed me; aren't the acts of logging in and signing up very different things?

Well, from the user's perspective, they don't appear to be. This Verge login dialog illustrates just how close the sign up and log in forms really are. Check out this animated GIF of it in action.

We've acknowledged that similarity by having either form accessible at any time from the two buttons at the bottom of the form, as a toggle:

And both can be kicked off directly from any page via the Sign Up and Log In buttons at the top right:

Pick common words

That's the problem with language, we have so many words for these concepts:

  • Sign In
  • Log In
  • Sign Up
  • Register
  • Join <site>
  • Create Account
  • Get Started
  • Subscribe

Which are the "right" ones? User research data isn't conclusive.

I tend to favor the shorter versions when possible, mostly because I'm a fan of the whole brevity thing, but there are valid cases to be made for each depending on the circumstances and user preferences.

Sign In may be slightly more common, though Log In has some nautical and historical computing basis that makes it worthy:

A couple of years ago I did a survey of top websites in the US and UK and whether they used “sign in”, “log in”, “login”, “log on”, or some other variant. The answer at the time seemed to be that if you combined “log in” and “login”, it exceeded “sign in”, but not by much. I’ve also noticed that the trend toward “sign in” is increasing, especially with the most popular services. Facebook seems to be a “log in” hold-out.

Work with browser password managers

Every login dialog you create should be tested to work with the default password managers in …

At an absolute minimum. Upon subsequent logins in that browser, you should see the username and password automatically autofilled.

Users rely on these default password managers built into the browsers they use, and any proper modern login form should respect that, and be designed sensibly, e.g. the password field should have type="password" in the HTML and a name that's readily identifable as a password entry field.

There's also LastPass and so forth, but I generally assume if the login dialog works with the built in browser password managers, it will work with third party utilities, too.

Handle common user mistakes

Oops, the user is typing their password with caps lock on? You should let them know about that.

Oops, the user entered their email as name@gmal.com instead of name@gmail.com? Or name@hotmail.cm instead of name@hotmail.com? You should either fix typos in common email domains for them, or let them know about that.

(I'm also a big fan of native browser "reveal password" support for the password field, so the user can verify that she typed in or autofilled the password she expects. Only Internet Explorer and I think Safari offer this, but all browsers should.)

Help users choose better passwords

There are many schools of thought on forcing helping users choose passwords that aren't unspeakably awful, e.g. password123 and iloveyou and so on.

There's the common password strength meter, which updates in real time as you type in the password field.

It's clever idea, but it gets awful preachy for my tastes on some sites. The implementation also leaves a lot to be desired, as it's left up to the whims of the site owner to decide what password strength means. One site's "good" is another site's "get outta here with that Fisher-Price toy password". It's frustrating.

So, with Discourse, rather than all that, I decided we'd default on a solid absolute minimum password length of 8 characters, and then verify the password to make sure it is not one of the 10,000 most common known passwords by checking its hash.

Don't forget the keyboard

I feel like keyboard users are a dying breed at this point, but for those of us that, when presented with a login dialog, like to rapidly type

name@example.com, tab, p4$$w0rd, enter

please verify that this works as it should. Tab order, enter to submit, etcetera.

Rate limit all the things

You should be rate limiting everything users can do, everywhere, and that's especially true of the login dialog.

If someone forgets their password and makes 3 attempts to log in, or issues 3 forgot password requests, that's probably OK. But if someone makes a thousand attempts to log in, or issues a thousand forgot password requests, that's a little weird. Why, I might even venture to guess they're possibly … not human.

You can do fancy stuff like temporarily disable accounts or start showing a CAPTCHA if there are too many failed login attempts, but this can easily become a griefing vector, so be careful.

I think a nice middle ground is to insert standard pauses of moderately increasing size after repeated sequential failures or repeated sequential forgot password requests from the same IP address. So that's what we do.

Stuff I forgot

I tried to remember everything we went through when we were building our ideal login dialog for Discourse, but I'm sure I forgot something, or could have been more thorough. Remember, Discourse is 100% open source and by definition a work in progress – so as my friend Miguel de Icaza likes to say, when it breaks, you get to keep both halves. Feel free to test out our implementation and give us your feedback in the comments, or point to other examples of great login experiences, or cite other helpful advice.

Logging in involves a simple form with two fields, a link, and two buttons. And yet, after reading all this, I'm sure you'll agree that it's deceptively complex. Your best course of action is not to build a login dialog at all, but instead rely on authentication from an outside source whenever you can.

Like, say, God.

[advertisement] How are you showing off your awesome? Create a Stack Overflow Careers profile and show off all of your hard work from Stack Overflow, Github, and virtually every other coding site. Who knows, you might even get recruited for a great new position!
26 Oct 21:05

It’s Time?

by Ken Parish

In the midst of all the Whitlam nostalgia over the last week or so I couldn’t help thinking of the contagious hope and excitement that was generated by the “It’s Time” campaign theme in 1972. It still sends tingles down my spine listening to it today.

The year of Whitlam’s election was my first year at the University of Sydney on a Commonwealth Scholarship. My parents certainly could not have afforded to send me there. By the next year it didn’t matter because tertiary education was free, at least for a while. That year I was also looking down the barrel of conscription when I turned 20 in 1973, with the strong possibility of ending up in Vietnam. That prospect also ended with Whitlam’s election.

Can you imagine running an election campaign for Bill Shorten and today’s ALP using the “It’s Time” theme?  Time for focus groups, expedient cynical policies; time for me-tooism on data retention and anti-terror laws, and sending the troops off to make a risky token gesture supporting the Americans fighting ISIL in Iraq. Time for tiptoeing towards supporting Abbott and Morrison on turning back asylum seeker boats. Time for doing a deal on a reduced Renewable Energy Target, while opposing essential tax reform because the focus groups dictate it.

Exciting, inspiring, no? I can hardly wait to rush down to Labor headquarters and renew my long lapsed membership.

23 Jul 06:04

Drake - 0 To 100 / The Catch Up | Listen for free at bop.fm

by Ben Horowitz

“Real quick, whole squad on that real sh*t
0 to 100, n***a, real quick”
—Drake, "0 to 100/The Catch Up"

For the audio version of this post click here.

17 Jul 00:44

How academics, ministry experts, and civil society are losing: is the government now for the few?

by Paul Frijters

The latest federal budget in Australia by the Liberal Party was a real break with the recent past in which politicians were reluctant to offend any large group of voters and in which the status quo with respect to entitlements was avidly kept. There was a bit of playing around with extra money under Labor – spent on projects like the NBN – and there were some attempts at taxing the richest sectors more, such as the carbon tax, but it was largely a case of ‘All quiet on the Western front’.

This budget was different and seems to herald a shift in orientation of our political elites, not just the Liberal Party. What seems to have happened is that the political elites now take their cue from well-organised interest groups, to the detriment of the unorganised majority, effectively trailing the US by about a decade. The US saw the same move towards a ‘money talks’ society about 10 years ago, including the lifting of the Glass-Steagall laws that were meant to prevent the kind of financial piracy that lead to the GFC. In the US the trend is again reversing, but here we are just getting to the crest of money-talks politics. This is dressed up as going towards ‘small government’, but in reality we are talking about Government for the few. It is an inequality increasing agenda that rewards topic-specific organisation. Let me expand.

As Ross Gittins has pointed out in a whole set of articles on the budget, the headline changes are quite dramatic for the majority, especially for young poor people: the Gonski reforms, benefitting the least able within the schooling system, have been axed; the Carbon Tax, a tax mainly on a couple of big firms (mines and electricity generators), has been repealed; the age-pension, which is one of the main transfer programs, has now been indexed to inflation rather than average wages, which implies a 2% reduction in relative terms per year and 25% within about 12 years; the public school system and the hospitals will similarly see their commonwealth subsidies indexed to inflation, ensuring the same 25% decline in about 12 years; the cuts in parenting support similarly hit large parts of the population, whilst the effective halving of the unemployment benefits for the young (via the 6-months-on, 6-months-off rules) are estimated by the Department of Social Services to eventually impoverish close to half a million people.

One might see all this as indications of a move towards ‘small government’ and ‘starving the beast of government of funds’. That is certainly the storyline kept up by the Coalition and one that business economists bandy around also. It was the story of the Bush years in the US. If you look closely though, you will find it is not about small government at all. For you would have missed all the areas where government just got bigger. Substantially bigger. So look at the other changes to see the full picture.

In terms of small fry, the government has increased its budget for infrastructure and armament by several billion a year, benefiting a few large construction companies and arms dealers.

The more radical increase in government spending comes via the liberalisation of the Hecs fees though: this can be expected to double if not treble the government subsidies for education in terms of new loans, which is currently a flow of around 6 billion a year. The government could thus quite possibly lend out an additional 10 billion AUS a year in coming years, or 50 billion over the next 5 years. That is an increase in the size of the government that dwarfs all the other changes, cuts or increases. It will not be counted as new government debt because of the presumption that these loans will be paid back in the future, but, as the UK experience suggests, non-repayment rates can easily go to 30-40%, particularly when the loans are indexed to nominal interest rates and hence blow up quickly for anyone who cannot repay them fast. A 40% non-payment rate would mean an effective cost blow-out of 4 to 7 billion a year, much bigger than any other item on the budget!

So just on the proposed changes to Hecs alone, one should see this budget as a massive expansion of government, not a move towards small government at all.

And one should be clear as to whom the winners and losers of this expansion will be: the losers will course be the students and their parents, which is the whole of the middle classes. The winners will be a quite small but well-organised group of top-administrators at the top 10 universities in the country who have lobbied for these changes for years. So for the benefit of a few hundred people, who wield large bureaucracies inside their universities that help them lobby politics on this topic, this government is expanding its involvement via government-backed loans quite massively.

Similarly, the axing of the Carbon Tax (something I don’t actually mind because the Carbon Tax was never going to have the effect it was supposed to have) benefits large generators and rich shareholders in London and New York and other overseas places (who own around 85% of all the shares of our biggest 3 mining firms).

The same theme of how the budget benefits very small but well-organised groups at the expense of millions of voters pertains to many other changes. Private schools for instance have not been affected by the reduced subsidies to public schools as their implicit subsidies (via tax write-offs) have been untouched. Indeed, with the temporary increase of the top rate of income tax, private schools are now subsidised more by the government, not less!

The reduction in Corporate Tax will also primarily benefit a few very large firms that make lots of profits, including those big three foreign-owned mining firms (BHP, Tinto, Extrata).

Then think of what has not been affected: there has been no move to reduce the tax write-offs used by the Super-rich; no move to unravel the medical cartels; no move to reduce the fees charged by the superannuation firms; no move against the fuel subsidies for the mining industry; no increase in accountability of financial advisers.

Worse, down the line, it is expected that we are looking at reductions in the marginal rate of taxation about $200,000, something that will benefit only a few percent of the population but will cost the other 99%.

We are clearly looking at the effect of years of lobbying of interest groups behind the scenes and a fundamental shift in the orientation of political elites on both sides of politics towards this lobbying, ie towards the ‘stakeholders’ they hang out with.

The writing was already on the wall in 2010 when the Labor government did not manage to push through a mining tax which would essentially have meant a grab for the profits of largely foreign shareholders (!!). It was obviously beneficial to the Australian population, but could not be pushed through due to concentrated lobbying and media efforts of the big foreign-owned mining companies. It also revealed the failure of civil society, top ministry advisers, and of academics who were caught off-guard by the media-blitz. The politicians saw ‘us’ lose.

This loss heralded a short period in which the Labor government effectively was in tax limbo-land, unable to push through any major change that would adversely affect the well-organised and the well-funded. Worse, it was co-opted to fund fanciful and inefficient projects like the NBN, which also benefited the few over the many. There were many other examples.

The shift in power that was hinted at following the battle over the mining tax in 2010 has now become much more apparent and ingrained: it seems to have become the actual policy of the government in power to do the bidding of the few. Like many newspapers, politicians are simply going with the stories that they get bombarded with, offered for free, which are the stories by the well-funded few (as well as the baby-boomers). These are our ‘Bush years’, whether or not the changes survive the Senate.

So we are looking at a shift in political influence in this country, away from the masses and towards the well-funded and the well-organised. For the moment, academics, ministry experts, and civil society, still oriented towards the wellbeing of the many, have no answer. Ministry experts who could formulate alternatives are being fired, left, right, and center. We are losing. Big time.

But what are the underlying reasons for this shift in our political elites to do the bidding of well-organised vested interests? What has eroded the relative ability of the many to have their needs represented and championed in our democracy? Where is this ‘regulatory capture’ of the government by the well-funded and the well-organised leading us as a society?

25 Feb 16:24

App-pocalypse Now

I'm getting pretty sick of being nagged to install your damn apps.

This-website-has-an-ipad-app

XKCD helpfully translates:

Xkcd-download-our-app

Yeah, there are smart app banners, which are marginally less annoying, but it's amazing how quickly we went from "Cool! Phone apps that finally don't suck!" to this sad, eye rolling, oh-great-of-course-you-have-an-app-too state of affairs.

"Would you like to install our free app?!?" is the new "It looks like you're writing a letter!"

— Jeff Atwood (@codinghorror) January 9, 2013

Four years, give or take a few months, if you were counting. So what happened?

Millions of pointless apps

Your platform now has a million apps? Amazing! Wonderful! What they don't tell you is that 99% of them are awful junk that nobody would ever want.

Let's start with the basics. How do you know which apps you need? How do you get them installed? How do you keep them updated? How many apps can you reasonably keep track of on a phone? On a tablet? Just the home screen? A few screens? A dozen screens? When you have millions of apps out there, this rapidly becomes less of a "slap a few icons on the page" problem and more of a search problem like the greater web. My son's iPad has more than 10 pages of apps now, we don't even bother with the pretense of scrolling through pages of icons, we just go straight to search every time.

Walledgarden-cover

The more apps out there, the more the app stores are clogged with mediocre junk, the more the overall noise level keeps going up, which leads directly to this profligate nagging. Companies keep asking how can we get people to find and install our amazing app instead of the one question they really should have asked.

Why the hell are we building an app in the first place?

I want to know who exactly is going to all the trouble of installing the McDonalds app on their device instead of simply visiting the McDonalds website in the browser as needed. What problem does that app solve for french fry enthusiasts that it needs to be permanently installed on your device? Why are they giving away free Big Macs just to get people to install this thing?

Fragmentation into parallel and incompatible app worlds

It was so much easier when iOS was totally dominant and the iPhone was the only player. Before the iPad and tablets. Before Android got decent in 4.0 and Google standardized the Play store. Now there are, at minimum, four radically different mobile platforms that every serious app player has to support:

  1. Android phone
  2. iOS phone
  3. iOS tablet
  4. Android tablet

(For extra credit: how many of these are actually "mobile"?)

Unless you're careful to build equivalent apps in all those places, it's like having multiple parallel Internets. "No, sorry, it's not available on that Internet, only the iOS phone Internet." Or even worse, only on the United States iOS phone Internet.

If you're feeling generous, we should technically include Windows 8 and Windows Phone in here too. All with different screen dimensions, development stacks, UI guidelines, and usage patterns. Oh and by the way, that's assuming no other players emerge as serious contenders in the computing device market. Ever.

At the point where you find yourself praying for a duopoly as one of the better possible outcomes, that's … not a good sign.

Paying for apps became a race to the bottom

Buying an app is the modern Support Your Favorite Small Software Vendor Day. I was always fine with dropping ten or twenty bucks on software I loved. I'm a software engineer by profession; apps are cheaper so I can buy even more of them.

Have you ever noticed that the people complaining about apps that cost $3.99 are the same people dropping five bucks on a cup of fancy coffee without batting an eyelash? Me too, and I'm with the coffee people. $3.99 for your app? Outraaageous!

Now, contrast this with your app, Mr. Developer. I don’t know you from Adam. You’re pitching digital Instant Refresher Juice 1.0 to me in the form of a new app. The return I’m going to get is questionable at best. I already have 30 apps on my phone, some of them very good. Do I need another one? I don’t use the 30 I have. The experience I’m going to get from adding one more app is not trustable. I’m assured of nothing. Last week I bought an app for 99 cents and it was terrible. I used it once, for 15 seconds. I could be shoving $1 straight down the toilet again for all I know. Your app, good sir, is a total gamble. Sure, it’s only a $1 gamble… but it’s a gamble and that fact matters more than any price you might place on it.

For some reason I don't completely understand, mobile app review systems are frequently of questionable value, so all you really have to go on are the screenshots and a bit of text provided by the developer.

Imagine you bought your coffee, only to open the lid and find it was only half full, or that it wasn't coffee at all but lemonade. If only 1 in 5 cups of coffee you bought actually contained coffee, a $3.99 price for that coffee starts to seem unreasonably high. When you buy an app, you don't really know what you're going to get.

Turns out, the precious resource here isn't the money after all. It's your time. In a world of millions of apps, free is the correct and only price for most apps except those rare few of extreme, easily demonstrable value – probably from well known brands of websites you already use daily. So hey, everything is free! Awesome! Right? Well…

When apps are free, you're the product

I know, I know, I'm sick of this trite phrase too. But if the market is emphatically proving that free is the only sustainable model for apps, then this is the new reality we have to acknowledge.

Geek-and-poke-pigs-free

Nothing terrifies me more than an app with no moral conscience in the desperate pursuit of revenue that has full access to everything on my phone: contacts, address book, pictures, email, auth tokens, you name it. I'm not excited by the prospect of installing an app on my phone these days. It's more like a vague sense of impending dread, with my finger shakily hovering over the uninstall button the whole time. All I can think is what shitty thing is this "free" app going to do to me so they can satisfy their investors?

For the sake of argument, let's say the app is free, and the developers are ethical, so you trust that they won't do anything sketchy with the personal information on your device to make ends meet. Great! But they still have to make a living, don't they? Which means doing anything useful in the app requires buying three "optional" add-ons that cost $2.99 each. Or there are special fees for performing certain actions. Isn't this stuff you would want to know before installing the app? You betcha. Maybe the app is properly tagged as "offering in-app purchases" but the entire burden of discovering exactly what "in-app purchases" means, and how much the app will ultimately cost you, is placed completely on your shoulders. You, the poor, bedraggled user.

The app user experience is wildly inconsistent

Have you ever tried actually using the Amazon app on iOS, Android, and Windows? iOS does the best, mostly because it's been an app platform for longer than the others, but even there, the Amazon app is a frustrating morass of missing and incomplete functions from the website. Sure, maybe you don't need the full breadth of Amazon functions on your phone, though that's debatable on a tablet. But natural web conveniences like opening links in new tabs, sharing links, the back button, searching within the page, and zooming in and out are available inconsistently, if at all.

The minute you begin switching between platforms – say you use an iOS tablet and an Android phone and a Windows 8 touch laptop, like I do – you'll find there are massive differences between the Amazon apps (and the eBay apps, and the Netflix apps, and the..) on these different platforms. At some point, you just get fed up with all the inconsistencies and oddities and quirks and say to hell with these apps, can I please just use the website instead?

Now, if your website is an awful calcified throwback to 2003, like eBay, then the mobile apps can be a valuable opportunity to reinvent your user interface without alienating all your existing users. If there's one thing I love about tablet and phone design it's that their small screens and touch interfaces force people to think simpler. This is a good thing. But if you don't eventually take those improvements home to the mothership, you're creating two totally different and incompatible UIs for doing the same things.

It seems like a fool's errand to dump millions of dollars of development time into these radically different, siloed app platforms when Amazon could have spent it improving their website and making that experience scale a bit better to every device out there.

The World Wide App

But that's not an option, because apparently the web is dead, and mobile apps are the future. I'm doing my best to resist a sudden uncontrollable urge to use my Ledge Finder app to find the nearest ledge to jump from right now.

The tablet and phone app ecosystem is slowly, painstakingly reinventing everything I hated about the computer software industry before the web blew it all up. Even fans are concerned:

I’m waiting for something that will unify the world of apps and make manually going to an App Store to find a new app as weird as typing in a URL to find a new website. My bet is that this won’t be Facebook. Instead, I would not bet against some young upstart, perhaps one inspired upon reading about a $19 billion deal, to go heads-down and come up with something crazy.

I'll have more to say about this soon, but I expect there to be an explosion of new computing devices all over the world in the next few decades, not a contraction. Sometimes the craziest solution is the one that's been right there in front of you the whole time.

[advertisement] Hiring developers? Post your open positions with Stack Overflow Careers and reach over 20MM awesome devs already on Stack Overflow. Create your satisfaction-guaranteed job listing today!
25 Dec 07:53

NetTuts.com: BDD With Behat

On NetTuts.com they've posted an introductory level tutorial for those that are exploring the world of testing and want to get their feet wet using some other tools. In this new post they introduce Behat, a human-readable functional testing tool written in PHP (but not just for PHP applications).

The BDD PHP framework Behat, allows you to test your PHP applications using human-readable sentences to write features and scenarios about how your applications should behave in order to test out its functionality. We can then run these tests to see if our application is behaving as expected. Let's spend about 15 minutes quickly going over Behat's installation process and learn how we can test our PHP applications behavior using the basics.

They help you get the tool installed (via Composer) and show you how to create your first feature file. They use a simple "adder" class for their examples that has two methods - add and display. The article covers features, scenarios and steps - including the code you'll need to add to the context file for any custom steps. Finally, they show how to run the tests and a sample of what the output looks like when everything's successful.

Link: http://net.tutsplus.com/tutorials/php/bdd-with-behat
14 Aug 21:48

MBA Mondays: Turning Your Team

by Fred

A serial entrepreneur I know tells me "you will turn your team three times on the way from startup to a business of scale." What he means is that the initial team will depart, replaced by another team, which in turn will be replaced by yet another team.

I have been closely involved with over 150 startups in my career and since roughly 1/3 of the startups we back get to real scale, that means I've seen the "startup to scale movie" over fifty times in my career and I can tell you this - my friend is right.

The people you need at your side when you are just getting started are generally not the people you will need at your side when you have five hundred or a thousand employees. Your technical co-founder who built much of your first product is not likely to be your VP Engineering when you have a couple hundred engineers. Your first salesperson who brings in your first customer is not likely to be your VP Sales. And your first community person is not likely to be your VP Marketing. 

Likewise, the first VP Engineering who figured out how to manage the unwieldy team left by your technical co-founder is not likely your VP Engineering when you have five hundred engineers. Your first VP Sales who built your first sales team is not likely the person who can manage a couple hundred million dollar quota. Companies scale and the team needs to scale with it. That often means turning the team.

The "turning your team" thing probably makes sense to most people. But executing it is where things get tricky and hard. How are you going to push out the person who built the first product almost all by themselves? How are you going to push out the person who brought in the first customer? How are you going to tell the person who managed your first user community so deftly that their services are no longer needed by your company?

And when do you need to do this and in what order? It's not like you tell your entire senior team to leave on the same day. So the execution of all of this is hard and getting the timing right is harder.

This is where serial entrepreneurs have a real leg up on first time entrepreneurs. They have seen the movie too and they played the starring role. So they know what the next scene is before it even starts. They know the tell tale signs of the company scaling faster than their team. And so they move more quickly to move the early leaders out and new leaders in. One of the signature faults of a first time founder is they are too loyal to their founding team and stick too long with them. 

If it is any consolation, the founding team makes most of the money when a company becomes successful. That technical co-founder who built the first product will likely end up with tens of millions of dollars, if not a lot more, if a business they helped start gets to five hundred or a thousand people. The VP Engineering of a five hundred person company will not likely have an equity package that is worth anywhere near that much.

So I generally advise entrepreneurs to be open and honest about all of this. Tell your early team that they may not make it all the way to the finish line but they will be handsomely compensated with equity and if you are successful, they will be too. And when it is time for them to go, think about how much they brought to the company and consider vesting some or all of their unvested stock on the way out. Also think about compensating them to stick around during the transition. And always make sure they leave the company with their head high feeling like the hero that they are. 

Here's the thing. Turning a team is not the same as firing someone for weak performance. You are firing someone for doing their job too well. They killed it and in the process got your company off to a great start and growing to a scale that they themselves aren't a great fit for. They may not be right for the job at hand, but they are a big part of the reason that the company is successful. That's the narrative that you need to have in your mind when you turn your team.

All of this is very hard, particularly if you are doing it for the first time. So get some mentors, advisors, and board members who have lived through this before. And listen to them about this. You may not want to listen to them too much about product and market stuff. Maybe you understand that better than they do. But when it comes to scaling a management team, those who have had to do it before will generally be right about the issues you are facing with your team. So their advice and counsel is worth a lot and you should pay close attention to it.

12 May 04:58

Big infrastructure, big uncertainty

by David Walker

One of the peculiar features of debates about big monolithic infrastructure projects, such as universal broadband networks and high-speed rail lines, is the way their supporters talk about them in public. To advocates, the wisdom of these projects is obvious. You can never have too much broadband! High-speed rail is the future! Why can’t we be like the visionaries who built the Snowy Mountains Scheme?!? $50 billion, $100 billion? “Chicken feed” is what we’ll call it in 20 years!

And indeed some opponents of these projects take the same attitude from the other side of the fence. Everything’s fine as it is! This new thing will be an enormous white elephant, obsolete before it is finished, you can bet on it!

What we don’t usually talk about is Knightian uncertainty – that is, risk you don’t know enough to quantify, or sometimes even recognise.

This was what former US defence secretary Donald Rumsfeld was talking about when he famously spoke of “unknown unknowns“. That Rumsfeld’s comment is frequently ridiculed just shows how alien considerations of uncertainty are in the public discourse*. Almost no-one wants to say, “well, we just don’t know”. There are things you might do if you just don’t know. They rarely get talked about.

But if you’re having a serious conversation about infrastructure, you have to talk about uncertainty. Uncertainty matters a lot for big monolithic infrastructure projects, such as universal broadband networks and high-speed rail lines: the time they take to complete and the technology environments in which they operate mean there’s great uncertainty about their ultimate benefits and ultimate demand. By the time you deliver them, the world may be in the process of passing them by. Demand risk is always a factor in these projects, but here we seem to have substantial demand uncertainty, which is worse.

This may be particularly true over the next few years, because predicting infrastructure demand growth seems a tough job right now. In the broadband business, you have to make a guess about the capability of mobile broadband and about willingness to pay for ultra-high-definition video streaming. In more traditional infrastructure, car and power use are currently and unexpectedly declining, and we don’t know whether these developments are structural or cyclical (and if they’re structural, we don’t know whether they are new trends or one-off adjustments).

My gut feel is that the shape of our transportation and communications networks is less predictable now than at any time in the past 50 years. Maybe I’m wrong. But that’s Knightian uncertainty for you: you can’t really size it up.

A few people are looking at the effects that technology forecasting has on infrastructure projects. From rail transport analyst Christian Wolmar:

One of the reasons for my scepticism about [UK high-speed rail project] HS2 is on the basis that it does not take into account future development of technology. Just look at how technology has changed since 1993 when mobile phones had barely taken root, Google, Facebook and Twitter were but twinkles in their founders’ eye and digital TV was just starting. Will there really be enough people wanting to pile into what are likely to be expensive trains in 20 years time to justify the huge expenditure on this project?

And here’s where I stick my neck out. The next big technology, one with such huge implications that it is impossible to being to predict them, is driverless cars … Perhaps they will start by being driven only on motorways but even that would have enormous consequences. It would combine many of the advantages of train travel with the flexibility of car use. Think trucks, too. The economics of transport would change as radically as they did when the railways were first developed. The time frame may be a decade or two, but the consequences will be much more far reaching than, say, the much talked-about electric cars. The driverless car – or rather motor vehicle – is the innovation that we ought all to be taking into account in our future thinking.

From University of Minnesota mathematician and network capacity expert Andrew Odlyzko:

How many times have you seen predictions and promises that better communications, such as faster Internet access, will stimulate telecommuting and decrease road congestion?
Such predictions are almost certainly wrong. At least they have been consistently wrong for about two centuries. Many, often very knowledgeable, observers, thought that transportation and communication were substitutes for each other. But the uniform experience to this age has been that they are complements, and grow in parallel. Yes, you may work from home, but chances are that you will make more trips to meet clients, or for family and other reasons, and in the end will travel more than before (barring major upsets, such as astronomical rise in price of fuels).

So what can we conclude? Most of all, that the future is hard to predict, so we have to prepare for the unexpected. Second, though, we should keep in mind that among the most common unexpected phenomena is the resilience of old technologies, services, and business methods, and their propensity to adopt some of the innovations that we work on.

If we conclude that big monolithic infrastructure projects need to factor in uncertainty, what do we do?

The simplest course is to raise our estimation of the risks. But how much? Bent Flyvbjerg recommends a technique called “reference class forecasting“, which predicts the outcome of a planned action based on actual outcomes in a “reference class”. But what’s the right reference class, and how useful are past projects, when the environment is changing fast?

Another option is to try to break down the project into smaller units, which we can tackle one by one. That only goes so far with high-speed rail, where we essentially have just three big legs (Melbourne to Canberra, Canberra to Sydney and Sydney to Brisbane). It does make a lot of sense for a fibre-to-the-premises broadband network, since such a network is by its nature mostly local in nature. It’s why we should be improving our existing broadband networks suburb by suburb, as suggested by Joshus Gans in a paper for CEDA back in 2007.

We should also, of course, adjust our thinking about such monolithic infrastructure – “megaprojects”, as Flyvbjerg has dubbed them. If the most recent study of the Melbourne-Sydney-Brisbane high-speed rail system says the environmental benefits mostly emerge from around 2057, for instance, we should probably avoid making very much of those benefits.

But first, we need to talk about big infrastructure as if uncertainty mattered. The unknown unknowns are real, and we need to acknowledge them.

* Note: You don’t need to agree with Rumsfeld about the conduct of US defence policy in order to think he was, on this occasion, talking sensibly about risk. As John Quiggin has noted in his own defence of Rumsfeld’s comments, the bloke could have usefully thought more and earlier about the repercussions of Knightian uncertainty for US attempts to reshape the Middle East through a war in Iraq.

10 May 09:30

Christopher Martinez: Static code analysis tools for PHP

Christopher Martinez has a recent post to his site that covers some of the static analysis tools available for PHP including the PHP Mess Detector, PHP CodeSniffer and the PHP Analyzer.

I believe in writing code that is easy to understand, easy to test, and easy to refactor. Yes, I realize that the statement above is pretty general and open to interpretation. Not everyone needs external tools to ensure quality in their code...but, I work on things from time to time that have absolutely no tests. [...] For whatever reason, this happens a lot more frequently in the PHP world. I'm guilty of not writing tests and checking how I write code, sometimes, too. Things are bright, though, for the PHP community - for quite some time now, we've had fantastic tools that assist us in writing better code.

He covers each of the tools, talks some about what they're good for and gives examples of their use, including output. He also talks some about the Pfff set of tools created by Facebook. He also talks some about how these tools fit into his daily work as a part of his pre-commit hooks in git.

Link: http://chrsm.org/2013/05/05/code-analysis-tools-for-php
02 Apr 07:07

The choices we made but never decided upon, part I.

by Paul Frijters
Nic Lowe

This is...quite thoughtful.

Let us pretend you are the benevolent elected dictator in Australia. It is 1980 and you have to decide on education and migration policy. Your wily political adviser comes to you with the following plan: he tells you it would be popular and cheap to stop inflicting difficult and painful education on Australia’s kids, instead importing foreigners who have gone through the pain of elite education.

Your adviser points out the many advantages: you can halt pay increases for teachers immediately which, as Andrew Leigh so nicely showed, gradually dumbs down the stock of teachers, particularly at the bottom end of the market. The incoming migrants have to buy a lot of stuff when arriving so you get immediate boosts to the economy, quite apart from the free human capital walking in.  You let those with a chip on their shoulder regarding the philosophy of current education do their thing. It’s the road of least resistance on all sides.

After some reflection you realise it’s a brilliant plan. All your kids are winners, none of them failing a class and each of them frequently getting rewards for breathing quietly, whilst they will end up in admin jobs where they mainly write reports for other admin people. The bright kids can invest in networks and fight it out with the other networkers to become the managers. Meanwhile the ‘tech’ foreigners do the hard yakka in the mines and behind the scenes. What is not to like about this joint proposal on education and visas?

The only thing not to like about the proposal is that it was never proposed. It was never consciously decided upon. But we ‘sort of’ did it this way anyway!

As far as I know hence, this choice was never consciously debated in Australia in the era of the 1980s, the Dawkins reforms. There wasn’t a clear first move towards it, with both Australia’s education and visa policy subject to numerous independent reviews, white papers, and complicated deliberations; all involving many heated debates about lofty goals and utopian visions, but none of which to my knowledge led to a conscious choice for the joint proposal above.

Yet the proposal above does describe how our education and visa policies actually co-evolved in the last 30 years: it is as if we consciously decided upon this path 30 years ago. Worse, it is as if we consciously decided upon it, and then purposely muddied the waters with thousands of pages of text pretending that we were choosing something else.

Whilst dumbing down schools and universities has become an art-form in itself(see, in particular the recent PISA results strongly suggesting that our school system is lagging internationally and this book on university decline), the reason that it has managed to go on without serious national backlash is because of the visa system: Australia’s industry can get the skills elsewhere, whilst most home-grown kids get good jobs anyway(with notable booms in the number of HR positions). In line with the large US-based debate on skill-biased change, one emerging view of the Australian labour market is that it too has a very skilled top end but with lots of workers not using all that many skills at all, stuck in dead-end jobs. Also, the resource boom has reduced the importance of human capital in the Australian economy via a Dutch disease effect.

It is not hard to see in hindsight what the lock-in mechanisms were to cement the choice: with human capital walking in for free, the pressure to improve elite education at university never materialised; with no pressure to improve schooling, the temptation to reduce teacher pay and give in to the hundreds of reform plans that effectively meant more administrators and less teachers was too hard to resist. And once the reliance on overseas skills became ingrained, there was a constituency of employers who became used to attracting foreign workers and thus no longer wanted to revert to domestic ones.

Similarly, the visa policies followed a road of least resistance: industries wanted skills Australian schools and universities were no longer providing, so politicians opened the gates and overseas countries provided the skilled foreigners who came for the great culture and wealth Australia has to offer.

And if you think about it objectively, the choice is not a bad one to make. It has basically allowed a generation of Australian kids to grow up in the happy belief that they are skilled without having to really push themselves, while at the same time allowing foreigners to improve their lot in life by relocating here. Where is the loss, one might ask? Australian culture is still doing fine and our country is still quite empty so the choice can be extended into the future without much trouble. It is certainly much easier politically than tackling the educational institutions, so why bother? As long as the boom lasts, the road-of-least resistance is to maintain the current course.