Shared posts

21 Aug 16:59

It’s cargobike love at first sight, but can we live together?

by mark simmons

When I first saw the spec on this cargobike way back sometime in 2018 I was blown away. This is maybe the perfect cargo bike, long-tail variety that is, for me. Dunno about you. I cannot see a time when I would transport a tiny child behind me where they can’t be seen, talked to and cared for when longjohns and box trikes exist? And I don’t even like kids!!!! But I do have a Chihuahua I carry most days on the front of my my bicycle if that counts!!!

Then I saw the photos. Oh wow. Looking sooooooo good. Solid, two batteries, big bags, big front basket, bloke riding it, chunky 20” wheels front and back (my favorite), and made by Tern! And haven’t they got the best PR pic ever with the kid giving a finger to the sky like they’re on a roller coaster. Genius.

I have a Tern Cargo Node, its a factory edition cargobike collaboration between Tern and Xtracycle. Its the nicest bicycle I have EVER ridden (so far!), so I was stoked when they announced the GSD. But what does GSD stand for? So far, from my research, and this isn’t from Tern, it means Get Shit Done. Funny if it’s true. Either way, it does, and then some.

I had this bicycle for a week to play with… I mean review ;-), mainly commuting with chores some days. My first impressions are that it’s like riding on a magic carpet – a sexy, blue, cargo carrying magic carpet. Well sort of. It’s so nice and smooth that I forgot what I was riding. This may be because I’m used to the gentle whine of a hub motor, the GSD mid-drive silence adds to the feeling of riding a non-electric bike. The Mid-drive also means the GSD feels like riding a regular bike as the torque sensing motor system only gives more if you pedal harder, unlike my hub drive where I can just turn it up regardless of my pedalling effort. I ended up really enjoying the torque sensor system as it required no real attention from me.

On my commute there is a long bridge. I powered up the bridge not fast (like on a Stromer or OHM) but in a measured and civilized way. The power and speed increase smoothly, so I always felt in control. This also meant that the steering was not negatively influenced by take off power either. In comparison, the long-tail Yuba Spicy Curry did feel faster, like a colt, you definitely have to hang on. The GSD is more like a pony, you will get there quick enough, but be un-flustered on arrival.

Riding down the other side of the bridge at 37kph it took about 30-35 feet to stop. I was hardly loaded and I did get some rear wheel skidding, but I think that’s the quickest I have ever stopped on any Bicycle. Gotta love hydraulic disc brakes. Especially on a cargobike that can carry kiddies.

But THE hill was next. A short lung buster called 3rd Street Hill. Usually I take the long way round through a park so the climb is less severe. But this is a review, therefore this cargobike captain went straight up it like a speeding lycra clad roadie!

Flattening hills is why I ride electric

Straight up, steady and way easier than I get on my bicycles, someone less heavy would speed up there, wonderful climb. And not that much noticeable battery fade either. A real test of the low torque power of the Bosch system. Almost enough to turn my head away from Hub motors.

I’m commuting in the dark at this time of year. So built in lights are great, no little batteries to fiddle with, just press and hold the + on the controller and we have light, nice. Its been said before, the headlight is fab, really fab, wide and bright beam, but not too bright to dazzle oncoming riders. It’s also super easy to aim down on a city bike path and aim high on the dark bits going home. But the rear light is half hidden by the bags so no side light, I expect the angle of visibility is less than 90 degrees, where 180 is preferable, though the bags do have reflective accents. Fixable with a $25 secondary seat post light which lights up the deck and adds a ton of side visibility.

Bags are big and good and tough. The hook system took me a while to get used to as quite unlike the more usual clip systems of other pannier/market bags. They are about as large as on my Tern Cargo Node cargobike, maybe a little bigger but they don’t have the sling system. They are however perfectly rectangular and deep unlike my Xtracycle bags on the Cargo Node that taper at the bottom. I am 90% sure the GSD ‘market style’ bags will suit 90% of users, especially a parent with odd shaped kids toys to haul and a couple of bags of groceries too. Durable simplicity is the Market bag’s strength, whatever the brand, whatever the cargobike.

Blue Tern GSD review riding the city electric cargobike
Photo courtesy of Tern

A machine of this quality is of course expensive. Locally I can order a GSD with a single battery and no accessories for $5,500 Canadian. But the spec is such that you don’t need to upgrade a thing which can account for quite a lot of money over the first year of ownership.

I adore the 20” wheels and tires. Wow. 2.5” wide rubber made by Schwalbe specially for this bicycle. The confidence this kind of wheel set-up gave me rolling down rough old Vancouver streets was amazing, even on roadside gravel I was solid all the way. In torrential Vancouver rain the GSD cargobike stuck to the line perfectly during any turn. These chunky wheels and the chunky frame just shouts strength and reliability. Plus a 400lb gross load limit, what’s not to like?

Well there are a few niggles I have about the GSD

  • At 6’4” with a 33” inseam I felt too close to the bars, a bit squashed up, I adjusted everything and it was just OK with the cockpit feel. I’m tall because I have a long back. A more ‘normally’ proportioned body will be fine.
  • I keep kicking the bags walking with it and pushing off.
  • I mentioned the half hidden back light already.
  • Hate keyless activation. My bikes have a key so if it’s stolen they can’t ride it away under power when the key is in my pocket. Dislike control panel removal as a key even more. I can have a spare key but not a spare panel.
  • It’s heavy and feels more like a moped than a bicycle. I could manoeuvre it on and off the Seabus just fine because it has a regular bicycle wheelbase, but if you have to lift an end it’s tough, and I’m a big strong lad but with a bad back so I have to be careful. It’s 60lbs with one battery, so over 70lbs with the double battery system I was trying. It was also a chore because of weight to get in and out of the elevator at home. All this said, does it matter? With kids and groceries it could weigh 150-200lbs without the rider and who’s going to lift that? It’s more of a commuter that lives in a concrete cave in the sky kinda gripe really.

To sum up

Would I buy one? If I had a garage I could simply roll into and park, yes. Or easily accessible bike room parking that’s 110% secure, yes. But I don’t, so I wouldn’t. The weight is the deal breaker for MY lifestyle. If you are in a house that want’s to get rid of a car, which is the GSD’s selling point, or a business that delivers, then yes, without a second thought, yes. It’s an awesome electric vehicle.

Update

Since I wrote this Tern have released the HSD which looks like it could negate all my living in a concrete cave in the sky gripes.
Also, there is an all new version of the GSD now.

Originally published February 2019 in Momentum Magazine.
Gently tweaked June 2021 for Lifestyle Cyclist Magazine.

The post It’s cargobike love at first sight, but can we live together? appeared first on Lifestyle Cyclist.

28 Jun 06:39

1x11 Shifting Components are in!

by noreply@blogger.com (VeloOrange)

by Igor

Sensah Components has been in the drivetrain game for a number of years producing shifters, derailleurs, cranksets, and cassettes for just about every type of drivetrain and bike. More recently, they've gotten into 1x (1by) drivetrains for mixed-terrain and off-road touring bikes (read: gravel). And so, we are pleased to now be offering SRX and CRX components for 1x drivetrains!

1x drivetrains are here to stay. They're easy to maintain, quick to set up, and are simple enough for anyone to hop onto. And as such, they are great for commuting, touring, and all-around adventuring. I've personally converted several of my bikes from double drivetrains and I really like the simplicity, minimal aesthetic, and clutch'd derailleur offerings for off-road use.

In testing out their components, I outfitted my Polyvalent Low Kicker with a suite of SRX drop bar shift/brake levers, rear derailleur, and 11-46 cassette. I've taken the bike on all sorts of rides and terrain from gravel to single track to pavé and I'm happy to report performance has been absolutely flawless. Let's go through each component and discuss their features!

SRX 11 speed Rear Derailleur

The SRX Rear Derailleur features aluminum alloy and steel construction for durability and has a max cassette range of up to 52t. 

The derailleur features a spring clutch so it keeps the chain from bouncing around too much on rougher terrain. I put mine on max and find the shifting action still easy.

There is also barrel adjustor (without detents) for quick adjustments.

SRX 1x11 Integrated Shift/Brake Levers

The SRX brake/shifter levers are 1x11 only and have excellent ergonomics. For anyone who has used integrated shifters, the shifting action will be right at home. To shift into a harder gear, a light tap of the blade will release cable tension and provide you with a satisfying clunk of the rear derailleur. Pushing the whole lever will increase cable tension and move you up (easier) the cassette. You can do 3 gears at a time with a full sweep.

The left lever is purely just a brake lever. Nothing much to see.

Each lever has a simple reach adjustment. A turn of a 2.5mm allen wrench can move the lever closer or further away from the handlebar. A major convenience for those with smaller hands. 

There is also a nice bit of flare in the blade for easier shifting and ergonomics.

CRX 11 speed Flat Bar Shifter

The Flatbar CRX shifter has the same pull ratio as SRX, so you can use the SRX Rear Derailleur. The shifter is simple and effective. There are two levers. The lower one can shift up the cassette changing a maximum of 5 gears with a full sweep.

The upper lever has dual-action meaning that you can shift into a higher gear with either a press of your thumb or a pull of your index finger.

11 speed 11-46 Cassette

While you mostly see 1x drivetrains on modern-looking bikes, a 1x conversion is perfect for older bikes, too! This 11 speed cassette uses Shimano HG splines, so if you have a 8-10 speed freehub body designed for Shimano HG cassettes, then you can upgrade to an 11 speed drivetrain.


While the derailleur technically goes up to 52t, we have found that 46t is about the max that provides consistently good shifts without significant jumps in gearing. 11-46 is a very generous range and when paired with a 42 or 44t chainring, you still have that magic sub 1:1 gear for steeper climbs.


I pull around our son in his trailer or our dog in her trailer up hills with a 42t x 11-46 drivetrain and don't find the need for anything lower. For us, anything lower would be slower and less efficient than walking.

11 speed Chains


What can I say?? They're 11 speed, solid pins, and have a simple quick link. There isn't much to them and they work well. And they're silver!

Other Chains

It's been pretty obvious that for the past 15 months consumables like chains have been hard to come by. So, we imported a bunch of 5-8, 9, and 10 speed chains. They're quality chains. And like the 11 speed version, they have solid pins and a simple quick link for installation. And they're silver!


28 Jun 06:37

All kinds of online marketplaces are creaking under scams

I put out a shout for a graphic designer on Twitter a couple of days ago. I was bombarded by direct messages from accounts that presented as people, and talked like people, and tweeted like people… but their portfolios were stuffed with generic logos clearly pumped out by software. Opportunist individuals, side-hustling their way to projects they then outsource? Or paint-by-numbers design farms that use fake “young designer” bots as a sales channel?

Then there are the Instagram ads for well-photographed products with well-put-together brands – that I then find on Alibaba being sold directly out of multiple factories. Ditto when you search on Amazon for digital scales for baking, or wristbands for running with key pockets, or STEM toys (to pick three recent examples) and the results are swamped with semi-identical products from a dozen different brands.

Except for the disingenuous authenticity, it’s unfair to call these scams. The system is functioning as intended. The system was design as a marketplace, and indeed buyers are being connected to sellers… only there are invisible intermediaries who are excellent at targeting the channel but contribute zero added value.

Though there are also actual scams to be found elsewhere:

  • Apps on the Apple App Store that pay for promotion, and fool users into extortionate subscriptions. (It only needs to work for a small percentage to be profitable.)
  • Facebook ads for products that take your money and never ship. (I’m pretty savvy and do my due diligence, but even I’ve been fooled once or twice.)

IN THE MIDST OF THIS there are legit new companies starting, and legit new people to work with. But it’s getting increasingly hard to find them (and once found, trust them) through the noise of the scams.


What spam is to communication, scams are to marketplaces.

Only there’s no way I can install an anti-scam filter.

My hunch is that after 20+ years of scaling marketplaces of all kinds, reducing friction and increasing activity, we’re hitting a wall similar to the malware wall hit by Windows (and ultimately “solved” by the shift to managed computing led by iOS), the spam wall hit by email (Gmail’s spam filter was a band-aid; ultimately comms moved off email into WhatsApp and corporate messaging), and the disinfo wall hit by large social network (not yet solved, but we can see attempted solutions in form of private Discords and the rise of the other cosy online spaces). Like these, the fix isn’t just more of the same.


So assume this problem is getting worse. What is to be done?

Two solutions from history no longer work in 2021:

  • Brands. A brand is a hostage – you know the company won’t do anything awful because they risk a brand which has taken time, money, and good behaviour to develop. Call it reputation. But we consumers can’t tell reputation directly, we can only look for signifiers: have we encountered the brand a lot; do other people appear to transact with it; does it look expensive; etc. And online, all of those signifiers are cheap to fake. A new scam brand can be indistinguishable from a established yet new-to-me trustworthy one.
  • Retailers. The other problem with brands is that you do want to buy from new ones, so one role of trusted retailers - in the past - has been to pass on that trust to the brands they select. Our brains believe that trust is a transitive property. But it isn’t: trusting Amazon doesn’t mean you can trust the merchants; trusting LinkedIn doesn’t mean you can trust the approaches you get there. Is there room for a retailer just like Amazon only it carefully vets all its merchants? Sadly I don’t believe so: a retailer is also about footfall, and such a selective retailer will never become the default shopping destination that Amazon is. Besides, we need a solution that works for Facebook ads (no Amazon) too.

I can speculate…

What if every brand had some kind of digital certificate, and anybody in my trusted networks could anonymously certify that they had had a good experience? (By networks I mean: mutuals on Twitter, Facebook, and Instagram; people I correspond with on email; and so on.) My initial model is HTTPS, which guarantees that your web browser has a connection with a certified endpoint, and no intermediaries have futzed with the data.

And then the certificate would be displayed as a badge wherever I see that brand in a channel, whether as a Facebook ad, in a list of Amazon results, via search, or on their own website.

Maybe instead of social signals, I could subscribe to a service that whitelists and blacklists brands, and use that as my source of trust instead. Perhaps credit card companies could also feed into it: when a purchase is made, the digital certificate transfered with the payment authorisation, and attached to any future chargeback or refund.

This would be something presented as an overlay on existing large web properties, so it probably has to be independent from them and built into the browser somehow. Instead of being yet another startup, could it be a protocol someone, something that everyone could adopt, large and small?

The key point is to decouple trust from the retailer itself (as marketplaces such as Amazon are unable to provide this), and to make the badge visible on every single discovery surface.

I’ll leave it as an exercise for the reader to extend this to freelancers and LinkedIn. To my kind it’s a similar-shaped problem. Neither is about identity (you are who you say you are) but about misrepresentation (your implied characteristics are the same as your actual characteristics).

We need big, distributed, imaginative solutions.


SEMI-RELATED:

Here is a story about the first thing I ever bought online.

Back in the late 90s there was this new thing called e-commerce. i.e. buying stuff on the web.

So in 1997 or maybe early 1998 I decided to buy something online for the first time. I mean, typing my credit number in an online form and everything, not just selecting from an online catalogue.

But I decided that, because e-commerce would plainly dominate in the future, I would purchase something that was in some way emblematic of the whole absurdity of e-commerce, to mark the occassion.

Here’s a photo of what I bought. I still have it. (Or rather: it lives in my mum’s garden.) It’s a garden ornament.

  • It’s ugly. It’s a grotesque!
  • It’s super heavy. Exactly the wrong kind of object to send through the post.
  • It’s not real. It’s not a stone ornament, it’s made of resin and filled with sand for the weight.

So it tickled me to get this nasty simulacrum as a kind of conscious foreshadowing of the rest of my online life.

But knowing what I know now, I should be thankful that I didn’t receive just a photocopy of a picture of the thing in an envelope, or that it actually turned up at all.

28 Jun 06:36

Galaxy Tab S7+

I impulse-bought this big Samsung slab which I guess represents the state of the art in Android tabletry and is trying to occupy an iPad-like spot in the ecosystem. It’s got issues but I’m keeping it. I’m writing this based on my perception that not many people have a tablet that’s not an iPad, so the territory is only lightly explored.

Credit is due to Nelson Minar, who has a sort-of-blog where he diarizes his personal tech divagations. It was his short, to-the-point piece on the S7+ that awoke the impulse and got me here. If you’re interested at all in this thing you should go read that. I’ll wait.

This is not gonna be an iPad comparo, if only because I’m not up-to-date on those. I occasionally use a two-year-old basic entry-level iPad and it’s cool, but I don’t think it’s trying to be what this is trying to be. I’m not too worried; I think that there are a lot of people who (like me) are pretty invested in either the iOS or Android ecosystems and, if they’re the latter and want a tablet, this is going to be a strong candidate.

[Update: Nelson dropped in a comment below pointing to his earlier post, Android Tablet: Samsung Galaxy Tab S5e, which talks more generally about Android tablets and software on them.]

Samsung S7+ tab displaying an MLB.tv ball game

The best part…

Nelson opened with the screen quality and wow, well yeah. if you want specs go look at the Wikipedia entry but let me tell you, it’s really big and really bright and really sharp.

The screen is made much more useful by this, uh, flat thing, that attaches magnetically to the back of the tab and has a little bulge on it to store the “S pen” stylus. It’s got a flap thingie at the bottom that you can bend out and back — the resistance is stiff — which is designed to prop up the tablet, both horizontally and vertically; the pen-holding bulge keeps it from falling over. Here’s a picture.

Galaxy Tab S7+ mounted vertically running Kindle.

This is Kindle displaying the 2nd page of Fritz Leiber’s Our Lady of Darkness, which by the way is the best fiction ever written, if your criterion is imaginative and skilful use of San Francisco as a backdrop.

I normally read books either on paper or on a Kindle Oasis, and like both. But I have to say I really like having a huge slab of bright, crisp, well-typeset text that holds itself up so I can scratch my butt or sip my coffee while I read. I’ve already inhaled one book on it and I’m sure there will be more.

The Economist app is brilliant too; many articles fit on a single page. Nelson mentioned comic books, which I haven’t tried yet.

Fast!

Really freaking fast, I mean; another nice thing about the device. I’ve never felt the urge to complain about my (now-outmoded I guess) Pixel 3’s performance, but this does everything faster. Plus once you’ve experienced 120Hz scrolling, it starts to feel addictive.

Android

Off the top there’s all this weird Samsung shit in your face but I followed Nelson’s advice and dropped in Nova Launcher, yielding a very Pixel-like experience. And Android, all these years later, has the best notification system of any computing environment, any form factor, that I’ve ever been near. To this day I’ll be working on my Mac and a couple of notifications will float up in the corner of the screen when I’m zoned in on something else; so what I do when I want to refresh context is (walking across the room first if necessary) grab my Pixel and pull down the notifications to see what’s happened.

Plus the gestural navigation and (maybe most important?) a Back button that basically always does the right thing.

The worst thing

It’s a real klunker. If you put the back-case-thingie on the back and the keyboard (yes, we’ll get to that) on the front, the combo is heavier than my wife’s 13" M1 MacBook Air. I usually just have the back thingie attached, because it’s so useful and also the pen-holding bulge is a comfy carry-grip. Feels heavier than I’d like.

Stripped of all attachments it’s acceptably light, I guess, but my hands don’t like the sharp corners.

Photography

Yep, it’s got a camera. At one point in history I wondered why tablets might need them, then I saw tourists walking around taking pictures with iPads and realized that they had attained the long-cherished ideal of WYSIWYG photography. So why not? Here are two pictures I took just now with it.

White rose, shot with Galaxy Tab S7+ Native honeysuckles, shot with Galaxy Tab S7+

They’re OK. What’s actually interesting is that they were not only shot but processed on the S7+ with Lightroom, which is actually pretty delightful on this device. Delightful enough, in fact, that I wonder if I should look at non-Classic Lightroom, which I think would let me edit my Fuji pix on the slab.

[Having said that, for some reason I can’t get Lightroom to auto-import the pictures.]

A keyboard, you said?

Indeed. I plugged it in and set it up and yeah, it works, but I was having severe cognitive dissonance. What’s it actually for? You’re not gonna set this contraption up to facilitate replying to a chat message. Don’t know about you, but I use a keyboard when I’m in creative mode, which often means writing. So, here’s a shot of the S7+ with keyboard beside my 16" MBP for context, set up for writing.

Galaxy Tab S7+ beside a 16" MacBook Pro

What, you wonder, might be on the screen? Well, Emacs, obviously, because that’s what I write this in. In fact, here’s a screen photo with the first few paragraphs of an early draft what you’re now reading.

My custom Emacs mode for blogging works fine, although the syntax coloring went off the rails somehow.

Emacs running on the Galaxy Tab S7+

Yaks were shaved. There seems to be no native Android Emacs? I’m surprised and disappointed. But you can install Termux, which gets you a perfectly acceptable shell environment and a pkg command that can install open-source packages from, uh, somewhere. Then it’s more work than you might think to get files and programs onto the device; I ended up using Curl mostly.

So yeah, I could in principle blog on this thing. Mind you, I’d need to get Perl and MySQL and so on running, but if Emacs can do it that ought to be possible.

OK, I kid. Normal people who write their blogs in a nice JS-browser environment would probably find themselves perfectly comfortable living their social-media lives on an S7+.

But that keyboard… it’s a pretty strange beast. It has 12 function keys and, down in the bottom left corner, Ctrl, Fn, Cmd, and Alt keys; to the spacebar’s right are keys labeled “Lang” and “Alt Gr”. Because I’m in Canada (I assume) some of the labels are bilingual and there’s a special key reserved for É. What’s just wrong is that some of the keys don’t produce what the labels say they do. While I was fooling with the shell I obviously needed “<” and “>”. There are keys with those symbols on them but they don’t emit those characters. Fortunately, my muscle memory took me to shift-, and shift-. which worked.

So something went off the rails here. Having said that, it’s a perfectly nice responsive keyboard and, with a bit of practice, I could live with it.

Miscellania

There’s a stylus called the “S Pen” that is said to be magically responsive, understand drawing pressure changes, and usable as a remote control so you can wiggle it around and drive the S7+ from across the table. Don’t ask me about drawing, one reason I like computers is that I have shaky hands, plus no shred of talent at drawing or penmanship. So it’s unlikely I’d ever use this thing.

It comes with a SIM card slot. Um, I guess that’s nice? Not sure what the scenario is that makes that interesting.

The battery life is OK. I binge-read most of a book in a single multi-hour sitting and that burned half the power. So it’s unlikely you’d ever run flat in a day; that’s all anyone really needs I think?

Samsung

I bought it direct from their website, which was cheaper than Amazon, and it showed up plenty fast. But they strongly de-emphasized the S7+ in favor of the smaller S7; near as I can tell there isn’t actually a dedicated S7+ page, and I had to do considerable backing and filling to actually order the big slab. Perhaps this is a consequence of me being in Canada?

Nelson says he thinks the S8 is imminent, which might also explain the weirdness. If he’s right, and the S8 turns out to be lighter and more graceful, I’ll be grouchy.

History

I have a bit, with Android tablets. Back in the fall of 2010 I took the first “Samsung Galaxy Tab” on a world tour. It was controversial because Google hadn’t managed to ship the first “official” Android Tablet, so Andy Rubin was pissed. Later, I repeatedly sang the praises of the Nexus 7, which I carried for years. Both of these were at the 7" form factor, so this is the only “big” tablet I’ve ever owned.

Does the world have a place for Android tablets? I dunno. But I’m holding on to this one for now.

28 Jun 06:34

A New Way to Share New York Times Stories

by The NYT Open Team

Times subscribers can now give up to 10 articles per month to family and friends who are non-subscribers.

An illustration of a hand holding a paper airplane on which five people are sitting.
Illustration by Joules Garcia

By Anna Mancusi, Erik Kernan, Nina Feinberg and Scott Sheu

We hear frequently from New York Times subscribers that they would like the ability to share Times journalism with non-subscribers. We know how enriching it can be to share articles with friends and family, but we also know that not everyone has a Times subscription.

We are excited to introduce gift articles for Times subscribers, which allows news subscribers to share up to 10 articles each month. Anyone who is sent a gift article will be able to read it, regardless of whether they subscribe to The Times; gift articles will not count towards the monthly free-article limit for non-subscribers.

Times subscribers are the biggest advocates for our journalism. By enabling them to more freely share our content with family and friends, we hope to make it easier for them to have meaningful conversations around the stories we publish.

An image of a Times article page with the gift articles button opened.
The new gift articles button sits next to the share tools and allows Times subscribers to share articles with family and friend who are not subscribed to The Times.

Through our research, we found that readers were more likely to engage with journalism that had been shared by someone they know rather than a company account. With gift articles, we hope to increase sharing, as well as bring in new audiences.

Research, Measure, Design

Building this feature was a collaborative effort that involved multiple teams and included rounds of user research, A/B testing, design iterations, API creation and countless discussions.

In early research studies, we learned that the vast majority of Times subscribers said that allowing friends and family to read the links they share would add value to their subscription overall. These insights gave us the confidence that building gift articles would meet a user need, grow our audience and add distinct value to a Times subscription.

As we began thinking about the new feature, we knew that making sure subscribers could find, understand and use it easily was extremely important. We spent some time developing rough user journeys that would encourage subscribers to share and reassure recipients that they would be able to freely read the full articles.

A slide showing user journeys for the gift article feature. The Subscriber Journey travels from Discover to Interact to Share, while the Recipient Journey goes from Discover to Access to Read.
To help shape what gift articles might look like, the team created rough user journeys.

These user journeys allowed our team to get on the same page about where and when this option would appear. But, they raised a number of questions about how we should describe the feature and how it would work.

Fortunately, we had the chance to conduct user research, which helped us think through how to frame the new feature. With the lessons from our research informing our designs, we built a back-end service to generate unique link codes and ran an initial A/B test in 2020.

In the test, we hypothesized that the ability to share unlocked articles would encourage more sharing overall. So, we included a prominent plain-text button directly below an article’s byline that read “Share unlocked article.” Normally, we require non-subscribers to register for free Times accounts before reading content on our app or website. However, for this test we wanted the articles to be freely available to non-subscribers; we still asked, but did not require, recipients to register for Times accounts.

These considerations led us to measure the percentage of subscriber sessions that resulted in a share and the percentage of shares that resulted in registration.

Our test showed a significant increase in subscriber sharing and a corresponding increase in referral traffic. Importantly, these increases did not come at the expense of normal sharing; readers still used our other share tools to post links to social platforms or send via email. While we saw a significant decline in the registration rate compared to our share tools, the rise in referral traffic and subsequent increase in registrations largely offset the impact. And because the benefit to readers was the primary goal of this feature, we deemed this an acceptable decrease.

Given the encouraging results from this first test, as well as the qualitative research that validated our approach, we decided to roll the feature out to news subscribers across our platforms.

But before we did, we had to replace the plain-text button. While it was acceptable for a test, it took up too much of the page and felt disjointed from our brand and style.

An image showing the evolution of the gift article icon from plain text, to an icon of a lock, then a hand and, finally, a gift.
The team tried a number of different images before settling on a gift icon.

Finding an appropriate icon was challenging. At first, we designed a lock symbol to match the concept of “unlocked articles” from our MVP test. Yet, when we brought this to a subsequent round of user testing, people said it suggested that the articles were inaccessible — and it made them hesitant to use it. Since this was the opposite of what we hoped to convey, we went back to the drawing board. We discussed using a hand symbol, but rejected this idea when it seemed as though The Times was requesting, rather than giving something.

This brought us to a gift symbol.

While we had some initial concerns about overlap with the gift subscriptions The Times sells, we decided the context was distinct enough that subscribers would not get mixed up. Ultimately, we felt the gift icon would do the most to draw user attention in a positive way, while fitting in with the other elements on the screen.

Feature release and beyond

Aside from unlimited article access, gift articles for Times subscribers is our first major subscriber-only feature. We hope that our subscribers will enjoy sharing stories with their family and friends, and that new readers will be able to discover what makes Times journalism unique.

As a product and design team, we will consider this feature a success if it meets the metrics we’re striving for. The feature is already available to the majority of subscribers and will soon be available on our apps. Over the next few months, we’ll be paying attention to how this feature impacts subscriber engagement and retention, and what new audiences it attracts.

But perhaps most importantly, the feature will be successful if it sparks conversations among curious people.

Anna Mancusi is a product director at The New York Times and leads a product group focused on subscriber retention initiatives.
Erik Kernan is a product manager focused on subscriber retention at The New York Times.
Nina Feinberg is a senior product designer at The New York Times.
Scott Sheu is an associate product manager and social product lead at The New York Times.

Special thanks to the teams that worked on gift articles for Times subscribers:
GMAX: Dominic Ancrum, Bon Champion, Amit Choudhary, Rashad Cureton, Courtney Downs, Robert Edwards, Anne Lindsley, Reza Malek, Patricia Martorana, Alex Paul, Sameer Sharma and Felix Torres

Audience Product: Justin Heideman, Andrei Kallaur, Jason Michaels, Victoria Niemeyer, Shelly Seroussi and Scott Sheu

Newsroom Audience: Anna Dubenko

Subscriber Advocacy: Olivia Appicelli, Mina Bansal, Nina Feinberg, Justin Fuller, Josh Hoeltzel, Tracy Huynh, Erik Kernan, Anisha Khanna, Reza Malek, Anna Mancusi and Steve Rickert

Meter: Mark Liechti, Ayo Odunsi, Binit Patel, Joselito Surdilla, Beatriz Trujillo, Ronny Wang and Dan Wittenberg

Care: Daniel Alesi, Honor Hawkins and Karen Pace

App Platforms: Sameer More

Data Platforms: Ed Gulczynski and Hannah Masuga

Corporate Communications: Linda Zebian


A New Way to Share New York Times Stories was originally published in NYT Open on Medium, where people are continuing the conversation by highlighting and responding to this story.

28 Jun 06:34

Office Talk

by Ton Zijlstra

This week our team is staying in a vacation park in the south of the Netherlands. All have their own cabin, except me. Family logistics mean I am spending most time at home, and commute to the holiday park.

This afternoon we discussed our office. What to do with it, how to make it more useful to us.

We opened an office exactly 2 years ago, and more than half of that time we didn’t use it much because of the pandemic. We opened the office because some of us need a place away from home to work. I am used to working either at home, en route, or at a client’s, and have been doing so for 17 years. Having an office, especially a centrally located one as we have in Utrecht, within a building with other facilities available to us (meeting rooms, restaurant/catering services, event spaces, roof terrace), to me is however very useful as a meeting place, and to be able to host groups. During the pandemic some of our team used it to escape the four walls of their limited living spaces in the inner city of Utrecht or Amsterdam. I handed my office keys to a new hire early on in the pandemic.

The central question today was, moving forward, given our pandemic experiences, and the likelihood of at least some measures being in place on and off, how do we want to use our office? And given that use, how do we want it to look / feel?

We split in three groups of three. That in itself was already an important first realisation for me: we can actually split in three groups of three. And the office should work well for all 9 of us, as well as for a handful of frequent collaborators.
In our little groups we discussed our ideal office, and shaped it with the material at hand. One group got to paint the office, another group to build with Lego (serious play is the applicable term I think), and the group I was in used clay.

20210623_160728

20210623_155153

20210623_150342

Patterns in the results were that, while it is still needed to have a few desks, most of them can be removed, that we want to make the office much greener with plants and more colourful in general, that shaping it as a social place is important, as well as a place where things can be created. A few immediate actions (such as removing two thirds of the desks, doing some painting, and adding plants) were decided upon for the summer. Another conclusion was that we simply cannot already know how office use post-pandemic will really be, meaning having plenty flexibility is key. Think furniture, devices, or dividers that can be very easily rearranged at will by those present. Think not investing in a ‘perfect’ design, but doing it as we go along.

28 Jun 06:32

Mozilla Racial Justice Commitments: One Year In

by Mitchell Baker

One year ago, we made a set of commitments to make diversity and inclusion more than a catchphrase or hot button topic. We decided to roll up our sleeves and get busy establishing significant goals, putting resources behind them and making sure that everyone, including our company leadership, was taking action to create a more diverse and equitable place at Mozilla and in society.

We have taken steps to address the issue of anti-Black racism and the lack of diversity and inclusion in our company, and hopefully, in society, through programming and people initiatives.  We have seen a significant increase in participation in diversity and inclusion initiatives, and perhaps, equally important, in our engagement survey results and in particular the increased scores on diversity and inclusion questions by people of color and women. While we have made strides on many of the goals established on June 18, 2020, we recognize this progress is the “First Step Toward Lasting Change.”  We continue to be committed through our actions and resources to improve Mozilla as a place to work for people of color and the internet for all.

1. Who we are: Our employee base and our communities

In our upcoming diversity and inclusion disclosure, you will find that we have greatly invested in improving diversity and enhancing a culture of inclusion at Mozilla. Through a balance of fun, education, celebration and conversations, we created safe spaces for people of color to share the totality of their human experience, honoring the beauty and joy of their lives and holding space to contend with the more sobering and harsh realities of race in society. 

We hosted three  panel discussions that each covered pertinent and insightful topics as designed by our Mozilla Resource Groups. There were the ones that gave us belly laughs – “What does it mean to lose your black card?” – and there were the ones that challenged us – “What is the impact of the model minority myth?” 

We held facilitated discussions designed to provide employees with an opportunity to engage in deep listening and sharing following an onslaught of racial violence across the U.S. These sessions, aptly named Gather @Mozilla, gave us an opportunity to collectively process some of the traumatic and triggering events happening around us. 

Our goal was to provide various options for employees to connect and learn. Recognizing that learning is a personal experience, we offered paths for individual learning and collective learning. We published resource libraries (and shared them publicly: Black History Month, Asian Pacific Islander Heritage Month), hosted virtual cooking lessons, convened talks with renowned authors, curated music playlists (Latin and Hispanic Heritage Month, Black History Month, Women’s History Month, API Heritage Month Playlist) and much more. By providing a breadth of opportunities to celebrate (Latin and Hispanic Heritage Month celebration, Black History Month celebration, Women’s History Month celebration, and Asian Pacific Islander Heritage Month celebration), we increased participation and invited our organization on a journey of co-creating an inclusive culture.  

As we round out the second half of 2021, we will be rolling out an Inclusion Champion program, working with DEI councils within each business group to promote organization-specific D&I programming, deploying a D&I skill development platform, and diversifying our talent acquisition pipeline.

2. What we build: Our outreach with our products

At Mozilla, we work to build a better internet and our products can help elevate the best of the internet. Through Pocket Collections, selections of stories curated by Pocket editors and guest editors, we introduced Collections that elevated diverse voices and gave insight into issues impacting BIPOC communities and the context around which they emerged (Racial Justice Collections, Essential Reading: Celebrating Juneteenth). We hoped to provide readers with content and perspectives they may not otherwise encounter.

We understood that when you point a finger at someone else, there is a finger pointing back at you. Thus, we launched a project that examined when and where biases creep into user research and design and initiated work efforts to reduce the amount of racist language in code (Remove all references to blacklist/whitelist within Gecko, Remove references to slave).

In the second half of 2021, our product teams will continue to identify opportunities to elevate diverse voices and combat unconscious biases in our products.

3. What we do beyond products: Our broader engagement with the world

We leveraged Dialogues & Debates, a speaker series, to address issues of A.I. and race and ethnicity and the challenges presented to communities of color because of this problem in tech and media. We had robust discussions about the use of technology to surveil historically vulnerable populations and used our network to call on fellow technology companies to be mindful of how they use technology in service to the criminal justice system instead of the communities of color we serve. The Mozilla Foundation launched campaigns calling on Nextdoor and Amazon Ring to pause their relationships with police departments and assess the impact of the platforms on users and communities of color. More than 28,000 people signed the petitions and several organizations partnered with Mozilla on escalation actions. 

We were able to have these critical community discussions and collaborations by being more thoughtful and intentional with the ways in which we used available funding from Mozilla Foundation in our commitment to social justice in tech and society. We granted 33% of Mozilla Foundation funds to black-led tech and social justice initiatives. Unfortunately, we fell short of our 40% target. While some may see this as a failure, we see it as an opportunity to acknowledge an area where we still need improvement and to commit to continuing to fund and elevate voices of color.

We also partnered with three Historically Black Colleges and Universities Engineering and Computer Science programs to promote the role of African Americans in tech, to engage in ethical computing discussions, and to cultivate relationships with aspiring scientists, designers and tech leaders so they understand there is a place for them in this industry.

Over the coming year, we are looking forward to deepening our relationships with institutions that serve and support communities of color and communities that have been historically marginalized. The First Step Toward Lasting Change squarely moves us along the journey of diversity, equity, inclusion and belonging but is not the middle nor final step. There remains significant room for improvement and we are committed to continue the course in closing the gaps that exist in tech and society.  

The post Mozilla Racial Justice Commitments: One Year In appeared first on The Mozilla Blog.

28 Jun 06:30

The Globe and Mail has built a paywall that knows when to give up

Sarah Scire, Nieman Lab, Jun 24, 2021
Icon

This is more about the Globe and Mail's automated paywall system, Sophi, which I covered here last week. "Sophi uses analytics to make decisions that balance the potential for ad revenue vs. the potential for subscriber revenue. 'It knows when to give up,' Verma said. 'It knows when not to bug someone because they’re not going to get money from them anyway.'" From my perspective, it's insidious, because I don't link to paywalls in OLDaily - but the AI might fool me into thinking there's no paywall when really there's just no paywall for me, but there might be one for you.

Web: [Direct Link] [This Post]
28 Jun 06:30

What becomes of journalism when everybody can write or cast?

by Doc Searls

Formalized journalism is outnumbered.

In the industrialized world (and in much of the world that isn’t), nearly everyone of a double-digit age has a Net-connected mobile device for sharing words they write and scenes they shoot.

While this doesn’t obsolesce professional journalists, it marginalizes and re-contextualizes them. Worse, it exposes the blindness within their formalities. Dave Winer lays this out clearly in They can’t see what they can’t see. An excerpt:

Journalism, academia, government and the corporate world all hire from the same talent pool.

They go to the same universities, get their news from the same sources. Corporate people take government jobs, then go back to the corporations. The people move fluidly in and out of each bucket.

So you get the same story, the reality they believe in, developed over centuries, that is radically different from the reality most other people experience. The story recited daily at CNN, MSNBC, The New Yorker, NYT. The world changes, again and again, and the story they tell is how angry this makes them, and how everyone must snap back.

New technologies can make change possible, like the one we’re using now.

We would never have Trump if it weren’t for Twitter.

Marshall McLuhan, dead since 1980, had something to say about that:

People don’t want to know the cause of anything. They do not want to know why radio caused Hitler and Gandhi alike. They do not want to know that print caused anything whatever.

Also,

All media work us over completely. They are so pervasive… that they leave no part of us untouched unaffected, unaltered… Any understanding of social and cultural change is impossible without a knowledge of the way media work as environments.

What could be more worked-over than our new nature as digital beings? Consider where and what you are reading right now, on your desk, your lap or in your hand. This was barely imaginable when the word “journalism” came into use around 1830.

What are we now? While remaining no less embodied than we ever were, we are incorporeal inhabitants of a digital world, absent of distance and gravity, where the only preposition that fully applies is with. Unless we aren’t. Ephemerality runs deep, and the Web is a whiteboard.

I don’t have an answer to the question in my headline. I tried one out three years ago, in my first, last and only TEDx talk, The story isn’t the whole story. To summarize, we gotta start local.

Because it is essential to be real with each other in the physical world, especially when fully relevant news actually happens. Conspiracy theories and other forms of bullshit will always be everywhere, but it’s harder for them to apply or survive when your building is on fire or your neighborhood is flooding.

Outside of that, I’m starting to think we need a new discipline: one that doesn’t start with the word journal. And decades may pass before we have it.

28 Jun 06:29

http://scripting.com/2021/06/23/162312.html?title=theyCantSeeWhatTheyCantSee

Dave Winer, Scripting News, Jun 25, 2021
Icon

Here's Dave Winer: "Journalism, academia, government and the corporate world all hire from the same talent pool. They go to the same universities, get their news from the same sources. Corporate people take government jobs, then go back to the corporations. The people move fluidly in and out of each bucket." I think that's true, and it certainly accords with my own observation. The second part, though, is where the hammer drops: "So you get the same story, the reality they believe in, developed over centuries, that is radically different from the reality most other people experience.... They think the problem is that we don't see what they see, but I think it's the other way. They hold on to a normalcy that is gone. They can't see what they can't see." Over the years I've called this the 'nexus'. Winer's point is that people inside the nexus don't realize their inside the nexus, or that there's a different reality outside the nexus. Via Doc Searls, who (being inside the nexus) misses Winer's point completely.

Web: [Direct Link] [This Post]
28 Jun 06:29

Global Warming

earthreport:Warming Arctic EU satellites recorded ground temperatures above 118 degrees Fahrenheit...
28 Jun 06:28

The Cloud Genie

by Rui Carmo

The first paragraph is sheer genius, so much so that I’m stealing a copy:

There’s a joke that I’ve always been partial to: a software engineering type rubs a lamp and a genie appears. The genie says that he’ll grant the engineer $1 billion, but only if they can spend $100 million in a single month with three rules. “You can’t gift it away. You can’t gamble with it. And you can’t throw it away.” The software engineer responds with “Well, can I use AWS?” The genie responds with “okay, there are four rules.”


28 Jun 06:28

Take control over your data with Rally, a novel privacy-first data sharing platform

by Mozilla

Mozilla teams up with Princeton University researchers to enable crowdsourced science for public good; collaborates with research groups at Princeton, Stanford on upcoming studies.

Your data is valuable. But for too long, online services have pilfered, swapped, and exploited your data without your awareness. Privacy violations and filter bubbles are all consequences of a surveillance data economy. But what if, instead of companies taking your data without giving you a say, you could select who gets access to your data and put it to work for public good?

Today, we’re announcing the Mozilla Rally platform. Built for the browser with privacy and transparency at its core, Rally puts users in control of their data and empowers them to contribute their browsing data to crowdfund projects for a better Internet and a better society. At Mozilla, we’re working on building a better internet, one that puts people first, respects their privacy and gives them power over their online experience. We’ve been a leader in privacy features that help you control your data by blocking trackers. But, being “data-empowered” also requires the ability to choose who you want to access your data. 

“Cutting people out of decisions about their data is an inequity that harms individuals, society and the internet. We believe that you should determine who benefits from your data. We are data optimists and want to change the way the data economy works for both people and day-to-day business. We are excited to see how Rally can help understand some of the biggest problems of the internet and make it better.”

Rebecca Weiss, Rally Project Lead

As a first step on this journey, we’re launching the new Rally research initiative, a crowdsourced scientific effort we developed in collaboration with professor Jonathan Mayer’s research group at Princeton University. Computer scientists, social scientists and other researchers will be able to launch groundbreaking studies about the web and invite you to participate. A core focus of the initiative is enabling unprecedented studies that hold major online services accountable.

“Online services constantly experiment on users, to maximize engagement and profit. But for too long, academic researchers have been stymied when trying to experiment on online services. Rally flips the script and enables a new ecosystem of technology policy research.”

Jonathan Mayer, Princeton’s Center for Information Technology Policy

We’re kickstarting the Mozilla Rally research initiative with our first two research collaborator studies. Our first study is “Political and COVID-19 News” and comes from the Princeton team that helped us develop the Rally research initiative. This study examines how people engage with news and misinformation about politics and COVID-19 across online services.  

Soon, we’ll also be launching our second academic study, “Beyond the Paywall”, a study, in partnership with Shoshana Vasserman and Greg Martin of the Stanford University Graduate School of Business. It aims to better understand news consumption, what people value in news and the economics that could build a more sustainable ecosystem for newspapers in the online marketplace.

“We need research to get answers to the hard questions that we face as a society in the information age. But for that research to be credible and reliable, it needs to be transparent, considered and treat every participant with respect. It sounds simple but this takes a lot of work. It needs a standard bearer to make it the expectation in social science. In working with Rally, we hope to be part of that transformation.”

Shoshana Vasserman, Assistant Professor of Economics at the Stanford Graduate School of Business

We are also launching a new toolkit today, WebScience, that enables researchers to build standardized browser-based studies on Rally. WebScience also encourages data minimization, which is central to how Rally will respect people who choose to participate in studies. WebScience was developed and open sourced by Jonathan Mayer’s team at Princeton and is now co-maintained with Mozilla. 

With Rally, we’ve built an innovative, consent-driven data sharing platform that puts power back into the hands of people. By leveraging the scale of web browsers – a piece of software used by billions of people around the world – Rally has the potential to help address societal problems we could not solve before. Our goal is to demonstrate that there is a case for an equitable market for data, one where every party is treated fairly, and we welcome mission-aligned organizations that want to join us on this journey. 

Rally is currently available for Firefox desktop users over age 19 in the United States. We plan to launch Rally for other web browsers and in other countries in the future. 

To participate in Rally, join us at rally.mozilla.org

————————————————————————————

Interested in joining Rally and want to know how it works?

When you join Rally, you have the opportunity to participate in data crowdsourcing projects — we call them “studies” — focused on understanding and finding solutions for social problems caused by the data economy. You will always see a simple explanation of a study’s purpose, the data it collects, how the data will be used, and who will have access to your data. All your data is stored in Mozilla’s restricted servers, and access to the analysis environment is tightly controlled. For those who really want to dig deep, you can read our detailed disclosures and even inspect our code

The post Take control over your data with Rally, a novel privacy-first data sharing platform appeared first on The Mozilla Blog.

28 Jun 06:28

When a country cancelled half its citizens

by Chris Grey
Five years have now passed since the Referendum, so for once I’m not going to provide the usual analysis of the week’s developments (perhaps the most important being that it looks as if the EU will agree to extend the grace period for chilled meats going from Great Britain to Northern Ireland). Nor am I going to give a summary what has happened during those years. There have been plenty of those already, including my own in Prospect magazine. Anyway, to do it properly would take at least a book - and for those who are interested I have written just such a book. It is called Brexit Unfolded: How no one got what they wanted (and why they were never going to) and was published by Biteback this week.

Instead, I’m going to focus on an important but remarkably little discussed aspect of the impact Brexit has had since 2016, and one which in the long-run I think may be amongst the most significant. As a point of departure, consider these lines in a speech that David Frost gave last Friday to the Konigswinter conference. There was much in the speech which was deeply questionable, or worse, but the particular lines are these:

“…there is no longer any serious debate on the subject [of Brexit] in Britain. No major political party advocates EU membership, and, while a proportion of the public may still regret Brexit, there is no energy behind a rejoin movement. Overwhelmingly we are now looking forward.”

Of course these are just the passing words of one (unelected) politician. But they come from the man who negotiated both the eventual Withdrawal Agreement and the Trade and Cooperation Agreement, and who is now in charge of the entirety of the UK’s post-Brexit policy. And I imagine that he expresses something that many pro-Brexit politicians (for Frost is, although wasn’t always, that) believe, or would like to believe. Yet what he said is manifestly untrue, and reveals a quite extraordinary complacency about what Brexit has done to this country.

Frost is right that there is no vibrant movement for the UK to rejoin the EU, though he ignores the way that Brexit has put new energy into the campaign for Scottish independence, with Scotland re-joining the EU a key part of that. He may also be right to imply that even those most opposed to Brexit have recognized the undeniable fact that Britain has left the EU. In that sense, the Brexiters have won. But what has winning entailed?

A nation divided – by design

Brexiters often present leaving the EU as a ‘national liberation’, and some even, quite insanely, compare it to independence from colonialism (insane not least because Britain freely joined and, as Brexit showed, was free to leave). But that has always had two obvious problems. One is that about half the country at the time of the vote, and more for the most of the period since, did not want it and were forced into it. So what sort of ‘liberation’ is that?

The other problem is that, quite as much as the EU, the target of the Brexiters’ ire always was, and still is, those of their compatriots they consider to be ‘the liberal metropolitan elite’ or ‘the establishment’ (categories made so elastic as to be meaningless), in contrast to the “ordinary, decent people” who voted to leave. Even in celebrating the fifth anniversary of “Independence Day”, Nigel Farage framed it in terms of ‘the people rejecting the establishment’, whilst Andrew Bridgen, one of the ERG ‘Spartans’, wrote of defending “the people’s Brexit against establishment sabotage”. Thus ‘owning the libs’ has been as much of a motivation as ‘liberation’ from the EU. The consequence is that it’s not just that the nation is divided over Brexit, but that Brexit, as a project, is deliberately divisive of the nation in treating only its supporters as the ‘people’.

A nation divided – in half

But what does it do to a country when roughly half of its population is turned upon in this way? And what might the particular implications be of doing so when that half is, in general terms, the more educated (57% of graduates voted remain, rising to 64% of those with a higher degree), more youthful (73% of 18-24s voted remain) and more economically-active (52% of those in work voted remain) part of the population? [Source of all figures: Lord Ashcroft Polls, 24 June 2016.]

From the very start, these people’s concerns were ignored or dismissed. They were told to ‘suck it up’, insulted as cry-babies, stereotyped as only interested in their Tuscan holiday homes and cheap Bulgarian nannies, demonised as ‘enemies of the people’ and ‘saboteurs’, and traduced as traitors. Yet, also from the start, there was a huge irony. Precisely because of the educational and social demographic of the vote (e.g. 57% of social classes AB voted remain), it was statistically likely that those who actually had to take responsibility for enacting Brexit were in many cases part of this demonised group. In any case, few Brexiters actually had the technical knowledge to do it: in general, people who understood what Brexit actually involved didn’t support it.

So although some Brexiters like to think of themselves as having initiated a revolution, it was an unusual one in requiring those who did not want it to do the hard work of enacting it. Some amongst the civil service no doubt accepted that as part of their professional duty – whilst all the time being belaboured for supposedly not doing so – or even actively embraced it despite their previous views, as seems to have happened with Frost himself. Others, including those in business and civil society organizations, have had no choice but to make what adjustments were necessary to deal with it.

With, no doubt, some exceptions, none of this amounts to acceptance that Brexit is a good idea, and why should it? Would Brexiters, had they lost the vote, have accepted that EU membership was a good idea? Certainly not, and, because remaining in the EU wouldn’t have needed any ‘enactment’, they would not have been called upon to do anything to make it happen. What would almost certainly have been the case, and we can be sure of this because it was what was happening before, is that no UK government would have pushed for more extensive integration with the EU, for example by joining the Euro or Schengen. Instead, and even more had there been a close vote to remain, it would have been recognized that this would be to disrespect and ignore the strong vein of anti-EU sentiment within the population. (This, by the way, is the reason why I think that, in such scenario, the UK would still have exercised its right to have an independent vaccine policy.)

Brexiters’ lack of magnanimity

The Brexiters in victory showed no such magnanimity. Instead, they pushed for the hardest of Brexits, not just making no concession to remainers but making no attempt to persuade, reassure, or even involve them. On the contrary, they sought to install Brexiters into key positions and hounded those they suspected of being remainers out of office. This is the real ‘cancel culture’ of recent years. They bemoaned the lack of buy-in, but made no attempt to secure it. Had they done so, we would very likely now see much less division, and perhaps a greater degree of support for Brexit. Instead, ‘the 17.4 million’ was used as a battering ram in order to treat 16.2 million like dirt. And now that Brexit has happened, the same treatment is still being meted out through the endless culture war against those stigmatised as ‘woke’ and unpatriotic in what Maheen Behrana aptly calls “the weaponisation of the metropolitan bogeyman”.

Of course, some of this is not new. For as long as I can remember there has been a noisy strand in politics and journalism lambasting ‘political correctness’, the ‘human rights brigade’ and the ‘bleeding heart liberals’. What changed with Brexit, though, was to enfold into one despised category the large and broad constituency of remainers. It is not even true that a distinction was drawn between remain voters who ‘accepted’ the result and ‘remoaners’ who didn’t. For as Theresa May, to take the most high-profile example, found, even embracing hard Brexit did not stop her being tagged ‘Theresa the remainer’.

The degradation of everyday life

What also changed with Brexit was not just to insult ‘the liberal metropolitan elite’ with words but with a policy which completely transformed almost every aspect of national economic and geo-political strategy.  That is not just an abstraction. Although somewhat masked by, and partly intertwined with, the effects of the pandemic, we increasingly see our everyday lives degraded and made more miserable and restricted by Brexit. Purchases from EU companies delayed, suspended or surcharged. Shortages of staff in so many areas of the economy as lockdown restrictions lift. Shortages of fresh produce in shops. The massive collapse of food and drink exports to the EU. The minor hassle of making customs declarations on packages posted to the EU, and what will be the extra hassles of holidaying there. The non-trivial loss of the Erasmus scheme for the young, the reduction of work opportunities and of easy continental retirement for older people. The shaming treatment, and often the loss, of our EU friends, colleagues and neighbours. The debilitating blight on families created by, and with plans predicated upon, free movement rights.

To complain of such things itself invites the jeering ridicule of some Brexiters. For so widely have they drawn the definition of the ‘liberal metropolitan elite’ that they treat things which in a relatively wealthy country are fairly routine expectations for a great many people as being marks of hugely entitled privilege. Thus they talk as if no ‘ordinary, decent’ person studies, works, holidays or retires abroad, or even orders goods from overseas. Perhaps even more extraordinarily, given the fact that most politicians of most political persuasions recognize the individual and national benefits of education, is the way the Brexiters sneer at education, and especially university education, as if it were something shameful.

All this is the more ludicrous since leading Brexiters from Jacob Rees-Mogg to Julia Hartley-Brewer are hardly short of money, lacking in privilege or, indeed, lacking university education or likely to deny it to their children. Nor, for that matter, is it true that leave voters in general were the under-privileged left behind – as the UK in a Changing Europe’s research shows, many were ‘comfortable leavers’.

Humiliating remainers has consequences

My point here is not so much to rehearse these now-familiar issues as to say that it is inevitable that they have had, and will continue to have, profound consequences. Both history and personal experience tell us that if you humiliate people then you will reap a harvest of resentment, anger and many other complex and destructive emotions. The fact that these, certainly for now, do not coalesce into a political movement to re-join the EU is irrelevant. Most erstwhile remainers recognize that this isn’t in prospect for the time being.

The important question is how this will now play out. The answer is bound to be complex, if only precisely because such a large and heterogenous group of people as remainers have been subsumed within the general categories of the ‘liberal metropolitan elite’ and the ‘enemies of the people’. It also bears saying that plenty of people who voted remain, like plenty who voted leave, didn’t and don’t feel very strongly about Brexit either way. But there are plenty who do.

It’s possible that last week’s Chesham and Amersham by-election result is an indication of the consequences, as Guardian journalist John Harris suggests. Remainers may have been a minority of Tory voters, but they were a very large minority: of those who voted Tory in the 2015 election, 42% voted remain in 2016. As political data analyst Christabel Cooper argues that whereas there has been much discussion of ‘Labour leavers’ these ‘Tory remainers’ have been rather ignored.

And it is not just that many Tory voters were remainers, it is that many of them do not approve of or subscribe to Johnson’s crude cultural attacks or his generally dishonest politics. Politicians like Dominic Grieve are (or were) representatives of a kind of Tory voter that still exists, and is most certainly conservative, but they, too, have been cancelled by Brexit. For that matter, it very likely that this group includes some Tory voters who supported Brexit.

It’s obviously foolish to draw many conclusions from a single by-election, and it’s worth recalling that many people, including me, thought that the LibDem victory in the December 2016 Richmond by-election might be a sign of things to come, which it wasn’t. That was partly because, as Cooper remarks, in the subsequent general elections Tory remainers’ hostility to Jeremy Corbyn was greater than their commitment to remain. That factor has now disappeared. Moreover, now, it is not just the Chesham and Amersham result but those of the recent local and Mayoral elections which arguably point to underlying shifts in voting patterns as a fall out from Brexit.

It’s not just about voting

In any case, there is more at stake here than how people vote. I’m not aware of any systematic research on this but my own experience and intuition suggest that there are other ways in which the demonization of remainers is playing out. One is the withdrawal of some from political engagement and comment, resigned to having lost the biggest battle of their generation. Instead, local and personal activities are prioritised.

Another, related, response is a rather bitter rejection of what, at least for progressives, used to be the social contract. Thus when some now hear of job losses in leave-voting areas the reaction is not one of solidarity with those affected. Instead, it is that they got what they voted for and will have to take the consequences. That may be rather graceless, and it ignores the fact that many in such areas, and many of those adversely affected, voted remain: Brexit is not a laser-guided missile the effects of which only target its supporters. But, like it or not, it is one consequence of all the jibes at remainers to ‘suck it up’.

Additionally, not least because many remainers are educated and have marketable skills, some of them are more actively opting-out of Brexit by emigrating. Again I’m only going by my own limited experience, although there is some evidence of an emerging ‘brain drain’, but I hear of more and more people who have, or are considering, leaving. Many are EU citizens, but many are British people who have skills which are in demand abroad, whether in EU countries or elsewhere. This is both a response to Brexit but also, of course, one of the costs of Brexit, adding to the problems of labour and skill shortages caused partly by the end of freedom of movement of people.

Clearly some Brexiters would respond ‘good riddance’, just as they defiantly speak of boycotting companies which re-locate, boycotting Spain, boycotting German goods, and, generally, boycotting ‘European muck’. That is all of a piece with the more general narrowing of Britain, and the meanness of its political discourse. But, be that is it may, leave voters as much as anyone else need goods and services and, especially given Britain’s ageing demographics, the coming years look likely to see real skill shortages, including in health and social care for the elderly. Any emigration of disaffected remainers is going to make that worse.

To reiterate, that disaffection is not simply sour grapes for having lost the 2016 vote, it is because of the way they have been treated since and the morphing of Brexit into a wider ‘Brexitification’ of politics that daily insults and belittles them, sometimes threatens them with violence, including threats which reference the murder of Jo Cox by a far-right terrorist, and advocates their trial for treason.

Anger, not acceptance

I am sure that what I have said here does not exhaust the ways that remainers are responding to what has been done to them. It may also be that, in time, a vibrant rejoin movement emerges. But my point is that its absence is not, as Frost seems to think, a sign that Brexit has been accepted. An opinion poll this week shows that precisely 0% of remain voters think Brexit is going ‘very well’, and just 8% that it is going ‘fairly well’ (interestingly a mere 10% even of leave voters think it is going ‘very well’, although a more substantial 35% think it is going ‘fairly well’ – still, hardly a great endorsement).

Meanwhile, another poll shows very little shift in the numbers who would still vote remain (or for that matter leave) if these were still options, an important caveat being that this is a poll of those who voted in 2016 (polling of those who did not vote or were too young to vote then shows a clear preference for remain, whilst of course those who have since died cannot by definition be surveyed).

So this is not ‘acceptance’ in any sense other than the truism that people know that, as a matter of fact, Brexit has happened. And, beneath the surface, there is far more going on than, as Frost has it, some “regret” about Brexit. Indeed, although I appreciate that Twitter is not representative and can be an echo chamber, the numerous responses to my tweet about Frost’s comment suggest that there is very considerable anger. At the very least, the polling evidence shows that the country remains as divided as ever.

An unprovoked attack on half the country

And how could there not be anger and division? Not just because of Brexit but all that the Brexiters have done since. That wasn’t automatically entailed by Brexit. It wasn’t necessary in order to honour the 2016 vote. It was a choice, motivated by the hatred of some and the opportunism of others.

Remainers did very little to provoke it. Yes, some insulted leave voters (just as some leave voters insulted them), although so far as I know they did not issue any death threats against pro-Brexit MPs. Yes, many expressed their outrage and opposition to Brexit (just as Brexiters did to EU membership). And, yes, many campaigned for another referendum to be held once the withdrawal terms were known (but, had it happened, nothing would have stopped people again voting to leave if they wanted).

None of that remotely justifies how they were treated after 2016, still less since Britain left the EU. It is not that they were bad losers, but that Brexiters were bad winners. It is what they have done with their victory, whilst always wallowing in a sense of their self-righteous victimhood, which has so divided and scarred the country. In effect, they have sought to cancel the half of the country that doesn’t agree with them. It would be very foolish indeed to imagine that this is not having consequences – profound and far-reaching, albeit unpredictable – on politics and society which will continue for years.

David Frost said something else this week. When asked what a successful Brexit would look like in ten years’ time he replied “I would say it’s a situation where nobody is seriously questioning Brexit - where it was self-evidently the right thing to do and the country feels comfortable with it”. Given the last five years, it‘s highly unlikely that this test will be met in the next ten, if indeed the country survives that long. Partly because of the flawed nature of Brexit, but no less because Brexiters have done all they can to ensure no such comfort is possible.

28 Jun 06:26

The Social Sector Needs a Meta Movement

Laura Deaton, Stanford Social Innovation Review, Jun 25, 2021
Icon

The entire article is an interesting read, but the author really gets to the point only in the last few paragraphs. " The social sector originated as a way to encourage wealthy white people to give alms to the desperate and ignored. It was not built to transfer power—just the opposite," writes Laura Deaton. "We should shed that history and build an engine for making the social sector a primary power center in its own right. It is the only sector that can accept and combine government, corporate, and philanthropic funding, and use that money for public good." The problem comes when you try to form "an advocacy network that connects currently disparate movements and aligns agendas in pursuit of common goals." Now there's a lot to be said for networks in social movements. But when you start 'aligning agendas', the goals of grassroots causes are swept away by the broader priorities of corporations and foundations, and you're back to a structure that entrenches power, and does not transfer it.

Web: [Direct Link] [This Post]
28 Jun 06:21

Ghost bike installation for cyclist who was killed last week in Brampton

by jnyyz

A cyclist described as a man in his 60s was hit last Thursday, June 17 at the intersection of Queen St E and Brampton. He died in hospital early the next morning. ARC was not able to find any further information. A ghost bike was installed in his memory this morning.

There was a small memorial of flowers on the southeast corner. You can see that this is a very busy intersection that would be terrifying to ride through. It was unnerving enough to cross it as a pedestrian as there was so much traffic raring to go in both directions as soon as a light turned green, plus slip lanes at all four corners.

The ghost bike was installed on the south west corner, which seemed to be the least developed corner. A hotel is being built on the northeast corner.

In actual fact, this particular ghost bike is the one that was put up in memory of Pasquale Alonzi who passed away last August. The ghost bike was in the town of Caledon, and ARC was informed that it was due to be removed because of a bylaw that said that roadside memorials couldn’t stay up for more than one year. With the kind permission of the family, the ghost bike was removed and reused at this new site, which ironically was just down HWY 50 from the old site.

Just for future reference, it took about 10 seconds to cut through this lock with an angle grinder. It takes about a minute to cut through a kryptonite lock.

Deepest condolences to the family and friends of the deceased.

28 Jun 06:20

What I've learned about data recently

What I've learned about data recently

Laurie Voss talks about the structure of data teams, based on his experience at npm and more recently Netlify. He suggests that Airflow and dbt are the data world's equivalent of frameworks like Rails: opinionated tools that solve core problems and which mean that you can now hire people who understand how your data pipelines work on their first day on the job.

Via @seldo

28 Jun 06:19

Notes on streaming large API responses

I started a Twitter conversation last week about API endpoints that stream large amounts of data as an alternative to APIs that return 100 results at a time and require clients to paginate through all of the pages in order to retrieve all of the data:

Any unexpected downsides to offering streaming HTTP API endpoints that serve up eg 100,000 JSON objects in a go rather than asking users to paginate 100 at a time over 1,000 requests, assuming efficient implementation of that streaming endpoint?

— Simon Willison (@simonw) June 17, 2021

I got a ton of great replies. I tried to tie them together in a thread attached to the tweet, but I'm also going to synthesize them into some thoughts here.

Bulk exporting data

The more time I spend with APIs, especially with regard to my Datasette and Dogsheep projects, the more I realize that my favourite APIs are the ones that let you extract all of your data as quickly and easily as possible.

There are generally three ways an API might provide this:

  • Click an "export everything" button, then wait for a while for an email to show up with a link to a downloadable zip file. This isn't really an API, in particular since it's usually hard if not impossible to automate that initial "click", but it's still better than nothing. Google's Takeout is one notable implementation of this pattern.
  • Provide a JSON API which allows users to paginate through their data. This is a very common pattern, although it can run into difficulties: what happens if new data is added while you are paginating through the original data, for example? Some systems only allow access to the first N pages too, for performance reasons.
  • Providing a single HTTP endpoint you can hit that will return ALL of your data - potentially dozens or hundreds of MBs of it - in one go.

It's that last option that I'm interested in talking about today.

Efficiently streaming data

It used to be that most web engineers would quickly discount the idea of an API endpoint that streams out an unlimited number of rows. HTTP requests should be served as quickly as possible! Anything more than a couple of seconds spent processing a request is a red flag that something should be reconsidered.

Almost everything in the web stack is optimized for quickly serving small requests. But over the past decade the tide has turned somewhat: Node.js made async web servers commonplace, WebSockets taught us to handle long-running connections and in the Python world asyncio and ASGI provided a firm foundation for handling long-running requests using smaller amounts of RAM and CPU.

I've been experimenting in this area for a few years now.

Datasette has the ability to use ASGI trickery to stream all rows from a table (or filtered table) as CSV, potentially returning hundreds of MBs of data.

Django SQL Dashboard can export the full results of a SQL query as CSV or TSV, this time using Django's StreamingHttpResponse (which does tie up a full worker process, but that's OK if you restrict it to a controlled number of authenticated users).

VIAL implements streaming responses to offer an "export from the admin" feature. It also has an API-key-protected search API which can stream out all matching rows in JSON or GeoJSON.

Implementation notes

The key thing to watch out for when implementing this pattern is memory usage: if your server buffers 100MB+ of data any time it needs to serve an export request you're going to run into trouble.

Some export formats are friendlier for streaming than others. CSV and TSV are pretty easy to stream, as is newline-delimited JSON.

Regular JSON requires a bit more thought: you can output a [ character, then output each row in a stream with a comma suffix, then skip the comma for the last row and output a ]. Doing that requires peeking ahead (looping two at a time) to verify that you haven't yet reached the end.

Or... Martin De Wulf pointed out that you can output the first row, then output every other row with a preceeding comma - which avoids the whole "iterate two at a time" problem entirely.

The next challenge is efficiently looping through every database result without first pulling them all into memory.

PostgreSQL (and the psycopg2 Python module) offers server-side cursors, which means you can stream results through your code without loading them all at once. I use these in Django SQL Dashboard.

Server-side cursors make me nervous though, because they seem like they likely tie up resources in the database itself. So the other technique I would consider here is keyset pagination.

Keyset pagination works against any data that is ordered by a unique column - it works especially well against a primary key (or other indexed column). Each page of data is retrieved using a query something like this:

select * from items order by id limit 21

Note the limit 21 - if we are retrieving pages of 20 items we ask for 21, since then we can use the last returned item to tell if there is a next page or not.

Then for subsequent pages take the 20th id value and ask for things greater than that:

select * from items where id > 20 limit 21

Each of these queries is fast to respond (since it's against an ordered index) and uses a predictable, fixed amount of memory. Using keyset pagination we can loop through an abitrarily large table of data, streaming each page out one at a time, without exhausting any resources.

And since each query is small and fast, we don't need to worry about huge queries tying up database resources either.

What can go wrong?

I really like these patterns. They haven't bitten me yet, though I've not deployed them for anything truly huge scale. So I asked Twitter what kind of problems I should look for.

Based on the Twitter conversation, here are some of the challenges that this approach faces.

Challenge: restarting servers

If the stream takes a significantly long time to finish then rolling out updates becomes a problem. You don't want to interrupt a download but also don't want to wait forever for it to finish to spin down the server.

— Adam Lowry (@robotadam) June 17, 2021

This came up a few times, and is something I hadn't considered. If your deployment process involves restarting your servers (and it's hard to imagine one that doesn't) you need to take long-running connections into account when you do that. If there's a user half way through a 500MB stream you can either truncate their connection or wait for them to finish.

Challenge: how to return errors

If you're streaming a response, you start with an HTTP 200 code... but then what happens if an error occurs half-way through, potentially while paginating through the database?

You've already started sending the request, so you can't change the status code to a 500. Instead, you need to write some kind of error to the stream that's being produced.

If you're serving up a huge JSON document, you can at least make that JSON become invalid, which should indicate to your client that something went wrong.

Formats like CSV are harder. How do you let your user know that their CSV data is incomplete?

And what if someone's connection drops - are they definitely going to notice that they are missing something, or will they assume that the truncated file is all of the data?

Challenge: resumable downloads

If a user is paginating through your API, they get resumability for free: if something goes wrong they can start again at the last page that they fetched.

Resuming a single stream is a lot harder.

The HTTP range mechanism can be used to provide resumable downloads against large files, but it only works if you generate the entire file in advance.

There is a way to design APIs to support this, provided the data in the stream is in a predictable order (which it has to be if you're using keyset pagination, described above).

Have the endpoint that triggers the download take an optional ?since= parameter, like this:

GET /stream-everything?since=b24ou34
[
    {"id": "m442ecc", "name": "..."},
    {"id": "c663qo2", "name": "..."},
    {"id": "z434hh3", "name": "..."},
]

Here the b24ou34 is an identifier - it can be a deliberately opaque token, but it needs to be served up as part of the response.

If the user is disconnected for any reason, they can start back where they left off by passing in the last ID that they successfully retrieved:

GET /stream-everything?since=z434hh3

This still requires some level of intelligence from the client application, but it's a reasonably simple pattern both to implement on the server and as a client.

Easiest solution: generate and return from cloud storage

It seems the most robust way to implement this kind of API is the least technically exciting: spin off a background task that generates the large response and pushes it to cloud storage (S3 or GCS), then redirect the user to a signed URL to download the resulting file.

This is easy to scale, gives users complete files with content-length headers that they know they can download (and even resume-downloading, since range headers are supported by S3 and GCS). It also avoids any issues with server restarts caused by long connections.

This is how Mixpanel handle their export feature, and it's the solution Sean Coates came to when trying to find a workaround for the AWS Lambda/API Gate response size limit.

If your goal is to provide your users a robust, reliable bulk-export mechanism for their data, export to cloud storage is probably the way to go.

But streaming dynamic responses are a really neat trick, and I plan to keep exploring them!

28 Jun 00:47

Markdown, Historical Writing, and Killer Apps

Like many technically inclined historians (for instance, Caleb McDaniel, Jason Heppler, and Lincoln Mullen) I find that I’ve increasingly been using the plain-text format Markdown for almost all of my writing.

The core idea of Markdown is that rather than use Microsoft Word, Scrivener, or any of the other pretty-looking tools out there, you type in “plain text” using formatting conventions that should be familiar to anyone who’s ever written or read an e-mail. (Click on Mullen’s or Heppler’s name for a better introduction than this, or see the Chronicle’s wrapup of approaches).

The benefits are many, but they’re mostly subtle:

  • A simple format like Markdown creates documents you’ll have not trouble reading in twenty years. I’ve been teaching a survey course this semester and had a hell of a time reading my old notes from generals which I took using EndNote; with Markdown, any web browser, text editor, or Microsoft Word descendant will have no trouble opening it.
  • It’s very easy to produce content that will look good in multiple media: I can make a course syllabus or personal CV with that formats nicely on a website and produces a clean looking PDF at the same time.
  • It becomes much easier to do things to a bunch of notes at the same time: bundle them into PDFs, search through all of your notes simulataneously, and so forth.

None of these, though, are a particularly strong sell for those who use a computer instrumentally: in reality, your Microsoft Words documents aren’t about to disappear, either. And there are disadvantages to giving up Word.

  • Things like footnotes with a citation manager are not very easy, even for the technically competent. 1{#fnref1.footnoteRef} Even footnotes without a citation manager are fairly clumsy.
  • The best tool for making your Markdown documents into attractive web pages , Pandoc, is not especially easy to install or configure if you don’t use the command line on a regular basis.
  • The core definition of Markdown is a little unclear: particularly in the last week, there have been some conflicts over the definition that will be confusing to newcomers. (Although the proposal that sparked them, “Common Markdown,” is likely to be a good thing in the long run)

The heart of Markdown’s appeal is its flexibility: to drive any adoption outside the hard core of people, you need a killer app built off of it that solves a problem. In the technology sector, that has been Markdown’s ability to easily handle links and snippets of computer code for those writing on two widely used sites, GitHub and Stack Overflow

Among historians, neither of those are very important. And the footnote problem is big enough that I generally wouldn’t recommend anyone to use Markdown, right now, unless they enjoy banging their head against the wall.

Lectures and Notes: the killer apps.

There are two places, though, where even historians don’t tend to use footnotes: lectures, and notes. And in both of these, Markdown makes some amazing things possible.

If there’s any reason for historians to use markdown, it’s in these two spheres. The reason I keep using Markdown is that it makes it possible for me to personally solve two problems that have driven me crazy:

  1. Quickly making slides decks to go alongside a lecture, and borrowing and reusing chunks of slides from one talk in another;
  2. Making heads or tails of the thousands of pictures you take while in an archival trip.

Markdown and lectures: multimedia and transposability.

First lectures. With Markdown, I’m able to write my own notes and create a slide deck at the same time. An example will help. Here’s a snippet from my lecture notes on the memory of the Civil War:

# Abolitionist memory of the war.

*Image: http://upload.wikimedia.org/wikipedia/commons/1/19/William-Tecumseh-Sherman.jpg* Caption: William Tecumseh Sherman

There's another set of people who aren't content to see it go: those who remembered the war as the period of national renewal, rebirth, and freedom. We remember World War II today as the "Good War," because we fought the Nazis and won. 

But unlike WWII, Civil War actually changed the country for the better. It abolished slavery. It instituted amendments that guaranteed citizenship to every American. It promised equal protection under the law.

Memory that's particularly strong among African Americans.

They remember Sherman differently.
Sherman not as maurauder but as unfifilled promise.
Sheman, you might remember, when he finally made it to the sea issued his famous **Field Order 15**

With some ancillary code I wrote, that does two things at once: builds a slide showing the wikimedia copy of Sherman’s grizzled mug, and creates a set of notes for me under the header “Abolitionist memory of the war” to go on the paper notes I’ll read from.

Later on, I’ll write another script that will find pull every phrase in boldface (like “Field Order 15”) from all my notes and put them onto a list of possible IDs for the midterm I can hand out. Another script could strip just the section headers and print out outlines for the lectures to hand out before class.

This is writing documents for multiple uses, and it can be incredibly useful. If, two minutes before class, I decide I want to switch the order I talk about the abolitionist memory of the war and the white supremacist memory of the war, I can just cut and paste the chunks of text, and all the slides associated with each will have their order switched.

Something like this could provide a really useful way to integrate and share resources, and free up some of the tedium with prepping lectures. But:

  • That syntax for including an image as a slide is my own, not standard Markdown. I’ve defined scripts for dropping in YouTube videos, images, captions, and some other predefined formats: but it would take a lot of work to define a set of them that make sense for anyone but me.
  • There are a lot of standards out there for working with HTML slides. None is winning, in part because none is anywhere as good as Keynote or Powerpoint for the average user. My code works with deck.js, one of the only HTML formats not supported by Pandoc; but there’s no obvious other standard to switch to.
  • Constructing slides that are more complicated than a single image with a title, or a numbered list, requires some serious HTML/CSS expertise. My scripts support that, but not in a pretty way.

Modern HTML allows some beautiful things: I can easily imagine a GUI for one of the standards that would make it easy to create slides for re-use in one of the competing platforms. But I think the standards are still evolving too rapidly in this sphere to make the way forward obvious.

Pull out the slide deck, and you still might have a useful tool here: something that generates a lecture notes for me, outlines for the students/course web page, and IDs for the test prep sessions. But I think there’s something even more valuable possible for archive notes.

Markdown and the Archives: integrating notes and photos

Markdown is a great language for taking archival notes. Archives are all about hierarchy: and Markdown easily lets you tag mutliple levels of headers (Series, Box, Collection, file…). But so is Microsoft Word: and there are plenty of outlining programs out there that are even better.

There are a few things that Markdown notes might do more easily than normal ones. Build a good enough web interface, and you could even click on a photo or quote in your notes and instantly get back a string that ascends the various headers to tell you where it is: Series 3a, Box 13, Folder 4, Letter on 4/18. But the place where there’s really an opportunity lies in Digital Photos.

Digital cameras have completely changed historians’ relations to archives in the last 15 years. (That is, in the subset of archives where cameras are allowed). We used to take notes: now, a massive part of our archival practice involves taking pictures, which have to be sorted through on our return.

When I’m wading through boxes, I tend to type the name of the box, and then some information about each folder followed by descriptions of the documents: if it’s especially useful or especially visual, I take a picture (or a series of several pictures). I think this is pretty similar to what most people do. It means that I end up with two separate timelines to sort through when I get home. 1) A bunch of textual notes that contain my impressions of the works and the rationales for why I copied them and what they are. 2) A stream of pictures with little context but their order to patch together their origin, sometimes with a close-up of a box or folder label thrown in to help.

The tough question is: how can you insert pictures into your notes? Unless you want to physically pick up your laptop and use the webcam for your pictures, it’s not obvious what the best way would be. And if you try to put more than a couple pictures into a Word document, it will crash right away.

Unlike the systems most historians use for notes, Markdown is plain text and has an easy method for inserting multimedia. That means that you can use it to integrate your archival photos directly into your notes; and that unlike Word, it can handle hundreds of images or thumbnails with aplomb.

The last challenge is knowing which parts of your notes go with which pictures. This is a surprisingly hard thing to solve: but there’s an existing answer in a second technology much beloved by the technology industry: version control.

Version control can get complicated, but in its simplest form it’s much like a wikipedia edit history: not just the current state of a file, but every previous revision is stored in memory.

So for archival notes, we just need to save the state of your archival notes every 10 or 15 seconds; match those markers against the timestamps of the photos from a digital camera; and insert the pictures into the text just in place.

When you want to review your notes, you just open them up in HTML format: thumbnails of every picture will appear in place, and you can click on them to get the full version.

For the technically savvy, I’ve put a set of scripts online that do just this. I use gitit to view the notes themselves so I can interlink between pages. A daemon handles the git commits: but that only works because I have always been a compulsive, several-times-a-minute saver of my documents.

What would a user-friendly platform look like?

My repo might be useful for those who are already comfortable with tools like version control: but those are the people who are already using Markdown anyway.

To make this useful for anyone else, we’d need a system with three easy, non-command line steps:

1. Installation

Puts Pandoc, Git, and a good Markdown editor on your computer at once.

2. Writing (in the archives)

This should resemble existing note taking as closely as possible: the user will need to make sure their camera’s clock is well-calibrated, but other than that it should look only like using a new text editor.

Whenever you type in the editor, it saves the files and runs git commit at close intervals. (Git experts may find the idea of automatic commits without a clear commit message cringe-inducing. Insofar as they have a point, edits should probably take place on a separate branch that is forked back into the main one periodically.)

3. Compilation (loading your pictures)

Imports photos from an sdcard or photo library, finds the version control files and matches photo times against them, and builds an html file for each document of notes.

What’s the platform?

Some of the technical components are obvious. I can’t imagine using anything other than git for version control; and though I use gitit to view files, I think that standalone html files are the only sensible way for most people to view their files. The scripting language for step three, as well, isn’t very important: I’ve used python, but anything with a set of hooks into git.

The big question is: what’s the text editor to be? I use emacs, and get the impression that most people writing in Markdown are using vim. Both of these are clearly bad choices for the ordinary historian. For all that Markdown can be written in any editor, the writing function also must support auto-save and auto-git-commit, so anything without a scripting interface is out. SublimeText has its selling points, but free’s probably the way to go.

That means, unless I’m missing a central player in the ecosystem, that the natural choice is the new Atom editor from Github. But perhaps there’s a more lightweight alternative?

Platform will also be an issue. The Mac is the obvious platform to capture a majority of historians: but a surprising number of people seem to take their notes with an iPad-keyboard array, which would call the whole stack into question.

Infrastructure

So that’s the proposal. Once historians see how great Markdown is for notes, maybe they’ll think about it for lectures; once they use it for lectures, maybe the footnote ecosystem will start to improve, and we’ll finally be able to distribute historical papers as text, making them more portable, more easily structured, and more lasting.

So, anyone want to try?

  • It took me a few hours of mucking about in Emacs Lisp to make inserting a link to something in my Zotero library almost as easy as it is under Microsoft Word; and if you want to configure the core behavior of Pandoc, it’s best to use Haskell. Even the “programming historian” may not have heard of either of these languages. Both (well, at least Haskell) have their strengths: but suffice it to say that neither has ever been anyone’s answer to the question “If I should only learn one computer language, which should it be?”↩
  • 28 Jun 00:47

    Finding the best ordering for states

    Here’s a very technical, but kind of fun, problem: what’s the optimal order for a list of geographical elements, like the states of the USA?

    If you’re just here from the future, and don’t care about the details, here’s my favorite answer right now:

    ["HI","AK","WA","OR","CA","AZ","NM","CO","WY","UT","NV","ID","MT","ND","SD","NE","KS","IA","MN","MO","OH","MI","IN","IL","WI","OK","AR","TX","LA","MS","AL","TN","KY","GA","FL","SC","WV","NC","VA","MD","DE","PA","NJ","NY","CT","RI","MA","NH","VT","ME"]

    But why would you want an ordering at all? Here’s an example. In the baby name bookworm, if you search for a name, you can see the interaction of states and years. Let’s choose “Kevin,” because it played such a role in my anachronism-hunting piece on Lincoln.

    Clearly the name took off around the start of the baby boom. But is there a geographical pattern? It’s very hard to say. It does look like the red names begin around 1955 in much of the country. But in a few, it’s not until the early 1970s. Which ones? Alabama, Georgia, North Carolina, South Carolina. That is, after substantial reading parsing over to the axis, it’s clear that most of those are southern states. But this is the sort of insight that should be immediately obvious. And there may be other connections we’re missing out on. The whole point of data visualization over tables is that you can pick out patterns using faster forms of cognition: requiring you to push over to the left to read off the names is a major loss.

    Alphabetical order makes it easy to find any individual state (assuming you know its name) but hard to see the way related states move with each other.  It means that to trace out regional variations over time, we tend to animate maps: but using time as the proxy for time makes cross-temporal comparisons much harder to tell. As Tufte says, comparisons should be enforced across the eyespan: relying on animation to trace out common names is a big problem. So there’s a dramatic interest in seeing different names pop up in (for instance) Reuben Fischer-Baum’s animation of baby names; but you have to watch the whole thing to think through questions like “what regions tend to adopt names early?” or “what’s the name that stays on top for the longest?”

    Putting it all into X-Y makes these questions easier. But that means we need to map states to X or to Y. Alphabetical order means that states are not arranged in a way that places states near others like them.

    So how could we make the states usefully arranged? We need some dimensionality reduction.

    Linear reductions

    One obvious way would be east-to-west or north-to-south: that starts out quite well, with all of New England:

    ME MA RI NH VT CT NY NJ PA DE MD DC VA NC SC WV OH FL GA MI KY IN AL TN IL WI MS MO LA AR MN IA TX KS NE OK SD ND WY CO NM MT UT AZ ID NV CA OR WA AK HI

    But quickly falls apart with Ohio, Florida, Georgia, and Michigan in immediate succession. If we plot the states, you can quickly see why. Rather than list orders, I’m going to show them as paths through a map: here’s what that looks like in this case.

     

    (By the way, you can see that the points are a little arbitrary: I’ve taken the first geonames hit for the state, which is sometimes the capital, sometimes the state centroid, and sometimes the most important city. Ideally I’d be using the population-weighted centroid, but in some ways I kind of like the results that come out of this.

    There are some other possibilities for linear dimensionality reduction (principal components comes to mind) but they’ll have the same fundamental problem. We want a metric that takes proximity more fully into account. Even non-metric multi-dimensional scaling fails: it handles a couple cases better (Jackson and St. Louis are in a more sensible order, for instance), but it still jumps erratically up and down, preventing any larger groups like “the south” from coming into sight:

     

    Hierarchical clustering approaches

    One possible approach, suggested to me by Miriam Huntley, is hierarchical clustering: using distances, we can cluster the states by proximity. Here’s the initial result of that:

    The individual groups are quite nice (New England is there, plus New York at the end), and every state is adjacent to an immediate neighbor. And while the groups have geographical coherence, they aren’t exactly the regions we know and love: the “mid-atlantic” runs down to South Carolina, and the midwest includes the gulf coast all the way to Tallahassee. The connections between the groups are scattered. Florida is next to Pennsylvania, and South Carolina to Massachusetts. Seen as a path, the weirdness of this is clear:

    Leaf ordering in dendrograms is arbitrary, however, and we can do better than this. Using a method developed by Bar-Joseph et al, and implemented in the “cba” library for R, we can reorder the dendrogram so that groups stay the same, but the leaves are ordered so that transitions from one group to the next are maintained.

     

     

    Now, the path looks considerably better:

     

    The clusters remain adjacent, but now the transitions are so smooth that it’s not obvious where one begins and the other ends. Instead, we get a serpentine path through the states that both ensures every path is between two adjacent states, and keeps paths generally inside the same region.

    Network approaches

    Can we do better? The strategy of plotting these as paths suggests that maybe this is an instance of the traveling salesperson problem, in which we want to travel through all the states minimizing the distance traveled. Why shouldn’t the “best” solution simply be the one where the overall sum of distances is the least?

    Inserting a dummy node as start- and end-point lets us view that: using the best method found by the “TSP” package in R (which is not guaranteed to be the optimal solution, since the traveling salesman is a notoriously difficult problem to solve), we get quite a different path:

     

    Rather than start in Maine, this route begins in Tennessee! After winding through the Midwest to West Virginia, it leaps to Vermont and then takes a beautifully practical course down the Eastern seaboard through Texas, through the great plains, and then takes up nearly an east-to-west ordering through the Mountain and Western time zones. While many of the regional choices here look better to me than the dendrogram solution (particularly the coherence of the south, the distance-optimizing strategy means that there are a few nearby states that have nothing in common: the leap from New Mexico to Montana, for example, and the extremely strange choice to place Washington DC between West Virginia and Vermont, ten nodes removed from either Maryland or Virginia, the closest geographical points. (In fact, I think the route could be improved by heading straight to Vermont from WV and putting DC in its rightful place: but it says something that out of the 7 algorithms in the free version of the TSP package, none was able to improve on this route).

    Fractal Curves 

    Another option is not to minimize travel distance but to maximize the likelihood that two points will be next to each other. That suggests filling the geographic region with some kind of fractal curve, and then positioning each state along the curve.

    This is an appealing way to think of arranging the country linearly: not as a network, but as iterable set of points. For just the United States, we could use some already-existing curve path. The most widespread linear mapping of points is the Zip code system: Samuel Arbesman has written about this on Wired, and includes a link to Robert Kosara’s ZipScribble maps. Here’s Kosara’s idea with a few minor changes (I use a rainbow spectrum, rather than coloring each state separately, and an Albers projection. And it appears that the zip database I have handy has something weird going on in southwest Georgia.)

    Space-filling curves

    The ZIP system isn’t especially logical, but there should be something similar that’s better. My first thought for this problem, which the whole post, was to use a Hilbert Curve. It turns out that Kosara has mapped that approach onto the Zip dataset.

    Using just the state points, it’s possible to draw a Hilbert curve that covers the continental United States, and then visit each state at the moment it’s closest to the curve. The actual path taken can then be simplified down to eliminate the intervening states. Here’s what that looks like, with both the Hilbert curve and the simplified route. I’ve shaded the Hilbert curve using a double rainbow so it’s easier to trace from its origin near the Bahamas (first making shore near South Carolina) to its exit off the coast of Los Angeles.

     

    I’m disappointed by the performance here. While there is some regional coherence (the stretch from Wisconsin to Kansas is well done, and the first jumps through the South are acceptable), the square binning forces some rather strange choices: the odd jag down to North Carolina, the detour to Colorado and Wyoming.

    There are other issues as well. Hilbert curves work best in square spaces, and the patches of ocean/Canada/Mexico that get filled are pretty far off limits. While I don’t show Alaska and Hawaii, for the other algorithms they’ve simply been tacked on at the end in a reasonable manner: here, though, a solution that includes Alaska and Hawaii makes some significant changes to the full arrangement and vastly increases the percentage of empty space, which tends to introduce odd decisions (like interposing Alaska between Oregon and Nevada.)

     

    I suspect there are ways of optimizing the Hilbert curve, or some similar fractal path, so that it better maps onto actual geographic spaces. That seems like an interesting avenue, potentially: but the initial results here seem worse, not better, than traveling salesman approximations.

    Conclusions and Deus ex Machina

    So on this particular set, the best results seem to come from, in descending order,

    1. Reordered hierarchical clustering

    2. Traveling Salesperson solutions

    3. Fractal Curves

    4. Quasi-linear dimensionality reduction (east-to-west, multi-dimensional scaling, etc).

    For the general problem (European countries, say, or counties in a state) I’d probably start with reordered hierarchical clustering or TSP solutions, at least until I learn how better to fit a fractal curve to an arbitary space.

    But for this particular problem, I’ve got an ace in the hole: there _are _conventional orderings of states that provide an acid test. In particular, we want something that matches to census regions.

    The ordering inside census regions is arbitrary, just like our clustering diagrams. So the best possible solution that includes some knowledge about the intrinsically _real _regions of the United States (the midwest, the south, etc.) is to combine the census regions with the optimal-dendrogram measures.

    Putting phony clusters just from the census regions looks like this:

    I can just plug those into a dummy distance matrix so that group membership trumps any other sorts of distance: and then allow geographical distance to sort out the spinning of those trees into a more sensible order.

    So, adding the constraint that census divisions and regions be kept intact, the optimal ordering looks like this: starting in Maine, traveling through the South west to Texas, skipping to the upper Midwest and then taking the same route west through the plains and mountains as the dendrogram:

    Is this the perfect ordering? To my mind, it’s not: but the flaws come straight from the census, not from the algorithm. West Virginia should not in the coastal south, it should be in the same division as Kentucky; the leap from Oklahoma to Wisconsin is unfortunate, and so is the one from Florida to Kentucky. Still, the census regions constrain is quite nice to have. And unlike the unguided paths, it preserves all but one of what I intuitively think of as the essential pairings: the Dakotas, the Carolinas, Alabama-Mississippi, Vermont-New Hampshire, Kansas-Nebraska, Colorado-Wyoming.

    So, let’s return to the original visualization to see what this new ordering helps us see. Remember, this original version revealed only with some serious axis-reading that the South starting using “Kevin” later.

     

     

    Here it is with the census-based ordering. (For a version you can hover over, see the browser itself). The southern states, two-thirds of the way down the page, clearly do begin later: but now it’s also immediately evident which of them _don’t _lag as much. There are also several patterns that are immediate evident which remain completely obscure in an alphabetical ordering: usage of “Kevin” is significantly higher around 1990 in the northeast, particularly the mid-Atlantic, than it is in the rest of country. And while the South waits the longest, a lag in the Arizona-New Mexico pairing is also clear.

     

    This style of display also makes subtler patterns visible. “Jennifer,” for example, rises a year later in the South than elsewhere. That would be lost as visual noise in an alphabetical ordering, but is completely clear here.

    Is a geographical ordering the best? Not always. Take “Madison“: its rise shows striped bands that don’t seem to be regional. Illinois, New Jersey, Washington DC, and New Mexico all avoid the wave. In fact, if you look closer, this is clearly a racial thing: “Madison” was most popular in states with overwhelmingly white populations. (Except Wisconsin, it seems). And aside from the bend through the southwest, there aren’t a whole lot of largely-minority states in any contiguous curve.

     

    But on another level, that just points out more the usefulness of _some _sensible ordering to start with.

    28 Jun 00:30

    ‘Malicious software’ is deleting WD My Book Live NAS drive data

    by Patrick O'Rourke
    WD My Digital

    If you use a WD My Book Live NAS, it might be a good idea to disconnect it from your network.

    According to a post on the device’s community forum that Ars Technica picked up, some WD My Book Live users have discovered that all of their data has been removed from their device.

    WD says that the data relates to “malicious software” that says that My Book Live and My Book Duo owners should disconnect their devices from the internet to protect their data.

    Several posts on WD’s forum detail how user data has been wiped entirely but that the file structure is still present — the folders are there, but they’re entirely empty.

    One user that goes by the username ‘JHS1‘ said the following:

    “I kept all my documents on this drive. All files gone, four empty folders for: pictures, music, that I never used. My pictures are stored elsewhere. Unfortunately or fortunately my last backup was 4.5 months ago. I thought I had my backup program set for scheduled backups, but that is another problem to look into. If this turns out to be a WD problem, I will look elsewhere for future storage equipment. I wish everybody good luck and if someone finds a solution, please post it.”

    Several other users echo the above situation. It’s unclear what the source of the malicious software is or if WD is currently working on a solution. WD sent the following statement regarding the issue to several publications:

    “The incident is under active investigation from Western Digital. We do not have any indications of a breach or compromise of Western Digital cloud services or systems.

    We have determined that some My Book Live devices have been compromised by a threat actor. In some cases, this compromise has led to a factory reset that appears to erase all data on the device. The My Book Live device received its final firmware update in 2015.

    At this time, we are recommending that customers disconnect their My Book Live devices from the Internet to protect their data on the device.

    We have issued the following statement to our customers and will provide updates to this thread when they are available: https://community.wd.com/t/action-required-on-my-book-live-and-my-book-live-duo/268147

    I’ve used an older WD My Cloud NAS drive to store files I need to share across multiple computers for several years. It’s unclear if this issue also affects this particular drive model given it’s not a My Book device, but I’m disconnecting the drive just in case.

    Source: Ars Technica

    The post ‘Malicious software’ is deleting WD My Book Live NAS drive data appeared first on MobileSyrup.

    28 Jun 00:30

    Windows 11 needs a TPM, here’s how to enable it on your computer

    by Jonathan Lamont

    Microsoft unveiled Windows 11 this week, and the upcoming software upgrade looks great. There’s considerable hype around it and, naturally, a bunch of people are taking the time to check if their PC or laptop can run the next version of Windows.

    We have an explainer on how to check for compatibility you can check out over here. The short version of it is to download Microsoft’s PC Health Check app and run the compatibility checker.

    However, several people have received warnings from the Health Check app saying their PC isn’t compatible with Windows 11, even if it meets all the minimum system requirements.

    Microsoft may updated the app to show the reason a computer fails, which should help users troubleshoot what’s going on. One common culprit appears to be the TPM 2.0 requirement. It’s listed as a system requirement for Windows 11 and it helps improve security by providing encryption for data. (The Verge has since published a great explainer about why Microsoft is requiring this).

    Along with being helpful for security, Microsoft has required TPM modules for Windows machines since 2016. That should mean that a relatively recent Windows machine should have a TPM — so why are so many people getting error messages?

    If your PC failed the check, make sure you have a supported CPU

    The TPM issue primarily seems to affect people who built their own desktop PCs. That’s what happened to me, and to fellow MobileSyrup staff reporter Brad Bennett — both of our custom PCs failed the Windows 11 compatibility check despite sporting relatively new hardware and largely surpassing the system requirements. (As an aside, my desktop is a weird Frankenstein of parts from 2016 and 2020 and is literally held together with zip ties, so I wasn’t totally surprised when it failed the compatibility test.)

    Despite Windows requiring a TPM, many custom computers have one that hasn’t been enabled. There are also a few other potential hiccups, such as certain boot settings. But, if you ran the PC Health Check compatibility test and your computer failed, don’t panic yet — there are a few things you should try first. Before we get into it, be warned that the solution is a little technical. If you’re not comfortable digging into the BIOS on your computer or don’t know how these things work, you may not want to do this.

    First things first: make sure your CPU supports Windows 11. Microsoft posted an extensive list of supported processors here. The simple breakdown is that Intel 8th to 11th Gen chips as well as several iterations of Xeon CPUs will be supported. On the AMD side, Ryzen 2000, 3000, 4000, 5000, Threadripper 2000 and 3000, Threadripper Pro 3000 and EPYC 2nd and 3rd gen CPUs will be supported.

    However, Microsoft also notes that those CPU requirements are a “soft floor.” The Verge offers a more detailed explanation, but it means that some older CPUs will still run Windows 11, they just may not have official support from Microsoft. Speaking of soft floors, the TPM 2.0 requirement is also a soft floor — TPM 1.2 and up should work fine, and, like with the CPUs, people will probably be able to get Windows 11 regardless. Microsoft just hasn’t been great at explaining it.

    If you have a CPU from the last five years, you have a TPM

    David Weston, director of enterprise and OS security at Microsoft, took to Twitter to clarify some of the confusion around the TPM requirement. Weston explains that “almost every CPU in the last 5-7 years has a TPM.”

    This is where the BIOS part comes in. If you have a recent Intel CPU in your computer, you can head into the BIOS and enable a setting called Platform Trust Technology (PTT). For AMD CPUs, it’s called Firmware TPM, or fTPM. Enabling these options should allow your device to pass the Windows 11 compatibility checker, but if your PC still fails, there may be a few reasons.

    For one, the checker also requires that your computer uses UEFI and Secure Boot — if those settings aren’t enabled, the compatibility check will still fail. Alternatively, your CPU may not be on the list of officially supported processors. Again, that doesn’t mean you won’t be able to get Windows 11 on it — when you run the Windows 11 installer, you may get a warning, but it should still install.

    How to enable your TPM

    With all that said, here’s how to check your BIOS. When you start up your PC, press the Delete key or F2 (or something else, depending on your motherboard manufacturer). That should interrupt the normal boot process and put you in the BIOS instead.

    Then, look for the PTT option if you have an Intel CPU, or the fTPM option on AMD. Unfortunately, where these options are and what everything looks like will differ significantly from manufacturer to manufacturer and even from motherboard to motherboard. I had no luck finding PTT on my own, but my BIOS software thankfully included a search function that helped me locate and enable PTT. There’s a good chance the option will be under the ‘Advanced’ section of your BIOS.

    While you’re in the BIOS, make sure your boot settings are correct. There should be a ‘Boot’ tab, so select that and then check to make sure UEFI and Secure Boot are enabled.

    If you need help finding these options, check your motherboard manual (if you don’t have it, most manufacturers offer digital copies on their website — just look up your motherboard model).

    Once done, save the settings and reboot. If everything worked, you should now have a TPM. An easy way to check if it worked is to press ‘WIN+R’ on your keyboard, type ‘tpm.msc’ and press enter. You’ll see a window that shows if Windows detects the TPM — if it does, you should be able to pass the compatibility check. It worked for me, although Bennett wasn’t able to pass after enabling fTPM on his AMD CPU — but he didn’t enable UEFI, so that may be where his issue lies.

    As another aside, it’s possible that enabling these settings could cause problems. Enabling PTT worked fine for me, but Bennett ran into some issues after enabling fTPM. He was eventually able to boot his computer again after some troubleshooting.

    Maybe just wait until Windows 11 arrives

    Unfortunately, some people may still be holding on to a really old CPU. It’s common for people who build their own PCs to upgrade the GPU every few years while holding onto the same CPU since the graphics card tends to become outdated quickly. If you have a technically not-supported CPU, or a CPU without a TPM, there are a few options.

    The first is to check if your motherboard has a spot to install a TPM. If so, you can buy a TPM online and install it yourself. However, that might be difficult (or expensive) right now since the confusion over the TPM requirement has caused demand to skyrocket. Now, people are scalping TPMs for a significant upcharge:

    The other option is to upgrade your PC with a newer CPU, although that may require a new motherboard as well and at that point, it may make sense to just build an entirely new PC.

    Thankfully, there’s still some time until Windows 11 arrives. And even when it does arrive, Microsoft will continue supporting Windows 10 until October 14, 2025. Windows 10 is also still getting the 21H2 update later this year. It will be the last feature update for Windows 10 and, while it’s not Windows 11, it’s better than nothing for those who can’t get Windows 11 on their machine.

    With that in mind, it may be better to just wait until Windows 11 releases and see how things shake out. As mentioned above, some of these new requirements are somewhat soft. Even if your PC fails the compatibility check, you may still be able to install Windows 11. Right now, we just don’t know. Before making any big decision, like buying a whole new PC, you may want to wait and see how the Windows 11 update shakes out.

    Unfortunately, this process is all quite confusing. As The Verge points out, this is the first significant shift in Windows hardware requirements in nearly a decade. Windows 8 brought significant changes in 2012, and since then, Microsoft hasn’t changed much. It’s no surprise that these new requirements are catching people off guard.

    Update 06/25/2021 at 3:59pm: Added more clarification around why Microsoft requires a TPM and included details about the updated PC Health Check app. A previous update added some more detail about buying a TPM module.

    Source: @dwizzzleMSFT, The Verge, Microsoft, Tom’s Hardware

    The post Windows 11 needs a TPM, here’s how to enable it on your computer appeared first on MobileSyrup.

    25 Jun 03:27

    The Best Folding Bike

    by Amy Roberts and Duncan Niederlitz
    A person shown standing infront of a brick wall holding a Dahon Mariner D8, our pick for the best folding bike.

    A folding bike may be the most convenient mode of transportation on two wheels.

    It can get you from point A to point B as readily as a full-size bike, but you can stash it in a car trunk, tuck it under a desk, or store it in a closet, on a boat, or inside an RV.

    To suss out which folding bike does it all best for most riders, we pedaled and shifted, folded and unfolded, and carried and maneuvered 15 popular models.

    After our 85-plus hours of research and testing, the Dahon Mariner D8 edged to the front of the pack, combining a comfortable ride and easy folding with good-enough components, all for a reasonable price.

    25 Jun 03:27

    Introducing Windows 11

    by Volker Weber

    Sehr schöner Blog-Eintrag, direkt bei Microsoft:

    At a time when the PC is playing a more central role in our lives, Windows 11 is designed to bring you closer to what you love.

    Was bei mir sofort hängen blieb:

    • Eine modernisierte UX, die mir ausnehmend gefällt
    • Windows Store für alle Typen von Apps
    • Microsoft will keinen Anteil am Umsatz von Spotify etc.
    • Android Apps aus dem Amazon Store
    • Widgets
    • News und Channels
    • Irgendwas mit Gaming, was ich nicht verstehe 🙂

    Zusammengefasst: Microsoft macht sich mächtig Mühe, relevant zu bleiben.

    Was mir gefehlt hat: Was ist mit Windows on ARM?

    Totalkatastrophe: Microsoft hat extra eine Media Site aufgesetzt, die es nicht schaffte, den Stream zu übertragen. Abhilfe: Man geht einfach auf Twitter und streamt dort.
    25 Jun 03:26

    Windows 11 will get one major update per year

    by Karandeep Oberoi

    One of the main criticism about Microsoft and Windows 10, in general, is how the company handles its OS updates.

    Throughout Windows 10’s lifespan, Microsoft aimed to release two major updates each year, however, this often resulted in stability and performance issues. But we learn from our mistakes, including Microsoft.

    With Windows 11, Microsoft is abandoning the strategy of two updates a year in favour of releasing one big upgrade per year, similar to what Apple and Google do with their respective OS.

    One big update per year gives Microsoft more wiggle room to add bigger and better features in the OS without compromising on stability and performance. Additionally, according to Microsoft, updates on Windows 11 are expected to be 40 percent smaller than its Windows 10 counterpart, resulting in faster download and installation.

    These once-a-year updates are scheduled to be released in the second half of each calendar year.

    Source: Microsoft

    The post Windows 11 will get one major update per year appeared first on MobileSyrup.

    25 Jun 03:26

    A storage crisis

    by Doc Searls

    The best new phones come with the ability to shoot 108 megapixel photos, record 4K video with stereo sound, and pack the results into a terabyte of onboard storage. But what do you do when that storage fills up?

    If you want to keep those files, you’ll need to offload them somewhere. Since your computer probably doesn’t have more than 2Tb of storage, you’ll need an external drive. Or two. Or three. Or more. Over time, a lot more.

    Welcome to my world.

    Gathered here for a portrait in a corner of my desk in Manhattan are 22 hard drives, plus three SD cards that each exceed the capacities of the drives they’re laying on. And then there’s the 2Tb one in the laptop I’m using now. That one has 357.33Gb available. Most of the others you see are also full, dead, or both. Five have FireWire connections, which my current laptop doesn’t comprehend at all. I also have a similar collection of drives in Santa Barbara.

    Photos occupy most of the data I’ve stored on all those drives. Currently my photo archives are spread between two portable drives and my laptop, and total about 7Tb. I also have a 5Tb portable drive for videos, which is back in Santa Barbara awaiting dubs off tapes. The portable photo drives are among those in the picture above. Earlier today, my laptop gave me this news about the main one, called Black 4Tb WD Photo Drive:

    That’s why I’m transferring its contents over to the 10Tb drive called Elements, on the far left. A progress report:

    About 5Tb of Elements is occupied by Apple Time Machine backups. After the transfer is done, there won’t be room for more backups. So my project now is figuring out what to do next.

    I could get some Network Attached Storage (NAS). I already have used 2012-vintage 18Tb QNAP one in Santa Barbara that I’ve never been able to make work.

    So I am tempted now to put it all in a cloud.

    I hadn’t bothered with that before, because upstream speeds have been highly sphinctered by ISPs for decades. Here in New York, where our ISP is Spectrum, our speeds have long run 100-400 Mbps down, but only 10 Mbps up. However…. I just checked again with Speedtest.net, and got this:

    And that’s over wi-fi.

    Now I’m encouraged. But before I commit to a supplier, I’d like to hear what others recommend. Currently I’m considering Backblaze, which is top rated here. The cost i $6/month, or less for unlimited sums of data. But I’m open to whatever.

    [Later…] Hmm. At that last link it says this:

    What We Don’t Like:

    Something I should mention is that some users have had bad experiences with Backblaze because of a not-so-apparent feature that maybe should be a lot more obvious: Backblaze doesn’t function as a permanent archive of all of your data, but instead as a mirror.

    In other words, if you delete files on your computer, or the drive fails and you’re connected to Backblaze’s website, Backblaze will see that those files are gone and will remove them from your online account, too.

    Granted, signing up for the forever version history option would eliminate any issues with this, but it still poses a problem for anyone using one of the limited version history options.

    Alas, the forever thing is complicated.

    To be clear, I want more than a mirroring of what I have on my laptop and external drives. I want to replace those external drives with cloud storage. Is that possible? Not clear.

    Alas, for all of us, this problem remains.

    Oh, and Spectrum now only measures under 10Mbps upstream. So forget the cloud.

    23 Jun 22:51

    Every so often I remember Mitterrand’s horrific last meal

    Every so often I think about the last supper of ex French president François Mitterrand, which had a horrifying beauty. He was dying of age and a long illness.

    There were oysters and so on, a feast, then a dish called ortolan.

    It’s a small songbird. To prepare it, the ortolan is drowned in a glass of Armagnac. This is not a metaphor. It is actually drowned, and then it is cooked in a cassoulet.

    It is illegal, but some chefs will make it.

    Then to eat it (which is how Mitterrand ate his):

    You place a white cloth over your head and pick the bird up with your fingers, and then you eat it whole, wings, feet, organs, head, everything except the feet. The ortolan is supposed to represent the soul of France.

    The white cloth is to create a closed sensory world of just taste and scent.

    The cloth is also, traditionally, to hide the act from God.

    After the meal Mitterrand didn’t eat again (by choice it seems). He died 8 days later.

    There was an extraordinary article in Esquire in 1998 by Michael Paterniti telling the story of ortolan, and Mitterrand’s meal… and also Parterniti’s experience in recreating it himself. It’s visceral prose.

    Here’s what I taste: Yes, quidbits of meat and organs, the succulent, tiny strands of flesh between the ribs and tail. I put inside myself the last flowered bit of air and Armagnac in its lungs, the body of rainwater and berries. In there, too, is the ocean and Africa and the dip and plunge in a high wind. And the heart that bursts between my teeth.

    – Micheal Paterniti (Esquire), The Last Meal (1998)

    (Paterniti also did this programme with NPR in 2008 which is a shorter read.)

    I don’t know what keeps drawing me back to this story.

    It’s shocking, for one. Real shock seems rare in the WEIRD 21st century, like boredom and like awe. I don’t mean shocking like a jump scare, or overwhelmed with horror. I mean the act has this enduring shockingness. No matter how many time I go back over the story, it’s this flawless crystal of beautiful, exquisite taste indivisibly joined to a central horrifyingly barbaric act - the drowning - like an equation somehow. The context in which Mitterrand chose the meal is part of it too. It seems emblematic of so much of the privilege and progress we have today, individually and as society, beauty with an horrific core, irreconcilable you would have thought. An artist or a writer would be able to decipher what’s going on, to diagram the entire thing, but for me it’s like staring at a Rothko painting. I can’t tell you why I can’t look away, but there it is.

    23 Jun 22:42

    Why Are Arrays So Hard For Beginners?

    Alfred Thompson, Computer Science Teacher, Jun 23, 2021
    Icon

    There are some seriously odd concepts in computer science. These concepts are like air for programmers - so normal they're not even noticable. But for beginners, it's like not knowing how to breathe. Here's Alfred Thompson: "Somehow the idea that the index is, in some sense, part of the name and not the value being stored is hard for some to grasp. Though come to think of it, I never talked about an index as being like a name... Perhaps I will discuss indexes as part of the name next time I teach arrays." That's the sort of thing I mean. When I was studying logic and programming I found the lessons full of such background concepts that were never stated, but just assumed to be understood. (Society is like that too, and uses this as one of its primary differentiators of power and class).

    Web: [Direct Link] [This Post]
    22 Jun 06:10

    Happy solstice. Happy indigenous people's day. ...

    by graydon
    Happy solstice. Happy indigenous people's day. Happy (delayed) juneteenth. Happy day of royal assent for bill C-15, enshrining a somewhat vague requirement to abide by UNDRIP in Canada's legal system.

    I walked to down and got some summer supplies. I did some hobby work, I read some Alison Bechdel, I watched the sun set.

    I felt relatively good. Six months ago I was a fair bit more anxious; today I feel a fair bit more calm. Maybe in another six months I'll feel different again.

    This entry was originally posted at https://graydon2.dreamwidth.org/290745.html. Please comment there using OpenID.
    22 Jun 04:13

    Sobeys is Uphill

    by peter@rukavina.net (Peter Rukavina)

    Really the only downside to cycling to get groceries at the Allen Street Sobeys is the hill climb from my house to the crest of Upper Prince Street. As arduous cycling goes, it ain’t no Andean trek. But it’s a hill. And on a hot tired day, maybe that’s enough to be too much.

    I thought I could play an end run around gravity tonight by sneakily cycling over the bridge to the Stratford Sobeys, benefiting both from the absence of the hill climb and from the delights of the “Extra” in that location.

    ‘Twas not meant to be, alas: I couldn’t figure out why the cycling was so sloggy-going on the way there. I was pulling a trailer, yes, but even with allowances there was a molasses quality to the ride.

    It was only when I turned around and essentially coasted all the way home, and then consulted a topographic map, that I realized that Sobeys in Stratford is at 23 meters above sea level while the crest of the Upper Prince hill is only 17 metres. So Stratford Sobeys is atop a higher mountain.

    Even that, and that it’s twice the distance from home, the Stratford Sobeys has one killer feature for the cyclist: almost all the route there is on dedicated cycle lanes (Hillsborough Bridge, MacKinnon Drive) or on streets with a marked cycleway (Water Street), whereas the ride up to Allen Street is a relative hellscape of parked cars backing up, squeezing through the needle of Upper Prince, and a tricky turn onto Allen Street.

    We’ve got some work to do.