Shared posts

15 Dec 04:56

Stuff that works :: Jabra Elite Pro 7 & Nothing ear (1)

by Volker Weber

Die neuesten Earbuds in meiner Sammlung haben sich bewährt und ich kann sie beide empfehlen. Beide bieten ANC und eignen sich auch zum Telefonieren. Beide Cases lassen sich induktiv auf Qi-Ladern auffüllen.

Die beiden Jabra Elite 7 Pro verschwinden komplett in der Ohrmuschel. Der Sound lässt sich an das Hörvermögen anpassen und zusätzlich über einen Equalizer verbiegen. Ich finde sie ausgesprochen bequem, aber das ist eine individuelle Erfahrung. Sehr gut: 8 Stunden Laufzeit mit ANC. 5 Minuten im Case bringen eine weitere Stunde und man kann jederzeit auch nur eins nutzen und das andere laden. Das Case ist schwierig zu öffnen, aber ich habe gelernt, damit zu leben. Aktuell sind die Elite 7 Pro mit 200 Euro noch recht teuer.

Die Nothing ear (1) waren meine Überraschung des Jahres. Im Bild sieht das das schwarze Sondermodell, normalerweise sind sie weiß und das gefällt mir sogar besser. Der kleine transparente Stiel ist ein guter Griff, um die Earbuds zu halten und zwischen Case und Ohr zu bewegen. Sie lassen sich viel leichter aus dem Ohr nehmen als die Jabras. Das Case ist recht groß, aber flach. Durch die Vertiefung im Deckel werden die Earbuds fest im Case gehalten und man weiß schon beim Anfassen, wie rum der Deckel aufgeht. Der Klang lässt sich nicht so fein regeln wie bei den Jabras, aber er passt bei mir genau. Aktuell kosten die Ear (1) unter 100 Euro.

Beide Headsets lassen sich durch zwei Tasten (Jabra) und zwei Touchfelder (Nothing) bedienen und konzentrieren sich dabei auf die wichtigsten Funktionen. Seht gut bei Jabra: Ein Tastendruck am linken Ohr schaltet das Mikro stumm. Das braucht man für Konferenzen. Beide Headsets arbeiten perfekt mit dem iPhone zusammen, weil sie den AAC Codec unterstützen. Dazu kann man auf dem iPhone in Telefonaten “Voice Isolation” einschalten und dann hört man keine Umgebungsgeräusche.

15 Dec 04:56

Back in the day when I thought Twitter was amazing…

by Nancy White
The blog post about Twitter from the amazing, generative and poetic Chris Lott is gone, but this snippet from 2017 remained in my draft blog posts.  Ruminate » Blog Archive » Twitter Community of Practice A paper presented here today analyzing a mobile social network application (kind like Twitter with groups, but only accessible from … Continue reading Back in the day when I thought Twitter was amazing…

Source

15 Dec 04:51

Leland Wilkinson, creator of The Grammar of Graphics, passed away

by Nathan Yau

Leland Wilkinson passed away on Friday, according to his daughter Amie. In visualization circles, he was best known for The Grammar of Graphics, which defined a system to describe and implement all statistical charts. Here’s Jessica Hullman on Wilkinson’s contribution:

My own admiration for Grammar of Graphics is partly why I chose to get into visualization back as a grad student. I remember thinking his concept of a frame was really important but underappreciated in any discussions I’d heard about visualization. I read it for the first time as a Ph.D. student and have been calling it my favorite book for years. Whenever I go back to reread chapters I always come away with some new appreciation. I even bring in a copy to pass around in my interactive visualization course, trying to get students to sense its influence and hopefully read it. Just looking at the examples is like an education in visualization.

Hadley Wickham, whose ggplot2 package in R is based on Wilkinson’s grammar, posted:

Lee Wilkinson is the reason that ggplot2 exists; not just because he wrote the Grammar of Graphics, but also because he was so kind and supportive to me when I was a young grad student thinking of trying to implement it. He will be missed.

The Grammar of Graphics was one of the first books I ever read on visualization, and the topics have stuck with me ever since. Wilkinson’s work no doubt influenced many others and helped push visualization to where it is today.

Tags: Leland Wilkinson

15 Dec 04:51

S18:E6 - How you can use music to learn code (Sam Aaron)

In this episode, we talk about how you can use music to learn code with Sam Aaron, creator and CEO of Sonic Pi. Sam talks about how programming basic games on calculators helped him learn to code, developing a passion for combining music and code, and creating Sonic Pi.

Show Links

Sam Aaron

Sam Aaron is a live coder who, through considering programming as performance, focusses on enhancing the productivity and power of modern programming languages and environments. Sam believes that a programming environment which has sufficient liveness, rapid feedback and tolerance of failure to support the live performance of music is an environment ripe for mining novel ideas that will not only benefit artistic practices themselves but also the computer industry and education more generally. In pursuit of this unique perspective Sam has created Sonic Pi, a free live coding environment designed to large both the needs of education and professional musicians.

15 Dec 04:51

The New LMS Rule: Transparency Working Both Ways

Lance Eaton, The Journal of Interactive Technology and Pedagogy, Dec 13, 2021
Icon

For my latest MOOC, I made sure I gathered no information about participants - no registrations, no names. Maybe I should write an article for a journal like this one. It reports on one instructor's self-admitted over-use of the tracking features in Instructure, and makes the suggestion that students should have the same access to data about instructors that instructors have on students. It's an interesting thought, and is a good blog post. But if the author is going to say things like "the pandemic increased that (power) imbalance through a form of disaster capitalism that had much of higher education more worried," then in an academic journal I'd like to see some evidence for that.

Web: [Direct Link] [This Post]
15 Dec 04:50

Climate change postcards from every country

by Nathan Yau

The effects of climate change can be seen around the world, in the present. The New York Times uses a mix of maps, charts, videos, illustrations, and photographs to imagine postcards sent from every country in the world to show what’s happening.

Tags: climate change, New York Times, postcards

15 Dec 04:50

The Great Bifurcation

by Ben Thompson

For the first several years of Stratechery I would write a year end article about “The State of Consumer Technology”; the last one I wrote, though, was in 2018, because consumer technology, dominated as it was by Apple and Google on the device side, and Google and Facebook on the services side, seemed rather stale and destined to descend into the world of politics and regulation (I was more optimistic about the enterprise, both in terms of the ongoing shift to the public cloud and the opportunity for SaaS companies).

That has largely proven to be the case, but it’s not the first time this has happened to technology; the pattern has happened twice before, and in each case the seeds of the next era were planted — usually by incumbents — while the previous era stagnated. And, in every case, the transition was marked by a reduction in lock-in and the devolvement of increasing amounts of autonomy to the individual user.

Tech 1.0: From Invention to IBM

The transistor, the foundation of modern computing, was invented at Bell Labs in 1947 by the solid state physics group led by William Shockley; nine years later Shockley moved to Mountain View, California to be close to his ailing mother in Palo Alto, where he started Shockley Semiconductor Laboratory. Eight of the researchers he hired, led by Bob Noyce, left the increasingly erratic Shockley a year later to found Fairchild Semiconductor, and in 1968 founded Intel with the support of Arthur Rock, one of the first venture capitalists.

The West Coast, though, was a sideshow compared to New York, where IBM had switched to transistors for the 7000 Series mainframe (as opposed to the 700 Series’ vacuum tubes); the real breakthrough was the modular and expandable System/360, which was the first computer bought by most companies, including the fictional SC&P from Mad Men:

There certainly was a connection to be drawn between IBM and the moon: IBM helped develop and track NASA’s initial exploratory flights and the eventual lunar mission. Here on earth, though, the Justice Department decided in 1969 that the company was in violation of antitrust laws; the case would be dropped 13 years later, but not before IBM voluntarily unbundled its software and services from its hardware, creating the first market for software.

Tech 2.0: King of the Hill

Notice those dates: by the time the Department of Justice sued IBM in 1969, Intel had already been founded; two years later an Intel engineer named Frederico Faggin designed the first microprocessor, the Intel 4004, which shrunk many of the functions of IBM’s room-sized computers to a single chip. Ten years after that IBM released the IBM PC, powered by Intel’s 8088 microprocessor.

The open nature of the IBM PC platform — at least once Compaq backward-engineered IBM’s BIOS — commoditized PCs; the real points of leverage in the PC value chain were Intel for chips and Windows for the operating system. The latter was a two-sided market: because so many businesses bought the Windows DOS-powered IBM PC, developers were motivated to make software for DOS; the more software for DOS, and later Windows (which was backwards compatible), the more that businesses sought out DOS/Windows-based computers. Over time more and more people who first used computers at work wanted similar functionality at home, which meant that DOS/Windows dominated the consumer market as well.

Thus was born another Justice Department lawsuit, this time against Microsoft’s alleged monopoly; that case was also eventually dismissed (although it lived on in various forms in the E.U. for years). Once again, though, the next paradigm that rendered the seeming monopolist’s lock-in immaterial was already in place: the Internet could be accessed from any computer, no matter its operating system. Moreover, in an echo of IBM’s voluntary unbundling of hardware and software, which created the conditions for tech’s next evolution, it was Microsoft that introduced the XMLHttpRequest API to Internet Explorer, which undergirded the Ajax web app architecture and Tech 3.0.

Tech 3.0: Software Eats the World

If Tech 1.0 was about hardware, and 2.0 software, 3.0 was about services. On the enterprise side this meant the development of the public cloud and software-as-a-service applications that required nothing more than a browser and a credit card; Marc Andreessen’s famous 2011 essay, Software Is Eating the World, is really about this transformation from software you installed on your computer to software you accessed over the Internet:

Companies in every industry need to assume that a software revolution is coming. This includes even industries that are software-based today. Great incumbent software companies like Oracle and Microsoft are increasingly threatened with irrelevance by new software offerings like Salesforce.com and Android (especially in a world where Google owns a major handset maker).

In some industries, particularly those with a heavy real-world component such as oil and gas, the software revolution is primarily an opportunity for incumbents. But in many industries, new software ideas will result in the rise of new Silicon Valley-style start-ups that invade existing industries with impunity. Over the next 10 years, the battles between incumbents and software-powered insurgents will be epic. Joseph Schumpeter, the economist who coined the term “creative destruction,” would be proud.

One way to think about this is that tech, for the first 50 years of its existence, mostly competed with itself: who could make the best operating system, the best database, the best ERP system. All of these were then adopted by legacy businesses who saw large efficiency gains. The SaaS revolution, though, saw tech turning its sights on the markets those legacy businesses served, bringing to bear entirely new ways of solving problems that started with the malleability and scalability of software as a core principle, not simply as a tool to do the same thing more efficiently.

The consumer space, meanwhile, has arguably been a decade behind the enterprise; e-commerce, for example, was primarily an Amazon story, and social media was all about Facebook. Both took analog concepts and digitized them: Amazon was the Sears and Roebucks catalog with far more products and far faster delivery; Facebook was literally named after a physical artifact, Harvard House face books.

What made Facebook so popular — and why the product retains its stickiness, even today — is that it is first and foremost the online representation of your offline relationships, whether those be family, classmates, friends, or co-workers. This also explains why the most successful Facebook add-ons, like Marketplace and Groups, are themselves often rooted in the physical world. These relationships represent a network effect — the more people you know who are on Facebook, the more valuable it is to you — and once again regulators have come knocking, in this case the FTC.

Note, though, that these cases are, if anything, becoming weaker over time, at least in terms of traditional antitrust concerns: while IBM’s market power was based on top-to-bottom integration that completely foreclosed competitors, Microsoft’s was about a two-sided network that was wide open for developers and users. Facebook, meanwhile, only has its users and their relationships to each other; that is the only thing preventing anyone from using another service.1

Which, of course, they are.

Social Networking 2.0

Last December I wrote what was, in retrospect, an important precursor to this piece (consider it my 2020 State of Technology article): Social Networking 2.0.

[My Bucks DM group is] not my only online community: while the writing of Stratechery is a solo affair, building new features like the Daily Update Podcast or simply dealing with ongoing administrative affairs requires a team that is scattered around the world; we hang out in Slack. Another group of tech enthusiast friends is in another Slack, and a third, primarily folks from Silicon Valley, is in WhatsApp. Meanwhile, I have friends and family centered in Wisconsin (we use iMessage), and, of course Taiwan (LINE for family, WhatsApp for friends). The end result is something I am proud of:

A drawing of Ben's Communities

The pride arises from a piece of advice I received when I announced I was moving back to Taiwan seven years ago: a mentor was worried about how I would find the support and friendship everyone needs if I were living halfway around the world; he told me that while it wouldn’t be ideal, perhaps I could piece together friendships in different spaces as a way to make do. In fact, not only have I managed to do exactly that, I firmly believe the outcome is a superior one, and reason for optimism in a tech landscape sorely in need of it.

The argument in that piece is that Facebook and Twitter represented Social Networking 1.0, where you were expected to be your whole self online; that expectation, though, was like a legacy company using computers to run their analog business model: it may have been more efficient, but it wasn’t at all an optimal use of technology. The entire magic of software is that it is malleable and scalable, and those qualities extend to users creating completely different personas and experiences based on the particular online community that they have joined.

Facebook, incidentally, like IBM and Microsoft in previous eras, has contributed to this evolution, particularly its acquisition and continued support of WhatsApp: while groups exist on all sorts of platforms, from Twitter to Facebook Groups to iMessage, WhatsApp seems to have a major share of these ad hoc private groups, particularly internationally but increasingly in the U.S. as well. What is notable about WhatsApp is that the key identifier is not your account, but rather your phone number; any sort of technological lock-in has completely disappeared.

Tech 4.0: The Metaverse

One of the biggest topics of 2021 has been the Metaverse, thanks in large part to Facebook’s pivot to Meta (even if Microsoft was first). It has been tricky, though, to define exactly what the Metaverse is: everyone has a different definition.

I think, though, I have settled on mine, and it starts with this comment from Meta CEO Mark Zuckerberg:

I think that the phrase “the real world” is interesting. I think that there’s a physical world and there’s a digital world, and increasingly those are sort of being overlaid and coming together, but I would argue that increasingly the real world is the combination of the digital world and the physical world and that the real world is not just the physical world. That, I think, is an interesting kind of frame to think about this stuff going forward.

I both agree and disagree with Zuckerberg; on one hand, I think he is absolutely correct that using “the real world” to only apply to the physical world is a mistake. Think back to those communities I described above that provide so much meaning to my life: those are almost completely online, but the sense of belonging is very real to me. Or think of this Article you are reading: it is nothing but endlessly replicable bits on the Internet, yet it is my career.

Where I disagree is with the idea that the physical world and the digital world are increasingly “being overlaid and coming together”; in fact, I think the opposite is happening: the physical world and digital world are increasingly bifurcating. Again, to use myself as an example, my physical reality is defined by my life in Taiwan with my family; the majority of my time and energy, though, is online, defined by interactions with friends, co-workers, and customers scattered all over the world.

For a long time I felt somewhat unique in this regard, but COVID has made my longstanding reality the norm for many more people. Their physical world is defined by their family and hometown, which no longer needs to be near their work, which is entirely online; everything from friends to entertainment has followed the same path.

Thus my definition: the Metaverse is the set of experiences that are completely online, and thus defined by their malleability and scalability, which is to say that the Metaverse is already here. Sure, today’s experience is largely denominated by text and 2D, but video is already a major medium, first in the form of entertainment and now a vital tool for work. This is a trajectory that, in my estimation, inexorably leads to virtual reality: if all that matters is digital, why wouldn’t you want the most immersive experience possible?

Crypto’s Role

This also explains why crypto is interesting. Stephen Diehl, in a scathing article entitled Web3 is Bullshit, writes:

At its core web3 is a vapid marketing campaign that attempts to reframe the public’s negative associations of crypto assets into a false narrative about disruption of legacy tech company hegemony. It is a distraction in the pursuit of selling more coins and continuing the gravy train of evading securities regulation. We see this manifest in the circularity in which the crypto and web3 movement talks about itself. It’s not about solving real consumer problems. The only problem to be solved by web3 is how to post-hoc rationalize its own existence.

The first part isn’t entirely unfair; scams and ponzi schemes are everywhere, and it seems clear that we are in the middle of an ever-inflating bubble. It’s also the case that an entire set of legitimate use cases are in reality regulatory arbitrage; crypto advocates are far too quick to ascribe all of the issues with the current monetary system to greed and corruption, without acknowledging that complex systems arise for very good reasons. And, along those same lines, Web3 evangelists often sound like overbearing regulators ascribing the dominance of the biggest tech companies to illegal lock-in, without acknowledging that Aggregators win because they provide the user experience consumers want (and which crypto applications currently sorely lack).

The second part, though, is less compelling to me, even if it is a restatement of the most common crypto criticism: “None of this digital stuff has any real world value.” The real world, of course, is the physical world, and I get the critique; I am very skeptical that crypto currencies are going to replace fiat currencies, or be otherwise useful in the physical world, or that DAOs are going to replace LLCs or corporations for real world companies.

Remember, though, my definition of the Metaverse as a set of experiences that are completely online. It is here that physical world constraints don’t make any sense. I can’t, for example, have a conversation with multiple, distinct groups of people at the same time, yet I do exactly that every day — I even have an entire monitor on my desktop devoted to nothing but chat clients! In this world having one account that represents my whole self, like that offered by Facebook, is a pain in the rear end; for now I have multiple accounts for each individual service, but as I noted, some of them are already based on my unique phone number. How much better if every account were based on a digital identifier unique to me and owned by no one, and now you understand the case for crypto wallets.

This gets to the other mistake Diehl makes in that article, which, ironically, echoes a similar mistake made by many crypto absolutists: there is no reason why the Metaverse, or any web application for that matter, will be built on the blockchain. Why would you use the world’s slowest database when a centralized one is far more scalable and performant? It is not as if WhatsApp or Signal are built on top of the plain old telephone service; they simply leverage the fact that phone numbers are unique and thus suitable as identifiers. This is the type of role blockchains will fill: provide uniqueness and portability where necessary, in a way that makes it possible to not just live your life entirely online, but as many lives simultaneously as your might wish, locked in nowhere.

The Great Bifurcation

I noted above that the physical and digital worlds are bifurcating, and this is happening to tech as well. Yesterday Elon Musk was named TIME’s 2021 Person of the Year, and while he is known for his tweets about Dogecoin and on-and-off-again support of Bitcoin, his biggest contributions to the world — electrical cars and reusable rockets — are very much physical. In fact, you could make the case that Tesla and SpaceX aren’t tech companies at all, but rather another example of tech-first companies set on remaking industries that only ever saw computers as a tool, not the foundation.

The Metaverse, in contrast, is not about eating the world; it’s about creating an entirely new one, from entertainment to community to money to identity. If Elon Musk wants to go to the moon, Mark Zuckerberg wants to create entirely new moons in digital space. This is a place where LLCs make no sense, where regulations are an after-thought, easily circumvented even if they exist. This is a place with no need for traditional money, or traditional art; the native solution is obviously superior. To put it another way, “None of this real world stuff has any digital world value” — the critique goes both ways.

In the end, the most important connection between the Metaverse and the physical world will be you: right now you are in the Metaverse, reading this Article; perhaps you will linger on Twitter or get started with your remote work. And then you’ll stand up from your computer, or take off your headset, eat dinner and tuck in your kids, aware that their bifurcated future will be fundamentally different from your unitary past.

I wrote a follow-up to this Article in this Daily Update.


  1. When it comes to e-commerce, the shift is similar: not only are other large e-commerce providers like Walmart a click away, individual merchants, largely powered by Shopify, are growing even faster than Amazon. 

15 Dec 04:44

Gute Nachrichten! Mozilla hat die Lösung gefunden!Für ...

mkalus shared this story from Fefes Blog.

b'Gute Nachrichten! Mozilla hat die L\xc3\xb6sung gefunden!
F\xc3\xbcr die Zukunft will Mozilla weiter in Firefox, Mozilla VPN, und seinen Lesezeichen-Dienst Pocket investieren, die nach eigenen Angaben von rund 100 Millionen Menschen genutzt werden. Au\xc3\x9ferdem soll "qualitativ hochwertige Werbung" durch von Mozilla als vertrauensw\xc3\xbcrdig eingestufte Partner geboten werden.
Schei\xc3\x9fe, Bernd! Nur hochwertige Werbung von vertrauensw\xc3\xbcrdigen Partnern? Wieso ist denn da sonst noch keiner drauf gekommen!1!!

Firefox scheinen die nur noch als Hebel zu sehen, um ihre anderen Dienste an den Mann zu bringen.'

13 Dec 07:52

Progressives’ Vaccine Mandate Meltdown

by Dave Pollard

Three people have sent me links to Paul Kingsnorth‘s recent two-part diatribe opposing vaccines and vaccine mandates. Two of them said they were persuaded by his argument. Here’s what I replied to them:

“The Vaccine Moment” is a warmed-over op-ed, driven by fear, poorly thought out, and pretty much fact-free. You know, like a Thomas Friedman piece.

I respect your opinion, and Paul Kingsnorth’s, though not without great sadness.

My viewpoints and decisions on CoVid-19 are informed by the science, in particular by the work of epidemiologists, a number of whom I worked with for two years with the Ontario Ministry of Health during the aftermath of the SARS pandemic, preparing for the “next” pandemic, which we are now struggling with.

I have no love of government bureaucracies, and even less for “law enforcement” agencies. But I believe absolutely in the wisdom of getting vaccinated, including getting a booster now, not only for my own protection but for the protection of my fellow citizens.

When Paul Kingsnorth says “It is [a] fact that these vaccines, whatever their efficacy in other areas, do not prevent transmission of the virus”, he is simply wrong, and in saying that he is spreading misinformation. The facts are:

  • Vaccines drastically reduce the likelihood of being infected, and, even when a vaccinated person is infected, they are dramatically less likely to infect others because their body produces much less of the virus.
  • When you are vaccinated, the probability of getting infected (and hence possibly getting chronic Long Covid diseases) is much lower, the probability of being hospitalized even if you do get infected is much lower, and the probability of dying if you are hospitalized is much lower.

On that basis, I have been double vaxxed and boosted, and did so as soon as I was able to and without hesitation or concern. We have a century of experience with vaccines, their development, and their risks. The risk is not zero, but is much lower than the risk of not being vaccinated.

My opinion on that is completely unconnected to my opinions on government oppression, or on what steps should be taken when people refuse to get vaccinated. Those are political issues, and they should not be muddied with the scientific issue of whether vaccination is effective.

I know people of colour, and people who are poor and/or sick, and the way they are treated by government and law enforcement and health agencies is deplorable. They have every justification in distrusting people and systems which have never done a thing for them except to label and harass them. I still encourage them to get vaccinated, because it is the informed and morally appropriate thing to do, not because if they don’t they’re going to end up in some “camp”.

For the same reason, I wear seat belts and I don’t drink and drive, because it is the informed and morally appropriate thing to do, not because I fear being arrested if I fail to comply.

Am I concerned about overreach by the rich and powerful in the exercise of control over citizens? Of course. We have been propagandized and conditioned by the rich and powerful all our lives to act in their interests (go to school, don’t challenge authority, go to work, be a ‘responsible’ adult ie responsible consumer, accept that corporations have the same rights as citizens, pretend you live in an actual democracy, do what you’re told, obey the rules etc). That’s ghastly, and it’s getting worse.

But to conflate this outrageous abuse of power with a belief that vaccines that happen to have been developed and urged by the powerful are evil by association, is just muddle-headed thinking, and it has largely created the horrible mess we’re in right now.

But we believe what we want to believe, and no one can change what we want to believe.

Paul is a brilliant thinker and has done some excellent research and writing on collapse and how we might best prepare for and/or cope with it. But when it comes to epidemiology he knows diddly-squat, and his spreading of misinformation on vaccines is, in a way, an abuse of the power he has acquired by virtue of his recognition as a leading thinker on collapse. He’s not alone — collapsniks from The Automatic Earth to Dmitry Orlov to Charles Eisenstein to Jim Kunstler have gone down the same path, pontificating on epidemiology when they know nothing about it. Doing a little personal research does not make you a credible expert on anything.

That’s my rant. I’m immensely disappointed in Paul’s articles, and in Dmitry’s and Ilargi’s and Charles’ and Jim’s. They are a sign of the madness that accelerating collapse is inflicting on all of us. We are no longer thinking clearly. We have given up on the truth as even something that exists, and are now reacting purely with our emotions, following the same pattern as the Trumpists and the other neo-fascists. It does not bode well for our capacity to deal with the catastrophic crises we are just beginning to witness.

As you probably know, I don’t believe we have free will or control over what we believe or what we do. So while I am disappointed, I am not surprised, and don’t blame anyone for their particular way of acting out their distress over what is happening. We’re all doing our best. What that will lead to, is anyone’s guess.

13 Dec 07:50

Twitter Favorites: [nealjennings] You've heard of elf on a shelf now get ready for... https://t.co/Zgf3QpnzOa

Neal Jennings @nealjennings
You've heard of elf on a shelf now get ready for... pic.twitter.com/Zgf3QpnzOa
13 Dec 07:49

Twitter Favorites: [lisawilliams] Rewatching Battlestar Galactica and eating some takeout Chinese. BSG is still really good but it's hardly SF unle… https://t.co/BFk8OjQ2Ta

Lisa Williams @lisawilliams
Rewatching Battlestar Galactica and eating some takeout Chinese. BSG is still really good but it's hardly SF unle… twitter.com/i/web/status/1…
13 Dec 07:49

Ubuntu 20.04 on a Lenovo Thinkpad E14

by Martin

Over the past couple of weeks, I’ve installed Ubuntu 20.04 on quite a number on new notebooks. This week’s challenge was to find a notebook that meets the following requirements: A 14″ display, a sturdy casing because it’s mostly used mobile, and ultra reliable operation, including Suspend to RAM. Suspend to Idle would not do, as one of the use cases is keep the notebook in the backpack overnight. Especially Suspend to RAM requires good BIOS and ACPI support, so I kept my eyes firmly in the Lenovo notebook camp, as they have a reputation for good Linux support. Also, the price should be below 1000 euros. In the end, I chose a Lenovo E14 for 689 euros with a 10th generation Intel i5-10210U, 16 GB of RAM and a small 256 GB SSD that will be replaced anyway. I’d say that’s a very good price for an E14 model.

Price and Production Date

The main reason the price was quite good was probably that this particular model (Lenovo E14, 20RA001DGE) with a 10th generation i5 processor is already on the market for almost one and a half years and hence, this might have been a clearance sale price. Notebookcheck reviewed this particular model already back in March 2020. So despite being a new notebook, the hardware is already a bit dated. But that was actually one of the reasons I bought this model, as by now, Linux kernel support should be rock solid. Interestingly enough, the production date indicated on a sticker at the bottom was June 2021. The same date is printed on a label on the RAM, and the sticker on the NVMe SSD says that this part was produced in February 2021.

Hardware Compatibility

And indeed, my Ubuntu 20.04 with Linux Kernel 5.11 that I booted from an SSD attached to a USB port properly detected and configured all hardware out of the box. Also, Suspend to RAM and resume worked without a flaw. Even though the price was rather low for an E14, it was equipped with an Intel AX201 802.11ax Wifi card, which I would say is the gold standard when it comes to Wifi and also when it comes to Linux support. E14 models with slightly newer hardware come with a different, and from my point of view, much inferior Wifi card.

Suspend to RAM

Since proper support of Suspend to RAM was a particular requirement, I checked the power consumption in this state. After 6h on standby, the battery level was still at 93%. I also checked power consumption with a charger attached and again saw a power consumption of under one watt. Excellent!

The Compromises

So, what is missing compared to a T14 model? For one thing, it ‘only’ supports USB 3.1 Gen 1, USB speed is limited to 5 GBit/s while the T14 supports USB 3.1 Gen 2, i.e. 10 GBit/s. No problem for most people. Also, there’s no chip card slot, which is used in many companies for authentication. Also not an issue for most people. And then, there’s no support for automatic firmware updates. Lenovo only seems to support that on higher-end models such as the T, X and L series. A bit of a shame, but again, not a problem. The biggest disadvantage of this model for me is that there’s only a single memory slot. In other words, there’s no dual-channel RAM operation, which impacts performance to a certain degree. More on that a bit later. But that’s it from my point of view.

The Pros

Like much more expensive models, charging over USB-C (PD) is supported. These days, that’s a must have to cut down on the number of chargers to carry. An external screen can be connected over the HDMI port or via USB-C. And a special goodie: This E14 model comes with an M.2 slot for an NVMe and a traditional SATA connector for older SSDs. There’s even a dummy HDD/SDD and a caddy installed. This is invaluable, as this allowed me to migrate a Samsung 960 Evo 2 TB SATA SSD. Sure, it’s not as fast as a recent NVMEe SSD for the M.2 slot, but that’s not an issue in this case. Also, this E14 model still has an Ethernet port built-in, so no need to carry an adapter. I see this as a nice bonus as well!

Cooling

One more thing: This E14 model has the cooling fan outlet at the bottom. I find this much better than blowing out hot air on the right, like most Lenovo models do. The downside of blowing out the hot air at the bottom is perhaps when the notebook is used in the lap or put on a blanket. And while we are talking about the fan, I noticed that it’s very quiet and doesn’t run very often. It even remains off when watching a Full-HD video on Youtube.

Performance

To see how things are performance wise, I compared CPU load when watching a full HD Youtube video. On my X13, overall CPU load on the 8 cores and 16 threads is around 15% while on this E14 model with 4 cores and 8 threads, CPU load is around 25-30%. Also, video calls work just fine with a similar CPU load, so there is enough headroom for doing other things simultaneously without a noticeable delay. My ffmpeg transcoding test resulted in an execution time of 9 minutes 21 seconds and a speedup of 5,11x. Compared to the 6 minutes 19 and a speedup of 7,56x on my AMD Ryzen 7 4750U powered X13, that’s 30% slower. Compared to the HP 17″ notebook with an 11th generation i5 processor I reviewed earlier, which took 7 minutes 54 seconds for the tasks and thus has a speedup of 6,05x, the difference is 15%. Apart from an 11th vs. 10th generation Intel i5 processor, which might give the HP notebook an edge over this E14 model, the HP has 2 RAM sockets that are both equipped, which allows for dual-channel RAM operation. As mentioned above, the E14 only has a single RAM socket which causes a 20% performance hit for my ffmpeg task on its own.

Summary

All in all, this E14 model meets the requirements perfectly and I’m glad I went for it, especially at this clearance sale type price.

13 Dec 07:47

servefolder.dev

servefolder.dev

Absurdly clever application of service workers and the file system API: you can select a folder from your computer and the contents of that folder will be served (just to you) from a path on this website - all without uploading any content. The code is on GitHub and offers a useful, succinct introduction to how to use those APIs.

Via AshleyScirra/servefolder.dev

13 Dec 07:47

2021 Series Survey

What with Covid and being semi-retired, I had plenty of reading time in 2021. In particular, I got to the final volumes of multiple long-running literary franchises. (Some of those final volumes had been published years before.) I think that every series listed here is a good investment of your time. Two warnings: It’s mostly pretty mainstream, and kind of sci-fi heavy. Quick table of contents: Broken Earth, The Expanse, Merchant Princes/Empire Games, Neapolitan Novels, The Sandman, Sandman Slim, Trickster Trilogy.

I offer links for each work: Amazon, because it’s convenient and if you follow these it might help sell panties and thus get my family free books, and Alibris, because they seem to be about helping independent booksellers (although their ownership is opaque).

Now, in alphabetical order…

Broken Earth

Premise: On a planet plagued by destructive seismic activity, some people can control earthquakes with their minds; they are feared and persecuted. We follow the adventures of three such women in a time of high crisis.

Each volume of this trilogy won N.K. Jemisin the Hugo award for the year it was published. What’s great about it is the complexity and believability of the characters, and the immersive intensity of the action; once you start reading you can’t stop.

What’s not so great (for me anyhow) was the slightly shaky world-building, my suspension of disbelief wavered every so often. Put any internal knowledge of physics on the back shelf and you’ll have fun reading these.

Best among these at: Taking you to really weird and intense places and leaving you with a strong impression of what it felt and smelled like to be there.

Amazon, Alibris.

The Expanse

Premise: Human civilization has spread across the Solar System and is beset by empire-building and colonialism. On an asteroid, an alien artifact is discovered, with momentous consequences; plenty enough for nine books and six TV seasons. We follow the adventures of a half-dozen or so lead characters and their doughty spacecraft, the Rocinante.

I mean, either you like Space Opera or you don’t. If you do, you’ll really like the work of James S.A. Corey, the pen name of Daniel Abraham and Ty Franck. Also, the TV series (its final season just shipping as I write this) is really very good.

The biggest criticism I can offer is that there’s a little too much talk about what the characters are thinking and feeling, could have been shown not told.

Best among these at: Clever plot construction, space battles.

Amazon, Alibris.

Merchant Princes/Empire Games

Premise: Our Earth has multiple timelines, similar to each other but different in interesting ways; certain people (and eventually, machines) are able to move between them.

There are a whole lot of books here! After writing the first few, author Charles Stross anthologized and reorganized a bit, so you have to do a bit of work figuring out what to buy to move from one end to the other. I’m a little surprised nobody’s taken this to TV yet, there are loads of juicy parts for good actors.

The thing about Stross is that he wants the audience to have a good time, and you get the sense that he’s having fun too. So at no point is there a feeling that anything’s taking itself too seriously. Loads of amusing characters and surprising situations and clever twists of fate.

Best among these at: Being convincingly set in the current real world, plus amusing variations. Also, nuclear explosions. Lots of nuclear explosions!

Merchant Princes Amazon, Alibris.

Empire Games Amazon, Alibris.

Neapolitan Novels

Premise: Two women grow up in a seedy part of Naples, then each makes a lot of bad choices as they move through their twentieth-century Italian/European lives.

Author Elena Ferrante is pseudonymous, although there has been a journalistic exposé which convincingly claims to have identified her.

Now, a confession: I haven’t actually finished the final book in the quartet, The Lost Child, because I got so mad at one of the characters (did I mention bad choices?) that I would have thrown the book across the room except for it was a Kindle. I actually think this is a symptom of it being a really good book. And I will go back and finish it.

The quality of the writing (and necessarily the translation) just can’t be praised too highly. This is beautiful, beautiful, stuff. The texture of life is examined and unwoven with a precision and intensity that I have never encountered anywhere else. Every page of the narrative is totally convincing: Yes, these people were in these situations and did these things and felt those things. The situations, growing as they do out of working-class bad-neighborhood society, are as exotic to most educated readers as any outer-space battle; except for being real.

There’s a minor geek-interest side-plot: One of the protagonists works her way out poverty via the IBM System/3, an important 1970’s piece of computer history.

Best among these at: Raw emotional intensity without loss of realism.

Amazon, Alibris.

The Sandman

Premise: Morpheus, or Dream, is one of the seven Endless, along with Destiny, Death, Desire, Despair, Delirium (formerly Delight), and Destruction (also known as ‘The Prodigal’). Dream is a complex character, not terribly admirable. Any attempt to say more is going either to be inadequate or require ten thousand words to get started up.

These graphic novels, written by Neil Gaiman and illustrated by a legion of wonderful artists, were published between 1989 and 1996. I’m really not big on graphic novels but I read Season of Mists on paper once many years ago; at the time, it didn’t change my mind about comic books. After buying a Samsung S7+ with a large and fabulous screen, on impulse I bought the whole 12-volume omnibus.

I just can’t say enough good things about the Kindle presentation of The Sandman on a big bright tablet; it was maybe the most intense reading experience I’ve had in years. It took me three months to get through this, and while it’s long, one reason was that sometimes I just needed a rest.

Among other things, the Kindle app helps deal with fanciful/explosive full-page compositions by gliding through the dialog bubbles one at a time in the right order. Not being a native comic-book citizen, I found this terrifically helpful.

Anyhow, the writing is fabulous, distinguished by its high velocity: You move from one frame to the next and realize that a whole lot of stuff must have happened and Gaiman hasn’t bothered to fill in the details because there’s enough there to deduce them. I admire this; and anyhow, the books are plenty long enough.

One small criticism of the big omnibus edition is the self-indulgent front and back matter for each volume, with fulsome praise from a panoply of famous names. No value is added and I just paged through them after the first couple.

Um, trigger warnings. I mean, all the trigger warnings, there is some extremely dark, brutal, awful stuff in here; I also took a reading break one week when I was having a tough time and just didn’t need the blackness.

I wonder if I now like graphic novels? Or perhaps this is entirely sui generis.

Best among these at: Making utterly strange and phantasmagorical scenarios believable. Delighting the eyes.

Amazon, Alibris.

Sandman Slim

Premise: James Stark returns to Earth after a decade in Hell. He is a monster who kills monsters. And, it turns out, something of a sensitive new-age guy when he’s not. Love, romance, and extreme violence.

This is author Richard Kadrey’s only real hit. The twelfth novel, which just came out, is advertised as the last, so we’ll see what he can do next. This series, to be honest, is perhaps less ambitious than some of the others here but it’s fun and you can’t spend all your time reading Ferrante, can you?

Stark is a really fine piece of character creation, and not the only one in this series, there are loads of interesting people and supernatural creatures, well-distributed across gender and other demographic divisions. Also, Kadrey has a major talent for dialogue; I’m not saying Elmore Leonard, but it’s not crazy to mention them in the same sentence.

And there’s action, lots of extremely violent action, in which many terribly undesirable creatures come to bad, messy ends, and then also other players of whom the reader will have grown fond. But in these books, you can after all come back from Hell and various other afterlife scenarios.

The world-building is necessarily somewhat theological and frankly ridiculous, but once you stop taking it seriously that’s not a problem.

For supernatural thrillers, these are nicely grounded, specifically in Los Angeles and its reflection in Hell. Just a taste: Stark is having a conversation with Lucifer (yes, that Lucifer) walking around in LA, and they stop for huge sloppy greasy burritos from a food truck. Lucifer is wearing a beautiful white linen suit and manages to eat the whole burrito with absolutely no food stains. Now that’s devilish.

Best among these at: Snappy dialogue.

Amazon, Alibris.

The Trickster

Jared is an indigenous high-school student in Kitimat, BC who’s got problems with family and addiction and school, despite his successful weed-cookie business. His mother is a real piece of his work and his father… well, there’s the story, his father is probably a Trickster, a powerful and malevolent supernatural creature with so many children they have to self-organize on Facebook. Jared eventually ends up in Vancouver at community college, making donuts for money and pursuing a struggle with unnatural enemies that builds to an apocalyptic climax.

Author Eden Robinson is, like Jared, from Kitimat and indigenous. She claims she was trying to write a “trashy reservation romance” but it got out of control and became three pretty big thick novels full of teen angst and magical violence.

There was a CBC-TV series that managed to deliver a lot of the Trickster flavor while wandering way away from the books’ plot-lines. I enjoyed it but it has apparently fallen apart after just one season partly because the director’s claims of indigenous heritage came into question.

This delivers you-are-there flavor for a lot of places that nobody will ever go and is bulging with lovable personalities. Most people are kind and decent, some are monstrous, and Ms Robinson has fun with both flavors. An entirely entertaining read end-to-end, often hysterically coffee-snorting funny. And also with plenty to teach about our indigenous people and the realities they face.

At this point, it may be a little surprising when I say that I enjoyed this series more than any of the others here. Granted, it’s possibly because it takes place partly in my home-town. But these are outstanding books.

Best among these at: Taking you to interesting real places that you’ve never been, and imaginary ones too. Making you laugh so hard the other people in the room will want to know if you’re OK.

Amazon, Alibris.

13 Dec 07:46

Why Are We Drawn to Stories That Serve Us Badly?

by Dave Pollard

The cognitive bias codex from wikipedia; if you want to print it out so it’s legible and useful, print the original over four letter-sized pages and paste them together (my printout is taped, tellingly perhaps, over my rarely-used TV). The model was developed by John Manoogian III and refined by Buster Benson; the online version includes links by TillmanR to the wikipedia articles explaining each bias.

Humans learn in a variety of ways: We learn by watching others, and seeing what happens. We learn from demonstration, and explanation. We learn from practice — trial and error, learning from our mistakes.

But one of the ‘stickiest’ ways we learn — by which I mean one of the ways our learning stays with us over time, even in the face of evidence that undermines what we think we’ve learned — is by listening to stories.

We are highly vulnerable to the messages of stories because their point of view, and their bias, is rarely right in our face. When we listen to a story and ‘relate’ to it, we imagine we are free to draw our own conclusions. To this extent, stories are subversive.

But worse than that, stories can be and often are manipulative. A story is often a sales pitch — hence our attraction to testimonials and ‘independent’ ratings of products and services. We are already predisposed to buy, and are looking for confirmation. If we read a bad review, that disposition can quickly slew 180º, as we look for confirmation of that story instead.

This manipulation employs, sometimes not even consciously, a whole bag of tricks that exploit our vulnerability to many cognitive biases. These biases are baked into our human DNA and our artifacts and processes, including our languages. Many of these biases seem to enhance the survival of our species, by simplifying learning and decision-making, making us more comfortable with what we’re facing, and drawing us together (since we can’t survive alone). Our brains are interested in what can help us do this, and the ‘truth’ — what’s ‘real’ and what’s really happening — seems to be of secondary importance.

And these biases show up, in spades, in our stories. Read through the explanations of these biases and it’s easy to find obvious examples where stories have exploited each of them.

At a retreat a few years ago, put on by Dark Mountain founders Dougald Hine and Paul Kingsnorth (yes, the irony), one of the things we worked on was a harvest of modern myths, some of them directly contradicting others. Note that myths aren’t necessarily wrong — a myth is any story believed collectively by a significant number or group of people. It’s not hard to understand why people hold to these myths, for the three reasons cited above (ease of action, comfort/reassurance, and social cohesion). We want to believe the stories of our peeps.

I largely gave up watching TV and going to films (and reading a lot of fiction and non-fictional “analysis” that was poorly supported) a few years ago. I just couldn’t get past the whoppers we were being asked to swallow.

The most popular movies always seem to be about (fictional or historic) heroic individuals who overcome ridiculous obstacles and horrific, ruthless enemies and prevail as models for us all. Or they’re about horror movie baddies who cannot be killed. Or they’re dramas and “suspense” movies about suffering — to make us feel better about our own lot by comparison? Or they’re comedies that belittle others, so that presumably we feel better about ourselves. Or they’re documentaries with magical, simplistic “we all just need to…” prescriptions at the end. Awful, manipulative stories, all of them.

And they all seem to have one-dimensional (slightly flawed hero/wronged villain) characters and a standard myth plot about inevitable struggle and inevitable progress. Why do we keep falling for these fictions? Why don’t we like movies and novels that present the world as it really is, in all its complexity and uncertainty and directionlessness, and which help us learn about the world and history and human nature, and imagine possibilities?

Why are we drawn to stories that serve us badly?

I don’t know the answer to this question, but I have some thoughts about
“wrong answers” to it. I don’t think it’s because humans are stupid. And while I think we have been (for understandable reasons) dumbed down and deprived of the opportunity to practice critical thinking and other skills and capacities, and to acquire relevant knowledge of history and science, I don’t think it’s this lack of skill, knowledge and practice that underlies our attraction to stories that serve us badly.

And although we’ve been brainwashed, propagandized and manipulated into believing falsehoods and half-truths because that serves the vested interests of the rich and powerful, I don’t even think it’s because of this that we’re attracted to, and buy, stories that serve us badly.

Most of my life I was attracted to such stories. My blog is full of utter nonsense that I once believed passionately, especially in the early years when the blog was almost embarrassingly popular. Why was I attracted to such stories as: 9/11 being “conceivably” an inside job, or the “dangerous” use of squalene in vaccines for the military simply because it was cheaper to use GIs as guinea pigs? Or business “management theories” based on “case studies” that were just fanciful inventions that had no more rigour or credibility than a slick PR piece?

Why did I want to believe these things were actually or probably true? Because they were novel and had a “ring” of truth? Because they flew in the face of “conventional wisdom” that I suspected was wrong, or boring? Because I fancied myself as counter-culture, or as an ersatz investigative journalist exposing the important truth of the lies of the rich and powerful?

I have no idea. It’s very possible that some of the things I believe fervently and write about now are dead wrong, or half-truths, and that by writing about them and believing them, I’m misleading myself and doing a disservice to my readers (the few who remain despite my changing my mind so dramatically on so many subjects).

Maybe our propensity to love and believe stories regardless of their truth or value, is just an evolved trait of the human animal that seems to have survived, even though it’s now a maladaptation that serves us badly, like our appendices or our fondness for sugars and salt.

All I know is that we humans seem drawn to, and even addicted to, our stories, and the beliefs they instil and reinforce in us, even when those stories and beliefs lead us astray. Much of our modern “civilized” world would probably not exist if we didn’t buy into all the stories that enable it to continue, that keep us doing what we do, even when it makes no sense.

Without our stories, where would we be?

13 Dec 07:44

In Berlin sucht man einen verlorenen Koffer mit Kassetten der DDR-Punkband die Firma

by Ronny
mkalus shared this story from Das Kraftfuttermischwerk.

b'\n

Schon im Juli ergab sich in Berlin diese kuriose und gleichsam traurige Situation, in der ein Mann einen Alukoffer in einer Tram stehen lie\xc3\x9f. Der Inhalt: nicht wenige Audiokassetten mit Aufnahmen der DDR-Punkband die Firma, die eigentlich allesamt digitalisiert werden sollten, was ohne die Kassetten nun nat\xc3\xbcrlich unm\xc3\xb6glich ist. Jetzt sucht man in Berlin nach diesem Koffer und vielleicht kennt jemand von euch irgendwen, der jemanden kennt und was geh\xc3\xb6rt haben k\xc3\xb6nnte. W\xc3\xa4re schon sch\xc3\xb6n, wenn die Dinger wieder auftauchen w\xc3\xbcrden.

\n

Berlin sucht einen Koffer mit Tonbandkassetten. \xc3\x9cber Facebook hat die Modedesignerin und Journalistin Jane Garber am Dienstag einen Aufruf gestartet. Sie war einst mit dem 2015 gestorbenen Frank Tr\xc3\xb6ger, genannt Tr\xc3\xb6tsch, dem S\xc3\xa4nger der Punkband Die Firma, zusammen. Offenbar hat sie auch einen Koffer mit Kassetten geerbt und wollte sich in diesem Sommer um die Hinterlassenschaften ihres Ex k\xc3\xbcmmern und die Tontr\xc3\xa4ger digitalisieren lassen.

'
13 Dec 07:44

Vancouver police identify Kitsilano park caretaker as homicide victim

mkalus shared this story :
I actually walked by there yesterday and wondered what had happen. Sad to hear. Related: Did you know that Vancouver Parks had live in caretakers? They ended this a few years ago. Only ones left are the people who already had the job. Imagine living in Stanley Park.

b'

Police in Vancouver are investigating the death of a\xc2\xa0city park caretaker as a homicide.

Officers\xc2\xa0were called to a home in Kitsilano near\xc2\xa0West 3rd Avenue and Macdonald Street on Friday around 11 a.m. PT where they found the body of 77-year-old Justis Daniel.

On Sunday, they confirmed that Daniel was the long-term caretaker of Tatlow Park.

The 1.4 hectare park has a field house, playground, tennis courts, a washroom and an onsite caretaker, according to the City of Vancouver.\xc2\xa0The caretaker residence in the park is the focus of the police investigation.\xc2\xa0

Police say the death is the 17th homicide of 2021, but so far have announced few details about\xc2\xa0why or how Daniel was killed.

"No arrests have been made and the investigation is ongoing," the VPD\xc2\xa0said in a release.

The Vancouver Board of Parks and Recreation said in a statement Sunday it is saddened that\xc2\xa0Daniel\' "life\xc2\xa0was cut short in such a senseless way.

"As a long-time caretaker at Tatlow Park, Mr. Daniel contributed to the enjoyability of the park each and every day," said the statement. "Our sincerest condolences are with his family and friends, as well as the park board staff and members of the community who came to know him."

Nearby residents also reacted with shock about the death.\xc2\xa0They say Daniel was well-known, had been at the park for decades and cared about his community.

WATCH | A\xc2\xa0Kitsilano resident reacts to the death of Justis Daniel:

\'It\xe2\x80\x99s a nice neighbourhood and he\xe2\x80\x99s just a nice old guy\'

A resident in Vancouver\'s Kitsilano neighbourhood reacts to the suspicious death of a park caretaker. 0:47

Investigators are looking for anyone with dashcam video or witnesses who\xc2\xa0were in the area of the park\xc2\xa0between 7 a.m. Thursday and 11 a.m. Friday.

Anyone with information is asked to call the VPD\'s Major Crime Section at 604-717-2500.

'
13 Dec 07:44

Frickin' Shaders With Frickin' Laser Beams

Cover Image

Hassle free GLSL

I've been working on a new library to compose GLSL shaders. This is part of a side project to come up with a composable and incremental way of driving WebGPU and GPUs in general.

#pragma import { getColor } from 'path/to/color'

void main() {
  gl_FragColor = getColor();
}

The problem seems banal: linking together code in a pretty simple language. In theory this is a textbook computer science problem: parse the code, link the symbols, synthesize new program, done. But in practice it's very different. Explaining why feels itself like an undertaking.

From the inside, GPU programming can seem perfectly sensible. But from the outside, it's impenetrable and ridiculously arcane. It's so bad I made fun of it.

This might seem odd, given the existence of tools like ShaderToy: clearly GPUs are programmable, and there are several shader languages to choose from. Why is this not enough?

Well in fact, being able to render text on a GPU is still enough of a feat that someone has literally made a career out of it. There's a data point.

Another data point is that for almost every major engine out there, adopting it is virtually indistinguishable from forking it. That is to say, if you wish to make all but the most minor changes, you are either stuck at one version, or you have to continuously port your changes to keep up. There is very little shared cross-engine abstraction, even as the underlying native APIs remain stable over years.

When these points are raised, the usual responses are highly technical. GPUs aren't stack machines for instance, so there is no real recursion. This limits what you can do. There are also legacy reasons for certain features. Sometimes, performance and parallelism demands that some things cannot be exposed to software. But I think that's missing the forest for the trees. There's something else going on entirely. Much easier to fix.

a puzzle

Just Out of Reach

Let's take a trivial shader:

vec4 getColor(vec2 xy) {
  return vec4(xy, 0.0, 1.0);
}

void main() {
  vec2 xy = gl_FragIndex * vec2(0.001, 0.001);
  gl_FragColor = getColor(xy);
}

This produces an XY color gradient.

In shaders, the main function doesn't return anything. The input and output are implicit, via global gl_… registers.

Conceptually a shader is just a function that runs for every item in a list (i.e. vertex or pixel), like so:

// On the GPU
for (let i = 0; i < n; ++i) {
  // Run shader for every (i) and store result
  result[i] = shader(i);
}

But the for loop is not in the shader, it's in the hardware, just out of reach. This shouldn't be a problem because it's such simple code: that's the entire idea of a shader, that it's a parallel map().

If you want to pass data into a shader, the specific method depends on the access pattern. If the value is constant for the entire loop, it's a uniform. If the value is mapped 1-to-1 to list elements, it's an attribute.

In GLSL:

// Constant
layout (set = 0, binding = 0) uniform UniformType {
  vec4 color;
  float size;
} UniformName;
// 1-to-1
layout(location = 0) in vec4 color;
layout(location = 1) in float size;

Uniforms and attributes have different syntax, and each has its own position system that requires assigning numeric indices. The syntax for attributes is also how you pass data between two connected shader stages.

But all this really comes down to is whether you're passing color or colors[i] to the shader in the implicit for loop:

for (let i = 0; i < n; ++i) {
  // Run shader for every (i) and store result (uniforms)
  result[i] = shader(i, color, size);
}
for (let i = 0; i < n; ++i) {
  // Run shader for every (i) and store result (attributes)
  result[i] = shader(i, colors[i], sizes[i]);
}

If you want the shader to be able to access all colors and sizes at once, then this can be done via a buffer:

layout (std430, set = 0, binding = 0) readonly buffer ColorBufferType {
  vec4 colors[];
} ColorBuffer;

layout (std430, set = 0, binding = 1) readonly buffer SizeBufferType {
  vec4 sizes[];
} SizeBuffer;

You can only have one variable length array per buffer, so here it has to be two buffers and two bindings. Unlike the single uniform block earlier. Otherwise you have to hardcode a MAX_NUMBER_OF_ELEMENTS of some kind.

Attributes and uniforms actually have subtly different type systems for the values, differing just enough to be annoying. The choice of uniform, attribute or buffer also requires 100% different code on the CPU side, both to set it all up, and to use it for a particular call. Their buffers are of a different type, you use them with a different method, and there are different constraints on size and alignment.

Only, it gets worse. Like CPU registers, bindings are a precious commodity on a GPU. But unlike CPU registers, typical tools do not help you whatsover in managing or hiding this. You will be numbering your bind groups all by yourself. Even more, if you have both a vertex and fragment shader, which is extremely normal, then you must produce a single list of bindings for both, across the two different programs.

And even then the above is all an oversimplification.

It's actually pretty crazy. If you want to make a shader of some type (A, B, C, D) => E, then you need to handroll a unique, bespoke definition for each particular A, B, C and D, factoring in a neighboring function that might run. This is based mainly on the access pattern for the underlying data: constant, element-wise or random, which forcibly determines all sorts of other unrelated things.

No other programming environment I know of makes it this difficult to call a plain old function: you have to manually triage and pre-approve the arguments on both the inside and outside, ahead of time. We normally just automate this on both ends, either compile or run-time.

It helps to understand why bindings exist. The idea is that most programs will simply set up a fixed set of calls ahead of time that they need to make, sharing much of their data. If you group them by kind, that means you can execute them in batches without needing to rebind most of the arguments. This is supposed to be highly efficient.

Though in practice, shader permutations do in fact reach high counts, and the original assumption is actually pretty flawed. Even a modicum of ability to modularize the complexity would work wonders here.

The shader from before could just be written to end in a pure function which is exported:

// ...
#pragma export
vec4 main(vec2 xy) {
  return getColor(xy * vec2(0.001, 0.001));
}

Using plain old functions and return values is not only simpler, but also lets you compose this module. This main can be called from somewhere else. It can be used by a new function vec2 => vec4 that you could substitute for it.

The crucial insight is that the rigid bureaucracy of shader bindings is just a very complicated calling convention for a function. It overcomplicates even the most basic programs, and throws composability out with the bathwater. The fact that there is a special set of globals for input/output, with a special way to specify 1-to-1 attributes, was a design mistake in the shader language.

It's not actually necessary to group the contents of a shader with the rules about how to apply that shader. You don't want to write shader code that strictly limits how it can be called. You want anyone to be able to call it any way they might possibly like.

So let's fix it.

Reinvent The Wheel

There is a perfectly fine solution for this already.

If you have a function, i.e. a shader, and some data, i.e. arguments, and you want to represent both together in a program... then you make a closure. This is just the same function with some of its variables bound to storage.

For each of the bindings above (uniform, attribute, buffer), we can define a function getColor that accesses it:

vec4 getColor(int index) {
  // uniform - constant
  return UniformName.color;
}
vec4 getColor(int index) {
  // attribute - 1 to 1
  return color;
}
vec4 getColor(int index) {
  // buffer - random access
  return ColorBuffer.color[index];
}

Any other shader can define this as a function prototype without a body, e.g.:

vec4 getColor(int index);

You can then link both together. This is super easy when functions just have inputs and outputs. The syntax is trivial.

If it seems like I am stating the obvious here, I can tell you, I've seen a lot of shader code in the wild and virtually nobody takes this route.

The API of such a linker could be:

link : (module: string, links: Record<string, string>) => string

Given some main shader code, and some named snippets of code, link them together into new code. This generates exactly the right shader to access exactly the right data, without much fuss.

But this isn't a closure, because this still just makes a code string. It doesn't actually include the data itself.

To do that, we need some kind of type T that represents shader modules at run-time. Then you can define a bind operation that accepts and returns the module type T:

bind : (module: T, links: Record<string, T>) => T

This lets you e.g. express something like:

let dataSource: T = makeSource(buffer);
let boundShader: T = bind(shader, {getColor: dataSource});

Here buffer is a GPU buffer, and dataSource is a virtual shader module, created ad-hoc and bound to that buffer. This can be made to work for any type of data source. When the bound shader is linked, it can produce the final manifest of all bindings inside, which can be used to set up and make the call.

That's a lot of handwaving, but believe me, the actual details are incredibly dull. Point is this:

If you get this to work end-to-end, you effectively get shader closures as first-class values in your program. You also end up with the calling convention that shaders probably should have had: the 1-to-1 and 1-to-N nature of data is expressed seamlessly through the normal types of the language you're in: is it an array or not? is it a buffer? Okay, thanks.

In practice you can also deal with array-of-struct to struct-of-arrays transformations of source data, or apply mathbox-like number emitters. Either way, somebody fills a source buffer, and tells a shader closure to read from it. That's it. That's the trick.

Shader closures can even represent things like materials too. Either as getters for properties, or as bound filters that directly work on values. It's just code + data, which can be run on a GPU.

When you combine this with a .glsl module system, and a loader that lets you import .glsl symbols directly into your CPU code, the effect is quite magical. Suddenly the gap between CPU and GPU feels like a tiny crack instead of the canyon it actually is. The problem was always just getting at your own data, which was not actually supposed to be your job. It was supposed to tag along.

Here is for example how I actually bind position, color, size, mask and texture to a simple quad shader, to turn it into an anti-aliased SDF point renderer:

import { getQuadVertex } from '@use-gpu/glsl/instance/vertex/quad.glsl';
import { getMaskedFragment } from '@use-gpu/glsl/mask/masked.glsl';
  
const vertexBindings = makeShaderBindings(VERTEX_BINDINGS, [
  props.positions ?? props.position ?? props.getPosition,
  props.colors ?? props.color ?? props.getColor,
  props.sizes ?? props.size ?? props.getSize,
]);

const fragmentBindings = makeShaderBindings(FRAGMENT_BINDINGS, [
  (mode !== RenderPassMode.Debug) ? props.getMask : null,
  props.getTexture,
]);

const getVertex = bindBundle(
  getQuadVertex,
  bindingsToLinks(vertexBindings)
);
const getFragment = bindBundle(
  getMaskedFragment,
  bindingsToLinks(fragmentBindings)
);

getVertex and getFragment are two new shader closures that I can then link to a general purpose main() stub.

I do not need to care one iota about the difference between passing a buffer, a constant, or a whole 'nother chunk of shader, for any of my attributes. The props only have different names so it can typecheck. The API just composes, and will even fill in default values for nulls, just like it should.

a puzzle

GP(GP(GP(GPU)))

What's neat is that you can make access patterns themselves a first-class value, which you can compose.

Consider the shader:

T getValue(int index);
int getIndex(int index);

T getIndexedValue(int i) {
  int index = getIndex(i);
  return getValue(index);
}

This represents using an index buffer to read from a value buffer. This is something normally done by the hardware's vertex pipeline. But you can just express it as a shader module.

When you bind it to two data sources getValue and getIndex, you get a closure int => T that works as a new data source.

You can use similar patterns to construct virtual geometry generators, which start from one vertexIndex and produce complex output. No vertex buffers needed. This also lets you do recursive tricks, like using a line shader to make a wireframe of the geometry produced by your line shader. All with vanilla GLSL.

By composing higher-order shader functions, it actually becomes trivial to emulate all sorts of native GPU behavior yourself, without much boilerplate at all. Giving shaders a dead-end main function was simply a mistake. Everything done to work around that since has made it worse. void main() is just where currently one decent type system ends and an awful one begins, nothing more.

In fact, it is tempting to just put all your data into a few giant buffers, and use pointers into that. This already exists and is called "bindless rendering". But this doesn't remove all the boilerplate, it just simplifies it. Now instead of an assortment of native bindings, you mainly use them to pass around ints to buffers or images, and layer your own structs on top somehow.

This is a textbook case of the inner platform effect: when faced with an incomplete or limited API, eventually you will build a copy of it on top, which is more capable. This means the official API is so unproductive that adopting it actually has a negative effect. It would probably be a good idea to redesign it.

In my case, I want to construct and call any shader I want at run-time. Arbitrary composition is the entire point. This implies that when I want to go make a GPU call, I need to generate and link a new program, based on the specific types and access patterns of values being passed in. These may come from other shader closures, generated by remote parts of my app. I need to make sure that any subsequent draws that use that shader have the correct bindings ready to go, with all associated data loaded. Which may itself change. I would like all this to be declarative and reactive.

If you're a graphics dev, this is likely a horrible proposition. Each engine is its own unique snowflake, but they tend to have one thing in common: the only reason that the CPU side and the GPU side are in agreement is because someone explicitly spent lots of time making it so.

This is why getting past drawing a black screen is a rite of passage for GPU devs. It means you finally matched up all the places you needed to repeat yourself in your code, and kept it all working long enough to fix all the other bugs.

The idea of changing a bunch of those places simultaneously, especially at run-time, without missing a spot, is not enticing to most I bet. This is also why many games still require you to go back to the main screen to change certain settings. Only a clean restart is safe.

So let's work with that. If only a clean restart is safe, then the program should always behave exactly as if it had been restarted from scratch. As far as I know, nobody has been crazy enough to try and do all their graphics that way. But you can.

One way of doing that is with a memoized effect system. Mine is somewhere halfway between discount ZIO and discount React. The "effect" part ensures predictable execution, while the "memo" part ensures no redundant re-execution. It takes a while to figure out how to organize a basic WebGPU/Vulkan-like pipeline this way, but you basically just stare at the data dependencies for a very long time and keep untangling. It's just plain old code.

The main result is that changes are tracked only as granularly as needed. It becomes easy to ensure that even when a shader needs to be recompiled, you are still only recompiling 1 shader. You are not throwing away all other associated resources, state or caches, and the app does not need to do much work to integrate the new shader into subsequent calls immediately. That is, if you switch a binding to another of the same type, you can keep using the same shader.

The key thing is that I don't intend to make thousands of draw calls this way either. I just want to make a couple dozen of exactly the draw calls I need, preferably today, not next week. It's a radically different use case from what game engines need, which is what the current industry APIs are really mostly tailored for.

The best part is that the memoization is in no way limited to shaders. In fact, in this architecture, it always knows when it doesn't need to re-render, when nothing could have changed. Code doesn't actually run if that's the case. This is illustrated above by only having the points move around if the camera changes. For interactive graphics outside of games, this is actually a killer feature, yet it's something that's usually solved entirely ad-hoc.

One unanticipated side-effect is that when you add an inspector tool to a memoized effect system, you also get an inspector for every piece of significant state in your entire app.

On the spectrum of retained vs immediate mode, this perfectly hits that React-like sweet spot where it feels like immediate mode 90% of the time, even if it is retaining a lot behind the scenes. I highly recommend it, and it's not even finished yet.

* * *

A while ago I said something about "React VR except with Lisp instead of tears when you look inside". This is starting to feel a lot like that.

In the code, it looks absolutely nothing like any OO-style library I've seen for doing the same, which is a very good sign. It looks sort of similar, except it's as if you removed all code except the constructors from every class, and somehow, everything still keeps on working. It contains a fraction of the bookkeeping, and instead has a bunch of dependencies attached to hooks. There is not a single isDirty flag anywhere, and it's all driven by plain old functions, either Typescript or GLSL.

The effect system allows the run-time to do all the necessary orchestration, while leaving the specifics up to "user space". This does involve version counters on the inside, but only as part of automated change detection. The difference with a dirty flag might seem like splitting hairs, but consider this: you can write a linter for a hook missing a dependency, but you can't write a linter for code missing a dirty flag somewhere. I know which one I want.

Right now this is still just a mediocre rendering demo. But from another perspective, this is a pretty insane simplification. In a handful of reactive components, you can get a proof-of-concept for something like Deck.GL or MapBox, in a fraction of the code it takes those frameworks. Without a bulky library in between that shields you from the actual goodies.

12 Dec 03:39

Flickr Release Notes, November 2021

by Carol Benovic-Bradley

Hello, Flickr members! Here’s a summary of what our team worked on in November 2021.

What’s changed

  • We removed the Contact Notifications tab from FlickrMail. This will improve loading of FlickrMail. You can still receive these notifications under the bell icon and via email, if you have that turned on in your Notification Settings.
  • Google Pay is now available as a payment option on desktop and mobile web in Safari.
  • We’ve launched more improvements to the Notifications Center and Settings based on your feedback, including:
  • When you hover over a group or profile picture, let’s say while you’re in the Notifications Center or your activity feed, a card displaying more information about that person or group will consistently appear.
Hovering over a group or profile picture now consistently displays more information

Bug fixes

  • In November, members reported that batch notifications for faves (e.g. “Carol and 83 others have faved your photo”) were broken. This has since been fixed. Fave away! :)

If you have any questions about these updates or about using Flickr, please write to our Support Heroes team. If you’re a Pro member, you’ll hear back from us within a couple of hours.

Wishing you all a happy and safe new year. See you on Flickr!

12 Dec 03:38

Yesterday I presented to the European working g...

by Ton Zijlstra

Yesterday I presented to the European working group of the IHO, the international hydrographic organisation. The European working group consists of the national hydrographic services, which are regularly part of the navy of the respective countries.

The online meeting was hosted and chaired by the French Service hydrographique et océanographique de la Marine (SHOM), operating out of Brest. I was pleasantly surprised that this French government organisation used the open source tool Big Blue Button for the video conference.

Kudos for using open source in the public sector!

12 Dec 03:38

My best holiday shopping tip? Mozilla

by Rebecca Smith

It’s that time of year again — when all of the mail carriers have overflowing trucks, malls are miraculously busy and budgets are tight. Yes, it is holiday shopping time. This year more than 84% of Americans plan to buy holiday gifts with estimates that Americans will spend at least as much on gifts as last year — $789 billion on people’s present purchases alone.  And as much as we love our family and friends, buying gifts for them can be just stressful. While Mozilla can’t make your impossible-to-shop-for dad any easier to shop for or fix the supply chain issues, we can help make the process of holiday shopping more enjoyable. 

Hey, with your holiday shopping a bit easier, maybe you will have some extra time to gift yourself exactly what you want this holiday season.

1. Jumping back into holiday shopping with Firefox mobile

Just because we all of a sudden have to purchase gifts for what feels like everyone we’ve ever known, that doesn’t mean that we suddenly have more time in our day. So here we are, searching the internet for gifts while inline for our morning coffee, during those 5-minute breaks between meetings, or right before we fall asleep at night. Firefox understands that, and our app homepage lets you jump back to open tabs as well as sites you’ve pinned and bookmarked. 

With holiday shopping you may want to pin things like:

  • The page comparing features of the latest phones you want to buy your family.
  • The complete Strategist holiday gift guide you feel will solve your gift-giving woes. 
  • The webpage your significant other not so subtly sent your way. 

How to pin a site: Tap the Settings menu (the three dots in the URL bar), then tap Add to top sites (Android) or Add to Shortcuts (iOS). To remove, rename or open a pin in a private tab, long-press to activate those options. 

2. Checking all gifts with our *Privacy Not Included holiday guide 

It is no secret that tech gifts are popular for the holidays — the U.S. alone is expected to spend $135 billion on hardware and tech services this season. The latest update of the *Privacy Not Included holiday buyer’s guide from the Mozilla Foundation includes research on 151 popular products. This allows you to make sure that the gift you are thinking about is not risking the privacy of your loved ones.

Or if you don’t know what tech gift is right yet, check out the Best Of category with the 22 products you can feel good about gifting. 

3. Organizing all of your gift ideas in Pocket

If you are choosing between multiple gifts, or are not sure that you’ve found exactly the right gift for your partner yet, you can keep all of your ideas organized in your Pocket app. Anytime you come across a gift you think your partner may like, save it to Pocket, and add a tag to that link. You customize the tag with the name of your recipient or “gift ideas”. Whatever keeps you organized and going!

Pro Tip: Save things you think your family and friends will like all year long and reduce the need for frantic searches just a couple of weeks before. Just search for the right tag in your Pocket app and all of your ideas will be in one place.

How to add a Pocket tag from Firefox: Hit the “Save to Pocket” button in the top right corner of Firefox, then tag your page and Save it. Later, when you open your Pocket app, your Saves will be there, tagged and organized.

4. Avoiding shopping spam with Firefox Relay 

The average American will spend $938 on holiday gifts this year (yes, you read that right), so any chance to save money is welcome. All of a sudden signing up for those annoying newsletters from every website ever sounds a bit more appealing if it will save you 10%-20% off everything. Feel free to sign up for all of the newsletters without compromising your personal email or spamming your main inbox with emails from e-commerce retailers you only shop at once or twice a year.

Firefox Relay is a free service available at relay.firefox.com where you’ll get five email aliases to use whenever you sign-up for an online account. If you need more than five email aliases you can sign up for Firefox Relay Premium. Relay Premium subscribers will receive one subdomain alias to create unlimited email aliases, the option to use your email aliases to reply to emails directly, and convenient customer support.

How it works:

5. Feeling good about using an indie company that puts people before profit as you shop

Small businesses depend on the holidays to hit their revenue goals, so while you shop indie this holiday season, you should do so with the indie browser. When you download Firefox, you’re choosing to support an independent tech company. Firefox is the only major browser backed by a non-profit fighting to give you more openness, transparency and control of your life online.

The post My best holiday shopping tip? Mozilla appeared first on The Mozilla Blog.

12 Dec 03:21

Introducing stack graphs

Introducing stack graphs

GitHub launched "precise code navigation" for Python today - the first language to get support for this feature. Click on any Python symbol in GitHub's code browsing views and a box will show you exactly where that symbol was defined - all based on static analysis by a custom parser written in Rust as opposed to executing any Python code directly. The underlying computer science uses a technique called stack graphs, based on scope graphs research from Eelco Visser’s research group at TU Delft.

Via Precise code navigation for Python, and code navigation in pull requests

12 Dec 03:21

What’s ahead for 2022

by Andrew Stadlen
The Calmers (Title Shot)

Hey Flickr!

Thank you for another great year of photo sharing. In a time when it’s perhaps harder than ever to stay connected, all of us at Flickr are grateful for what you bring to the community. Before 2022 begins, we would like to recap what we accomplished over the past year and what will guide us in the new year.

Recapping 2021

In 2021, we had a continued focus on fostering connection on Flickr. Our work was centered on improvements and experiences that made it easier to connect with your fellow Flickr members and to share and be inspired by amazing photography.

  • We started the year with our very first #MyFlickrYear, a personalized report celebrating how active members on Flickr share and connect with each other.
  • We made it easier to report content or behavior that violates Flickr’s community guidelines.
  • We put renewed focus into the Flickr Commons and announced that we’ll be launching a foundation to grow and nurture this program.
  • We launched a new Notifications Center and Settings to give you more control and visibility into how people are interacting with you, your photos, and groups on Flickr.
  • The Flickr mobile widget brings photography inspiration directly to your mobile phone’s home screen.
  • We collaborated with several fantastic organizations, including FUJIFILM, LEGO, and Black Women Photographers, to create moments of inspiration and opportunity for photographers of all backgrounds and interests.
  • Our Support Heroes team focused on delivering exceptional support: we respond to everyone that writes in with a question or feedback about Flickr, and we are committed to answering questions from Pro members within 4 hours. If you ever have a question, don’t hesitate to write to us!
  • Our Trust and Safety team worked to make sure that Flickr is a safe platform. We’ve doubled down on our efforts to combat CSAM. This team is also instrumental in making sure that all Flickr members are aware of Flickr’s Community Guidelines and is committed to upholding them.
  • Our Operations team worked year round – sometimes on weekends and late evenings! – to make sure that Flickr runs smoothly and effectively. And if for any reason Flickr is not up and running, restoring the site is their top priority.
  • Our Engineering team published their vision and guiding principles.

This is far from an exhaustive list.

Throughout the year, you showed up with support for your fellow Flickr members and feedback to help us continue improving. Thank you!

What’s ahead in 2022

Over the past few years, all of our efforts have been guided by making Flickr the best place to be inspired by photography, to connect with others who share your interests, and to share your work with the people who matter to you. Putting these three concepts of inspiration, connection, and sharing at the center of our decision-making was validated by the community in a recent survey and we will continue to use these core tenets to guide our work in 2022.

In the short term, here’s some of what we’re working on:

Inspiration

  • #MyFlickrYear will be making a return soon, and this year we hope to make it available to even more Flickr members. Stay tuned!
  • Explore takeovers that highlight the Flickr community’s expansive interests and inspiring photography.
  • Experimenting with exclusive features and perks for Pro members.
  • Keeping Flickr a safe place where everyone feels comfortable sharing their photography and working with our members and partners to uphold Flickr’s Community Guidelines.

Connection

  • Continued improvements to the Notifications Center based on your feedback.
  • Delivering exceptional support to all Flickr members – we respond to Flickr Pros within four hours.

Sharing

  • Photo prompts, contests, and discussions that bring together Flickr members that share the same interests and desire to improve their photography.

Looking forward, we are excited about continuing these efforts into 2022. In order to strengthen Flickr as a sustainable, independent platform and to support ongoing investment into this community we all love, we will be increasing the price of Flickr Pro membership. You can always find the latest pricing information in this FAQ.

To everyone on Flickr, thank you for being here and making Flickr your home for inspiration, connections, and sharing. Your love of Flickr and photography keeps us going and we are committed to continually improving your experience. Please stay tuned to the Flickr blog for continued release notes and updates on what we’re working on.

See you in 2022!

12 Dec 03:20

wheel.yml for Pyjion using cibuildwheel

wheel.yml for Pyjion using cibuildwheel

cibuildwheel, maintained by the Python Packaging Authority, builds and tests Python wheels across multiple platforms. I hadn't realized quite how minimal a configuration using their GitHub Actions action was until I looked at how Pyjion was using it.

Via @simonw

12 Dec 03:16

Debunking

mkalus shared this story from xkcd.com.

b'Mark Zuckerberg has only neutral feelings toward Peppa Pig, who he understands is a fictional character, and he blames the coronavirus pandemic on other factors.'
12 Dec 03:15

Not my Brexit

by Chris Grey
The evidence that Brexit is causing mounting damage has been growing since the transition period ended, and has been catalogued in almost every post on this blog since then. It is also to be found on Professor Gerhard Schnyder’s Brexit Impact Tracker, Yorkshire Bylines’ Davis Downsides Dossier, the now closed ‘Keleman Archive’ of 1000 examples, the Brexit database, and a newish substack blog I have only just come across, Nick Tyrone’s This Week in Brexitland. The latest dollop of evidence comes with an excellent new report on the impact of Brexit on services industries by Professor Sarah Hall and Matt Heneghan for the UK in a Changing Europe centre.

That damage isn’t just economic. Those who believed promises of a boost for Britain’s global stature have been rewarded with at best international bemusement and at worst a country deemed untrustworthy and liable to break international law and treaty obligations. Those expecting a revivified national democracy have instead seen an illegal prorogation of parliament and a spectacular and ongoing power grab by the Executive.

Yet, for reasons ranging from the Trappist vows both Tory and Labour politicians seem to have taken, through to media and public pre-occupation with Covid, relatively little is heard of this Brexit damage (though, just this morning, the Express seems to be catching on). Those politicians, journalists, and academic or other experts who do speak of it are ignored or traduced by Brexiters as ‘remoaners’. Brexiters themselves, perhaps unsurprisingly, are reticent to discuss what is happening and when they do the guiding theme is to evade responsibility for it. Astonishingly, the government itself, according to leaked documents, has no measure of whether Brexit has been a success or failure. For some, Brexit was simply ‘the right thing to do’ so the consequences are irrelevant. Others ignore or deny the evidence of Brexit damage. Still others just disown it on the basis that ‘this is not my Brexit’.

Businesses suffer in silence

Amongst those we might expect to be making a noise and to be heard are businesses. It is they that are bearing the brunt of the economic damage, especially those which trade with the EU. A new report this week showed that cross-Channel delays are actually worsening, and there is a lot more pain coming in just three weeks’ time when the UK begins to introduce full import controls. As with all the new trade barriers, it is small firms which will struggle to cope the most and a recent Federation of Small Businesses survey suggests that only a quarter of such firms that import from the EU are ready for what they will face. Beyond trade, almost all business sectors are suffering from labour and supply shortages.

If we hear relatively little public clamour from businesses, and where we do it is more likely to be from representative bodies rather individual companies, it is for good reason. The story of Brexit and business is a complicated one, well told by Iain Anderson, Chairman of Cicero Group, and also discussed in a Mile End Institute podcast featuring Nicole Sykes (then Head of EU negotiations for the CBI), me, and Professor Tim Bale of Queen Mary, University of London. In brief, businesses have been reluctant to speak out for fear of attracting government reprisals in one form or another, and of alienating leave voting customers.

Meanwhile Brexiters have never forgiven the fact that most businesses and their representative bodies opposed, albeit not very vociferously, Brexit before the referendum, and often warned against hard Brexit and no-deal Brexit in the years thereafter. One consequence is that throughout the Brexit process businesses have tended to be excluded from government consultation unless very clearly pro-Brexit, reflecting the cult-like approach to Brexit taken by both the May and Johnson administrations. Even now, anything they say is likely to be dismissed as remainer ‘negativity’ or ‘fearmongering’ as happened over warnings about HGV driver shortages and fuel supplies. On that occasion, an anonymous “senior government source” also ominously threatened the Road Haulage Association that “we will deal with them when this is over”.

To the extent business is able to influence government Brexit policy it is very much behind the scenes, as seems to have happened in relation to this week’s news that the UK is likely to postpone introducing its independent chemicals industry regulatory system. It follows previous postponement of things like the UKCA quality mark after intensive lobbying from the manufacturing sector. Backtracking on such things is welcome, and it could even be a prelude to abandonment, but neither is cost-free. The problem is the disruption and uncertainty, as well as the lobbying effort, engendered by the original rushed timescales and the ideological symbolism of creating meaningless independence. In a similar way, a National Audit Office report this week suggested that in rushing to make trade deals the government is neglectful of consulting businesses and consumer groups and, hence, of their substantive needs.

Pro-Brexit business voices

Overall, it seems fair to say that whilst businesses may be resigned to the reality of Brexit, they continue to be critical of the way it is being done. It is certainly not ‘their Brexit’, and whilst the state is on some accounts supposed to be the executive committee of the bourgeoisie that doesn’t seem to apply to Johnson’s ‘f*** business’ Brexit regime.

But what of those business leaders who were pro-Brexit? One difficulty here is that, compared with politicians, journalists, academics and media commentators, the numbers involved are quite small. Repeated attempts to construct pro-Brexit business umbrella groups, the most sustained being the Alliance of British Entrepreneurs (ABE), have made little headway. It is hard to know who such groups speak for anyway – the ABE, for example, “does not offer formal membership”, and what little is known about it is, to say the least, underwhelming.

At all events, both before and since the referendum, whenever a pro-leave business person appeared in the media you could list on virtually the fingers of one hand which of the white, middle-aged and mainly titled men heroically taking on ‘the establishment’ it would most likely be. Tim Martin (66), Sir James Dyson (74) Lord (Anthony) Bamford (76), Lord (Simon) Wolfson (54), Sir Jim Ratcliffe (69) and Sir Rocco Forte (76) just about covered the main possibilities.

So far as I know neither Sir James nor Sir Anthony – both of whom have had extensive legal disputes with the EU (£) - have ever suggested that Brexit has had any downsides. In fact Dyson this week re-iterated (£) his opposition to the EU’s approach to regulation (though since in the example he uses, his own legal case, he won and the regulation was dropped, it isn’t a strong argument for Brexit unless he believes that British regulations are infallible). To the extent that he worries that the UK may continue to align with many EU regulations, which it probably will, it’s at least possible that he will come to think it was ‘not his Brexit’. As for Sir Jim, he too hasn’t recanted, although his decisions in 2020 to shift production of the Ineos Grenadier from Wales to France and his own tax domicile from the UK to Monaco were perhaps not exactly ringing endorsements for Brexit Britain.

Amongst the other high-profile figures, their most obvious criticism has been over the consequences of ending freedom of movement of people. Tim Martin of Wetherspoons spoke in June of the need to liberalise the immigration system, with special treatment for “countries geographically closer to the UK”. Lord Wolfson, CEO of Next, has also repeatedly called for more open immigration, especially given the labour and supply shortages which have emerged this year.

They were joined last weekend (£) by Sir Rocco, Chair of Rocco Forte Hotels, who also wants a relaxation of immigration controls which are too restrictive for his, and many other, businesses which face the fact that “it is obvious that there is no ready supply of labour available in the UK to fill vacancies”. Forte has the grace to note that “I am sure that some will say I should have been careful what I wished for in supporting Brexit. I would respond that the only issue determined in the referendum result was that the UK should decide its own immigration system. As a Brexit supporter, I wanted proper control over our borders, not their virtual closure”. It’s not his Brexit, it seems.

Yet this does not go so far as to take responsibility for the fact that he used his authority as a business leader to decry those warning of precisely what he is now bemoaning. For example, writing in the Daily Mail in July 2018, he said “concerns voiced by the big business lobby are little more than scaremongering, like the claim that any restriction on European freedom of movement will badly hurt recruitment by British firms. This is untrue. My family's hotel chain hired staff from all over the world, including Europe, long before the EU was even created. We will continue to do so after Brexit, especially because so many young people from Europe want to come here to learn English. Contrary to the hollow warnings from the pro-EU campaigners, migration controls will not mean an end to European migration.”

Wolfson, too, argues that Brexit only meant the UK setting its own immigration policy, rather than being anti-immigration per se. Yet before the referendum, when it would have counted more, his formulation that we must “place our trust in the collective intelligence and endeavour of Britain’s 30 million-strong workforce” did not, at the least, do anything to challenge the centrality being put upon immigration control during the campaign.

In fact, as I noted in my previous post, Brexiters on the free-market right were not, in general, especially bothered about immigration. But they chose to ‘ride the tiger’ of a project which had as one of its central selling points, if not the central point, profoundly anti-immigration sentiment. Within that same central narrative, softness on immigration, and opposition to Brexit, was associated with the self-interest of the “big business lobby” referenced by Forte. Moreover, in the post-referendum drive towards hard Brexit, it is undoubtedly the case that continuing freedom of movement was critical to the argument that soft Brexit would not honour the referendum result. Yet, far from questioning hard Brexit, both Forte and Martin made the case for an even harder ‘no-deal Brexit’, whilst Wolfson suggested that no-deal would cause only “mild disruption”.

The consequence of how Brexit was sold and then executed, whatever they may have wanted or expected, was to make it virtually impossible for the post-Brexit immigration regime to be a liberal one and, certainly, this government isn’t going to deliver it. So, in the absence of any suggestion that they now recognize they made a mistake in championing Brexit, the position of Forte, Wolfson and Martin would appear to be that of ‘this isn’t my Brexit’. It is a position shared by many of those, such as former Brexit Party MEP June Mummery, who regard what has been delivered by Brexit as a betrayal of the fishing industry. Indeed, following the complaint of one enthusiastic leaver about the airport queues he now experiences, “this isn’t the Brexit I voted for” has become shorthand to denote those who, whilst still supporting Brexit, rue some of its consequences.

Johnson and Frost: it’s not our Brexit, either

That some of the most committed Brexiters would regard what got delivered as, at best, unsatisfactory and, at worst, as betrayal was baked into Brexit from the beginning, of course, because what it would mean was never specified. So it’s not a surprise that they are now saying that it isn’t their Brexit or (which is slightly different, as it also suggests that their Brexit was real Brexit) that Brexit hasn’t been done properly.

What is far more surprising, not to say downright grotesque, is the fact that Boris Johnson and David Frost, whose Brexit it actually is, also consider that Brexit hasn’t been done properly, at least as regards the Northern Ireland Protocol. And they, too, suggest it is not their Brexit but, rather, the one forced on them by remainer MPs in 2019 and by the EU taking advantage of their temporary weakness (the golden rule for understanding Brexiters is that they are never responsible and are always the victims).

That has been their repeated complaint for months now, and it looks very much as if their attempts to negotiate a different Brexit to the one they agreed, signed and said was a triumph will now drag into the New Year. That may still bring a triggering of Article 16, although the fact that it has not yet been used, whereas in the summer the implication from Frost was that it would happen very quickly if UK demands weren’t met, suggests a degree of caution on the part of, presumably, Johnson. That may be as a result of the clear messages from the EU that it would make a robust response and/or because of US pressure, including the delayed removal of steel and aluminium tariffs. If that caution holds, and Article 16 isn’t used, we can expect years of being told that, yet again, the Brexiters didn’t get the Brexit they wanted because of EU bullying and US meddling.

History will judge?

This idea that Brexit hasn’t been ‘tried properly’ resembles, as I and many others have frequently remarked, the claims of some apologists for communism. It’s a slippery argument but at least has the merit of acknowledging that things haven’t gone as promised. But, again like some apologists for communism, they have a different trick in their book which is to say that it cannot be evaluated on present appearances or achievements but only in the (very) long-run. Be patient, comrades, our great cause will be proved right by the onward march of history.

It’s a trick that has already been deployed by Jacob Rees-Mogg, who suggested it might be 50 years until the success or otherwise of Brexit is known. Lord (Digby) Jones, the buffoonish former head of the CBI who declared that “there’s not going to be any economic pain” from Brexit, subsequently suggested a timeframe of 100 years for judgment. This week it has surfaced again in an article on the Conservative Home website by Ryan Bourne of the CATO Institute (formerly Head of Policy at the Institute for Economic Affairs and, apparently, though all information about the group’s membership seems to have disappeared, at one time part of Patrick Minford’s Economists for Brexit).

Bourne argues that it is premature to gauge the economic effects of Brexit, and also wrong to conflate Brexit with whatever the Johnson government does or does not do, because the real issue is the possibility that British institutions will prove more economically liberal and more likely to create permissive regulatory regimes than those of the EU. Bourne believes this will be so, but suggests that whether it proves to be the case will only be known in the long-term, which he implies to mean something like 30 years.

Needless to say, this is sophistry of the highest order, arising solely because it has become impossible for any half-way serious person to argue that Brexit has been an economic success so far, or even that such success is imminent. So, conveniently, we are told to defer giving our verdict for a few decades. But Brexit was never proposed to the British people as something which might, perhaps, in many decades be a success but then again might not. Had it been, far fewer people would have voted for it than did – almost certainly too few to have won the referendum. Moreover, if Brexiters had really been serious about the long-term nature of their project, they would not have pushed so recklessly to ‘get it done’, treating all extensions, whether to the Article 50 process or the Transition Period, as treachery rather than careful preparation for an epochal change.

Avoiding accountability

It may be that in 30, 50 or 100 years there will be a consensus view but, in the absence of some decisive ‘Berlin Wall’ moment, it’s equally likely that Brexit will continue to be contested. Indeed we can already see how Brexiters have created the conditions for never having their project judged at all.

For when forecasts are made of Brexit damage, they invariably dismiss them as ‘just predictions, no one can know for sure what will happen’. But as soon as there is actual data about Brexit damage they say ‘ah, but this cannot be proved to be the result of Brexit’ - most notably, so far, because of the pandemic, but 30 years on there will be any number of other reasons to give, and it will get ever-harder to disentangle Brexit from them. Meanwhile, any time that anything good happens to the UK, or on any occasion that the EU experiences difficulties, they will say that this ‘proves’ it was right to leave.

In this way Brexiters protect themselves in a hermetically sealed bubble where both prospective and retrospective scrutiny is ruled out. It will always be either too early to say or too late to know. It is this which is the real significance of the ‘history will judge’ test. For Brexiters, it at worst defers and at best avoids accountability for what they have done. If nothing else it ensures that those who advocated Brexit will be so old, and so distant from having had any power or influence, as to be totally unaccountable for what they promised and what they did.

Take the high-profile pro-Brexit business leaders mentioned above: in 30 years’ time the youngest of them will be 84 and the oldest 106. Amongst the political leaders, Johnson and Farage will both be 87 whilst veteran Eurosceptic Bill Cash will be 111 years old. If we apply the Digby Jones time frame of 100 years, they will all be long dead. And if, by chance, some of the more youthful Brexiters are still around, or some miracle – or horror-film nightmare - of cryogenic storage preserves, say, Jacob Rees-Mogg to face the music, well, they will always have the first and last-ditch evasion of responsibility available. Yes, they may conceivably admit, it was all a ghastly mistake, but only because what was done was ‘not my Brexit’.

 

I will be taking a break from blogging for the rest of the year. My review of Brexit since the end of the Transition Period will be published in the December print edition of Byline Times. And if you haven’t read it, or are stuck for a Christmas present, you could always buy my book Brexit Unfolded. How none got what they wanted (and why they were never going to) published by Biteback earlier this year and available from all good booksellers.

12 Dec 03:09

Wallpaper: Inside Apple Park

by Volker Weber

Global exclusive! Led by Evans Hankey and Alan Dye, the Apple Design Team holds enormous sway over our evolving relationship with technology. Opening the doors to their studio at Apple Park in Cupertino for the first time, they offered us a deep dive into the working processes behind their latest creations

More >

12 Dec 02:43

Filtering Lessons

Last week, AWS announced (docs, blog) event filtering for Lambdas reading from SQS, DynamoDB, and Kinesis. Lambda’s new filtering is consistent with EventBridge’s. Also announced was faster, slicker, cheaper S3/EventBridge integration. These features rely on software I helped write. The problem turns out to be more interesting than you might expect, by a factor of 10¹² or so. I think it’d be really great if the code were open-sourced.

The core idea is that in modern applications, streams of data flow around between services, and very often you want to select just some of the data out of the stream. For example, if you’ve written a few Lambdas, quite likely some of your code was about looking at the input events and deciding whether to process or ignore them. Or deciding which other software to dispatch them to. If you can achieve the same effect by writing pattern-matching rules, you can subtract code and save money: a win-win.

Interesting why?

It’s not as though these filtering features are doing anything new or innovative or surprising; this sort of thing goes back as far as 1973, with grep. There are lots more landmarks along that road; for example a lot of smart people seem to like jq these days. So, why should anyone care about this software?

First of all, the stream it’s filtering might be carrying millions of events per second. Second, people will want to post a million rules against the stream. (Neither of those numbers are hypothetical.) So it’s a million times a million times trickier than you might think.

Bearing that in mind, we created a library which I’ll call the Matcher. Its query expressions are a little idiosyncratic, but people seem to like them; that blog I linked to earlier is probably the best introduction I’ve seen.

The performance envelope is, um, unusual. But pleasing. “Good enough for EventBridge” is decent testimony. Even stronger is “good enough for S3”. Check out this short thread from Usman Khalid, GM for EventBridge and some other stuff, a guy I really enjoyed working with at AWS. He explains that the event-matching logic has been pushed down into S3, which doesn’t even emit the events unless they match a rule. Now, “inside of S3” is a very special place. An almost unimaginable amount of customer data lives in there, subject to an almost unimaginable transaction rate, and everyone is very very latency-sensitive.

How it works

Well, that’d be a secret. At least I thought so until I sat down to write this, and discovered U.S. Patent 11,068,487: Event-stream searching using compiled rule patterns, filed in 2015 but just issued last July. Patent theory says that the disclosure should be sufficient to enable a competent practitioner to implement the technology. Heh, good luck with that. I know a better way than patents to share engineering progress.

Open-source why?

The Matcher would be useful for a lot of people, if the AWS experience is any guide. It’s easy to integrate. It’d be a significant and visible give-back from AWS to the software commons from which it’s profited so richly. I also think it’d be a useful teaching tool (see below).

Having said that, and to be brutally honest, my reasons are selfish. I enjoyed working on this so much, and would like to do it some more. So from that point of view, AWS would get some free time out of an engineer with highly relevant experience.

I might also like to make versions of it in other programming languages.

And courseware

I applied plenty of lessons hard-won in decades of programming to this software, and learned useful new ones. So there are deep software-tech blog pieces screaming to be written. In fact, I think I could probably put together a 400-level CompSci course on “Topics in high-performance software design” using the Matcher as a lab substrate.

I’ll support that claim with one interesting and counter-intuitive lesson. But first…

History

When I started at AWS on December 1, 2014, my hiring manager said “Our idea is we’re gonna have an event bus carrying notifications of everything that’s happening in everybody’s AWS account and you can subscribe to it, select the events you care about, and process them with Lambdas.” So, it wasn’t my idea; and that manager is now a VP who owns a huge swathe of AWS Networking.

But I helped write the PR/FAQ and was senior engineer on the dev team.

I couldn’t stop thinking about matching the rules that defined the subscriptions. My mind hadn’t stretched to millions times millions yet, but it seemed obvious that the biggest customers, who would have the most events, would also have the most rules. So if you coded up a simple-minded loop like “for each event, retrieve all the rules posted by the event’s account and check one by one to see if they match” that’d probably be bad.

So I coded up most of release 1.0 of the Matcher by January, with a couple of breakthroughs happening on my Mom’s couch when we visited her in Saskatchewan for Christmas.

EventBridge (launched as “CloudWatch Events”) worked out pretty well and soon other groups started using the Matcher. It grew plenty of features over the years, some only exposed internally. Naturally, like any other widely-used piece of software that started out lean and mean, it became fatter and more complex. While there’s a ton of my code there, I’m sure it’s a minority by now.

I met some wonderful people from other AWS and Amazon teams who said “Oh we need Matcher but it also needs to do X, mind if I code that up?” Working at AWS was a fun job, and Matcher was one of the most-fun parts.

Now, I promised a non-obvious finding that emerged during the Matcher work…

Lesson: JSON is faster than binary

Here’s a surprising statement: If you want to filter messages very, very quickly, you can do it much faster if they’re in JSON as opposed to a strongly-typed “binary” format like Protobufs or Avro.

No, really. Because for those formats, you have to parse the whole binary blob into a native data structure, which means looking at all the fields and allocating memory for them. You really want to minimize memory allocation for anything that happens millions of times per second.

JSON, however, allows stream parsing. Consider for example, the Jackson library’s nextToken() or Go’s json.Tokenizer.

Which means you can skip through each event, only keeping around the fields that you’re interested in filtering on, which in most event-processing scenarios is very few of them. So yeah, you do have to parse strings of digits into numbers, but only for the fields you care about.

And there are better structures for storing a small selection of data fields than JSON’s messy, weakly-typed tree. I’ve measured; the performance delta between parsing incoming events into an object and just skipping through their fields is freaking huge.

That’s just one take-away; there are plenty more. I’d sure like to share them.

[Update: I’ve seen feedback that there’s no reason in principle that you couldn’t build streaming parsers for the strongly-typed binary data formats and that, for Avro in particular, it ought to be really fast. As far as I can tell, there isn’t much in the way of such APIs out there at the moment; sounds like fun little programming project.]

12 Dec 02:40

A reverse chronology of some Python features

by Brett Cannon

I occasionally hear people lament that Python is "bloated", "too big", "going enterprise", or some other phrasing to suggest there was once an "ideal" version of Python that had less bloat and was closer to what the person wanted.

But then this lament is also often followed by, "but I would want this one feature from a newer version of Python". 🤨 What I think a lot of people don't realize is that various features are interconnected in ways that may not be obvious. And this can go beyond the typical cause-and-effect scenario where you can't cleanly separate inspiration, etc. from each other.

Take, for instance, pattern matching. Plenty of people complained that it wasn't going to be necessary and they could live without it. But the adding of pattern matching necessitated writing a new parser for Python. That new parser allowed for improved error messages (something I have yet to hear anyone say was a bad thing to introduce and for a lot of people their favourite feature in Python 3.10). So if you don't want the match statement, are you willing to give up the better error messages as a trade?

To help those who wish for the "good old days" of some older Python version, I thought I would write down a bunch of the major language features that were added in Python in reverse chronological order. Start from top and work your way down until you come across a feature you aren't willing to give up. The Python version which has that feature is the version you would be okay going back to.

By the way, you also have to give up all subsequent performance improvements when you go back as new features lead to code clean up, new insights on how to make something better, etc. And I can tell you right now, if you like performance you are probably going to be making Python 3.11 your favourite version of Python. 😁

I'm going to stop at Python 3.1 because Python 3.0 is intertwined with Python 2.6 and the list is way too long to want to write out. And I'm not going past Python 3.0 because it's time to move on already. 😁

12 Dec 02:30

“This is important, tell me why it’s great” – A Fallacy

Ted Curran, Ted Curran.net, Dec 10, 2021
Icon

Here's the gist: "This fallacy takes for granted that this content is “important” without doing the hard work of letting learners experience for themselves what’s genuinely good or useful about it" and it "puts learners in the position of having to instantly come up with reasons why this new framework is the best idea ever." An example of this is a learning exercise where you introduce a new technology and then have participants come up with applications before ever having actually used it.

Web: [Direct Link] [This Post]