Rolandt

There’s an itch I’ve been picking at for some time. Everyone wants me to be more productive. All I need to do is listen to their podcast, buy their book , install their software, or implement their system. I’ve spent too much time and energy chasing those promises with limited return.

Productivity metrics are appealing because they’re easy. Count the number of notes you’ve captured to your personal knowledge management system. Plot the number of words you wrote today. Reward the programmer who wrote the most lines of code last week.

Productivity matters if you’re turning out cars or cellphones. Not a great metric if you’re in a more creative line of work. Einstein wasn’t lauded for how many papers he wrote. Picasso wasn’t praised for being prolific.

We appreciate this distinction in clearly creative realms. We haven’t managed to transfer that appreciation to less obviously creative spaces. More importantly, we haven’t grappled with the reality that we now all operate in creative realms.

Peter Drucker made a step in this direction when he wrote The Effective Executive (see Effective executives are design thinkers for my review). Although Drucker coined the term “knowledge worker”, he didn’t extend his analysis of effectiveness to them. But we knowledge workers now drive the economy and we don’t have good ways to sort out how to manage ourselves appropriately.

I’d like to spend the next few weeks taking a deeper look at what it might mean to shift from thinking about efficiency to thinking about effectiveness. Can we think and talk about effectiveness in ways that can better shape how we go about doing our creative work?

The post Dethroning productivity: becoming more effective appeared first on McGee's Musings.

Here are notes and links from my talk at Southern California Linux Expo. This is not exactly what I said but the links should be right.

Introduction: limits of individual protection

I need to start with a disclaimer. Privacy tools will not get you privacy, but it's important to do them anyway.

• Most privacy tools and settings make you different from others, so more fingerprintable (hi, Linux nerds)

• Out of band surveillance, which privacy tools can't touch, is still there (and the more you participate in the modern economy, the more you're surveilled)

• companies fail to comply with existing laws (yes, they complain that they're "confusing" and somehow that seems to work in a lot of cases.) And they play legal tricks to build up and use friendly jurisdictions.

• the more you try to protect yourself, the more crappy ads you get. How long can you stand getting the miracle cure ointment ad featuring the "before" picture—before you give up and try to get tracked so you get more legit ads?

By itself, no set of privacy tools is going to be able to get you to an acceptable level of privacy. If that's what you came for, you can split, go get a coffee or something. Privacy tools are worth doing, but only as part of a cooperative program to address a much larger set of problems.

Am I here to argue the privacy point of view or the pro-business point of view? Yes. First of all, surveillance advertising competes directly with product innovation for investments. The economics literature on the effects of shifting investment away from surveillance is going to be interesting. In the meantime, we have evidence from mobile games and mobile game advertising, which is a fast-moving behavioral economics lab that helps spot trends early. As mobile game developers lose much of their targeting info to Apple's App Tracking Transparency, they compensate by investing more in content and gameplay.

Surveillance also puts us at greater risk from internal strife and external adversaries. Bob Hoffman told the European Parliament, Tracking is also a national security threat. The Congress of the United States has asked U.S. intelligence agencies to study how information gleaned from online data collection may be used by hostile foreign governments. (More on that: Microtargeting as Information Warfare by Prof. Jessica Dawson of the Army Cyber Institute) All right, all right, fine, you came for open source privacy tools advice so I will include some. We'll do some fun tips, but I will try to put them in context. Here's number one.

Privacy tip: fix YouTube

This one is an easy example of a tracking risk. The video you came to watch is great, but the stuff that gets suggested after that is the first step down the rat hole. A case before the Supreme Court right now covers YouTube running ISIS recruiting videos. And one of their biggest stars was just arrested in Romania for human trafficking. The recommendation algorithm on there will take you to some pretty dark places, pretty fast. But there are also videos on there you have to watch for school, or work.

What I want is a YouTube setup that will not just protect my privacy, but also reduce the number of YouTube impressions I generate. (Also I want to visit my family members at college or at a legit job, not at terrorist training camp.)

This requires two browser extensions. First, use the Multi-Account Containers extension to put YouTube in a separate browser container. That's a separate space with its own logins, storage, and cookies.

Click the extension, select "Manage Containers" and make a new container for YouTube. Then go to YouTube and choose "always open this site in container."

Second, use Enhancer for YouTube to turn off the problematic features.

• Turn off autoplay

• Hide "related videos"

• Enable "Theater Mode" to make main video wider (and fill in the space left by related videos)

• All your other least favorite features of YouTube can also be hidden.

Don't use the native mobile app for YouTube. There are also alternate front ends for YouTube and other services. LibRedirect is an extension to automatically redirect to those services. But they aren't as future-proof, because it's easier for YouTube to break them.

Privacy violations as part of a larger problem

Privacy is downstream from other problems. Surveillance capitalism isn't a thing because advertisers care about your personal data. If you do a "Right to Know" most of what you get is wrong. According to Oracle, I am a soccer mom of four.

I hate to break it to you, but those startups that tell you that you can make money selling your personal info to advertisers are wrong. Your data has value as a short-term intermediate reaction product, as an input to two kinds of deceptive practices.

• First of all, the B2C scams: ad targeting lets the legit advertisers show their ads to the affluent users who can afford stuff. And they typically target younger people. Targeting generally cuts off at age 34 or 54. So what do social media platforms do with all the ad slots that the legit advertisers don't buy, and that go to older and poorer people? Right, they get the not so legit ads.

• Second is the B2B scam: tricking advertisers into sponsoring content that they would choose to avoid, such as Russian disinformation and copyright infringement. User tracking lets ad intermediaries sell an ad impression that claims to reach a high-value user, but on content that no reasonable advertiser would choose to support.

Hey, look, it's a real car ad on a scraper site.

How can such a large industry get away with this kind of conduct for so long? Professor Tressie McMillan Cottom wrote,

From buying a gallon of milk to making a dinner reservation, all the way up the chain to electoral malfeasance, so many of our interactions feel weighted away from social connection in favor of extracting every ounce of unfair advantage from every single human activity. Not to overstate it, but a pluralistic democracy simply cannot function when most of its citizens cannot trust that the arrangements that they rely on to meet their basic needs are roughly fair.

To put the privacy problem in context, privacy violations as we see them are downstream from changes in business norms. We're pulling on one tentacle of a much bigger threat than just the obvious personal privacy issues such as swatting, identity theft, price discrimination, and investment scams.

The relative payoffs of production and predation (or “making” versus “taking”) are determined by legal mechanisms for enforcing contracts and protecting property rights, but also by social norms and interpersonal trust. — Stephen Knack

The more common that scams become in our markets and society, the more trust we burn, the more unnecessary risk we take on, and the more wealth we miss out on. This is not an advertising problem, as Bob Hoffman said—it's a surveillance problem. Some kinds of advertising can have economic signaling value that does help build trust, but only if the medium is set up for it. (Fixing digital ads to pull their weight and fulfill the signaling role of advertising is a hard problem, and I should bug Rory Sutherland some more about how to solve it.)

Privacy tip: mobile apps

Native apps can track you in ways that web sites on a well-behaved browser can't. In-app browsers inject JavaScript. Apps contain tracking SDKs. Privacy filters are limited.

Banning TikTok is a start, but all apps containing the TikTok SDK are TikTok, as far as harvesting your info goes. And other apps and/or SDKs could be sharing your info with other people you don't want it shared with. There are a few ways to deal with surveillance apps.

• Delete your account

• Switch to using the service on the web

• Limit your use of the app (For example, make a habit of checking Signal before you check surveillance apps, so people learn that it’s a better way to reach you.)

Remove all the surveillance apps from your phone that you can. This doesn't necessarily mean cut off people who you can only reach on a surveillance app.

Social connections mean you live longer, so apps required to communicate with friends and family are worth spending some of your personal "privacy risk budget" on. In general I accept some risks on social connections and collaborating, but I try to be stricter when it comes to shopping and entertainment.

Good news: privacy is on the way up

Right now we're in the middle of some positive privacy trends: more effective enforcement of existing privacy laws and regulations, more interest in new ones, and some software improvements that make some kinds of tracking harder. Here in California, some recent news includes

• the Sephora case, which makes it clear that common surveillance marketing practices are sales under the CCPA

• the Attorney General's office has been clarifying rules on Global Privacy Control, and doing a sweep of law-breaking apps

• CPPA, the new privacy agency, is starting up

More states are probably going to get privacy laws this year, so those outside California will be able to join in.

We have some advantages if we sound like we know what we're doing. Surveillance and the attitudes underlying it are not changing right away, but you can shift things if you approach each action with something easy for the other end to do, and act like you know what you're doing. They're going to balance the time required to act on your privacy mail against the risk of not acting on it, and if the risk looks big enough, easier to do the right thing.

Enough of those and we shift the relative expected payouts of surveillance and non-surveillance investments. (Like RCRA got many companies to just cut down on hazmat instead of dealing with the required record-keeping.)

Privacy tip: mobile phone settings

Check your phone settings. On Apple iOS there are two settings for surveillance ads: one for most companies' ads, and one for Apple's own ads. Don't forget to check both. (Yes, you probably have to scroll down for the second one. Well played, Apple.)

• In Settings, go to Privacy & Security, then Tracking, and make sure Allow Apps to Request to Track is turned off.

• Also in Settings under Privacy & Security, find Apple Advertising and make sure that Personalized Ads is turned off.

On Android, you can open Settings, go to Privacy, then Ads and select Delete advertising ID.

More info: How to Disable Ad ID Tracking on iOS and Android, and Why You Should Do It Now | Electronic Frontier Foundation

If you get these settings right, your mobile app ads will get really crappy, really fast, but you're limiting mobile time and maximizing web time, right?

General principle: trying to herd the money away from the worst places.

We said we would try to retrain the surveillance business.

Address the worst practices first. Doing the easy stuff first can create the wrong incentives. That's why I put YouTube first here. Then level up your privacy skills and toolset.

Level 1: mix of effective and ineffective privacy practices and tools

Level 2: effective privacy practices and tools

Level 3: effective practices and tools applied in an effective way

Remember you are not going to be able to get individual protection that's meaningful while still participating in society. It's more about driving transformation.

Privacy tip: Google Chrome

This is a big year for Google Chrome.

• Manifest v3: coming next year, will limit the ability of content blocking extensions to block dynamically. On the Google Chrome browser, ad blockers, along with tracking protection tools that work in a similar way to ad blockers, will soon be limited in what they can do. If your chosen privacy tools and settings are not going to be supported, you might have to switch browsers. Browser compatibility has gotten a lot better recently, so if you switched because a site you like was broken on your old browser, please check it again. (This may or may not affect other Chromium-based browsers.)

• Privacy Sandbox: a variety of projects, including a mix of some actual privacy features, one big on-browser ad auction (an ambitious project) and some anti-competitive shenanigans. It looks like 2023 will be the summer of double ad JavaScript on Google Chrome: you'll still have the old cookie-based stuff, but you'll also have an experimental ad auction running inside the browser.

Because Google both competes as an adtech intermediary and releases a browser, antitrust concerns mean they have to try to make the browser fair to all the other adtech intermediary companies they compete with. This is now the subject of an ongoing investigation by the Competition and Markets Authority in the UK. James Rosewell, CEO and founder of a mobile data company called 51Degrees, started a long, complicated process.

If you do decide to keep Google Chrome, there is a bunch of brouhaha about the impending end of third-party cookies, but you can turn them off today without breaking much, if anything. (Sites already have to support browsers that don't do third-party cookies.) From the Ad Contrarian newsletter:

1. Open the Chrome browser. Click the three dot thing in the upper right corner.
2. Click "Settings"
3. In the left column click "Privacy and security"
4. Click "Cookies and other site data"
5. Click "Block third-party cookies"

Google Chrome also has new in-browser advertising features, confusingly lumped together as Privacy Sandbox. Check chrome://settings/privacySandbox.

The text in this settings screen is not especially helpful. Google's "Topics API" is a general-purpose system to categorize users based on sites you visit. There's nothing about it that's limited to ads. Will probably be more useful for price discrimination, and worse, helps incentivize deceptive practices to drive web traffic. (Looks like that message is in open-source Chromium, so filed an issue. Anyway, if you have Google Chrome, turn this off.

Update 17 Jul 2023: If you have a web site, or if you administer desktop systems with Google Chrome on them, you can do a few more steps to protect others: Google Chrome ad features checklist

The attribution tracking chain

This is an oversimplified chart, but it's a good start for learning about the "attribution tracking" chain and how to break it.

If you break the link between surveillance data on what you buy and surveillance data on what ads you saw, then it's harder to justify investments in surveillance advertising— remember, we're trying to move money from surveillance to other investments here.

That's the main reason why you have YouTube on a browser container that is never used for anything else. It interferes with the attribution link between a video view and a sale.

Privacy tip: remember to vote

California has the CPRA because people voted for Proposition 24 in 2020. The CPRA isn't perfect, but voting made a difference. While you're voting, please don't eliminate a candidate from consideration just because they're using the big surveillance platforms. They're hard to avoid completely. In today's environment it's generally better to make a little progress than to achieve privacy purity but lose the actual election.

Breaking the chain: where?

Once you have a mental model of the attribution chain you can allocate your time most effectively.

• Software tools are usually lower effort but shorter range: they can protect an activity like a web session but can't reach out to affect server to server communications.

• Legal tools can reach out further, but are higher effort. And companies don't always comply.

Different surveillance threats can be addressed at different levels. In general, earlier and more automated is easier, but legal tools have a longer reach, since they can touch systems that you don't have a network connection to, and that automated tools can't see.

Here's a list of places to make a difference, from earliest to latest.

• Don’t do a trackable activity (example: delete a surveillance app, don’t visit a surveilled location)

• Don’t send tracking info (example: block tracking scripts)

• Send tracking info that is hard to link to your real info (examples: use an auto-generated email address system like Firefox Relay, or churn tracking cookies with Cookie AutoDelete)

• Deny consent or send an opt out when doing a tracked activity: Global Privacy Control

• Object, exercise your right to delete, or opt out later, after data has been collected but before you are targeted (CCPA Authorized Agents, RtD automation tools like Mine)

Privacy tip: CCPA script

Delete your info from the largest surveillance firms. Here's a partial list. Later on we'll cover how to make a personalized list based on who has info on you.

• Acxiom

• Epsilon

• LiveRamp

• Neustar

• Oracle

• Verizon

I do these pretty quickly with Mutt and shell.

    ccpa privacy@example.com

The script is here: CCPA opt out, nerd edition

The sample letters, as templates, are in this privacy-docs Git repository.

Teaching the escalation path

If you're protected from a company by automated tooling, you can mostly relax. (This applies to most of the companies that can only track you by an identifier stored in a third-party cookie. Turn off third-party cookies and you're fine.)

If they accept a GPC, almost as good. Remember that GPC applies to all uses of your information, not just the current session.

Manual opt outs are where things start to get time-consuming. Most opt outs are either not compliant, or take advantage of loopholes in the regs to make you do extra work.

If they give you any grief on an opt out, and make opt-out as hard as a "right to know" (Rtk), then go ahead and escalate to an RtK. According to one study, a manual RtK can cost a company $1400 to handle.

Then if they give you grief on your RtK, you can file a complaint with the AG.

Be patient. Right now there are way more companies that have your personal info than companies that have a qualified CCPA manager. Be patient, and remember, any individual request might end quickly or turn into a long thread. The object is to change the expected payoffs for investments in surveillance relative to other investments.

Privacy tip: keep your Facebook account

Why does it matter if a company is sending my info to Facebook if I never log in to Facebook (or related sites?)

Just as the lowest point on a toxic waste site is generally a good place for a sampling well to check the progress of remediation, Facebook is a good place to sample for your personal info. It's a low point to which most of the firms who have your info will eventually share it.

Remember, get a Facebook account, but don't install the mobile app.

• click on your face

• select Settings and Privacy from the menu, then Settings

• Select "Ads" on the left column

• Under "Ad Preferences" select Ad Settings

(We're getting close...)

• Select "Audience-based advertising"

• Now scroll down and select "see all businesses"

You're probably going to get a lot of these. Let's have a look at a few.

First is a store where I actually bought something. Facebook's biggest advertiser at one point.

Then a DTC retailer that I've never heard of. Who sold them a list of "hot prospects?" I remember when this list used to be all car dealers.

Then two surveillance marketing companies—they can get a Right to Delete.

Finally another retailer, wait a minute, I haven't bought from this one either. The closest I have come to this one is the year that they had the LinuxWorld conference and the Talbots managers meeting at the same convention center in Boston. Remember, surveillance marketing and scam culture are two overlapping scenes—you can often see when a legit company has been sold a customer list that isn't. (maybe the scammer who got the car dealers has moved on to retailers?)

Of course, while you're there, building a list of who to send opt outs to, don't buy anything from the ads.

Privacy tip: Browser checkup

Remove extensions you aren't sure about. A lot of spyware and adware gets through.

Run the EFF checkup tool, "Cover your Tracks". Sad but true: the more custom your Linux setup is, the more fingerprintable you are. Technical protections won't cover you, browser protection needs to combine the technical and the legal.

Turn on Global Privacy Control. This will automate your California do not sell for sites you visit. Still not supported everywhere, but will have more effect as more companies come into compliance and more jurisdictions require companies to support GPC.

Content blocking. This next one is a tricky subject, so I'm not going to say anything about it. I'll just quote the FBI's public service announcement, Cyber Criminals Impersonating Brands Using Search Engine Advertisement Services to Defraud Users.

It all comes back to scam culture. Be warned when looking for an ad blocker on browser extension directories. A lot of them are spyware or malware.

Some privacy extensions I use include:

• ClearURLs to remove tracking parameters from URLs, and often speed up browsing by skipping a redirect that's just there for tracking.

• Cookie AutoDelete. Cleans up cookies after leaving a site. Not for everyone—it does create a little extra work for you by making you log in more often and/or manage the list of sites that can set persistent cookies. But it does let you click agree with less worry since the cookie you agreed to is going to be deleted.

• Facebook Container because, well, Facebook.

• NJS. This minimal JavaScript disable/enable button can be a good way to remove some intrusive data collection and growth hacking on sites where the real content works without JavaScript.

• Personal Blocklist is surprisingly handy for removing domains that are heavy on annoyances and surveillance, but weak on actual information, from search results.

• Privacy Badger blocks tracking scripts and will also turn on Global Privacy control for you, by default. This will not have much of an impact right away but will start to do more and more as more companies come into compliance. (More companies are required to comply with California privacy law than there are people who understand how to comply California privacy law.)

Privacy tip: Authorized Agent and related services

These are relatively new, still getting more and more effective as more companies come into compliance.

• Consumer Reports Permission Slip

• incogni

• Mine (not an authorized agent service, an automation tool for your own opt outs)

If you have tried an authorized agent service in the past and gotten nowhere, try again. Because of the Sephora case last year and the enforcement sweep this year, companies are staffing up. And hey, if you have made it this far, you could probably qualify for one of those privacy manager jobs.

Hope to see you at SCALE next year.

Conference program listing

Most of us have built up a collection of privacy tools, including browser extensions, settings, and services. But privacy threats keep changing, while at the same time new kinds of tools and services have become available. This talk will help maximize the value of your privacy toolset for today's best options, while helping you plan for the future.

Not only can you protect yourself as an individual, but your choices can help drive future investments away from surveillance into more productive areas. (Surveillance marketers and their investors think they can train us -- but with the right tools we can train them right back.)

Today most of us are at level 1 or 2 on privacy.

Level 1 You do something about privacy and take a mix of effective and ineffective actions

Level 2 You take mostly effective actions, but don't allocate your time and resources for maximum effect

Level 3 You take effective actions, efficiently selected and prioritized

Ready to level up? Now that California law codifies our right to check out how our personal information is shared, that means we have an opportunity to optimize our privacy toolkits and habits, and focus where it counts. We'll cover:

• Ad blocker myths and facts, and why the surveillance marketing business loves some ad blockers

• The most important privacy extensions for most people (and they're not what you'd think)

• Corporate surveillance about you that never touches your device, and how to reach out and block it

• Don't use the law on a problem that a tool can solve faster, but don't try to stretch a tool to solve a problem that needs the power of the law

• Where to add extra protection for special cases

Each of your individual privacy choices has a bigger impact that just the protection that it provides for you. In the long run, your real impact will be not so much in how you’re protected as an individual, but in how you help drive future investments away from surveillance and toward more constructive projects.

Recent related links

How Can Advertisers Stop Funding Piracy? Block Sellers, Not Domains

This Is What Happened When I Tried Using AdChoices (Spoiler: It Didn’t Work)

Notes on Privacy Extremism

The Cookie Mullet

WTF is the global privacy control?

Big changes coming for GDPR enforcement on Big Tech in Europe?

Websites Selling Abortion Pills Are Sharing Sensitive Data With Google

The Triumph Of Safari, by Magic Lasso

Google to SCOTUS: Liability for promoting terrorist videos will ruin the Internet

Schools sue social networks, claim they “exploit neurophysiology” of kids’ brains

How are programming languages similar, and how do they differ? Most arguments about this are based on lists of language features, but another way to tackle the question is to ask how different languages behave in practice. This paper does that by looking at how flaky tests in C, Go, Java, JavaScript, and Python are similar and how they differ. The authors find that things like concurrency are hard everywhere, but other things (like resource management) vary from language to language. They also found that less than a dozen strategies for fixing flaky tests accounting for 85% of fixes, which suggests that explicitly teaching people bug-fixing and refactoring patterns for tests would be beneficial. Finally, they found that people either fix flaky tests right away or leave them broken for a long time, suggesting either that some tests are useful and some are not, or that some projects have a culture of clean coding and others don't with little middle ground.

Keila Costa, Ronivaldo Ferreira, Gustavo Pinto, Marcelo d'Amorim, and Breno Miranda. Test flakiness across programming languages. IEEE Transactions on Software Engineering, pages 1–14, 2022. doi:10.1109/tse.2022.3208864.

Regression Testing (RT) is a quality-assurance practice commonly adopted in the software industry to check if functionality remains intact after code changes. Test flakiness is a serious problem for RT. A test is said to be flaky when it non-deterministically passes or fails on a fixed environment. Prior work studied test flakiness primarily on Java programs. It is unclear, however, how problematic is test flakiness for software written in other programming languages. This paper reports on a study focusing on three central aspects of test flakiness: concentration, similarity, and cost. Considering concentration, our results show that, for any given programming language that we studied (C, Go, Java, JS, and Python), most issues could be explained by a small fraction of root causes (5/13 root causes cover 78.07% of the issues) and could be fixed by a relatively small fraction of fix strategies (10/23 fix strategies cover 85.20% of the issues). Considering similarity, although there were commonalities in root causes and fixes across languages (e.g., concurrency and async wait are common causes of flakiness in most languages), we also found important differences (e.g., flakiness due to improper release of resources are more common in C), suggesting that there is opportunity to fine tuning analysis tools. Considering cost, we found that issues related to flaky tests are resolved either very early once they are posted (<10 days), suggesting relevance, or very late (>100 days), suggesting irrelevance.

Cat Hicks (who spoke at our first two live events), Carol Lee (who's speaking next month), and their colleague Morgan Ramsey have released a new report: Developer Thriving: The Four Factors that Drive Software Developer Productivity Across Industries. It's important, insightful work, and I hope it will spark useful conversations among your team.

Burnaby violin virtuoso Lache Cercel has distinguished himself as a global pioneer in Roma jazz.

It all began when the Bucharest-born musician was about 12 years old, dreaming up improvisational pieces in his head. However, he didn’t know back then what he was imagining could be called jazz. He was unfamiliar with this word because he was living behind the Iron Curtain.

“We didn’t have access to the tapes,” Cercel tells Pancouver over Zoom.

However, when he was 15 years old, Cercel was visiting Romania’s Black Sea area. And he was astonished to hear French jazz violinist Stéphane Grappelli on an ultra-short-wave radio.

The teenager had an epiphany. He suddenly realized that there actually was a musical genre beyond the classical, classical café, and Roma folk music that he was learning in Romania.

It also became immediately clear that he had imagined something very real.

“I was so happy in my heart,” Cercel recalls. “I said ‘it exists.’ It’s true. I’m not wrong. There is something else.”

Cercel shares this story in advance of his Sunday (March 19) concert with his Roma Swing Ensemble at Pyatt Hall in downtown Vancouver. It’s one of three shows in the Jaz’N’theViolin Concert Series, curated by jazz violinist and music historian Kit Eakle.

After hearing Grappelli for the first time, Cercel was eager to listen to more of this master of the violin. But his records weren’t available in Romania. So Cercel asked his cousin, then working at a Romanian airport, to encourage a pilot to find Grappelli’s music on one of his journeys to the West.

“They brought me a tape,” the musician recalls. “I used to study with that tape over and over. I made duplicates and I learned all the improvisations, thinking they were songs.”

Cercel reached musical pinnacle in Romania

He became hooked on swing as he rose to prominence as a classical musician in Bucharest. In 1986, the Romanian government conferred on Cercel an “Artist of the People” citation in recognition of his brilliance.

Cercel chats about his life with the same enthusiasm that he exhibits on stage. And what a life it’s been—marked by interrogations by Romanian officials, immigration to Canada, and performances at international festivals.

To fully appreciate his journey, it’s essential to understand the roots of the Roma people, as well as their connection to different music.

The Roma are a widely dispersed Indo-Aryan ethnic group. Their roots go back to North India, primarily Rajasthan.

Well before the British slave trade brought Africans to the Caribbean and North America, Roma were considered property in the European principalities of Wallachia and Modovia.

Back as the 14^th century, the Crown, Orthodox church, and upper-class landowners treated the darker-skinned Roma people as slaves. And Roma remained enslaved in this part of the world until the mid 1850s.

In 1862, Wallachia and Moldovia united to become the Romanian United Principalities. Therefore, a history of oppression is as much a part of Cercel’s heritage as it is for African Americans.

“I find, myself, similarities between the Black people and the Roma people, who were in the same slavery,” Cercel says. “Maybe we were sold to Africa, too, from Rajasthan.”

He says this because the Roma people were also present across North Africa.

Violin is part of American Black culture

Through the concert series, Eakle hopes to raise awareness about the violin’s deep roots in African-American life and its connection to jazz music.

“There are records back to the 1600s of there being Black fiddle players,” Eakle says in the same Zoom call.

In fact, Eakle notes that the violin was very much an African instrument. He adds that this may be why enslaved Black people picked it up so quickly in the U.S. South.

“Probably, a major privileged position on the plantation was to be the fiddle player and to play for dances,” he states.

Eakle adds that during the emergence of blues music, many African-American fiddlers played in the Mississippi Delta area. For example, he says, Muddy Waters played his first gig with fiddle player Henry “Son” Sims.

Because the Roma people also have a rich history with the violin and were also enslaved, Eakle included Lache Cercel and his Roma Swing Ensemble in the series.

There’s another thing that Roma and African Americans have in common. Some have demanded reparations to Roma people as compensation for slavery they endured on Romanian lands. Also in North America, many have called for reparations for Black Americans.

Cercel has varied musical lineage

One of Cercel’s grandfathers was a talented violinist living in France before moving to Romania prior to the Second World War.

“After the Communists took over, the government decided to make him a teacher,” he says.

One of his grandfather’s students was the son of the culture minister. Another student was his father, which is how his dad met his mother.

“I grew up in this family where I have two sides of music—classical, from my grandpa, who was the one who put the violin in my hands when I was six,” Cercel reveals. “Then, my father trained me to also have knowledge in folk and Roma music.”

Cercel has woven these musical strands together with his love of improvisation and Grappelli to forge a new path in Roma jazz.

This shouldn’t come as a huge surprise because the Roma people have been shaping musical traditions wherever they went.

As far back as the 11^th century, they were part of troubadour ensembles. They spread music around Europe after music notation was invented by Guido d’Arrezo. Even though the Roma were considered to be slaves, Cercel says that the emperor in Rome allowed them to travel because there weren’t enough of them to influence other people.

Yet despite facing such intense discrimination, they still had a considerable impact. Composers such as Franz Liszt, Ludwig van Beethoven, Franz-Josef Haydn, and Franz Schubert are believed to have incorporated Romani influences into their compositions. And Roma musicians played a major role in the development of flamenco in 19th-century Andalusia in southern Spain.

Interrogated by the Securitate

Cercel points out that the Roma also influenced the development of classical café music in the Austro-Hungarian Empire in Central Europe. And it was related to the Ottomans’ love of coffee.

He says that after the Ottomans were expelled from Vienna in 1683, they left tremendous amounts of this beverage. That created a conundrum for café owners and the new rulers.

“They said, ‘What are you going to do with this coffee? Please do not ask us to listen to classical music because it’s too formal and people aren’t wanting to dress up and get prepared,’ ” Cercel says.

He adds that they also didn’t want folk music in the cafés because it was too lively—and the women, in particular, weren’t interested in that. Cercel maintains that this led to the growth of salon music, then popular in the Netherlands and England, which could be enjoyed while eating a light dinner.

“The name became classical café concert,” Cercel says. “It was interpreted in the classical manner. Czardas, which everybody knows, is part of that music.”

He goes on to say that Johannes Brahms’s Hungarian Dance compositions and Boulangère music were also part of that tradition. Cercel grew up with it, as well as learning about French opera composer George Bizet, French music theorist Anton Reicha, and folkloric Roma music.

Cercel’s departure from Romania came with considerable discomfort. He recalls being interrogated intensely by Romanian government’s Securitate before coming to Canada in a musical exchange program in 1987 and settling in Victoria. He decided not to return and as a new immigrant, he performed with Original Balkan Jam.

Two years later, the Iron Curtain fell and the dictator, Nicolae Ceaușescu, was overthrown.

In 2019, Lache Cercel and the Roma Swing Ensemble performed with accordionist Bob Kozak.

Roma Swing Ensemble blends jazz with classical

To this day, Cercel describes himself as “apolitical”. He points out that the word “Rom” actually means “human”, but his people were often held in lower esteem. He believes that this can affect how people think.

While he appreciates the love that he’s felt from friends in Canada and cherishes his freedom, there is still a residue of fear.

“That’s why I don’t touch computers,” Cercel says. “I don’t write too much because in my interrogation in Romania, they always told you to write it down.”

In 1997, he met jazz guitarist and viola player Don Ogilvie. Together, they formed the Roma Swing Ensemble, which blends jazz with classical, Roma folk songs, and other world music. In 2007, the band toured China.

Over the years, they’ve performed with guitarist Stephen Nikleva, bassists Sam Soichet and Kyle Hagen, and drummer Paul Townsend, along with vocalists Jennifer Layne, Merrier Ben Amor, and Laura Crema.

Their musical influences include 20^th-century Romani-Belgian jazz guitarist Django Reinhardt and Romani-Romanian violin virtuoso and composer Grigoraș Dinicu.

Cercel says that his upcoming concert will reflect the Roma people’s contributions to world music. To him, that’s a metaphor for how we should be looking at society as a whole.

“Our wish is to understand how we are all one nation—and to bring peace and harmony to the world by seeing this variety,” Cercel says.

For more information about the Jaz’N’theViolin Concert Series, visit MusicKit.com. Tickets are available here. Follow Charlie Smith on Twitter @charliesmithvcr. Follow Pancouver on Twitter @PancouverMedia.

The post Roma violinist Lache Cercel brings world of music infused by centuries of oppression to Jaz’N’theViolin Concert Series appeared first on Pancouver.

By Andrew Cheng, Simon Fraser University

At the 95th Academy Awards, Daniel Scheinert and Daniel Kwan’s Everything Everywhere All at Once took home wins in acting, editing and directing categories, and also won the coveted best picture award.

The quirky and expansive sci-fi action film is famed for its absurdist take on the idea of a multiverse, yet it is still grounded in realness and humanity: In one masterfully choreographed scene in Everything Everywhere, three languages are spoken by three generations of one chaotic immigrant family over dinner.

As a linguist, I am interested in how screenwriters and directors use language in film to do much more than just convey dialogue: language and subtitles also provide characterization, establish the tone of a scene and ground fictional stories in reality.

Increasing use of non-English languages and subtitles demonstrates both a trend toward linguistic realism in Hollywood and also broader acceptance of linguistic diversity in a globalized society.

But filmmakers must negotiate the tension between narrative goals and audience engagement. By examining different ideologies held by filmmakers and audiences alike, we can also still sometimes find long-standing associations between foreign language and villainy.

Jumping for Oscar winning joy! Presenting your Best Original Screenplay winners: the Daniels. #Oscars95

Photo Credit: @landonnordeman pic.twitter.com/VqM4dOmJkw

— The Academy (@TheAcademy) March 13, 2023

Languages breaking through

As South Korean filmmaker Bong Joon-ho described in his Golden Globes acceptance speech for 2019’s Parasite, when audiences move past the “one-inch tall barrier of subtitles”, a richer world of film awaits.

Telling authentic stories in a modern, globalized and multicultural world means fairly portraying the languages spoken and signed all throughout our society. Hence, at times, the need for subtitles.

In another scene in Everything Everywhere, spoken language is eschewed altogether: two rocks converse silently while gazing over a barren canyon, with black and white subtitles appearing on a bright blue sky. This simple yet emotionally resonant scene was somehow enough to move many viewers to tears.

Although Todd Field’s Tár did not clinch any Oscar wins this year, star Cate Blanchett notched multiple other acting awards for her role as the titular orchestra conductor. In some of the film’s best scenes, the maestro uses only German during orchestra rehearsals and subtitles are dropped completely.

Non-Germanophone audiences might not understand the words Blanchett says, but they are instead encouraged to focus on the visual aspects of the character’s communication: her facial expressions, physical tics, and tremors and the building tension, both visible and audible, between her ego and everyone around her.

The Oscar performance of “Naatu Naatu,” from the Indian Telugu-language blockbuster RRR, which received a standing ovation, was evidence that performances can cross language boundaries by leaning into creative elements far beyond words.

Two sides of language representation

The flip side of positive aspects of linguistic representation is that filmmakers, free to use the tool of multilingualism to provide characterization and establish tone, and even to showcase threatened languages, are also free to use it to entrench certain linguistic stereotypes, namely the association between foreignness and villainy.

The warriors of the underwater Talokan kingdom in Ryan Coogler’s Black Panther: Wakanda Forever never speak a word of English. In this Marvel Cinematic Universe story, the fictional kingdom of Atlantis was repositioned as a Mesoamerican-inspired society.

The language of Talokan is the real-world language of Yucatec Maya, spoken by Indigenous people in present-day Guatemala, Belize, and southern Mexico.

Sharing an Indigenous language

Yucatec Maya is stigmatized and marginalized in the geographic areas where it is traditionally spoken, one of the long-lasting consequences of colonization.

But Coogler’s creative decision to use Yucatec Maya as the language of Talokan, as well as to borrow heavily from Mayan and Aztec cultures for visual inspiration, has delighted some Maya-speaking Indigenous moviegoers who say they saw their culture represented and are happy to see the language widely shared.

Two strong minor characters, Namora and Attuma, speak only in Yucatec Maya throughout the entire movie, with English subtitles in an elegant blue script.

Archetypes of antagonism?

Yet, these characters are the primary villains of the story, portrayed as powerful and physically otherworldly. Thus, while the characters are not one-dimensional, their language seems to narratively underscore certain archetypes of antagonism, beyond simply showcasing a real Indigenous language of the world.

This is especially noticeable when contrasted with the protagonists, the mostly anglophone Wakandans. Although the language of Wakanda is the real-world language of Xhosa, spoken in South Africa, in Wakanda Forever, it was mostly relegated to greetings and asides. Instead, the protagonists speak a kind of pan-African accented English. Unlike in James Cameron’s Avatar: The Way of Water, viewers are never explicitly told that what the audience hears has been “translated” for us.

Refreshing to see, hear more languages

In Avatar, which took the Oscar for best visual effects, the protagonists all speak Na’vi, the language of the species living across the fictional Pandora.

Most of the language has been “translated” in the mind of the narrator, Jake Sully. In a neat expositional trick, the main character narrates the beginning of the film and explains that he learned the alien language well enough that it just sounds like English to him.

This way, audiences won’t have to read for the entire film, and the heroes (and actors) can speak only English without sacrificing the “realism” of the science-fictional universe.

It has been refreshing to see, in recent years, many productions in film achieve critical and commercial success in the anglophone world despite being in non-English languages and using subtitles. There’s much more room for films to highlight both linguistic diversity as well as authentic characterization of characters who speak and sign without resorting to potentially harmful tropes.

And I’m sure audiences wouldn’t mind hearing more Na’vi in Avatar 3.

Andrew Cheng, Postdoctoral Researcher, Linguistics, Simon Fraser University

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The post <i>Everything Everywhere All at Once</i> and other Oscars 2023 films show a trend towards linguistic realism in Hollywood appeared first on Pancouver.

My general rule for writing about new developments in the Haskell compiler or core libraries is to wait until they’ve been available for a year or so. This avoids putting a reader into a situation where the tutorial they’re reading involves features that they can’t use yet, perhaps because they’re not yet able to install the latest compiler, or because they’re working within a larger codebase where not all of their dependencies support the latest stuff yet. GHC2021 has now been available for about a year and a half, and there are now three major GHC releases — 9.2, 9.4, and 9.6 (as of just a few days ago) — so I’m quite happy to go ahead and begin giving a new recommendation: Use GHC2021 as the language for your Haskell projects.

Share

My skeletal default Cabal file looks like this nowadays:

cabal-version: 3.0
name: my-package-name
version: 0.0.0.0

common base
    default-language: GHC2021
    ghc-options: -Wall
    build-depends: base

library
    import: base
    hs-source-dirs: library
    exposed-modules: ...

The available choices for default-language are Haskell98, Haskell2010, and GHC2021.

A brief history of 27 years

Haskell has gone through a number of versioning schemes as things have evolved over the decades. The recorded history of Haskell begins with Haskell 1.0 in 1990. A few other releases went on in this style; many are lost to time, though we can find Haskell 1.3 from 1996. Then at the end of the decade, a new language standard was published: Haskell 98, this time named for the year in which it was written. With the exception of some lesser revisions a few years later, Haskell 98 was the definition of “official” Haskell for another decade.

There were two major Haskell compilers — Hugs and GHC — and, because coders gonna code, new features continued to be introduced into each of them. I wasn’t around in this era and I don’t know much about Hugs, so I will speak only of GHC. By default, GHC was a Haskell98 compiler; but if you compiled with the -fglasgow-exts flag, then you got all the new stuff that GHC added that wasn’t in the language specification. Some fun additions during the early years were:

• Scoped type variables were added in GHC 4.02 (1998) and expanded upon in GHC 6.4 (2005)

• Functional dependencies first appeared in GHC 5.00 (2001)

• Generalized newtype deriving is from GHC 5.04 (2002)

• Template Haskell, GHC 6.0 (2003)

• GADTs, GHC 6.4 (2005)

GHC 6.4 was the first version to include the package manager, Cabal. It seems to have been recognized around this time that Haskell was losing sight of the dream of having multiple compilers. If package authors increasingly relied on compiler-specific features — that is, if everybody just kept publishing packages that could only be built with the -fglasgow-exts flag — then there would effectively be two programming languages, GHC Haskell and Hugs Haskell, and libraries could not be shared between them.

Thus the language pragma appears in GHC 6.6. Rather than declaring “I require the GHC features,” this allows a package to specify precisely which non-standard features it requires. It is feasible, then, for two compilers to incorporate some of the same features, creating opportunity for library interoperability without requiring that the two compilers stay perfectly in sync.

In 2010, we got a new official language specification: Haskell 2010. There hasn’t been once since. I can’t speak for anyone else, but my sense is: We have lost the dream. Modern Haskell is GHC. This is not to say that the entire system of having a stable language specification with many named extensions is a pointless exercise; I think the extension mechanism provides excellent points of reference for discussing language features, tracking how the compiler has changed over time, and seeing which features people are using. But it does seem clear that the specification document is losing relevance, and that this has resulting in a general lack of motivation to work on a new version of it. Language extensions are now the thing at the top of everyone’s mind.

It was clear that we needed a new language version, though. Because over the past twenty or so years, Haskell has been accumulating a hefty pile of language extensions, and many of them have become de facto standards. But if nobody wants to take on the substantial task of writing (or funding the writing of) a sequel to the 300-page document that is Haskell 2010, then it just isn’t happening.

So what we got instead in 2021 was sort of a new language specification, but defined in the laziest way possible. (Not a criticism.) GHC 2021 is defined as Haskell 2010 plus a list of language extensions.

Changes in GHC 2021

Below we give a complete list of the differences between Haskell 2010 and GHC 2021. The list of extension names can be found the GHC manual. If you haven’t written Haskell in ten years — or if you’ve been using Haskell and need an overview of what language features you don’t have to explicitly turn on anymore — the remainder of this article outlines what you need to know to get up-to-date.

Expanding deriving power

The set of stock-derivable classes (e.g. Eq, Ord, Show) has been expanded to include Functor, Foldable, Traversable, Generic, Data, and Lift.

Standalone deriving clauses are now permitted. For example,

data Ix a = Ix Int a deriving Eq

can now be written as

data Ix a = Ix Int a

deriving instance Eq a => Eq (Ix a)

A newtype is now permitted to receive instances derived from its underlying type. In the following example, Show is a stock-derivable class, but Semigroup and Monoid are newtype-derived using this feature.

import Data.Foldable (fold)
import Data.Monoid (Sum (..))
import Numeric.Natural (Natural)

newtype Total = Total (Sum Natural) 
  deriving (Show, Semigroup, Monoid)

λ> fold [Total 5, Total 10, Total 3]
Total (Sum {getSum = 18})

More explicit type information

The forall keyword is added, letting you explicitly bind type variable names. This is called existential quantification, and it makes several other features possible.

If you bind a type variable with forall in parentheses within a larger type, that’s called a higher-rank type, and it is not possible in Haskell 2010.

If you bind a type variable with forall in some definition’s type signature, you can refer to that type variable within the lexical scope of the definition; this feature is called scoped type variables.

One way to use a scoped type variable is in a type application, which is an important tool for reducing polymorphism to resolve type ambiguities using the @ keyword. For example:

{-# language AllowAmbiguousTypes #-}

import Data.List (intercalate)

showAll :: forall a. (Enum a, Bounded a, Show a) => String
showAll = intercalate ", " $ map (show @a) $
    enumFromTo minBound maxBound

λ> showAll @Ordering
"LT, EQ, GT"

Type signatures are now allowed on methods within instance definitions. For example:

import Data.Semigroup (Semigroup (..))

data Pair a = Pair a a

instance (Semigroup a) => Semigroup (Pair a) where
   (<>) :: Pair a -> Pair a -> Pair a
   Pair a b <> Pair c d = Pair (a <> c) (b <> d)

The line beginning with “(<>) ::” is an instance signature. It is not permitted in Haskell 2010 but is allowed in GHC 2021. The instance signature is not allowed to contradict the polymorphic type of (<>) as defined by the Semigroup class; it only reiterates the type for clarity.

Kind annotations and kind signatures now supported. These look just like type annotations and type signatures, using the :: keyword. The difference is that while a type annotation specifies the type of an expression, a kind annotation specifies the kind of a type. For example, consider the following datatype definition.

import Data.Kind (Type)

newtype NumberAction context number =
  NumberAction (context number)

There are two ways to make the kinds of its type parameters explicit. One is with kind annotations:

newtype NumberAction (context :: Type -> Type) (number :: Type) =
  NumberAction (context number)

The other is by adding a standalone kind signature:

type NumberAction :: (Type -> Type) -> Type -> Type

newtype NumberAction context number =
  NumberAction (context number)

In this example, writing the extra kind information has no consequence; it serves only to make things explicit. Kind annotations are sometimes necessary, however, because kind polymorphism is now enabled by default. This means that, in the absence of an explicit kind annotation, the compiler will now infer polymorphic kinds when possible. This is generally something you will only encounter if you are doing something exotic.

New ways to write numbers

There are some new ways to write numeric literals. Underscores can be inserted into a number wherever you like. Typically we use this in the same way you would use commas in English to increase readability: for example, we write 1_000_000 to represent one million (1,000,000). In contrast with 1000000, the underscores make it is easier to comprehend at a glance how many zeroes there are.

Binary literals are now allowed, prefixed with 0b. The list [1, 2, 3, 4, 5] can be written in binary like so:

[0b0, 0b1, 0b10, 0b11, 0b100, 0b101]

Floating-point literals can now be expressed using hexadecimal notation, prefixed with 0x. For example, a and b below are equivalent:

a = 0xb.fe
b = 11 + (254 / 256)

Slightly nicer-looking imports

It is now permitted to move the qualified keyword to the end of an import declaration. This can lead to more compact import lists in some aligned formatting styles. For example,

import           Data.List (intercalate)
import qualified Data.List as List

can now be written as:

import Data.List (intercalate)
import Data.List qualified as List

Bits of abbreviated syntax

\(a -> (a,b)) can be written as (a,) and \(b -> (a,b)) can be written as (,b). The abbreviated forms are called tuple sections.

Postfix operators are now allowed. It resembles the syntax for operation sections, but is for operators that are unary rather than binary. This feature is not commonly used.

import Data.IORef

(++) :: Num a => IORef a -> IO ()
(++) i = modifyIORef' i (+ 1)

λ> do { i <- newIORef 0 ; (i ++) ; (i ++) ; readIORef i }
2

Bang patterns allow you to prefix a variable pattern with ! to force evaluation of the matched value to weak head normal form, equivalent to applying seq.

When you are constructing or deconstructing using record syntax and a local variable has the same name as a corresponding record field, the name no longer has to be written twice. This feature is called named field punning. Consider the following example that does not use field puns:

data ShippingInfo =
    ShippingInfo{ name :: String, address :: String }

readShippingInfo :: IO ShippingInfo
readShippingInfo = do
    name <- getLine
    address <- getLine
    pure ShippingInfo{ name = name, address = address }

displayShippingInfo :: ShippingInfo -> String
displayShippingInfo ShippingInfo{ name = name, address = address} =
    "Name: " <> name <> "\n" <>
    "Address: " <> address <> "\n"

Using named field puns, these functions can be written more conveniently as:

readShippingInfo :: IO ShippingInfo
readShippingInfo = do
    name <- getLine
    address <- getLine
    pure ShippingInfo{ name, address }

displayShippingInfo :: ShippingInfo -> String
displayShippingInfo ShippingInfo{ name, address } =
    "Name: " <> name <> "\n" <>
    "Address: " <> address <> "\n"

Support for Void

Case expressions without any cases are now permitted. This is generally needed only when dealing with the Void type, which has no constructors. For example, the absurd function is defined using an empty case expression:

absurd :: Void -> a
absurd x = case x of { }

Another limitation pertaining to types with no data constructors is relaxed: Previously they weren’t allowed to have a deriving clause, and this is now permitted. For example, the definition of the Void type itself now looks like this:

data Void
  deriving (Eq, Data, Generic, Ord, Read, Show)

Multi-parameter classes

Perhaps the most bold change in GHC 2021 is that multi-parameter type classes are now permitted. For example, the MonadError class from the mtl package, reproduced below, has two parameters, e and m.

class (Monad m) => MonadError e m | m -> e where
    throwError :: e -> m a
    catchError :: m a -> (e -> m a) -> m a

Like many multi-parameter type classes, the MonadError class also has a functional dependency (the “ | m -> e” at the end of the first line). This is not enabled by GHC 2021; it still requires separately enabling the FunctionalDependencies language extension.

Relaxed rules for classes, instances, constraints

Flexible contexts and flexible instances are now permitted; these extensions lift some restrictions that were previously imposed upon what sorts of typeclass constraints were permissive.

It is now permissible to use a type alias in an instance definition. For example, consider the following class and instances from Relude. Since String is an alias for [Char], the last instance is not permitted in Haskell 2010, but it is fine in GHC 2021.

class ToText a         where toText :: a -> Text
instance ToText Text   where toText = id
instance ToText LText  where toText = LT.toStrict
instance ToText String where toText = T.pack

Class methods were previously not allowed to have constraints on class variables, such as the (Eq a) contraint in the example below. This restriction has been deemed unhelpful, and so it has been removed.

class Seq s a where
  fromList :: [a] -> s a
  elem     :: (Eq a) => a -> s a -> Bool

A type context (the stuff to the left of the => arrow) can now be anything of the Constraint kind. This includes, most notably, constraint aliases. For example, we can define the following:

import Data.Kind (Type, Constraint)

type All :: Type -> Constraint
type All a = (Enum a, Bounded a)

This defines All to be a constraint satisfied by types that belong to both Enum and Bounded. This alias can then be used in the context of a type signature such as:

all :: All a => [a]
all = enumFromTo minBound maxBound

A new way to write datatypes

There is a new form of data declaration. Instead of an = symbol, the where keyword opens a block which contains a list of type signatures for all the datatype’s constructors. The following two type definitions are equivalent:

data Maybe a = Nothing | Just a

data Maybe a where
    Nothing :: Maybe a
    Just :: a -> Maybe a

Which format you choose is a matter of preference. The old form is usually more concise. The new form is called GADT syntax because it is sometimes more convenient for writing the sorts of complicated datatypes that require the GADTs (Generalised Algebraic Data Types) language extension. GADTs were too exotic to standardize into GHC 2021, but this syntax designed for it is innocuous enough to be now supported by default.

The writing will continue until I run out of money!

Subscribe now

Happy Haskelling, and everybody stay (type-) safe out there.

Running LLaMA 7B on a 64GB M2 MacBook Pro with llama.cpp

I got Facebook's LLaMA 7B to run on my MacBook Pro using llama.cpp (a "port of Facebook's LLaMA model in C/C++") by Georgi Gerganov. It works! I've been hoping to run a GPT-3 class language model on my own hardware for ages, and now it's possible to do exactly that. The model itself ends up being just 4GB after applying Georgi's script to "quantize the model to 4-bits".

The open release of the Stable Diffusion image generation model back in August 2022 was a key moment. I wrote how Stable Diffusion is a really big deal at the time.

People could now generate images from text on their own hardware!

More importantly, developers could mess around with the guts of what was going on.

The resulting explosion in innovation is still going on today. Most recently, ControlNet appears to have leapt Stable Diffusion ahead of Midjourney and DALL-E in terms of its capabilities.

It feels to me like that Stable Diffusion moment back in August kick-started the entire new wave of interest in generative AI - which was then pushed into over-drive by the release of ChatGPT at the end of November.

That Stable Diffusion moment is happening again right now, for large language models - the technology behind ChatGPT itself.

This morning I ran a GPT-3 class language model on my own personal laptop for the first time!

AI stuff was weird already. It's about to get a whole lot weirder.

LLaMA

Somewhat surprisingly, language models like GPT-3 that power tools like ChatGPT are a lot larger and more expensive to build and operate than image generation models.

The best of these models have mostly been built by private organizations such as OpenAI, and have been kept tightly controlled - accessible via their API and web interfaces, but not released for anyone to run on their own machines.

These models are also BIG. Even if you could obtain the GPT-3 model you would not be able to run it on commodity hardware - these things usually require several A100-class GPUs, each of which retail for $8,000+.

This technology is clearly too important to be entirely controlled by a small group of companies.

There have been dozens of open large language models released over the past few years, but none of them have quite hit the sweet spot for me in terms of the following:

• Easy to run on my own hardware
• Large enough to be useful - ideally equivalent in capabilities to GPT-3
• Open source enough that they can be tinkered with

This all changed yesterday, thanks to the combination of Facebook's LLaMA model and llama.cpp by Georgi Gerganov.

Here's the abstract from the LLaMA paper:

We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.

It's important to note that LLaMA isn't fully "open". You have to agree to some strict terms to access the model. It's intended as a research preview, and isn't something which can be used for commercial purposes.

In a totally cyberpunk move, within a few days of the release, someone submitted this PR to the LLaMA repository linking to an unofficial BitTorrent download link for the model files!

So they're in the wild now. You may not be legally able to build a commercial product on them, but the genie is out of the bottle. That furious typing sound you can hear is thousands of hackers around the world starting to dig in and figure out what life is like when you can run a GPT-3 class model on your own hardware.

llama.cpp

LLaMA on its own isn't much good if it's still too hard to run it on a personal laptop.

Enter Georgi Gerganov.

Georgi is an open source developer based in Sofia, Bulgaria (according to his GitHub profile). He previously released whisper.cpp, a port of OpenAI's Whisper automatic speech recognition model to C++. That project made Whisper applicable to a huge range of new use cases.

He's just done the same thing with LLaMA.

Georgi's llama.cpp project had its initial release yesterday. From the README:

The main goal is to run the model using 4-bit quantization on a MacBook.

4-bit quantization is a technique for reducing the size of models so they can run on less powerful hardware. It also reduces the model sizes on disk - to 4GB for the 7B model and just under 8GB for the 13B one.

It totally works!

I used it to run the 7B LLaMA model on my laptop last night, and then this morning upgraded to the 13B model - the one that Facebook claim is competitive with GPT-3.

Here are my detailed notes on how I did that - most of the information I needed was already there in the README.

As my laptop started to spit out text at me I genuinely had a feeling that the world was about to change, again.

I thought it would be a few more years before I could run a GPT-3 class model on hardware that I owned. I was wrong: that future is here already.

Is this the worst thing that ever happened?

I'm not worried about the science fiction scenarios here. The language model running on my laptop is not an AGI that's going to break free and take over the world.

But there are a ton of very real ways in which this technology can be used for harm. Just a few:

• Generating spam
• Automated romance scams
• Trolling and hate speech
• Fake news and disinformation
• Automated radicalization (I worry about this one a lot)

Not to mention that this technology makes things up exactly as easily as it parrots factual information, and provides no way to tell the difference.

Prior to this moment, a thin layer of defence existed in terms of companies like OpenAI having a limited ability to control how people interacted with those models.

Now that we can run these on our own hardware, even those controls are gone.

How do we use this for good?

I think this is going to have a huge impact on society. My priority is trying to direct that impact in a positive direction.

It's easy to fall into a cynical trap of thinking there's nothing good here at all, and everything generative AI is either actively harmful or a waste of time.

I'm personally using generative AI tools on a daily basis now for a variety of different purposes. They've given me a material productivity boost, but more importantly they have expanded my ambitions in terms of projects that I take on.

I used ChatGPT to learn enough AppleScript to ship a new project in less than an hour just last week!

I'm going to continue exploring and sharing genuinely positive applications of this technology. It's not going to be un-invented, so I think our priority should be figuring out the most constructive possible ways to use it.

What to look for next

Assuming Facebook don't relax the licensing terms, LLaMA will likely end up more a proof-of-concept that local language models are feasible on consumer hardware than a new foundation model that people use going forward.

The race is on to release the first fully open language model that gives people ChatGPT-like capabilities on their own devices.

Quoting Stable Diffusion backer Emad Mostaque:

Wouldn't be nice if there was a fully open version eh

It's happening already...

I published this article on Saturday 11th March 2023. On Sunday, Artem Andreenko got it running on a RaspberryPi with 4GB of RAM:

I've sucefully runned LLaMA 7B model on my 4GB RAM Raspberry Pi 4. It's super slow about 10sec/token. But it looks we can run powerful cognitive pipelines on a cheap hardware. pic.twitter.com/XDbvM2U5GY

- Artem Andreenko 🇺🇦 (@miolini) March 12, 2023

Then on Monday, Anish Thite got it working on a Pixel 6 phone (at 26s/token):

@ggerganov's LLaMA works on a Pixel 6!

LLaMAs been waiting for this, and so have I pic.twitter.com/JjEhdzJ2B9

- anishmaxxing (@thiteanish) March 13, 2023

And then a research lab at Stanford released Alpaca - an instruction fine-tuned version of the model. I wrote more about that in a follow-up post: Stanford Alpaca, and the acceleration of on-device large language model development.

Follow my work

Everything I write on my blog goes out in my Atom feed, and I have a very active Mastodon account, plus a Twitter account (@simonw) where I continue to post links to new things I've written.

I'm also starting a newsletter at simonw.substack.com. I plan to send out everything from my blog on a weekly basis, so if email is your preferred way to stay up-to-date you can subscribe there.

More stuff I've written

My Generative AI tag has everything, but here are some relevant highlights from the past year:

• A Datasette tutorial written by GPT-3 - 31 May 2022
• How to use the GPT-3 language model - 5 Jun 2022
• First impressions of DALL-E, generating images from text - 23 Jun 2022
• Using GPT-3 to explain how code works - 9 Jul 2022
• Stable Diffusion is a really big deal - 29 Aug 2022
• Exploring the training data behind Stable Diffusion - 5 Sep 2022
• Prompt injection attacks against GPT-3 - 12 Sep 2022
• A tool to run caption extraction against online videos using Whisper and GitHub Issues/Actions - 30 Sep 2022
• Is the AI spell-casting metaphor harmful or helpful? - 5 Oct 2022
• A new AI game: Give me ideas for crimes to do - 4 Dec 2022
• AI assisted learning: Learning Rust with ChatGPT, Copilot and Advent of Code - 5 Dec 2022
• How to implement Q&A against your documentation with GPT3, embeddings and Datasette - 13 Jan 2023
• Bing: "I will not harm you unless you harm me first" - 15 Feb 2023
• I talked about Bing and tried to explain language models on live TV! - 19 Feb 2023
• In defense of prompt engineering - 21 Feb 2023
• Thoughts and impressions of AI-assisted search from Bing - 24 Feb 2023
• Weeknotes: NICAR, and an appearance on KQED Forum - 7 Mar 2023
• ChatGPT can't access the internet, even though it really looks like it can - 10 Mar 2023

ChatGPT's API is So Good and Cheap, It Makes Most Text Generating AI Obsolete

Max Woolf on the quite frankly weird economics of the ChatGPT API: it's 1/10th the price of GPT-3 Da Vinci and appears to be equivalent (if not more) capable. "But it is very hard to economically justify not using ChatGPT as a starting point for a business need and migrating to a more bespoke infrastructure later as needed, and that’s what OpenAI is counting on. [...] I don’t envy startups whose primary business is text generation right now."

Via Hacker News

I've successfully run LLaMA 7B model on my 4GB RAM Raspberry Pi 4. It's super slow about 10sec/token. But it looks we can run powerful cognitive pipelines on a cheap hardware.

— Artem Andreenko

We introduce Alpaca 7B, a model fine-tuned from the LLaMA 7B model on 52K instruction-following demonstrations. Alpaca behaves similarly to OpenAI’s text-davinci-003, while being surprisingly small and easy/cheap to reproduce (<600$).

— Alpaca: A Strong Open-Source Instruction-Following Model

This is a long discussion about using something like chatGPT do do grading tasks, and is therefore a good demonstration of the need for people to understand that there are different types of AI that perform different tasks. I outline them here. In a nutshell, grading is a classification task - this is a recognition problem (which is what a grammar-checker is doing). The provision of feedback is a requirement for an explanation of that classification. Maybe generative AI could help here, but chatGPT wasn't designed for anything like this. There's no point in asking it to "offer advice on how to revise and improve the following text" if the AI has no real mechanism for recognizing and classifying good or bad content. People should stop treating the latest AI thing as though it were a tool for doing everything.

The story: "Koko, a mental health nonprofit, found at-risk teens on platforms like Facebook and Tumblr, then tested an unproven intervention on them without obtaining informed consent. 'It's nuanced,' said the founder." I don't think it's particularly nuanced in this instance, and that most people world agree the research was unethical. But it's not exactly like there are ethical protocols for developing AI-enabled products and services. Sure, there are guidelines for the ethical development of AI, but not really for when the AI should be applied in a health, educational or social setting. I'm not talking about whether the AI is ethically sourced, or creative, or bland, or whether it is accurate, but whether it is a good idea to use intervention X in application area Y - we don't have a research protocol for testing that, or at least, I haven't found one.

Based on analysis by economics professor Caitlin Myers, FiveThirtyEight provides a hypothetical map that shows how access changes in terms of distance to travel and increase in patients at nearby clinics:

New bans will have outsized impacts on who can get an abortion, how far they have to drive for it and how long they have to wait for an appointment. A new analysis by Caitlin Myers, an economics professor at Middlebury College who studies abortion, illustrates how abortion access could continue to dwindle this year if key states like Florida and North Carolina pass additional restrictions.

You can select any combination of possible states and the map updates to show the shifts. Roll over any county to see the nearest county that provides access. Useful and informative.

Tags: abortion, ban, FiveThirtyEight

amnhnyc:

CoelaCANTh? More like coelaCAN! Scientists once thought that coelacanths went extinct some 70 million years ago during the Cretaceous. Then, one turned up in 1938 when a fisherman caught a living coelacanth off the coast of South Africa. Plot twist: Decade laters, another species of coelacanth was caught.

Did you know? These ancient “fish” are actually more closely related to land animals. Their paired fins are lobelike and have joined bones, like arms and legs. Coelacanths live in deep saltwater environments and can grow up to 6 feet (2 m) long. They have an organ filled with a jelly-like substance located near the front of their heads, a trait found in no other vertebrate.

Scientists think this “rostral organ” helps the coelacanth hunt, allowing it to detect low-frequency electrical signals emitted by prey. What’s more? Unlike most other fish, coelacanths give birth to live young—a fact that remained unknown until 1975, when the Museum’s first coelacanth specimen was dissected and found to be pregnant with five embryos.

Photo 1: A living coelacanth (Latimeria chalumnae), wrecklessmarine, CC BY-NC 4.0, iNaturalist
Photo 2: Marjorie Courtenay-Latimer, the curator of the East London Museum in South Africa, who discovered the 1938 coelacanth.
Photo 3: From left to right: Ichthyology Curator James Atz, Dr. Charles Rand, Ichthyology Curator C. Lavett Smith, © AMNH Library (circa September 1975)

#science #discovery #amazingnature #wildlife #amnh
https://www.instagram.com/p/Cpk7X2DLeMj/?igshid=NGJjMDIxMWI=

Every clack of a well-made keyboard can bring joy. If you spend most of your day typing, coding, or gaming, a mechanical keyboard can be a comfortable, customizable upgrade over the shallow, drab keyboard that came with your computer.

After spending months testing the most promising mechanical keyboards, we recommend the Keychron V3 Max, Keychron V5 Max, and Keychron V6 Max. These keyboards provide the best typing experience, and they offer the most extra features for their surprisingly reasonable prices.

Here at Mozilla, we are the first to admit the internet isn’t perfect, but we are also quick to point out that the internet is pretty darn magical. The internet opens up doors and opportunities, allows for people to connect with others, and lets everyone find where they belong — their corners of the internet. We all have an internet story worth sharing. In My Corner Of The Internet, we talk with people about the online spaces they can’t get enough of, what we should save in Pocket to read later, and what sites and forums shaped them.

This month we chat with Annie Rauwerda, the woman behind Depths of Wikipedia, which highlights weird and unexpected entries from the online encyclopedia. The project started as an Instagram account and has since expanded to Twitter, TikTok and a live comedy show. We talk to her about her obsession with memes, editing Wikipedia pages and finding the goofy corners of the internet. 

What is your favorite corner of the internet? 

I really like goofy corners of the web, like this archive of computer mice and this archive of rotating food. I love Discord so much. I love Twitter most of the time. And I love Wikipedia, particularly the timeline of the far future. 

What is an internet deep dive that you can’t wait to jump back into?

I’m obsessed with finding the origin of meme images. I recently tracked down the hot pink bitch named Breakfast and the Wikipedia high five couple. I spend a lot of time on the lost media subreddit. 

What is the one tab you always regret closing?

I edit Wikipedia a lot, and occasionally I come across a guideline or gadget that’s super helpful but hard to find (Wikipedia’s guidelines are a total labyrinth). Thank god for command + shift + t!!

What can you not stop talking about on the internet right now? 

I’ve been really into crossword construction! And making silly Venn diagrams!

What was the first online community you engaged with?

When I was in elementary school, my dad quit his job to be a stay-at-home parent and started pouring tons of energy into being the coolest dad ever. I grew up in Grand Rapids, Michigan, where there’s a ton of show in the winter, and he would make massive snow castles. It was right as YouTube was getting big, and he posted a video of the fort. We got comments from people around the world. It was so cool!

What articles and videos are in your Pocket waiting to be read/watched right now?

I love all the features in Quanta. And I’ve been meaning to finish The Curse of Xanadu in Wired from 1995. 

If you could create your own corner of the internet, what would it look like?

You know how people create extensive, time-intensive projects dedicated to random things like bread tags or candy cross-sections or notes left in library books or etymology? Probably something like that. I love archives, especially silly archives!

Wikipedia turned 22 this year. What about it keeps contributors like you so dedicated to editing and maintaining its pages?

There are a lot of canned answers I could give about a shared commitment to free knowledge, and I’m sure that’s part of it, but the real reason I edit is that it’s pretty fun. You get addicted.

Annie Rauwerda has been running Depths of Wikipedia since March 2020, when she started it as a sophomore at the University of Michigan. She’s also a writer whose work has appeared in Input Magazine and Slate.

Save and discover the best articles, stories and videos on the web

Get Pocket

The post The Depths of Wikipedia creator on finding the goofy corners of the web appeared first on The Mozilla Blog.

Ad blockers are a specific type of browser extension (i.e. software that adds new features or functionality to Firefox). Using ad blockers, you can eliminate distraction and frustration associated with online ads popping up across your internet travels. Here are six of our favorite ad blockers that make the web a whole lot easier to enjoy. 

uBlock Origin

A gold standard among ad blockers, uBlock Origin is extremely efficient at stopping all types of internet ads, including video pre-rolls and pop-ups. It works great by default, but also affords users plenty of customization options should you want to tweak your content filters. 

AdBlocker Ultimate

AdBlocker Ultimate is also very capable at removing all varieties of internet ads. There are no “acceptable” ads or whitelisted advertisers. The extension also blocks many trackers and helps detect malware.

AdGuard AdBlocker 

AdGuard AdBlocker is a highly effective ad blocker that works well on Facebook and YouTube. It also smartly allows certain types of ads by default, such as search ads, since those may actually be helpful to your searches, as well as “self-promotion” ads (e.g. special deals on site-specific shopping platforms like “50% off today only!” sales). 

Ghostery

Block ads and most hidden browser trackers by default with Ghostery. This slick extension also scores points for being very intuitive and easy to operate. It’s simple to set Ghostery’s core features, like enabling or disabling Enhanced Ad Blocking and Anti-tracking. 

Popup Blocker (strict)

Popup Blocker (strict) blocks all pop-up requests from any website by default. However, a handy notification window gives you the control to accept, reject or open pop-ups as you please. 

Webmail Ad Blocker

Use Webmail Ad Blocker to clean up your web-based email by removing ads littering the inboxes of Gmail, Hotmail, Outlook.com, Yahoo Mail and more. 

Take control of your online advertising experience by checking out our latest Ad Blockers Collection. And if you’re really into add-ons, follow our Mozilla Add-ons Blog and consider building your own extension to add to the collection!

Take control of your online advertising experience by checking our latest Ad Blockers Collection. And if you’re really into add-ons, follow our Mozilla Add-ons Blog and consider building your own extension to add to the collection!

Get Firefox

Get the browser that protects what’s important

The post Ad blocker roundup: 6 ad blockers to improve your internet experience appeared first on The Mozilla Blog.

In case you haven’t heard, there’s an ongoing conversation happening about your personal data. 

Earlier this year, United States President Biden said in his State of the Union address that there needs to be stricter limits on the personal data that companies collect. Additionally, a recent survey found that most people said they’d like to control the data that companies collect about them, yet they don’t understand how online tracking works nor do they know what they can do about it. Companies are now trying and testing ways to anonymize the third-party cookies that track people on the web or get consent for each site or app that wants to track people’s behavior across the web. 

These days, who can you trust with your personal data? Mozilla. We have over a decade of anti-tracking work with products and features that protect people, their privacy and their online activity. Today, we’re announcing the official rollout of one of our strongest privacy features, Total Cookie Protection, to automatically block cross-site tracking on Firefox Android. 

Yes, companies gather your data when you go from site to site

Before we talk about Total Cookie Protection, let’s talk about cross-site tracking. These days our in-person transactions like shopping for groceries or buying gifts for friends have now become commonplace online. What people may not be aware of are the other transactions happening behind the scenes. 

For example, as you’re shopping for a gift and going from site to site looking for the right one, your activity is being tracked without your consent. Companies use a specific cookie known as the third-party cookie, which gathers information about you and your browsing behavior and tracks you when you go from site to site. Companies use the information to build profiles and help them make ads targeted at convincing you to purchase, like resurfacing an item you were shopping for. So Mozilla created the feature Total Cookie Protection to block companies from gathering information about you and your browsing behavior.

Your freedom from cross-site tracking now available on Firefox Android 

Meet Firefox’s Total Cookie Protection, which stops cookies from tracking you around the web and is now available on Firefox Android. Last year, Firefox rolled out our strongest privacy feature, Total Cookie Protection across Windows, Mac and Linux. Total Cookie Protection works by maintaining a separate “cookie jar” for each website you visit. Any time a website, or third-party content embedded in a website, deposits a cookie in your browser, Firefox Android confines that cookie to the cookie jar assigned to that website. This way, no other websites can reach into the cookie jars that don’t belong to them and find out what the other websites’ cookies know about you. Now, you can say goodbye to those annoying ads following you and reduce the amount of information that companies gather about you whenever you go online.

Firefox’s Total Cookie Protection covers you across all your devices

Whether you’re browsing at your desk or your phone, now you’ll get Firefox’s strongest privacy protection to date. Firefox will confine cookies to the site where they were created, thus preventing tracking companies from using these cookies to track your browsing from site to site. To seamlessly work across your devices, sign up for a free Firefox Account. You’ll be able to easily pick up your last open tab between your devices. Bonus: You can also access your saved passwords from your other devices by signing up for a free Firefox Account.

Get your most secure and private Firefox Android today!

Download Firefox Android

For more on Firefox:

• Don’t “accept all cookies” until you’ve seen this video
• Mozilla Explains: Cookies and supercookies
• How Firefox’s Total Cookie Protection and container extensions work together
• Firefox rolls out Total Cookie Protection by default to all users worldwide

The post Firefox Android’s new privacy feature, Total Cookie Protection, stops companies from keeping tabs on your moves appeared first on The Mozilla Blog.

Hot on the heels of getting borg assembled to go into my server closet, I decided to get myself a Beelink U59 Pro mini-PC to use as a beefier thin client, and these are my notes on it.

Spoiler: Low-end Intel machines are still a better idea than ARM single-board computers for general use–a lesson I’m apparently doomed to relearn every few years.

Although I have been very happy with my thin client setup over the past six months, a few things happened as I made more and more use of it:

• The Raspberry Pi (first a 3, then a 4) was fast enough to provide a good experience against my old server, but it definitely the weak link when rendering a remote desktop streamed from borg. I still got a good, usable experience at 2560x1080, but… I wanted more.
• Try as I may, I couldn’t get the Pi to be performant when rendering both a 2560x1080 and a 3440x1440 display (which is as far as a 4K HDMI port output can drive my LG Ultrawide), and it crashed whenever I tried to overclock it.
• I have been wanting to fiddle with music gear again for a few months, and I just can’t bring myself to do it on a laptop. I’ve tried time and time again, but I need a permanent location to have everything plugged in.
• I also can’t do that on an ARM machine (well, other than a Mac mini, but I’m not made of money)¹.

So I started thinking about my options, and since I did entirely too much research into mini PCs before assembling borg, I knew pretty much what the landscape looked like regarding computing power, noise and power consumption, and it was only a matter of watching for a suitable deal–and this was it.

The U59 has a few things I like:

• The Celeron N5105 compares pretty favorably to the i7-6700 I was using as a server before borg. And that includes integrated Intel UHD graphics, hardware video transcoding and virtualization support.
• This CPU is rated at 10W, which is important in a box I want to be powered on 24/7.
• It has two full size HDMI ports, each able to go up to 4K. No more cables for ants.
• It also has two Gigabit Ethernet ports (not 2.5Gbps, but I haven’t felt the need for that yet).
• It has five USB ports to plug in anything I need on my desk (mostly USB-A, but one USB-C with display output).
• The machine is trivial to open, and RAM, SSD and the SATA bay are easily accessible (and not soldered on). Yes, it is a proprietary motherboard, but so is everything else in this range, and at least it’s maintainable.
• You get dual-channel RAM in the 16GB configuration (which is what I went for, and which gives you pretty good iGPU performance).

A beefier machine (with 2.5Gbps or PD/Thunderbolt) would be nicer, but the (relatively) fast integrated graphics at such a low TDP (and price point) was the key factor for me.

I couldn’t get anything comparable near the EUR 200 price range with a Ryzen chip, and even though are new Intel chips coming out (like the N100) that will have an even lower TDP of 6W, this was the right performance at the right time.

It bears noting that I had absolutely zero intention of actually getting another Intel machine after assembling borg, but I got a pretty good deal on the 16GB RAM/512GB SSD configuration (just a little over EUR 200 with a coupon) and at this price point it blows literally everything with an ARM processor out of the water performance-wise².

But I can always re-purpose it as firewall or Proxmox host if by any chance this doesn’t pan out. In fact, my long-term plan is to rotate it out to act as a media server (in a reprise of what I did when I replaced a Pi with a Z83ii six years ago), and it will do very nicely for that.

Unlike most mini PCs I’ve seen, this was actually designed to be opened without tearing off rubber feet (and has a cute little pull tab to help you open the case):

The case is “unapologetically plastic” but sturdy, and discreet enough to keep on sight (well, except the power button, which is too visible by far).

But I like the looks and the general port layout. I’d have preferred all the ports–except maybe the USB-C port–were around the back, but there wouldn’t be enough room:

It’s worth noting that you get two SATA ribbon cables in the box, plus the usual VESA mounting assortments and HDMI cables.

The U59 Pro came with a pre-activated Windows 11 Pro OEM license that booted straight into the out-of-the-box experience, installed updates, and was ready to go within 30m or so (which is par for the course with Celerons).

Thanks to the Intel UHD graphics and the speedy SSD, the desktop experience (both local and remote) was very good. Snappy browsing, decent performance with the Windows Subsystem for Linux, etc.

Everything went smoothly until I started installing Bitwig and a few VST plugins. That was when I realized things weren’t going to be snappy enough for my liking in Windows (at least not with the Antimalware Service Executable dragging the installs down).

And since I prefer to have Linux on my personal machines, I decided to install Fedora (yes, it’s grown on me). Since I want to preserve the Windows install (I might need it later), I popped open the case and slotted in a spare SATA SSD. Within an hour or so, I had Fedora, Bitwig, WINE and most of what I wanted installed.

And, of course, everything works out of the box in Fedora–I have it set to use my Bluetooth keyboard and mouse combo, and I was soon using it to run both my displays simultaneously via Wayland³.

I can’t confirm power draw properly since Linux tools like powertop can only measure the CPU and state 2W on an idle state and 10W at load, which matches the N5105‘s nominal 10W TDP, and I don’t have a socket meter.

But I think that compares favorably with a Pi 4, which I believe will go a bit past 6W at full tilt. which is around half of what the N5105 is likely to be spending doing comparatively more work (either doing audio or driving two displays).

The PSU is rated for 36W (against the Pi’s 15W PSU), but the CPU is fast enough that I don’t really think I can really make the machine draw that much power in “normal” use–unless I really go to town on Bitwig plugins.

Like I wrote above, even though the N5105 only has 4 cores and no hyper-threading, it compares very well against the i7-6700 I am using as a server, and I can run Bitwig and 4-5 plugins without any issues (yet–I haven’t really tried more).

I also installed VS Code, Blender and Godot 4.0 for playing around, and nothing felt slow when editing and tweaking existing files, which makes sense since (again) I’ve been doing that remotely on the i7-6700, and both CPUs are roughly matched.

But I also tried a couple of emulators, and it’s no speed daemon (no, you can’t really game on it). The thing is… I can do most of my 3D modeling on it in a pinch, and that is saying a lot.

And, of course, it blows the Raspberry Pi out of the water graphics-wise (which was kind of the point).

The N5105 cannot drive my LG Ultrawide at its full 5120x2160 resolution, but it can run an accelerated RDP session from borg across both displays (2560x1080 and 3440x1440) with consummate ease, and it is glorious.

Even full-screen rotations in Blender come across quite smoothly, and if I only use the smaller display, you’d be hard pressed to notice it’s a remote session.

But I wanted a solid way to compare the experience and performance improvements for my daily use, and as a sort of benchmark, I resorted to my 3D site map.

I still have a few challenges with it because the graph library is pretty compute-intensive (not to mention I’m throwing thousands of densely interconnected nodes at it), but it is a nice yardstick because it uses both CPU and GPU, and I can load it up on any machine with ease.

So I set up 1920x1080 browser windows both locally and on borg (using Remmina as an RDP client, set to LAN speed and automatic session encoding) and got these somewhat unscientific measurements with intel_cpu_top running on the U59 while I rotated the graph around:

• Local: GPU at 48%, CPU at an average of 59% across all 4 cores.
• Remote: GPU at 14%, CPU at less than 10% across any core, even if I did something wild like rotating or zooming the graph quickly.

And yes, the RDP session is being GPU accelerated at both ends–local GPU load decreased to 8%-10% once I closed the browser on borg.

I also tried repeating the UFO test benchmarks I did with the Pi, and it was pretty boring–I got 57-60fps in remote sessions, and a locked 60 locally.

Another thing I noticed was that remote audio was perfect. With the Pi as remote desktop client, I would get crackly audio from alerts or irritating pauses when peeking at a YouTube video remotely–now I can watch video remotely without any significant issues.

So from the perspective of upgrading my remote desktop experience, the U59 is a resounding success–sessions are significantly snappier than on the Pi, with full screen updates and quick scrolling almost indistinguishable from running things locally (when using the 2560x1080 monitor display alone), and I can use two monitors now just fine.

This is a bit of an afterthought, but generally good news–regular readers will know one of the reasons I like my remote setup is the complete absence of noise–and I originally wanted a completely fanless machine, but that just isn’t feasible at this price point.

I picked the U59 partly because reviews specifically mentioned it being very quiet under average load. So I took a chance, and it paid off.

Since I run almost nothing that can make all of the CPU cores run at full tilt, I don’t hear the fan at all, other than a short burst at power-on/reboot–and I preemptively “solved” that by mounting it out of sight under my desk.

I also checked it regularly, and it doesn’t get noticeably warm, at least in Winter weather. I have lmsensors on all of my machines given the wide temperature swings we get here over the year, and with 18^oC in the office, the CPU sits between 59^oC and 64^oC.

This is OK, but not as low as I would have liked–the i7-6700 I have in a stuffy closet measures about the same with constant load, but then again it has a Noctua cooler.

Time will tell how the U59 will perform when it’s 35^oC outside (and 28^oC inside without AC), but so far I’m happy with it.

Now I think it’s time to stop setting up machines and getting something done.

Hardware week again.

Cleanup day, not just physically.

• Resumed weekly tidying of office. Considering adding a ritual sacrifice of my optimism and creativity (in the shape of a cookie) before turning on my work machine.
• Was again stung by Fedora‘s insistence in kneecapping its OS by removing media codecs in a new install.
• Spent a while trying to get ChatGPT to help with creating a set of Azure Monitor queries in my personal subscription, which it was utterly useless for since it just… made up stuff (and confused Kusto queries with the simpler feature set in Log Analytics).

Some radio stuff.

• Set up a LilyGo 433MHz board to run OpenMQTTGateway, which was childishly simple to do, but doesn’t seem to immediately pick up all the 433MHz stuff I’m interested in–I guess that will require sitting down with my SDR dongle for a while and tuning a few things. Was somewhat amused to see an Acurite Grill/Meat Thermometer 01185M pop up almost immediately, though.
• Fixed some of my home automation, namely some overdue battery swaps and the LG Smart TV API integration that was broken by my having switched routers a few days ago.

Impromptu ISP veteran meetup day.

• Walked 5Km in an attempt at doing some actual exercise.
• Listened to another Oxide and Friends episode during the above.
• Took a little time before dinner to check on Steam Linux updates and fool around with emulators, with mixed results.
• Finally managed to get an Xbox controller to pair with Fedora by setting:

# head -2 /etc/bluetooth/main.conf
[General]
Privacy=device

• Cleared out a mass of duplicates from my Calibre library.
• Spent a fair bit trying to get sdf running on my iPad Pro, but scikit would have none of it.

Woke up early again.

• Had a stab at getting py-sdf (a more complete fork) to use CUDA with cupy, but numpy array type coverage isn’t there yet. Ended up starting a fork to experiment.
• Tried to install my Kontakt VSTs in Fedora by resorting to an older installer, but Native Instruments’ software is just completely user hostile.
• Learned the hard way that CAD Sketcher will only work “out of the box” in Fedora 37 if you install Blender via flatpak due to different Python runtimes (also, updated my notes regarding how to set it up in mm to edit STL files directly).
• Played around with Steam and CEMU with borg plugged into my LG Ultrawide for a few minutes of glorious enjoyment. If it wasn’t for the fan noise when the GPU ramps up, I’d likely never move it to the closet.

Wound down for the weekend.

• Tried to proactively clean up office after work to save time on Monday.
• Spent a while reading up on CUDA development and clearing my personal backlog, drafts, random organizational chores.

Decided to investigate 3D modeling options.

• Personal inbox zero.
• Got a somewhat usable modeling worflow going with py-sdf, meshview and Jupyter inside VS Code:

It can’t really replace OpenSCAD, given the massive amount of libraries for it out there, but might be a good alternative for other things.

• Practiced using CAD Sketcher to design a simple enclosure, which was somewhat of a failure. Realized I still remember a fair amount of how to use Blender, just not the bits I need.
• Collated and posted my notes on the Beelink U59 Pro.

Low-level and electronics stuff.

• Upgraded my ZigBee coordinator and routers to new firmware versions to see if I can fix the niggling issues I’ve been having with devices falling off the network¹:

cc2538-bsl.py -p /dev/ttyUSB0 -evw CC2652R_router_20221102.hex
cc2538-bsl.py -p /dev/ttyUSB0 -evw CC2652R_coordinator_20221226.hex

• Tried to unbrick another CC2652R1F adapter I had put aside a while back, first with an FTDI adapter and later using a Pi as an impromptu adapter, but openocd just couldn’t detect the JTAG interface no matter what I did.
• Rebuilt one of my ESP-01 prototypes on a clean breadboard. I have a dozen of the things and might as well make use of them.
• Looked for updated SDR applications for the Mac. Turns out there aren’t many (SDR Angel is an old fave, gqrx hasn’t been updated in a year, etc.).
• Cleaned up and posted these notes.

Photo provided by Sarens

I have this morning received a Press Release from a Belgian crane company that helped install the tunnel boring machines for the Broadway Subway in Vancouver. The reason I have decided to post on this bog about it is that it makes some quite remarkable assertions about the benefits of this project. My commentary appears after their quoted text below. Not all of the Press Release is included here.

“This new project, scheduled to open in 2026, will cover a 5.7 km extension of the Millennium line between VCC-Clark Station and Broadway and Arbutus, and will create more than 130,000 direct and indirect construction jobs.

“Sarens had a direct participation in this project, by contributing with its technical team and specialized machinery to the assembly of the tunnel boring machines (TBM) used for the construction of the subway section of the line.

“This new line will represent an important environmental advance by exponentially reducing road traffic in the area, and reducing greenhouse gas emissions by up to 9,800 tons of CO2 per year by 2030.

“Construction work on the new Broadway Subway Project is already underway on the tunneling phase prior to completing the six underground stations along its 5.7 km route. Once inaugurated, this line, which serves as an extension of the Millennium Line, will reduce the travel time between VCC-Clark and Arbutus to just 11 minutes, saving the average transit commuter almost 30 minutes a day.

“This new subway line will consist of a subway section of approximately 5 km, while the remaining 700 meters will be built above ground to connect to VCC-Clark Station. Sarens, world leader in heavy lifting, engineered transport and crane rental, has participated at the request of Broadway Subway Project Corporation, an Acciona-Ghella joint venture, in the assembly of the tunnel boring machines (TBM) used for the construction of the subway section.

“For this particular job, Sarens used one of its Liebherr LR 1400-2 cranes in SDB configuration. To facilitate the transfer of the TBM parts into the tunnel portal, the team used a 70 m main boom with a 135-ton counterweight and a 170-ton Superlift counterweight. In addition, support cranes such as the LTM 1095 or the Terex Explorer 5800 were used.

“As the operation is within the City of Vancouver, the crane operators had to deal with a heavily congested site and tight lifting schedules. In addition, due to the limited visibility available for the installation maneuver of the various parts of the TBM, the technical team had to work under close coordination established by radio.

“Broadway Subway Project is a project driven by the province of British Columbia, which will be operated by TransLink once it is commissioned in 2026. It will cover a total distance of 5.7 km between VCC-Clark and Arbutus, a corridor recognized as the second most important business center in British Columbia.

“This new subway line is expected to generate more than 130,000 direct and indirect jobs. In addition, thanks to the project’s adherence to the Community Benefits Agreement, a significant portion of these positions will employ women, indigenous people and traditionally underrepresented sectors, which will benefit the community economically and create a highly skilled technical workforce base.”

The claim that this project will “exponentially” reduce road traffic is not supported by any evidence, nor is it consistent with the plan of building a subway. The whole idea of burying the line is to protect the current capacity for road traffic. There are no plans to reduce the amount of road space dedicated to moving and parking vehicles. If there were, the preferred alternative would have been light rail running on the surface. But the whole purpose of SkyTrain has always been to keep out of the way of the cars. The only difference is while the rest of the region’s rapid transit system is on elevated guideways this one is all in tunnel; just like most of the Canada Line in Vancouver.

The only way to reduce traffic is to either reduce the amount of road space dedicated to moving vehicles – and that includes parking spaces too – or start charging a fee for using the road. Neither of those are popular with Vancouver voters – especially those who like driving large cars and trucks, mostly with just the driver most of the time. Just look at the fuss made when one exclusive bike lane was put in Stanley Park.

Traffic expands and contracts to fill the space available. Roads can carry many more people when they use sustainable transportation modes – buses, bikes and walking. If road users insist on bringing their SUVs for every trip, congestion is inevitable but tends to adapt over time. Gridlock, when it does occur, is due to rare events and people using their cars and trucks to block intersections. This blog has made these statements many times over the years and no-one has ever managed to solve traffic congestion by building more road space. Simple geometry means modes that carry more people per hour than SOVs, and given priority in their own lanes will greatly increase the utility and attractiveness of streets. People also like spaces where they can sit down and watch the passing scene. As long as they are not deafened by engines and tire noise and choked with exhaust fumes. Broadway in New York City being a great example. Times Square now sees far more activity than it ever did when filled with cars.

Most of the people who will use this new stretch of subway currently use transit. The 99 B-Line being one of the busiest bus routes. Since the subway line will only get as far as Arbutus Street many will continue westwards on what will be a shorter B-Line route. It could get Rapid Bus treatment, but I wouldn’t hold my breath waiting for that.

I expect that the Broadway subway will attract new transit users, but even if there are a lot of people who stop driving and start riding transit all the time, don’t expect that to translate to fewer cars on the street absent a user fee or a reduction in road width dedicated to cars and trucks. When Toronto decided to replace the streetcars on Yonge Street with a subway, traffic in downtown increased – since there were no longer streetcars holding up the cars while people got on and off them.

When the Broadway subway opens do not expect any reduction in people driving. The people who do switch modes will quickly be replaced by others hoping for a quicker drive along Broadway with fewer buses competing for space. Within a very short period of time any available road will be occupied. Guaranteed.

A nice new B-Line bendy bus (hybrid diesel electric) on its way from UBC. Once the subway opens this service will terminate at Arbutus Street Station.

No decision on the extension of the Broadway Subway to UBC has yet been announced.

My photo CC license for non-commercial use

This post was revised on March 13, 2023 to correct the country of the crane company.

POSTSCRIPT

Actually City staff are making a real dog’s breakfast about how to utilise road space on Broadway after the current construction phase is over and trains start running.

Anthony Floyd posted on Mastodon

“City of Vancouver staff have recommended against Active Mobility lanes on a post-subway-construction Broadway, despite overwhelming public support and direction from council last year.”

The recent staff report can be found at

https://council.vancouver.ca/20230329/documents/pspc2.pdf

This story is a bit encouraging and a bit unsettling. It's encouraging because it would be good to see WordPress sites become full-fledged members of the fediverse (by which I mean both sending posts and receiving comments from the fediverse). It would represent a huge uptick in support and use for a distributed social network, and could finally eliminate the need for Facebook and Twitter. But it's unsettling because Automattic has acquired the plugin. Does that mean they're going to monetize it? Take it out of open source (source is currently here)? I don't know. Relkated: Ben Werdmuller comments.