For a few years now I’ve had my eyes on Pulumi. Every time I looked at it I came away with a sentiment of “hmm, that seems nice”. I never got around to using it though and nobody around me had either so it was hard to get a sense of whether it would live up to my hopes. I recently decided to redo some of my personal infrastructure and since Pulumi has Hetzner Cloud support I decided to give it a go and see what would happen.
However, with millions of new active users rushing into Mastodon, I’m forced to reevaluate that. I think I may have become too focused on what I saw of as the limits of a federated setup (putting yourself into someone else’s fiefdom), without recognizing that if it started to take off (as it has), it would become easier and easier for people to set up their own instances, allowing those who are concerned about setting up in someone else’s garden the freedom to set up their own plot of land.
I hadn’t thought to use Mastodon lists until I read the Frustration with lists chapter of Martin Fowler’s Exploring Mastodon, in which he writes:
I like lists because they allow me to divide up my timeline to topics I want to read about at different times. They are frustrating because the tools to manage them in Twitter are very limited, so it’s more hassle to set up the kind of environment I’d like. Mastodon also has lists, sadly its current management tools are equally bad.
This seemed like a good challenge for Steampipe. To tackle it, I first needed to add some new tables to the plugin to encapsulate the list APIs: mastodon_list and mastodon_list_account. I’ll save that story for another time. Here I’ll just show that together they enable queries like this.
select
l.title as list,
array_agg(a.username order by a.username) as people
from
mastodon_list l
join
mastodon_list_account a
on
l.id = a.list_id
group by
l.title
That’s a useful view, and I’ve now included it, but it didn’t address Martin’s specific need.
To manage these lists, I really need a display that shows every account that I follow in a table with its lists. That way I can easily see which list each account is on, and spot any accounts that aren’t on a list.
For that I needed to add a list column to the Following tab.
This was the original query.
select
url,
case when display_name = '' then username else display_name end as person,
to_char(created_at, 'YYYY-MM-DD') as since,
followers_count as followers,
following_count as following,
statuses_count as toots,
note
from
mastodon_following
order by
person
The new version captures the above join of mastodon_list and mastodon_list_account, and joins that to the mastodon_following (people I follow) table. It’s a left join which means I’ll always get all the people I follow. If you’re not on a list, your list column will be null.
with data as (
select
l.title as list,
a.*
from
mastodon_list l
join
mastodon_list_account a
on
l.id = a.list_id
),
combined as (
select
d.list,
f.url,
case when f.display_name = '' then f.username else f.display_name end as person,
to_char(f.created_at, 'YYYY-MM-DD') as since,
f.followers_count as followers,
f.following_count as following,
f.statuses_count as toots,
f.note
from
mastodon_following f
left join
data d
on
f.id = d.id
)
select
*
from
combined
order by
person
That query drives the new version of the Following tab.
It’s pretty sparse, I’ve only just begun adding people to lists. And honestly I’m not sure I’ll want to keep doing this curation, it’s the kind of thing that can become a burden, I need to play around some more before I commit. Meanwhile, the default sort puts unlisted people first so they’re easy to find.
To provide a better way to find people who are on lists, I expanded the List tab in a couple of ways. It had included a dropdown of lists by which to filter the home timeline. Now that dropdown has counts of people on each list.
input "list" {
type = "select"
width = 2
sql = <<EOQ
with list_account as (
select
l.title
from
mastodon_list l
join
mastodon_list_account a
on
l.id = a.list_id
),
counted as (
select
title,
count(*)
from
list_account
group by
title
order by
title
)
select
title || ' (' || count || ')' as label,
title as value
from
counted
order by
title
EOQ
}
I also used this query to expand the List tab.
select
l.title as list,
array_to_string( array_agg( lower(a.username) order by lower(a.username)), ', ') as people
from
mastodon_list l
join
mastodon_list_account a
on
l.id = a.list_id
group by
l.title
The result is the list / people table on the right.
I know that some won’t cotton to this SQL-forward programming model. But for others who will, I wanted to show a few detailed examples to give you a sense of what’s possible at the intersection of Mastodon and Steampipe.
If you’re not tuned into SQL (like I wasn’t for a very long time), here’s your takeaway: as SQL goes, this stuff is not too scary. Yes there are joins, yes there’s an array_agg that transposes a column into a list. It’s not beginner SQL. But lots of people know how to use join and array_agg in these ways, lots more could easily learn how, and with SQL ascendant nowadays these are skills worth having.
A lot has changed since 2009, which led us to re-architect Acquia Cloud starting in late 2019. This effort, labeled "Acquia Cloud Next" (ACN), became the largest technology overhaul in Acquia's history.
In 2013, four years after the launch of Acquia Cloud, Docker emerged. Docker popularized a lightweight container runtime, and a simple way to package, distribute and deploy applications.
Docker was built on a variety of Linux kernel developments, including "cgroups", "user namespaces" and "Linux containers":
In 2008, Eric W. Biederman (Red Hat) introduced user namespaces. User namespaces allow a Linux process to have its own set of users, and in particular, allow root privileges inside process containers.
In 2008, IBM created the Linux Containers Project (LCP), a set of tools on top of cgroups and user namespaces.
Docker's focus was to deploy containers on a single machine. When organizations started to adopt Docker across a large number of machines, the need for a "container orchestrator" became clear.
Long before Docker was born, in the early 2000s, Google famously built its search engine on commodity hardware. Where competitors used expensive enterprise-grade hardware, Google realized that they could scale faster on cheap hardware running Linux. This worked as long as their software was able to cope with hardware failures. The key to building fault-tolerant software was the use of containers. To do so, Google not only contributed to the development of cgroups and user namespaces, but they also built an in-house, proprietary container orchestrator called Borg.
When Docker exploded in popularity, engineers involved in the Borg project branched off to develop Kubernetes. Google open sourced Kubernetes in 2014, and in the years following, Kubernetes grew to become the leading container management system in the world.
Back to Acquia. By the end of 2019, Acquia Cloud's infrastructure was delivering around 35 billion page views a month (excluding CDN). Our infrastructure had grown to tens of thousands of EC2 instances spread across many AWS regions. We supported some of the highest trafficked events in the world, including coverage of the Olympics, the Australian Open, Weather.com, the Mueller report, and more.
Throughout 2019, we rolled out many "under the hood" improvements to Acquia Cloud. Thanks to these, our customers' sites saw performance improvements anywhere from 30% to 60%, at no cost to them.
That said, it became harder and harder to make improvements to the existing platform. Because of our scale, it could take weeks to roll out improvements to our fleet of EC2 instances. It was around that time that we set out to re-architect Acquia Cloud from scratch.
Acquia's journey to ACN started prior to Kubernetes and Docker becoming mainstream. Our initial approach was based on cgroups and Linux containers. But as Kubernetes and Docker established themselves in the market, it became clear we had to pivot. We decided to design ACN from the ground up to be a cloud-native, Kubernetes-native platform.
In March of 2021, after a year and a half of development, my little blog, dri.es, was the first site to move to ACN. Getting my site live in production was a fun rallying point for our team. Even more so because my site was also the first site to launch on the original Acquia Hosting platform.
I never blogged about ACN because I wanted to wait until enough customer sites had upgraded. Fast forward another year and a half, and a large number of customers are running on ACN. We now have some of our highest traffic customers running on ACN. I can say without a doubt that ACN offers the highest levels of performance, self-healing, and dynamic scaling that Acquia customers have relied on.
ACN continuously monitors application performance, detects failures, reroutes traffic, and scales websites automatically without human assistance. ACN can handle billions of pageviews, gracefully deals with massive traffic spikes, all without manual intervention or architectural changes. Best of all, we can roll out new features in minutes or hours instead of weeks.
There is no better way to visualize this than by sharing a chart:
The "web transaction times" of a large Fortune 2000 customer that upgraded their main website to Acquia Cloud Next. You can see that PHP (blue area), MySQL (dark yellow area) and Memcached (light yellow) became both much faster and less volatile after migrating to Acquia Cloud Next. (The graph is generated by New Relic. New Relic defines the "web transaction time" as the time between when the application receives a HTTP request and when a HTTP response is sent.)
Customers on Acquia Cloud Next get:
Much faster page performance and web transaction times (see chart above)
5x faster databases compared to traditional MySQL server deployments
Faster dynamic auto-scaling and faster self-healing
Improved resource isolation - Nginx, Memcached, Cron, and other services all run in dedicated pods
To achieve these results, we worked closely with our partner, AWS. We pushed the boundaries of certain AWS services, including Amazon Elastic File System (EFS), Amazon Elastic Kubernetes Service (EKS), and Amazon Aurora. For example, AWS had to make changes to EKS to ensure that they could meet the scale at which we were growing. After 15 years of working with AWS, we continue to be impressed by AWS' willingness to partner with us and keep up with our demand.
In the process, AWS made upstream Kubernetes contributions to overcome some of our scaling challenges. These helped improve the speed and stability of Kubernetes. We certainly like that AWS shares our values and commitments to Open Source.
Last but not least, I'd be remiss not to give a big shoutout to Acquia's product, architect, and engineering teams. Re-architecting a platform with tens of thousands of EC2 instances running large-scale, mission-critical websites is no small feat.
Our team continued to find creative and state-of-the-art ways to build the best possible platform for Drupal. For a glimpse of that, take a look at this presentation we gave at Kubecon 2022. We learned that by switching our scaling metric from Kubernetes' built-in CPU utilization to a custom metric, we could reduce the churn on our clusters by ~1,000%.
Looking back at ACN's journey over the past 3+ years, I'm incredibly proud of how far we have come.
Having been housebound for the past two days, I was able to read this remarkable book about a family of fourteen, twelve children, of which six–six!–of the sons became schizophrenic in their teens and twenties. An amazing feat of research which follows the family’s story, as well as those of the doctors working to understand this puzzling, difficult and disabling disease.
This is from a press release I got this morning. The message is too long to make it feasible for a Mastodon post (500 characters) but I will post a link there back to this post as I think this is important information. TL:DR version – we don’t need a fusion power source – we’ve already got one – the sun! By the time the technology gets up to scale and comparable cost to solar will be TOO LATE. We need lots more renewables now not in the distant future
Guest post from Garry Cinnamon, of Cinnamon Energy
Scientists at Lawrence Livermore National Laboratory have announced a fusion breakthrough using lasers. The future of clean, limitless energy according to Forbes! A game-changer for climate according to PBS!
Using a laser and power plant system about the size of a sports stadium, the experiment generated a net power output of about one megajoule. This fusion power plant can blast the laser about 10 times a week. Sounds impressive.
Not to burst anyone’s bubble, but one megajoule is equivalent to 0.278 kwh — about the same amount of energy a single solar panel will generate in an hour from that fusion power plant 92 million miles away.
Press coverage somehow missed the fact that the energy output of this test is de minimis. They also missed the facts that fusion releases vast quantities of dangerous neutron radiation, that this radiation will contaminate surrounding equipment (just like fission reactors), that that we have not yet engineered a way to capture the heat from laser or tokamak fusion, and that there is no good source for all the tritium fuel necessary for fusion (other than more fusion reactors).
It takes at least 20 years to get a new nuclear fission plant permitted and constructed. At this point we don’t even have a working prototype laser fusion plant — that could take another 30 years. Realistically, we’re 50+ years away from getting commercial laser or tokamak fusion power plants working at scale. In the mean time, deploying billions of lowly solar panels is the safest, most reliable and least expensive way to generate the energy we need.
To learn more about the realities of a fusion power breakthrough, please listen to this week’s Energy Show.
Last week, dancer and DJ Stephen ‘tWitch’ Boss died from suicide at age 40. Like many, I was incredibly shocked and saddened by the news.
As a scholar of Black entertainment history, I also reflected on the longer history of Black male entertainers dancing or telling jokes to their deaths despite cultivating a public image as “pure love and light,” which is how tWitch’s former co-producer, Ellen DeGeneres described him on her Instagram upon hearing of his death.
There have been so many tragic and unexpected deaths of young Black men in the entertainment industry that websites, such as BestOfDate, and Ranker have formed to document them.
While these sites are primarily documenting the deaths of rappers, they are also creating a narrative around Black men that values their personas more than the lives they actually lived.
If there is a common thread running through the seemingly unexpected deaths of Black male celebrities, it’s that few around them were made aware of their struggles. When singer-songwriter Prince died in 2016 at 57 from an accidental overdose of fentanyl, even his closest friends did little to address his drug addiction. Similarly when Chadwick Boseman, actor and star of Black Panther died of Stage 4 colon cancer in 2020 at age 43, no one in the industry knew that he had been battling the disease. While these deaths were given a medical cause, I believe the larger issue of Black male celebrities not talking about their struggles plays an undeniable role.
When a celebrity’s image matters more to the public than their real-life challenges, it is often referred to as the parasocial relationship.
How parasocial relationships have changed
First coined in 1956 by sociologists Donald Horton and R. Richard Wohl, the “para-social interaction,” was a kind of psychological relationship experienced by how television audiences related to performers. Today, parasocial interactions apply to social media platforms. As audiences are repeatedly exposed to media personas, we develop illusions of intimacy, friendship and identification.
They’re at our fingertips and in front of our eyes every second of every day. Clinical psychologist Bethany Cook told Stylecaster that “social media allows the untouchable to become touchable.”
And the lines between reality and fiction are increasingly more blurred than when Horton and Wohl conducted their study.
In reality, the networks of intimacy that we develop with celebrities are based on impersonal forms of communication.
For example, two days before tWitch died, he posted a dance video to his Instagram with his wife, Allison. While dancing Instagram posts come off as pure fun, they are mostly a marketing strategy to increase brand awareness, not an innocent glimpse into a dancer’s “off-time.”
Today’s celebrity and performer are involved in the curation of webs of intimacy and presumed friendships which makes it difficult to see reality. For example, when a celebrity we follow is struggling with a mental health issue.
Significantly, there is a long history of Black male performers burying mental health issues until they tragically and unexpectedly die.
Black men have been dying on-and-off stages for centuries
William Henry Lane (1825–1852), also known as Master Juba, was the first Black dancer to reach international acclaim on both sides of the Atlantic. Born in Providence, R.I., he is remembered not only as the originator of African American tap dance but has been hailed as “the Jackie Robinson of the American stage”.
By the 1840s, Lane was billed as one of the greatest dancers of his time, no small feat if you consider that throughout the 19th century (and most of the 20th), Black performers did not get regular work unless they fit themselves into the mold cast for them by white casting directors.
However, because it was the 19th century, Lane was often forced to wear the burnt-cork mask of blackface minstrelsy, as he danced. As the sole Black performer on white stages, Lane worked day and night for 11 years in Britain until he died at only 27 years old.
As cultural sociologist Michael Pickering observes in Blackface Minstrelsy in Britain, by most accounts, Lane “had quite literally danced himself to death.”
tWitch was one of the first Black dancers on So You Think You Can Dance to catapult into the mainstream. His unique combination of personality and hip hop moves made him one of the most memorable and beloved members of the show.
While the reasons tWitch took his life are still unknown, the legacy of Lane’s death, which was the result of physical and mental exhaustion lingers eerily in his passing.
It’s time to listen to the whisper
At the end of my book, Uncle: Race, Nostalgia, and the Politics of Loyalty, I write that “Uncle Tom is our collective whisper.” Meaning that when Black men are always smiling, happy, loyal and constantly performing, that state of “on-ness” comes at a cost.
Redd Foxx in 1966.
(John E. Reed/Coast Artist Management0
In centuries past, working oneself to death meant that performers died suddenly like Lane or the legendary comedian Redd Foxx, who suffered a heart attack on set in 1991 after working in the industry for 56 years. The trailblazing dancer, actor, and choreographer Gregory Hines, who revitalized tap dance in the 1980s, also died young at age 57 after a short battle with cancer.
Today, it is more likely that Black celebrities—especially those who make a career of entertaining primarily white audiences—suffer in silence until they die suddenly, take their own lives and/or have violent public outbursts.
The “slap heard around the world,” for instance, at the 2022 Oscars was not just about two Black male entertainers having an inappropriate altercation; it was a glimpse into Black mental health where the cost of playing the “nice guy,” as Tayo Bero argued in a piece for the Guardian, takes an often-invisible toll.
A 2015 report by the U.S. National Center for Health Statistics found that only 26.4 per cent of Black and Hispanic men ages 18 to 44 who experienced daily feelings of anxiety or depression were likely to have used mental health services, compared with 45.4 per cent of non-Hispanic white men with the same feelings.
The Mental Health Commission of Canada reports similar disparities noting that between 2001 and 2014, 38.3 per cent of Black Canadians with “poor or fair self-reported” mental health used mental health services compared with 50.8 per cent of white Canadians.
Black male celebrities who are chasing white approval are self-destructing in front of our very eyes. The whispers have become non-stop noise. It’s time for celebrities with power to do more than post condolences on social media. They need to be part of the process to create sustainable structures and supports for Black men in the industry. When that happens, the parasocial relationship might be key to changing lived realities.
My hope is that Stephen ‘tWitch’ Boss’s death does not overshadow the life he lived. And that the entertainment industry finally breaks down the wall of shame that keeps too many in the closet about their mental health struggles.
If you or someone you know is struggling with suicidal thoughts or mental health matters, you can get help here: Talk Suicide Canada: 1-833-456-4566 (phone) | 45645 (text between 4 p.m. and midnight ET)
Leslie Dos Remedios has lots to talk about these days.
The Vancouver theatre artist is preparing to direct For Now, a play on gender, sexuality, and privilege. Green Thumb Theatre commissioned this work by Scott Button.
In addition, she’s acting in the monthlong run of the Arts Club’s Me Love BINGO!: Best in Snow. And next month, Dos Remedios performs two roles in the world premiere of Instantaneous Blue. It’s a Mitch and Murray Productions play about a family’s struggle with dementia.
“I’ve honestly never been busier,” Dos Remedios tells Pancouver over Zoom. “The last three years have been bananas busy for me, which is great.”
Aaron Craven wrote the semi-autobiographical Instantaneous Blue. The show revolves around a couple who must juggle work responsibilities with caring for aging parents and bringing a child into the world.
“We’re exploring this kind of generational gap between the person becoming a caregiver for parents, but also becoming a caregiver for their own children—and what it’s like having those personal resources really pulled thin,” Dos Remedios says.
In the first act, she plays a care aide named Grace. Then she becomes Reyna, another care aide, in the second act.
The show also includes performances by Patti Allan, Tom McBeath, Charlie Gallant, Kayla Deorksen, Jesse Miller, and Eric Breker.
The poster for Instantaneous Blue conveys what happens in the brain of someone with dementia.
Dos Remedios family dealt with dementia
She has tremendous respect for health-care workers who look after patients with dementia and manage hopes and expectations of family members.
“They’re also setting them up with social supports that they might need later on, giving them options on how to cope with their loved ones that are in care,” Dos Remedios says.
She points out that many health workers are people of colour or new immigrants. And family members sometimes criticize them over how they care for an elderly person.
Dos Remedios witnessed this system firsthand before her grandmother passed away last September with dementia. It came at a complicated time in the actor-director’s life, just as she was juggling a demanding work schedule and raising a three-year-old.
She points out that during the pandemic, people must make appointments to visit elders in care homes.
“If anything went wrong with getting the kid ready, I missed the window to see my grandma,” she says.
An unconventional BINGO! play
Meanwhile, in Me Love BINGO!: Best in Snow, Dos Remedios plays an assistant to Kyle Loven, who’s in drag telling stories while managing bingo games.
“It’s set up like a bingo hall,” she says. “It’s very interactive.”
She relates that the room is supposed to look unprofessional. Audience members become participants by interacting with others at their tables.
This is not a performance in which people simply sit in a darkened theatre. They’re not watching a traditional theatrical performance.
“It’s asking people to go to an uncomfortable space,” Dos Remedios emphasizes.
She acknowledges that Me Love BINGO! isn’t for everyone—and some have walked out.
“The Arts Club tried something new,” the theatre artist adds. “That was so different than a lot of the shows they produce. It’s not The Sound of Music. If you want The Sound of Music, go see The Sound of Music.”
She’s also thrilled with this unconventional show’s creative process and loves the artists involved in it.
“This is the most outside of a traditional theatre form that I’ve ever worked in with Me Love BINGO!” Dos Remedios declares. “To me, that was a really special experience.”
Leslie Dos Remedios plays an assistant to bingo director Kyle Loven in Me Love Bingo! Arts Club photo.
Theatre doesn’t need to be a grind
When she studied theatre at York University and Studio 58, she was taught that the director stood at the top of a hierarchy. In those days, Dos Remedios says, instructors demanded that students “become a Studio 58 type of actor or UBC type of actor or SFU type of actor”.
After being hired to teach acting several years later, Dos Remedios offered a similar type of training. She would instruct students on the importance of learning their lines and let them know when they weren’t working hard enough.
Then it dawned upon her that this teaching methodology wasn’t nurturing individual expression. Now, Dos Remedios embraces an altogether different approach.
“In my personal philosophy as a teacher and even going into directing, I think we, as theatre artists, can marry rigour with humanity,” the actor-director insists. “I really, truly believe that.”
She acknowledges that this might not be the easiest way to create theatre—and that’s okay. “I think we need to be starting to think about a new way to create work and to create artistic expression.”
This approach makes it easier for people who might be caregivers to continue working, including parents of young children.
On the positive side, Dos Remedios has seen some changes in the theatre world, including in recent projects that she’s associated with. But she’s also saddened by the number of talented theatre artists who’ve quit. Others switched to becoming screen actors.
“They’re like, ‘If I’m going to work this hard, I’m going to get paid film and TV bucks. I’m not going to do the theatre thing anymore.’ To me, we’ve lost so many amazing artists to that,” Dos Remedios laments.
Leslie Dos Remedios will direct For Now, which was written by Scott Button (above). Photo courtesy of Green Thumb Theatre.
Directing with humanity
She maintains that it’s possible to create exceptional theatre within a flexible workplace while keeping reasonable working hours.
“It’s my job as the director to be clear—find the clarity in the story and find the clarity in the directing,” Dos Remedios says. “And if I do my job properly, we don’t need to work a 48-hour workweek. No other industry does that at the same pay rate we do.”
Green Thumb Theatre recently announced that it’s accepting submissions from Lower Mainland performers to play the roles of Bex and Jacob in For Now. “It focuses on LGBT issues—trans queer youth,” she says.
According to Dos Remedios, this production has been cast authentically, with trans and queer artists playing roles for these characters. Furthermore, she wants people to know that For Now is not another “queer trauma play”.
“It’s about the joy of discovery,” she says. “It’s about the evolving friendship between these two characters and how it changes when one person’s identity shifts.”
As a queer artist of Chinese ancestry, Dos Remedios says that she would like to see more stories about marginalized people go in this direction.
“I want my presence to be the story—my presence telling whatever play it is, whatever story it is,” she declares. “The fact that I’m in it, the fact that I’m there, makes it an Asian Canadian story, makes it a queer story. It doesn’t have to be about ‘I’m coming out’ or ‘there’s a racial attack on me’.”
For Now, which is intended for students in Grade 7 and up, is about the friendship between a queer teen from a privileged background and a working-class straight teen. Green Thumb Theatre photo.
Discrimination in theatre world
Dos Remedios has a Portuguese surname because her ancestral roots are in Macau, a city near Hong Kong that was once a Portuguese colony. Pancouver asks if she’s ever felt discrimination in her professional working life because she’s a person of colour.
“It can feel very whitewashed when you’re one of two people of colour in a staff of 20,” she replies.
Moreover, she’s felt at times like she has to be the one to call out what another person said because it felt weird to her. That’s because none of the white people present was saying anything.
On a couple of occasions, Dos Remedios belies that she was invited to audition because she’s an equity artist, even though there were no intentions to cast her.
“Feeling that kind of tension—hostility—in the room has definitely happened,” Dos Remedios says.
She’s also well aware of potential career repercussions for speaking out.
Once when she did this, she recalls this as the response: “Well, we’re artists. We’re supposed to push the boundaries. I’m not going to apologize for this, that, and the other thing.”
Finding a work-life balance
So, how should younger artists of colour respond to situations like this? Dos Remedios says that she never wants to tell an artist of colour anything that puts them in a position where they might feel endangered or unsafe.
“I would say: go to somebody you trust. Go to somebody—an advocate—who you maybe doesn’t have skin in the game and who’s going to be able to say, ‘Hey, I can speak to people on your behalf.’ ”
Over the longer term, she hopes that more companies think about how the impact of their reporting structures. Dos Remedios believes that boards of directors can ensure that a Black, Indigenous, or person of colour shouldn’t have to file a complaint to a person whom they report to.
Meanwhile, in the past three or four years, she has stopped basing milestones for achievement and success on other people’s approval and validation.
Rather, Dos Remedios focuses far more on what makes her happy. And this has coincided with finding a great deal more theatrical work, even as she raises her three-year-old child and seeks a good work-life balance.
“I don’t think we live in a world of ‘the show must go on’ anymore with COVID and all that,” Dos Remedios says. “I think we have to be movable, bendable, flexible, and adaptable.”
The Arts Club presents Me Love Bingo!: Best in Snow until January 1 on the Newmont Stage at the BMO Theatre Centre. Mitch and Murray Productions presents Instantaneous Blue at the Waterfront Centre from January 5 to 22, with Mondays off.Follow Charlie Smith on Twitter @charliesmithvcr and Pancouver on Twitter @PancouverMedia.
…a rare and unique holiday letter from Nancy and Larry White I’m not sure this will ever get sent. I’m not sure where I have viable snail mail addresses to so many of you, so it may also end up as a blog post. (Yup, just became a blog post!) I realize I SOOO enjoy … Continue reading Holiday Missives With AI Help…
I’ve been discussing Mastodon UX wishlists with some new acquaintances there. This excerpt from A Bloomberg terminal for Mastodon concludes with part of my own wishlist.
In a Mastodon timeline, a chatty person can dominate what you see at a glance. When we participate in social media we are always making bids for one another’s attention. As publishers of feeds it’s wise to consider how a flurry of items can overwhelm a reader’s experience. But it’s also useful to consider ways that feed readers can filter a chatty source. Steampipe’s SQL foundation affords an easy and natural way to do that. Here’s part of the query that drives the list view.
select distinct on (list, person, hour) -- only one per list/user/hour
person,
url,
hour,
toot
from
data
order by
hour desc, list, person
It was easy to implement a rule that limits each person to at most one toot per hour. Next steps here will be to apply this rule to other views, show the number of collapsed toots, and enable such rules on a per-person basis.
As a warmup exercise, I decided to first add a simple control for boosts that enables me to see my home timeline with or without boosts. To give technically-inclined readers a sense of what’s involved in doing this kind of thing with Steampipe, I’ll describe the changes here. I’m obviously biased but I find this programming environment to be accessible and productive. If it seems that way to you as well, you might want to try out some of the items on your own UX wishlist. And if you do, let me know how it goes!
Here are the original versions of the two files that I changed to add the new feature. First there’s home.sp which defines the dashboard for the home timeline.
dashboard "Home" {
tags = {
service = "Mastodon"
}
container {
// a text widget with the HTML links that define the menu of dashboards
}
container {
text {
// a block that displays the HTML links that form a menu of dashboards
}
card {
// a block that reports the name of my server
}
input "limit" {
width = 2
title = "limit"
sql = <<EOQ
with limits(label) as (
values
( '50' ),
( '100' ),
( '200' ),
( '500' )
)
select
label,
label::int as value
from
limits
EOQ
}
}
container {
table {
title = "home: recent toots"
query = query.timeline
args = [ "home", self.input.limit ]
column "person" {
wrap = "all"
}
column "toot" {
wrap = "all"
}
column "url" {
wrap = "all"
}
}
}
}
And here’s the new version. It adds an input block called boosts, and passes its value to the referenced query.
dashboard "Home" {
tags = {
service = "Mastodon"
}
container {
// a text widget with the HTML links that define the menu of dashboards
}
container {
text {
// a block that displays the HTML links that form a menu of dashboards
}
card {
// a block that reports the name of my server
}
input "limit" {
// as above
}
input "boosts" {
width = 2
title = "boosts"
sql = <<EOQ
with boosts(label, value) as (
values
( 'include', 'include' ),
( 'hide', ' ' ),
( 'only', ' 🢁 ' )
)
select
label,
value
from
boosts
EOQ
}
}
container {
table {
// as above
args = [ "home", self.input.limit, self.input.boosts ]
}
}
}
Steampipe dashboards are built with two languages. HCL (Hashicorp configuration language) defines the UX widgets, and SQL fills them with data. In this case we’re selecting static values for the boosts input. But any Steampipe query can run there! For example, here is the input block I use on the dashboard that filters the timeline by the list to which I’ve assigned people.
input "list" {
type = "select"
width = 2
title = "search home timeline"
sql = <<EOQ
select
title as label,
title as value
from
mastodon_list
order by
title
EOQ
}
Now here is the referenced query, query.timeline, from the file query.sp which contains queries used by all the dashboards.
query "timeline" {
sql = <<EOQ
with toots as (
select
account_url as account,
case
when display_name = '' then user_name
else display_name
end as person,
case
when reblog -> 'url' is null then
content
else
reblog_content
end as toot,
to_char(created_at, 'MM-DD HH24:MI') as created_at,
case
when reblog -> 'url' is not null then '🢁'
else ''
end as boosted,
case
when in_reply_to_account_id is not null then ' 🢂 ' || ( select acct from mastodon_account where id = in_reply_to_account_id )
else ''
end as in_reply_to,
case
when reblog -> 'url' is not null then reblog ->> 'url'
else url
end as url
from
mastodon_toot
where
timeline = $1
limit $2
)
select
account,
person ||
case
when in_reply_to is null then ''
else in_reply_to
end as person,
boosted || ' ' || toot as toot,
url
from
toots
order by
created_at desc
EOQ
param "timeline" {}
param "limit" {}
}
And here is the new version of that query.
query "timeline" {
sql = <<EOQ
with toots as (
// as above
),
boosted as (
select
$3 as boost,
boosted,
account,
in_reply_to,
person,
toot,
url
from
toots
order by
created_at desc
)
select
account,
person ||
case
when in_reply_to is null then ''
else in_reply_to
end as person,
boosted || ' ' || toot as toot,
url
from
boosted
where
boost = boosted
or boost = 'include'
or boost = 'n/a'
EOQ
param "timeline" {}
param "limit" {}
param "boost" {}
}
The original version uses a single CTE (aka common table expression aka WITH clause), toots, to marshall data for the concluding SELECT. The new version inserts another CTE, boosts, into the pipeline. It uses $3 to reference param "boost" {} which maps to the self.input.boosts passed from home.sp
The SQL code is all standard. Postgres is the engine inside Steampipe, and I sometimes use Postgres-specific idioms, but I don’t think any of those are happening here.
The HCL code may be unfamiliar. Steampipe uses HCL because its core audience are DevSecOps pros who are familiar with Terraform, which is HCL-based. But its a pretty simple language that can be used to describe all kinds of resources. Here the resources are widgets that appear on dashboards.
The other thing to know, if you want to roll up your sleeves and try building your own dashboards, is that the developer experience is — again in my biased opinion! — pretty great because if you’re using an autosaving editor you’ll see your changes (to both HCL and SQL code) reflected in realtime.
To illustrate that, here’s the screencast we included in our blog post introducing the dashboard system.
Not shown there, because we wanted to focus on the happy path, is realtime feedback when your SQL queries provoke Postgres errors. The experience feels very much like the one Bret Victor champions in Inventing on Principle. The core principle: “Creators need an immediate connection to what they’re creating.”
Here’s the wrong way that too often constrains us.
If there’s anything wrong with the scene, or if I go and make changes, or if I have further ideas, I have to go back to the code, and I edit the code, compile and run, see what it looks like. Anything wrong, I go back to the code. Most of my time is spent working in the code, working in a text editor blindly, without an immediate connection to this thing, which is what I’m actually trying to make.
And here is the right way.
I’ve got this picture on the side, and the code on the side, and this part draws the sky and this draws the mountains and this draws the tree, and when I make any change to the code, the picture changes immediately. So the code and the picture are always in sync; there is no compile and run. I just change things in the code and I see things change in the picture.
We want to work the right way wherever we can. The experience isn’t available everywhere, yet, but it is available in Steampipe where it powerfully enables the experimentation and prototyping that many of us are inspired to do as we delve into Mastodon.
If you want to try this for yourself, please check out the setup instructions for the plugin that maps Mastodon APIs to Postgres tables, and the dashboards that use those tables, and ping me (on Mastodon if you like!) with any questions you may have.
Mic Check helps you with your microphone setup to make you podcast-ready. And Enhance Speech makes your voice recordings sound as if they were recorded in a professional studio.
Here are two samples spoken in German. The first one is recorded with just the laptop microphones of a Lenovo ThinkPad Z13. You will notice the room reverb and if you wear headphones, you might even hear a Dyson vacuum behind the next door.
Before enhancement
I uploaded this mp3 file to the Adobe website where it was processed and made available for download as a wav file. Even without headphones you should be able to hear a stark difference. These are not two recordings with two microphones. The second sample is just enhanced from the first sample.
The same recording after enhancement
Adobe is beta-testing a podcast suite for live recording on their website and these tools are both free to use. You have to have a registered Adobe account.
It’s interesting to hear people describe the experience of migrating away from big tech sites like Twitter and Facebook and moving to Mastodon as a return to the web “the way it used to be.” I have to admit, I share that feeling. I did lots of blogging and writing during the 2000’s and the moment we’re in now reminds me a lot of those times.
So it’s really resonated with me to see what feels like an old school civic hacking project spring up in Philadelphia. In no small part because it is a project focused on telephony software and apps (which I spent the better part of 2000’s blogging about). The PhilTel project is a collective of phone hackers that is using open source software to enable free phone calling from pay phones in Philadelphia. They are doing some very interesting things, and if you haven’t read up on the work of this group you should.
Reading about this project reminds me of the civic hacking movement the way it used to be. Creative, talented people using the skills that they have to make their city better by assisting those that need help. Not waiting for official sanction or support, just applying their skills to solve a problem that they think needs to be fixed. It’s pretty cool to see.
As much as learning about this project strums my civic heartstrings, I don’t want to go back. We’ve learned so much since those early days of civic hacking. We know that we need to think carefully about who gets to decide what projects get worked on, who gets to work on them, and who we build them for (and with) to make cities better places.
But I do think it’s worth remembering what made the early days of the civic hacking movement so special. The sense of community, the sense of possibility, the belief that groups of people outside of government can make a difference in their community. Savvy public sector leaders now understand the importance of harnessing this enthusiasm, of using it help drive innovative solutions. It’s common now to see civic hacking groups working directly with cities and states. It wasn’t always that way.
If nothing else, the PhilTel project is a reminder of the power of civic hacking to highlight ways that cities can be better. To show how innovative thinking can foster creative solutions to complex problems.
We're back in the UK to see family over Christmas (our first trip back since 2019). Here are a few notes from the past couple of weeks.
Datasette 0.63.3
In addition to the Datasette 1.02a2 alpha (described in detail here) I also published a small bug fix release for the 0.63.x stable branch. Quoting the release notes in full:
Fixed a bug where datasette --root, when running in Docker, would only output the URL to sign in as root when the server shut down, not when it started up. (#1958)
You no longer need to ensure await datasette.invoke_startup() has been called in order for Datasette to start correctly serving requests - this is now handled automatically the first time the server receives a request. This fixes a bug experienced when Datasette is served directly by an ASGI application server such as Uvicorn or Gunicorn. It also fixes a bug with the datasette-gunicorn plugin. (#1955)
That second fix ended up taking longer than expected.
The root of that fix was that back in Datasette 0.63 I introduced the need to call await datasette.invoke_startup() as part of Datasette's setup process - mainly to trigger plugins that might need to run their own async setup code.
This turned out to break a bunch of unexpected things - most notably, it affected any time people wanted to run Datasette using an ASGI handler such us Gunicorn or Uvicorn.
The core problem was that the Datasette() class constructor can be called synchronously, but needed a subsequent await ... call to run those async def setup methods.
I realized that a neater way to handle this would be to introduce a mechanism such that the first time anyone attempted to run an HTTP request through Datasette - an operation that always involved an await - the invoke_startup() method would be called automatically.
I got that working, but in doing so I ran into a longer-running set of problems.
Datasette has around 1,200 tests at this point, and parts of the test suite date back to the start of the project and no longer reflect my preferred way of writing tests.
I've started running into "too many open files" errors running the test suite on macOS, and have so far not quite tracked down the best way to keep open file handles under control.
Test failures were hampering my efforts to fix the issue, so I used this as the impetus to refactor a large chunk of the test suite.
Several hundred of Datasette's tests now share a single in-memory fixtures database - previously, these tests were using a fixtures.db database file created in a temporary directory.
There's still more test refactoring that I want to do, described in this issue, but I'm happy with the progress I've made so far.
datasette-ripgrep, cosmetic upgrade
I built datasette-ripgrep a couple of years ago - it's a Datasette plugin that provides a UI for running ripgrep code search queries - and linking to the results. It's very handy for finding uses of APIs that I might want to deprecate.
It was only a year ago that I opened the 2021 Year in Review by noting that the news felt like a bit of a drag; the contrast to 2022 has been stark. The biggest story in tech not just this year but, I would argue, since the advent of mobile and cloud computing, was the emergence of AI. AI looms large not simply in terms of products, but also its connection to the semiconductor industry; that means the impact is not only a question of technology and society, but also geopolitics and, potentially, war. War, meanwhile, came to Europe, while inflation came to the world; tech valuations collapsed and the crypto bubble burst, and brought to light one of the largest frauds in history. All of this was discussed on Twitter, even as Twitter itself came to dominate the conversation, thanks to its purchase by Elon Musk.
Stratechery, meanwhile, entering its 10th year of publishing, underwent major changes of its own; a subscription to the Daily Update newsletter transformed into a subscription to the Stratechery Plus bundle, including:
Sharp Tech, my new twice-weekly hour-long show with Andrew Sharp.
Sharp China, a new weekly show with Andrew Sharp and Sinocism’s Bill Bishop.
Stratechery Interviews, meanwhile, became its own distinct brand, befitting its weekly schedule and increased prominence in Stratechery’s offering. I am excited to see Stratechery Plus continue to expand in 2023.
This year Stratechery published 33 free Weekly Articles, 111 subscriber Updates, and 36 Interviews. Today, as per tradition, I summarize the most popular and most important posts of the year on Stratechery.
The five most-viewed articles on Stratechery according to page views:
AI Homework — It seems appropriate that this article, written after the launch of ChatGPT, was the most popular of the year because AI is, in my estimation, the most important story of the year. This article used homework as a way to discuss how verifying and editing information will not only be essential in the future, but already are. I wrote two other articles about AI:
The AI Unbundling — AI is starting to unbundle the final part of the idea propagation value chain: idea creation and substantiation. The impacts will be far-reaching.
Meta Myths — Meta deserves a bit of a discount off of its recent highs, but a number of myths about its business have caused the market to over-react. See also:
Instagram, TikTok, and the Three Trends — Trends in medium, AI, and user interaction underpin Instagram’s response to TikTok, and will determine Meta’s long-term moat.
Shopify’s Evolution — Shopify should build an advertising business to complement Shop Pay and the Shopify Fulfillment Network. An additional challenge for Shopify is the changing nature of Amazon’s moat:
Beyond Aggregation: Amazon as a Service — Amazon’s new Buy With Prime announced the arrival of Amazon Logistics as a Service, and is a big red flag for Shopify.
Digital Advertising in 2022 — The advertising has shifted from a Google-Facebook duopoly to one where Amazon and potentially Apple are major forces. Speaking of Apple:
Data and Definitions — Explaining exactly why Apple’s approach to ATT is anti-competitive.
Nvidia In the Valley — Nvidia is in the valley in terms of gaming, the data center, and the omniverse; if it makes it to future heights its margins will be well-earned.
Semiconductors and Geopolitics
Geopolitics, including the Russian invasion of Ukraine and relations with China, were major stories this year; semiconductors figured prominently in both.
Tech and War — The reaction to the Ukraine invasion has been a demonstration of tech capabilities; those capabilities may be the key to compelling China to pressure Russia.
Political Chips — Chips are the clearest example that economic efficiencies will not be the ultimate decider of technology’s end state: politics will play an important role.
A central theme on Stratechery has always been platforms and Aggregators.
OpenSea, Web3, and Aggregation Theory — OpenSea is positioned as another Aggregator, which is evidence that Web 3 is a layer on top of the Internet, not a replacement.
Gaming the Smiling Curve — The spate of recent acquisitions in the gaming space — Take-Two and Zynga, Microsoft and Activision, and Sony and Bungie — make sense in the context of the Smiling Curve.
This year saw a lot of upheavel in the streaming space; some of these outlooks have already came true (Netflix and ads), remain to be seen (Warner Bros. Discovery), or aren’t looking too good (consolidation may happen in streaming, but cable is looking like a weak player).
Why Netflix Should Sell Ads — Netflix has been resolutely opposed to selling ads, prioritizing the user experience; however, the market conditions for streaming have changed, and so should Netflix.
Cable’s Last Laugh — Cable companies survived the great unbundling thanks to selling Internet service; they may be best place to make the bundle of the future.
The intersection between tech and society has never been more clear than over the last few months as Twitter, a relatively small and unimportant company in business terms, has dominated the news, thanks to its societal impact.
The Current Thing — If businesses are subject to Aggregation Theory, then so are ideas: this is the root of the “The Current Thing” meme, and it should drive a re-evaluation of how we think about moderating content on the Internet.
Zero-COVID and Free Speech — Zero-COVID is possible [Update: it’s not], but few of us in the West are willing to pay the costs; the exact same reasoning applies to free speech; in both cases China-lite is the worst possible strategy.
Narratives — What Elon Musk got wrong about Twitter, journalists and VCs got wrong about FTX, and Peter Thiel got wrong about crypto and AI — and why I made many of the same mistakes along the way.
Other Company Coverage
Microsoft continues to show strength, Apple didn’t raise prices (although, in retrospect, the below Article overstates the case), Meta continues to pursue the Metaverse, and what a private Twitter might have been.
Microsoft Full Circle — Microsoft has come full circle from the company that cared more about Windows than Office; the retirement of the Office name is possible precisely because Microsoft gave up on Windows and went to the cloud.
Stratechery Interviews
This year Stratechery Interviews became a standard weekly item, with three distinct categories:
Intel CEO Pat Gelsinger about IDM 2.0, the acquisition of Tower Semiconductor, and what went wrong at Intel
Nvidia CEO Jensen Huang about the history of Nvidia, manufacturing intelligence, and how GPUs undergird Aggregation Theory; then, six months later, the Ada Lovelace GPU and the Omniverse Cloud
This was a new type of interview I launched this year: given that it is impossible to cover startups objectively through data, I asked founders to give their subjective view of their businesses and long-term prospects.
Roblox Ads, Walmart Land, Internal Economies — Roblox announced an advertising platform; I’m skeptical about the value to brand advertisers, but that doesn’t mean it won’t be valuable to Roblox itself.
I am so grateful to the subscribers that make it possible for me to do this as a job. I wish all of you a Merry Christmas and Happy New Year, and I’m looking forward to a great 2023!
James Bennett provides an opinionated guide to setting up Python tools for linting, code formatting and and other code quality concerns. Of particular interest to me is his section on packaging checks, which introduces a whole bunch of new-to-me tools that can help avoid accidentally shipping broken packages to PyPI.
Last night after work I decided to see how easy it would be to run a TakahēActivityPub instance under Piku, my tiny Python-oriented PaaS.
Self-hosting Mastodon is all the rage, but having to deal with a full-blown installation of Ruby (which is always a pain to install properly, even if you use rbenv), plus the abomination that is Sidekiq and the overall Rube Goldberg-esque architectural approach that is almost mandatory to deal with the complexities of ActivityPub is just something I don’t want to maintain. Ever. Even inside Docker.
Which is why I have been developing my own ActivityPub server using Sanic and a very lightweight asyncio-based approach at handling all the transactional aspects of ActivityPub atop SQLite. And let me tell you, I honestly wish the protocol was less about doing what boils down to P2P webhooks with PEM signatures embedded in requests.
Blue birds are cool. Well, at least flightless ones.
But Takahē is now aiming to support client apps as of version 0.6, is built on Django (which I have always loved as a framework), and it saves me from the trouble of building everything from scratch, so… I had to try it out.
More to the point, Django is exactly what Piku was originally designed to run.
Besides running as a WSGI app, Takahē uses an async stator to handle all the background tasks (which is also exactly the pattern I aim for and designed Piku to support), so I just had to see how easy it was to get it running under Piku on very low-end hardware.
I have a 4GB Raspberry Pi 4s set up as an SSD-backed Proxomox server, hosting several different arm64 LXC containers I use for developing stuff. I love it because I can use LXC CPU allocations to throttle things and make sure they run fast enough on very low-end hardware, plus I can just snapshot, mess up and restore entire environments.
So I set up an Ubuntu 22.04 container with 1GB of RAM and access to 2 CPU cores, capped to 50% overall usage–which is roughly the performance of a Raspberry Pi 2 give or take, albeit with a fully 64-bit CPU.
I deployed Piku, set up a CloudFlare tunnel, and then went to town.
Here’s the configuration I used, annotated. First the ENV file:
# Yes, I went and got it to use SQLite, and it nearly worked 100%
TAKAHE_DATABASE_SERVER=sqlite:////home/piku/takahe.db
# This is what I eventually migrated to (more below)
# TAKAHE_DATABASE_SERVER=postgres://piku:<password>@localhost/takahe
# I actually love Django debugging, and with it on I can see the inner workings
TAKAHE_DEBUG=true
# You know who uses this password, don't you?
TAKAHE_SECRET_KEY=pepsicola
# No, it's not the one I'm actually using.
# Anyway, this next one breaks a little on Piku, so I need to revise parsing for this case.
TAKAHE_CSRF_TRUSTED_ORIGINS=["http://127.0.0.1:8000", "https://127.0.0.1:8000"]
TAKAHE_USE_PROXY_HEADERS=true
TAKAHE_EMAIL_SERVER=console://console
TAKAHE_MAIN_DOMAIN=insightful.systems
TAKAHE_ENVIRONMENT=development
TAKAHE_MEDIA_BACKEND=local://
TAKAHE_MEDIA_ROOT=/home/piku/media
TAKAHE_MEDIA_URL=https://insightful.systems/media/
TAKAHE_AUTO_ADMIN_EMAIL=<my e-mail>
# I also added a Redis server later, without any relevant impact in RAM usage
TAKAHE_CACHES_DEFAULT=redis://127.0.0.1:6379/0
SERVER_NAME=insightful.systems
# This is all Piku config from here on down
# I need IPv6 off for sanity inside Proxmox
DISABLE_IPV6=true
LC_ALL=en_US.UTF-8
LANG=$LC_ALL
# This ensures nginx only accepts requests from CloudFlare, plus a few extra tweaks
NGINX_CLOUDFLARE_ACL=True
NGINX_SERVER_NAME=$SERVER_NAME
# These are caching settings for my dev branch of Piku
NGINX_CACHE_SIZE=2
NGINX_CACHE_TIME=28800
NGINX_CACHE_DAYS=12
# This has nginx cache these prefixes
NGINX_CACHE_PREFIXES=/media,/proxy,/static/admin
# This maps static user media directly to an nginx route
NGINX_STATIC_PATHS=/media:/home/piku/media,/static:static,/robots.txt:static/robots.txt
PORT=8000
# You want to set these, trust me. I should make them defaults in Piku.
PYTHONIOENCODING=UTF_8:replace
PYTHONUNBUFFERED=1
TZ=Europe/Lisbon
# This tells uWSGI to shut down idle HTTP workers
# Saves RAM, but startup from idle is a bit more expensive CPU-wise
UWSGI_IDLE=60
# We need to run at least 2 uWSGI workers for Takahe
UWSGI_PROCESSES=2
# Each worker will have this many threads
# (even though I'm only giving this 2 cores)
# to match the original gunicorn config.
UWSGI_THREADS=4
In essence, I removed gunicorn (which I could use anyway) to let uWSGI handle HTTP requests and scale down to zero (saving RAM). And yes, Piku also supports release activities, thanks to Chris McCormick.
And that was it. Zero code changes. None. Nada. And I can use exactly the same setup on any VPS on the planet, thanks to Piku.
After a little faffing about with the media storage settings (which I got wrong the first time around, since Takahē also uses /static for its own assets), I had a fully working ActivityPub instance, and, well… John Mastodon just happened to sign up:
Every ActivityPub server ought to make this their demo account.
Takahēnearly works with SQLite, but sadly it relies on JSON_CONTAINS, which is an unsupported feature in SQLite (but one which PostgreSQL excels at).
The upshot of this was that the stator worker was very sad and bombed out when trying to handle hashtags–but all critical stuff worked, so there might well be a workaroud.
But I took some time after breakfast to migrate the database, and since my Django skills are rusty, here are my notes:
# Open a shell to Piku
ssh -t piku@activitypub.lan run takahe bash
sudo apt install postgresql
python manage.py dumpdata > /tmp/dump.json
sudo su - postgres
psql
-- Set up the database
create user piku;
create database takahe;
alter role piku with password '<mysecret>';
grant all privileges on database takahe to piku;
alter database takahe owner to piku;
# Reset all the migrations, just in case
find . -path “*/migrations/*.py” -not -name “__init__.py” -delete
find . -path “*/migrations/*.pyc” -delete
# Reapply them
python manage.py makemigrations
python manage.py migrate
# Wipe all default entities
python manage.py shell
from django.contrib.contenttypes.models import ContentType
ContentType.objects.all().delete()
# Load everything back
python manage.py loaddata /tmp/dump.json
Overall, I’m quite impressed with the whole thing. Even with such measly resources and Linux’s tendency to take up RAM with buffers, Takahē under Piku is taking up around 100MB per active worker (2 web handlers, plus the stator worker), plus less than 50MB for PostgreSQL and nginxtogether.
So I’m seeing less than 512MB of RAM in actual use, and a steady <10% CPU load inside the container as the stator keeps picking up inbound updates, handling them (including any outbound requests) and doing all the messy housekeeping associated with ActivityPub:
These are the stats a few hours later in the day, after publishing this post.
But here’s the kicker: Since this is being capped inside LXC, that is actually around 5% overall CPU load on the hardware–which should translate to something like 2% of CPU usage on any kind of “real” hardware.
With only one active user for now (but following a few accounts already), this is very, very promising.
I have no real plans to leave mastodon.social for my own domain, but using Takahē to host a small group of people (or a company) with nothing more than a tiny VPS seems entirely feasible, and is certainly in my future.
Right now, I’m going to try to contribute by testing various iOS clients (I will be using the Takahē public test instance as well) and do some minor tweaks to my install, namely:
Setting up nginx caching. Cloudflare is already caching one third of the data, but I want to bulk up this setup so that I can eventually move it to Azure, and I’ve been meaning to add that to Piku anyway.
Fine-tuning the stator to see how it scales up or down (I might want to try to scale it down further).
Trying gunicorn to see if it makes any difference in overall RAM and CPU.
Seeing if I can get it to work on Azure Functions (that is sure to be fun, although the current SDK failed to install on my M1 and I haven’t tried since).
Look at how media assets are handled and see if I can add a patch to support Azure Storage via my own aioazstorage library.
Deploy on my k3s cluster, to get a feel for how much it would cost to run on spot instances.
There goes my holiday break, I guess…
Update: A Few Days Later
I’ve since sorted out nginx caching in Piku (and will soon be merging it to main), which makes things significantly snappier. I’ve also filed #287 to improve caching via Cloudflare and #288 to have nginx immediately cache assets (which works for me, at least).
Before that, I had some fun tuning stator pauses and filed #232, which resulted in a tweak that lowered idle CPU consumption to a pretty amazing 3% in my test instance.
With the caching tweaks, gunicorn doesn’t have any real advantage against uWSGI workers, although I suspect that may be different in higher-load instances.
I’ve also tossed the source tree into an Azure Function and got it to “work”, but not fully. Right now I’m not sure that is worth pursuing given I still need an external database, but I’m really curious to try again in a few months’ time.
This year has been packed full of fantastic Pixelmator Pro updates and we’re wrapping it up with a bang. Today, we’re releasing Pixelmator Pro 3.2.3 with a groundbreaking new feature called Deband, which lets you remove posterization and compression artifacts from images with just a click. And, to top it all off, we’re getting festive with a collection of gorgeous, holiday-themed templates for social media, print, and more.
Deband
Color banding (or posterization) is a common type of image artifact especially noticeable in low-quality photos featuring gradients or large areas of solid color. Instead of smoothly blending together, colors jump abruptly from one shade to the next, forming distinct bands of color. While posterization isn’t particularly difficult to get rid of — you can blur it out or add noise to hide it — it is very much a labor-intensive task. Or, it was. With the magic of machine learning, we’ve been able to turn color debanding into an effortless, one-click process. And the results are simply incredible!
Before
After
We’ve trained the Deband algorithm to intelligently analyze colors, gradients, and textures in images, determine where the affected areas are, and seamlessly smooth them out without touching finer details in other parts of the image. Notice how all the tiny stars in the desert photo above remain crisp after color banding is removed. What’s more, Deband also tackles different compression artifacts (in heavily compressed JPEGs, for instance) so you can enjoy all-around better-looking images.
Holiday Templates
It’s the giving season and as a small gift from the Pixelmator Team, we’ve created a collection of new, holiday-themed templates you can easily customize and share with your friends and family. In this update, you’ll find 18 stunning, artist-designed templates for creating social media posts, stories, posters, and greeting cards which are fully optimized for sharing online and printing.
Today’s update is free to all existing users and is available from Mac App Store.
And that’s all for the updates this year — it’s truly been a great one for Pixelmator Pro. Let’s make 2023 even better!
Colophon: Recent publications, upcoming/recent appearances, current writing projects, current reading
How Apple could open its App Store without really opening its App Store (permalink)
Last week, Mark Gurman published a blockbuster story in Bloomberg, revealing Apple's plan to allow third-party Ios App Stores to comply with the EU's Digital Markets Act. Apple didn't confirm it, but I believe it. Gurman's sourcing was impeccable:
This is a huge deal. While Apple's "curated" approach to software delivers benefits to users, those benefits are unreliable. As I explain in a new post for EFF's Deeplinks blog, Apple only fights for its users when doing so is good for its shareholders. But when something is good for Apple shareholders and bad for its customers, the shareholders win, every time:
To see how this works, just consider Apple's record in China. First, Apple removed all working VPN apps from its Chinese App Store, to facilitate state spying on its Chinese customers:
Apple claims that its App Store is a fortress that protects its users against external threats. But the Iphone is designed to block its owners from choosing rival app stores, which means that when Apple betrays its customers, the fortress walls become prison walls. Governments know this, and they rely on it when they demand that Apple compromise its customers to totalitarian surveillance:
Now, there's an interesting contrast here. When the DFBI demanded that Apple backdoor its devices to aid in the prosecution of the San Bernardino shooters, Apple took its customers' side, bravely refusing to compromise its devices:
That was the right call to make. Does it mean that Apple doesn't value privacy for its Chinese customers' privacy as much as it values it for American customers? Does it mean that Apple respects the CCP more than it respects the FBI?
Not at all. It just means that China was able to threaten Apple's shareholders in ways that the DoJ couldn't. Standing up to the Chinese government would threaten Apple's access to 350 million middle-class Chinese potential customers, and an equal number of Chinese low-waged workers who could be tapped to manufacture Apple devices under brutal labor conditions at rock-bottom prices.
Standing up to the FBI didn't threaten Apple's shareholders the way that standing up to the CCP would, so Apple stood up for its American users and sold out its Chinese users.
But that doesn't mean that US Apple customers are safe. In the US, Apple defends its customers from rival commercial threats, but actively prevents those customers from defending themselves against Apple's own commercial threats.
Famously, Apple took its customers side over Facebook's, adding an amazing, best-in-class, one-click opt-out to tracking, which is costing Facebook $10 billion per year. You love to see it:
On the other hand…Apple secretly continued to its customers' clicks, taps, gestures, apps and keystrokes, even after those customers explicitly opted out of tracking, and used that data to build nonconsensual dossiers on every Ios owner for use in its own ad-targeting business:
Apple defended its customers against Facebook's predation, but not its own. When Apple's shareholder interests are on the line, Apple's App Store becomes a prison, not a fortress: because Apple controls which software you can install, it can (and does) block you from installing apps that extend its block on commercial surveillance to Apple itself.
Then there's the app tax. Apple charges app makers a 30% commission on all their sales, which means that certain businesses literally can't exist. Take audiobooks: audiobook sellers have 20% gross margins on their wares. If they sell their audiobooks through apps and pay a 30% vig to Apple, they lose money on every sale. Thus, the only Ios app that will sell you an audiobook is Apple's own Apple Books.
Apple Books requires authors and publishers to wrap their books in Apple's DRM, and the DMCA makes it a felony to supply your own readers with a tool to convert the books you published to a rival's format. That means that readers have to surrender every book they've bought on Apple Books if you switch platforms and ask them to follow you. It's not just social media that turns creators into digital sharecroppers.
It's not any better when it comes to the businesses that can eke out an existence under the app tax's yoke. These businesses pass their extra costs on to Apple's customers, who ultimately bear the app tax burden. Because every app maker has to pay the app tax, they all tacitly collude to hike their prices. And because mobile is a duopoly, the app tax is also buried in every Android app, because Google has exactly the same app tax as Apple (Google will also be forced to remove barriers to third-party app stores under the DMA).
All this to say that it is a terrible error to impute morals or values to giant corporations. Apple and Google are both immortal colony organisms that view human beings as inconvenient gut flora. They are remorseless paperclip-maximizing artificial life forms. They are, in other words, limited liability corporations.
"If you're not paying for the product, you're the product" sounds good, but it's absolutely wrong. You can't bribe a paperclip-maximizing colony organism into treating you with dignity by spending money with it. Companies' treatment of you depends on what they can get away with – not their "personalities." Apple doesn't respect privacy – it thinks it can make more paperclips by giving some of its customers some privacy. As soon as Apple finds a way to make more paperclips by spying on those you (say, by starting its own internal adtech business), it will spy on you, and the $1000 you spent on your Iphone will not save you.
Once you understand that corporate conduct is a matter of power, not personality, then you understand that the way to prevent companies from harming you is to meet their power with countervailing power. This is why tech worker unions matter: organized labor has historically been the most important check on corporate power, which is why tech companies are so vicious in the face of union drives:
Beyond labor, two other forces can discipline corporate conduct: regulation and competition. The biggest threat to a business's customers is that business's own shareholders. A company might defend its customers against a rival, but they will never defend its customers against its own shareholders.
Regulation and competition both impose costs on shareholder who abuse their customers: regulation can punish bad conduct with fines that come out of shareholder profits, and competition can create a race to the top as businesses seek to poach each others' customers by offering them progressively better deals.
Which brings me back to the DMA, the EU's pending regulation forcing Apple to open its app store, and Apple's leaked plans to comply with the regulation. This is (potentially) great news, because rival app stores can offer Apple customers an escape hatch from mandatory surveillance and price-gouging.
But the devil is in the details. There are so many ways that Apple can use malicious compliance to appear to offer a competitive app marketplace without actually doing so. In my article for EFF, I offer a checklist of fuckieries to watch for in Apple's plans:
Forcing software authors in Apple's Developer Program. Not only does this force developers to pay Apple for the privilege of selling to Iphone owners, but it also forces them to sign onto a Bible-thick EULA that places all kinds of arbitrary limits on their software. It's not enough for Apple to open up to rival app stores – it also must not sabotage rivals who produce competing SDKs for Ios.
Forcing App Store criteria on rival app stores. Apple mustn't be permitted to turn legitimate vetting for security or privacy risks into editorial control over which apps Ios users are allowed to use. Apple may not want to carry games that highlight labor conditions in high-tech manufacturing sweatshops:
It's fine to say that app stores must submit to third-party security certification, but they should be free to choose Apple out of a field of qualified privacy certifiers.
Requiring third-party app stores to process payments with Apple. The app tax should be disciplined by competition. Allowing Apple to extract 30% from transactions in its rivals' app stores would defeat the whole purpose of the DMA.
Arbitrarily revoking third party app stores. It's foreseeable that some third-party app stores would be so incompetent or malicious that Apple could revoke their ability to operate on Ios devices. However, if Apple were to pretextually shut down third-party app stores, it could sour Iphone owners off the whole prospect of getting apps elsewhere.
Apple must not be permitted to use its power to shut down app stores in an anti-competitive way, but distinguishing pretextual shutdowns from bona fide ones is a time-consuming, fact-intensive process that could leave customers in limbo for years.
One way to manage this is for regulators to dangle massive fines for pretextual shutdowns. In addition to this, Apple must make some provision to continue its customers' access to the apps, media and data from the app stores it shuts down.
All of this points to the role that regulators pay, even (especially) when it comes to disciplining companies through competition. The DMA is overseen by the EU Commission, which has the power to investigate, verify and approve (or reject) the standards that Apple sets for privacy, security, and app stores themselves. The Commission should anticipate and fund the regulators needed to manage these tasks quickly, thoroughly and efficiently.
Finally, Europeans shouldn't have all the fun. If Apple can do this for Europeans, it can do it for every Apple device owner. If you bought an Ios device, it's yours, not Apple's, and you should have the right to technological self determination that Europeans get when it comes to deciding which software it runs.
"Chokepoint Capitalism: How to Beat Big Tech, Tame Big Content, and Get Artists Paid, with Rebecca Giblin", on how to unrig the markets for creative labor, Beacon Press/Scribe 2022 https://chokepointcapitalism.com
"Attack Surface": The third Little Brother novel, a standalone technothriller for adults. The Washington Post called it "a political cyberthriller, vigorous, bold and savvy about the limits of revolution and resistance." Order signed, personalized copies from Dark Delicacies https://www.darkdel.com/store/p1840/Available_Now%3A_Attack_Surface.html
Red Team Blues: "A grabby, compulsive thriller that will leave you knowing more about how the world works than you did before." Tor Books, April 2023
This work licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net.
Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution.
DaveO@uncleweed
Photo from a couple of weeks ago at Rural Caprine (Goat) Farm where we met, captured by our first "official tourist… twitter.com/i/web/status/1…
I finished Melvin Sheldrake’s “Entangled Life” today. The book covers how fungus interact with humans and the rest of life. It’s a great read; each chapter discusses different aspects of fungus. The writing is personal, multi-faceted, kind and curious.
One chapter is about lichens, the complex symbiotic forms of algae and fungus that adapt so well to the hardest places on earth to live. Another covers mycorrhizal networks, the intertwingled lines of plant roots and fungal hyphae that run underneath the soil of forest floors.
The chapter on psilocybin and other entheogenic mushrooms is very interesting. Sheldrake addresses the question of whether humans and mushrooms have a symbiotic relationship in the same way lichens or plant root networks do. He concludes that probably not. Although we benefit greatly from the effects of entheogens, they don’t need us in the same way. If the relationship ended, the mushrooms would just go on living their lives under the floors of Yucatan jungles.
The audiobook is read by Sheldrake himself. He’s got a flat, soft-voiced delivery that takes a little getting used to, but eventually brings the book’s sincerity and genuine curiosity out. I liked his reading of his LSD trip to concentrate on the nature of hyphae and other mysteries of the fungal world.
I came away from the book liking Sheldrake and interested in working with fungi. I already make wine and bread, but I’d like to see about re-invigorating the fungal networks in my garden in the country. I’m going to do some investigation of what fungi like our soil in the Eastern Townships of Quebec, and maybe try to help them along. We have a great mushrooming store, the Mycoboutique, in my neighbourhood, which I am going to visit to find out about spores.
The majority of scholarship on platform governance focuses on for-profit, corporate social media with highly centralized network structures. Instead, we show how non-centralized platform governance functions in the Mastodon social network. Through an analysis of survey data, Github and Discourse developer discussions, Mastodon Codes of Conduct, and participant observations, we argue Mastodon’s platform governance is an exemplar of the covenant, a key concept from federalist political theory. We contrast Mastodon’s covenantal federalism platform governance with the contractual form used by corporate social media. We also use covenantal federalist theory to explain how Mastodon’s users, administrators, and developers justify revoking or denying membership in the federation. In doing so, this study sheds new light on the innovations in platform governance that go beyond the corporate/alt-right platform dichotomy.
And I also published an op-ed in the Toronto Star this morning. The op-ed discusses the fact that Twitter is blocking links to Mastodon instances, including my own, Scholar.social. However, Twitter is not blocking links to Gab, the alt-right social network that hosts self-identified Nazis. It seems Musk is concerned about competition from well-moderated, tiny Mastodon instances, but not so much from the alt-right.
Since taking over Twitter, Elon Musk has done nearly everything wrong, destroying tens of billions of dollars in value in a few short weeks. Last night he conducted a binding Twitter poll to see if he should step down, and received a resounding vote of no confidence. What can leaders learn from Musk’s short and … Continued
“Secure Element” (SE) chips have traditionally taken a very closed-source, NDA-heavy approach. Thus, it piqued my interest when an early-stage SE chip startup, Cramium (still in stealth mode), approached me to advise on open source strategy. This blog post explains my reasoning for agreeing to advise Cramium, and what I hope to accomplish in the future.
As an open source hardware activist, I have been very pleased at the progress made by the eFabless/Google partnership at creating an open-to-the-transistors physical design kit (PDK) for chips. This would be about as open as you can get from the design standpoint. However, the partnership currently supports only lower-complexity designs in the 90nm to 180nm technology nodes. Meanwhile, Cramium is planning to tape out their security chip in the 22nm node. A 22nm chip would be much more capable and cost-effective than one fabricated in 90nm (for reference, the RP2040 is fabricated in 40nm, while the Raspberry Pi 4’s CPU is fabricated in 28nm), but it would not be open-to-the-transistors.
Cramium indicated that they want to push the boundaries on what one can do with open source, within the four corners of the foundry NDAs. Ideally, a security chip would be fabricated in an open-PDK process, but I still feel it’s important to engage and help nudge them in the right direction because there is a genuine possibility that an open SDK (but still closed PDK) SE in a 22nm process could gain a lot of traction. If it’s not done right, it could establish poor de-facto standards, with lasting impacts on the open source ecosystem.
For example, when Cramium approached me, their original thought was to ship the chip with an ARM Cortex M7 CPU. Their reasoning is that developers prize a high-performance CPU, and the M7 is one of the best offerings in its class from that perspective. Who doesn’t love a processor with lots of MHz and a high IPC?
However, if Cramium’s chip were to gain traction and ship to millions of customers, it could effectively entrench the ARM instruction set — and more importantly — quirks such as the Memory Protection Unit (MPU) as the standard for open source SEs. We’ve seen the power of architectural lock-in as the x86 serially shredded the Alpha, Sparc, Itanium and MIPS architectures; so, I worry that every new market embracing ARM as a de-facto standard is also ground lost to fully open architectures such as RISC-V.
So, after some conversations, I accepted an advisory position at Cramium as the Ecosystem Engineer under the condition that they also include a RISC-V core on the chip. This is in addition to the Cortex M7. The good news is that a RISC-V core is royalty-free, and the silicon area necessary to add it at 22nm is basically a rounding error in cost, so it was a relatively easy sell. If I’m successful at integrating the RISC-V core, it will give software developers a choice between ARM and RISC-V.
So why is Cramium leaving the M7 core in? Quite frankly, it’s for risk mitigation. The project will cost upwards of $20 million to tape out. The ARM M7 core has been taped out and shipped in millions of products, and is supported by a billion-dollar company with deep silicon experience. The VexRiscv core that we’re planning to integrate, on the other hand, comes with no warranty of fitness, and it is not as performant as the Cortex M7. It’s just my word and sweat of brow that will ensure it hopefully works well enough to be usable. Thus, I find it understandable that the people writing the checks want a “plan B” that involves a battle-tested core, even if proprietary.
This will understandably ruffle the feathers of the open source purists who will only certify hardware as “Free” if and only if it contains solely libre components. I also sympathize with their position; however, our choices are either the open source community somehow provides a CPU core with a warranty of fitness, effectively underwriting a $20 million bill if there is a fatal bug in the core, or I walk away from the project for “not being libre enough”, and allow ARM to take the possibly soon-to-be-huge open source SE market without challenge.
In my view it’s better to compromise and have a seat at the table now, than to walk away from negotiations and simply cede green fields to proprietary technologies, hoping to retake lost ground only after the community has achieved consensus around a robust full-stack open source SE solution. So, instead of investing time arguing over politics before any work is done, I’m choosing to invest time building validation test suites. Once I have a solid suite of tests in hand, I’ll have a much stronger position to argue for the removal of any proprietary CPU cores.
On the Limit of Openness in a Proprietary Ecosystem
Advising on the CPU core is just one of many tasks ahead of me as their open source Ecosystem Engineer. Cramium’s background comes from the traditional chip world, where NDAs are the norm and open source is an exotic and potentially fatal novelty. Fatal, because most startups in this space exit through acquisition, and it’s much harder to negotiate a high acquisition price if prized IP is already available free-of-charge. Thus my goal is to not alienate their team with contumelious condescension about the obviousness and goodness of open source that is regrettably the cultural norm of our community. Instead, I am building bridges and reaching across the aisle, trying to understand their concerns, and explaining to them how and why open source can practically benefit a security chip.
To that end, trying to figure out where to draw the line for openness is a challenge. The crux of the situation is that the perceived fear/uncertainty/doubt (FUD) around a particular attack surface tends to have an inverse relation to the actual size of the attack surface. This illustrates the perceived FUD around a given layer of the security hierarchy:
Generally, the amount of FUD around an attack surface grows with how poorly understood the attack surface is: naturally we fear things we don’t understand well; likewise we have less fear of the familiar. Thus, “user error” doesn’t sound particularly scary, but “direct readout” with a focused ion beam of hardware security keys sounds downright leet and scary, the stuff of state actors and APTs, and also of factoids spouted over beers with peers to sound smart.
However, the actual size of the attack surface is quite the opposite:
In practice, “user error” – weak passwords, spearphishing, typosquatting, or straight-up fat fingering a poorly designed UX – is common and often remotely exploitable. Protocol errors – downgrade attacks, failures to check signatures, TOCTOUs – are likewise fairly common and remotely exploitable. Next in the order are just straight-up software bugs – buffer overruns, use after frees, and other logic bugs. Due to the sheer volume of code (and more significantly the rate of code turnover) involved in most security protocols, there are a lot of bugs, and a constant stream of newly minted bugs with each update.
Beneath this are the hardware bugs. These are logical errors in the implementation of a function of a piece of hardware, such as memory aliasing, open test access ports, and oversights such as partially mutable cryptographic material (such as an AES key that can’t be read out, but can be updated one byte at a time). Underneath logical hardware bugs are sidechannels – leakage of secret information through timing, power, and electromagnetic emissions that can occur even if the hardware is logically perfect. And finally, at the bottom layer is direct readout – someone with physical access to a chip directly inspecting its arrangement of atoms to read out secrets. While there is ultimately no defense against the direct readout of nonvolatile secrets short of zeroizing them on tamper detection, it’s an attack surface that is literally measured in microns and it requires unmitigated physical access to hardware – a far cry from the ubiquity of “user error” or even “software bugs”.
The current NDA-heavy status quo for SE chips creates an analytical barrier that prevents everyday users like us from determining how big the actual attack surface is. That analytical barrier actually extends slightly up the stack from hardware, into “software bugs”. This is because without intimate knowledge of how the hardware is supposed to function, there are important classes of software bugs we can’t analyze.
Furthermore, the inability of developers to freely write code and run it directly on SEs forces more functionality up into the protocol layer, creating an even larger attack surface.
My hope is that working with Cramium will improve this situation. In the end, we won’t be able to entirely remove all analytical barriers, but hopefully we arrive at something closer to this:
Due to various NDAs, we won’t be able to release things such as the mask geometries, and there are some blocks less relevant to security such as the ADC and USB PHY that are proprietary. However, the goal is to have the critical sections responsible for the security logic, such as the cryptographic accelerators, the RISC-V CPU core, and other related blocks shared as open source RTL descriptions. This will allow us to have improved, although not perfect, visibility into a significant class of hardware bugs.
The biggest red flag in the overall scenario is that the on-chip interconnect matrix is slated to be a core generated using the ARM NIC-400 IP generator, so this logic will not be available for inspection. The reasoning behind this is, once again, risk mitigation of the tapeout. This is unfortunate, but this also means we just need to be a bit more clever about how we structure the open source blocks so that we have a toolbox to guard against potential misbehavior in the interconnect matrix.
My personal goal is to create a fully OSS-friendly FPGA model of the RISC-V core and their cryptographic accelerators using the LiteX framework, so that researchers and analysts can use this to model the behavior of the SE and create a battery of tests and fuzzers to confirm the correctness of construction of the rest of the chip.
In addition to the work advising Cramium’s engagement with the open source community, I’m also starting to look into non-destructive optical inspection techniques to verify chips in earnest, thanks to a grant I received from NLNet’s NGI0 Entrust fund. More on this later, but it’s my hope that I can find a synergy between the work I’m doing at Cramium and my silicon verification work to help narrow the remaining gaps in the trust model, despite refractory foundry and IP NDAs.
Counterpoint: The Utility of Secrecy in Security
Secrecy has utility in security. After all, every SE vendor runs with this approach, and for example, we trust the security of nuclear stockpiles to hardware that is presumably entirely closed source.
Secrecy makes a lot of sense when:
Even a small delay in discovering a secret can be a matter of life or death
Distribution and access to hardware is already strictly controlled
The secrets would rather be deleted than discovered
Military applications check all these boxes. The additional days, weeks or months delay incurred by an adversary analyzing around some obfuscation can be a critical tactical advantage in a hot war. Furthermore, military hardware has controlled distribution; every mission-critical box can be serialized and tracked. Although systems are designed assuming serial number 1 is delivered to the Kremlin, great efforts are still taken to ensure that is not the case (or that a decoy unit is delivered), since even a small delay or confusion can yield a tactical advantage. And finally, in many cases for military hardware, one would rather have the device self-destruct and wipe all of its secrets, rather than have its secrets extracted. Building in booby traps that wipe secrets can measurably raise the bar for any adversary contemplating a direct-readout attack.
On the other hand, SEs like those found in bank cards and phones are:
Widely distributed – often directly and intentionally to potentially adversarial parties
Protecting data at rest (value of secret is constant or may even grow with time)
Used as a trust root for complicated protocols that typically update over time
Protecting secrets where extraction is preferable to self-destruction. The legal system offers remedies for recourse and recovery of stolen assets; whereas self-destruction of the assets offers no recourse
In this case, the role of the anti-tamper countermeasures and side-channel minimization is to raise the investment necessary to recover data from “trivial” to somewhere around “there’s probably an easier and cheaper way to go about this…right?”. After all, for most complicated cryptosystems, the bigger risk is an algorithmic or protocol flaw that can be exploited without any circumvention of hardware countermeasures. If there is a protocol flaw, employing an SE to protect your data is like using a vault, but leaving the keys dangling on a hook next to the vault.
It is useful to contemplate who bears the greatest risk in the traditional SE model, where chips are typically distributed without any way to update their firmware. While an individual user may lose the contents of their bank account, a chip maker may bear a risk of many tens of millions of dollars in losses from recalls, replacement costs and legal damages if a flaw were traced to their design issue. In this game, the player with the most to lose is the chipmaker, not any individual user protected by the chip. Thus, a chipmaker has little incentive to disclose their design’s details.
A key difference between a traditional SE and Cramium’s is that Cramium’s firmware can be updated (assuming an updateable SKU is released; this was a surprisingly controversial suggestion when I brought it up). This is thanks in part to the extensive use of non-volatile ReRAM to store the firmware. This likewise shifts the calculus on what constitutes a recall event. The open source firmware model also means that the code on the device comes, per letter of the license, without warranty; the end customer is ultimately responsible for building, certifying and deploying their own applications. Thus, for a player like Cramium, the potential benefits of openness outweigh those of secrecy and obfuscation embraced by traditional SE vendors.
Summary
My role is to advise Cramium on how to shift the norms around SEs from NDAs to openness. Cramium is not attempting to forge an open-foundry model – they are producing parts using a relatively advanced (compared to your typical stand-alone SE) 22nm process. This process is protected by the highly restrictive foundry NDAs. However, Cramium plans to release much of their design under an open source license, to achieve the following goals:
Facilitate white-box inspection of cryptosystems implemented using their primitives
Speed up discovery of errors; and perhaps more importantly, improve the rate at which they are patched
Reduce the risk of protocol and algorithmic errors, so that hardware countermeasures could be the actual true path of least resistance
Build trust
Promote wide adoption and accelerate application development
Cramium is neither fully open hardware, nor is it fully closed. My goal is to steer it toward the more open side of the spectrum, but the reality is there are going to be elements that are too difficult to open source in the first generation of the chip.
The Cramium chip complements the eFabless/Google efforts to build open-to-the-transistors chips. Today, one can build chips that are open to the mask level using 90 – 180nm processes. Unfortunately, the level of integration achievable with their current technology isn’t quite sufficient for a single-chip Secure Element. There isn’t enough ROM or RAM available to hold the entire application stack on chip, thus requiring a multi-chip solution and negating the HSM-like benefits of custom silicon. The performance of older processes is also not sufficient for the latest cryptographic systems, such as Post Quantum algorithms or Multiparty Threshold ECDSA with Identifiable Aborts. On the upside, one could understand the design down to the transistor level using this process.
However, it’s important to remember that knowing the mask pattern does not mean you’ve solved the supply chain problem, and can trust the silicon in your hands. There are a lot of steps that silicon goes through to go from foundry to product, and at any of those steps the chip you thought you’re getting could be swapped out with a different one; this is particularly easy given the fact that all of the chips available through eFabless/Google’s process use a standardized package and pinout.
In the context of Cramium, I’m primarily concerned about the correctness of the RTL used to generate the chip, and the software that runs on it. Thus, my focus in guiding Cramium is to open sufficient portions of the design such that anyone can analyze the RTL for errors and weaknesses, and less on mitigating supply-chain level attacks.
That being said, RTL-level transparency can still benefit efforts to close the supply chain gap. A trivial example would be using the RTL to fuzz blocks with garbage in simulation; any differences in measured hardware behavior versus simulated behavior could point to extra or hidden logic pathways added to the design. Extra backdoor circuitry injected into the chip would also add loading to internal nodes, impacting timing closure. Thus, we could also do non-destructive, in-situ experiments such as overclocking functional blocks to the point where they fail; with the help of the RTL we can determine the expected critical path and compare it against the observed failure modes. Strong outliers could indicate tampering with the design. While analysis like this cannot guarantee the absence of foundry-injected backdoors, it constrains the things one could do without being detected. Thus, the availability of design source opens up new avenues for verifying correctness and trustability in a way that would be much more difficult, if not impossible, to do without design source.
Finally, by opening as much of the chip as possible to programmers and developers, I’m hoping that we can get the open source SE chip ecosystem off on the right foot. This way, as more advance nodes shift toward open PDKs, we’ll be ready and waiting to create a full-stack open source solution that adequately addresses all the security needs of our modern technology ecosystem.
I’m snapshotting this so we can remember the weekend when one of the world’s most powerful CEOs peaked (hopefully) at mis-reading the planetary room while watching the World Cup final and alienated thousands upon thousands of users and key influencers on a social network he paid tens of billions for and is now hemorraging value.
This has been a rollercoaster of raving lunacy, and I am very glad I moved to Mastodon early on.
I decided to stay around on Twitter so I have a front row seat to its meltdown, and it has not disappointed so far–the layoffs and arbitrary “leadership” were just the start, and the tech industry has effectively disavowed Twitter as a company worth working for at a speed that makes the traditional dissing of Meta or Oracle look like a walk in the park.
As far as the general media are concerned, the visible damage has come from the “flip-flop” strategy for banning journalists and influencers for disagreeing with Musk. That has turned into an outrageous soap opera that (also hopefully) peaked with the announcement of a (quickly reversed, apparently) anti-competitive policy regarding linking to or promoting other social networks (like Mastodon) just yesterday.
But the fundamental damage has to do with trust. Trust from users, advertisers, and even governments. This past month is a blight Twitter may well never overcome unless Musk sells it again to an entity that can manage it properly in he public eye.
Furthermore, his plans for charging a subscription to a walled garden with a tiered system are a two-edged sword–it will drive away additional users in the long run, and make regulators perk up even more.
I would have loved to have seen the EU sue Elon. And I’m betting the FTC has been quietly taking notes, too.
Grisly entertainment, but such is the way of our times.
I am in a flare up of my mobility and pain issues (arthritis/joint pain) which means I cannot really do much, am using my manual wheelchair in the house, need to rest quite a lot and am not exactly on the top of my thinking game. Am on prednisone and many other drugs (like gabapentin which makes me feel like a zombie) to deal with it.
The real distinction this time to me is that I can look back over many years of this happening and see the patterns very well. My new normal baseline, in pain and mobility, is pretty decent and predictable and I’m stronger and healthier than I have been since 2005 or so.
So a flare up looks like going from 50 to 10mph rather 30 to 10 to 20 to 15 to 25 to 10 — more of a “suddenly flattened, but temporary” than a hideous wobble in a chaotic unpredictable journey. I feel less afraid that I will be down for a year, and fairly confident that in a couple of weeks I’ll be cautiously bouncing back to normal. Fingers crossed that is true!
It is easier to see what is happening and react appropriately when it isn’t just a hideous chaos of up and down in pain level.
I think the differences are mainly that I know how to manage better, I’ve been wiser in respecting my limits (not THAT much wiser but, better than I was) and limiting what I do — like not traveling a lot and having really consistent sleep. Another difference from 10 years ago is that if I’m in bad pain, I :::stop:::! I stop and rest and adjust my lifestyle and my expectations temporarily, rather than taking pain drugs and powering through ie making it worse.
Now, when I did “power through” it was because I felt I had to, or I wanted to – like going to Vienna to speak at a conference with both feet in soft casts, in a manual wheelchair, by myself, in the dead of winter in the freaking snow. I wanted to do things to be a good parent, or for my career, or because I worried it might be my last chance to travel if my health got worse. I did not want to lose my jobs or career and would end up using all my PTO/sick/vacation time when ill and then feeling under a cloud and then going off on leave when things got unbearable. (Now, I feel like I can take time out of work and come back and trust my job will be there.) So, I respect my past choices, but I’m glad I decided to change my priorities to center more on making space for rest and healing.
This is not at all to imply that anything is “all in my mind” or that the power of positive thinking can cure chronic illness. It took a pretty radical shift in some underlying ways of thinking for me to change my approach (with help from a fabulous, smart therapist) to try to handle my activities and condition in a more sustainable way.
In the early days, Acquia was one of the fastest-growing companies in the US. Like most startups, we'd raise money, convert that money into growth, raise money again, etc. In total, we raised nearly $190 million in seven rounds of funding.
At some point, all companies that take this approach have to become self-sustainable. Acquia wasn't any different.
When Acquia did a CEO search in 2017, we had just started that transformation. We hired Mike Sullivan as our CEO to help us grow, while quickly becoming financially independent at the same time.
When Mike told me he decided to leave Acquia at the end of the year, I was sad, but not completely surprised. While there is always more work to do, Mike has accomplished the mission we had set out for him: we continued our growth and became self-sustained. Mike is leaving us in the strongest financial position we have ever been.
Mike will be succeeded by Steve Reny, who has been Acquia's Chief Operating Officer (COO) for 4.5 years. Steve has been guiding all aspects of Acquia's customer success, professional services, global support, security, and operations. And before joining Acquia in 2018, Steve held executive leadership positions at other companies, including as CEO, COO, CFO, head of sales, and head of corporate development.
Everyone at Acquia knows and loves Steve. This CEO transition is natural, planned, and minimally disruptive.
I have a deep appreciation for everything Mike has done for Acquia. And I'm excited for Steve. Not everyone gets to lead one of the most prominent Boston technology companies. As for me, I continue in my role as CTO, and look forward to partnering with Steve.
I believe strongly in Acquia's mission, our purpose, and our opportunity. I have a deep-rooted belief in the critical importance of the web and digital experiences. It's how we communicate, how we stay in touch with loved ones across the world, how we collaborate, how we do business, how we learn, how we bank, and more.
Because of the web's importance to society, we need to help ensure its well-being and long-term health. I think a lot about helping to build a web that I want my children to grow up with. We need to make sure the web is open, accessible, inclusive, safe, energy efficient, pro-privacy, and compliant. Acquia and Drupal both have an important part to play in that.
Tina Robinson @actualtina
Pluralistic: Daily links from Cory Doctorow.
DaveO @uncleweed
AliceAvizandum on Twitter.
Donna Respirator (AliceAvizandum)
on Wednesday, December 21st, 2022 2:54pm
