Rolandt

Last year, I wrote a post linking to all of my weeknotes from the year along with a short synopsis. That’s a bit boring, so instead I’m going to choose a photo from each month that hasn’t previously appeared on this blog and talk about some of the things I did.

In total, I did 213 days of work and took 47 days of holiday this year; I observed some public holidays and not others, for various reasons. I’m not going to talk about work much in what follows. You can see all of my weeknote posts here.

January

I started and ended the year on walks with Aaron Hirtenstein in Northumberland National Park. As one does, I made a couple of lists of things I wanted to do more of in 2022 and things I wanted to do less of. While I’ve been pretty fastidious in avoiding things that sap my energy, I haven’t done such a great job at things that give me more.

My daughter came back from school with Covid and so, inevitably, I caught it (although my wife didn’t until later in the year, and my son still hasn’t had it). I didn’t feel great, but worked through it. WAO got into a good rhythm of co-working on projects, and I started working on the Zappa project with the Bonfire team.

February

Hannah and I went away for a night in Newcastle, which included a wonderful meal and cocktails at Khai Khai. It was so good, I ended up taking the whole family for my birthday later in the year.

I kept telling people I had not ‘Long Covid’ but ‘Medium Covid’ as although I was OK, I couldn’t really do much exercise without my heart rate massively spiking. I did start doing more walking every day, however, in an attempt to build up my capacity for the Hadrian’s Wall walk in April.

Later in the month I started a course on ‘tech ethics’ which, although I didn’t finish due to travel, I thought was really thought-provoking. I also met some interesting people.

March

I did my first proper work trip since the pandemic in March, a multi-city trip in The Netherlands as part of the Dutch National Libraries conference. I spent the weekend beforehand in Amsterdam with Hannah, and Laura Hilliger and her husband came over from Germany at the same time so we had dinner together. It was so good to see people IRL again!

Due to Russia’s unprovoked attack on Ukraine, I donate a lot of my remaining crypto directly to the Ukrainian government, and my kids ended up donating all of the charity money they set aside from their pocket money each month. On our home front, we bought a new bed, and I started taking Feverfew tablets which had a real effect on reducing the frequency and intensity of my migraines.

April

As planned, I took three weeks off work in April. Walking Hadrian’s Wall in 72 hours was a real achievement for me, after only really recovering from Covid in March. I had been supposed to walk it with Aaron, but he came down with it on the Sunday we were supposed to meet. So I just went for it. Took me a full week to recover!

Team Belshaw also went on holiday to Croatia, which was excellent — if a little intense between the Bora wind, watching Hadjuk Split vs Dinamo Zagreb, and experiencing an earthquake on our last night… On our return, I ran my first 10k of the year, and we got hospitality tickets for Newcastle United vs Liverpool at St James Park.

May

A busy month, which included my team (Sunderland) being promoted to the Championship after languishing in League 1 for too long. My dad and I went to the playoff semi-final. I also went down to London to present at the Learning Technologies conference, meeting up with Bryan Mathers and Oliver Quinlan while I was there, and ducking my head into the Outlandish offices. I attended the Thinking Digital conference, probably for the last time.

Back home, I completed a Sociocracy facilitators course with some friends, and tried semi-successfully, to replace my home broadband with a 5G connection. I got into a pretty good routine of three gym sessions, two runs, one swim, and one yoga session per week.

June

A quieter month full of routines and including a celebration of Fathers’ Day. My son earned his first Open Badge, which felt like a bit of A Moment given my 11-year history with the project. I did a bit of solo parenting as Hannah was away for work, and my daughter tried out for new football teams.

July

Hannah and I went away to York to celebrate 20 years since we were engaged on Prince Edward Island, Canada. Team Belshaw travelled to Sheffield to watch England’s Lionesses thrash Sweden in the Women’s Euros semi-final.

Hannah completed a 26.2 mile Mighty Hike for charity, while I started running every day and ran my fastest 10k for a decade. I bought a new laptop, painted my office in the rain, and took the kids to football and basketball tournaments. A busier month than I was expecting!

August

I flew to Denver and then on to Boulder, Colorado right at the end of July for The Badge Summit which happened right at the start of August. I had a wonderful time, sharing an amazing Airbnb with Laura and Anne, as well as Mark, Julie, and Don from Participate. In addition to presenting at the event, I went hiking in the mountains, watched the Lionesses win the Euros, and experienced the Meow Wolf immersive art installation.

Hannah got Covid right before I went, so I unceremoniously shipped off the kids to my parents. She had it quite bad, and was incapacitated for a few days, spending time in bed. I think she was doubly concerned as a family member had Long Covid at that time.

After the excitement of the US, we had a week before going to a wedding and then heading off on holiday in France. I ran an accidental half-marathon and found my way back by speaking broken French to random people in the countryside.

September

September was a weird month. I ended up in hospital twice (once to do with my heart, once to do with my brain) due to overdoing it. A pigeon got absolutely rammed-stuck in our car’s grille. I set up exercise.cafe as an escape from Strava. Our daughter got into Newcastle United’s Emerging Talent Centre. I bought a Steam Deck.

October

I got back into running and had a couple of enjoyable runs in Lille where I was for the ePIC conference. I got the Eurostar there, which was a first. I got vaccinated (my fourth) and baked a cake for Hannah’s birthday.

After toying with the idea of getting an electric vehicle, we ended up realising we needed a second car instead and bought a little VW Up! Team Belshaw went on holiday in an Airbnb near Dundee, and our son played an exhibition basketball game before a pro fixture.

November

As I predicted years ago, everyone started showing up on the Fediverse after Twitter took a dive. I set up a non-Mastodon instance at wao.wtf (was Misskey, now a fork called Calckey). Team Belshaw went out for an excellent Sunday dinner, as we do most months now, up the Northumbrian Coast. We had a lovely walk on the beach afterwards.

Aaron and I led an Away Day for LocalGov Drupal down in London together with Nathan Brown. It was a great success, and as a result the project is now incorporated as a co-op. For the first time ever, the World Cup was a winter one and started in late November, which was awesome. Just don’t mention… well, everything other than men kicking a football around.

December

I’d planned to take three weeks off in April, August, and December this year. It didn’t quite work out like that, but close enough. Our kids don’t return to school until 10th January, so I’m taking my three weeks off in a way that straddles the last two weeks of 2022 and the first week of 2023.

I used some AI tools for various things, including audio transcription. Like many people, I’ve been messing about with AI art tools, but ChatGPT was something quite unexpectedly different.

Laura and I appeared on the OEG Voices podcast. I was in full-on wind-down mode from December 1st and focused my attention on sampling many different types of mince pies. Of course my wife’s and my mother’s are the best!

I went walking in Northumberland National Park with Aaron, in the Peak District with Bryan, and up at Simonside with the family. I celebrated my 42nd birthday. I made a nut roast from scratch. I bought gifts for everyone else and then a Mac Studio for myself as it’s the end of my own company’s financial year.

Tonight, to see in the New Year, we’re heading up into the Northumbrian hills for Allendale Tar Bar’l. We used to go every year when I was a teenager, and Hannah and I have been before we had kids. It’ll be the first time our two have experienced it. I just hope it doesn’t rain too much. Snow is absolutely fine 🙂

In Lists and people on Mastodon I showed how I added a list column to the following tab of the Mastodon browser I’m building. That was a step in the direction of easier and more powerful list management. It enables me to see whether the people I follow are assigned to lists, and to consider who should be on a list (or perhaps on a different list).

Today, as I began to use that new affordance in earnest, I discovered a new challenge. In order to assign someone to a list, or change a list assignment, I clicked the link in the account_url column to open that person’s profile in the Mastodon web app. That was fine for accounts on my home server, mastodon.social. An account URL like Shelley Powers’ https://mastodon.social/@burningbird brings me to Shelley’s profile on my home server where the list manager is available.

But if I’m following someone elsewhere, like Ward Cunningham at https://mastodon.radio/@k9ox, the account URL brings me to Ward’s profile on that server where the list manager isn’t available. In order to assign Ward to a list I had to capture his account URL, paste it into the search box in my home server’s web app, and then click the resulting link: https://mastodon.social/@k9ox@mastodon.radio.

That got old real fast, so I adjusted the following tab to display the latter flavor of URL which I’ll call an instance-qualified URL.

Steampipe provides a few ways to make that adjustment. As a user of the dashboard, you can use Postgres’ regular expression functions to do the transformation in the SQL query that drives the view. But you’d rather not have to. It’s much nicer if the plugin does that for you, so the SQL can just refer to a column called instance_qualified_url.

I chose the latter approach. As the author of a Steampipe plugin you want to make life as easy as possible for users of the plugin. When you’re the author of both the plugin and the dashboard, as I am in this case, you can enjoy a nice virtuous cycle. As the dashboard evolves you discover ways to improve the plugin, which leads to more use of the dashboard, which suggests further opportunities to improve the plugin. I have been greatly enjoying the coevolution of these two components!

Adding a new column to a Steampipe table

To make the change, I extended the structure that defines the columns of the tables mapped from Mastodon Account API. A Steampipe plugin defines columns using a list of structs like this.

...,
{
	Name:        "url",
	Type:        proto.ColumnType_STRING,
	Description: "URL for the account.",
},
...,

That struct says: “When the name of a top-level field in the API response is url, tell Steampipe to make a database column with that name and with the Postgres type text.”

You can also transform values in API responses to synthesize new columns that don’t appear in API responses. Here’s the struct I added for this case.

...,
{	
	Name:        "instance_qualified_account_url",
	Type:        proto.ColumnType_STRING,
	Description: "Account URL prefixed with my instance.",
	Transform:   transform.FromValue().Transform(instanceQualifiedAccountUrl),
},
...

That one says: “Send the API response to the transform function instanceQualifiedAccountUrl, and use its result as the value of the column.

Here’s the function.

func instanceQualifiedAccountUrl(ctx context.Context, input *transform.TransformData) (interface{}, error) {
	url := input.Value.(*mastodon.Status).Account.URL
	qualifiedUrl := qualifiedUrl(ctx, url)
	return qualifiedUrl, nil
}

It delegates the real work to another function.

func qualifiedUrl(ctx context.Context, url string) string {
	plugin.Logger(ctx).Debug("instanceQualifiedUrl", "server", homeServer, "url", url)
	re := regexp.MustCompile(`https://([^/]+)/@(.+)`)
	matches := re.FindStringSubmatch(url)
	if len(matches) == 0 {
		return url
	}
	person := matches[1]
	server := matches[2]
	qualifiedUrl := fmt.Sprintf("%s/@%s@%s", homeServer, server, person)
	plugin.Logger(ctx).Debug("instanceQualifiedUrl", "qualifiedUrl", qualifiedUrl)
	schemelessHomeServer := strings.ReplaceAll(homeServer, "https://", "")
	qualifiedUrl = strings.ReplaceAll(qualifiedUrl, "@"+schemelessHomeServer, "")
	plugin.Logger(ctx).Debug("qualifiedUrl", "qualifiedUrl", qualifiedUrl)
	return qualifiedUrl
}

Why? Two different sets of column definitions need the same transformation. instanceQualifiedAccountUrl works with responses from the Account API. But account URLs also appear in the Status API that drives timeline views. Those use a different transform function, instanceQualifiedStatusUrl, to do the same transformation for a different API response.

From account URLs to status URLs

The instanceQualifiedAccountUrl column solved the original problem. I was able to remove my plugin-author hat, put on my dashboard-author hat, and refer to account urls as instance-qualified URLs in all the tabs that display them. Any such link now leads to a profile that I view through the lens of mastodon.social and that enables me to use the web app’s list manager directly, without the cumbersome copy/paste/search procedure.

My happy dance didn’t last long, though. Newly sensitized to that copy/paste/search friction, I realized it was still happening when I try to reply to items that appear in a timeline view. Here is a recent example: https://techpolicy.social/@mnot/109610641523489182.

That’s the URL displayed in the dashboard. When I click it I land on Mark’s server and can view the item, but if I try to reply I’m confronted with the dreaded copy/paste/search operation.

No problem! I’ll use a similar transform! Not so fast. I can form an URL like https://mastodon.social/@mnot@techpolicy.social/109610641523489182 but it doesn’t go anywhere.

If I do the copy/paste/search operation, I land on a similar-but-different URL: https://mastodon.social/@mnot@techpolicy.social/109610641692667630. It has the same structure but a different toot ID. This URL is also the one that appears in the web app’s home timeline, which is why I can reply directly from that view.

I’m out of my depth here so I’ll just end with an appeal for help. It makes sense that a home server will assign its own id to an item fetched from a foreign server, and that the web app will use that id. But I’m not seeing a way to aquire that id directly from the API. I suspect it’s possible to acquire it by way of search, but doing that for every item in a timeline will quickly exhaust the tight budget for API requests (just 300 every 5 minutes).

So, Lazy Mastodon, am I just stuck here or is there a way to transform foreign status URLs into instance-relative status URLs?

Update: Solved!

After chatting with Jari Pennanen I took another look and realized the needed ID was available in the API response after all, I just wasn’t using it (facepalm). And in fact there are two flavors of the ID — one for original toots, another for boosts. Columns for both cases are added here and the tweak to make the dashboard use them here.

Here is the result.

Thanks for being my rubber duck, Jari! The instance-qualified toot and reblog URLs make this dashboard massively more useful.

—
1 https://blog.jonudell.net/2022/11/28/autonomy-packet-size-friction-fanout-and-velocity/
2 https://blog.jonudell.net/2022/12/06/mastodon-steampipe-and-rss/
3 https://blog.jonudell.net/2022/12/10/browsing-the-fediverse/
4 https://blog.jonudell.net/2022/12/17/a-bloomberg-terminal-for-mastodon/
5 https://blog.jonudell.net/2022/12/19/create-your-own-mastodon-ux/
6 https://blog.jonudell.net/2022/12/22/lists-and-people-on-mastodon/
7 https://blog.jonudell.net/2022/12/29/how-many-people-in-my-mastodon-feed-also-tweeted-today/
8 https://blog.jonudell.net/2022/12/31/instance-qualified-mastodon-urls/
9 https://blog.jonudell.net/2023/01/16/mastodon-relationship-graphs/
10 https://blog.jonudell.net/2023/01/21/working-with-mastodon-lists/
11 https://blog.jonudell.net/2023/01/26/images-considered-harmful-sometimes/
12 https://blog.jonudell.net/2023/02/02/mapping-the-wider-fediverse/
13 https://blog.jonudell.net/2023/02/06/protocols-apis-and-conventions/
14 https://blog.jonudell.net/2023/02/14/news-in-the-fediverse/
15 https://blog.jonudell.net/2023/02/26/mapping-people-and-tags-on-mastodon/
16 https://blog.jonudell.net/2023/03/07/visualizing-mastodon-server-moderation/
17 https://blog.jonudell.net/2023/03/14/mastodon-timelines-for-teams/
18 https://blog.jonudell.net/2023/04/03/the-mastodon-plugin-is-now-available-on-the-steampipe-hub/
19 https://blog.jonudell.net/2023/04/11/migrating-mastodon-lists/
20 https://blog.jonudell.net/2023/05/24/when-the-rubber-duck-talks-back/

Another year has come and gone. I can’t believe it. I think I’ll look back on 2022 as a year of milestones. It’s been a year of new jobs, new challenges, new products, and new hobbies. This is my second year piecing my year together from my monthly vibechecks. I think I’ve boiled those posts down to some major themes from the past year.

My new job at Luro

The biggest change this past year is my employment, I’m working full-time on Luro. What began as a side project app has grown into it’s own little stealth startup. And when I say startup, I mean startup. Over the course of the last year, we took a bit of pre-seed investment, made our first hires, re-architected the app, and onboarded our first round of private beta users.

There have been ups and downs starting a SaaS business in the headwinds of an economic recession, but I’ve never been more sure of Luro’s value and potential. It’s an absolute joy to see people’s eyes light up when they understand Luro and then start describing the product back to us. I know I’ve been a bit cagey or cryptic about what Luro is and what it does but that should all change in 2023.

Thanks to everyone who has contributed directly and indirectly to Luro so far. I appreciate all the thoughts, feedback, and code you’ve all contributed.

Five Hundred ShopTalks

Chris and I went published our 500th episode of ShopTalk in 2022. Another year of wonderful guests and lots of Chris and I talking about websites. Behind the scenes: Chris sold a blog, I started a company, and we trade off each month on who has the back pain. With this year a wrap, it’s a surreal feeling to be looking back at 11 years of podcasts and 546 episodes.

The D-d-d-discord is going strong and there’s a lot of Shopomaniacs from all over the world in there. It’s fun that people are chatting around the clock. It’s the right mix of nuanced #hotdrama. We fell off on the YouTube channel due to general busy-ness, but I’ve got some ideas brewing for next year. If you or your company would like to sponsor some of the YouTubes, reach out.

And that’s a good time to mention, the plan for next year is to self-sponsor the show. That’s right. No ads, except for a CodePen or Luro spot potentially. If a good partner comes along and wants to reach thousands of listeners each week, reach out and we might consider it, but we’re going #NoAds for a bit. Yowza.

My Web Components Course

In April, Frontend Masters invited me out to do a workshop on Web Components. They recorded the workshop and offer it along with all the dozens and dozens of training courses on their site. It’s a little over four hours in length and is a beginner course designed to take you from a zero-to-intermediate experience with Web Components.

Another sub-goal of this project was to have an answer to the oldest of all Shop Talk questions: How can you give me money? I’m joking, but it’s nice to be able to point people to a resource that makes me money if they find it valuable. Thousands of people have taken the course so far and while it’s not “course on React” popular, I’m pleased on the progress so far. The nice thing about a course on Web Components is that as long as the Web Platform has Web Components, the content should be relevant.

A lot of time and effort went into this project. Over a hundred hours at least. I extrapolated a workshop from a talk, built a guidebook, and researched a ton of questions I had along the way. Behind the scenes, Frontend Masters and I talked about the course for over a year, so wrapping that project up was a relief. I enjoyed working with the team there and publishing a course is a career milestone for me.

Watch my course on Frontend Masters

My Gundam Phase

2022 was the year I entered my Gundam phase. It started with making my first plastic “gunpla” models (in Japanese, “gunpla” or “plamo” refer to “Gundam Plastic Model”). It hooked me after the first model. I ended up making over 20 of these little robot buddies in varying in size, quality grade, and price. Most of them are ~$15 and a great way to kill an evening or two, but they can also be hundreds of dollars if you want.

Getting into the hobby led me to trying to watch the entire Gundam franchise starting from the original 80s to current day and mixing in some of the modern day series. The Gundam series timeline is complex and there’s thousands of episodes to catch up on. It’s a unique connection to a show when you’re building a robot from one of the shows you’re watching.

Am I a grown-ass man making little plastic robots for his bookshelf? Yes. Do I know it’s Level 900 dork shit? Also, yes. For me, it’s way more satisfying than something like LEGO.

ADHD, PARA, and “Projects have an end date”

I spent a lot of time this year understanding my brain. I’m pretty convinced I have adult ADHD that I self-medicate with caffeine. I should get myself formally diagnosed but that would require remembering and taking time to make a doctor’s appointment and well… there’s the problem. It’s a vicious cycle.

The best way I can describe my brain is as a boat with a sensitive rudder. Any thought or suggestion (internal or external) nudges the rudder in a different direction. It’s not uncorrectable, but it does divert course. Whatever I can do to lessen the nudges, the better. This all sounds negative but there’s a lot of positives that come with having a brain like this, as others know. You may forget to eat and show up 10 minutes late to an important meeting because reading a wikipedia article on whale songs took precedence, but I wouldn’t trade it for a neurotypical brain.

To manage some of the mental chaos, I utilized a framework called PARA. While I didn’t strictly adhere to PARA throughout the year, embracing it made me take inventory of all the Projects and Areas in my brain. The most challenging idea in PARA was that “Projects have an end date” because most of my projects are never-ending. This made me look long and hard at projects and activities I’m involved in and forced me to reevaluate some commitments. As work/life-stress compiled, I stepped out of some projects; a DnD group, and open source group, and a podcast to name.

I miss those activities, but I think it’s for the better. I’m buying back mental bandwidth. I can tell my brain is closer to its natural state when it thinks about making a dozen different video games all the time.

Checking in on my 2022 KPIs

On to the full transparency report, checking in on last years goals I set for myself.

You can see I somewhat succeeded on the metric based objectives. For the failures:

• I fell off of pickleball due to a sociopath at my club. I had one good month of exercising consistently and while I can celebrate that, my goal wasn’t to exercise, it’s to make my health part of my job.
• I still haven’t figured out what relaxes me 😣 but I do think I’ve identified some of my stressors as well as physiological and psychological responses. I guess that’s something.

Gunpla quasi-fits into the relaxation category and I certainly leaned into it. It’s a methodical, linear process but I think I’d label it as a “constructive occupation” of my time rather than a way to relax. I could harness more of the “Zen and the Art of Gunpla Maintenance” aspects of the hobby I’m sure. Perhaps this was the one tool I found this year that offers some calm.

Books should fit into the relaxation category as well, but I use audiobooks as a reward mechanism for doing tasks I don’t want to do (e.g. wash the dishes, etc). So. Not always relaxing. Feeding information addiction isn’t super relaxing either. It’s chasing dopamine for my dopamine-starved brain. And since I’ve had kids, it’s hard to find a quiet place to read in the house.

Looking towards 2023

Next year. Well. Luro. That’s the big one. We’ve got some nice new features on deck that I think you’ll love. I may even get to blog for my job. But after a big monorepo refactor we’re setup to handle more users. I’m looking forward to cutting some more feature as well as responding to user feedback.

I’m excited for the Spring and baseball to start back up. It makes for a busy time, but it’s nice to be outside watching kids be kids. I expect they’ll wrangle me into the scorekeeper job again but that’s okay. Inject that Americana into my veins.

I think a good mantra for next year might be “2023: Watch more TV.” I have over-indexed on books and weird anime that I’d like to catch up on some of the cultural zeitgeist.

I have some SMART goals (loosely held), broken down by quarter. I’ll track progress here:

Let’s try this. But my true goal in life is to be so bored I read a magazine. Nothing else to do. No open tasks. Me, so bored with nothing to do, that I pick up a magazine and read some of the articles. The goal is to experience boredom.

In lieu of my regular weeknotes (I took two weeks off for the holidays) here's a look back at 2022, mainly in terms of projects and things I've written about.

January

• What's new in sqlite-utils 3.20 and 3.21: --lines, --text, --convert - some neat new features for sqlite-utils that expanded its capabilities in terms of log processing.
• Datasette 0.60: The annotated release notes - improvements to Datasette's internal APIs for writing to the database, plus a new plugin hook for custom table filters.

February

• Help scraping: track changes to CLI tools by recording their --help using Git - a new twist on git scraping, which I used to try and spot when AWS released the Lambda Function URLs I had been waiting for.
• Google Drive to SQLite - a project for creating a SQLite database of metadata for files in your Google Drive.
• Tutorial: Exploring a database with Datasette - the first official Datasette tutorial.
• Tutorial: Learn SQL with Datasette - tutorial number two.

March

• shot-scraper: automated screenshots for documentation, built on Playwright. This was a really fun new project which I expanded in all sorts of directions as the year went on.
• Instantly create a GitHub repository to take screenshots of a web page - using shot-scraper and GitHub Action template repositories.

April

• Pillar Point Stewards, pypi-to-sqlite, improvements to shot-scraper and appreciating datasette-dashboards - Pillar Point Stewards was a project I built with Natalie Downe to manage the stewardship program she was running for our local reef.
• Building a Covid sewage Twitter bot (and other weeknotes) describes how I built @covidsewage to publish the latest Covid sewage numbers for the San Francisco Bay Area.
• Automatically opening issues when tracked file content changes was even more GitHub Actions + Git scraping hackery, this time to alert me when a file I was tracking in another GitHub repository changed by opening a new issue.

May

• Datasette Lite: a server-side Python web application running in a browser. This may have been my most impactful project of the year: I got Datasette (a Python server-side web application) working entirely in the browser, using WebAssembly and Pyodide!
• A Datasette tutorial written by GPT-3 was the first of my public explorations of GPT-3. I spent much of the rest of the year exploring its capabilities further.

June

• How to use the GPT-3 language model - I realized that most people I knew hadn't even tried GPT-3 yet, so I wrote a tutorial to help them get started.
• Weeknotes: Datasette Cloud ready to preview - I continued to make progress on Datasette Cloud throughout the year, but it's still not quite at a place where I'm happy to throw it open to a wider audience.
• A tiny web app to create images from OpenStreetMap maps - another shot-scraper inspired project, which gives me the ability to easily create map images with custom markers.
• Joining CSV files in your browser using Datasette Lite - a massive upgrade to Datasette Lite in terms of its usability for real-world problems.
• First impressions of DALL-E, generating images from text - I got access to the DALL-E preview.
• s3-ocr: Extract text from PDF files stored in an S3 bucket - I originally built this for the San Francisco Microscopical Society, but I'm confident it can be useful for all sorts of other archival projects.

July

• Using GPT-3 to explain how code works - more fun with GPT-3.
• sqlite-comprehend: run AWS entity extraction against content in a SQLite database was another Microscopical Society project.
• Tutorial: Cleaning data with sqlite-utils and Datasette - adapted from my talk at HYTRADBOI.

August

• Plugin support for Datasette Lite is really fun: you can now add ?plugin=datasette-mp3-audio to the URL to install plugins.
• Analyzing ScotRail audio announcements with Datasette---from prototype to production which used that new feature in Datasette Lite!
• Building a searchable archive for the San Francisco Microscopical Society was the inspiration for several other projects I built this year.
• Stable Diffusion is a really big deal. I was definitely right about this! Generative AI got very, very big over the rest of the year.

September

• Exploring the training data behind Stable Diffusion was the collaboration I did with Andy Budd to explore the LAION dataset used to create Stable Diffusion. This has played a meaningful role in the ongoing conversation about the ethics of generative AI.
• Prompt injection attacks against GPT-3 ended up as a series of posts about a security vulnerability in many applications built on top of GPT-3 that has so far proven stubborn in the face of potential fixes.
• TIL: Deploying Python web apps as AWS Lambda functions - I'd been wanting to figure this out for years!
• Exploring 10m scraped Shutterstock videos used to train Meta's Make-A-Video text-to-video model
• A tool to run caption extraction against online videos using Whisper and GitHub Issues/Actions describes my winning entry in the Bellingcat Hackathon, taking advantage of OpenAI's Whisper speech to text model.

October

• Software engineering practices described a set of less obvious software engineering practices I think are well worth adopting in 2022.
• Measuring traffic during the Half Moon Bay Pumpkin Festival was another fun Git scraping hack.
• Weeknotes: DjangoCon, SQLite in Django, datasette-gunicorn talked about some of my work relating to DjangoCon 2022.
• The Perfect Commit took one of the key ideas from my DjangoCon talk and expanded it into a standalone essay.

November

• Designing a write API for Datasette set the scene for my work towards Datasette 1.0.
• Tracking Mastodon user numbers over time with a bucket of tricks was more Git scraping plus a new way of taking advantage of SQLite and WebAssembly.
• Coping strategies for the serial project hoarder is the detailed write-up of my talk from DjangoCon 2022.

December

• Datasette's new JSON write API: The first alpha of Datasette 1.0 - the first alpha! It's finally starting to happen.
• AI assisted learning: Learning Rust with ChatGPT, Copilot and Advent of Code describes my attempts to learn Rust using the just-released ChatGPT, plus Copilot. I got to day 15 before the time commitment for the increasingly difficult puzzles became too much.
• Over-engineering Secret Santa with Python cryptography and Datasette was a fun quick project which gave me a chance to explore Python's cryptography library.
• Datasette 1.0a2: Upserts and finely grained permissions - the last alpha release of the year.

Yesterday we swapped one Swiss lake for another, visiting dear friends on Lake Zug for NYE. The weather is fantastic although very unseasonal. Which gave us an amazing final sunset for 2022 and clear views entering 2023.

The final sunset of 2022

2023 first morning view

I recently wrote about completing Advent of Code 2022 using a different programming language (or two) every day for 25 days.

That note focused on the strategy, tactics and logistics involved in using 26 languages in 25 days without saying much about the languages or the experience itself.

Using so many languages in such a short span provided insight into tradeoffs in language design.

Here are my two high-level reflections from the experience:

• Good program design in the first part of each puzzle – especially more functional techniques and abstractions – tended to make the second part easier. So, in general, functional languages seemed to have the advantage in the puzzles.

• Using better algorithms and data structures was more important than having a “faster” programming language. There was never a time where rewriting in another language felt like the right way to get better performance.

Read on for more specific reflections on language design.

Click here to read the rest of the article

In the early 2000s, the University of Toronto asked me to teach an undergraduate course on software architecture. After three runs I told the university to cancel it because of a lack of material: I’d bought a dozen textbooks on the subject, but between them, they devoted a total of less than 30 pages to describing the designs of actual systems.

Frustrated by that, Andy Oram and I persuaded some well-known programmers to contribute chapters to a book called Beautiful Code. Entries described everything from figuring out whether three points are on a line to the ground station software for the Mars Rover, but the breadth that made them fun to read also meant they weren’t particularly useful for teaching.

To fix that, Amy Brown and I (and later Tavish Armstrong and Mike DiBernardo) edited a four-book series called The Architecture of Open Source Applications. In the first two volumes, the creators of fifty open source projects described their systems’ designs; the third book explored the performance of those systems, while contributors to the fourth built scale models of common tools to demonstrate how real ones worked.

These books were closer to what an instructor would need for an undergrad class on software design, but still not quite right. Students wouldn’t be familiar with most of the problem domains, and since each author used the programming language of their choice, most students wouldn’t be able to read most examples. “They’ll get the sense of it” misses the point: discussion and critique of design often hinges on small details, so anything that distracts readers from seeing those details inhibits learning.

So here we are in 2023, and I think I’ve finally created what I wanted twenty years ago. Software Design by Example: A Tool-Based Introduction with JavaScript builds tiny versions of Git, Mocha, the JavaScript VM, and other tools in order to show how they work, but more importantly, to show how experienced programmers think about software design. All the material is available under open licenses, and all royalties from book sales go to support the Red Door Family Shelter in Toronto.

I had three learner personas in mind as I wrote:

• Aïsha started writing VB macros for Excel in an accounting course and never looked back. After spending three years doing front-end JavaScript work she now wants to learn how to build back-end applications. SDXJS will fill in some gaps in her programming knowledge and teach her some common design patterns.
• Rupinder is studying computer science at college. He has learned a lot about the theory of algorithms, and while he uses Git and unit testing tools in his assignments, he doesn’t feel he understands how they work. SDXJS will give him a better understanding of those tools and of how to design new ones.
• Yim builds mobile apps for a living but also teaches two college courses: one on full-stack web development using JavaScript and Node and another titled “Software Design”. They are happy with the former, but frustrated that so many books about the latter subject talk about it in the abstract and use examples that their students can’t relate to. SDXJS will fill those gaps and give them starting points for a wide variety of course assignments.

I hope to blog about one chapter each week day in January. Feedback is always welcome; while I’ve had to disable comments on this blog (see the explanation in the footer of my site’s home page), you can file issues in in the book’s GitHub repository. I hope you find the material useful, and I’m happy to answer questions by email.

I am grateful to the creators of diagrams.net, Emacs, ESLint, Glosario, GNU Make, LaTeX, Node, NPM, Standard JS, SVG Screenshot, WAVE, and all the other open source tools used in creating these lessons: if we all give a little, we all get a lot. I would also like to thank Darren McElligott, Evan Schultz, and Juanan Pereira for early and ongoing feedback; any errors, omissions, or misunderstandings that remain are entirely my fault.

Notes:

1. Each post in this series contains a list of the terms defined in the corresponding chapter. In my experience, a list like this is the quickest way to get a reliable overview of what a lesson is about. You can find the corresponding definitions in the book’s glossary.

2. I’m translating this book into Python, and adding a few more examples along the way. If you’d like to give early feedback on that material, please reach out.

Twitter is creaking. Social media seems less fun than ever. Maybe it’s time to get a little more personal.

Source: Bring back personal blogging – The Verge

My first exposure to blogging was during a project I joined (more like, weaseled my way into) that was looking to put wee blogs on phones that folks could search for and engage with (it was an interesting idea and could still be something cool). Around that same time, I was slated to join a team working on a digital multimedia diary at the start of 2004. Turns out, just before I joined the team, at an ex-officio meeting I attended in late 2003, they chose to name the product Lifeblog, forcing all of us to jump into blogging (and retconning blogging features into the product, haha).

From 1.0 to 2.0
I’d already been online for many years. I had used various sorts of bulletin boards and forums to post stuff and engage with folks. I had jointly run a proto-blog for a company (mostly news, analysis, and commentary in reverse chronological order – but no comments or permalink or feeds). And I had a few pages for family updates on Geocities (Athens 1066 was the main one). This was during the roaring 1999-2001.

Yet, blogging was different. Blogging took various nice elements from being online and gave online writing key features to help build engagement and ease of publishing. Not to mention, there were some hosted services, such as TypePad (where this blog got started at cognections.typepad.com) and Blogger (purchased by Google in 2003), hosted service that became THE thing in 2004 (blogging was a regular cover story in 2005).

Back in the day, a Tuesday, to be precise
Because our Lifeblog product was about blogging, we had to dive into the world of bloggers and blogging. As far as I know, our team was the first at Nokia to talk about our products and to engage with bloggers to earn goodwill for a product launch. Over 2004 and 2005 I did a lot of traveling and speaking and posting about blogging and mobiles.

Fast forward to end of 2007, Nokia corporate comms brought me on to build and run the Nokia corporate blog, which we called Nokia Conversations. This was not the first Nokia corporate blog. The first one was the S60 blog, set up by Phil Schwarzmann, who followed me after I had left the S60 team. I was a sorta godparent to the blogs he set up and ran. Of course, I turned to Phil to replace me when I left Nokia and the Nokia Conversations team.

Why didn’t the Nokia Lifeblog team have a blog? Well, individually, some of us did, and we talked about what we did, our products, and such. But blogging was so new. As I recall, someone told my boss ‘We don’t make celebrities at Nokia.’ Really, none of us wanted to be a celebrity blogger riding on the Nokia brand. We just wanted to promote our products. Tho that led to the other rub – the whole blogging about our product seemed so contrary to how Nokia had been marketing all their products.

So in the end, the team that finally brought blogging to Nokia was sorta not allowed to actually have one.

By the time Phil set up the S60 blogs, things were more accepted, and marketing teams were more experimental. By the time I got back into the game, Nokia Conversations was able to go big, be experimental, drive huge changes in Nokia comms, and do it while having fun (thanks to a forgiving and creative leadership).

Still here
This blog started on cognections.typepad.com, by my records, in mid-January 2004 (on a Tuesday, actually). I moved the domain to Molecularist.com, I think around 2008/09, just before my move to the US.

I was a heavy blogger in the day. And I’ve had the good fortune to have various jobs where blogging was part of the role. No more so than Nokia Conversations, of course, but also at Children’s Boston (more videos and Facebook than traditional text blogging), IBM, Owl. Indeed, posting something online is always my go to move at any org I’m at.

But, as I saw way back in 2005, social media morselized the web – fragmenting where people ended up, spreading convos across (off the top of my head) Facebook, YouTube, Instagram, SnapChat, and of course Twitter, not to motion so many other places people post stuff, such as Yelp, Amazon, Wattpad, and all the ones that have died in the past 15 years (looking at you Flickr).

And folks follow the conversations. So, for me, this blog became less of an outlet, especially as feed readers and the like died down, I lost contact with so many I knew back in 2005-2009, and everyone moved to mostly Twitter. I usually end up having spurts here when there are no other places for me to write things down, or when I have a brainwave and just need to write it down.

You know the line ‘dance like no one is watching.’ Blogging to me these days is just like that. The things in 2005-2009 that made blogs hum with activity are no longer around or widely used. So I don’t give a damn and ‘blog like there’s no one watching.’

Social media is dead, long live social media
What struck me from reading the article I link to above, is that these past few weeks I’ve been reviewing my online presence, like I do every end of year. I had soured on Twitter so long ago, but in the past few years, I heavily curated my Twitter feed, focusing on makers and the like. Tho in the past few months, I basically shut off the spigot on all my online and offline feeds, narrowed down to two websites and a print magazine. I occasionally hit Discord for two communities, mostly to troubleshoot electronics things I’m working on. And I use LinkedIn for work.

But, for sure, this blog, the one that predated all of the others, is still limping along.

I know what it takes to grow a blog, and not sure that’s the path I want to take. Growing a blog is hard work, like any channel. I’ll leave that to someone who wants to pay me. Haha.

I’ll be content with posting something once in a while for now, mostly for me, not working hard on changing things up.

And if you happen to be one of the rare person who is actually reading this, let me know. Perhaps that will encourage me to write more.

What do you think of the Verge article? Do you blog? When did you start? Why do you blog?

Image by Nile from Pixabay

Gentle reader, you have been directed to this page because you recently said, without sarcasm:

“Everyone was on Twitter.”

This phrase is utter nonsense, as a quick bit of math will show. The population of the world is 8 billion people, a threshhold it just recently crossed. The number of people on Twitter is up for debate, but we’ll be generous and presume 250 million people, or 0.25 billion people. So, in billions of people, the ratio of people on Twitter was:

^.25⁄₈ = ¹⁄₃₂

Only one in 32 people were on Twitter, or roughly 3% of the world’s population, which is clearly not “everyone”. Given the obvious incorrectness of the statement leads to some questions:

What exactly did you mean by “everyone”?

More importantly:

Why is it so important to get other people to believe this prima facie falsehood?

“Bring back personal blogging” argues Monque Judge on The Verve, a plea to revert from the current Web 2.0 to some some Web 1.0 goodness. This kind of article tends to appear every couple of years, but there is a point I find very convincing: Blogging (or whatever you’d like to call it) ensures that we stay in control of our own data:

The biggest reason personal blogs need to make a comeback is a simple one: we should all be in control of our own platforms. If what is happening on Twitter hasn’t demonstrated it, our relationship with these social media platforms is tenuous at best. The thing we are using to build our popularity today could very well be destroyed and disappear from the internet tomorrow, and then what?

It sometimes feels like archiving is a lost art—we all feed social networks and online platforms with unprecedented amounts of data, hardly accounting for the fact everything might vanish when the ownership of a network changes (as seems likely with Twitter’s ongoing nosedive) or the business model collapses.¹

Many—if not most—of the online platforms and networks I’ve ever used on the internet have stopped existing or withered away, something that is strangely commonplace on the web. And yet we hardly stop to think that everything we share might simply get lost over time.

Invoking an “archive” for the ephemera we share online may feel like overkill: Yet even the quick, off-the-cuff ideas and comments, the visual impressions, jokes, memes all add up to something important. Together they draw a picture of our individual perceptions and preoccupations, of what we deemed important at this turbulent time.

Whenever I read history books, it surprised me to learn how many people used to keep diaries and wrote extensive letters that documented what they were thinking at a given time. Today, I can’t think of a single person who does that—countless thoughts and observations are living on social media network’s servers instead, and all could be gone one day.

Judge ends with a plea for the new year that I can absolutely get behind:

(…) Carve your space out on the web. Tell your stories, build your community, and talk to your people. It doesn’t have to be big. It doesn’t have to be fancy. You don’t have to reinvent the wheel. It doesn’t need to duplicate any space that already exists on the web — in fact, it shouldn’t. This is your creation. It’s your expression. It should reflect you.

1. Kashmir Hill makes a similar point in her article “Your Memories. Their Cloud. She talks about the fragility of digital files, and warns us to keep our own archive rather than relying exclusively on some internet company’s cloud. Today’s monoliths are tomorrow’s failed tech companies.↩︎

Stories of New York City in Covid Time

by Eleanor Lerman

I got an advance reader copy in my mailbox. A collection of nine short stories of people who live in New York – or who are travelling there – in the midst of the coronavirus pandemic.

So this is a thin book, just under 160 pages. none of the stories actually feels complete. These are people, mostly single, all out of their regular occupations, but hanging on in a city that many have abandoned. Just as you are getting to know them the story ends and a new one begins. There are some common themes: women with long black hair and a taste for goth makeup. People suffering from severe back pain at a time when normal health services are no longer available. The author of the stories has black hair. Quite possibly she has a back ache too. She knows New York. People are attracted to the Village and Washington Square Park. But often find themselves in the less desirable outer limits of the subway service – but they are still in the City.

The epidemic is not over now. Not yet. But the mood has changed from when this book was written. People have stopped wearing masks – mostly. Travel has restarted but gets disrupted. Restaurants have reopened and people are using transit again, but in lower numbers. Management would like everyone to be back in the office but has to reluctantly accept that remote work is what a lot want to continue. Especially in places where the cost of living is high and rising. In the stories the idea that prices have dropped for desirable places pops up now and again but that is not what is happening now. These stories are of a rare time and a unique space. There is something special about New York City. And that magic – dead at the time of these stories – seems to be reviving now.

The pandemic is now far worse in China, which is where it started, and where lockdowns are still being enforced. Other places were not actually in formal lockdown, thought it might have felt like that. Cruise ships are sailing again. The planes are no longer just flying to reserve their spots at the terminals. But the chaos of lost baggage, delays and confusion are more to do with the impacts of climate – no longer “change” but “crisis”. Huge backlogs of cancellations and missed connections. A whole different set of stories, rather than the folks who managed to hang on in the City even if they no longer had their former well paid jobs, in the stories scraping by wondering what happens next while we readers are in what happens next, which is nowhere like “business as usual” no matter how much business wishes it was.

In terms of overloaded emergency rooms, and rising death rates, plus increasing numbers of people who have had multiple infections or who suffer from “long Covid” the pandemic is nearly as bad as it was at the earlier peaks, but now a high percentage have had multiple vaccinations which work – at least for a few months – but deteriorate rapidly afterwards. Public Health officials are still on the defensive. Simple ideas like hand washing and being kind don’t seem to have a lot of impact on an airborne virus that has the ability to produce a continuous series of variations, each being nastier and more virulent than the one before. We would like to think that we can learn to live with it, just as we have with the flu, the common cold and HIV – but that does seem to be an illusion. Nevertheless, there are indoor parties, the theatres and concert halls are open and the tourism industry seems to be back with bang. There is not a shred of this new reality in these short fictions, where time seems to have stopped. We do not mask very much. There are still many open schools that have no modern ventilation or even box fans surrounded by HEPA filters. Kids are getting sick – and not just with covid but all the other childhood diseases which have resurfaced thanks to a combination of political opportunism and vaccine “hesitancy”. Plus, of course, plenty of deliberate misinformation.

I am not sure that this reviewer can actually recommend this book. Some of the stories have already appeared in magazines and would have been timely then. Now? I am not so sure. Actually I wonder if there needs to be the sequels to some of these stories, so we know how these stories work out. If they did. Certainly good writing.

The following is extracted from the press materials that came with the book. I had not read this before I wrote the review above.

“For award-winning author and poet Eleanor Lerman, New York
City remains the most vibrant and important urban center in the
world. The idea that it would never recover from the pandemic was
an affront not only to New York but to cities everywhere struggling
to deal with the effects of coronavirus.
A lifelong New Yorker, Lerman was disturbed by pontifications that
the city was “dead,” that everyone was leaving, that it would never
regain its place of prominence in American life or be able to offer
the remarkable range of experiences that only a city with a diverse
population and a storied history of welcoming immigrants, artists,
workers, and dreamers, both gay and straight, could provide.
As writers do, she turned her feelings into inspiration.

The Game Café: Stories of New York

City in Covid Time by Eleanor Lerman

Mayapple Press

Paperback; December 2022

ISBN: 978-1-952781-13-1

$22.95; 6 x 9; 160 pages

Another year gone by, another 8000 km or so. A record distance over the year, just a bit beyond what I did the past two years, despite the fact that I did fewer long rides. Cyclemeter says I’m a bit over 8500 km, whereas veloviewer gives a slightly smaller total.

January:

I took advantage of a cold snap to ride the clear ice on Grenadier Pond in the course of testing out some pants. I could hear the ice sing.

Local bike advocate Janet Joy Wilson took a new job in the Big Apple, so she invited a few of us along for a group ride to mark the occasion.

Late January was unusually cold, and so I ventured out to Toronto Island to ride on the ice in the canals, with a bit of crunchy snow on top.

February:

Lots of local protests in support of the freedom convoy in Ottawa disrupted traffic in the downtown area on weekends. Didn’t affect biking so much.

March:

TCBC organized a ride to show support for extending the Bloor bike lanes all the way into Mississauga. The group was small because the ride had been postponed due to weather at the last minute, but some cyclists showed up anyway. So we went ahead and rode out from Runnymede and were met with a group coming the other direction at the bridge over Etobicoke Creek.

The official ride on March 20 happened with a much larger group including Midori and I on the tandem.

April:

I made a quick trip out to Portland for a wedding, and also checked out two cargo bike shops that I missed during my last visit, one of which was Splendid Cycles.

Cycle Toronto organized a ride to celebrate the success of the bike lanes on Shaw St. We were joined by long time supporter Councillor Mike Layton.

May:

Bike for Mike 2002 had rainy weather, but nevertheless I had a good time, and it was for a good cause.

May the fourth was the perfect day for a Star Wars themed ride.

I explored a bit of the Uxbridge to Lindsay rail trail. Didn’t make it as far as Neverland.

The Ride of Silence was back in person for the first time in three years but I was not able to attend.

The first ghost bike ride of the year was for Joshua Okoeguale, a 16 year old who was killed in Oshawa.

The annual bike month group commute was back this year.

HPVDT had a chance to test their tandem bike at a wind tunnel at Western University.

June:

The annual fund raising bike ride on the Gardiner and DVP was rebranded the Ride for Brain Health. I was doing ride support with TBN as per usual, but I also met up with colleagues from my department at the beginning.

A quick trip to Hamilton to see a promotion of the Keddy Access Trail.

July:

I had a streak of continuous days of bike riding that stretched back to Boxing Day 2020, but somehow I forgot to ride on July 1, so my streak ended at 517 days.

A quick trip to Woodstock NY to go to a concert by Nexus percussion. Got in some riding by the Ashokan Reservoir. Got to see the stage where 4′ 33″ was premiered.

Got a Switch e-bike conversion. Initial impressions were positive.

A ghost bike ride in Hamilton for Brian Woods, who was killed riding to his work at Limeridge Mall.

Doing a little exploring by bike of an unimproved section of the Etobicoke Creek Trail.

Another Burlington to Niagara ride with TBN.

Mike Layton decides not to run for re-election. The cycling community in Toronto has lost one of its strongest advocates.

August:

A number of years ago, I was on an organized ride from Seattle to Vancouver, but due to a flat tire and other issues, I ended up completing the ride but leaving a gap of about 100 km. I went back this year to fill in that gap. It was punishingly hot, but there was ice cream at the end.

Some nice gravel riding on the left coast.

A ride to promote safety on Parkside Drive, and to protest police ticketing of cyclists in High Park.

A TBN ride to Lake Simcoe.

September:

Testing our tandem speedbike at Downsview.

The World Human Powered Speed Challenge was back this year after two years of cancellations. Unfortunately our tandem crashed and we did not set any records.

Cycle Toronto organized fund raising rides in different areas of the city. I rode with the Scarborough group.

October:

A night time march down Yonge St to promote road safety.

A gravel ride between Belwood and Luther Marsh.

A ride with TBN during peak fall colours.

Third ghost bike ride of the year, this time near Streetsville.

Pre Halloween ride with the Neon Riders.

Hallowe’en Bike Parade.

November:

Checking out another section of the G2G trail.

The annual ride to remember Road Traffic Victims. It was cold and windy.

A TBN ride from Hamilton to home.

Dammit, we couldn’t get through one year without a ghost bike installed in Toronto. RIP Kartik Saini.

December:

Another ride down Yonge St with Santa.

A pair of pogies arrived from a small company in Ukraine, naturally in the colours of Ukrainian Flag.

I’ll also note in passing that an updated map of all ghost bike locations in the GTA has been posted. Thanks to Ingrid Buday for her work on this.

For some year end coverage of some of the upgrades to bike infra and associated public consultations for future projects, visit Rob Z’s blog.

Also see this year end summary from David Shellnut, the Biking Lawyer.

Wishing you all a safe year for 2023, with plenty of tailwinds!

I usually explain that a taxonomy is a structured kind of controlled vocabulary, which is list of terms (or concepts) usually used to tag content to aid in its retrieval. The structure can be hierarchical, faceted, or a combination. Other people have defined taxonomies for a general audience in more simplistic ways as a kind of hierarchical classification system. So, while a taxonomy has two main features (naming and structure), my preferred definition has focused on the controlled vocabulary and naming aspect, whereas other definitions focus on the hierarchical classification aspect of taxonomies. However, a taxonomy and a classification system are not necessarily the same. While it is understandable that a definition is simplified for a general audience, it should not be simplified to the extent of being misleading.

I have blogged previously on the differences between taxonomies and classification systems, so I won’t repeat all the differences again.  The main point is that a classification system is generic and rigid and is intended to be used widely, such as the Dewey Decimal Classification for libraries, whereas a taxonomy tends to be customized for a particular use case and context and is flexible and undergoes changes.

How does a taxonomy of concepts for information retrieval relate to a scientific taxonomy? They are similar, and the differences are not so great that there should be considered different meanings of the word “taxonomy.” If we consider that taxonomies are systems to name and organize things hierarchically, then a taxonomy for information retrieval, comprised of terms for tagging and retrieving content (documents, images, etc.), can be considered a taxonomy of a controlled vocabulary, in contrast to taxonomies of things, such as organisms. This is a slightly different perspective than to consider a taxonomy as a kind of controlled vocabulary, as I previously had. The following diagram illustrates a possible way to consider how information-retrieval taxonomies related to classification systems and controlled vocabularies.

Wikipedia provides the definition at the entry for Taxonomy: “A taxonomy (or taxonomical classification) is a scheme of classification, especially a hierarchical classification, in which things are organized into groups or types.” But then it goes on to say, “it may refer to a categorisation of things or concepts.” Thus, an information-retrieval taxonomy is a categorization of concepts (also called terms in a controlled vocabulary). It is not a classification system, since the goal is not to classify things, not even the things tagged with the taxonomy concepts, but rather to organize the set of concepts that have been identified as appropriate for tagging and retrieving a set of content.

I’ve been rather quiet the past week as I’ve been enjoying — and enduring — the holidays with my family. But the end of the year is approaching, and I thought I should do some sort of year-end wrap up. Why not, I thought, talk about some of the fun and quirky videogames I’ve played over the past year? Lots of unusual games have been coming out with unconventional art, game mechanics, and themes, a nice complement to the impressive but familiar AAA games that we see every year. So here’s a rundown on some of the games that caught my attention in 2022. Images are taken from the Steam pages of the games.

Immortality. This has been one of the highest-rated games of the entire year, and with good reason. It was developed by Sam Barlow, building on the design concepts of his earlier interactive film video game, Her Story (2015).

The starting premise is simple enough. An actress, Marissa Marcel, starred in three movies, made in 1968, 1970, and 1999, none of which were ever released, and Marcel’s fate is also unknown. Your task is to figure out what happened to Marcel and why her movies were never released. You have what amounts to a film spooler, and starting with just one movie clip, which you can move through forward and backward at will, you click on interesting objects in a scene, which unlocks new scenes that have some sort of symbolic connection. In this way, you can slowly piece together the tangled story.

At some point very soon in beginning the game, however, something very unexpected happens, and I was totally blown away when it did. Suddenly you find that there is a lot more to the story than you could possibly imagine, and that you, the viewer, may also be becoming part of the story.

The film clips are incredibly well-acted and filmed, and include scenes from the 3 fictional movies themselves as well as behind the scenes shots, screen tests, and more. As the game unfolds, you find that you are unraveling multiple stories layered on top of each other: the plots of the 3 movies, the story of Marissa Marcel, and a deeper, more sinister story that underlies it all.

The two leads Manon Gage and Charlotta Mohlin give excellent performances. The different scenes discovered range from unsettling, or even frightening, to deeply and profoundly moving.

Immortality has an “end game,” but it is likely that you won’t have exposed the full story by the time you reach it. Fortunately, you can keep searching, looking for that hidden clip that will make everything come together. I played 11 hours total.

Choo-Choo Charles. This game went viral before release when its single developer released video clips of the gameplay, a first-person shooter where the player uses weapons on a train to fight Choo-Choo Charles, a terrifying spider-legged monster train.

This game is quite short — only 3 hours, at most — and has rather limited game mechanics, but it is a surprisingly good time! Most of the game involves traveling around the island by train, doing chores for islanders in exchange for weapons and train upgrades, while fighting off Charles’ unpredictable attacks. There are some very crude stealth sections that could have been improved, but the game is gorgeous and the train driving and the shooting are very satisfying. Basically, if you’ve watched the trailer of the game and it looks fun, you’ll probably enjoy it!

Excavation of Hob’s Barrow. This retro-styled point-and-click adventure game captures the spirit of those classics like Grim Fandango and Day of the Tentacle, but with a grim cosmic horror twist!

The player controls antiquarian Thomasina Bateman, who is writing a book on the barrows of England. She is called to the village of Bewlay by a strange letter that invites her to investigate Hob’s Barrow, but when she arrives, she finds the letter-writer nowhere to be found and the locals decidedly hostile to her investigations. Through the solving of various puzzles — ingenious, but not too hard — Thomasina is drawn closer and closer to Hob’s Barrow and the horrifying secrets within.

There is a feeling of doom throughout the game, as it is made clear in the beginning that things will end very badly. The game captures an atmosphere of cosmic horror well, and the pixel art is well done. This game took me a bit over 5 hours to finish, and I found the experience and the conclusion very rewarding.

Night at the Gates of Hell. Speaking of retro experiences, Night at the Gates of Hell is a quite unusual one! A 90s-style shooter with low-polygon graphics, NatGoH is inspired by Italian supernatural horror films like the zombie films of Lucio Fulci.

The game is chock full of jump scares and the plot makes very little sense, but this is completely in keeping with the movies it homages! There is a mix of silly humor thrown in, and some low-resolution nudity that made me chuckle. It is a quirky game, and somewhat clunky at times — in the final battle at the end of the game, the number of zombies on screen caused significant game lag. But I was hooked on its weird twists and turns and changes of location, and ended up wrapping things up in 4.5 hours.

Dorfromantik. For something completely different, there’s Dorfromantik, a very placid tile-laying and landscape-building game!

The premise is simple: you have a finite number of tiles, and you place them one at a time, in order, to build up the countryside. Certain tiles will have objectives associated with them, like putting together 7 connected water tiles, and achieving these objectives gives you extra tiles to continue.

There is a score to the game, but it is designed to be very peaceful and non-competitive! It even avoids giving you a standard “game over” screen, giving you instead a “session completed” message. You can build outwards towards certain special locations, which will unlock new colorful tiles to make the perfect little world. The game has a lovely soundtrack, and it is a perfect diversion when you need a little escape time. I’ve played 15 hours so far.

Strange Horticulture. This contemplative game places you as the new proprietor of a horticulture shop, where you must attend to the needs of customers who come in with a variety of ailments and troubles.

Using your horticulture guide, you not only have to figure out what plant will help a customer, you need to identify the various plants in your collection from the often limited information in the guide! As the game progresses, you will have the opportunity to wander away from your shop and find more plants, adventures that are described as simple text adventures.

There are more sinister workings at hand, as well; you learn that there are forces of good and evil in the community, and each of them needs your services! Depending on the choices you make, you can save the world — or possibly end it.

Nothing in the game is timed, so it is a great game for those nights when you want something to ponder with the TV on in the background. The story, and the game mechanics, have plenty of twists and turns and clever puzzles to keep it entertaining for its full run time, which was almost 7 hours for me.

Vampire Survivors. The most surprising game of the year was Vampire Survivors, thrown together by a single developer using purchased assets. He released it in early access in 2021 for only $4, and it became so successful that it turned into a full-time project that entered full release this October.

Vampire Survivors basically introduced an entirely new type of gameplay, which I have heard referred to as “bullet heaven.” In “bullet hell” games, the entire screen ends up filled with enemy bullets, which must be continuously dodged. In Vampire Survivors, this is reversed: the player builds up an arsenal of ever more powerful weapons, which eventually turns them into a tornado of death, mowing down endless hordes.

The innovative aspect of Vampire Survivors is that you do not aim your weapon. Your only active control is motion, dodging and weaving amongst scores of enemies. Your weapons, that you collect as you level up, fire automatically and each with their own unique pattern. At each level, you get a choice of upgrades, and eventually you learn what types of weapons and their upgrades work well together. Occasional treasure chests from killing more advanced enemies give additional upgrades, and if you survive long enough you can become literally unstoppable. It is actually a great feeling the first time you realize that you’ve chosen your tools so well that you can stand still and just watch the death unfold. Every round lasts only 30 minutes at most, and you start anew after that.

There are numerous characters that can be unlocked through play, new weapons, new tools, new levels and many unlabeled secrets. The game rewards exploration and repeat play, and thanks to the fixed time limit, it is a perfect game to play to kill a few minutes between meetings or whatever.

I’ve played 33 hours, and haven’t even gotten close to unlocking everything yet! And haven’t even purchased the new DLC for it!

Dwarf Fortress. Speaking of games made by small teams, Dwarf Fortress finally got its Steam release, with a Tutorial and updated graphics!

For those not familiar, Dwarf Fortress is considered the most insanely detailed colony sim ever created. It has been in development since 2002, and its Alpha build was released in 2006. It is the passion project of Tarn and Zach Adams, and the original version used ascii characters to represent various people, items, features, and monsters.

The whole idea of Dwarf Fortress is to found and grow a colony of dwarves that have moved to a new land. You have to build your underground fortress from scratch and worry about every detail of keeping your colony alive: food, drink, shelter, defense, crafting, trade, happiness, and all sorts of other stuff I haven’t thought of. The world is procedurally generated, giving it a history before your dwarves arrived, and every dwarf has their own personality, likes, dislikes, aptitudes, and flaws.

Part of the fun of the game is that such a complex colony with so many variables is inevitably going to go disastrously wrong at some point, be it from raiders, starvation, or just digging in the wrong place! Each game, and each failure, tells a unique story.

The brothers had been working on Dwarf Fortress for years with donations optional. More recently, they decided to make it more user friendly with proper graphics and a tutorial, and charge for it on Steam, because they have bills to pay like all of us. It’s actually heartwarming that there was such a positive response, and sales have been so good, that they’ve been able to get themselves the health insurance coverage that they needed. Someone on Steam asked, “are people going to pay $30 for a game that’s been free for years?” and there are thousands of responses simply saying “yes.”

I haven’t delved too deeply in the game yet, as it is intimidating, even with the tutorial! But I’m eager to tell some tragic and hilarious stories.

The Wandering Village. Another unusual colony simulator came out this year, and the twist is that the colony has been built on the top of a massive wandering colossus!

The villagers form a symbiotic relationship with the colossus, helping to take care of it while it in turn keeps them safely above the poisonous post-apocalyptic world.

The graphics are great, the management is detailed but not overwhelming like Dwarf Fortress. I’ve only sunk a couple of hours into this game so far, but looking forward to more.

UPDATE:

Stray. How did I overlook Stray when I first wrote this post??? A game where you basically and primarily play a cat doing cat-like things!

There has never been a better simulator that lets you experience the purity of being a cat, from knocking over piles of books to scratching on carpets. There is a story, too: you are a feline that has accidentally fallen into the post-apocalyptic remains of a city populated by robots. With the aid of a robot companion you pick up, you solve puzzles and help other robots with their problems with the overall goal of escaping and returning to the surface. There are some minor complaints about some extended combat-type sequences and an ending that is too abrupt, but this was one of the most charming games of the year. I finished it in 6 hours.

***************************

Let me close out with a couple of suggestions that are games that I have started, but haven’t fully finished yet but am enjoying!

Pentiment. A game by Obsidian Entertainment, Pentiment is a narrative adventure and mystery set in 16th century Europe at a time of great religious and societal upheaval. You play as Andreas, an artist working at an Abbey in the Bavarian Alps. When a murder is committed at the Abbey, Andreas finds himself drawn into the investigation and the intrigue around it.

I haven’t played very far yet (I’m basically up to the murder), but can attest to the amazing art style of the game, which is patterned after the literary art drawn by monks at the time.

One of the really neat things, even early on in the game, is realizing how much you have to watch what you say! Free expression wasn’t really a thing back then and the game lets you know when you have chosen a statement that will be remembered (and possibly come back to bite you).

I’m looking forward to seeing where the story goes…

Impasto. Finally, let me give a shoutout to Impasto, a horror game inspired by the late works of the great painter Francisco Goya! You are almost certainly familiar with his terrifying Saturn Devouring His Son.

In the game, which is free, you play as a descendant of Goya, coming to the home of the artist to collect your inheritance. Soon you find yourself drawn into the world of Goya’s paintings, representing his fraying sanity.

This game was an undergraduate student project, and it is quite amazingly lovely and creepy. Another game I’m looking to delve into fully.

*********************************

So those are some of the unusual games I’ve played this year! Feel free to leave your own discoveries in the comments! And Happy New Year!

Many people pursue photography while working in another field to pay the bills. But it’s extremely rare for these hobbyists to win prestigious awards against competitors from across the country.

Vancouver dentist George Ching-Yuen Lo, on the other hand, has won several prizes, including the Sony National Award for Canada for his photograph Fighting bald eagles.

Nearly eight years ago, he captured this image (above) at the Fraser Bald Eagle Festival along the Harrison River.

Fighting bald eagles is on the wall in his dental practice at 909 Davie Street in Vancouver. In October, Explorasian Festival posted more of his photos on its website.

Lo is also a pioneer in microscopic dentistry, which led him into photography.

Earlier in his career, he believed that he needed to learn more about hand-eye-brain coordination. Therefore, in the 1990s, he began taking evening lessons to improve his skills with a camera.

According to the Explorasian website, Lo bought his first digital SLR camera in 2007. He used a telephoto lens for the “Fighting bald eagles” photograph.

Lo family linked to Chinese history

Lo was born in Hong Kong and is well known within Vancouver’s Chinese community. But few Vancouver residents are aware of his intriguing family history.

His maternal grandfather, Huang Shao-hsiung, was a significant figure in 20th-century Chinese history. The son of a schoolteacher, Huang graduated from Baoding Military College in Hebei Province in 1916. Later, he co-founded what became known as the New Guangxi Clique.

In 1923, the founder of modern China, Dr. Sun Yat-sen, appointed Huang as commander-in-chief of the anti-rebel army in Guangxi, which is a mountainous region bordering Vietnam. As a member of the clique, he played a key role in this part of Southern China coming under the jurisdiction of the national government.

Huang was the civil governor of Guangxi from 1924 to 1929, and also became a state councillor in the national government.

UBC historian shone light on Huang’s activities

UBC history professor emerita Diana Lary devoted a great deal of attention to Huang in her 1974 book, Region and Nation: the Kwangsi Clique in Chinese politics, 1925 – 1937 (Cambridge University Press). She noted that the “most notable distinction” between Huang and his fellow Guangxi colleagues and more typical provincial warlords was their interest in modern ideas.

As a result, they were not merely local warlords but regional militarists who were responding to the collapse of the central state in China.

“Their horizons extended beyond the boundaries of Kwangsi,” Lary wrote. “Their birth, into educated if declining families, gave them some pretensions to higher values, originally to the universalist values of Confucianism, now to the concept of the nation; their education had introduced them to a new concept of the role of the soldier; and their experience outside the province had attracted them to the revolutionary ideas of the Kuomintang.”

The New Guangxi Clique taxed the opium trade and created the University of Guangxi.

From 1932 to 1934, Huang was China’s minister of the interior, followed by a two-year stint as chairman of the Zhejiang provincial government, and nine months as chairman of the Hubei provincial government. From 1937 to 1946, he was again chairman of the Zhejiang provincial government.

Another relative was the foreign minister

That’s not Lo’s only influential ancestor. His great uncle on his dad’s side, Lo Wen-ken (a.k.a. Lo Wen-kan and Luo Wengan), was China’s foreign minister under Zhang Zuolin’s government in 1928 and again in 1932 under Chiang Kai-shek.

In that position, Lo Wen-ken pleaded with western powers to intervene to fight Japan, which had taken over Manchuria and created a puppet state. Chiang replaced him as foreign minister that year and he later became minister of justice.

Lo Wen-ken studied law at Oxford University in the early 20th century, which was very rare for a person from China. During his career, he also held positions as minister of finance and president of the Supreme Court in Beijing.

Charlie Smith is the editor of Pancouver. Follow him on Twitter @charliesmithvcr. Follow Pancouver on Twitter @PancouverMedia.

The post Vancouver dentist Dr. George Lo wins plaudits for his photography, but few are aware of his intriguing family history appeared first on Pancouver.

By Gabriel Yiu

With October’s announcement of an additional $50 million—for a total of $100 million—from the B.C. government, the Vancouver Art Gallery’s new home is set to become one of the city’s most iconic landmark architectural statements.

Provincial funding adds to a growing conglomeration of committed public and private donations, including $29.3 million from the federal government and more than $190 million in private donations. This includes $100 million from the Audain Foundation and $40 million from the Chan Family Foundation.

The injection of private and public funding has enabled a second redesign of Swiss studio Herzog & de Meuron’s original blueprint. The new design, released to the public in 2021, is a collaboration between Herzog & de Meuron, Vancouver architectural firm Perkins & Will, and Coast Salish artists Debra Sparrow, Skwetsimeltxw Willard (Buddy) Joseph, Hereditary Chief Chepximiya Siam’ Janice George, and Angela George.

The new Vancouver Art Gallery is set to stand 68 metres tall and will host 45,000 square meters of floor space. The designers seek to make the massive structure “approachable” by emulating traditionally woven textiles with interwoven copper-toned metals. Along with the use of mass timber, this will give the structure an inviting and earthy feel.

The amalgamation of contemporary design and Indigenous influences, including the use of materials culturally important to local First Nations, creates a truly spectacular architectural work. It seeks to embody the primeval grandeur of the ancient forests iconic to the West Coast landscape.

No doubt, the Vancouver Art Gallery will continue its growing traditional of centering traditional and contemporary works of Indigenous artists local to our Coast Salish territories and nationwide.

Its impressive building façade mirroring the West Coast landscape and the rich art and culture of local Indigenous peoples will anchor the gallery close to Indigenous histories and roots.

Now, the gallery must seek to answer the next big question. How will its upcoming exhibitions reflect the soul of this diverse city’s people, especially the diasporic communities of colour who form the majority of the city’s population?

Chinese artists often overlooked

As a lifelong aficionado of Chinese art and culture, I am often dismayed that in a region inhabited by half a million people of Chinese heritage and descent, the enormous artistic talent of this community has gone mostly unseen and unacknowledged by the Vancouver Art Gallery, and by extension, Vancouver’s general population over the past decades.

Vancouver is home to many of the world’s most renowned Chinese Canadian painters. But despite their international fame, their work has rarely been prominently featured at the Vancouver Art Gallery. (One exception came in 2017 with the 12-week exhibition called Pacific Crossings: Hong Kong Artists in Vancouver, coinciding with the 20th anniversary of the return of Hong Kong to China.)

The gallery has missed an opportunity to serve as the artistic and cultural hub of the diverse city that is North America’s Asia-Pacific Gateway. With the new gallery building infusing the city with a new sense of anticipation and excitement for local art, my great hope is that it can finally capture and showcase the full extent of this city’s talent.

In the next section, I will introduce readers to some of these unknown masters who have been our neighbours for years and decades.

Professor Johnson Chow (周士心)

Vancouver was home to the late Professor Johnson Chow for over half his life. Professor Chow is a scholar of Chinese calligraphy and painting, and the number of students he has mentored and influenced range in the thousands. His painting style originated from the Southern School of Chinese painting, often called “literati painting”, a style that emphasizes the expression of the artist’s inner realities over the accurate portrayal of the outward likeness of objects.

Having immigrated to Canada in 1980, some of Chow’s most influential works include Ancient Glacier in Los Angles, Iguazu Falls in Brazil, The Grand Canyon, The Rocky Mountains in Canada and Jiuzhaigou in Autumn. These paintings, many of which are of landscapes in the Americas, are departures from the traditional Chinese painting styles and techniques. The named paintings were all completed in Vancouver.

Chow’s works are displayed as a permanent collection at the Hong Kong Museum of Art (HKMA). In 2011, the HKMA presented a grand exhibition featuring over 60 pieces of Chow’s finest paintings. There is also a museum dedicated to showing his work in Suzhou, China. This was his birthplace and home before he lived in Hong Kong for two decades. Aside from his prolific painting albums, Chow has penned and published extensive collections essays and books on art instruction, art theory, and art history.

Pui Sun Tung (董培新)

Another master artist of international renown, Pui Sun Tung has recently celebrated his 80th birthday at his home in Vancouver. His work has been shown in numerous solo exhibitions in cities across China, Hong Kong, and Taiwan.

Last year, Hongkong Post transformed eight of his Romance of the Three Kingdoms paintings into postal stamp collections. Tung’s work started a new chapter for Chinese illustration, drawing from the rich and chivalric stories and historic events of traditional martial arts novels. With outstanding technique and conceptualization, Tung brings to life the epic and mystical scenes of romance and martial arts from master storytellers like the legendary writer Jin Rong.

Yang Shanshen (楊善深) and Carrie Koo (顧媚)

Yang Shanshen and Gu Mei (also known as Carrie Koo) are two ubiquitous Vancouver names in international art auctions of Chinese paintings and calligraphy.

The late Yang Shanshen immigrated to Vancouver in 1988. He is a renowned artist of the Lingnan, or Cantonese School of Chinese painting. It’s a revolutionary and innovative artistic movement influenced by the Japanese Nihonga visual arts, and seeks to create new school and styles of art through the fusion of Han Chinese and Western, ancient and modern styles. Last year, the Yang Shanshen Lingnan Art Museum was established in Taishan, China, to commemorate and display his work.

Carrie Koo was a Hong Kong pop star before immigrating to Canada and making Vancouver her home. She studied painting under the tutelage of Zhao Shao’ang (趙少昂), a contemporary of Yang Shanshen and considered one of the great Lingnan artists, as well as under contemporary water-ink painter Lui Shou-Kwan (呂壽琨). Carrie Koo’s landscape paintings, which were part of the 2017 VAG exhibition, play with haze and light/shadow contrast and are perfect demonstrations of Western-Chinese fusion and mastery.

A Permanent Exhibition of Chinese Canadian Artists for Vancouver

Vancouver is truly a land of “crouching tigers, hidden dragons,” a phrase that describes great talents walking undetected among the population. Yet great art should never remain hidden and unseen. I have only scratched the surface of Vancouver’s hidden repository of international talents. It is not an exaggeration to say that some of these talents are of history-making calibre, yet they remain overlooked by mainstream media and the Vancouver Art Gallery.

I hope that the new chapter of the Vancouver Art Gallery will include a permanent exhibit of Vancouver’s vast Chinese Canadian talents and serve as a hub where people can come from all over the world and experience the cultural richness of North America’s Asia-Pacific Gateway. This will also help the Vancouver Art Gallery become an institution that truly reflects the contemporary vibrancy of our diverse city.

I believe that this vision is not my wish alone. I suspect that all the living artists named above and their collectors, patrons, family, and all of the artists and art enthusiasts whom they have inspired will also support the establishment of a permanent exhibition of Chinese Canadian artists at the Vancouver Art Gallery.

The contributions of Chinese Canadian immigrant artists to the international art community should be celebrated as a proud part of Canada’s cultural and artistic heritage. A gallery truly reflective of Vancouver’s diversity cannot only curate Euro-American works and developments, while ignoring the artistic history being made right here in our own neighbourhoods.

Vancouver is uniquely positioned as an intersecting point of global movement of people and goods. It is past due time that Vancouver Art Gallery fully taps into the city’s abundant talent and endeavour to become an international hub of artistic exchange.

Gabriel Yiu is a writer and art lover in Vancouver.

The post Gabriel Yiu: Vancouver Art Gallery’s new chapter should reflect city’s cultural wealth appeared first on Pancouver.

Via this comment on Hacker News I started exploring the ClickHouse Playground. It's really cool, and among other things it allows CORS-enabled API hits that can query a decade of history from the GitHub events archive in less than a second.

ClickHouse

ClickHouse is an open source column-oriented database, originally developed at Yandex but spun out into a separate, VC-funded company in 2021. It's designed for big data analytical queries - in a similar space to HBase, BigQuery and DuckDB.

It turns out it can do that trick with HTTP range queries where you can point it at the URL to a Parquet or .native.zst file (ClickHouse native format, optionally compressed using Facebook Zstandard) and run queries without downloading the entire file first.

Exploring the Playground

The ClickHouse Playground is a free hosted environment for trying out ClickHouse. You can access it here:

https://play.clickhouse.com/play?user=play

Try this query, taken from the ClickHouse Everything You Always Wanted To Know About GitHub (But Were Afraid To Ask) tutorial:

SELECT count() FROM github_events WHERE event_type = 'WatchEvent'

github_events

The github_events table contains a copy of the GH Archive - a project that archives and makes available the public GitHub timeline, I think using data from the public events API. GH Archive then makes that data available as compressed newline-delimited JSON in this bucket. The archive stretches back to February 2011, and is constantly updated.

The ClickHouse demo table is continually updated with the latest archived data, by this script, which runs every 10 minutes.

You can do all sorts of fun stuff with it. Here's my recent activity acrosss all of GitHub:

SELECT
  created_at,
  actor_login,
  repo_name,
  event_type,
  title
FROM
  github_events
WHERE
  actor_login = 'simonw'
  AND repo_name != 'simonw/disaster-data'
ORDER BY
  created_at DESC
LIMIT
  100

This link executes that query - note how it includes a base64 encoded copy of the SQL query following the # in the URL.

There are 77 tables total in the Playground instance - you can get a list of them like this:

SELECT database, name FROM system.tables

API access

You can access the API via curl like this:

curl 'https://play.clickhouse.com/' \
  -X POST \
  -H 'Authorization: Basic cGxheTo=' \
  --data-raw $'
    SELECT created_at, actor_login, repo_name
    FROM github_events
    WHERE event_type = \'WatchEvent\'
    ORDER BY created_at DESC LIMIT 100'

This defaults to returning TSV without column headers, like this:

2023-01-01 03:59:59	Willmac16	nlohmann/json
2023-01-01 03:59:59	Samrose-Ahmed	Stebalien/stash-rs
2023-01-01 03:59:57	CodePromoter	aplus-framework/image

To get back data in JSON instead, add ?default_format=JSON to the URL. Here I'm piping that through jq to pretty print it:

curl 'https://play.clickhouse.com/?default_format=JSON' \
  -X POST \
  -H 'Authorization: Basic cGxheTo=' \
  --data-raw $'
    SELECT created_at, actor_login, repo_name
    FROM github_events
    WHERE event_type = \'WatchEvent\'
    ORDER BY created_at DESC LIMIT 1' | jq

Output:

{
  "meta": [
    {
      "name": "created_at",
      "type": "DateTime"
    },
    {
      "name": "actor_login",
      "type": "LowCardinality(String)"
    },
    {
      "name": "repo_name",
      "type": "LowCardinality(String)"
    }
  ],
  "data": [
    {
      "created_at": "2023-01-01 03:59:59",
      "actor_login": "Willmac16",
      "repo_name": "nlohmann/json"
    }
  ],
  "rows": 1,
  "rows_before_limit_at_least": 341429632,
  "statistics": {
    "elapsed": 0.925636889,
    "rows_read": 341540363,
    "bytes_read": 10567093585
  }
}

More format options are documented here.

CORS requests from JavaScript

This pattern works for running queries from JavaScript. CORS is enabled - I pasted this into the Firefox DevTools console on https://www.example.com/ and it returned the results I expected:

r = await fetch("https://play.clickhouse.com/?user=play", {
  method: "POST",
  body: `SELECT
        created_at,
        event_type,
        actor_login,
        repo_name,
        number,
        title,
        body
      FROM
        github_events
      WHERE
         actor_login = 'simonw'
      ORDER BY
        created_at desc
      LIMIT
        100
      FORMAT JSON`,
});
d = await r.json();

Here I'm using FORMAT JSON at the end of the query itself, and passing the requested user as ?user=play rather than sending an Authorization header.

Non-COVID Respiratory Diseases

This article reports that BC CDC says that this year’s flu shot is about 50% effective. That sounds bad compared to like the measles vax or the first results for the mRNA vax, but half is still a big fraction!

In addition to not updating their COVID-19 statistics (which is why there was no BC blog posting yesterday), they also did not update their flu statistics or wastewater data.

At 13:00 on Friday Dec 30, the wait time at BC Children’s Hospital’s wait time is only 1:33, so at least anecdotally, the crisis is easing here.

In the US overall, influenza-like illnesses have clearly peaked (red line for this year):

In the US’ Pacific Northwest, upper respiratory infections have at least plateaued:

Recommended Reading

If you want to read a 2022 COVID-19 in BC retrospective (from someone else, not me), here you go.

I am sitting looking at a long list of organizations that send me emails asking for money at this time of year. It starts off with publications. You can claim up to $500 for amounts you paid in 2022 for qualifying subscription expenses. You can find out more about the digital news subscription at that link. (At the time of writing they were still talking about the 2021 tax return on that first link.) I get The Tyee, The Maple, The Narwhal and the National Observer – so that $500 limit isn’t hard to reach.

There should also be some acknowledgment that Canadian not for profit activist organisations are NOT registered charities. One that failed to point this out until AFTER I had sent them money was Greenpeace Canada. So the following are those that while I support them in other ways, won’t give a usable tax receipt: 350.org, Open Media, Lead Now, Sum of Us, Dogwood, Evidence for Democracy, Democracy Watch. And in the US Next City, Oil Change, Inside Climate News, Climate Central, DeSmog, Change.org, Avaaz, Eco Literacy, Stand.earth

So now for the list of the lucky winners

Transport Action

BC Humanist Association

West Coast Environmental Law

Green Party of BC – which gets a different tax treatment as a political party than charities

Amnesty International

David Suzuki Foundation

Georgia Strait Alliance – which currently has issues with credit cards and PayPal but bank transfers still work

SPEC

I have added URLs to each of the organisations’ home page – which is not necessarily a donation link.

Fair Voting BC just sent me an email which says

“A brief reminder that there are a couple of days left to make a donation and get a 2022 tax receipt for supporting the case. To donate, please click . “

I lasted tweeted on Dec 22. (It was, unsurprisingly, a link to a blog post about Mastodon.) Today I wondered what percentage of the people who appear in my Mastodon timeline today also appeared on Twitter today.

To start, I wrote this query which tries to match Twitter and Mastodon usernames. When it finds a match, it reports the day on which that person last tweeted.

with mastodon as (
  select
    substring(username from 1 for 15) as username, -- twitter names are max 15 chars
    'from:' || substring(username from 1 for 15) as query -- we will query twitter using, e.g., 'from:judell'
  from
    mastodon_toot
  where
    timeline = 'home'
  limit
    500
)
select
  m.username as mastodon_person,
  t.author->>'username' as twitter_person,
  max(to_char(t.created_at, 'YYYY-MM-DD')) as last_tweet_day
from 
  mastodon m
left join
  twitter_search_recent t -- see https://hub.steampipe.io/plugins/turbot/twitter/tables/twitter_search_recent
on
  t.query = m.query
group by
  mastodon_person,
  twitter_person
order by
  last_tweet_day desc

This is my favorite kind of Steampipe query: two different APIs, each represented as a Postgres table, combined with a SQL JOIN.

The result looks like this, with nulls for failed matches.

+-----------------+-----------------+----------------+
| mastodon_person | twitter_person  | last_tweet_day |
+-----------------+-----------------+----------------+
| AlanSill        | null            | null           |
| Colarusso       | null            | null           |
| ...                                                |
| williamgunn     | null            | null           |
| xian            | null            | null           |
| ...                                                |
| futurebird      | futurebird      | 2022-12-29     |
| glynmoody       | glynmoody       | 2022-12-29     |
| ...                                                |
| khinsen         | khinsen         | 2022-12-23     |
| blaine          | blaine          | 2022-12-23     |
+-----------------+-----------------+----------------+

Next I created a table from the above query.

create table public.mastodon_twitter as 
  -- sql as above

And then ran this query.

select
  last_tweet_day,
  count(*)
from
  mastodon_twitter
where
  last_tweet_day is not null
group by
  last_tweet_day
order by
  last_tweet_day desc

Here’s the result.

+----------------+-------+
| last_tweet_day | count |
+----------------+-------+
| 2022-12-29     | 36    |
| 2022-12-28     | 6     |
| 2022-12-27     | 1     |
| 2022-12-26     | 1     |
| 2022-12-25     | 2     |
| 2022-12-23     | 2     |
+----------------+-------+

The 500 toots represented here were created by 93 people who tooted today.

select count(*) from mastodon_twitter

+-------+
| count |
+-------+
| 93    |
+-------+

Of those 93 people, 48 have matching usernames.

select count(*) from mastodon_twitter where last_tweet_day is not null

+-------+
| count |
+-------+
| 48    |
+-------+

Of the 48 with matching usernames, 36 also tweeted today.

So there’s my answer: 75% of the people who appeared in my Mastodon home timeline (when I sampled it just now) also appeared on Twitter today.

—
1 https://blog.jonudell.net/2022/11/28/autonomy-packet-size-friction-fanout-and-velocity/
2 https://blog.jonudell.net/2022/12/06/mastodon-steampipe-and-rss/
3 https://blog.jonudell.net/2022/12/10/browsing-the-fediverse/
4 https://blog.jonudell.net/2022/12/17/a-bloomberg-terminal-for-mastodon/
5 https://blog.jonudell.net/2022/12/19/create-your-own-mastodon-ux/
6 https://blog.jonudell.net/2022/12/22/lists-and-people-on-mastodon/
7 https://blog.jonudell.net/2022/12/29/how-many-people-in-my-mastodon-feed-also-tweeted-today/
8 https://blog.jonudell.net/2022/12/31/instance-qualified-mastodon-urls/
9 https://blog.jonudell.net/2023/01/16/mastodon-relationship-graphs/
10 https://blog.jonudell.net/2023/01/21/working-with-mastodon-lists/
11 https://blog.jonudell.net/2023/01/26/images-considered-harmful-sometimes/
12 https://blog.jonudell.net/2023/02/02/mapping-the-wider-fediverse/
13 https://blog.jonudell.net/2023/02/06/protocols-apis-and-conventions/
14 https://blog.jonudell.net/2023/02/14/news-in-the-fediverse/
15 https://blog.jonudell.net/2023/02/26/mapping-people-and-tags-on-mastodon/
16 https://blog.jonudell.net/2023/03/07/visualizing-mastodon-server-moderation/
17 https://blog.jonudell.net/2023/03/14/mastodon-timelines-for-teams/
18 https://blog.jonudell.net/2023/04/03/the-mastodon-plugin-is-now-available-on-the-steampipe-hub/
19 https://blog.jonudell.net/2023/04/11/migrating-mastodon-lists/
20 https://blog.jonudell.net/2023/05/24/when-the-rubber-duck-talks-back/

Mitigation Measures

This article says that $14.8 million in quarantine fines have been given in BC, MB, ON, and Atlantic Canada. The article hints that most of that money will not be collected, but doesn’t really explain why.

This article says that 54% of Canadians would be okay with a mask mandate if cases rise. (They didn’t say by how much, though.) Almost a third (31%) of Canadians say that they usually or always wear a mask in public indoor spaces.

This article reports that PHAC mailed out six million postcards, urging people to get vaccinated. The article didn’t say which population the postcards were sent to, but I hope it was to people who were unvaccinated. (I did not get a card, and I’m up-to-date.)

A bunch of countries (including the US) are requiring COVID-19 tests on arrival from China, but this article says that Canada doesn’t show any signs of doing the same. (In case you hadn’t heard, China has dropped their Zero COVID policy and, predictably, their case counts have soared. Don’t fuck with exponentially-growing viruses!)

Long COVID

This paper from the USA says that people with loss of smell/taste who got platelet-rich plasma therapy (PRP) got somewhat better than those who did not get PRP.

This paper from the USA found that fully vaccinated patients with severe auto-immune disease (like rheumatoid arthritis) were half as likely to have Long COVID symptoms at 28 days, and only 10% as likely to have symptoms at 90 days.

This preprint from the USA found that overweight/obese adults taking the diabetes drug metaformin immediately after a positive COVID test were about half as likely to get Long COVID; those taking ivermectin were no more or less likely to get Long COVID; those taking fluvoxamine were 36% more likely to get Long COVID.

This paper from the USA compared people who had had COVID-19 with people who had cold or flu, and also with propensity-matched controls. Compared to the cold/flu/pneumonia group, COVID-19 patients were more likely to have palpitations, hair loss, fatigue, chest pain, dyspnea, joint pain, and obesity in the post-infection period. Fine.

Now the part that I don’t understand: there were a ton of things that were significantly more common in the COVID-19 group compared to the cold/flu/pneumonia group but which were not more common in COVID-19 patients than in the controls! I don’t understand how that could happen.

The symptoms for which that was true includes: pulmonary embolism, hypoxemia, other respiratory failure, dependence on oxygen,
pneumonia, tachycardia, unspecified anemia, heart failure, hypertension, hyperlipidemia, muscle weakness, and diagnoses of type 1 diabetes.

They found that loss of smell, anxiety, mood disorders, increased risk of stroke, cerebral hemorrhage weren’t any more common in COVID-19 and cold/flu/pneumonia groups.

I find this deeply, deeply strange, as I have seen lots of papers which say that lots of those symptoms are much more common in people who had had COVID-19 than people who had not. (Why am I telling you about this if I don’t believe the results? Because I think it’s important to report what is happening, regardless of whether I believe in it. Within limits.)

Variants

As this Twitter thread explains, the F486P mutation is difficult for SARS-CoV-2 to achieve: it requires two mutations (not one, like is almost always the case) plus the base pair changes to get there are relatively high energy. As this specific tweet in the thread points out, “F486P is not a mutation you would expect to see unless there are very strong selective forces favoring it.” XBB.1.5 has this mutation, and is starting to make strong headway in the US. This blog post talks more about XBB.1.5.

Treatments

This paper from China says that their oral drug, VV116, is as effective as Paxlovid with slightly fewer side effects.

This preprint from Australia says that Molnupiravir can cause viable, persistent mutations in SARS-CoV-2.

Vaccines

There’s a vaccine which is grown in a relative of a tobacco plant that was developed in Canada by a company called Medicago. Medicago was a joint venture between Phillip Morris International (yes, the tobacco company) and Mitsubishi Tanabe Pharma. Medicago was slow, but eventually made a vax that works. (PHAC has approved it.) However, they couldn’t get WHO to approve it because WHO has rules against doing anything with tobacco companies. This article says that Medicago somehow kicked Phillip Morris out of the partnership, presumably so they could have another go at getting WHO approval.

This report from the USA says that a bivalent mRNA booster (Omicron+COVID Classic) gave 73% additional protection against hospitalization than a monovalent COVID Classic booster between September 8 and November 30, 2022.

Pathology

This preprint from Australia says that people with COVID-trained immune systems produce T-cells really fast — within a day after SARS-CoV-2 infection — and high levels of T-cells predicted a short case.

This press release from a university in Australia says that COVID-19 can cause problems in depth perception, but my understanding is that it should only be temporary.

The Flipper Zero is a cute “multi-tool” inspired by a Johnny Mnemonic character that, among other things, does really well as a digital radio scanner/transceiver.

Back in July of 2020, and again in September of 2020, I wrote about tracking costs in Reclaim Cloud.

Costs in the Cloud

A Follow-up on Costs in the Cloud

Two years later things have developed a fair amount with my personal use of Reclaim Cloud, and I think it’s worth sharing because as I move more and more of my web properties to the cloud, the costs have changed quite a bit from the $87.96 I was spending monthly in September of 2020. In fact, I’m now spending close to $400 per month for my Reclaim Cloud environments alone, not to mention my Cloudflare bill and my terrible domain habit Anyway, below is the billing breakdown by environment for the first 30 days of December 2022:

December 2022 costs by environment for my personal Reclaim Cloud

I went from 12 environments in 2020 to 21 environments in 2022. My monthly bill went from $87.96 in September of 2020 to $397.45 for the first 30 days of December. That is a significant jump, and it’s worth looking at how those costs break down. I have roughly two main groups of servers I manage. The first is the bava fleet of servers, which includes the bava.blog (in stereo), bava.tv (PeerTube), bavaghost (Ghost), bavacast (Owncast), bavameet (Jitsi), bavasocial (Mastodon), and two WordPress backup environments with three days of on-demand backups of the blog across two different regions. Let’s look at the numbers:

• bavacast                $8.51 (Owncast instance I should probably keep off unless in use)
• bavacast-clone    $0.02 (temp environment while moving bavacast to 1-click installer)
• bavaghost             $19.61 (Instance of Ghost I am running alongside my blog)
• bavameet             $2.93 (Jitsi instance I only turn on when needed)
• bavamulti-1         $24.52 (WordPress Multiregion primary site for the bava.blog)
• bavamulti-2         $24.49 (WordPress Multiregion secondary site for the bava.blog)
• bavasocial            $25.18 (Test Mastodon instance for bava.blog I’m still playing with)
• bavatube               $33.81 (PeerTube instance that hosts over 1000 videos)
• bavatube-clone   $14.21 (Clone of PeerTube to test version 5 upgrade)
• env-7614939       $5.21 (Regular nightly backups of bava.blog on UK environment)
• env-9758267       $1.12 (Regular nightly backups of bava.blog on WC environment)

This comes to a subtotal of $159.61. If I was a bit more frugal I could probably eliminate $60-$70 by turning bavacast off more regularly, getting rid of the bavatube-clone, abandon the multiregion setup, and deciding not to run a separate mastodon instance for bavatuesdays. By eliminating those environments I can easily keep the personal hosting bill for the bava properties well below $100 per month. Not necessarily cheap by shared hosting standards, but the difference is what you can do. I’m running a pretty ridiculous multiregion setup that is overkill for my blog, but a testing ground for what is possible for sites that need to stay up no matter what. This is that space where my personal sites overlap with my professional research and development, so being able to spend the money to be able to run a lot of these next-generation apps that would never run in cPanel is crucial.

Also, I think the elephant in the room is owning your own media and managing it comes at a steep cost. Using YouTube or wordpress.com or Zoom or Twitch would be arguably cheaper, but not only would I feel dirty, but the cost is their terms and my data. Freedom ain’t free! So it is worth it for me to have the route around those options when possible to explore alternatives at a manageable cost.

The other properties I have been consolidating in Reclaim Cloud are the various ds106 sites. Namely ds106.us, the WordPress multisite that was a beast on shared hosting, but purrs like a kitten on Reclaim Cloud. Then there is the mighty ds106radio running on Azuracast in the cloud; the newly christened Mastodon server social.ds106.us; the old gold tilde space ds106.club; and finally the listen.ds106rad.io Apache server I just spun up last month to move it off cPanel. There is also the old ds106.tv environment running an Ant Media server I just shut down. That has been replaced with the ds106.tv server running on PeerTube, and I’m not deleting it until I know everything is synced to the new instance. So, let’s do the math on the monthly costs to host the ds106 empire:

• ds106.us                    $34.82 (This is an active WordPress Multisite with years of archives, the daily create, assignment bank and much more)
• ds106radio                $50.54 (Instance of Azuracast running ds106radio)
• ds106social               $57.98 (Mastodon instance running social.ds106.us)
• ds106.club                $5.72  (Fun old school tilde server that’s still running purely for the joy it brings me)
• ds106.tv                    $34.68 (The old Ant Media server we are running, the new Peertube instance is running at $12 per month)
• listen2ds106radio   $5.71 (an apache server I am testing to run archived HTML sites from)

And that’s the ds106 breakdown at a subtotal of $189.45 per month. You’ll notice that Mastodon and Azuracast are the most expensive environments, but once I retire the Ant Media Server, the cost could go down another $15-$20 monthly. But I might actually increase costs for ds106.us given I want to see if a multiregion setup works for that WordPress setup. Would love to get it running through Cloudflare and have failover, CDN action and DDoS protection, not to mention image compression on the fly. We’ll see, enough folks still use that server that I do not want to get cavalier with it, but it would be nice to treat ds106.us as mission critical infrastructure, even if it’s not in the eyes of many

So, with $159.61 for the bavaproperties and $189.45 per month for ds106, where is the unaccounted $50? That would be a couple of personal projects outside bavatuesdays I am playing with, namely No Copyright Intended which is a Peertube instance with all my archived VHS tapes, that runs $28.18 per month. Then there is Antonella’s Ghost blog that runs $14.34 per month, and finally a couple of sites I use to track my domains so I remember to renew the growing horde of online addresses I’m regularly accumulating (that’s another post). And with that you have just about $400 per month, sounds crazy, but then think about all the sites it is powering, and how damn well they work

The other cost, besides domains, is running the DNS for these sites through Cloudflare. I have the Cloudflare Pro Plan at $20 per month (mainly for bavatuesdays, the other domains are free) and I pay an additional $5.00 per month for the Argo smart routing of traffic for the bava. On top of that I pay another $8.80 for the 80 GB of accelerated traffic through Argo monthly (the first GB is free), which acts as a kind of speed boost. Finally, I pay another $15.50 for the load balancing feature through Cloudflare that I use to manage the bava multiregion setup. That breaks down to $5 for basic load balancing, another $10 for geo-routing traffic to the closest server, and $0.50 for DNS queries beyond the initial 500,000. Is it overkill? Definitely. Am I learning a ton about how amazing Cloudflare is and what it enables for sites that need to be constantly online? Absolutely.

So, at the end of the day, excluding domains, I am paying just about $450 for hosting my various bavatuesdays sites and all the ds106 instances. To be clear, I can be so extravagant with these resources cause I help run a hosting service—that fact is not lost on me. At the same time it is helping me put costs in the cloud in some perspective in an attempt to understand what the real costs of hosting your own data are, not to mention running larger community sites. I can afford to do this for ds106.us because it is Dr. Oblivion’s illegitimate love child, and bava is my bread and butter, but not everyone is in the same boat. But the more I put into the cloud the more I start to frame this exchange in terms of the cost of peace of mind and performance for the pain and suffering I went through when trying to run bavatuesdays and ds106.us on a shared server. Perhaps this whole post is more an argument for minimal computing to shed the weight of the past, but that is not how I roll. I’m a web hoarder and I’m just gonna need to move to Cloud City!

lousolversons:

LOUSOLVERSON’S REVIEW OF 2022
Best Film - Everything Everywhere All At Once, Dir. Daniel Kwan, Daniel Scheinert

Let’s start our 2023 predictions off with some thoughts on artificial intelligence. With ChatGPT, Silicon Valley seems to have gotten a bit of its mojo back. After two decades spent simmering the magic of Apple, Google, Amazon and Facebook into a sticky lucre of corporate profit, here was the kind of technological marvel the industry seemed to have forgotten how to make – a magical tour de force that surprised, mystified, and delighted millions.

Even better, ChatGPT didn’t come from any of those corporate titans – not directly, anyway. Instead it came from a non-profit artificial intelligence research laboratory called OpenAI.  Founded in 2015 with a mission of furthering “responsible AI,” OpenAI is backed by some of the most celebrated names in Valley technology – LinkedIn’s Reid Hoffman, PayPal’s Peter Theil, Tesla’s Elon Musk among them. Now this was more like it!

ChatGPT seemed to burst from nowhere – but of course, like Google or TikTok before it, its success leverages years of consumer behavioral data and decades of academic research in mathematics, artificial intelligence, and linguistic models. Over the past seven years, OpenAI has evolved its corporate structure to incorporate a for-profit model and more traditional venture investment schemes – with all their attendant complexities.  Now owned in large part by the very investors who gave us tech’s last two decades of mixed blessings, it remains to be seen if OpenAI will remain true to its mission of ensuring “that artificial general intelligence benefits all of humanity.”

But let’s be real here: It takes capital to build at-scale AI applications – a lot of it. For all its tickling of the popular imagination, ChatGPT lacks a business model. And one of the most ironclad mandates of money is that money sown must become money reaped. Which  takes us to the question driving my first predictions for the coming year: ChatGPT will drive several significant innovations in digital business models. The first will be for ChatGPT itself – it will start to license its technology to at-scale clients, initially to OEMs who will blend ChatGPT with their own offerings. The next two will come via ChatGPTs first big new clients. Google, which played an integral – if largely unsung – role in the technology behind ChatGPT – will launch a ChatGPT-like version of its core search offering. In enterprise markets, Microsoft, which invested a cool billion in the for-profit iteration of OpenAI – will launch a ChatGPT-inspired service aimed at its largest corporate clients.

Google: Conversational Search

“Oh sh*t, Google’s screwed.” That’s the consensus of scores of hot-takes on ChatGPT’s launch. “‘Code Red’ for Google’s Search Business,” declared the New York Times. The NY Post, naturally, took it further: “‘Scary’ AI ChatGPT could eliminate Google within 2 years.” And Casey Newton, one of my favorite tech reporters, said ChatGPT makes Google “feel positively prehistoric.” 

Ouch. I tend to disagree with all those hot takes, but as the old Valley trope states, only the paranoid survive, and certainly ChatGPT’s success is a reason for the folks at Google to be looking over their shoulders. Or perhaps more fittingly, into the mirror, where they likely see a company that’s developed an unflattering middle-age paunch. Could it really be outrun by a smaller, more agile version of itself? Is that even possible anymore?

I’m quite sure the board and major investors in Alphabet, Google’s parent company, are not only asking these questions, they’re demanding answers. And those answers will most likely take the form of a new product from Google in 2023 that I’ll call “Conversational Search.” (If you’ve read this site for the past two decades, that term will certainly resonate!).

Here’s how I imagine it might work. Pairing the open APIs and source code of OpenAI (assuming the newly for-profit company will allow it), Google’s massive trove of voice data, and/or its own internal chat platform, Google will build a novel conversational interface to its flagship Google search application. Text-based search has always had what I call a “modal” problem: often the first answer to a query isn’t accurate. Many in the search field wish they could pop up a modal dialog after an unsuccessful search, asking “Did you mean…?” This would allow the engine to both refine results, and gather critical data that would allow it to better answer the query next time. But there’s a problem: More than 50 percent of users will abandon their search when they see a modal dialog box.

The ChatGPT model of conversational “prompt and response” solves for this problem, providing a fresh context for how humans like to gather information (in essence, by talking to each other). The company will probably dub its first efforts in conversation search as “experimental” – Gmail was famously in beta for five years – but this will be deadly serious project.

Plus, it’ll be fun. Imagine a mashup of Google’s high-fidelity search with the serendipity and human-like conversational tone of ChatGPT. Unlike the stilted voice prompts of Alexa, Google Home, or (shudder) Siri, Conversational Search would be like talking with endlessly wise, patient, and intelligent guru. Pulling such a feat off would take and extraordinary amount of work, CPU cycles, and scale, but…Google is capable of all that and more. Plus, Google is strongly motivated to figure out a business model for Conversational Search, and it’s the one company both most likely to pull it off, and with the most to lose if it doesn’t. Marketers have been crying out for brand-friendly  innovations in digital advertising, and Conversational Search could be just the ticket (for more on that, I’ll link to a future post here, once I’ve written it).

Microsoft: Enterprise Explorer

Microsoft also has a consumer search business (Bing, anyone?) but the company makes its money in enterprise software, and it’s already in the business of selling AI solutions to big companies worldwide. What I’ll call “Enterprise Explorer” could be a hugely successful – and profitable – upsell to its top clients, who wouldn’t mind paying, say, another $10 million or so a year to have a useful, sexy, and energizing new application at their disposal.

So what would Enterprise Explorer (E2, to be corporate cute) be? Built, again, from a mashup of OpenAI technology and Microsoft’s Azure compute platform, E2 would address some of ChatGPT’s most annoying problems – its indifference to truth, for example, or the biases inherent to its Web-scale training corpus. The idea would be this: Train a specific ChatGPT instance on just the body of data owned or operated by a particular corporation. Most large companies have access to petabytes of internal data – everything from customer databases to internal messaging and document management platforms, all accreted over decades. Add in partner data – cleaned and secured through industry-standard methods like data safe havens – and you could hit a tipping point in terms of pattern recognition and results. E2 could spark a revolution in accessing, querying, and delivering enterprise- and industry-specific  intelligence – finally paying off decades of empty promises about the power of digitization and “executive information systems.” Imagine every employee being able to – quite literally – ask the enterprise questions about itself. The mind…boggles. As with Google and Conversational Search, pulling off such a feat would require a staggering amount of innovation and work. And again, just as with Google, Microsoft is deeply motivated to do exactly that.

So to summarize, my first three predictions are this: One, that ChatGPT finds a business model, two, that Google launches an initially experimental Conversational Search interface, and three, that Microsoft announces or launches an Enterprise Explorer-like application for its major Azure clients.

—

This is the first in a series of posts exploring my 2023 predictions. Here’s a link to the second post, and the third. And here’s a link to the summary post. Thanks for reading!

Previous predictions:

Predictions 2022

2022: How I Did

Predictions 2021

Predictions 21: How I Did

Predictions 2020

2020: How I Did

Predictions 2019

2019: How I did

Predictions 2018

2018: How I Did

Predictions 2017

2017: How I Did

Predictions 2016

2016: How I Did

Predictions 2015

2015: How I Did

Predictions 2014

2014: How I Did

Predictions 2013

2013: How I Did

Predictions 2012

2012: How I Did

Predictions 2011

2011: How I Did

Predictions 2010

2010: How I Did

2009 Predictions

2009 How I Did

2008 Predictions

2008 How I Did

2007 Predictions

2007 How I Did

2006 Predictions

2006 How I Did

2005 Predictions

2005 How I Did

2004 Predictions

2004 How I Did

I love advertising – particularly digital advertising. There, I said it. Was that so hard? Well, yes, the industry I’ve partnered with for more than three decades can be very difficult to defend – and the past ten or fifteen years have been particularly bad. I’m tempted to say that everything after Google Adwords was a net negative in the world, including Facebook, which was the bastard child of Google, and even the open web and programmatic advertising (a development I’ve previously called “heroic” and “the greatest single artifact of humankind”).

It’s fair to say I have a complicated relationship with what’s come to be called “ad tech” – we developed the first ad servers and banner ads at Wired in the 1990s, I wrote a book about the business and its breakout star (Google) in the early 2000s, I started an advertising-driven open-web business that nearly reached escape velocity around the same time, I still chair an adtech and data-driven descendant of that business today, I’ve work closely with the largest advertiser on the planet for nearly 15 years, I sit on the board of LiveRamp, an essential component of today’s digital marketing ecosystem, I’ve started or advised or invested in countless media companies – most of which are dependent on advertising in one form or another.

So yeah, I love advertising. And I kind of hate it too. With those caveats now duly noted, let’s get into what might be some of the most important developments in the space over the coming year.

First, let’s talk the elephants in the room: Apple, Amazon, Google, and Meta. Remember when they seemed immutable characters in some Marvel franchise – dubbed “The Four” by Prof. Galloway?  Five years in, each of those companies has their share of worldly troubles, but no one paying attention would argue they’re going anywhere soon. They’re still four of the most valuable companies on earth, though Facebook (nee Meta) has really been working hard at failing these past few years. So what will happen with them this coming year?

Throughout 2023, we’ll hear talk of “the new duopoly” – Amazon and Apple. Both have far smaller advertising businesses than either Meta or Google, but Amazon and Apple have strategic advantages that will allow them to steal significant share: They are much closer to the consumer than their rivals. Apple, of course, owns a massive consumer data platform (iOS and the iPhone), and despite the insanely great contradiction inherent in a “privacy” company building a data-driven advertising business, Apple will likely grow past $10 billion in advertising revenues in 2023. (I’d write thousands of words on the hypocrisy of Apple crippling the entire ad tech ecosystem even as it becomes what it claims it hates, but for now others have done a far better job). Regardless, Apple will be a major digital advertising story next year.

So will Amazon. Here’s another player with a superior data – this company knows what you look at, what you covet, and what you buy. That’s pretty much the entire bottom of the funnel, and over the past five years, Amazon has quietly built a $30+billion advertising business on top of it. Expect that to grow to nearly $40 billion in 2023 – and spark a wave of competitors in what is now known as the “retail media” business – advertising networks built on top of retail and consumer data (Target, Walmart, and many others have already built such businesses, and while they are viable, they pale in comparison to Amazon’s capacities).

Combined, Amazon and Apple will likely grow $15 billion or more in revenue in just one year – and that growth will come largely at the expense of its “Big Four” rivals Google and Facebook, each of which need tens of billions in growth each year to support their sagging stock prices. Expect one hell of a war between these four in 2023.

Next up in the advertising related predictions: Netflix. I’m sure you’ve been following the Netflix Finally Capitulates To The Advertising Model story, but just in case, here’s a primer. For more than a decade Netflix stood on holier-than-thou ground, claiming it would never allow advertising on its platform (that’d be the Netflix CEO in 2020). Industry folk predicted Netflix would start selling ads regardless (that’d be me, also in 2020). 2022 proved the year it happened. Given the press loves a good “I told you so” story, initial reviews of Netflix’s advertising business launch were laden with loving spoonfuls of schadenfreude. “Tepid,” wrote Variety, arguably the most important industry voice in Netflix’s world. “Least popular,” sniffed the Wall St. Journal, noting, of course, that Netflix’s stock has plummeted over the past year. “Giving money back to advertisers,” gloated Digiday.

So what will happen in 2023? By year’s end Netflix’s experiment in advertising will be seen as a triumph. No company is more motivated, more data-driven, and has hired more accomplished industry veterans than Netflix, and if anyone is going to figure out what has so far been a total shitshow (that’d be the connected television market), it’s going to be Netflix.   They’ll test, experiment, fail, learn, test some more, and figure out exactly how many ads each of us will tolerate, and they’ll translate that consumer sentiment into data-laden, at-scale advertising products that brand marketers will buy on sight. By this time next year, those spoonfuls of schadenfreude will have turned into paeans of praise.

And finally, Twitter. Oh, Twitter. In past years I’ve focused lengthy posts on Twitter – what the company should do, what it should avoid, who should buy it. Not this year. This year I’m focusing my predictions solely on one thing: Twitter’s mortality. Which is to say, the business that once constituted 90 percent of its revenue base, advertising, is perilously close to death. Given the company is no longer public, it’s impossible to know how badly Space Voldemort has damaged Twitter’s once-sterling reputation amongst many in the advertising business. However, insiders I’ve spoken to estimate Twitter’s US business, which is its largest, is down more than 60 percent year on year. Rough math would therefore put Twitter’s annual revenues – on track to be more than $5 billion in 2022 – at something like $3 billion on a go forward basis. That makes its consumer data business insanely important – Twitter’s data sales are a nearly half-billion-dollar profit driver that almost no one understands. So here’s my prediction: In 2023, Elon will tire of Twitter, driven as much by the reality of his waning wealth at Tesla (prediction #8, here) as by the sheer biological reality of endorphin fatigue. He’ll hire a real CEO who commands respect in the ad world, contractually obligate himself to not meddle, and find some way to claim victory in what will otherwise become a world-class Harvard Business School Case in What Not To Do.  Given all this, in 2023 Twitter will rebound, and by the end of the year, the stories will about the miraculous rebirth of The Bird, because, well, that’s always been Twitter’s story.

So there are three more predictions for 2023: A war between duopolies, a Netflix comeback, and Twitter’s phoenix rising. My first post, which focuses on AI-based predictions, is here. My third, focused on markets, is here. And the summary of all of them is right here. Thanks for reading!

—

This is the second in a series of posts exploring my 2023 predictions. Previous predictions:

Predictions 2022

2022: How I Did

Predictions 2021

Predictions 21: How I Did

Predictions 2020

2020: How I Did

Predictions 2019

2019: How I did

Predictions 2018

2018: How I Did

Predictions 2017

2017: How I Did

Predictions 2016

2016: How I Did

Predictions 2015

2015: How I Did

Predictions 2014

2014: How I Did

Predictions 2013

2013: How I Did

Predictions 2012

2012: How I Did

Predictions 2011

2011: How I Did

Predictions 2010

2010: How I Did

2009 Predictions

2009 How I Did

2008 Predictions

2008 How I Did

2007 Predictions

2007 How I Did

2006 Predictions

2006 How I Did

2005 Predictions

2005 How I Did

2004 Predictions

2004 How I Did

I’ve used the image above for many years, mainly because I love how surprised the guy looks as he gazes into the crystal ball. Or maybe he’s just sat on something unpleasant. In any case, it pretty much sums up my approach to this, my 20th edition of annual predictions. I sit down, I might have an adult beverage on hand, and I just write until I feel like I’m done.

While reviewing my ’22 predictions (I did pretty well!) I promised to do something new: One post per predictions, ten posts total. But as I began that promised work, I realized it would test the limits of even my most dedicated readers (I see you, kids). So instead I wrote three long form posts, each with three or four predictions apiece. The first focused on AI, the second on advertising, and the third on markets, with a bonus call related to the ’24 election. Having now written all of them, I’m going to summarize them briefly in this “master post.” Grab your own favorite beverage, have a wonderful New Year, and read on!

1. ChatGPT finds a business model. Because of course it will. Which leads to…
2. Google launches a ChatGPT-inspired search interface. Because paranoia. Related…
3. Microsoft launches “Enterprise Explorer” – because there isn’t a big company CEO who doesn’t want some AI to play with.
4. There’ll be a war between the duopolies of Google/Facebook and Amazon/Apple. Grab your popcorn.
5. Netflix will triumph. I know…but the next one’s even more far fetched…
6. Twitter will rebound. (I’ll leave it there for now)
7. Crypto will go sideways in ’23. But that’ll be a good thing.
8. Tesla will continue to tank, and not because of Twitter.
9. Tech IPOs will make a comeback by EOY.
10. Trump will pull out of the ’24 presidential race.

Well that was the quickest predictions post I’ve ever written! Let me know your thoughts – but not on Twitter, as I’m taking an extended pause from the bird site. I’m on Mastodon, or LinkedIn, or leave your comments here…and see you in ’23, which promises to be one helluva interesting year.

—

Previous predictions:

Predictions 2022

2022: How I Did

Predictions 2021

Predictions 21: How I Did

Predictions 2020

2020: How I Did

Predictions 2019

2019: How I did

Predictions 2018

2018: How I Did

Predictions 2017

2017: How I Did

Predictions 2016

2016: How I Did

Predictions 2015

2015: How I Did

Predictions 2014

2014: How I Did

Predictions 2013

2013: How I Did

Predictions 2012

2012: How I Did

Predictions 2011

2011: How I Did

Predictions 2010

2010: How I Did

2009 Predictions

2009 How I Did

2008 Predictions

2008 How I Did

2007 Predictions

2007 How I Did

2006 Predictions

2006 How I Did

2005 Predictions

2005 How I Did

2004 Predictions

2004 How I Did

There are only a couple more weeks to visit one of Canada’s premier museums before it closes for an extended period. After January 15, the Museum of Anthropology at UBC will shut down until late 2023 for building improvements and seismic upgrades to the Great Hall. Until that date, museumgoers can still take in Xicanx: Dreamers + Changemakers / Soñadores + creadores del cambio. This exhibition features the work of artists of Latin American ancestry in the United States.

Traditionally, they were referred to as Chicanos or Chicanas. However, in recent years, Xicanx has emerged as a more inclusive, gender-neutral, and anticolonial term, just as Filipinx sometimes replaces Filipinos and Filipinas.

MOA curator of education Jill Baird and Americas Research Network executive director Greta de León co-curated the exhibition. It includes the work of 33 Xicanx artists.

“Art is the vehicle for conveying messages politically,” de León says in the video below. “You can go back to the muralist traditions in Mexico in the beginning of the last century—and how it’s incorporated in the buildings and the walls.

“It’s what we see,” she continues. “It’s what we walk.”

Video: Greta de León speaks explains why art is a vehicle for conveying messages.

Xicanx presents diversity of artforms

According to de León, art is an “essential core” of the Xicanx civil-rights movement.

One of the featured artists is Los Angeles painter and muralist Judy Baca. Her Tres Maria is included in the show and is part of her series of portraits showing a strong Xicanx woman. In September, Baca spoke along with fellow artists Sarah Castillo, and Cathy Vargas as part of the MOA’s Xicanx Speaks! series.

In addition to paintings, the exhibition presents installation, photography, print, music, and film. “This is the first time a big Chicano exhibition is in Canada,” de León says.

The post Museum of Anthropology at UBC will go on hiatus after Xicanx: Dreamers + Changemakers exhibition ends on January 15 appeared first on Pancouver.