Rolandt

Here’s another dramatic aerial view. There’s nearly a century separating the two images – the Archives image was shot in 1927, so there’s no bridge to the North Shore yet – and very little there if you did manage to cross on the ferry. The picture was taken by Pacific Airways, apparently for the Union Steamship Company.

The West End in 1927 was mostly houses, although the smart money had already moved on to the CPR’s relatively recently released Shaughnessy district, so many of the big old houses (twenty to thirty years old houses), were being divided up or used as guest houses and rooming houses. There were apartment buildings sprinkled throughout the area, and many more were being built in this period, replacing some of those earlier houses.

On the waterfront it’s possible to make out both Englesea Lodge that was right on the water’s edge, and nearby the Sylvia Apartments, both designed by Seattle architect W P White. Between them were two piers, the older (and longer) with a pavilion at the end. It was built around 1905 and demolished in 1938. On a holiday like today in 1927 a band would probably be playing here, and there might be another at the bandstand on the roof of the changing pavilion to the southeast.

Today’s West End has a mix of lower density buildings, some already built in 1927, and far more mid and higher towers, built for the most part (in this part of the area) from the 1950s to the 1980s. The relatively recent West End Plan has encouraged development in certain parts of the area, and in Trish Jewison’s image (taken from the Global News helicopter) it’s possible to spot a number of tower cranes on Davie Street near Denman. There are five new rental towers being built there. The picture was taken in the spring, so the Empire Landmark was still standing, although the revolving restaurant had already been removed; now there just a big hole in the ground, awaiting the construction of a pair of condo towers with social housing over a retail and office podium.

Image source: City of Vancouver Archives CVA 374-181 and Trish Jewison on twitter.

0933

Melanie Mitchell, Aeon, Dec 26, 2019

I understand the concern, but I think it relies on a myth that won't - in the long run - bear scrutiny. The argument here is essentially that human drivers depend on a wealth of knowledge known as 'common sense' - "the mostly tacit ‘core knowledge’ that humans share – knowledge we are born with or learn by living in the world." The myth here is that there is 'core knowledge', that it is common, and that it is required by an autonomous car. The sort of person who uses 'common sense' to plough through a pile of leaves or into a flock of pigeons is the sort of driver who has accidents because "nobody could have predicted" the hidden tree trunk or the damage a pigeon can cause. The AI, meanwhile, will learn a very different 'common sense', one focused on road safety, and hence not be misled. Via The Next Web.

Web: [Direct Link] [This Post]

For creative work, you can't cheat. My believe is that there are 5 creative hours in everyone's day. All I ask of people at Shopify is that 4 of those are channeled into the company.

— Tobi Lutke

Last September I wanted to go to a conference in Sweden and my boss asked me what I would do different from my attendance at the US edition of the event the prior April.

I said, “I’m gonna make a bioreactor demo.”

I wrote about the demo here on our corporate blog. In short, to me a bioreactor is no big deal; it’s just a flask with bugs and media. But for others, there’s a real nerd factor, especially with sensors stuffed into the flask, cables streaming data, and the network connectivity I was demonstrating.

Yes, the pic up to the right is the demo I built.

And then something clicked
Before I get to what clicked, I wanted to mention a bit about the demo, the times, and what it all means.

I had chosen to build the demo around an old Arduino Duemilanove I had lying around. I had bought it ten years ago for my son and I to work on and learn. He and I tried to make sense of it all back then, but I really couldn’t find the info and code to help us make the most of it.

Fast forward to 2019 and I was able to find sensors, libraries, example code, and hardware to whip up my demo in a few weeks of nights and weekends type effort. To me, it was a huge difference from ten years before.

And from the ten years before that.

Once upon a time…
I always envied how some folks could build and program hardware. When I was working at MIT, I’d meet hardware folks (can you say Media Lab?) or walk through the halls seeing all the hardware being hacked (that’s where I fell in love with microcontrollers, wanting to build smart robots). In grad school, my advisor was always programming or getting interesting hardware built for the lab. At Nokia, I met hardware folks at Nokia and outside Nokia, who could not only program hardware, but were building really cool gizmos in clever ways. Indeed, it was because of those folks during my time at Nokia, that I learned about the intersection of hardware, software, and design; of Patchube (then Xively, then Google IoT); Arduino; Raspberry Pi; and hardware hacking (yes, I know some of that stuff is from after my time at Nokia, but the early threads were all there).

Alas, while I had deep experience in designing proteins and organisms (ho-hum to me but perhaps magic to you), I had no clue how to string up an LED with the right resistor and getting something to make it blink (ho-hum to you, but perhaps magic to me).

Sigh.

Back to today
Building the demo took me back to those times. And the tools and information I now had access to gave me superpowers to build something I never thought I’d ever be able to.

But after I built the demo, I sort of stepped back and let things cool.

Sort of.

I had picked up a Raspberry Pi Zero at the Sweden event and started playing with that, learning Linux, Kali hacking, and Python.

And then I started digging into the hardware hacking world, wondering how far that rabbit hole went.

That’s when things just clicked.

A new vice
By October, less than two months from when I first started working on the demo, I was hooked, daily reading hardware hacking sites like Hackaday and Hackster.io; perusing community hardware sites like Crowdsupply, Thingiverse, and Tindie; visiting hardware catalogs and building wish lists (yes, Adafruit, that’s me visiting your site every-frakkin-day); and learning about microcontrollers, resistors, capacitors, PCBs, Discord, Raspberry Pi, systems on a chip and single board computers, retro-gaming, Python, Micro Python, CircuitPython, GPIOs, FPGAs, IC2 and SPI, and so much more. And I’ve met and read stuff and watched videos galore of folks who make and break hardware (helpful to work for a hardware company, too, we have so many smarties).

Then I had a new project idea and bought a few items to make it real: a brand new microcontroller, some sensor boards, a soldering iron. I got it working in no time (thanks to readily available libraries and coding guides). I even made my first PCB for it (just holes and traces to connect things together, no components yet, so far). And, yes, I’ve already broken a few things, humorously so.

OMG, so fun. And woe to anyone who gets me taking about it all. Haha.

Watch that wallet
Along with the amazing community resources of examples, libraries, tools, and guides, the hardware has become so cheap (giving you permission to break things*). You can get a wifi- and BT-capable microcontroller with a ton of memory (for a microcontroller) that can run on and charge a battery and is an inch long for $20. Complex, ready-to-use sensors can be $5 or $15. PCBs can be designed with free tools and ordered online for a few bucks. And just about everything is on Amazon (tho, I prefer to buy from the manufacturer directly, to support them).

But beware: I work hard to keep myself to buying multiple items at a time (to save on shipping). And I work hard to keep from buying everything in sight. It all adds up. And fast. Haha.

What’s next?
I’m surprised by where I have come in three months – the confidence, the knowledge, the skills. And I’ve only scratched the surface. Though it’s clear to me that _at this moment_ this has become a daily obsession that is occupying so much of my spare time, and there’s no sign of it slowing down.

In my innovation work, I tend to do a chocolate and peanut butter ideation exercise. So I wonder if the hardware hacking were the chocolate, what would be the peanut butter I can mix it with? Though, I don’t mean something like hardware and brewing, which is what I am working on with one of my projects. I mean something bigger, something I might mix with hardware to do something completely different, that might catapult me to a new phase in my life.

What might that be?

Dunno.

For now, I’m quite excited running up this hardware hacking learning curve.

Where might it lead me?

*My wife says, in skiing, “If you’re not falling down, you’re not trying hard enough.” Sort of works for life, in general, too.

Garret is back in the blogging-saddle over on dangerousmeta!

“Blogs have been largely relegated in the face of the instant-satisfaction social media. Those that are being linked on commercial news services regurgitate universal knowledge, inspirational quotes, and ‘best practices’ boilerplate. In addition, we are now tracked, tabulated, diced, sliced and served to corporate interests on a golden platter. I don’t need to tell you why or how – we all know we’ve signed our souls away for the terrifyingly bad user interfaces of services like Facebook, Tumblr, Instagram and others. Did I say bad? I mean purposely bad, to encourage you to do what they want, while making it truly difficult to do what you want.
[…]
[W]e all need a kick in the ass to stop with the easy-to-consume poison Kool-Aid, and start caring about this. Others of my blog generation have made the move, it’s past time for me to do so.

I call “time” for the renaissance of the independent self-hosted weblog. I don’t care what you use, or how you use it, but get the hell off the commercial social services and get an independent weblog.

Own your information. Control your information. “

Welcome back Garret. We’ve missed you!

After my dive into my exported Delicious data, and looking at the service its current owner Pinboard offers, I found myself awake last night. I used the time to think about link curation, research and the social and analytical aspects of bookmarking. That against the litmus test of Networked Agency and Indieweb principles. Concluded I need to experiment with rolling my own. First sketches made, and coming up with a doable experimentation plan.

Bookmarked Linqurator

Tim Bray, Ongoing, Dec 25, 2019

These are reflections from Tim Bray (who knows a thing or two about protocols) on Twitter's @Bluesky proposal to create a decentralized social network. Will it work? he asks. Probably not. There are too many things that can't be fixed via the protocol alone - things like "the messy political mechanisms behind our imperfect but essential legal and regulatory frameworks." But maybe it could work, he says, with carefully designed APIs and the sort of AI-based indirection proposed by Stephen Wolfram. And there needs to be some mechanism to define 'verifiable or not' in the network's algorithms, he says. It's a long shot, he argues. But is it worth trying? Absolutely.

Web: [Direct Link] [This Post]

At my day job they say “Think big!”, and @bluesky is that. I can’t stop thinking about it, and bloggers can’t help thinking in public. This work can simultaneously be probably-doomed and obviously-worthwhile.

I’m aligned with where Twitter is starting this conversation. First, speaking under my AWS-employee hat: One of our standard soundbites is that people and enterprises should strive to avoid “undifferentiated heavy lifting”, i.e. wrangling software infrastructure. The basic machinery behind Twitter needs to keep track of who said what and make it available to others. Let’s go further and grant (as I would) that Twitter exhibits the essential nouns and verbs of text-centric social media: Annotated linking, following, echoing, answering, liking, muting, blocking. Which all feels to me like undifferentiated heavy lifting.

Where’s the value?

Twitter’s @bluesky hypothesis is that the real business opportunity in social media is extracting from the planetary-scale tangle a fragment stream that meets the needs of individuals sufficiently well that their attention can be monetized, and does so without being a vector for hate, bigotry, and ignorance.

It’s easy to be dubious of this claim and still convinced the project is worth pursuing. I’m both: I persist in resisting Twitter’s pleas to replace my people-I-follow-in-order feed with their “top tweets” flavor. But self-evidently I’m weird; I have the privilege and education and space and time to obsess over certain aspects of life and their truths and falsehoods, and invest in careful curation of my feed. It’s not unreasonable for people who are starved for time and less privileged to pose a simple request to Twitter (or a competitor): “Please inform me about the world quickly, in a way that enriches my life, without leading me into pathological social dysfunction.”

The second starting point I share with Twitter: Social media as currently constituted does a terrible job. The platform works better for GamerGaters than for humanists; better for Donald Trump than for the Dalai Lama; better for incels than for working mothers. So, what is to be done? The value of the quest hardly seems open to question.

What is the Internet?

It isn’t a place or a thing, it’s a lot of computers which implement protocols written down in IETF RFCs (and to a lesser degree other organizations’ publications) that reflect inspiration and evolution and are known to achieve certain effects in practice. So in the @bluesky launch thread it’s unsurprising to find belief in protocols as foundational to the project. One of the two documents they quote is Protocols, Not Platforms: A Technological Approach to Free Speech by Mike Masnick.

I’ve read Masnick carefully but I’m mostly unconvinced. He seems still to dream the foundational Internet dream that if we can just put people in touch with each other things will be better. Plus there’s market fundamentalism: Let people choose the products that most please them and this will lead to good outcomes and to products which are relatively free of dysfunction.

Which I’d like to believe but don’t. The free market is obviously a useful macroeconomic tool, but only when its freedom is carefully circumscribed. Every market needs a supporting framework of contract law and of regulation: Against fraud, misrepresentation, theft, Ponzi schemes and of course to promote safety: Aviation safety, electrical safety, chemical safety, nutritional safety, and so on. Can a protocol support the messy political mechanisms behind our imperfect but essential legal and regulatory frameworks?

The other issue that seems insufficiently considered is organized hostile action. Empirically, we observe powerful well-funded parties launching concerted efforts to bend the path of societal conversation towards Trump or away from Jesus’ Good-Samaritan lesson; or to boost Brexit. If a protocol could be effective in resisting this sort of adversary, that would be great.

I’m aware of no conclusive evidence that this is either possible or impossible. So I’d be willing to give it a shot. But I’d also broaden the focus; if there is a social-networking technology out there that can ameliorate the current dysfunction, it seems just as likely to be found in a carefully-thought-through API framework, or in some of Stephen Wolfram’s early-stage proposals (also linked in the @bluesky thread) for a level of indirection in ML-based recommendation algorithms.

Block what?

Both @jack in his @bluesky thread and his CTO following up nodded in the direction of blockchain. Any regular ongoing reader can hear my snorts of derision from anywhere on the globe. I’m not going to relitigate the argument at this point; the absence of succesful applications of the technology, all these years into it, should have removed the need. But I think the appeal to blockchain is a symptom of a powerful instinct that whatever we do to fix social media, it has to be involve extreme non-optional transparency. (Blockchain does this but unfortunately fails to be useful for other unrelated reasons.)

I share this instinct and would go further: A powerful focus on transparency and truth is an absolutely necessary (but of course not sufficient) precondition for addressing social media’s sins.

Existence proof

There is a successful online community where truth is the common currency: Wikipedia. Yes, it’s flawed in multiple important ways, but its insistence on publishing only what’s backed by evidence, and its rough consensus on what evidence is acceptable, is at the end of the day heartwarming. Its great flaw is not inaccuracy but inattention; the community of editors is generally insufficiently diverse and specifically mostly-male, and this reflects into Wikipedia’s pages.

Wikipedia’s contract with the world is simple: Any material that is not verifiable will eventually be removed. So obviously the meaning of “verifiable” matters and is thus a source of controversy. Healthy, reasonable, sensible controversy. Which we observe to have an imperfect but highly usable result, relatively free of the currient pathologies of other civic discourse.

Am I saying that the @bluesky solution comes down to “remove anything not verifiable”? No, but I’d be inclined to suggest that “verifiable vs not” should be an important input to the @bluesky algorithms and protocols.

Testing

Suppose the @bluesky team ends up looking at a few big ideas — let’s ignore whether they’re protocols or algorithms or ML models or some other sort of thing. My first question is “how do you tell if they work?” At AWS, something relatively simple, like the performance of a throttling algorithm under stress, can be difficult to test without, you know, putting it under stress.

We’ve developed incremental nondestructive imperfect techniques for evaluating this kind of thing, and you’d need something like that to make useful decisions about @bluesky ideas. You’d need to run them at scale in the context of an automated harness, and you’d need to do it more or less all the time, forever.

The construction of this infrastructure is, I believe, apt to be one of the hardest parts of @bluesky.

Will it work?

I doubt it. Social-media dysfunction seems closely related to aspects of human nature at the individual and societal levels. A solution might end up being more or less equivalent in difficulty to World Peace or Curing Cancer.

Which is obviously not a reason to not try.

In meinem Keller steht ein kleines Rechenzentrum in Form eines ziemlich kapitalen 19"-Schranks. Daraus habe ich in den letzten Tagen große Mengen Elektronik und Aluminium entsorgt. Ich zerlege diese Geräte so gut es geht in ihre Fraktionen, um sie dann zu recyceln. Darunter waren ein paar PCs, bis zu 20 Jahre alt und auch zwei kapitale DL380 Server. Am meisten Arbeit haben mir 25 Festplatten gemacht, aus denen man 20 bis 30 Schrauen herausdrehen muss, bis man die Datenträger isoliert hat:

Das ist nur ein kleiner Anfang. Übrig behalten habe ich nur die Scheiben, die Distanzringe, zwei Permanentmagnete und ca. 700 Schrauben, alle von Hand herausgedreht. Die Magnete stammen aus den Linearmotoren und sind ausnehmend kräftig. Damit lassen sich die Daten auf den Scheiben perfekt löschen. Und danach werden das sehr schöne Untersetzer. :-)

IT-Hardware hat bei mir ein sehr langes Leben. So habe ich gerade einen Sony SDM-M81 an meinen Raspberry-Pi angeschlossen. Den TFT-Monitor habe ich vor 17 Jahren gekauft und der tut immer noch. Ich "drehe" eine Menge Test-Hardware, aber aus der eigengen IT quetsche ich mehr heraus als die meisten anderen.

	sillygwailo shared this story from China Channel:
	None of the books to avoid are the one that got me interested in Chinese history (The Private Life of Chairman Mao: The Memoirs of Mao’s Personal Physician by Li Zhisui), but it’s not on the list of books to read, either.

---

Essential reads for your China library

‘Tis the season of merriment and listicles. Before we break for Christmas, we’ve updated our list of essential China books, in case our latest staff picks weren’t enough for you. We’ve split this into five lists of five: books on contemporary China; books on Chinese history; books from Chinese voices; Chinese classics; and a new fifth section, China books to avoid. (We don’t mean that these books have no value – they are all informative – but that they are ultimately misleading and so you should avoid them in favour of others.)

We hope this is useful as an open sesame for new China watchers, or for old China hands to plug holes in their bookshelf. The lists are designed as all-you-need to pack your bag/Kindle to grasp that aspect or perspective of China, without being overwhelming. Naturally, we have missed out a plethora of wonderful books. But, we hope, this is only the beginning of your reading. See also our lists of 12 must-read Chinese fiction books, and a dozen Chinese films to watch.

5 Books on Contemporary China

Out of Mao’s Shadow: The Struggle for the Soul of a New China by Philip P. Pan (2008) – A very readable work, featuring stories from the post-Mao era that shaped China as it is today, the perfect primer on the “new China”

Factory Girls: From Village to City in a Changing China by Leslie Chang (2008) – Sensitively told and deeply researched, this gives the human (and women-driven) side of the factory boom, with Chang’s own history woven in

Country Driving: A Chinese Road Trip by Peter Hessler (2010) – The third book in New Yorker correspondent Hessler’s China trilogy, both the most up-to-date and the widest in scope, from southern factory to northern farm

Age of Ambition: Chasing Fortune, Truth, and Faith in the New China by Evan Osnos (2014) –  A series of profiles and articles on a range of Chinese dreamers and dissidents, and a winner of the National Book Award

Street of Eternal Happiness: Big City Dreams Along a Shanghai Road by Rob Schmitz (2017) – A portrait of a Shanghai street and its occupants that brings out the range of ordinary and extraordinary lives in China

***

5 Books on Chinese History

The Search for Modern China by Jonathan Spence (1990) – The grandaddy of China history books, taking China’s modern story from the late Ming dynasty up until 1989, this doorstopper is also an absolute pleasure to read

The Heritage of Chinese Civilisation by Albert M. Craig (2001) – One of the best primers on the vast sweep of China’s dynastial history and culture, this volume is clear but has the added advantage of not being overly long

A Bitter Revolution: China’s Struggle with the Modern World by Rana Mitter (2004) – Mitter provides a broad span of 20th century Chinese history through an original and compelling lens, the pioneering May 4th movement

Wealth and Power: China’s Long March to the Twenty-first Century by Orville Schell and John Delury (2013) – Scholarly researched profiles of 14 individuals who made modern China what it is, a gripping approach

China in the 21st Century: What Everyone Needs to Know by Jeffrey Wasserstrom and Maura Cunningham (2018) – Intuitive Q&A structure, with concise, clear answers to the key questions about China’s past and present

***

5 Books from Chinese Voices

Chairman Mao Would Not Be Amused: Fiction from Today’s China ed. Howard Goldblatt (1995) – The perfect introduction to contemporary Chinese literature, with a range of short stories from fiction writers

China Candid: The People on the People’s Republic by Sang Ye (2005) – Oral histories of ordinary Chinese lives, from barefeet doctors to PLA soldiers – a grassroots portrait of China since 1949 in the style of Studs Terkel

The Corpse Walker: China From the Bottom Up by Liao Yiwu (2008) – Surprising and often moving narratives of ordinary people from all over China’s countryside, bringing out everyday experiences with empathy

China in Ten Words by Yu Hua (2010) – A Chinese perspective of the nation’s transition from Cultural Revolution to opening up and the contradictions of 21st century China, compelling non-fiction written by a master novelist

This Generation: Dispatches from China’s Most Popular Literary Star (and Race Car Driver) by Han Han (2012) – Social commentary from the superstar Chinese blogger, on the new generation of young Chinese born after 1980

***

5 Chinese Classics

Classical Chinese Poetry: An Anthology, trans. David Hinton (ancient, pub. 2010) – The most comprehensive collection out there giving a feel for the origins of Chinese poetry and culture, by one of the best translators in the business

The Analects of Confucius by Confucius, trans. Simon Leys (5c. BCE, pub. 1997) – For ancient Chinese philosophy, Confucius may be a conventional recommendation, but Simon Leys brings the heart of Chinese thought to life

Dream of the Red Chamber by Cao Xueqin, trans. David Hawkes (1760s, pub. 1974) – The last and richest of China’s four classic novels, in stylish translation by David Hawkes and John Minford (in, alas, five volumes)

The Complete Fiction of Lu Xun trans. Julia Lovell (1920s, pub. 2010) – China’s George Orwell, or closest equivalent, and an essential writer to have read firsthand. At least read ‘The True Story of Ah Q’ and ‘Diary of a Madman’

Fortress Beseiged by Qian Zhongshu (1947, pub. 2004) – A wonderful novel, hilariously funny and a razor sharp indictment of Chinese society in the late 1930s, this often overlooked but still relevant classic is a modern must

***

5 China Books to Avoid

The Coming Collapse of China by Gordon Chang (2001) – We’re still waiting for this collapse of China’s economy and politics, 18 years after the book came out. This is why you never make a prediction in a book title

When China Rules the World by Martin Jacques (2009) – From the other end of the prognostication spectrum. Some insightful analysis, but in the end these birds-eye-view big-thesis China books just aren’t worth it

Mao: The Unknown Story by Jung Chang and Jon Halliday (2005) – Mao was a complex figure who caused horrific suffering, but this book demonizes him without nuance, and ultimately does disservice to impartial history

Lost on Planet China, by J Maarten Troost (2008) – It might be cruel to single this one out over others, but it’s an example of the “look how wacky this place is” China memoir, which plays for cheap laughs but is shallow as a puddle

Death by China: Confronting the Dragon, by Peter Navarro (2011) – The book that started the new yellow peril scare, and catalysed Trump’s trade war. A classic example of exaggerated scaremongering in the China Threat genre
∎

The post 20 China books to read (and 5 to avoid) appeared first on China Channel.

For years I had been an active user of Delicious, the social bookmarking service. I started using it in 2004, a year after its launch, and stopped using it in 2015. By then the service had been repeatedly sold, and much of its useful social features had been deprecated. It’s one of those great services Yahoo bought and then never did anything with. As I describe in a posting on bookmarking strategies last year, Delicious was useful originally because it showed you who else had bookmarked the same thing as you, and with which tags. It allowed me to find other people with similar interests, and especially if they used very different tags than me for a page they would be outside my own communities and networks (as ‘tribes’ will gravitate to a shared idiom). I’d then start following the blogs of those other people, as a way of widening my ‘very large scale antenna array’ of feed reading. Tags were pivots for triangulation. Delicious is one of those tools that were really social software, as opposed to a social media platform with its now too common self-reinforcing toxicity.

The current owner of Delicious is Pinboard, and according to Wikipedia the Delicious site was officially made inactive last August. That became obvious visiting my Delicious profile in the past weeks (on the original de.licio.us url, not the later delicious.com), as it would regularly result in an internal server error. Today I could access my profile.

My delicious profile

I decided to download my Delicious data, 3851 bookmarks.

After several attempts resulting in internal server errors, I ended up on the export screen which has options to include both notes and tags.

Delicious export screen

The resulting download is a HTML file (delicious.html), which after opening at first glance looked disappointing as it did not show tags, nor the date of bookmarking, just the description. Loosing most context would make the list of bookmarks rather useless.

My delicious html export

However, when I took a look at the source of the HTML file, I found that thankfully tags and dates are included as data attributes of the bookmarks. The HTML is nicely marked up wit DT and DD tags too, so it will be no problem to parse this export automatically.

My delicious html export source showing data attributes

My original notion was to import all bookmarks with their tags and notes, as back dated blog entries here. But randomly clicking on a range of links tells me that many of those bookmarks no longer resolve to an active web page, or redirect to some domain squatting spam outfit. So bringing the bookmarks ‘home’ into my site isn’t useful.
As the export includes tags, I can mine the list for bits of utility though. The collection contains a wide variety of open data usage examples I collected over the years, and that is of interest as a historical library, that I could try and match against the internet archives, using the bookmarking dates. Most other stuff is no longer of interest, or was ephemeral to begin with, so I won’t bother bringing that ‘home’. I will add the delicious export to the other exports of Twitter and Facebook on my NAS drive and cloud as archive. I have now removed my profile from the Delicious website (after several attempts to overcome internal server errors, and it is now verifiably gone).

Consider the following: Santa brings toys to all the good girls and boys.

There are 7,713,468,100 people in the world in 2019, around 26.3% of which are under 15 years old. This works out to 2,028,642,110 children (persons under 15 years of age) in the world this year.

Santa doesn’t seem to visit children of every religion, so we’ll generalize and only include Christians and non-religious folks. Collectively that makes up approximately 44.72% of the population. If we assume that all kids take after their parents, then 907,208,751.6 children would appear to be Santa-eligible.

What percentage of those children are good? It’s impossible to know; however, we can work on a few assumptions. One is that Santa Claus functions more on optimism than economics and would likely have prepared for the possibility that every child is a good child in any given year. Thus, he would be prepared to give a toy to every child. Let’s assume it’s been a great year and that all 907,208,751.6 children are getting toys.

That’s a lot of presents, and, as we know, they’re all made by Santa’s elves at his North China Pole workshop. Given that there are 365 days in a year and one of them is Christmas, let’s assume that Santa’s elves collectively have 364 days to create and gift wrap 907,208,752 (rounded up) presents. That works out to 2,492,331.74 presents per day.

Almost two-and-a-half million presents per day is a heavy workload for any workshop. Let’s look at two paradigms that Santa might employ to hit this goal: concurrency, and parallelism.

A sequential process

Suppose that Santa’s workshop is staffed by exactly one, very hard working, very tired elf. The production of one present involves four steps:

1. Cutting wood
2. Assembly and glueing
3. Painting
4. Gift-wrapping

With a single elf, only one step for one present can be happening at any instance in time. If the elf were to produce one present at a time from beginning to end, that process would be executed sequentially. It’s not the most efficient method for producing two-and-a-half million presents per day; for instance, the elf would have to wait around doing nothing while the glue on the present was drying before moving on to the next step.

Concurrency

In order to be more efficient, the elf works on all presents concurrently.

Instead of completing one present at a time, the elf first cuts all the wood for all the toys, one by one. When everything is cut, the elf assembles and glues the toys together, one after the other. This concurrent processing means that the glue from the first toy has time to dry (without needing more attention from the elf) while the remaining toys are glued together. The same goes for painting, one toy at a time, and finally wrapping.

Since one elf can only do one task at a time, a single elf is using the day as efficiently as possible by concurrently producing presents.

Parallelism

Hopefully, Santa’s workshop has more than just one elf. With more elves, more toys can be built simultaneously over the course of a day. This simultaneous work means that the presents are being produced in parallel. Parallel processing carried out by multiple elves means more work happens at the same time.

Elves working in parallel can also employ concurrency. One elf can still tackle only one task at a time, so it’s most efficient to have multiple elves concurrently producing presents.

Of course, if Santa’s workshop has, say, two-and-a-half million elves, each elf would only need to finish a maximum of one present per day. In this case, working sequentially doesn’t detract from the workshop’s efficiency. There would still be 7,668.26 elves left over to fetch coffee and lunch.

Santa Claus, and threading

After all the elves’ hard work is done, it’s up to Santa Claus to deliver the presents – all 907,208,752 of them.

Santa doesn’t need to make a visit to every kid; just to the one household tree. So how many trees does Santa need to visit? Again with broad generalization, we’ll say that the average number of children per household worldwide is 2.45, based on the year’s predicted fertility rates. That makes 370,289,286.4 houses to visit. Let’s round that up to 370,289,287.

How long does Santa have? The lore says one night, which means one earthly rotation, and thus 24 hours. NORAD confirms.

This means Santa must visit 370,289,287 households in 24 hours (86,400 seconds), at a rate of 4,285.75 households per second, nevermind the time it takes to put presents under the tree and grab a cookie.

Clearly, Santa doesn’t exist in our dimension. This is especially likely given that despite being chubby and plump, he fits down a chimney (with a lit fire, while remaining unhurt) carrying a sack of toys containing presents for all the household’s children. We haven’t even considered the fact that his sleigh carries enough toys for every believing boy and girl around the world, and flies.

Does Santa exist outside our rules of physics? How could one entity manage to travel around the world, delivering packages, in under 24 hours at a rate of 4,285.75 households per second, and still have time for milk and cookies and kissing mommy?

One thing is certain: Santa uses the Internet. No other technology has yet enabled packages to travel quite so far and quite so quickly. Even so, attempting to reach upwards of four thousand households per second is no small task, even with even the best gigabit Internet hookup the North Pole has to offer. How might Santa increase his efficiency?

There’s clearly only one logical conclusion to this mystery: Santa Claus is a multithreaded process.

A single thread

Let’s work outward. Think of a thread as one particular task, or the most granular sequence of instructions that Santa might execute. One thread might execute the task, put present under tree. A thread is a component of a process, in this case, Santa’s process of delivering presents.

If Santa Claus is single-threaded, he, as a process, would only be able to accomplish one task at a time. Since he’s old and a bit forgetful, he probably has a set of instructions for delivering presents, as well as a schedule to abide by. These two things guide Santa’s thread until his process is complete.

Single-threaded Santa Claus might work something like this:

1. Land sleigh at Timmy’s house
2. Get Timmy’s present from sleigh
3. Enter house via chimney
4. Locate Christmas tree
5. Place Timmy’s present under Christmas tree
6. Exit house via chimney
7. Take off in sleigh

Rinse and repeat… another 370,289,286 times.

Multithreading

Multithreaded Santa Claus, by contrast, is the Doctor Manhattan of the North Pole. There’s still only one Santa Claus in the world; however, he has the amazing ability to multiply his consciousness and accomplish multiple instruction sets of tasks simultaneously. These additional task workers, or worker threads, are created and controlled by the main process of Santa delivering presents.

Each worker thread acts independently to complete its instructions. Since they all belong to Santa’s consciousness, they share Santa’s memory and know everything that Santa knows, including what planet they’re running around on, and where to get the presents from.

With this shared knowledge, each thread is able to execute its set of instructions in parallel with the other threads. This multithreaded parallelism makes the one and only Santa Claus as efficient as possible.

If an average present delivery run takes an hour, Santa need only spawn 4,286 worker threads. With each making one delivery trip per hour, Santa will have completed all 370,289,287 trips by the end of the night.

Of course, in theory, Santa could even spawn 370,289,287 worker threads, each flying to one household to deliver presents for all the children in it! That would make Santa’s process extremely efficient, and also explain how he manages to consume all those milk-dunked cookies without getting full. 🥛🍪🍪🍪

An efficient and merry multithreaded Christmas

Thanks to modern computing, we now finally understand how Santa Claus manages the seemingly-impossible task of delivering toys to good girls and boys the world-over. From my family to yours, I hope you have a wonderful Christmas. Don’t forget to hang up your stockings on the router shelf.

Of course, none of this explains how reindeer manage to fly.

Several months ago, I started to jot down ideas about what I'd cover in my annual review of what's happened over the course of the past 12 months in the field / industry /promotion of education technology. For the past decade, I've churned out a multi-part series on the dominant trends and narratives. Typically this project has taken all of November and all of December to write, based on all the Friday "week-in-reviews" that I’d written over the course of the year. But for the past year or so, I've turned my attention away from the day-to-day news as I've researched and written a book on teaching machines and the history of personalization and behaviorism.

So, I'm not sure I have a lot to say about what happened, what stories were told in 2019. I mean, I always have something to say, but this year, it’s not the well-researched, well-linked series that I've published in the past.

What I have noticed in 2019 from my perch of not-paying-full-attention would probably include these broad trends and narratives: the tangled business prospects of the ed-tech acronym market (the LMS, the OPM, the MOOC); the heightened (and inequitable) surveillance of students (and staff), increasingly justified as preventing school shootings; the fomentation of fears about the surveillance of Chinese tech companies and the Chinese government, rather than a recognition that American companies -- and surely the US education system itself -- has long perpetuated its own surveillance practices; and the last gasp of the (white, male) ed-tech/ed-reform evangelism, whose adherents seem quite angry that their bland hype machine is no longer uncritically lauded by a world that is becoming increasingly concerned about the biases and dangers of digital technologies.

Of course, the latter is just wishful thinking. And even if the wish comes true, the damage of ed-tech evangelism -- particularly the evangelism of trade publications that have spent the past decade repeating nothing but industry spin -- is already done. Even if these publications fade away, the breathless stories about the possibilities of brainwave-reading mindfulness headbands and "mind-reading robot tutors in the sky" continue to be told. Dangerous practices and products are normalized. Stories about the inevitable ed-tech revolution continue to be influential and persuasive, particularly among administrators and politicians and those who can spout the key points of the latest airport-bookstore business-section best-seller at their cocktail parties and in their wine caves. The ed-tech happy hour.

(Deliberate) Misinformation -- about what ed-tech can do, about the problems it will solve, about what sort of circumstances students and schools and society are now facing, about what sort of future new technologies will necessarily give us -- is picked up and wielded by far too many education leaders and decision-makers. That's what a decade of ed-tech social media and PR have wrought: hashtag gurus and fake news.

My hope for the new decade is we can keep the lies out of their mouths. Because it is from their mouths that these lies form the basis for policies and expenditures that have proven to be deeply damaging to education -- public education in particular. But we can't do so if we keep cultivating an ed-tech (information) ecosystem that fosters inaccuracies and inequalities.

(Coming soon: the Hack Education Ed-Tech Decade in Review)

Writing the Hack Education end-of-year series has always reminded me of how very short our memories seem to be. By December, we’ve forgotten what happened in January or June. And we’ve certainly forgotten what happened a year ago, two years ago, a decade ago. Or at least, that’s one way I can rationalize how someone like Chris Whittle can get such a glowing profile in The Washington Post this year for his latest entrepreneurial endeavor.

There are rarely consequences for the people — men mostly — in ed-tech who’ve demonstrated that they are dangerous or dishonest. Their reputations get laundered; their transgressions erased. Joi Ito, Nicholas Negroponte, the MIT Media Lab — who will remember their connections to convicted sex trafficker Jeffrey Epstein? Who will hold them accountable? How soon ’til they’re back in the game with a new story, a new startup?

One of the problems with a lot of ed-tech journalism, I’d argue, is that it is has not been particularly interested in accountability. Some of that, thankfully is changing. Trade publications have been far less committed to explaining what ed-tech is or does or was and far more committed to proselytizing what it might be or might do — all good, all positive of course. Far too many articles — and this is surely what its venture capitalist and philanthropist backers hope — have not reflected the landscape but have tried instead to shape it. That’s why stories about the golly-gee-whiz prospects of learning to code, game-based learning, social emotional learning, artificial intelligence, blockchain transcripts, and tutoring — by chatbots or by gig workers — still fill the pages of these publications. It’s not that these things are necessarily trends; it’s that certain folks very much hope they will be.

And so that we don’t forget and so that we can hold some of these people and companies accountable, here is a list of some of what did actually happen this year, jotted down for my own memory mostly and so I can see the full arc of the decade’s storytelling:

• Silicon Valley darling AltSchool “called it quits” after raising almost $180 million and promising to personalize education through surveillance
• WeWork closed its K-12 school this year. (It also had a failed IPO and ousted its co-founder.) People with knowledge of the school described it as “run like the company of WeWork, subject to constant changes or ‘disruption,’ sometimes without full consideration for the children these changes impact.” Good work, Adam and Rebekah
• Pearson sold its US K-12 curriculum business for $250 million
• Pearson said it would shift its strategy and make all its textbooks “digital first” (which you just know is going to cost students more)
• Cengage and McGraw-Hill announced a merger that would make the new company the second largest textbook publisher in the US. (The deal still faces opposition)
• David Wiley announced that this year’s OpenEd Conference would be the last
• “Operation Varsity Blues.” News of the college admissions scandal, in which 50+ people including celebrities like Felicity Huffman and Lori Loughlin were involved in a criminal conspiracy to influence undergraduate admissions, broke in March. And the saga — sentencing, jail time, probation, and so on — has played out all year. It’s not an ed-tech story per se. But of course these sorts of things always prompt some bright-eyed entrepreneur to argue that “technology will fix it!” — in this case, the blockchain
• The College Board planned to add an “adversity score” to the SAT, ostensibly to help college admissions officials recognize students’ socio-economic privilege, but had to go back to the drawing board after pushback on the idea
• Instructure announced it would be acquired by private equity firm Thomas Bravo — although it’s not clear that the deal will actually go through. (This year, Instructure also acquired Portfolium and MasteryConnect)
• PowerSchool acquired the K-12 LMS Schoology
• Amazon got into the lesson-plans-for-sale business, launching a program called Ignite (not to be confused with its controversial lesson-plans-for-free business, Inspire)
• Carnegie Mellon announced it would open source its digital learning software. Something about "learning engineers"
• 2U did not have such a great year. The OPM company acquired the coding bootcamp Trilogy Education in April, but its stocks fell sharply when it had to adjust its growth projections for 2019
• Udacity got a new CEO
• Coursera raised $109 million in venture company because investors are cray
• ASU ends its MOOC experiment, Global Freshman Academy
• “Mind reading robo tutor in the sky” company Knewton was acquired by Wiley for $17 million — LOL — having raised over $180 million. LOL LOL LOL
• Emily Tate wrote in July about what happens when an online tutor for VIPKid witnesses child abuse. Spoiler alert: nothing
• There were far too many data breaches for me to list here — check out the K-12 Cyber Incident Map — but I will note the one at Pearson that exposed the data of some 13,000 school and university accounts (each of those potentially exposing thousands of students’ information)
• Amazon was sued for Alexa’s recording of children’s voices without parents’ consent
• Google was fined $170 million for violating children’s privacy on YouTube
• Google got into the anti-plagiarism business
• Plagiarism detector TurnItIn was acquired for $1.75 billion (which certainly explains why Google got into the anti-plagiarism business, eh?)
• “Flawed Algorithms Are Grading Millions of Students’ Essays,” Motherboard reported in August
• The University of Alaska system braced for campus closures and layoffs after its operating budget was slashed by 41%. (Vultures, offering online alternatives, swooped in.)
• Maker Media abruptly shut down operations and laid off its staff, but Edsurge — whose investors include Maker Media founder Dale Dougherty — insists that this will have no effect on the “maker movement” or on the Maker Education Initiative
• ISTE announced it would acquire the ed-tech trade publication Edsurge

So much innovation and “edsurgency.” And if we’re not careful, if we do not hold these entrepreneurs and barkers and politicians accountable, if we do not remember their failures and falsehoods, then we will find that all this will just repeat itself on into the next decade.

Matthew Laird @matthewlaird
Doan's Berrytime Raspberry Kolsch is really really good.

Roland Tanglao 猪肉面 @rtanglao
Happy Holidays to that happy poet in a cottage in Japan!: rolandmicroblog.com/2019/12/24/hap…

When your car’s tire-pressure warning light comes on, or if you get a flat tire, a good portable air compressor can top up your tires and help you get back on the road quickly and easily without a detour to a gas station. After researching more than 100 models and testing 36, we recommend the Viair 77P as the best tire inflator for most drivers. It’s one of the quickest and quietest models we’ve tested, its pressure gauge is accurate and simple to read, it’s compact and easy to stow in a vehicle, and it has a sturdy metal body that belies its affordable price.

mkalus shared this story from mrjamesob on Twitter.

---

Twitter is banning animated PNG files, also known as APNGs, from its platform following an attack on the Epilepsy Foundation’s account.

An attacker sent out animated images from the foundation’s account, which could have potentially caused seizures in photosensitive people. The animated images included strobing light effects.

We want everyone to have a safe experience on Twitter.

APNGs were fun, but they don’t respect autoplay settings, so we’re removing the ability to add them to Tweets. This is for the safety of people with sensitivity to motion and flashing imagery, including those with epilepsy. https://t.co/Suogtrop1u

— Twitter Accessibility (@TwitterA11y) December 23, 2019

The social media giant found a bug that allowed users to bypass the autoplay settings and add several animated images in a single tweet.

Existing tweets that have APNG images won’t be deleted from Twitter, but going forward only GIFs will be allowed. GIFs are allowed because they follow the autoplay settings and won’t play if someone has turned this setting off.

Source: Twitter Via: The Verge

The post Twitter bans PNG files following attack on Epilepsy Foundation’s account appeared first on MobileSyrup.

My autistic son made me see the season anew – a Christmas tree is not a fish!:

I remember on the way home from nursery one December day, my son pointed at a small plastic fish in a charity-shop window. In the next window, I showed him a Christmas tree, and, in a phase when he was obsessed with defining things negatively, according to his own idiosyncratic system of what they were not, he replied: “It’s not a fish!” For at least two more years, every time we saw a Christmas tree one of us would say: “It’s not a fish!” and fall about laughing. I know, the levels of you-had-to-be-there are off the scale, but this is the way with autism. It’s hard to explain, and highly specific. The point is, even with all the challenges – and there are many – if you just open the door of your mind to autism, the world is reborn. And it’s incredible.

Exactly.

While Sony has largely exited the phone manufacturing business, the Japanese tech giant has remained in the smartphone ecosystem in a significant way by producing image sensors for other companies.

However, the company says it’s struggling to keep up with demand.

In an interview with Bloomberg, Sony semiconductor head Terushi Shimizu noted that the company’s image sensor factories are working constantly through the holidays for the second consecutive year. Even with that extra work, though, Shimizu said production is falling behind.

“Judging by the way things are going, even after all that investment in expanding capacity, it might still not be enough,” Shimizu told Bloomberg. “We are having to apologize to customers because we just can’t make enough.”

It’s unclear how Sony might address the production concerns. However, the company has a major incentive to resolve the issue. In its Q2 2019 earnings report earlier this year, Sony said it’s targeting 60 percent of its revenue to come from its image sensor business by March 2026.

Source: Bloomberg

The post Sony says it ‘just can’t make enough’ image sensors to keep up with phone demand appeared first on MobileSyrup.

A security researcher says he was able to exploit a bug in Twitter’s Android app and matched 17 million phone numbers to users’ accounts.

He found that it was possible to upload full lists of generated phone numbers through the platform’s contacts upload feature. He explained to TechCrunch that if you upload your phone number, it fetches user data.

The researcher generated more than two billion phones numbers and then uploaded them to Twitter through the Android app.

He was able to match records to users located in France, Germany, Iran, Greece, Turkey, Israel and Armenia. Twitter then blocked the flaw on December 20th.

In one of the cases, he was able to identify an Israeli politician through their matched phone number. Although he did not notify Twitter, he says he warned high-profile Twitter users to warn them.

Source: TechCrunch 

The post Twitter for Android bug matched 17 million phone numbers to accounts appeared first on MobileSyrup.

Jules Evans, OneZero, Dec 24, 2019

The funny thing, for me, is that I'm actually comfortable with all this chaos. The whole internet thing doesn't bother me. Why not? Here's how Jules Evans characterizes the problem: "Social media has also totally transformed our politics, in a decade. It’s ripped apart our sense of trust in media and politicians.... The trick mirror turns everything into a performance, so political debate becomes wrestling.... We have reached a stage of collective consciousness, through globalization and the internet, and we can’t quite handle it."

So why am I OK with all this? Maybe it's because I grew up in a small town, maybe it's because I was writing from an early age, maybe it's because I never trusted media and politicians, having seen from the inside how they fabricate reality, maybe it's because I read a lot of science fiction and to me all this is normal. But my advice is: don't lose your head over all this, it's not nearly as bad as you are being led to believe, and it serves only to obscure the real problems - the abuse of power, authority and wealth - that we have always had.

Web: [Direct Link] [This Post]

As the year winds down, I am reflecting on progress made in the global movement to build great cities and communities, as well as new challenges. 

In the last few years, cities are facing the effects of climate change and housing affordability more acutely than ever before. But most change starts locally, and cities are tackling these issues in new and exciting ways. I am particularly impressed with how local governments and citizens are working to build family-friendly cities, as featured in my series, Playmakers Around the World.

Unfortunately, new challenges have arisen alongside current ones. Particularly how we treat each other on the roads and in our streets. The demand for SUVs and large trucks has grown in North America, along with increasing pedestrian and cyclist fatalities. This issue is only getting worse, and I hope to write more about it in 2020. We also are seeing a rise in racism, intolerance for refugees, and disconnection and isolation in cities.

My hope is that urbanists and citizens continue working to show larger institutions how to tackle these complex issues by starting small and making an impact locally. 

As we embark on a new year and a new decade, I hope you take time over the holidays to connect with your loved ones and your community.

Merry Christmas, Happy Holidays, and Happy New Year!

If you want to catch up on some urbanist reading, here are the most popular articles posted on This City Life in 2019:

1.) Playmakers Around The World: Tim Gill wants kids to be free to take risks

2.) Out Today! A Comprehensive Book on Building Child-Friendly Cities

3.) Is Vancouver a family-friendly city?

4.) New Series! Playmakers Around the World

5.) Playmakers Around the World: Başak Incekara creates city building workshops for children in Turkey

Photo: My son at Vancouver’s climate strike protest this year.

Creating a Slack client for Windows 3.11 is an amazing accomplishment - even building a 16bit app for Windows requires an older version (Windows 10 does have a 32bit version, but finding a copy is tough). The walkthrough is detailed enough to use a tutorial, and the christmas break starts today...

Every day, millions of people rely on independent websites that are mostly created by regular people, weren't designed as mobile apps, connect deeply to culture, and aren't run by the giant tech companies. These are a vision of not just what the web once was, but what it can be again.

---

Think of every time you've sent someone a Snopes link to debunk a spurious story that's been shared online. The casual way we might look up the credits for an album on Discogs, or for a movie on IMDB. The absurd details of popular culture captured on TV Tropes and Fandom, linguistic oddities documented on Urban Dictionary, technical questions answered on Stack Overflow, lyrics we quote from Genius, memes on Know Your Meme — all of these are a powerful and valuable record of the world around us, created and captured by millions of ordinary people. And there is, of course, Wikipedia standing astride them all, as perhaps the pinnacle of people-created web references.

Now, these kinds of sites are far from perfect. Each ecosystem of information has too many barriers to creation. Their communities of moderators and contributors are often exclusionary, echoing the gatekeeping of the media and institutions that preceded them. Some of the information on the sites is inaccurate, or skewed.

But even with all their flaws, the existence of dozens of massive, collectively-maintained, curated and organized libraries of communal culture are still something like a miracle of the web. Tellingly, these kinds of sites rarely get launched these days, and the ones that have survived all follow a fairly common set of patterns. They often start as a labor of love from one person, or one small, tightly-knit community. The knowledge or information set that they record is considered obscure or even worthless to outsiders, until it becomes so comprehensive that its collective worth is undeniable.

Their business models have evolved as the internet has evolved, and they tend to start as pretty pure web experiences, that have then had to iterate, often with limited resources, to accommodate the dominance of search engines, the rise of the mobile web, the pervasiveness of social networks, and the societal challenges of organized harassment and targeted misinformation. Through it all, they've grown and adapted, and handled the inevitable community challenges. Many have diversified their business models with everything from memberships and subscriptions to merchandise and events.

But here's the thing: Taken together, these sites are as valuable as any of the giant platforms run by the tech titans.

For as much video as we see on YouTube, as many photos as we browse on Instagram, there is just as much time, attention and energy spent every day on exploring and referencing these deep databases. They don't have fancy filters or complex recommendation algorithms, but they meet a variety of deep human needs around creation and expression and often, they also help people simply do their jobs.

A Web At Risk

At Glitch, we find a tremendous amount of inspiration in the open-ended creativity of these communities, but I think everyone who loves the web finds joy from the seemingly-endless ideas captured on these sites. We just don't think of them as a cohesive whole like we do with the big apps that live behind a button on our phones. We urgently need to pay attention to this cohort of sites, though, because their position is precarious. Just as we've seen with Google introducing its algorithms and devastating the first generation of the social web, with the rise of native mobile apps and social networks like Facebook and Instagram limiting our links and locking us into their walled gardens, these people-made web communities are deeply vulnerable to the whims of the big players. We have to recognize their collective value before they're facing an existential threat.

If we're going to build a new web, and a new internet, that respects our privacy and security, that doesn't amplify abuse and harassment and misinformation, we're going to need to imagine models of experiences and communities that could provide a better alternative. There's not going to be a "Facebook killer". But there could simply be lots of other sites, that focus on a different, more constructive and generative, set of goals.

The good news is, we don't have to imagine what that more human, more expressive, more valuable web could look like. We just have to pay attention to the fact that we visit it every day.

I released Datasette 0.33 yesterday. The release represents an accumulation of small changes and features since Datasette 0.32 back in November. Duplicating the release notes:

• rowid is now included in dropdown menus for filtering tables (#636)
• Columns are now only suggested for faceting if they have at least one value with more than one record (#638)
• Queries with no results now display “0 results” (#637)
• Improved documentation for the --static option (#641)
• asyncio task information is now included on the /-/threads debug page
• Bumped Uvicorn dependency 0.11
• You can now use --port 0 to listen on an available port
• New template_debug setting for debugging templates, e.g. https://latest.datasette.io/fixtures/roadsideattractions?context=1 (#654)

The last three items deserve extra explanation.

Port 0

I run a lot of Datasette instances on my laptop - I often have 5 or 6 running representing different projects or experiments.

Eventually this means I start to hit port conflicts. I'll run datasette mydb.db -p 8005 and get an error because port 8005 is already in use by something else.

I asked on Twitter if there were any downsides to having Datasette automatically pick an unused port... and @davids and @dominicrodger pointed out that port 0 is a Unix convention for "pick a port for me".

It turned out Datasette already supported this! If you ran datasette --port=0 the underlying Uvicorn server would indeed pick an unused port and start serving on it.

There was just one catch: the URL output to the console would show port 0, so it wasn't clear which port to actually visit. This was a bug in Uvicorn.

Open source to the rescue... I filed an issue with Uvicorn and then constructed a pull request. Tom Christie merged it five hours later and shipped a release. Datasette 0.33 depends on that updated version of Uvicorn, so --port=0 now works as expected!

template_debug

I'm always looking for new ways to add debugging and development tools to Datasette. I added support for ?_trace=1 to see executed SQL queries back in May for example.

While thinking through a refactoring of Datasette's template rendering to better support plugins, I realized there was an opportunity for better debugging around templates.

Datasette supports custom templates, which means template authors need a clear way to see what variables are available to their template logic.

I tried adding a ?_context=1 parameter to dump out the template context, and it instantly helped me better understand what was going on.

There was just one problem: what about secret information? By default Datasette doesn't include any secrets (signing keys, API tokens etc) in its template context - but Datasette plugins are allowed to add things to the context, and I can't guarantee that a future plugin won't add something that shouldn't be exposed by the ?_context=1 debugging parameter.

The solution was to have context debugging only available if the new template_debug configuration setting is turned on:

$ datasette --memory --config template_debug:1

I've set up the latest.datasette.io demo to run with this setting, so you can now view the context for any page on that site, like so:

• https://latest.datasette.io/?_context=1
• https://latest.datasette.io/fixtures?_context=1
• https://latest.datasette.io/fixtures/roadside_attractions?_context=1

Niche Museums

Natalie and I are in the UK this week. We spent a few days in London and managed to get in a few niche museums, including the first four on this week's new listings on Niche Museums:

• Mail Rail in London
• Museum of Funeral History in London
• London Mithraeum in London
• Bank of England Museum in London
• Reserva Cerro Ancón in Panama City
• Buckfast Abbey in Devon
• Dennis Severs’ House in London

I also made a small fix to the site's Atom feed: it now respects newlines in the entries, by rendering the description using Markdown.

Here's the implementation. Since the Atom feed is defined by a SQL query (using datasette-atom) I took advantage of Datasette's ability to load custom plugins from a per-site plugins/ directory and added a custom SQLite SQL function for rendering markdown, then added that function call to the query that defines the feed.

nick shackleton-jones, aconventional, Dec 23, 2019

I am sensitive to this and see it all the time: "Keep an eye out for this social architecture. You will start to notice it several times a day: the instinctive (infantile) deference to some beardy bloke with academic titles. Bear in mind that they may be part of the problem."

Web: [Direct Link] [This Post]

A Gini coefficient is a simple measure of income equality. A coefficient of 0 indicates perfect equality (everyone has the same amount), while a coefficient of 1 indicates perfect inequality (one person has everything while everyone else has nothing). Highly unequal countries like South Africa have an (official) Gini coefficient around 0.65, while less unequal countries like Canada have a coefficient around 0.35.

Gini coefficients can be applied to other things, such as the number of commits made by each contributor to a Git repository. I was curious: how (un)equal are contributions to software projects? And are contributions to lessons more or less unequal? To find out, I crawled the histories of five Software Carpentry lessons and five widely-used numerical Python libraries. The results are undoubtedly wrong (I’m not trying to merge records for people with multiple email addresses, for example), but I was surprised:

1. that the coefficients are so similar,
2. and that they are so unequal when measuring the number of commits,
3. but that they are so much more equal when counting insertions minus deletions (i.e., number of lines contributed),
4. and that there is so much more variability when counting lines. (I don’t know if taking the ratio of Gini coefficients is meaningful, but it gives an idea of the scale of the disparities.)

I have a stack of background reading to do, but it’s shaping up to be a fun little project. If you know of other simple ways to measure the evenness of contributorship, or if you can explain why measuring by contributions and by lines gives such different answers, I’d enjoy hearing from you.

Project	Gini (Commits)	Gini (Lines)	Ratio
Git Lesson	0.7867	0.0306	25.69
Python Lesson	0.8250	0.0982	8.40
R Lesson	0.7899	0.0299	26.42
Shell Lesson	0.7955	0.0362	21.96
SQL Lesson	0.8101	0.0462	17.52
Lessons Average	0.8014	0.0482	20.00
NumPy	0.9097	0.0105	86.89
Pandas	0.8743	0.0306	28.55
Scikit-Image	0.8547	0.2495	3.42
Scikit-Learn	0.8836	0.0492	179.51
SciPy	0.8821	0.0047	185.94
Projects Average	0.8809	0.0601	96.86

The right jolly old elves over at Alteryx created a “Santalytics” challenge back in 2016 to see if their community members could help Santa deliver presents to kids all across the globe.

They posted data for four challenges along with solutions and I’ve made a git repo & RStudio project with the challenges and solves for two of the four (I was going to try to have all four done but December has been a cruel master when it comes to allowing for free time).

Most of tasks are pretty straightforward and range from basic joining and grouping to some spatial optimizations (but all very do-able with a little elbow grease). The featured image at the top of the blog is one solution to finding “distribution hubs” for all the presents.

You can find the starter Rmd and data files over at your favorite social coding site:

• My self-hosted Gitea
• Sourcehut
• Bitbucket
• GitLab
• GitUgh

FIN

Give Santa a hand and blog your approach to solving each challenge!

All four of our offspring are home for Christmas this year (w00t!!!) so this is likely the last blog post of 2019. Many blessings to all as you celebrate this time of year and catch y’all in 2020!

The PureOS team are proud to have created a brand new stable release of PureOS and delivered it to lots of users in 2019, so much in fact we had to add download mirrors to keep up with the demand. If that wasn’t enough we also created a new rolling release and are very close to finishing reproducible builds. Read the rest of the PureOS team’s 2019 year in review to find out more and discover what we have planned for 2020!

Stable and Testing Releases

Our new stable release is called Amber and is designed to provide the security of coreboot with the stability of a well tested distro. This new version of PureOS retains its status as a Free Software Foundation “free distro” and we’ve added a sibling called Byzantium which is a rolling release.

We’ve improved our release cadence so that we can do in-depth testing with some regularity. Byzantium is now being updated directly from the mothership, Debian testing, for those who want the latest and greatest. We’ve been pleasantly surprised by the adoption of the two distros and I am looking forward to announce the availability of Byzantium ISO images early in the New Year. Some of you might have noticed that while it is possible to upgrade Amber to Byzantium (details in our forum) it’s a big change with lots of new packages. If you’re not feeling adventurous, no need to worry, we’re going to continue to update Amber throughout 2020 with Amber updates and security fixes.

Our new rolling release was made possible by the flexibility of the Laniakea tool which we’ve invested significant resources in. Purism proudly sponsors the project and is also contributing to its broader adoption in the community as a way of building out the ecosystem of Free Software itself. Laniakea holds the logic to create new images as well as update packages which has allowed us to implement continuous delivery. Lanikea has allowed us to use Debian binary packages without having to rebuild each one by hand. Without Laniakea there simply would be too much work for the team. To top it off Matthias Klumpp created a tool that clones the entire Debian build process and let’s you maintain a large Debian derivative with relatively little effort. It is currently powering the Librem 5 and x86 builds distributions.

Mirrors

The PureOS traffic has grown considerably this year and we set up mirrors so that it is more easily accessible. PureOS now has mirrors on the US West Coast, Europe, and its own original instance. We hope this will streamline your software updates.

Librem Software Stack

On many distros the software stack begins at the kernel but our Librem software stack starts at the BIOS level, specifically coreboot. This year we’ve supported the Pureboot team in packaging coreboot, flashrom, and the TPM updating code. These tools are now in the PureOS repos and have successfully been used to manipulate the TPM on the Librem 13 and 15 and  allow our users to take ownership of that chip. As heads matures and is integrated into Pureboot we’ll have more ways to protect your computer.

Aside from packaging code to change TPM secrets and flashrom, we’ve also packaged some udev instructions that work with the Librem Key. We always try to push patches up to Debian first but sometimes our patches make it into our distro more quickly that upstream. As we continue to package things for PureOS that will likely remain the case.

Reproducible Builds

Our work on reproducible builds has progressed and we have reduced our diff down to roughly 3 Megabytes in a multi-gigabyte distro. Once we’re doing consistent builds of Byzantium we’re confident that we can close that gap and reproducibly build for PureOS and Pureboot. Coreboot is already reproducible and much of the code in our repos is also reproducible. Putting all that work together into a complete reproducible ISO is tantalizingly close.

PureOS in 2020

Next year we will be including a Quality Assurance tool (hint: OpenQA), continue our work on reproducible builds and the PureOS store. We are continuing our convergence work on PureOS to optimize the experience for all our current and future devices, whatever form they take.

Read more about PureOS on our blog and install PureOS on your computer or a virtual machine.

The post 2019 Year in Review: PureOS appeared first on Purism.