Shared posts

18 Dec 04:56

No you should not seriously consider whether we are living in a simulation

I’m reading Superintelligence, a surprisingly dull book given its topic is, well, the possibility of a rampant superintelligent AI destroying civilization and what we might do to prevent that sort of thing. Along the way, the author speculates about how a caged hyperintelligent AI (not joking here) might incorporate reasoning about the the simulation hypothesis into its plans to escape confinement.

The answer to the question of “how do we know we aren’t living in a simulation” is similar to “how do you know the universe wasn’t snapped into existence 5000 years ago?”, “how do you know you aren’t dreaming right now?” or “how do you know you aren’t just a brain in a vat?” On some level, we can’t really know these things with certainty, but we don’t seriously consider these possibilities because we have simpler models that explain reality as well or better.

However, without any judgement, I report that some people take this idea of reality being a simulation pretty seriously. Elon Musk apparently believes we are likely living in a simulation and is rumored to have a team of people trying to figure out a way of breaking out of the simulation. I’ll admit: the first time I read that second article I had to double check that I was not reading The Onion.

Anyway, I want to spend the rest of this post picking apart the simulation hypothesis, but first, let’s hear the argument out! Enter Nick Bostrom:

Many works of science fiction as well as some forecasts by serious technologists and futurologists predict that enormous amounts of computing power will be available in the future. Let us suppose for a moment that these predictions are correct. One thing that later generations might do with their super-powerful computers is run detailed simulations of their forebears or of people like their forebears. Because their computers would be so powerful, they could run a great many such simulations. Suppose that these simulated people are conscious (as they would be if the simulations were sufficiently fine-grained and if a certain quite widely accepted position in the philosophy of mind is correct). Then it could be the case that the vast majority of minds like ours do not belong to the original race but rather to people simulated by the advanced descendants of an original race.

It is then possible to argue that, if this were the case, we would be rational to think that we are likely among the simulated minds rather than among the original biological ones.

Therefore, if we don’t think that we are currently living in a computer simulation, we are not entitled to believe that we will have descendants who will run lots of such simulations of their forebears.

Whoa there! First problem: one can’t just extrapolate from a few decades of computing trends (where computers have indeed become exponentially more efficient) arbitrarily far into the future, without considering that there are theoretical, physical, and practical limits to computing power, even for an advanced civilization. To put things in perspective, we cannot today build computers powerful enough to accurately simulate everything happening in a thimble full of blood; what makes us think we can simulate a planet, a solar system, or a universe?

Something underappreciated here is that the computation happening in the wild, including the computation done by our universe’s laws of physics, tends to be irreducible if we simulate it at sufficient fidelity. (And by all available evidence, our reality is “computed” at high fidelity down to quantum scales, not some vastly simplified approximation like what is done in video games.) It’s only when we consider much simpler idealizations of reality that we’re able to simulate or predict reality using more limited amounts of compute. Stephen Wolfram been beating this drum for a while, though I like to just view it as a consequence of Rice’s theorem or the halting problem: in general, you can’t fast forward the program to know what it does, you have to run it, and that requires energy, space, and time and eventually reaches physical limits. Though much is unknown, it is very possible that our universe cannot be simulated much more efficiently than by building an effectively universe-sized computer. This does not bode well for hypothetical future civilizations planning to simulate our current reality within our current universe.

A more fundamental issue with the simulation hypothesis is it requires speculating about an “outer universe” capable of simulating our current reality. Notice we can’t really speculate about this because nothing we can observe in our current reality places any constraints on such an outer universe. Is it similar size to our own? With similar computing power? Do its laws of physics work similarly? Anything goes here and is consistent with our observations of reality. So we can conjure up the idea that there is an outer reality but can’t actually investigate or test it.

A related objection is we have no idea what sort of activities civilizations in a hypothetical outer universe would engage in, so even if computers powerful enough to simulate our current universe are possible in some outer universe, will creating simulations of our currrent reality be something that is done?

Together, these points establish the simulation hypothesis as having roughly the same logical status as debating how many angels can fit on the head of a pin or what would happen if Superman and Cyclops fired their laser eyes at each other: it is speculation about the properties of things that can’t be disproven or shown to exist.

I find it odd that these presumed creators of the simulation we find ourselves in have left no trace that it is in fact a simulation and everything we observe seems indicative that reality is what it is. Give us a sign, dammit! The arguments one sometimes hears for this lack of evidence (“only when we reach a certain stage of technical advancement will we be able to detect the simulation”) remind me of other unfalsifiable explanations that conspiracy theorists invent: one can always make up reasons for why the thing you believe doesn’t have any observable evidence (or why observations contradict your belief).

But forget all that. We have no need to resort to outer universes simulating our current universe to explain reality, so why introduce that? Let’s all just stop there and get back to (what I sure hope is) reality.

18 Dec 04:55

Firefox Announces New Partner in Delivering Private and Secure DNS Services to Users

by Mozilla

NextDNS Joins Firefox’s Trusted Recursive Resolver Program Committing to Data Retention and Transparency Requirements that Respect User Privacy

Firefox announced a new partnership with NextDNS to provide Firefox users with private and secure encrypted Domain Name System (DNS) services through its Trusted Recursive Resolver Program. The company has committed to putting user privacy first in efforts to modernize DNS.

For more than 30 years, DNS has served as a key mechanism for accessing sites and services on the web. DNS is the Internet’s directory. It translates names we know like ​www.firefox.com​ to numeric Internet addresses that a computer understands. Almost every activity on the Internet begins with a DNS request.

The Domain Name System (DNS) is one of the oldest parts of internet architecture, and remains largely untouched by efforts to make the web safer and more private. Malicious actors can spy on or tamper with users’ browsing activity and DNS providers, including internet service providers (ISPs), can collect and monetize a user’s browsing activity.

Over the last two years, Firefox, in partnership with other industry stakeholders, has been working to develop, standardize, and deploy DNS over HTTPs (DoH). DoH aims to protect that same browsing activity from interception, manipulation, and collection in the middle of the network.

But encrypting DNS data with DoH is only the first step. Requiring the companies handling this data have rules in place – like the ones outlined in the TRR program – ensures that the access to that data is not being abused, is a necessary second.

“For most users, it’s very hard to know where their DNS requests go and what the resolver is doing with them.” said Eric Rescorla, Firefox CTO. “Firefox’s Trusted Recursive Resolver program allows Mozilla to negotiate with providers on your behalf and require that they have strong privacy policies before handling your DNS data. We’re excited to have NextDNS partner with us in our work to put people back in control of their data and privacy online.”

Our trusted recursive resolver program aims to standardize requirements for three areas: limiting data collection and retention from the resolver, ensuring transparency for any data retention that does occur, and limiting any potential use of the resolver to block access or modify content. By marrying the right technology – DoH – and strict operational requirements for those implementing it, we are improving user privacy by default by finding good partners, establishing legal agreements that put privacy first, and shipping a product we believe is best by default.

“We applaud Mozilla’s leading stance on privacy and we are proud to partner with them to offer the choice of a modern, fast and no-logs trusted DNS resolver to the Firefox community,” said Romain Cointepas, Co-founder, NextDNS.

NextDNS launched in March 2019 providing a fully customizable, modern and secure DNS resolver. Since then the company has worked to continue to improve the service and has released DNS-over-HTTPS apps for all major platforms (iOS, Android, macOS, Windows, Linux) and routers.

NextDNS is the latest resolver to join the TRR program. Cloudflare joined the program in 2018.

“Cloudflare joined the program back in 2018 with its launch of 1.1.1.1, the public DNS resolver built around the principle of privacy-first. We believe that giving consumers the ability to choose the fastest, most privacy-respecting DNS is a win-win. It’s good for them and it’s good for the Internet,” said Matthew Prince, co-founder & CEO, Cloudflare. “We hope more ISPs and DNS providers will follow this lead so we can finally encrypt one of the Internet’s most important protocols.”

While the TRR program, and its privacy first policies, is specific to Firefox’s implementation of DoH, we believe that all internet users are entitled to these protections. As the work to implement DoH continues, we look forward to bringing more partners into the TRR program who are committed to bringing the DNS system into the 21st century with the privacy and security protections users deserve and hope the rest of the industry follows suit.

The post Firefox Announces New Partner in Delivering Private and Secure DNS Services to Users appeared first on The Mozilla Blog.

18 Dec 04:55

This may happen to you. No, it probably will.

by Volker Weber
Hackers hit Norsk Hydro with ransomware. The company responded with transparency.

Make your plans now.

More >

18 Dec 04:55

Surface Headphones :: Zweite Chance

by Volker Weber

8e7c03a0706bafef031200efd22d25f1

Im letzten Winter habe ich die damals neuen Surface Headphones verrissen. Zu teuer, zu schlecht, unbequem. Aber wie das so ist: Preise ändern sich*, Software bekommt Updates, selbst mein Kopf ist schmaler geworden. Deshalb bekommen die Headphones eine zweite Chance. Ich mag sehr viel an diesen Headphones, etwa die Bedienung und dass sie auch zum Telefonieren taugen. Und das will ich mir alles mit mehr Geduld anschauen. Beim ersten Test war ich unter zu starkem zeitlichen Druck. Nun, da sich die erste Review-Welle verlaufen hat, wird mein Testzeitraum nicht in Tagen sondern in Monaten gemessen. Ich freue mich drauf.

München, 7. Februar 2019. Mit den Surface Headphones macht Microsoft seine ersten Over-Ear-Kopfhörer ab dem 7. März 2019 auf dem deutschen Markt verfügbar. Erhältlich werden die komfortablen Kopfhörer im Microsoft Store sowie über ausgewählte Surface Fachhändler zu einem Preis von 379,99 Euro (UVP; inkl. MwSt.). Geschäftskunden beziehen die Kopfhörer über einen autorisierten Reseller.

Heute reden wir nicht mehr von dicken 380 sondern schlanken 219 Euro. Und es gibt entscheidende Softwareupdates.

Und sehr schön: Surface Headphones können mit bis zu acht Geräten gepairt und zwei gleichzeitig verbunden sein.

Annotation 2019-12-17 155035 Annotation 2019-12-17 154644 Annotation 2019-12-17 154907

Cortana habe ich abgeschaltet, Siri erreiche ich durch lange Berührung einer der beiden Ohrmuscheln. Kein aptX oder AAC, diese Einschränkung bleibt weiterhin. Telefonieren funktioniert 1a, aber mir fehlt das Monitorsignal des PLT8200. Der spielt die eigene Sprache zurück in die Hörmuschel, damit man sich selbst besser hört. Ich helfe mir, in dem ich ANC runterdrehe, wenn ich telefoniere. Diese Bedienung durch die rotierenden Ringe (links ANC, recht Lautstärke) ist das beste an diesem Headset.

18 Dec 04:55

brucesterling: *Been there, got the...

18 Dec 04:54

Physically disabled Apple employee sues tech giant for dismissal in Ontario

by Aisha Malik
Apple

A 33-year-old Ontario man is suing Apple for lost wages and damages after claiming that he was dismissed from working at an Apple store for requiring a wheelchair.

He worked as an “Apple Genius” for more than eight years and says that Apple repeatedly refused to work on a plan to accommodate him. The Statement of Claim alleges that he was never given a reason as to why his request for an accommodation plan was denied.

The man says that his health suffered as a result of harassment, bullying and a toxic work environment. The allegations made in the claim need to be proven in court.

He worked at the Apple store in Square One in Mississauga for six years. He then switched to the Apple store in Sherway Gardens in Etobicoke and worked there for two years before he was dismissed in July 2019.

Before starting at the store in Sherway Gardens he was told that the store would be equipped with an appropriate table and automatics door, but this never happened.

The action was filed in the Ontario Superior Court of Justice by a Toronto employment lawyer. The man’s lawyer says that Apple may have failed to comply with the province’s Accessibility for Ontarians with Disabilities Act.

The Act infers that the man should have received an accommodation plan from Apple.

Source: Monkhouse Law 

The post Physically disabled Apple employee sues tech giant for dismissal in Ontario appeared first on MobileSyrup.

18 Dec 04:52

Measuring Software Tools

by Greg Wilson

I think there are four interesting ways to measure a software tool. To make them seem more deserving of serious attention, I’m going to use mathematical notation (subscripts!) to label them:

  1. The median discovery time is how long it takes half of the people who ought to be using the tool to find it.

  2. The median time to proficiency is how long it takes half of the people who stick with the tool to reach the point where they can use it without constantly consulting the manual or feeling frustrated.

  3. The average improvement in efficiency measures the benefits of the tool on tasks the user was already doing. This can either be how much time it saves (if the amount of work stays constant) or how much more work they can do (if the time stays constant).

  4. The increase in reach measures how many new tasks the user can tackle, i.e., how many things they can do with the tool that they couldn’t do before.

Funders care most about #4 because that’s what gets grant recipients on the cover of Nature. Developers mostly focus on #3, though they often describe what they’re doing in terms of #4: for example, they might describe an online text formatting tool as “revolutionary” when what it’s really doing is a task everyone is already familiar with in a less demanding way.

The world looks very different from the average user’s point of view, though. I’m belatedly starting to appreciate how important #1 is—Sarah Lin recently posted ten quick tips for making stuff findable that I wish I’d thought about ten years ago—but as an educator, I think #2 is the most important factor for most people. If something is so hard to learn that most people give up or never reach the point where they can do things without a lot of online searches and cursing then #3 and #4 are irrelevant. Good lessons are necessary but not sufficient: you can’t document your way out of a usability hole (cough Git cough), but not having decent tutorials pretty much guarantees that only the most obsessive and/or privileged of potential users will last long enough to see a tool’s benefits.

This is why I’m so weary of seeing funding announcements that don’t split money 50/50 between building new things and teaching people how to use them. Academia and the tech industry look down on training as second-class work; both congratulate themselves on how many people accomplish X without asking how many gave up along the way or how many aren’t blogging or tweeting about it because X makes them feel stupid every time they use it and they blame themselves instead of its creators. A shift in funding priorities won’t change that overnight, and might not get funders as many Nature covers to brag about, but it will help a lot of people and make the world a better place.

18 Dec 04:50

These are the stupidest contract terms I saw this year. Can you beat them?

by Josh Bernoff

I recently got the opportunity to run a writing workshop for a huge social media company. That meant I also got the opportunity to see their standard supplier contract, which had some pretty outrageous stuff in it. Freelancers, I challenge you: have you seen contract terms worse than this? Let me be clear about what … Continued

The post These are the stupidest contract terms I saw this year. Can you beat them? appeared first on without bullshit.

18 Dec 04:50

"The two most salient facts of our reality are ecological collapse and income inequality, and the..."

“The two most salient facts of our reality are ecological collapse and income inequality, and...
18 Dec 04:50

Senákw Project Approved by Squamish First Nation in Vancouver

by Sandy James Planner

senakw-squamish-vancouver-kitsilano-december-2019-11

senakw-squamish-vancouver-kitsilano-december-2019-11

There’s some new information on the proposed Senákw project in Vancouver which is on Squamish Nation land near Burrard Street and Vanier Park in Kitsilano. Earlier this year a three billion dollar project was announced at this site which hopes to build 6,000 dwelling units in eleven towers.

This massive project has been ratified by the Squamish Nation in a voting process, and it is intended to be built in an equal venture agreement with Westbank Development Corporation, the same organization  that has built Vancouver House.

As reported in the Vancouver Courier with Frank O’Brien , Peter Mitham  and  Hayley Woodin building 6,000 units in 11 towers “would require buildings of between 55 to 60 storeys, based on comparison with other residential towers proposed but not yet built in Vancouver.”

Squamish Nation Councillor Khelsilem indicated that while the percentage of rental versus strata units had not yet been decided, the project is seen as a long term economic development project. While Westbank will guarantee the loan for the development, and provide any needed equity, the Squamish Nation will be providing their lands.

Leases will run for 120 years, and build out could take ten years. It is intended that  rental units will have a 110 year lease and condos will have a 99 year lease paid up front, with the understanding that the condos turn back to rental units upon lease expiry.

The project will be built in five phases, with the first potentially commencing in 2021.

While the lands are going to be exempt from foreign buyers tax, speculation tax, and rent control, the Squamish Nation will be the governing authority. The Nation can collect property taxes to cover the provision of services and amenities, and will need to make agreements with the City of Vancouver for sewer, water, and other services.

As one of the largest infrastructure projects of its kind in Canada, the structuring and set up of the development will attract national interest. As Khelsilem notes on his twitter page the Nation has “access to federal tools to bring Provincial regulations into force” that could include the Residential Tenancy Act.

The history of these lands and how the Squamish First Nations have been treated is documented in this YouTube video by Douglas Harris, a UBC law professor. The history is horrendous. The section from 25:00 documents some of the specifics.

 

Images: DailyHive PostMedia

18 Dec 04:49

Larry Cudney’s Revenge – The Story of the Vancouver Special

by Gordon Price

A big shout-out to author Jesse Donaldson:

“Land of Destiny: A History of Vancouver Real Estate” is a fun, fascinating book … that more than delivers on its title. His publisher Anvil Press will host a Vancouver launch Dec. 19 at 6 p.m at Resurrection Spirits, free to the public.

Here’s an excerpt from The Tyee: 

Larry Cudney hated architects. In fact, he hated the entire architectural profession. For a time, years earlier, while still a young intern with a local firm, he had harboured dreams of becoming one himself, until a falling-out with the company prevented him from obtaining the certification he needed. …

Working as a draftsman from his cramped office on Main Street and 33rd Avenue, he designed single-family homes (the only buildings a draftsman could legally design), and his work was known for being simple and practical …

… sometime in the mid/late-1960s, Cudney sat down and drafted the plans that would become his legacy. It came to be known as the “Vancouver Special,” and for the next 20 years, it would be the most widely-discussed — and hated — type of housing in town. …

“Those brash new houses with slightly pitched roofs and aluminum balconies (known in the trade as Vancouver Specials), which are now squeezed into lots where once a single house stood in a magnificent garden are here not just to stay, but to increase,” complained the Sun, in 1978.  …Between 1965 and 1985, an estimated 10,000 Vancouver Specials were built, and by 1980, according to a Young Canada Works survey, eleven per cent of Hastings-Sunrise, and five per cent of Marpole were made up of Vancouver Specials. And as more and more were built, the backlash only grew. …

“Right now, to buy a house in the city’s east side, you have to have $20,000 in assets and a $20,000 income,” wrote the Sun’s Mary McAlpine in 1978. “Most young people with children don’t have that sort of money. The people who do are developers who tear down the house and put up Vancouver Specials …

But in the years that followed, attitudes — including city council, and the Sun’s McMartin — began to change. For many lower-income and immigrant families, council later recognized, the Vancouver Special was their only chance for home ownership. In 1987, City Councillor Gordon Price even praised the architectural style as “a tradition of our cultural diversity,” and “worthy of heritage preservation.* …

In 2005, a renovated Vancouver Special was awarded the Lieutenant-Governor’s Award for Innovation in Architecture. …

Privately, Larry Cudney was said to have been proud of the disgust his brainchild had engendered. “Creating a completely tasteless form of housing,” stepdaughter Elizabeth Murphy later opined, “was his revenge on the architect profession with which he was in conflict.”

 

*It’s true!  I remember saying that.  Still do.  But with respect to heritage preservation, I meant only that we should designate an intact original and perhaps try to save a complete block like the one above.  Let the rest evolve or eventually be replaced by higher density ‘missing-middle’ alternatives.  

Vancouver has always been in need of some kind of Vancouver Special.  The two-storey carpenter-built single-family houses along streetcar lines in the 1890s and 1900s were the originals.  Even West End one-bedroom apartments in West End highrises in the 1960s were a form of simple, affordable, mass-produced housing.  So in a different way was the illegal basement suite.  Now it’s the modular house for the otherwise homeless.  But with the high land costs, design controls, heritage preservation, and inflexible zoning, we aren’t likely to see another version anytime soon.

 

17 Dec 04:54

Over Promise and Under Deliver

Nikon's been in a bit of a conundrum for quite some time with mirrorless cameras. By pulling their first effort completely (the Nikon 1/CX series) and then seeming to rush back in with the Z mount, they find themselves playing defense, not offense. …

17 Dec 04:48

A Tale of Two Cities & Two Different City Councillors

by Sandy James Planner

19794898_web_duty1

19794898_web_duty1

Here is a tale of two municipalities and two councillors.

As reported by Aaron Hinks in the Peace Arch News White Rock has a Dogs on the Promenade Task Force, evaluating whether dogs should be allowed on leash on the town’s famous waterfront promenade. White Rock council has approved a six month trial period for dogs to be permitted which ends March 31, 2020.

While there has been some complaints of dogs disturbing wildlife and complaints of unleashed dogs and defecation, there has been no complaint about canine aggression or biting.

And on December 12  White Rock received a morning complaint about dog feces being left along the public walkway. The chair of the Dogs on the Promenade task force Councillor Scott Kristjanson personally went out that morning to clean up the “doggy debris”.

“I beat staff,” Kristjanson said, and offered to share photographs of him handling the business.” 

And Kristjanson noted that the temporary acceptance of dogs on the promenade had unintended consequences~seniors with small dogs that did not have access to vehicles were using the promenade to walk their dogs, and were thrilled to socialize with the community that way.

dona-brady

dona-brady

Counter this with the actions of this City Councillor in Cleveland who was upset that the pastor of the Denison Avenue United Church was going to open up the church for homeless people to stay warm and sleep during the coldest part of the winter. Councillor Dona Brady “told the pastor of Denison Avenue United Church of Christ more than a month ago that she would not allow it to offer homeless people a cot for a safe night’s sleep.”

And how did she do that? While the church’s Metanoia operation  brought 13 people in a night and offered a meal, access to bathrooms and a shower, a by-law officer showed up with a list of code violations. That officer returned the next day too. As Michael McIntyre writes in Cleveland.com that City Councillor refused to meet with the church to discuss her concerns, and did not return the journalist’s calls. Of those 13 people that were taking advantage of sleeping in the warmth of the church, some did live on the streets in the immediate neighbourhood.  Metanoia also operated in another church in downtown Cleveland providing a warming centre, a meal and sleeping facilities. That location received a fire inspection and was told that a 45 person limit was in effect, meaning dozens of homeless were turned away at night.

A lawyer who was the representing the church’s Metanoia organization called any violations “micro stuff that doesn’t involve the health and safety of people on or around the property.We also have a difference of opinion as to whether we are violating anything here,” he said, noting that he doesn’t believe the building’s use has changed because it is not being used as a residence.”

There you have it~one White Rock City Councillor that steps out of his role to go on site and actually perform a scoop up of doggy do, and a Cleveland Councillor that steps over her role to keep out the most vulnerable instead of working to be part of the solution. And what is at stake is how we react to problems, how we think of people and ourselves, and how we lift up each other.

As the church’s minister, Pastor Nozomi Ikuta states “I am wired more toward obedience…But, she added, she had core commitments to keep: “love, compassion, generosity, community, respect.”

7WUVFLQNRNE3LNW624SCSB4PWI

7WUVFLQNRNE3LNW624SCSB4PWI

 

 

Images: Peace Arch News, Cleveland People, Cleveland.com

Thanks to Tom Horsman for link

 

17 Dec 04:47

A Modular Couch Update and Giveaway with Article!

by Alison Mazurek
600sqft_Article_giveaway1

We have had our Solae sofa for over a year now. In celebration of the couch surviving our small-space-hard- wearing family and a friendship forged with Article, Article is generously offering a $500 giveaway here on the blog. It’s in fact the first giveaway ever hosted here on the blog!

GIVEAWAY CLOSED ///// To enter comment below with your favourite current Article piece. Also be sure to be following @Article and @600sqftandababy on Instagram. When you’ve commented below click*** this link ***to finalize your entry! A random winner will be drawn on Dec 19th. Article also mentioned they will be having a boxing day sale on December 26th so if you win you might want to use it on that day!

This seemed like the perfect opportunity to update you on how the Solae has been working out for us. I don’t always get the chance to respond to older comments and questions so hopefully I can capture all of that here. Here is the original post about our Solae (A Modular Sofa for our Small Space) and a video we made with Article about our space (Small Space Living with Kids: A Video with Article).

Layout

We’ve been so happy with the layout and how it has opened up our living room, allowing for more floorspace without compromising on seats. I’m especially grateful for the additional ottoman when we are hosting a lot of friends. The kids still jump and run on the couch though there have been some accidents so we have been trying to cut back on it a bit. I would say the couch is medium firm and hasn’t softened a ton since we got it. Which bodes well for the long term, I think.

Not sure if you can notice in the pictures but we have played around with the layout a bit putting a side table between two of the sections at times. That’s the benefit of a modular sofa, you don’t have to stick to one layout. Sometimes I am asked if the separate sections are uncomfortable. I would say, yes I don’t prefer to sit in between the two pieces but there are enough spots without a crack that it doesn’t bother me. Mae on the other hand loves to wedge herself in the crack, it’s her favourite spot, haha.

I’ve also noticed that we spend more time on the floor sitting up against the couch because of it’s low proximity to the ground and the comfy sides.

I’ve even had a few friends buy the same couch (in different configurations) after sitting on ours. I’ve been known to offer acquaintances to come over and try it out, ha! A couch is a big decision and I’m happy to facilitate, not saying I can invite everyone reading this over ;).

600sqft_Article_giveaway4

Care

There was some initial pilling but it has calmed and we also had a darker blanket that was contributing to the pilling. I’ve swapped out the blanket for a softer grey wool blanket that doesn’t cause pilling. As for stains… well we keep eating to the kitchen (except popcorn on movie nights) and somehow magically we haven’t spilled any wine on it (and now that I’ve written that out I probably will this weekend). I’ve wiped down any marks with a cloth and soap and water and that has worked so far.

600sqft_Article_giveaway3

Movie Night

We often set up the projector on Friday nights and I don’t usually get a photo of the layout so I set it up here so you can get a sense of the cozy couch/bed we make. There are more cords when it is actually playing and yes we stress about the projector being touched or bumped by little ones. We have a highly sensitive 6 year old who scares easily so we tend to rotate the same movies… current movie favourites: Sing, It Takes Two, Daddy Daycare, Toy Story (1 and 4), Frozen. Current Christmas Favourites…. The Grinch (Jim Carrey version), Home Alone.

Movie Night set up… Solae couch, side table and projector that projects on the wall bed.

Movie Night set up… Solae couch, side table and projector that projects on the wall bed.

This post is in partnership with Article. All words are my own. Thanks for supporting the brands that support this blog.

17 Dec 04:47

Ein Schelm ...

by Volker Weber

Schöner Burn der Deutschen Bahn:

Das ist das, was BLöD und Angsthasen für Deutschland verstanden haben:

Das hat Greta Thunberg gemeint:

Und das ist wirklich passiert >

17 Dec 04:45

✚ How to Make Interactive Frequency Trails with D3.js

by Peter Cook

Layering time series data or distributions with this method can change the feel and aesthetic versus a multi-line chart or small multiples. In some cases, frequency trails let you show more in less space. Read More

17 Dec 04:45

Let’s get #deepreal

by Doc Searls

Deepfakes are a big thing, and a bad one.

On the big side, a Google search for deepfake brings up more than 23 billion results.

On the bad side, today’s top result in a search on Twitter for the hashtag #deepfake says, “Technology is slowly killing reality. I am worried of tomorrow’s truths that will be made in shops. This #deepfake is bothering my soul deeply.” In another of the top results, a Vice report is headlined Deepfake Porn Is Evolving to Give People Total Control Over Women’s Bodies.

Clearly we need an antidote here.

I suggest deepreal.

If deepfake lies at the bottom of the uncanny valley (as more than 37 thousand sites online suggest), deepreal should be just as highly out of that valley. As the graphic above (source) suggests, the deeply real (I added that) is fully human, and can elicit any level of emotional response, as real humans tend to do.

So what do we know that’s already deepreal?

Well, there’s reality itself, meaning the physical kind. A real person talking to you in the real world is undeniably human (at least until robots perfectly emulate human beings walking, talking and working among us, which will be icky and therefore deep in the uncanny valley). But what about the digital world? How can we be sure that a fully human being is also deeply real where the prevalent state is incorporeal—absent of physical existence?

The only way I know, so far, is with self-sovereign identity (SSI) technology, which gives us standardized ways of letting others know required facts about us (e.g. “I’m over 18,” “I’m a citizen of this country,” “I have my own car insurance,” “I live in this state,” “I’m a member of this club.”) Here’s some of what I’ve written and said about SSI:

  1. The Sovereign Identity Revolution (OneWorldIdentity, 21 February, 2017)
  2. New Hope for Digital Identity (Linux Journal, 9 November 2017)
  3. Some Thoughts About Self-Sovereign Identity (doc.blog, 16 March 2019)
  4. Some Perspective on  Self-Sovereign Identity (KuppingerCole, 20 April 2019)
  5. Thoughts at #ID2020 (Doc Searls Weblog, 19 September 2019)

As I put it in #4 above, “The time has come to humanize identity in the networked world by making it as personal as it has been all along in the natural one.” I believe it is only by humanizing identity in the networked world that we can start to deal with deepfakes and other ways we are being dehumanized online. (And, if you’re skeptical about SSI, as are some I shouted out to here, what other means to you suggest? It’s still early, and the world of possibility is large.)

I also look forward to discussing this with real people here online—and in the physical world. Toward that, here are some identity tech gatherings coming up in 2020:

I also look forward to playing whack-a-mole with robots faking interest in this post; and which, because I’ll succeed, you’ll not see in the comment section below. (You probably also won’t see comments by humans, because humans prefer conversational venues not hogged by robots.)

17 Dec 04:44

Comments and Collatz Conundrum

by Matt

Over the summer Terence Tao, a Fields Medal-winning mathematician considered one of the best of his generation, got an anonymous comment on his WordPress blog post from 2011 exploring the Collatz conjecture — one of the most persistent problems in math — suggesting he explore the problem for “almost all” numbers. Terence has been a regular WP.com blogger since 2007 and he and his commenters make extensive use of our LaTeX feature to express and embed equations.

That anonymous comment led him to an important breakthrough on the Collatz Conundrum, as Quanta Magazine reports. If you want great comments, you as the author have to participate in them and Terence is incredibly active in engaging with the commenters on his site.

I’ve always said that comments are the best part of blogging, but this is a particularly cool example. Here’s Terence’s latest post on it, with an excellent comment thread following.

17 Dec 04:44

Designer, you can run a usability study: Usability Mentorship at Mozilla

by Holly Collier

Authors: Holly Collier, Jennifer Davidson

On the Firefox UX team, a human-centered design process and a “roll up your sleeves” attitude define our collaborative approach to shipping products and features that help further our mission. Over the past year, we’ve been piloting a Usability Mentorship program in an effort to train and empower designers to make regular research part of their design process, treating research as “a piece of the pie” rather than an extra slice on the side. What’s Mozilla’s Firefox UX team like? We have about twenty designers, a handful of user researchers, and a few content strategists.

This blog post is written by Holly (product designer, and mentee), and Jennifer (user researcher, and mentor).

A soda can in a coozy that says “User research is a team sport,” sitting on a table with people & laptops in the background.
photo: Holly Collier; A coozy gift from Gemma Petrie. Credit for the phrase goes to Leslie Reichelt at GDS.

Why should I, a designer, learn user research skills?

Let’s start with Holly’s perspective.

I’m an interaction designer — I’ve been designing apps and websites (with and without the help of user research) for over a decade now, first in agencies and then in-house at an e-commerce giant. Part of what drew me to Mozilla and the Firefox UX team a year ago was the value that Mozillians place on user research. When I learned that we had an official Usability Mentorship program on the Firefox UX team, I was really excited — I had gotten a taste of helping to plan and run user research during my last gig, but I wanted to expand my skill set and to feel more confident conducting studies independently.

I think it’s really important to make user research an ongoing part of the product design process, and I’m always amazed by the insights it produces. By building up my own user research skill set, it means that I’m in a better position to identify user problems for us to solve and to improve the quality of the products I work on.

How does the mentorship program work?

And now onto Jennifer. She’ll talk about how this all worked.

A little bit about me before we dive in. I’m a user researcher — I’ve been in the industry for 6 years now, and at Mozilla on the Firefox User Research team for 3 years. I’ve worked at a couple big tech companies (HP & Intel) before coming to Mozilla. Prior to that, I worked hard at internships and got a PhD in Computer Science, focused on Human Computer Interaction. I love working at Mozilla, especially with designers like Holly, who are passionate about user research informing product design!

At Mozilla, our research team conducts three types of research (as written by Raja Jacob and Gemma Petrie):

  • Exploratory: Discovering and learning. Conducting research around a topic where a little is known. This type of research allows us to explore and learn about a problem space, challenge our assumptions on a topic, and more deeply understand the people we are designing for.
  • Generative: Generative research can help us develop concepts through activities such as participatory design sessions or help us better understand user behavior and mental models related to a specific problem/solution space.
  • Evaluative: Evaluative research is conducted to test a proposed solution to see if it meets people’s needs, is easy to use and understand, and creates a positive user experience. Usability testing falls under this category.

Like most organizations, we routinely have more designs that need usability testing than we have researchers. Gemma Petrie, our most senior User Researcher (a Principal User Researcher), started the mentorship program as a way to address this problem in her previous role as interim Director of User Research. By spreading usability testing abilities more broadly across the Firefox UX team, we could ensure that more designs got tested and ensure that our dedicated researchers could continue to do exploratory and generative research.

Because all of our designers and content strategists had different levels of familiarity with usability testing, Gemma brought in an external consultant to kick-off this effort and run a usability testing workshop with the entire UX team. This workshop was recorded so it can be cross-referenced later, and so that new team members can watch it as part of their onboarding.

At Mozilla, a mentorship project starts somewhat informally. Designers and content strategists “raise their hand” to show interest, and each researcher on the User Research team is a mentor. A designer gets paired with a mentor to figure out a (hopefully) low stress, low-stakes project to work on together. The designer takes the reins, and the researcher helps out along the way.

While we don’t have a strict curriculum, after a designer shows interest, each mentorship roughly follows these steps:

  • Pre-work: Watch the recorded usability training and fill out a simple intake form to describe their desired project.
  • First meeting — Set the bounds: To keep things simple, we restrict the method to a usability test on usertesting.com. This isn’t a survey, a foundational piece of work, or anything huge. The goal is to improve the design at hand.
  • After the first meeting — Homework: Look at past examples and come up with the research purpose and a draft of research questions.
  • Plan and protocol: Work hand-in-hand with the research mentor to create a research plan and protocol. Then collect feedback on the research questions from project stakeholders and write a protocol for the usability test tasks.
  • Analysis: One of our other researchers, Alice Rhee, created a great “analysis tips” document that we share with mentees: Set up a spreadsheet, watch the pilot video, make any necessary adjustments to the test, and then go from there. Direct quotes are captured, along with success or failure of tasks. Some quotes are bolded that are candidates to become “highlights” later.
  • Synthesis: Record answers to all research questions based on the summaries from the analysis. Is there anything missing? Anything you’re unsure about? Meet with research mentor to talk through this part.
  • Report: Use an existing report to get started. Start with a background and methods section, then clearly answer each research question.
  • Presentation: Work with Product Manager to schedule a time to share findings with the impacted team. Record it. Put it in the User Research repository.

But how does the mentorship program really work?

Let’s have Holly tell us about what she learned from her experience testing one of Firefox’s apps.

Stand on the shoulders of giants

We identified a product that needed usability testing: Firefox Lockwise for Android (then called “Firefox Lockbox”), a new password manager app that works in conjunction with logins that are saved in the Firefox browser. It’s in my team’s practice area, so I thought it would be a good chance to get to know a new product, but it was also a good fit in terms of my experience with the Android platform (all of my previous involvement in user research was on mobile apps).

There were a lot of materials available to help me get started — sample protocols, decks, analysis spreadsheets. The Firefox User Research team is great about documenting and saving research artifacts. The Lockwise team also had conducted in-person usability testing on their iPhone app the previous summer, so I had some usability questions to start with.

Firefox Lockbox app loader screen with screen recorder UI visible.
Firefox Lockbox prototype in the usertesting.com screen recorder.

Designing and piloting an effective test protocol feels a lot like… product design and prototyping!

The process of gathering requirements, designing, piloting and revising before releasing a usability test to participants felt similar to the process of problem definition, design, prototyping and iteration we use for product design:

Requirements Gathering: This particular test had many constraints and requirements. Because this was a remote, unmoderated test, and because users had to have a Firefox account with Sync enabled in order to test the Lockbox app, the protocol for the test was pretty extensive.

Protocol Design: Getting high-quality test results required thinking through the test experience from the test taker’s point of view while also achieving our research goals:

  • How do I ensure that people to see everything we need them to see?
  • How do I construct questions to be clear but not leading?
  • How can I extend the protocol beyond usability to cover comprehension and desirability (make sure we’re designing the “right” thing) but also keep the test as short as possible?

Piloting: Before launching the real test, we launched a prototype of the test called a “pilot” and watched videos of a few participants to make sure the test instructions were understood and that the test was functioning as designed. Getting out of this pilot stage was challenging because of problems we discovered and had to troubleshoot along the way:

  • Our protocol had multiple sections and required a lot of steps before participants saw the actual thing that we were testing, so there were lots of potential failure points (and as a result, a lot of iterations around the wording for this part of the protocol before we got it right).
  • When a few participants weren’t seeing expected pieces of important functionality in the prototype, we figured out through talking with engineering that we needed to change the participant screening criteria to limit it to specific Android operating systems.
  • After getting a few recordings of participants screens that were totally black except for the usertesting.com mobile video recorder interface, we figured out that the prototype for our app, a password manager, had code in it that wouldn’t allow the mobile video recorder to capture participants’ screens. Our engineering team made us a special build for the rest of the tests. The lesson here: Talk with your engineering team, early & often!
Black screen with only screen recorder UI visible.
The prototype wouldn’t allow the screen recorder to capture participants’ screens.

Once we addressed the issues with the protocol and the prototype that we uncovered during our pilot, we launched the test, and I moved on to watching participant videos and taking lots of notes (direct quotes!) that I’d mine for insights later.

Analysis and report writing don’t have to be scary

Analyzing the test data and delivering the findings (and recommendations!) was the most intimidating part of the process for me. As a designer, I’ve always looked to the research findings deck as a ‘beacon of truth’ in the design process. Now that I was running the research, I felt a lot of responsibility to deliver that same truth and guidance.

I worked through those feelings of intimidation by triple-checking my sources, mapping my data (including quotes) to the research questions and getting the following awesome perspective from Jennifer:

  • Usability tests are mostly about observation — just tell the story of what you saw.
  • The findings don’t need to be the ‘tablets coming down from the mountain,’ they just need to be accurate and backed by the data.
  • Design recommendations are suggestions, not directions. By phrasing them as “How might we?” questions (rather than being prescriptive about solutions), I could frame problems for the Lockwise team and rely on them to use their deep knowledge of the product and space to solve them.
The videochat interface, showing the Lockwise for Android findings presentation and attendees’ faces.
The distributed Firefox Lockwise team during the findings presentation.

As it turned out, giving the findings presentation to the Lockwise team actually ended up being one of my favorite parts of the whole process. Telling the story was fun and inspired great conversation, and the Lockwise team really appreciated the “How might we?” format I used for the design recommendations.

How did the usability testing mentorship go for mentor?

Jennifer, take it away!

Helping Holly out with this project helped make tacit knowledge explicit. I’ve done many, many usability tests. I am almost on auto-pilot when I conduct them. So it was a great exercise to actually explain the process with a co-worker and perform a needed amount of reflection on my process. Especially with Holly, who was willing to learn and asks great questions.

Here are some of those great questions that made me reflect on my process:

How do I know when the pilot phase is over?

The pilot is over when the protocol “works” as intended. That means there are no show-stopping bugs in the platform that prevent someone from doing the task — and if there are showstoppers, adjusting the protocol. It also means that your questions have been phrased in a way that people understand. You can only determine this through observing a couple pilot participants.

Can I use my pilot data in the report?

The academic in me says that if the protocol changes at all from the pilot to running the rest of the tests, no, you can’t use the pilot data in the report. The industry researcher in me says that sure, you can include the results as long as you mark it clearly as pilot data.

How do I present ‘bad news’ to a team?

Most usability tests have at least some good news. Start with that! Be clear about when you’re going to deliver the bad news and come prepared with “How might we?” questions or recommendations on how to improve the experience.

How do I make sure I take my personal bias of how I understand this app out of the process?

Acknowledge your bias. Know what it is going in and voice it to your mentor. Do exactly what Holly mentioned earlier and double and triple-check your results against the videos, direct quotes, and research questions. Do you still feel like you might be stretching the interpretation of a result? Check with your mentor, or anyone who’s done a usability test before. Have a co-worker who isn’t very close to the project review your results and recommendations before you present them to the wider team.

And now, in parting.

Holly, the designer & mentee says:
Designers, you can do user research! Since completing the mentorship, I have conducted several other studies, including a usability/concept test and an information architecture research study. It’s become a regular part of my design practice, and I think the products I work on are better for it.

Jennifer, the user researcher & mentor says:
User researchers out there: you can run your own usability testing mentorship program!
In the spirit of open source, here are some examples:

Thank you to Gemma Petrie, Anthony Lam, and Elisabeth Klann for reviewing this blog post. And a special thanks to Gemma Petrie for setting up the Usability Mentorship Program at Mozilla.

Also published on the Firefox UX Blog.


Designer, you can run a usability study: Usability Mentorship at Mozilla was originally published in Firefox User Experience on Medium, where people are continuing the conversation by highlighting and responding to this story.

17 Dec 04:40

✚ How to Make a Grid Map with Histograms in R, with ggplot

by Maarten Lambrechts

Layout multiple charts in a single view. Then adjust the scales appropriately for maximum comparability and a unified graphic. Read More

17 Dec 04:36

‘Scootergeddon’ – or Passing Fad?

by Gordon Price
mkalus shared this story from Price Tags.

Vancouver, get ready.

Via Dianna

From the San Francisco Chronicle:

San Francisco will allow 750 more electric rental scooters onto the streets Sunday as the second phase of a pilot program rolls out.

That brings the total number of scooters allowed in the city to 3,250. It’s a mark of cautious approval for a clean but controversial technology, which still leaves people worried about illegal sidewalk riding and injuries. …

Two months into San Francisco’s pilot program, Supervisor Aaron Peskin cautiously voiced support.

“I would say so far, not bad, compared to ‘scootergeddon’ of last year,” Peskin said, comparing the unregulated scooters that swarmed city streets in spring 2018 to an apocalypse. “Although there is still some sidewalk riding and still some injuries, all in all the rollout has been going well.” …

A May report from Boston Consulting Group found that scooter companies are hardpressed to make money: The devices have an average life span of three months, but companies need four months to break even per device. Longerlasting batteries may help.

The report also forecast consolidation in the industry, and that is indeed happening. In June, Bird bought Scoot, a San Francisco company, and its coveted city permit. …

“We don’t know whether these jobs are going to exist five or 10 years down the road, whether scooters are a passing fad,” said Doug Bloch, political director for Teamsters.

17 Dec 04:36

Astronauts Complete First Excursion to Repair Cosmic Particle Detector

mkalus shared this story from NASA Image of the Day.

Station Commander ​​Luca Parmitano of the European Space Agency conducts repairs, while attached to the Canadarm during the first spacewalk to repair the Alpha Magnetic Spectrometer.

17 Dec 04:36

Those commercials where the husband buys his wife, like, two giant pickup trucks for Christmas are even more psychotic than the Pelotan wife, but it doesn't register because they've been running ads like that for years.

by existentialcoms
mkalus shared this story from existentialcoms on Twitter.

Those commercials where the husband buys his wife, like, two giant pickup trucks for Christmas are even more psychotic than the Pelotan wife, but it doesn't register because they've been running ads like that for years.




1190 likes, 93 retweets
17 Dec 04:36

The total area of solar panels it would take to power the world, Europe, and Germany. This map is from Nadine May’s thesis [source: ow.ly/ga7Z50xBrRU] pic.twitter.com/aLoos4hm5T

by Rainmaker1973
mkalus shared this story from Rainmaker1973 on Twitter.

The total area of solar panels it would take to power the world, Europe, and Germany. This map is from Nadine May’s thesis [source: ow.ly/ga7Z50xBrRU] pic.twitter.com/aLoos4hm5T





155 likes, 64 retweets
17 Dec 01:33

Support: Optical Heart Rate Tips

by garminblog
mkalus shared this story from garminblog's YouTube Videos.

From: garminblog
Duration: 01:15

Learn some tips that can be used to get a more accurate reading from the optical heart rate sensor on your Garmin watch.

For more help, visit http://support.garmin.com

17 Dec 01:33

Vancouver, 1972. Photography by Greg Girard. pic.twitter.com/GRf6G9q408

by moodvintage
mkalus shared this story from moodvintage on Twitter.

Vancouver, 1972. Photography by Greg Girard. pic.twitter.com/GRf6G9q408





120 likes, 20 retweets
17 Dec 01:33

Who here remembers LimeWire? pic.twitter.com/hk2WKZvyix

by moodvintage
mkalus shared this story from moodvintage on Twitter.

Who here remembers LimeWire? pic.twitter.com/hk2WKZvyix





242 likes, 37 retweets
17 Dec 01:33

RT @castlvillageman: The ultimate example of the failure of the UK educational system. pic.twitter.com/qsz67sv6MC

by castlvillageman
mkalus shared this story from ottocrat on Twitter.

The ultimate example of the failure of the UK educational system. pic.twitter.com/qsz67sv6MC



Retweeted by ottocrat on Monday, December 16th, 2019 8:50pm


4056 likes, 1053 retweets
17 Dec 01:32

Logging to SQLite using ASGI middleware

I had some fun playing around with ASGI middleware and logging during our flight back to England for the holidays.

asgi-log-to-sqlite

I decided to experiment with SQLite as a logging mechanism. I wouldn’t use this on a high traffic site, but most of my Datasette related projects are small enough that logging HTTP traffic directly to a SQLite database feels like it should work reasonable well.

Once your logs are in a SQLite database, you can use Datasette to analyze them. I think this could be a lot of fun.

asgi-log-to-sqlite is my first exploration of this idea. It’s a piece of ASGI middleware which wraps an ASGI application and then logs relevant information from the request and response to an attached SQLite database.

You use it like this:

from asgi_log_to_sqlite import AsgiLogToSqlite
from my_asgi_app import app

app = AsgiLogToSqlite(app, "/tmp/log.db")

Here’s a demo Datasette instance showing logs from my testing: asgi-log-demo-j7hipcg4aq-uc.a.run.app

As always with Datasette, the data is at its most interesting once you apply some facets.

Intercepting requests to and from the wrapped ASGI app

There are a couple of interesting parts of the implementation. The first is how the information is gathered from the request and response.

This is a classic pattern for ASGI middleware. The ASGI protocol has three key components; a scope dictionary describing the incoming request, and two async functions called receive and send which are used to retrieve and send data to the connected client (usually a browser).

Most middleware works by wrapping those functions with custom replacements. That’s what I’m doing here:

class AsgiLogToSqlite:
    def __init__(self, app, file):
        self.app = app
        self.db = sqlite_utils.Database(file)
    # ...
    async def __call__(self, scope, receive, send):
        response_headers = []
        body_size = 0
        http_status = None

        async def wrapped_send(message):
            nonlocal body_size, response_headers, http_status
            if message["type"] == "http.response.start":
                response_headers = message["headers"]
                http_status = message["status"]

            if message["type"] == "http.response.body":
                body_size += len(message["body"])

            await send(message)

        start = time.time()
        await self.app(scope, receive, wrapped_send)
        end = time.time()

My wrapped_send() function replaces the original send() function with one that pulls out some of the data I want to log from the messages that are being sent to the client.

I record a start time, then await the original ASGI application, then record an end time when it finishes.

Logging to SQLite using sqlite-utils

I’m using my sqlite-utils library to implement the logging. My first version looked like this:

db["requests"].insert({
    "path": scope.get("path"),
    "response_headers": str(response_headers),
    "body_size": body_size,
    "http_status": http_status,
    "scope": str(scope),
}, alter=True)

sqlite-utils automatically creates a table with the correct schema the first time you try to insert a record into it. This makes it ideal for rapid prototyping. In this case I captured stringified versions of various data structures so I could look at them in my browser with Datasette.

The alter=True argument here means that if I attempt to insert a new shape of record into an existing tables any missing columns will be added automatically as well. Again, handy for prototyping.

Based on the above, I evolved the code into recording the values I wanted to see in my logs - the full URL path, the User-Agent, the HTTP referrer, the IP and so on.

This resulted in a LOT of duplicative data. Values like the path, user-agent and HTTP referrer are the same across many different requests.

Regular plain text logs can solve this with gzip compression, but you can’t gzip a SQLite database and still expect it to work.

Since we are logging to a relational database, we can solve for duplicate values using normalization. We can extract out those lengthy strings into separate lookup tables - that way we can store mostly integer foreign key references in the requests table itself.

After a few iterations, my database code ended up looking like this:

with db.conn:  # Use a transaction
    db["requests"].insert(
        {
            "start": start,
            "method": scope["method"],
            "path": lookup(db, "paths", path),
            "query_string": lookup(db, "query_strings", query_string),
            "user_agent": lookup(db, "user_agents", user_agent),
            "referer": lookup(db, "referers", referer),
            "accept_language": lookup(db, "accept_languages", accept_language),
            "http_status": http_status,
            "content_type": lookup(db, "content_types", content_type),
            "client_ip": scope.get("client", (None, None))[0],
            "duration": end - start,
            "body_size": body_size,
        },
        alter=True,
        foreign_keys=self.lookup_columns,
    )


def lookup(db, table, value):
    return db[table].lookup({
        "name": value
    }) if value else None

The table.lookup() method in sqlite-utils is designed for exactly this use-case. If you pass it a value (or multiple values) it will ensure the underlying table has those columns with a unique index on them, then get-or-insert your data and return you the primary key.

Automatically creating tables is fine for an initial prototype, but it starts getting a little messy once you have foreign keys relationships that you need to be able to rely on. I moved to explicit table creation in an ensure_tables() method that’s called once when the middleware class is used to wrap the underlying ASGI app:

    lookup_columns = (
        "path",
        "user_agent",
        "referer",
        "accept_language",
        "content_type",
        "query_string",
    )

    def ensure_tables(self):
        for column in self.lookup_columns:
            table = "{}s".format(column)
            if not self.db[table].exists:
                self.db[table].create({
                    "id": int,
                    "name": str
                }, pk="id")
        if not self.db["requests"].exists:
            self.db["requests"].create({
                "start": float,
                "method": str,
                "path": int,
                "query_string": int,
                "user_agent": int,
                "referer": int,
                "accept_language": int,
                "http_status": int,
                "content_type": int,
                "client_ip": str,
                "duration": float,
                "body_size": int,
            }, foreign_keys=self.lookup_columns)

I’m increasingly using this pattern in my sqlite-utils projects. It’s not a full-grown migrations system but it’s a pretty low-effort way of creating tables correctly provided they don’t yet exist.

Here’s the full implementation of the middleware.

Configuring the middleware for use with Datasette

Publishing standalone ASGI middleware for this kind of thing is neat because it can be used with any ASGI application, not just with Datasette.

To make it as usable as possible with Datasette I want it made available as a plugin.

I’ve tried two different patterns for this in the past.

My first ASGI middleware was asgi-cors. I published that as two separate packages to PyPI: asgi-cors is the middleware itself, and datasette-cors is a very thin plugin wrapper around it that hooks into Datasette’s plugin configuration mechanism.

For datasette-auth-github I decided not to publish two packages. Instead I published a single plugin package and then described how to use it as standalone ASGI middleware in its documentation.

This lazier approach is confusing: it’s not at all clear that a package called datasette-auth-github can be used independently of Datasette. But I did get to avoid having to publish two packages.

datasette-configure-asgi

Since I want to do a lot more experiments with ASGI plugins in the future, I decided to try solving the ASGI configuration issue once and for all. I built a new experimental plugin, datasette-configure-asgi which can be used to configure ANY ASGI middleware that conforms to an expected protocol.

Here’s what that looks like at the configuration level, using a metadata.json settings file (which I should really rename since it’s more about configuration than metadata these days):

{
  "plugins": {
    "datasette-configure-asgi": [
      {
        "class": "asgi_log_to_sqlite.AsgiLogToSqlite",
        "args": {
          "file": "/tmp/log.db"
        }
      }
    ]
  }
}

The implementation of this plugin is very simple: here’s the entire thing:

from datasette import hookimpl
import importlib


@hookimpl
def asgi_wrapper(datasette):
    def wrap_with_classes(app):
        configs = datasette.plugin_config("datasette-configure-asgi") or []
        for config in configs:
            module_path, class_name = config["class"].rsplit(".", 1)
            mod = importlib.import_module(module_path)
            klass = getattr(mod, class_name)
            args = config.get("args") or {}
            app = klass(app, **args)
        return app

    return wrap_with_classes

It hooks into the asgi_wrapper plugin hook, reads its configuration from the datasette object (using plugin_config()), then loops through the list of configured plugins and dynamically loads each implementation using importlib. Then it wraps the ASGI app with each of them in turn.

Open questions

This is where I’ve got to with my experiments so far. Should you use this stuff in production? Almost certainly not! I wrote it on a plane just now. It definitely needs a bit more thought.

A couple of obvious open questions:

Python async functions shouldn’t make blocking calls, since doing so will block the entire event loop for everyone else.

Interacting with SQLite is a blocking call. Datasette works around this by running SQL queries in a thread pool; my logging plugin doesn’t bother with that.

Maybe it should? My hunch is that inserting into SQLite in this way is so fast it won’t actually cause any noticeable overhead. It would be nice to test that assumption thoroughly though.

Log rotation. This is an important detail for any well designed logging system, and I’ve punted on it entirely. Figuring out an elegant way to handle this with underlying SQLite databases files would be an interesting design challenge - relevant issue.

Would my SQLite logging middleware work with Django 3.0? I don’t see why not - the documentation covers how to wrap entire Django applications with ASGI middleware. I should try that out!

This week’s Niche Museums

These are technically my weeknotes, but logging experiments aside it’s been a quiet week for me.

I finally added paragraph breaks to Niche Museums (using datasette-render-markdown, implementation here) As a result my descriptions have been getting a whole lot longer. Added this week:

17 Dec 01:32

Star Trek: Picard renewed for Season 2 ahead of January premiere on Crave

by Bradly Shankar
Star Trek: Picard

CBS All Access has renewed its Star Trek: Picard series for a second season ahead of the first season’s premiere on January 23rd.

In Canada, Bell Media holds the exclusive rights to Picard and will broadcast the series on its CTV Sci-Fi channel in English and Z channel in French. Each episode will begin streaming on Bell’s Crave service the following day.

As the name suggests, Picard features the return of the iconic USS Enterprise captain, played once again by Patrick Stewart. The series marks Stewart’s first time in the role since 2002’s Star Trek: Nemesis and explores the fallout of the destruction of Romulus.

While CBS has yet to officially comment on the renewal, The Hollywood Reporter notes that the company is likely waiting to make one once it’s confirmed a new showrunner for the series’ second season. As it stands, the current showrunner, novelist Michael Chabon (The Amazing Adventures of Kavalier and Clay), is reportedly looking to shift duties to another creator as he moves onto other projects.

The most recent look at the series came from New York Comic-Con in October via a new trailer. Check it out below:

Image credit: CBS

Source: The Hollywood Reporter

The post Star Trek: Picard renewed for Season 2 ahead of January premiere on Crave appeared first on MobileSyrup.