Shared posts

14 Nov 05:27

My Python Development Environment, 2020 Edition

My Python Development Environment, 2020 Edition

Jacob Kaplan-Moss shares what works for him as a Python environment coming into 2020: pyenv, poetry, and pipx. I'm not a frequent user of any of those tools - it definitely looks like I should be.

Via @jacobian

14 Nov 05:27

Datasette 0.31

Datasette 0.31

Released today: this version adds compatibility with Python 3.8 and breaks compatibility with Python 3.5. Since Glitch support Python 3.7.3 now I decided I could finally give up on 3.5. This means Datasette can use f-strings now, but more importantly it opens up the opportunity to start taking advantage of Starlette, which makes all kinds of interesting new ASGI-based plugins much easier to build.

14 Nov 05:27

Die nächste Amazon-Disruption :: Klamotten

by Volker Weber

bff770c92ca461e88ab416d9cc909ffc

Neulich haben wir im Circus Merino diskutiert, weil Klamotten aus Merinowolle nicht so schnell müffeln und seltener gewaschen werden müssen. Ideal für Reisende. Also habe ich mal bei Amazon gesucht und bin dort auf Meraki, eine Amazon-Eigenmarke gestoßen. Und dann einen Wollpullover für 28 Euro bestellt, um mal zu schauen, was man für so wenig Geld bekommt.

d998e64d596cc27b59e14cd32eb25330

Was soll ich sagen? Tadellos. Habe ihn jemandem gezeigt, der Klamotten am Stoff und nicht am Label einordnet und der sagte: "Mindestens 79 Euro." Wer die Margen von Groß- und Einzelhandel kennt, der weiß, dass Amazon auch bei 28 Euro noch Geld mit so einem Pulli verdienen kann. Meraki merke ich mir. Die haben genau die Basics, die mir gefallen.

More >

14 Nov 05:26

Snow!

by jnyyz

So we ended up getting a decent amount of snow overnight.

Bike lane snow clearance on Annette was inconsistent. Here it is at Keele.

As per usual, the bike lane was in much better shape on Dupont, especially between the Annette/Dupont junction and Symington.

Clear sailing on Bloor between Lansdowne and Dufferin.

Most sections of the Bloor bike lane were clear.

Side streets like Palmerston were still somewhat snowbound.

Late afternoon update:

On the way home, the Bloor bike lane was in good shape.

I was also pleased to see that the Shaw contraflow lane had been plowed.

One basic tenet of winter biking is that even when a street (like this section of Hallam) is plowed, the plow leaves plenty of margin around parked cars, and so you have to be comfortable taking the lane.

Annette was in better shape than this morning.

Its going to be cold overnight, so watch for black ice in the morning!

14 Nov 05:26

How Much You Should Be Saving for Retirement

by Nathan Yau

There are a lot of variables to consider, but for people of middle income, here's a suggestion, based on when you start saving and when you want to retire. Read More

14 Nov 05:26

Snow covered peaks somewhere near Baffin Bay in...

Snow covered peaks somewhere near Baffin Bay in late October

14 Nov 05:26

New Bytecode Alliance Brings the Security, Ubiquity, and Interoperability of the Web to the World of Pervasive Computing

by Mozilla

New community effort will create a new cross-platform, cross-device computing runtime based on the unique advantages of WebAssembly 

MOUNTAIN VIEW, California, November 12, 2019 — The Bytecode Alliance is a newly-formed open source community dedicated to creating new software foundations, building on standards such as WebAssembly and WebAssembly System Interface (WASI). Mozilla, Fastly, Intel, and Red Hat are founding members.

The Bytecode Alliance will, through the joint efforts of its contributing members, deliver a state-of-the-art runtime environment and associated language toolchains, where security, efficiency, and modularity can all coexist across the widest possible range of devices and architectures. Technologies contributed and collaboratively evolved through the Alliance leverage established innovation in compilers, runtimes, and tooling, and focus on fine-grained sandboxing, capabilities-based security, modularity, and standards such as WebAssembly and WASI.

Founding members are making several open source project contributions to the Bytecode Alliance, including:

  • Wasmtime, a small and efficient runtime for WebAssembly & WASI
  • Lucet, an ahead-of-time compiler and runti
  • me for WebAssembly & WASI focused on low-latency, high-concurrency applications
  • WebAssembly Micro Runtime (WAMR), an interpreter-based WebAssembly runtime for embedded devices
  • Cranelift, a cross-platform code generator with a focus on security and performance, written in Rust

Modern software applications and services are built from global repositories of shared components and frameworks, which greatly accelerates creation of new and better multi-device experiences but understandably increases concerns about trust, data integrity, and system vulnerability. The Bytecode Alliance is committed to establishing a capable, secure platform that allows application developers and service providers to confidently run untrusted code, on any infrastructure, for any operating system or device, leveraging decades of experience doing so inside web browsers.

Partner quotes:

Mozilla:

“WebAssembly is changing the web, but we believe WebAssembly can play an even bigger role in the software ecosystem as it continues to expand beyond browsers. This is a unique moment in time at the dawn of a new technology, where we have the opportunity to fix what’s broken and build new, secure-by-default foundations for native development that are portable and scalable. But we need to take deliberate, cross-industry action to ensure this happens in the right way. Together with our partners in the Bytecode Alliance, Mozilla is building these new secure foundations—for everything from small, embedded devices to large, computing clouds,” said Luke Wagner, Distinguished Engineer at Mozilla and co-creator of WebAssembly.

Fastly:

“Fastly is very happy to help bring the Bytecode Alliance to the community,” said Tyler McMullen, CTO at Fastly. “Lucet and Cranelift have been developed together for years, and we’re excited to formalize their relationship and help them grow faster together. This is an important moment in computing history, marking our chance to redefine how software will be built across clients, origins, and the edge. The Bytecode Alliance is our way of contributing to and working with the community, to create the foundations that the future of the internet will be built on.”

Intel:

“Intel is joining the Bytecode Alliance as a founding member to help extend WebAssembly’s performance and security benefits beyond the browser to a wide range of applications and servers. Bytecode Alliance technologies can help developers extend software using a wide selection of languages, building upon the full capabilities of leading-edge compute platforms,” said Mark Skarpness, VP, Intel Architecture, Graphics, and Software; Director, Data-Centric System Stacks.

Red Hat: 

“Red Hat believes deeply in the role open source technologies play in helping provide the foundation for computing, from the operating system to the browser to the open hybrid cloud,” said Chris Wright, senior vice president and Chief Technology Officer at Red Hat. “Wasmtime is an exciting development that helps move WebAssembly out of the browser into the server space where we are experimenting with it to change the trust model for applications, and we are happy to be involved in helping it grow into a mature, community-based project.”

Useful Links:

About Mozilla

Mozilla has been a pioneer and advocate for the web for more than 20 years. We are a global organization with a mission to promote innovation and opportunity on the Web. Today, hundreds of millions of people worldwide use the popular Firefox browser to discover, experience, and connect to the Web on computers, tablets and mobile phones. Together with our vibrant, global community of developers and contributors, we create and promote open standards that ensure the internet remains a global public resource, open and accessible to all.

The post New Bytecode Alliance Brings the Security, Ubiquity, and Interoperability of the Web to the World of Pervasive Computing appeared first on The Mozilla Blog.

14 Nov 05:26

The 4 things you must do AFTER you’ve finished your manuscript

by Josh Bernoff

In the monumental effort of writing a book, the biggest milestone is handing in your finished manuscript. Whew, that felt good. You’ve done 95% of the work. But you’re not done. Unless you prepare for the rest of the work, your book won’t be as good as it ought to be — and it might … Continued

The post The 4 things you must do AFTER you’ve finished your manuscript appeared first on without bullshit.

14 Nov 05:26

Immense wealth isn’t good for your reality sense. | Paul Krugmanhttps://t.co/jFD4TIxPpz| Stowe Boyd...

Immense wealth isn’t good for your reality sense. | Paul Krugmanhttps://t.co/jFD4TIxPpz| Stowe Boyd...
14 Nov 05:26

Plan by Squamish Nation for unique, super-dense development on False Creek sets off wave of praise

by Frances Bula

So, we kicked off last week with the story that the Squamish Nation has updated its plans for the land it owns around the south end of the Burrard Bridge, with a project that would have 6,000 units in 11 towers, one of them 56 storeys.

My story in the Globe and the follow-up story are here and here. Text below.

There’s been a huge wave of interest and response to the story, with calls coming in to Squamish Nation councillor Khelsilem from across Canada and even Britain.

For some people exasperated with city rules, the plan is being welcomed almost vengefully, like a giant middle finger to the city’s planning department.

Others are simply fascinated by the architectural design, which has echoes of First Nations themes, and the unusual approach.

I should note that not everyone is thrilled, like Vancouver Councillor Colleen Hardwick. Apparently there are also a lot of exchanges on various Facebook pages that express a lot less enthusiasm for the project than what is being heard more publicly.

As I noted in a series of tweets later in the week, there are many questions still to be answered. But it’s going to be a fascinating ride.

I can’t help but feel that there’s is an inevitable counterpoint coming to this first wave of euphoric enthusiasm over the Senakw development. There is much to praise about the first, utopian phase, but so many details still to come and some likely readjustments on the horizon.

There will be a lot more scrutiny once the public knows more about what the rent levels are likely to be. The Squamish are rightly not in the business of subsidizing housing for non-nation members, but 4,000 or so apartments at very high rents would leave a sour taste.

Although overall costs of the project should be lower than, for example, developer Ian Gillespie’s Vancouver House, given free land and minimal parking, they won’t be nothing. Very high concrete towers are the most expensive form of construction and the higher they go, the more $

That means bringing some portion of apartments to below-market or even at-market rates will require a subsidy from elsewhere, the most probable source being the federal government’s CMHC programs. That will take some negotiating and no one knows the outcome of that yet.
Then there’s the question of what rules will apply to those renting. The Residential Tenancy Act won’t apply. There have been scattered instances of unpleasant fights between First Nations and tenants when rents went up by rates way over the usual allowable limit.

The Vancouver Sun had a good story exploring the options for renter rules on the land. But no one knows yet what those will eventually be.

I get that a lot of people are gleeful over this project because of a raft of frustrations many have with current city-planning practices that seem to work counter to a lot of goals related to living more sustainably or building more efficiently.
But, even without city interference, surprises and compromises are coming. I just hope we don’t do that thing (so familiar in present-day politics) of having a violent infatuation and then, when it turns out the person or thing is not 100-per-cent perfect, switching to vitriol.
And, one final thought. In spite of what the cautions I’m saying everyone should bear in mind, it has been a pleasant surprise to a lot of people in and around this project that the mention of “Indigenous” has NOT set off the usual troll pack of awfulness.

BTW, for the many of you asking, the other big pieces of Vancouver land under First Nations control will not have the same freedom as this piece of Squamish land. I triple-checked with the city on this and they said:

Hi Frances, here’s the info on this.

The three projects you had asked about (Jericho lands, Heather lands, Liquor Distribution branch site at Broadway/Renfrew) which are being led by MST Development Corporation (“MST DC”) on behalf of the MST partnership, are owned by corporations and are not on federal or reserve lands.  As such, the development of those lands will be subject to all municipal laws and by-laws in respect of use and development of land.  For developments of this scale, the normal process would be a high level policy statement, rezoning and then the development permit and building permit process.  There will be extensive public engagement in this process and public hearings in front of Vancouver City Council for the rezonings.  This is unlike the proposed Senakw project which is on Squamish Nation reserve land and as such Squamish Nation’s land use planning jurisdiction applies and not the City of Vancouver’s.

 

 

VANCOUVER
SPECIAL TO THE GLOBE AND MAIL
FOR SUBSCRIBERS

SHARE

 

 

The development will be built on part of the traditional land of the Squamish Nation in the neighbourhood known as Kitsilano.

A First Nations-led development proposal in the heart of Vancouver has doubled in size from its original plans to include 6,000 units of mostly rental apartments in 11 towers, a project expected to be worth billions of dollars.

The Senakw development, to be built on part of the traditional land of the Squamish Nation in the neighbourhood now known as Kitsilano, across the False Creek inlet west of downtown Vancouver, will bring a level of density and building style to the area unlike anything there now.

In April, the Squamish announced a plan for 3,000 units on the site. Squamish councillor Khelsilem told The Globe and Mail Monday that the nation’s planning group has decided to double the size and potential value. Residents of Kitsilano have traditionally opposed buildings of more than a few storeys, and community activists earlier in the year said they hoped the Squamish would consult them on the project.

“The reasonable expectation my people have is they’ve seen this whole city built up around them with very little benefit. Now they are wanting us to go this far,” said Khelsilem, who uses one name. “We as council have a responsibility to make sure the highest value is created on an asset they own.”

 

The planned development would be one of the largest private First Nations investment projects in the country.

The City of Vancouver would have very little influence on the plan, because the land is owned by the Squamish Nation and not subject to local zoning or bylaws. Mayor Kennedy Stewart has called the housing project an opportunity to demonstrate the city’s commitment to reconciliation with Indigenous communities.

It would be one of the largest private First Nations investment projects in the country, expected by the Squamish to be in the billions of dollars, and turn the First Nation into a major developer in Vancouver’s lucrative housing market.

The 4.7-hectare project must be approved by the Squamish Nation’s approximately 3,000 members in a referendum on Dec. 10.

They’ll have to decide if they are comfortable with the idea of a 50-50 profit-sharing partnership with Vancouver developer Ian Gillespie of Westbank Corp. The Squamish would provide the land, and Westbank would do the construction.

Mr. Gillespie’s company is just finishing the Vancouver House project across the inlet, which, at 56 storeys, will be the same height as the tallest tower of Senakw.

A letter from the council to nation members says that, through a formal request for proposals and an assessment by the accounting firm Ernst & Young, “council concluded that a partnership with Westbank … will generate highest [financial value] and best-use leasing opportunities at Senakw.”

The project is relatively free from city constraints because it is on Indigenous land.

The letter notes that, under regulations from the federal department Indigenous Services Canada, members’ approval is needed to allow the nation to enter a business relationship with Westbank.

Squamish Nation members are also being asked to approve the idea of having as many as 30 per cent of the units potentially sold on a leasehold basis, with a maximum 120-year lease. This is a new idea for the Squamish, who have typically done traditional rentals, such as their arrangements with a mall in West Vancouver.

In a referendum 10 years ago, the nation approved a much smaller development on the land, which is near Burrard Bridge, with only two towers. Those never went ahead because of a downturn in the housing market.

The firm Revery Architecture has designed the towers to echo elements of totem poles and reflect the mountains and sky of the North Shore, Khelsilem said.

Because it is on Indigenous land, the project is relatively free from city constraints.

As a result, planners are looking at providing parking for only 10 per cent of the apartments, far below the usual minimum, and abandoning the city’s preference for towers that sit on podiums of townhouses.

The project will be marketed to people who don’t own cars or are willing to sell them to live downtown.

Instead, the 11 towers will rise straight up. That will leave 80 per cent of the land available for publicly accessible space, some of which will be parks.

Khelselim said reducing the amount of parking so significantly will result in huge cost savings for everyone who eventually lives there, since an individual car stall would cost $80,000 to $120,000 to build.

Instead, the project will be marketed to those who don’t own cars or are willing to sell them to live downtown. The nation’s planning team is also looking at how to get a streetcar running on a former rail line that borders the Squamish land.

The team still has to negotiate with the city over water, sewer and waste-removal services, but the general idea is that the Squamish will collect taxes on all of the units, including the half owned by Westbank, and pay for services with that.

Khelselim said the project will provide a lot of opportunities for work for Squamish members in the expected six-year construction period.

Building could begin in 2021 if the project is approved in the referendum.

 

 

VANCOUVER
SPECIAL TO THE GLOBE AND MAIL
FOR SUBSCRIBERS

The current plan for the Senakw project by the Squamish makes it potentially the densest development in Metro Vancouver.

Major developments in Vancouver normally generate tens of millions of dollars’ worth of amenities for the community, including affordable housing. But the city will have almost no leverage to get any of that from a massive new project planned by the Squamish Nation.

The 11-tower project, which has doubled in size since it was first proposed, will dramatically alter a swath of Vancouver’s downtown waterfront.

The project is to have 6,000 mostly rental units and is planned in partnership with Vancouver developer Ian Gillespie of Westbank Corp. Any other project like it would typically generate an automatic requirement for 20 per cent of the units to be affordable, along with other amenities such as child-care buildings, heritage and social programs.

But Mayor Kennedy Stewart said repeatedly Tuesday that the city has little power to ask for anything like that.

“The only real say we have is on the [infrastructure] service agreement,” said the mayor, who stressed that he is 100-per-cent supportive of the development and believes it will help with the city’s efforts at reconciliation with First Nations groups.

“There’s a constitutional issue here because those are reserve lands and they are not subject to city processes.”

When the former 1986 Expo lands were being planned for development in the 1990s, former mayor Gordon Campbell introduced a policy requiring developers to contribute amenities for the whole community in exchange for getting significant density and, therefore, condo space to sell.

The current plan, reported by The Globe and Mail on Tuesday, for the Senakw project by the Squamish makes it potentially the densest development in Metro Vancouver, at 1,277 units per hectare. Condo towers in the most dense section of downtown – the 1300-block Richards, 900- and 1000-block Seymour – come in at 1,058 and 833 per hectare, respectively.

But because this development is taking place on Squamish reserve lands, the city’s only negotiating tool is its control of the significant new services that will be needed in the area – water, sewer, roads, police and fire protection. Currently, the land is little-used, sitting next to Vanier Park on Kitsilano Point.

In comparison, when Mr. Gillespie got approval for a massive condo project at the Oakridge Centre mall that he is developing with QuadReal Property Group, the team was asked to provide about $148.8-million in community amenities, including 290 units of social housing, a nine-acre park and a 100,000-square-foot civic centre that will include a seniors’ centre, library and child-care space. As well, they had to provide another 290 units of market rental housing.

On Tuesday, Mr. Stewart said the huge increase in rental supply from the Squamish project will benefit the city.

However, the city cannot impose any requirements about the type or cost of the housing.

Mr. Stewart said he had early discussions with Squamish members about the possibility of some lower-cost housing in the project. But those speaking for the nation have made it clear that it’s not the responsibility of the Squamish to provide subsidies for Vancouver residents’ housing.

Khelsilem, the Squamish Nation councillor who has been the main spokesperson for the project, said the planning team at the council is considering how to provide some below-market apartments for Squamish members.

The inability of the city to have much control over the project is concerning to at least one city councillor.

While some, such as OneCity’s Christine Boyle, say the nation has the right to do whatever it wants with its own land, Non-Partisan Association Councillor Colleen Hardwick said it’s concerning there is no ability to impose any rules.

“There’s no oversight. I have big concerns about that,” said Ms. Hardwick, who called the project oversized, over-the-top and “inconsistent with the values of a livable city.”

14 Nov 05:25

Windows 10 1909 starts rolling out

by Volker Weber
Today, Microsoft is starting to make 1909 (Build 18363.418) available to "seekers," meaning those who proactively go to Windows Update, check for updates and subsequently opt to download it immediately. As it has been doing with recent updates, Microsoft will be throttling delivery, providing the new feature update over the next few weeks to users with devices that Microsoft has deemed most likely to accommodate the 1909 feature update successfully.

I have been running 1909 on all machines for at least a month now. No complaints.

More >

14 Nov 05:25

The Google Squeeze

by Ben Thompson

In 3Q 2014 Google had $16.5 billion in revenue and $2.8 billion in profit. I proceeded to write an article entitled Peak Google. Fast forward to last quarter, and Google had $36 billion in revenue and $6.7 billion in profit, increases of 118% and 139% respectively. It is difficult to imagine being more wrong!

For the record, my thesis was not that Google’s revenue and profit growth were over; rather, like Microsoft in the 2000s, Google would continue to grow but that its relevance had peaked, in large part because brand marketing would become much more important on the web.

Frankly, this explanation makes things worse in a way: it is certainly true that some types of advertising have, as I predicted, worked much better on platforms like Facebook or Instagram (and it’s also true that Facebook gave up on competing for advertising on 3rd-party sites); it is also worth noting that much of that advertising is less traditional brand advertising, meant to increase brand affinity for future conversions, than it is demand-generating direct advertising (as opposed to Google’s demand-capturing direct advertising in Search). What truly misses the mark, though, is the suggestion that Google’s relevance has in any way decreased.

Five Years of Growth

I have owned up to getting the Peak Google article wrong in the Daily Update, particularly this post in 2017, but in the interest of accountability — and, naturally, this article — a quick review is in order.

First, I should have been clear from the get-go that the analysis did not apply to YouTube. Not only is YouTube a natural fit for brand advertising, which is traditionally video-based, it was also barely monetized at that point; clearly significant growth was coming from that property alone.1

Second, despite the fact I spent most of the early years of Stratechery writing about mobile and the extent to which its impact was underrated, particularly by those in the United States who had already adopted personal computers, I underrated mobile’s impact! First, mobile dramatically increased the number of users Google served in both developed and developing countries. Second, mobile dramatically increased usage from existing users, as the Internet was now in people’s pockets or purses, not only their desks or backpacks. Google’s market was in the process of getting much larger.

The biggest mistake, though, was in underestimating just how far Google could go in terms of showing users more ads, even once you accounted for more users using Google more often.

MOAR Ads

The first and most obvious way that Google showed users more ads was by literally inserting more ads into mobile search results. This was a development I tracked closely, wondering just how far the company would go as it added first a third ad, and then a fourth. I wrote in that Daily Update:

Admittedly, this observation has been largely critical: is Google’s revenue growth due to actual increased engagement or simply due to stuffing the screen with ever more ads? In fact, the appropriate answer is “Who cares?” I suspect my disproving attitude stemmed from Peak Google: the company continued to defy my general narrative, so surely the cause must be something untoward like a modern paid inclusion model. And, well, that’s kind-of-sort-of what it is…

That certainly may be vaguely distasteful — it was much more edifying for Google to argue, as they did a decade ago, that folks simply using the Internet meant they made more money — but that doesn’t mean it isn’t effective, and frankly, I haven’t given it enough credit.

Just as important, though, is the way in which Google responded to the threat posed by vertical search alternatives. Back when I wrote Peak Google, there was a lot of talk that mobile was a problem for Google because the new app paradigm would make it more likely that end users would go around Google. They would use Yelp for local search, Amazon for shopping, or Expedia for travel; sure, URLs and bookmark managers may have been too confusing for most users, but the “App Revolution” meant a vertical search engine was only a tap away.

In fact, though, this threat ended up being overstated for several reasons.

First, it turned out that users didn’t want to juggle multiple apps any more than they wanted to juggle multiple URLs. Search in the built-in browser was still the easiest and most obvious place to start.

Second, Google was willing to pay whatever was necessary to ensure it was the default search engine for those built-in browsers, including billions of dollars on an ongoing basis to Apple.

Third, Google started to transform mobile results in particular to be much more useful: instead of forcing users to click a link for an answer — something mobile users dislike about as much as downloading new apps — Google would give it to them; they didn’t even need to “feel lucky”. Most importantly, though, when it came to vertical search categories, Google would offer an entirely new kind of results page.

Expedia and TripAdvisor

This reason to discuss this topic now is because of disappointing earnings results from Expedia and TripAdvisor. From CNBC:

Shares of Expedia and TripAdvisor both reached new year-to-date lows during midday trading on Thursday, tumbling as much as 25%. The stock plunge comes after both the travel service stocks reported third-quarter earnings misses after the bell Wednesday. Both companies pointed to weakened visibility in Google search results as a long-term revenue headwind.

Expedia CEO Mark Okerstrom said on Expedia’s earnings call:

What we saw was a continued shift of essentially the free links further down the page, by other modules that were inserted and ultimately a shift of traffic from the SEO channel over to some of the other products whether it’s flight metasearch or hotel metasearch over time. Now of course as related to the hotel product, the lodging product, we are able to pick up some of that volume and that ultimately resulted in spending more on sales and marketing than we had otherwise would have. We are happy with the returns that we saw on it, but ultimately, not as good returns as we would see from the SEO channel.

TripAdvisor CEO Steve Kaufer said on TripAdvisor’s earnings call:

We did see some incremental SEO headwinds over the course of the quarter. It’s always hard to know exactly what Google is doing. We think of it as how far down the page are we is our organic result. And I think you’re seeing this across the industry as Google has gotten more aggressive. We’ve been predicting this of course for the past many years. We talked about it on our last call. We know that this SEO piece is an ongoing trend and we’re not predicting that it’s going to turn around.

Expedia and TripAdvisor used to be the same company; they play in closely related spaces. Expedia is an “OTA” — Online Travel Agent — where you can book hotels, plane tickets, etc.; TripAdvisor is focused on reviews but monetizes as a meta-search engine, i.e. by referring users to OTAs (although TripAdvisor, with its “instant booking” product, is not far from being an OTA itself).

OTAs and Aggregation Theory

OTAs have always been a special case when it comes to Aggregation Theory; like Aggregators, they serve customers on a zero marginal cost basis, and they have power over supply (hotels, primarily) by virtue of delivering them demand. The hangup for me is how they acquire that demand: first and foremost from Google.

Expedia’s Google play is straightforward: deliver highly-ranked answers to common queries like “Tickets to Tokyo” or “Hotels in Sydney”, and also become very good at buying search ads. TripAdvisor, meanwhile, leverages its reviews to rank highly on a whole host of terms related to traveling, and then offers booking functionality alongside those reviews.

What is notable in both cases, though, is that it is Google that ultimately owns the customer relationship, which is why I have always hesitated to call OTAs Aggregators:

Google owns demand

This arrangement between OTAs and Google has long been beneficial to both sides. Google drives traffic to the OTAs, which can monetize that traffic via commissions extracted from suppliers.2 Google, meanwhile, not only receives relevant results it could serve to customers, but also makes billions of dollars from OTAs buying search ads.

Google and the OTAs value chain

What has changed, starting with Google’s search results, which then spilled over to these companies’ financial results, is the hotel module:

Google hotel search results

First, note just how many screens you now have to scroll to reach organic results — at least 3 on an iPhone 11 Pro which is 812 points tall.3 Again, this isn’t necessarily new — Google has been adding ads for a while — but what makes the hotel module compelling is that, while it is easy to ignore the ads, the module is genuinely useful! You have a map of the city with prices of various hotels, an opportunity to specify your dates, and several options to click through on.

Here is the rub, though, at least from an OTA perspective:

Google Partners

In case you’re not sure what “Google Partners” means, here is a screen you get when you click on one of those hotels and look at the prices:

Google hotel module listings are ads

Everything in the hotel module is an ad, or perhaps more accurately, paid-inclusion. This particular example does not have Expedia.com (but it does have Hotels.com, which Expedia owns) or TripAdvisor, but they are in others; it is this module that Okerstrom was referring to when he said:

Now of course as related to the hotel product, the lodging product, we are able to pick up some of that volume and that ultimately resulted in spending more on sales and marketing than we had otherwise would have. We are happy with the returns that we saw on it, but ultimately, not as good returns as we would see from the SEO channel.

I would think not; the SEO channel is free, the hotel module isn’t.

Aggregating OTAs

At this point the conclusion seems easy, no? Google being evil, yet again. In fact, while I understand the frustration of Expedia and TripAdvisor, I think it is a bit more complicated.

Start with the theoretical perspective: the stable structure of an Aggregator-dominated market is that the Aggregator controls demand and suppliers come onto the Aggregator on the Aggregator’s terms. In other words, there are three players in the value chain: suppliers—Aggregator—demand. Notably, though, that has not been the case in travel, where Google has controlled demand but OTAs have controlled supply.

One way to achieve equilibrium would be for Google to become the one OTA to rule them all. Indeed, this would be difficult to compete with (and was a fear when Google acquired ITA in 2010). The truth, though, is that OTAs have put in significant effort to bring suppliers on board, and they deal with all of the pesky payment and customer support issues that Google loves to eschew. Instead Google has realized it can get OTAs to effectively pay Google to take care of the messy parts for them.

The Hotel Module in the Value Chain

With the hotel module, Google captures demand more efficiently, which not only makes Google search more attractive to end users, but also transforms OTAs into suppliers, paying to provide the service that Google doesn’t want to. It is a textbook example of what Tren Griffin calls Wholesale Transfer Pricing:

Wholesale transfer pricing = the bargaining power of company A that supplies a unique product XYZ to Company B which may enable company A to take the profits of company B by increasing the wholesale price of XYZ.

In this case the unique product is demand — users. And this is where I am tempted to defend Google: at the end of the day, the company has the dominant position in its value chain largely by providing a better product. Search was better to start, but Google didn’t rest on its laurels: it made search better on mobile in particular with these sorts of modules, and while users could download another app or go to a different URL, they simply don’t want to.

At the same time, I get the frustration of the OTAs specifically and all of Google’s suppliers generally: if not even four ads will deter users, and if Google is going to pay whatever it takes to be the default search engine, then isn’t it unfair for the company to collect rent in this way?

Competing With Google

Here it is worth at least considering the biggest OTA of all, Booking Holdings. The company reported its earnings a day after Expedia; from Morningstar:

Booking Holdings Inc. reported third-quarter results that beat expectations. Profit at the online travel site was $1.95 billion, or $45.54 a share, up from $1.77 billion, or $37.02 a share, a year earlier. Adjusted earnings were $45.36, up 20% from a year earlier. Analysts polled by FactSet were expecting $44.50 a share. Revenue was $5 billion, up from $4.8 billion a year earlier. Analysts were expecting $4.85 billion.

Booking Holdings CEO Glenn Fogel argued on the company’s earnings call that the company was relatively insulated from Google’s actions:

Regarding SEO, we saw some headwinds in the SEO channel that did create some modest pressure, but it’s a small channel for us.

Fogel added later on:

In the end, what’s most important for us to get customers to come to us directly. We’ve talked about this a lot in the past. It’s one of the things that I think is very important. For us to have our own future is to create a service that is so wonderful, so good that people just naturally will come back to us directly. And we will not be as dependent on other sources of traffic.

This seems like unequivocally a good thing, no? Booking knows it can’t depend on the Google channel, that its future is best secured by innovating and building a customer experience that convinces users to go to Booking directly. That is competition working to the benefit of customers!

I had a similar thought while reading this profile of Yelp CEO Jeremy Stoppelman; the ‘hotel module’ was long ago preceded by the ‘local module’ on Google search, much to Yelp’s consternation. What always gave me pause about Yelp’s complaints, though, is that, as I noted earlier, the company was at one time held up as the canonical threat to Google on mobile; why didn’t the company earn more direct customers, and instead spend so much time and energy kvetching about Google’s search results? That is why I found this bit of that profile compelling:

It also surfaces reviews algorithmically via recommendation software. It segregates reviews that the software flags for being solicited or biased, or because it doesn’t know enough about a user. Which means Yelp hides almost 30% of the reviews posted to its site, according to the company. This review filter is, to put it mildly, enormously unpopular among businesses…

“I’m sure we could have been making a lot more money if we allowed ourselves to be compromised and just said: Anything goes on Yelp. You want 5 stars? Tell your friends to go write a bunch of reviews for you and they’ll be on Yelp and then you can advertise. And wouldn’t it be wonderful?” said Stoppelman.

Instead, Yelp went another route. It is vigilant about reviews, and has passed on some easy ways to make money from users’ data. It doesn’t let businesses target users who happen to be walking by with an ad, for example. Despite persistent rumors, it’s hard to imagine Yelp fitting in as an acquisition target for Big Tech — in just two interviews with BuzzFeed News, the outspoken Stoppelman took shots at Facebook, Amazon, and Google…

“When I look out at other companies,” Stoppelman said, “I see other priorities, namely growing revenue as much as possible. So why didn’t Facebook crack down on certain types of content, or why did they allow sensational stories or stories that are not true to blast across the network and get amplified so much? Had they had the foresight to say, ‘Hey, this is bad for the world’ or ‘This is bad for our long-term brand, we should shut it down,’ it probably wouldn’t have turned into an eventually traumatic political issue.

“But at the end of the day, collecting attention is the way that they make money, and they dial up the algorithm — the same as YouTube, same for Google. You know, it’s like Google and Facebook did the same thing: Use the algorithm to optimize for maximum attention. And if you optimize for maximum attention, you’re leaning into human nature of rubbernecking at train crashes, and all the worst stuff that humanity can provide. And that’s where you end up. And I’m sure it was like rocket fuel for their business, but now we’re paying the price.”

This is by far the most compelling pitch I have heard Yelp give for itself: “The big companies are full of spam and misinformation, while we take the time to get reviews right.” It is hard not to wonder just how much more popular Yelp’s product might be if this message were spread as stridently as its anti-Google arguments.

And, of course, there is Amazon: more product searches start on Amazon than Google, not because Amazon spent its energy complaining about Google favoring its own shopping results, but because Amazon went out and delivered a better experience for users.

Monopoly Concerns

I remain very concerned about monopoly, particularly, when it comes to consumer tech, digital advertising; this Wall Street Journal story is an excellent overview of how Google makes it extremely difficult to compete (for competitive ad-tech companies) and extremely difficult to go elsewhere (for its customers).

What gives me pause about search, on the other hand, is that there are not constraints on user movement. It really is trivial to use Yelp, or Amazon, or Booking, both on the web and on a smartphone. Is customer inertia something that requires regulation, or is it a possible spur to making products that are that much more compelling?

One answer, perhaps, lies in Google’s behavior itself: unlike traditional monopolies, it is hard to argue that Google’s product isn’t getting better. Sure, OTAs need to pay to play on the hotel module, but the hotel module is a genuine improvement over 10 blue links. The same can be said of the other areas where Google gives answers instead of options. I absolutely get the argument that this might be an unfair extension of Google’s search dominance, but the possibility of stifling innovation, both directly and also its incentives, are worth consideration.

I wrote a follow-up to this article in this Daily Update.

  1. It remains a big problem that we don’t know exactly what YouTube’s financials are; if Google won’t tell us the SEC should make them
  2. I’m using “Commissions” here broadly; there are multiple monetization models for OTAs, including selling rooms directly, charging hotels fees after-the-fact, etc.
  3. A “point” is the functional equivalent of a pixel in the user interface; an iPhone 11 Pro has a 3x retinal display which means that three physical pixels represent one “point”
14 Nov 05:24

Stuff I’ve been reading (October 2019)

Things I finished reading in October 2019:

Books

  • Braun, Lundy. Breathing race into the machine: The surprising career of the spirometer from plantation to genetics. University of Minnesota Press, 2014.
  • Clare, Eli. Brilliant imperfection: Grappling with cure. Duke University Press, 2017.
  • Duster, Troy. Backdoor to eugenics. Routledge, 2004.
  • Eyal, Gil. The Autism Matrix. Polity, 2010.
  • Foucault, Michel. The history of sexuality: An introduction, volume I.* Trans. Robert Hurley. New York: Vintage, 1990.
  • Koopman, Colin. How we became our data: A genealogy of the informational person. University of Chicago Press, 2019.
  • McGuire, Anne. War on autism: On the cultural logic of normative violence. University of Michigan Press, 2016.
  • McHoul, Alec, and Grace, Wendy. A Foucault primer: Discourse, power and the subject. Melbourne University Press, 1992.

Papers

  • Ebben, Hannah. “The desire to recognize the undesirable: De/Constructing the autism epidemic metaphor and contagion in autism as a discourse.” Feminist Formations 30.1 (2018): 141-163.
  • Friedner, Michele. “Deaf bodies and corporate bodies: new regimes of value in Bangalore’s business process outsourcing sector.” Journal of the Royal Anthropological Institute 21.2 (2015): 313-329.
  • Friedner, Michele. “Deaf Capital: An exploration of the relationship between stigma and value in deaf multilevel marketing participation in urban India.” Medical anthropology quarterly 28.4 (2014): 502-518.
  • Goodley, Dan. “Understanding Disability: Biopsychology, Biopolitics, and an In-Between-All Politics.” Adapted Physical Activity Quarterly 35.3 (2018): 308-319.
  • Hacking, Ian. “Between Michel Foucault and Erving Goffman: between discourse in the abstract and face-to-face interaction.” Economy and society 33.3 (2004): 277-302.
  • Jenkins, Richard. “Erving Goffman: A major theorist of power?.” Journal of Power 1.2 (2008): 157-168.
  • Kitchin, Rob. “Thinking critically about and researching algorithms.” Information, Communication & Society 20.1 (2017): 14-29.
  • Mallett, Rebecca, Katherine Runswick-Cole, and S. Timimi. ”The commodification of autism.” Re-thinking Autism: Diagnosis, Identity and Equality (2016): 110.
  • Roscigno, Robin. ”Neuroqueerness as Fugitive Practice: Reading Against the Grain of Applied Behavioral Analysis Scholarship.” Educational Studies 55.4 (2019): 405-419.
  • Seaver, Nick. “Algorithms as culture: Some tactics for the ethnography of algorithmic systems.” Big Data & Society 4.2 (2017): 2053951717738104.
  • Stark, Luke & Hoffmann, Anna Lauren, ”Data Is the New What? Popular Metaphors & Professional Ethics in Emerging Data Culture”. Journal of Cultural Analytics. May 2, 2019. DOI: 10.31235/osf.io/2xguw
14 Nov 05:21

Orbit DB

OrbitDB, Nov 12, 2019
Icon

Something that's bothered me for a long time has been how to have a local database that syncs with an online or cloud database. Orbit addresses this. From the website: " OrbitDB is a serverless, distributed, peer-to-peer database. OrbitDB uses IPFS as its data storage and IPFS Pubsub to automatically sync databases with peers. It’s an eventually consistent database that uses CRDTs for conflict-free database merges making OrbitDB an excellent choice for decentralized apps (dApps), blockchain applications and offline-first web applications." I haven't tried it yet, but it's on my list.

Web: [Direct Link] [This Post]
14 Nov 05:19

A Universal “Hostless” Substrate for a Post-Serverless Future

Brooklyn Zelenka, Vimeo, Nov 12, 2019
Icon

This is a smart, detailed and forward-looking presentation delivered at the Scandanavian Developers' Conference this past week. It looks at the concept of 'universal computing' from the perspective of three major innovations: global storage systems (including, for example, IPFS), digital security and self-sovereign identity, portable computing where you provide the data and identity as needed, and the application provider never actually has access to these. A lot of this is related to some of the stuff I've been talking about with respect to content-addressable resource networks and the like. It's hard to imagine the world of Facebook and Twitter being overturned, but Brooklyn Zelenka is talking about a 2-5 year time frame, so be ready. You can also view her slides here in Noti.st

Web: [Direct Link] [This Post]
14 Nov 05:18

consolidation? meh.

This is an answer to a question I got about adtech consolidation.

The Wall Street Journal ran a great piece by Keach Hagey and Vivien Ngo, all about how Google is finally helping to fix the design of that terrible "Lumascape" slide, by letting them add some whitespace and make the logos bigger.Who says Google doesn't do great things for graphic design? Here it is: How Google Edged Out Rivals and Built the World’s Dominant Ad Machine: A Visual Guide - WSJ

So, is adtech consolidation bad for publishers?

Not really. Adtech consolidation is overdue. Any normal Internet business would have consolidated a long time ago. (Ever notice you get mostly Zoom meeting invitations now, and not links to rival conferencing systems?)

What matters is the adtech/publisher split.

  • In a sustainable ad-supported business you have a split of about 85% publisher/15% intermediaries.

  • The web is 40%/60% with the publisher getting the small slice.

It doesn't matter if the 60% goes to one company, a few companies, or a Lumascape with thousands of companies. It's still unsustainable for the publisher. When publishers look at the diagrams showing multiple advertisers bidding for an impression on their sites, they're ignoring the other side—the same advertisers bidding for the same user eyeballs on cheaper sites.

Content is commodified, the publisher lacks market power to command a decent price for ads delivered to their own audience, and no tweaking of market share in the adtech business will change that. Real-time bidding works today because too many people have the user's data, and can target them.

Today's big tech companies come at every problem with the same tool: take whatever business is adjacent to me, and try to turn it into a commodity. That works great when your product is an internet service and the business adjacent to you is an operating system. A copy of Linux is a commodity, but it's actually worth more than a copy of HP-UX or Solaris.

But commodification is not the highest value model for the advertising business, which depends on feedback between brand equity and content reputation. Brands are worth more in a market with high-value content sites, and ad-supported content sites are worth more in a market with high-value brands. So what matters to the publisher is not the number of adtech vendors participating in your commoditization. What matters is the number of saleable ad impressions that are in direct competition with your ad impressions. (A site trying to sell an ad on a story that cost $10,000 to report, edit and shoot is competing against a site running an ad on a recycled racist cartoon or a pirated Nickelback song that cost them nothing.)

Adtech's job is simple: to facilitate putting impressions on cheap content into the same market as impressions on expensive content. It might matter a little bit how many companies are involved, but that's not the main story. The survival of any ad-supported site depends on market power, and part of reclaiming market power is making it harder or more expensive to reach your same audience on cheaper sites. That depends on privacy tools and regulations.

The good news is that CCPA is coming January 1. CCPA is not just a compliance issue or a cost center. If we handle it right, it's a way to reclaim some market power by limiting the number of saleable ad impressions on low-reputation sites.

Bonus links

Targeted ads are one of the world's most destructive trends. Here's why | Arwa Mahdawi

How big tech is dragging us towards the next financial crash

‘There is a trade-off’: How Immediate Media is preparing for a cookie-less future

Advertisers emerge as buyers for ad tech

The Death of the Rude Press

Google's civil war

It’s shockingly difficult to escape the web’s most pervasive dark patterns

14 Nov 05:18

“They are indistinguishable”

by Andrea

NPR: Math Looks The Same In The Brains Of Boys And Girls, Study Finds.

“There’s new evidence that girls start out with the same math abilities as boys.

A study of 104 children from ages 3 to 10 found similar patterns of brain activity in boys and girls as they engaged in basic math tasks, researchers reported Friday in the journal Science of Learning.

“They are indistinguishable,” says Jessica Cantlon, an author of the study and professor of developmental neuroscience at Carnegie Mellon University.

The finding challenges the idea that more boys than girls end up in STEM fields (science, technology, engineering, and mathematics) because they are inherently better at the sort of thinking those fields require. It also backs other studies that found similar math abilities in males and females early in life.”

NPR: New Study Challenges The Assumption That Math Is Harder For Girls. “Research shows that when boys and girls as old as 10 do math, their patterns of brain activity are indistinguishable. The finding is the latest challenge to the idea that math is harder for girls.”

“Geary says differences seem to show up later and involve very high-level math tasks. His own research has found that in most countries, female students perform just as well as male students in science-related subjects. Yet paradoxically, Geary says, females are less likely to get degrees in fields like math and computer science if they live in wealthier countries with greater gender equality.”

12 Nov 04:21

North Shore Housing? Thanks Tsleil-Waututh and Squamish!

by Barry Rueger
mkalus shared this story from Price Tags.

If you’ve been following the plans by the Squamish nation to build 6000 units of housing near the Burrard Bridge, you’ll appreciate the sheer bravado of the local Tsleil-Waututh Nation.

Instead of waiting years for a District of North Vancouver council to finally approve a significant housing development, they’ve applied to the federal government to add the 45 hectares of the target property to their reserve lands. This would mean they could proceed without council approval.

Or, as one grouch on Twitter described it:

Last year Darwin Properties and the Tsleil-Waututh presented an ambitious new project called the Maplewood Innovation District, defined as “a new district offering an innovative mix of employment, education, recreational and limited residential and community uses in a campus-style structure.”

It would have built student and rental housing, commercial and studio space, and a variety of amenities on largely vacant land between the Mount Seymour Parkway and the Dollarton Highway. The proposal described plans for “a mix of housing tenures including 680 rental housing units (450 non-market and 230 market) and 220 market ownership units to co-locate jobs and residents and to support employment-generating uses.“


In July of 2018 the outgoing council chose to punt the proposal ’til after the election. The new council, who seem to be opposed to any development larger than coachhouses, quickly rejected it.

A second, much reduced version was also deferred in May of this year “until after the OCP review.” It seems likely though that opinions like those of council member Lisa Muri played a significant part in that decision. She was quoted in the North Shore News as saying:

“I didn’t use to converse with [Darwin] and now I do. Keep your enemies close, is what my initial thought was,” she said. Still, she said the project felt exclusionary for anyone who wasn’t a young tech employee on a bicycle.

“I felt like it was a party I wasn’t invited to,” she said.

The Tsleil-Waututh have now applied to the Government of Canada to annex the land and add it to their reserve holdings, which essentially moves it out of the reach of District council. (Aside from negotiations on shared infrastructure, like water and sewer.)

Last month, the adjacent Squamish nation announced their own plan to build 1,000 units of new housing, much of it geared to income, on the North Shore and in Squamish. The Hiy̓ám̓ ta Sḵwx̱wú7mesh Housing Society specifically wants to build housing for their own members but the development will still impact the rest of the North Shore housing market.

More than one pundit has suggested that the reason for the incredible traffic jams at both North Shore bridges is because so many people who work on the North Shore can’t afford to have a home here.

I guess it turns out that the biggest obstacle to settling on the North Shore was, in fact, Settler Culture.

12 Nov 04:17

Last weekend it was 30 years ago the Wall/iron ...

by Ton Zijlstra

Last weekend it was 30 years ago the Wall/iron curtain fell. I sat in front of the tv deep into the night watching German live tv. Having visited Eastern Germany just two years before it, I cried watching. I felt a strong urge to go there, but it wasn’t my place I also felt, not my personal history being made and I didn’t want to be a spectator in their midst. It was my neighbour’s, Rainer’s, history, who fled Eastern Germany with his parents as the iron curtain came up in ’61, while his sister stayed as she was freshly in love. Their lives separated for almost 3 decades, punctuated by his visits as often as he was allowed in. Their parents never being allowed back in. I remember in ’87 getting into an argument with an East-Berlin civil servant who told us the wall was there as a ‘Anti-Fascist Protection Rampart’ against the/us fascists in the West. It was so much cognitive dissonance for me that the Wall was supposedly there to keep me out, but yet here I was let in, just not them let out. A prison with the key on the inside. He couldn’t acknowledge my comment obviously, which I couldn’t in turn accept as a teenager. Now, as I’ve worked in various places and settings, where diplomacy is the only way forward, allowing others to save face, and room to manoeuvre to get somewhere, I know why there was no way at all I would have ‘won’ being right that day in East-Berlin city hall. Yet I was, empathy against bureaucracy is/should always be right.

Last weekend it was also 81 years ago that Nazis roamed the streets in Germany trashing Jewish owned businesses, homes and synagogues, and the murder of hundreds and internment in concentration camps of some thirty thousand German citizens of Jewish faith, Kristallnacht.

Oddly enough it is in the regions most affected by the separation of the Wall, Thuringia and Saxony, that the fascist echoes ring the loudest in Germany these days, where AfD. ‘alternative for Germany’ finds an electoral feeding ground. Or maybe it is not surprising, as societal complexity may make you yearn for the (imagined) simplicity of something that never was, exchange the simplicity of one authoritarian construct for another, or at least to blame someone, anyone, some mythical Other, for the baffling complexity around us.

All three of these things are important and current to my own life in different ways. As is the next.

Today I was in Brussels, a very complicated city in an equally complicated country in its own right. Whatever you can say about this place, and here too nationalist (or rather regionalist) sentiments boil, there is beauty in the elegance of dealing with otherness here. Where you are being spoken to in French or Dutch, and answer in a mixture of French, Dutch and English, or vice versa, and it’s all fine. Where the hotel barman tonight has a distinctive Flemish/Dutch name, Gert, yet is Walloon, and we both shrug and say, yeah Europe is complicated, and we appreciate each other’s efforts to be understood and embrace how our lands have been the crossroads of so many different things. It’s fitting that the EU has its institutions in both Brussels and Luxembourg, the two linguistically most confused/mixed cities in Europe.

E and I often remark to each other when we encounter situations like me and the barman above how ‘Europe works’. Last time I was here in Brussels, over dinner I sat next to a family of 4 who amongst themselves effortlessly conversed in Italian, Dutch and German, while fluently ordering in French. I texted to E that very phrase, “Europe works”, not for the first time. It’s our regular shorthand for what the EU has achieved, starting from co-governing the coal and steel works of 6 nations in 1951 so that none of us could build up a war machine without the others being able to stop it, to what the EU is now and how it plays out practically in the lives of us, our networks and the people we encounter across the continent. I intimately know the divisions national borders created within my own family, as well as the deep pain on all sides and resentment of World War II. Equally, I deeply know in my bones what we’ve all gained when freedom of movement kicked in 27 years ago (it’s a tangible sensation every time I personally or professionally do something where there before was a fence), as well as how it makes my professional life possible. Yet across the EU, which from my travels around the world I know to be a place of such enormous abundance (which is not a synonym for perfection nor utopia), resentment against those gains has built. While I recognise the things that feed into that resentment, all too often it smacks of Kristallnacht. It reeks. German rock-band BAP, singing in the Cologne dialect which in itself is a testament to the age-old connection between my own lands of origin and Germany (I can use may own Dutch dialect deep into Germany without issue of being understood, and Cologne’s dialect is akin to my own), sketches out the political pantomime of the copy-cat fascists perfectly. Their 1982 song is just as pertinent in 2019.

Trust your nose. Your nose points out the arsonists. You’ll know when et rüsch noh Kristallnaach, when it reeks of Kristallnacht.
My nose makes me an EU citizen.

12 Nov 04:15

Remembrance Day 2019 at Victory Square, Vancouver

by ahamedia
12 Nov 04:15

Weeknotes: Python 3.7 on Glitch, datasette-render-markdown

Streaks is really working well for me. I’m at 12 days of commits to Datasette, 16 posting a daily Niche Museum, 19 of actually reviewing my email inbox and 14 of guitar practice. I rewarded myself for that last one by purchasing an actual classical (as opposed to acoustic) guitar.

Datasette

One downside: since my aim is to land a commit to Datasette master every day, I’m incentivised to land small changes. I have a bunch of much larger Datasette projects in the works - I think my goal for the next week should be to land one of those. Contenders include:

I’m going to redefine my daily goal to include pushing in-progress work to Datasette branches in an attempt to escape that false incentive.

New datasette-csvs using Python 3.7 on Glitch

The main reason I’ve been strict about keeping Datasette compatible with Python 3.5 is that it was the only version supported by Glitch, and Glitch has become my favourite tool for getting people up and running with Datasette quickly.

There’s been a long running Glitch support thread requesting an upgrade, and last week it finally bore fruit. Projects on Glitch now get python3 pointing to Python 3.7.5 instead!

This actually broke my datasette-csvs project at first, because for some reason under Python 3.7 the Pandas dependency used by csvs-to-sqlite started taking up too much space from the 200MB Glitch instance quota. I ended up working around this by switching over to using my sqlite-utils CLI tool instead, which has much lighter dependencies.

I’ve shared the new code for my Glitch project in the datasette-csvs repo on GitHub.

The one thing missing from sqlite-utils insert my.db mytable myfile.csv --csv right now is the ability to run it against multiple files at once - something csvs-to-sqlite handles really well. I ended up finally learning how to use while in bash and wrote the following install.sh shell script:

$ pip3 install -U -r requirements.txt --user && \
  mkdir -p .data && \
  rm .data/data.db || true && \
  for f in *.csv
    do
        sqlite-utils insert .data/data.db ${f%.*} $f --csv
    done

${f%.*} is the bash incantation for stripping off the file extension - so the above evaluates to this for each of the CSV files it finds in the root directory:

$ sqlite-utils insert .data/data.db trees trees.csv --csv

github-to-sqlite releases

I released github-to-sqlite 0.6 with a new sub-command:

$ github-to-sqlite releases github.db simonw/datasette

It grabs all of the releases for a repository using the GitHub releases API.

I’m using this for my personal Dogsheep instance, but I’m also planning to use this for the forthcoming Datasette website - I want to pull together all of the releases of all of the Datasette Ecosystem of projects in one place.

I decided to exercise my new bash while skills and write a script to run by cron once an hour which fetches all of my repos (from both my simonw account and my dogsheep GitHub organization) and then fetches their releases.

Since I don’t want to fetch releases for all 257 of my personal GitHub repos - just the repos which relate to Datasette - I started applying a new datasette-io topic (for datasette.io, my planned website domain) to the repos that I want to pull releases from.

Then I came up with this shell script monstrosity:

#!/bin/bash
# Fetch repos for simonw and dogsheep
github-to-sqlite repos github.db simonw dogsheep -a auth.json

# Fetch releases for the repos tagged 'datasette-io'
sqlite-utils github.db "
select full_name from repos where rowid in (
    select repos.rowid from repos, json_each(repos.topics) j
    where j.value = 'datasette-io'
)" --csv --no-headers | while read repo;
    do github-to-sqlite releases \
            github.db $(echo $repo | tr -d '\r') \
            -a auth.json;
        sleep 2;
    done;

Here’s an example of the database this produces, running on Cloud Run: https://github-to-sqlite-releases-j7hipcg4aq-uc.a.run.app

I’m using the ability of sqlite-utils to run a SQL query and return the results as CSV, but without the header row. Then I pipe the results through a while loop and use them to call the github-to-sqlite releases command against each repo.

I ran into a weird bug which turned out to be caused by the CSV output using \r\n which was fed into github-to-sqlite releases as simonw/datasette\r - I fixed that using $(echo $repo | tr -d '\r').

datasette-render-markdown

Now that I have a releases database table with all of the releases of my various packages I want to be able to browse them in one place. I fired up Datasette and realized that the most interesting information is in the body column, which contains markdown.

So I built a plugin for the render_cell plugin hook which safely renders markdown data as HTML. Here’s the full implementation of the plugin:

import bleach
import markdown
from datasette import hookimpl
import jinja2

ALLOWED_TAGS = [
    "a", "abbr", "acronym", "b", "blockquote", "code", "em",
    "i", "li", "ol", "strong", "ul", "pre", "p", "h1","h2",
    "h3", "h4", "h5", "h6",
]

@hookimpl()
def render_cell(value, column):
    if not isinstance(value, str):
        return None
    # Only convert to markdown if table ends in _markdown
    if not column.endswith("_markdown"):
        return None
    # Render it!
    html = bleach.linkify(
        bleach.clean(
            markdown.markdown(value, output_format="html5"),
            tags=ALLOWED_TAGS,
        )
    )
    return jinja2.Markup(html)

This first release of the plugin just looks for column names that end in _markdown and renders those. So the following SQL query does what I need:

select
  json_object("label", repos.full_name, "href", repos.html_url) as repo,
  json_object(
    "href",
    releases.html_url,
    "label",
    releases.name
  ) as release,
  substr(releases.published_at, 0, 11) as date,
  releases.body as body_markdown,
  releases.published_at
from
  releases
  join repos on repos.id = releases.repo
order by
  releases.published_at desc

In aliases releases.body to body_markdown to trigger the markdown rendering, and uses json_object(...) to cause datasette-json-html to render some links.

You can see the results here.

Releases SQL results

More museums

I added another 7 museums to www.niche-museums.com.

  • Dingles Fairground Heritage Centre
  • Ilfracombe Museum
  • Barometer World
  • La Galcante
  • Musée des Arts et Métiers
  • International Women’s Air & Space Museum
  • West Kern Oil Museum
12 Nov 00:13

Oristano, Sardinia

Stephen Downes, Flickr, Nov 11, 2019

Web: [Direct Link] [This Post]
12 Nov 00:13

Painting with CSS

Jason Kottke, kottke.org, Nov 11, 2019
Icon

Jason Kottke writes, " Believe it or not, the image above was made using only HTML & CSS by developer Diana Smith. It’s coded by hand and built for Chrome — you can check it out here. The source code and accompanying CSS is not as extensive as you might think." Even funnier is the same page rendered in Internet Explorer. Yes, it's as bad as you thought.

Web: [Direct Link] [This Post]
12 Nov 00:13

The Paradox of Self-Consciousness

Markus Gabriel, Edge, Nov 11, 2019
Icon

Markus Gabriel outlines the position of 'New Realism', which on the face of it isn't as realist as we might thing. There are, he writes, two fundamental tenets. "Number one is that we can know reality as it is, in itself.... we know how things are. That's claim number one. There is actual objectivity." That's probably a common-sense sort of realism that most people hold. However, "the second claim... is that the world does not exist. What I mean by this is that there is no single unified account of all the facts." For example, there's no fact of the matter regarding whether "there are unicorns in movies." There's no fact of the matter regarding whether "objects in virtual reality are real." This challenge is deeper than it may appear at first glance. Can we know something if we don't know everything? Can we know anything about our selves if we don't know everything?

Web: [Direct Link] [This Post]
11 Nov 23:47

The Best Headphones for Running

by Lauren Dragan
The Best Headphones for Running

Whether you’re prepping for your annual marathon or lacing up sneakers for the first time, the Plantronics BackBeat Fit 3150 wireless earbuds are the running headphones you should be wearing. They don’t seal out the surrounding world, so cars and people can’t sneak up on you. With seamless controls and no cable connection, these true wireless earbuds provide the most satisfying experience of the 40-plus running headphones we’ve tested.

11 Nov 23:34

The Sound Of Your Voice

by Anil Dash
The Sound Of Your Voice

Even though I watched the medium of podcasting being created since its inception, I'd always resisted a little bit participating myself. I think I just felt more at home in this format, blogging, as that's where I'd found my voice.

Well, that might be a bit hard to believe now that I've been on lots of different podcasts. But it feels like I'm really finally getting the hang of it, and I hope you'll check out some of the recent conversations that I've been lucky enough to get to visit.


The Weeds: The Internet We Lost

This was a really great conversation with Matthew Yglesias — especially because we didn’t just  go into some old visions of "the good old days of the web" (which never actually existed), but because we got to talk about the evolution from those days — and how the independent web is  resurgent today. You’ll want to hear this!


The New Rules of Work: Communities Are Diverse. Why Isn’t Tech Talent?

This conversation with Kathryn Minshew started by talking about the structural barriers that keep tech from being inclusive, but we also offered some hope about people and companies that are doing the hard work today.


Developer Tea

This is one of the first times anyone's ever asked me about the deeper motivations for why I got into technology in the first place. For all the criticisms of tech that I often have, there's something really meaningful about why I've always loved coding and tech, and we got to dive into it in this two-part interview.


Switched On Pop: Why U Love 2 Listen 2 Prince

Oh, you can't miss this one. With the upcoming release of the Deluxe Edition of Prince's 1999, I went on Switched On Pop to let my funk freak flag fully fly, and we had a blast. You are gonna love this one.

The Sound Of Your VoiceThe Sound Of Your Voice
The Sound Of Your Voice
11 Nov 23:34

1st snowy commute 2019

by jnyyz

The forecast was for 10-15 cm of snow, the first real snow of the season, and so of course all the news coverage went overboard about how bad it might get.

Riding in this morning, it was just starting to snow. Poppy for Remembrance Day.

The ride back around 4:30 was just fine. Downtown, there was barely 3-5 cm of snow, and there were more cyclists out than I expected.

We’ll give this dad a pass for riding on the sidewalk since this section of Wallace is always very narrow.

Even side streets weren’t too snowy yet.

Just enough snow to cover the bike lane markings on Annette.

Ride safe, everyone. Tomorrow will be colder, so it might be icy.

Update: tragically a cyclist was hit near Bloor and Dufferin, which is part of my usual commute home.

11 Nov 23:33

Google Chrome plans to notify users when a site loads slowly

by Brad Bennett

In an effort to push developers to make faster websites, Google Chrome will soon place badges on sites that load slowly.

So far, the tech giant hasn’t laid out any specifics, but it says it may showcase sites that load quickly with a green loading bar on mobile, and sites that load slowly with an icon that says, ” Loading… Usually loads slow.”

This is likely bad news for website developers who don’t want Google to place a ‘Slow’ tag on their page. That said, if it does push sites to load faster, then this is a win for consumers.

These are only examples, and Google says this will take place in a “number of forms and we plan to experiment with different options, to determine which provides the most value to our users,” in a Chromium blog post posted on November 11th.

As Google outlines in the press release, the first step is to look at the website’s historical load times and determine if it loads at a reduced pace all the time. After this, it plans to determine if sites are loading slowly because of network speeds.

Beyond the first two examples mentioned, Google says it might also showcase a website’s typical loading time in the context menu when you click on the hyperlink.

Near the end of the blog post, the company says that “Our long-term goal is to define badging for high-quality experiences, which may include signals beyond just speed. ”

In the long run, this might mean that Google will push users towards sites that run better rather than reputable websites.

Source: Google

The post Google Chrome plans to notify users when a site loads slowly appeared first on MobileSyrup.

11 Nov 23:33

Microsoft’s latest sizzle reels make me want to use its most boring products

by Jonathan Lamont

It started with new videos showing off SharePoint and Windows Terminal. Now, Microsoft can’t seem to stop putting out stunning product videos that make its most boring software look enticing.

The latest videos from Microsoft show off Yammer, Tasks and the Fluid Framework. Again, they all make these rather mundane corporate software offerings look much more exciting.

Take Yammer, for example. Companies use it as a private social networking platform. Microsoft is refreshing it with a new design and improving integration with Teams, SharePoint and Outlook. The new sizzle reel shows off the new design using reams of paper to represent conversations and little buttons to represent avatars. Additionally, the video highlights the program’s new dark mode.

Honestly, it makes Yammer look incredibly fun to use and not at all like the boring corporate social platform that it is.

Similarly, Microsoft Tasks’ video uses glass blocks to represent different tasks. They shimmer and bounce between different Surface products and Office apps. I’ve never wanted to make a to-do list so much in my life.

Finally, the Fluid Framework video, while also stunning, doesn’t really explain what it is unless you’re already familiar with the concept. For those who aren’t familiar, Fluid Framework is Microsoft’s vision for the future of collaborative work.

Instead of compartmentalized apps like Word, for example, Fluid envisions a component-based web document where works can collaborate by adding things like text, graphs and more.

The Verge points out that design agency Tendril produced some of the videos, which show a clear trend in how Microsoft displays and envisions its products.

While the products themselves may not be as exciting as these videos make them out to be, the company is doing an excellent job making people consider them.

Source: The Verge

The post Microsoft’s latest sizzle reels make me want to use its most boring products appeared first on MobileSyrup.

11 Nov 23:33

Apple reportedly plans to release an iPad Pro with dual cameras in 2020

by Patrick O'Rourke
Apple

Apple’s next iPad Pro could feature dual cameras, according to a new report.

Along with an augmented reality (AR) headset and glasses, a new report courtesy of Bloomberg’s often-reliable Mark Gurman indicates that Apple’s 2020 iPad Pro will feature a dual lens camera setup.

Along with the pair of shooters, the new iPad Pro is also tipped to feature a “small hole for the 3D system.” This corroborates an earlier rumour that indicated Apple’s next iPad Pro would feature a time-of-flight 3D sensor.

This 3D sensor is described by Gurman as “the centrepiece” of Apple’s AR plans and seems to be an evolution of the technology currently powering Face ID. The technology will allow users to make 3D reconstructions of rooms, objects and people.

Gurman goes on to say that Apple also plans to add the sensor to its 2020 iPhones, along with 5G functionality.

The same report also discussed Apple’s plans to release an augmented reality headset in 2021, followed by a pair of smart glasses in 2023. Though there are some differences, this information corroborates an earlier report that recently appeared courtesy of The Information.

Additionally, Gurman says that Apple’s upcoming VR and AR focused headset will be focused on gaming, watching videos and virtual meetings.

The headset was reportedly originally slated to be released in 2020 but was pushed back. The Information’s report indicated that Apple’s smart glasses project will release in 2022, though Bloomberg states that smart glasses are slated for 2023.

Given the initial excitement surrounding virtual reality and augmented reality seems to have died down, it will be interesting to see how Apple approaches the technology. The company has a history of taking established categories and building on them in meaningful ways, so there’s a strong possibility Apple will do the same with AR and VR.

Source: Bloomberg, The Information

The post Apple reportedly plans to release an iPad Pro with dual cameras in 2020 appeared first on MobileSyrup.