Shared posts

14 Mar 05:08

COVID-19 Coronavirus Infographic Data Pack

by David McCandless

We made an infographic of the best COVID-19 / Coronavirus charts floating around, plus some of our own – all with the latest data

We’ll plan to keep it updated every few days.

» See the graphic
» Explore the Data: bit.ly/COVID19-DATA

related
» Look at all infectious diseases in context under the Microbescope
» Mountains out of molehills: a timeline of media-inflamed fears
» Our response to the 2009 H1N1 Swine Flu epidemic was the fastest ever (this is day 69)

14 Mar 05:08

Facet: Deduping

When you fire an event into the cloud, can you be sure it’ll only come out again once? It turns out that sometimes they come out more often than they go in. This may or may not be a problem in your application. If it is, there are techniques to help work around it.

“At-least once”

This is a phrase you’ll hear a lot when you hang around with eventing/messaging people, (and cloud people generally). Builders work so hard at making sure everything gets delivered that they can end up doing it more than once.

How can that happen?

Here’s a scenario: Suppose your software wants to retrieve and process events from a bus or topic or stream or whatever the service is called. And suppose you retrieve one, then something goes wrong before you can acknowledge it. For example, the host your code’s running on might have failed. Or your code just crashed (mine never has bugs, but they tell me it happens). Or your software is written in Java and unfortunately went into a 45-second stop-the-world garbage-collection stall.

When any of these happen, the eventing software will get the idea that you didn’t successfully receive the data, and since its primary purpose in life is to deliver reliably, it will try again. Which means you end up processing it twice.

On the way in…

events in

…on the way out.

events out, with dupes

Production note: For pictures of events in action, I’ve adopted this technique, created by Colm MacCárthaigh, of using emojois. It’d be a good all-purpose solution except for I’ve found the emjoi repertoire and representation highly nonportable between Mac and Windows.

Sometimes duplicates aren’t your fault. Suppose one data center gets cut off from all the others in an AWS region. Yes, we try really hard to arrange for multiple redundant connections, but shit happens. A little birdie told that one day a few years ago, a badly-built bridge in Beijing fell down and it had been carrying three different network providers’ fibres.

We call this situation a network partition. When it happens, both sides of the partition will try really hard not to lose any data. The details can get complicated, but duplicate messages are a common result.

Obviously this could be a problem for your app.

What to do?

There are situations where you can ignore the problem. Since duplicates are rare, in an analytics app or anything else that’s doing stats, they’re probably not a big deal. But there are lots of situations where they are.

If your app has a database that does transactions, you’re probably OK, because you can safely remember when you’ve seen each event’s unique ID, and just discard duplicates.

Another useful technique is idempotency. That is to say, structure your application such that API calls can be repeated without changing the result. An example is anything that can be expressed as a pure HTTP PUT request. You can set a field to a given value as many times as you like without doing any damage. Designing an app to work this way is tricky. But it’s an option that’s worth investigating, because having idempotent operations tends to produce apps that are robust in the face of all sorts of common failure scenarios.

Here’s one thing to note: Duplicates are rare, but when they do happen, they tend to come in clusters (think about that bridge in Beijing). I don’t know if that fact is useful in the context of your app, but just in case.

But there are some apps that just can’t live with dupes.

“Exactly once”

That‘s the terminology used for software that comes with built-in de-duping. One example would be SQS FIFO. Upon encountering this capability, you might ask yourself “why don’t I just use this for everything?” It turns out, just as with FIFO, de-duping isn’t free and in fact isn’t particularly cheap.

It can also get kind of complicated and there are more details than you might think. Consider this blog: How the Amazon SQS FIFO API Works. It dives deep on all the details, which ends up taking over two thousand words.

My advice would be to teach your software to live with duplicates, if at all possible. “At least once” systems are just part of the cloud landscape.

14 Mar 05:08

Facet: Point-to-Point vs Pub/Sub

When there’s an event in the cloud, how many different receivers can receive it? There are two plausible answers: Just one, or anyone.

[This is part of the Event Facets series.]

Postman delivers letter

Point-to-point

The idea is that any given message can only be read by one receiver. You might imagine that, as the old photo above suggests, the sender supplies an address, choosing the receiver. But that’s actually a pretty rare scenario. What’s much more common is the way that SQS works.

In SQS, when you send a message, you have to pick which queue you’re sending it on. Any number of receivers can be polling the queue, but only one will receive any given message. The API is clever; once you’ve received a message and processed it successfully, you’re supposed to delete it. There’s a time window after you’ve received it during which nobody else can see it. If the message isn’t deleted in time, it’ll pop back into availability. It’s called the “visibility timeout” and there are APIs for setting and adjusting it.

Think about a back-end system in one of Amazon.com’s warehouses, picking up order events from the Web site. You want enough event readers in the warehouse to keep it busy, and you want to have each order processed by only one of them; so this flavor of point-to-point is just the ticket.

It’s not a rare scenario at all; the Apache Artemis message broker calls it “anycast”.

Publish and Subscribe

Credit: Terje Skjerdal from Høvåg, Norway / CC BY

Pub/Sub

It’s short for “Publish and Subscribe”, which nobody ever says. This is probably the default mode for most of the software that has the word “event” in its name. The idea is, you publish your events onto a Bus or a Stream or a Topic or a Queue — there’s no standard terminology. Then receivers subscribe to the whatever-it’s-called and the events on it are in principle available to all of them.

There are all sorts of variations in how subscriptions work; push vs pull, filter vs firehose; they’re important enough to deserve their own entries in this series.

There are loads and loads of apps in every sector of business that are a good fit for pub/sub. I’d be amazed if there are readers in a business of any size that doesn’t have some running.

At AWS, we have multiple services that do pub/sub: Kinesis, Managed Kafka, MQ, and then my favorite, SNS, which is used by more or less everyone to do more or less anything. The way it works is, you publish messages to at Topic, and then you subscribe receivers to it. They can be SQS queues, Lambda functions, mobile-device messaging, and arbitrary HTTP endpoints. And you can have as many subscribers as you want, SNS will do the fan-out.

14 Mar 05:08

Don’t Explain How Gamification Systems Work To Your Members

by Richard Millington

One client’s community members have been gaming the system.

I suspect they research what platform we use, looked at which behaviors are rewarded, and then grouped together to rise rapidly.

If they hadn’t tried to occupy every position on the leaderboard, they might’ve gotten away with it (at least for a while).

Once members know precisely how points are allocated and what behaviors are rewarded they quickly optimise their behavior to match. This isn’t a good thing. If members know being endorsed as a topic expert is worth 20 points they will find 10 buddies to join and endorse them.

Give vague instructions about how the system works rather than specific ones. Focus on the intent of the system rather than the mechanics of it.

i.e. “If you create content that people like you will earn points” is better than “every like on an answer you create is worth 3 points”

“The best way to earn points is to be helpful to other members and build a good reputation amongst your peers” is a lot better than “each answer you give is worth 2 points”.

The other benefit here is it’s a lot easier to tweak and improve the system if members don’t know precisely how it works.

14 Mar 05:08

iOS 14 and Handwritten Input

by Rui Carmo

This is somewhat uplifting amidst the downpour of bad news.

I read the initial leaks and was particularly interested in both the new watch features (blood oxygen monitoring is particularly interesting at this point) and “proper” mouse/trackpad support, but missed the mentions of PencilKit and handwritten input in standard text fields (which is something I already enjoy thanks to MyScript third-party keyboards and would love to keep using into the future).

Let’s hope they don’t bungle it like they did iOS 12 (and most of 13 so far).


14 Mar 05:08

Coronavirus is our Chernobyl

President Trump Is Unfit for This Crisis. Period. | Jennifer Senior:At a Friday news conference at...
14 Mar 05:08

"Our society is constructed to reward the rich in good times and punish the poor in bad times."

“Our society is constructed to reward the rich in good times and punish the poor in bad...
14 Mar 05:07

Settecani Italy where Taps Turned Water into Wine

by Sandy James Planner

wine-tap

wine-tap

Just the very best story from Settecani  Italy where villagers in this town of 200 people had a wee surprise when they turned on their water taps~on Wednesday March 4, the local premium wine,  a sparkling Lambrusco Gasparossa flowed through the spout instead of water.

Enterprising locals in this village near Modena immediately grabbed buckets to start capturing the wine which is dry and barely sweet.  It turned out that a “defective silo” was leaking wine into the local water system. Since wine has a higher liquid pressure than water in pipes, the town’s residents truly had the best liquid on tap in their homes.

Sadly the mistake was quickly rectified, but not before litres of this well loved local wine had been procured from the taps.

While the local council apologized and ensured everyone knew there was no threat to health with the mature wine coursing through the water system, some residents responded that the problem was fixed too quickly, and wondered if the problem  could not reoccur later in the day.

And the winery? It apologized as well, reassuring residents  that there were no hygienic risks. “It is only wine that was already for bottling” the winery confirmed.

The video below shows what that Italian premium red sparkling wine flowing through kitchen taps looks like.

14 Mar 05:07

Things Debuts Modernized Apple Watch App

by Ryan Christoffel

The Apple Watch has come a long way in five years, and apps are only starting to catch up. Many Watch apps received the majority of their development attention with the first or second versions of watchOS, before the days of LTE service, independence, and SwiftUI. Those early Watch apps were hamstrung by OS limitations, but in the last few years as the platform has evolved, most apps never adapted to what’s possible now.

Things 3.12, releasing today, exists for just that purpose: it addresses the task manager’s former Watch client shortcomings, making it a truly capable companion for Things on iPhone, iPad, and Mac.

Things’ new Watch app has been completely rebuilt from the ground up so that everything just works better than ever before. The UI remains virtually untouched, but the few new features plus under-the-hood improvements make a big difference.

Sync is perhaps one of the biggest areas of frustration I’ve encountered with Watch apps over the last five years. iPhone-to-Watch communication has just never been as solid as it should be, including with Things, where updates to the Watch app depended on its iPhone counterpart. This wasn’t a problem if you only used Things on those two devices, but most users also depend on the iPad and/or Mac versions, and often changes in those places wouldn’t be reflected on your wrist. Now that issue’s solved thanks to Things Cloud coming to the Watch.

Starting today, Things for Apple Watch depends on Things Cloud for its syncing. With a free Things Cloud account, which most Things users will already have, the Watch client now gets its data directly from the cloud rather than relying on its iPhone companion; you’ll see this in action thanks to a helpful cloud icon that indicates syncing is in progress.

Whether you’re near your iPhone or away from it, Things for Apple Watch will stay up-to-date on changes made across all your devices automatically. This change means that another previous frustration – Things’ complications being inaccurate – has been solved too; complications now update in the background based on the latest Things Cloud data.

Alongside Things Cloud’s debut on Apple Watch, there are a few other key new features such as the ability to add new to-dos to Today rather than the Inbox. This has always been the behavior I’ve wanted from Things, so I’m delighted to see the option added via the app’s Settings screen.1 Also, when creating new tasks you can now write them out with watchOS’ Scribble feature rather than being limited to dictation. Finally, if you’re a multi-Watch user Things now works across multiple devices, and you can remove dates from tasks in your Today view via the date picker.


Things was relatively feature-rich on the Apple Watch before, but due to its reliance on older watchOS technologies, it wasn’t able to provide as reliable an experience as Watch users deserve. I’m thankful that’s changed today with the debut of version 3.12. Things for Apple Watch is no longer a second-class citizen to its iPhone, iPad, and Mac counterparts; rather, thanks to Things Cloud integration it’s become just as rock-solid as those other versions.

Things is available on the App Store.


  1. Force-press anywhere in the main view to find the Settings screen. ↩︎

Support MacStories Directly

Club MacStories offers exclusive access to extra MacStories content, delivered every week; it’s also a way to support us directly.

Club MacStories will help you discover the best apps for your devices and get the most out of your iPhone, iPad, and Mac. Plus, it’s made in Italy.

Join Now
14 Mar 05:07

New Things for Apple Watch

by Mick

Since New Year, we’ve been working on a big update – and it’s all about modernizing Things for Apple Watch. All the app’s internal parts have been rebuilt, it now syncs directly with Things Cloud, and we’ve added some much-requested features!

Things for Apple Watch now syncs directly with Things Cloud
Things for Apple Watch – now with cloud sync!

Five Years On

We first created Things for Apple Watch five years ago, when watches were still slow and entirely dependent on your phone for connectivity. In order to get changes from your watch to the cloud, the iPhone app had to act as a middleman, ferrying changes to and from the cloud on the watch app’s behalf. While it seemed like a fine idea at the time, in practice it turned out to be slow and error-prone.

In the years since, the Apple Watch has come a long way. Devices are faster, have more storage, support background refresh, connect to WiFi, and many are free-roaming on LTE. Compared to 2015, it’s a huge leap forward.

In short, we knew that big improvements were possible for Things – and well overdue. So in January, we disassembled our watch app and began replacing all the parts: the sync, data layer, complication integrations, background refresh, Siri integration – everything under the hood is completely new. But without doubt, the biggest change today is the sync: we’ve brought Things Cloud to Apple Watch.

Things Cloud on the Watch

The most important change in this release is that Things on your watch will now sync directly with Things Cloud.

Now
Things for Apple Watch now syncs directly with Things Cloud.
Before
Before, Things on your iPhone acted as the middleman.
Now
Before

Syncing directly with Things Cloud is far superior to the previous implementation, and we couldn’t be more excited to get this into your hands.

Even when you’re away from your phone, the watch syncs instantly with the cloud and you can be confident that you’re always looking at the latest data.

When you enter a project, for example, or a to-do with a checklist, you’ll no longer have to wait for that content to load. Everything is already on your watch and ready to go.

We’ve also improved the quality of data on the watch face. Since both your phone and watch now sync to the cloud, we’re at liberty to choose whichever data is more recent and display that in your complication. After all, you want to be sure when you glance at your wrist that the info’s up to date ⌚️👀

Of course, all of this requires a Things Cloud account. If you don’t already have one, you can create one for free.

More New Features

Even though today’s update is mainly focused on the app’s foundation, we were able to sneak in a few of your most-requested features.

Other new features
  • Add to Today

    By default, the watch app saves new to-dos to your Inbox; you can now set Today as the default instead. To access this setting, open the watch app and force press in the main list.

  • Remove from Today

    Many of you complained that there was no way to remove a to-do from Today – that is to say, put it in Anytime. You can now do this by tapping Clear in the date picker.

  • Scribble support

    When adding new to-dos, or appending some notes, you can now scribble instead of speaking the text 👆

  • Multiple watches

    Incidentally, the new implementation of sync brings another benefit: many of you wrote in saying you use a spare watch for sleep tracking and really missed having Things on it. Now you can!


If you love your Apple Watch, we think you’ll love Things 3.12. The addition of Things Cloud makes a world of difference and we find ourselves using the app more than ever. Take it for a spin today and let us know what you think!

14 Mar 05:07

Worry if nobody worries :: And vice versa

by Volker Weber

14 Mar 05:07

Making Working From Home Work

by Andy Abramson

Two days ago, I wrote about the free offers coming out of telecom companies to help with the Coronavirus situation. Today, I'm seeing that longtime friend, repeat client, and leader at Dialpad, Craig Walker, has penned a nice piece on what Dialpad is doing in light of the Coronavirus with regards to his staff around the globe as well as showing how they are helping businesses adapt to the workplace changes brought on by Corona. As a shareholder in Dialpad, Craig's candor and approach to what he and his team are doing for their staff really comes through.

The reasons for both messages are clear and simple. All companies have to keep going, as they all have lots to lose, and like so many others, Dialpad can use what they sell to keep their staff working. Equally rewarding to me is to see how Craig and the Dialpad team have basically resurfaced "Dialpad Free" a program I helped launch back in the early winter of 2017 as way of attracting new business. 8x8 has done something similar pushing out a reminder this week about their free video meetings service they announced back in November.

SignalWire also has a good post on being experienced via FreeSwitch when it comes to working from home.

 

 

 

14 Mar 05:06

Why Slower Cities Matter & Why the Provincial Government Got This Wrong

by Sandy James Planner

In a discussion with CBC Radio’s Gloria Macarenko talking about British Columbia’s new speed cameras, I noted that traffic congestion is actually good for you, as it mitigates carbon emissions with slower speeds. The Insurance Institute for Highway Safety (IIHS) also found that slower speeds reduce the number of crashes, using  Boston as an example.

I have already written about the terrible decision of  the Provincial Government, who despite the unanimous motion from the Union of British Columbia Municipalities decided that they would not allow cities to designate residential zones as 30 km/h.  Instead, municipalities will have to sign each and every street with expensive signage, showing once again that the rights of vehicular drivers trump those wishing to live in connected, walkable and bikeable communities.

Even New York City is testing “neighbourhood slow zones”, reducing speeds to 20 mph (30 kmh) on roads within neighbourhood areas.

Philadelphia is now designing “slow-speed corridors” and has over 20 neighbourhoods applying for that designation. Portland Oregon has brought speeds down to 20 mph (30 kmh) in their residential streets. Other cities are following suit, as slowing speeds contributes to the Vision Zero philosophy of increasing safety benefits to all community users.

The more a street or road is designed to accommodate speed, the broader they are and the less bendy they are. They are inhospitable to live next to, and research shows that higher speed roads only gain their occupants an  additional 48 seconds for every 1.2 kilometers.

The City of Edmonton has also instituted 30 km/h zones in residential areas and a 40 km/h maximum for arterial roads . They also have an online “Estimated Time of Arrival” tool online for drivers to see that their overall driving time is not really changing.

There is also a psychological discussion too about slower speed~the faster drivers travel, the less they are aware of the environment they are moving through.  Richard Sennett talks about pre-automobile streets where congestion was accepted as normal. He notes that it was the development of streets designed like boulevards that began the idea that lack of congestion was associated with the right to speed.

It is the concept of the driver’s right to uncongested freedom of movement at speed that has been a 20th century construction, now accepted as an entrenched given.

That’s where the Province of British Columbia got stuck.

Instead of visioning cities as having the right to lower speeds across the board in neighbourhoods to create more sociable spaces and community cohesion, the Province chose not to question the driver’s right to their accepted expectation of how roads are supposed to be all about the driver.

This is not about delaying motorists. This is about making city streets useable for other forms of transport besides a vehicle. If autonomous vehicles were also programmed to go at 30 km/h in neighbourhoods, they would be predictable obstacles for cyclists and pedestrians to get out of the way of, and also make transit, and active transportation more appealing.

It is simply time to change the paradigm that speed and vehicular travel matter more than the rest of neighbourhood residents and other road users. It is the 21st century.

Photo by Lucas Ettore Chiereguini on Pexels.com

 

 

 

14 Mar 05:05

Moving Online in Pandemic #3: What to STOP doing!

by Nancy White
How to use the Liberating Structure TRIZ to figure out what NOT to do for online meetings!

Source

14 Mar 05:04

My exploration of Rust and .NET

If this were a twelve-step meeting

ME: "Hi, my name is Eric, and I compulsively write code that hardly anybody else wants."

YOU (all together): "Hi Eric"

TLDR

I have been working on stuff that facilitates .NET development using Rust. My progress has reached a point where it looks like this could maybe become a real thing, but I'm not sure what to do with it. So I have decided to let you listen in while I ramble a bit.

Sometimes I write blog posts in a format I might describe as "pretending that the reader is asking me questions". This is one of those.

What is it?

The project I am discussing here, which does not yet have a name, consists of two main parts:

  • A "compiler" that takes LLVM bitcode from rustc and converts it into a .NET assembly
  • A tool to generate Rust bindings for other .NET assemblies so that Rust code can call existing .NET libraries

So the result is that Rust code can call .NET code, and vice versa.

Here's a Rust function which takes a string literal of digits, converts it to a .NET string, and then calls System.Int32.TryParse() on it:

fn int_tryparse_out_parm() -> bool {
    let s = "30579";
    let s_clr = System::Text::Encoding::UTF8().GetString_1(s.as_bytes());
    let mut result = 0;
    let b = System::Int32::TryParse_2(&s_clr, &mut result);
    return b && (result == 30579);
}

Is this open source? Can I try it?

Sorry, not at this time. This project is currently taking place in a private repo until I decide if or how to take it forward.

What are all these terms?

Yeah, since this blog entry might be read by people who know .NET or Rust but not both, let's define some things.

  • In .NET, an "assembly" is code in its compiled form. A shared library. The file suffix is .DLL, regardless of whether the OS platform is Windows or Linux or Mac or whatever.

  • In Rust, the build tool is called "cargo". It manages dependencies on both local projects and external packages.

  • In Rust, a package is a "crate". The main package registry is https://crates.io

  • In .NET, a package is a "NuGet package". The main package registry is https://nuget.org

  • The .NET system is built around a core runtime called the Common Language Runtime (CLR). The low level language is called Common Intermediate Language (CIL), or MSIL, or just IL.

  • Rust is compiled to native code through LLVM, a huge and popular set of compiler tools and libraries. (LLVM was originally created right here in the town where I live, at the University of Illinois at Urbana-Champaign.)

  • In Rust, referencing a part of a string or array is called a "slice". In .NET, this is called a "Span". Each side has memory safety rules about how such things can be used.

If .NET developers want interop with Rust, can't they just use P/Invoke?

Yes, well, that is the approach a sane person would use, isn't it?

What I'm exploring here is an alternative which could offer deeper integration between the Rust and .NET worlds.

How deep could that Rust/.NET integration go?

A lot deeper than P/Invoke, but probably not as deep as F#.

For those of you who use F#, you know how it constantly feels like you're straddling two worlds? The F# and C# worlds are both visible, and rather different.

  • Seq is an alias for IEnumerable.

  • For async, you can use the C# flavor or the F# flavor.

  • There are the F# collections, but you still have access to the stuff in System.Collections.Generic.

  • F# types like records and discriminated unions are not exposed to C#.

  • Naming conventions for identifiers are different

Rust in this project feels kinda like that, but the dissonance between Rust and .NET is even greater. For example, Rust has no garbage collector. Rust types like struct and enum are not are not exposed for use by other .NET stuff.

So even though P/Invoke is not involved, there is still a boundary between the Rust world and the world that C# sees.

How does the boundary between Rust and CLR work?

From the Rust perspective, the boundary is FFI ("extern"). Any Rust function which needs to be called by non-Rust CLR code should be "extern". The generated bindings for Rust to call other .NET stuff are in the form of extern functions. Just as with normal Rust, the only things that can cross that FFI boundary are simple types.

From the CLR perspective, interop with Rust/CLR code requires some of the same things as interop with native code through P/Invoke. Even when compiled for the CLR, Rust memory is "unmanaged" memory. The need to Marshal still applies. The binding generator handles a lot of this.

What currently works? What doesn't?

Nothing in this project is exhaustively tested. When I say something "works", I am speaking of a quality level such as "proof of concept". I have a small but growing test suite.

  • Converting LLVM bitcode to CIL generally works. I still find a bug every so often, and I expect to find more.

  • The binding generator can handle most of the surface area of the .NET class libraries.

  • Reference objects are wrapped in a struct and the methods for that class are placed in an impl block for that struct.

  • From the Rust point of view, things depending on the libcore and liballoc layers are working.

  • But Rust libstd is not implemented. The .NET bindings will become the basis for a .NET implementation of libstd. In other words, Rust file I/O (for example) would be implemented by calling .NET file I/O instead of libc.

  • Using Rust crates seems to work if they support no_std. For example, serde works.

  • Operator overloading in a C# type gets implemented on the Rust side using the corresponding traits.

  • The binding generator does not currently have a way to present a generic type as a Rust generic type.

  • The binding generator currently skips generic methods that have their own type parameters. I haven't figured out what to do about these yet.

  • .NET Task objects are wrapped on the Rust side like any other. I am hoping it will be possible to integrate Task with Rust futures and await, but I haven't looked closely at that yet.

Just how cool is this project?

I was hoping you would ask that, even though it's like asking a Mom how cute her baby is...

Here's something pretty cool: The binding generator has stuff to convert a Rust closure into a C# delegate.

fn mul_delegate() -> bool {
    let d = System::Func_i32_i32_i32_::create(|a,b| a * b);
    let x = d.Invoke_2(3,4);
    return x == 3 * 4;
}

Now THAT is one seriously cute baby.

(Yeah, I know, you're distracted by the ugliness of System.Func<int,int,int> becoming System::Func_i32_i32_i32. Just nod and be polite, okay?)

Is this project related to that Wasm stuff you did?

Kinda. Not really.

My wasm2cil work is on GitHub here:

https://github.com/ericsink/wasm2cil

There have been some copy-and-paste episodes between wasm2cil and this project, but the two are generally quite different, because WebAssembly and LLVM bitcode are quite different.

What does the implementation of this project look like?

The compiler and the binding generator are both written in F#, but the resulting assemblies have no F# dependencies.

Assembly generation is currently done using Mono.Cecil. I keep looking at System.Reflection.Metadata and thinking about it.

If this project has a future, it should eventually be re-written in Rust. Any decent compiler should be able to self-compile.

How does the CIL generation work?

First, generate LLVM bitcode using "cargo xbuild" with a custom target and a special linker.

Then, use LLVMSharp to read that .bc file, and Mono.Cecil to write it out as a .NET assembly.

The conversion of bitcode to CIL is where most of the magic happens.

Wait, I thought LLVM bitcode was non-portable?

It is. The way I am using bitcode is "a pathway to many abilities, some considered to be unnatural".

I do really wish that LLVM bitcode was available in a well-specified portable subset of some kind. A little googling has revealed that others have wanted this too, but the notion has apparently never gained any real traction.

Wasm would be a great alternative if it supported (a) 64 bits, and (b) optional use of non-sandboxed memory. For (a), I assume that Wasm64 will eventually happen, but I see no reason why they would do (b). Wasm's reason for existence is safety, not to provide the compiler world a portable assembly language.

So I find myself using LLVM bitcode and trying to deal with the places where arch-specific stuff creeps in.

Strangely, even though everybody says that LLVM bitcode is not portable, it's actually kinda close. For example, my CIL code generator doesn't actually care whether the Rust custom target arch is x64 or ARM64 -- it works fine either way.

I'm confused -- how can you compile a low-level language for the CLR?

Recall that CLR stands for COMMON Language Runtime. The CLR actually has good support for lower-level languages, even though none of the mainstream CLR languages actually make much use of those features.

Would this bc-to-CIL approach work for C/C++ too?

In theory, yes.

In more practical terms, confining my efforts to Rust is making things much simpler. C/C++ has several decades more baggage.

Isn't the Rust memory model really different from C#?

Yep. But here again, the CLR offers garbage collection, but it doesn't force you to use it. Nothing prevents us from using manual memory management.

Rust code compiled for the CLR still has the same memory model as any other Rust code.

How does Rust liballoc work?

Currently the allocator I give to Rust is implemented by simply calling AllocHGlobal and AllocHFree. I threw this in as a temporary solution, figuring I would need to replace it with a real malloc of some kind. Strangely, it isn't as slow as I expected.

Overall, how is the performance of this project?

I've done no real benchmarks, but roughly speaking, it appears the performance of "Rust CLR" is in the same ballpark as "Rust native".

I've seen some cases where "Rust CLR" actually seems to be faster, but I haven't investigated thoroughly. It makes sense that the JIT would win at least some of the time.

How do you deal with Rust's lack of function overloading?

That issue has been quite a problem. Naming things is hard.

You may have already noticed in the Rust code snippets above that names are a bit weird. For example, how do I generate a binding for System.Console.WriteLine(), which has multiple overloads?

fn write_string(s : &String) {
    let s_clr = System::Text::Encoding::UTF8().GetString_1(s.as_bytes());
    System::Console::WriteLine_1(&s_clr);
}

Under the hood, the binding generator gives every method an ugly-but-unique name like "void__WriteLine__1__String". At a higher layer, Rust traits are used to provide friendlier names to the developer.

But how to deal with those names? There are several approaches to this kind of problem, and all of them have tradeoffs. My current path is to implement all of them, and then experiment with actual usage to see how they work out.

For example, Rust can simulate function overloading using traits if number of parameters does not vary. So one of the techniques I'm taking for a test drive is to add a suffix _N for the number of params. This is the approach shown in the snippet above, where WriteLine has taken on the name WriteLine_1, because it has one parameter.

Another approach is to use tuples, but the extra set of parenthesis is ugly.

Of course, for all the methods that have only one overload, I could just keep the name. But if an overload gets added later, it will break.

There are lots of things here that still need to be figured out.

Wait, isn't .NET just about C#?

It is true that here in 2020, C# is by far the dominant language for .NET.

But the architecture was originally designed to support multiple languages. After all, the core component is called the COMMON Language Runtime.

What other .NET languages are there?

In the beginning, there was C# and VB.NET.

In hindsight, I suspect the multi-language architecture of .NET was primarily driven by Microsoft's desire to avoid leaving their Visual Basic developers behind. VB.NET is still around, and it still has many developers using it. But I assume (without evidence) that the VB segment is not growing, and probably shrinking.

The other .NET language often mentioned is F#, which might be described as .NET variant of OCaml. FWIW, I describe myself as an F# fan, although I also like and use C# a greal deal.

According to The .NET Language Strategy (Mads Torgersen, 2017), when counting developers using these languages, they speak of "millions" of people using C#, and "hundreds of thousands" for VB.NET, and "tens of thousands" for F#:

https://devblogs.microsoft.com/dotnet/the-net-language-strategy/

I don't know which CLR language is in fourth place, but it's probably a DISTANT fourth place.

It is worth noting that Microsoft has one more .NET language that is rarely discussed, and wasn't even mentioned in that Torgersen blog entry: C++/CLI. I'll stay with tradition and not talk about it.

Bottom line, if you assume that .NET and C# are synonymous, you will annoy F# and VB.NET fans, but sadly, you will be correct enough for most contexts.

Have their been any others?

Lots of them. Wikipedia has a whole page:

https://en.wikipedia.org/wiki/List_of_CLI_languages

My comments on selected cases:

  • I have never used Nemerle, but I always thought it looked interesting. Apparently the developers have been hired by JetBrains.

  • RemObjects has Oxygene. This is a Pascal-ish language for .NET, somewhat like the spiritual descendant of Delphi. It seems to be actively developed, but I cannot recall ever hearing this company or their products mentioned out in the wild.

  • Years ago, the LLVM project had a backend which could generate CIL. It apparently died of neglect and was removed.

  • PeachPie is an interesting case. It's a PHP compiler for .NET.

So there's some interesting stuff here, but the bottom line is: Every CLR language not named C# has lived a constant struggle for viability.

So then why in the world are you doing this?

Recall that the first thing I did in this blog entry was admit to a compulsion.

If I attempt to explain my actions beyond that, a couple of things come to mind.

First of all, when I look at the kind of leadership coming from Microsoft right now, I see an emphasis on building bridges between .NET and other ecosystems. Because of .NET Core, this is a new era for .NET, and it seems reasonable to wonder how things in the future might go very differently than they have in the past.

I also wonder about the possibility that Rust might be a special case.

How might Rust be "a special case"?

Rust is simply a masterpiece. It is an amazing achievement of programming language design and implementation. There is nothing else like it.

I believe that the popularity of Rust will continue to grow rapidly. And let me immediately say that I might be wrong. The best technology doesn't always win.

But Rust is showing very strong signs of momentum. For example, as of January 2020, Rust has been Stack Overflow's "most loved language" four years in a row:

https://stackoverflow.blog/2020/01/20/what-is-rust-and-why-is-it-so-popular/

So my claim that Rust could become hugely popular may not be correct, but it's not the craziest thing I've ever said.

What if Rust explodes in popularity over the next 5-10 years? It seems likely to me that if Rust becomes really big, there could be a significant number of .NET people interested in using it.

What is the goal of this blog entry?

One great thing about the Internet is its ability to connect people with niche interests. Recently I was in my kitchen making coffee and found myself wondering if Captain America could beat a lion barehanded. Sure enough, people on the Internet have been discussing that important topic for many years.

With over 7 billion people on the planet, I'm probably not the only one interested in the possibilities of developing for .NET using Rust.

I mean, there's gotta be at least two more of you. If that darn Covid-19 scare weren't cancelling all our conferences, the three of us could get together for coffee and chat.

As we are all currently trapped in our homes hoarding hand sanitizer, we'll need to figure something else out. But it would still be nice to know who you are.

How will you take this project forward?

I honestly don't know yet. There are lots of variables in the equation. I could make this open source. I could create a niche proprietary commercial product. And there are many shades of gray in between.

Regardless of how it goes forward, if this is to become more than my fun side project, there are questions about sustainability.

14 Mar 05:04

Tone.js Interactive Music Web Framework

Dylan Schiemann, InfoQ, Mar 11, 2020
Icon

This short article introduces readers to the Tone.js Javascript web music framework. This is a set of functions that application developers can use to create interactive music in their applications. If you sample the demo apps you can get a sense of what's possible. The framework is open source and you can view the source on GitHub.

Web: [Direct Link] [This Post]
14 Mar 05:04

Alec Couros was used by scammers to catfish thousands of women and he's a victim too

Bridget Judd, ABC News, Mar 11, 2020
Icon

The truly sad thing about this story is that it's running in 2000, some 13 years after Alec Couros's photos first started to be used in catfishing scams ('catfishing' is where a person assumes a fake ID and pretends to be a potential romantic partner in an effort to defraud you). He "estimates the number of victims to be in the thousands." Though social networks claim to be taking action against it, I've watched Couros's efforts over the years to stop the problem come to no avail. Via Aaron Davis, who recommends more reading: posts by Alan Levine, Dean Shareski and Alec Couros himself. Also the case of Lydia Abdelmalek provides another detailed case.

Web: [Direct Link] [This Post]
14 Mar 05:04

What happens when cities don’t prioritize families?

image

Last week, Vancouver’s former chief city planner, Brent Toderian, started a media frenzy when he revealed that his child lost a lottery to attend their local kindergarten. This means that after giving up his car to live in Vancouver’s walkable downtown, he can’t even walk his kids to the school across the street from his condo.

“We moved literally across the street from the school, so, ironically, we are the furthest away from the next school. We are the worst impacted by not getting in,” he said in an interview with Daily Hive.

His son is one of over 250 Vancouver children who are put into lotteries for their local schools, and often end up shuttled to schools outside of their communities. Last year, there were initially 269 kindergarten kids wait-listed at their catchment schools, according to the Vancouver School Board. In September, 115 children started kindergarten at a school outside their catchment area.

image

Toderian says the outcome is a “result of bad planning, and I don’t mean by our family.”

How does this happen? I can’t imagine that planners, who are hired to think about this kind of stuff, can’t predict the future demand for local schools when working on large and small scale planning projects for the city. 

It makes me think that perhaps it just isn’t a priority for the City of Vancouver. If it were, it would be a primary consideration whenever any large development or new community is built, like when the City took a massive piece of industrial land and converted it into Vancouver’s Olympic Village over 10 years ago. 

Vancouver’s Olympic Village is a gorgeous, walkable community that is full of families. It likely received millions of dollars in community amenity contributions from developers. This is why the City was able to build affordable housing, including family-friendly co-op housing, a lovely park, and a playground in the neighbourhood. 

But guess what is missing?

A school.

How does one plan and build all that density and not build a school?

image

Apparently before the 2010 Olympics, the city intended to build a school for its newest community. But it was not until October of 2019, that Vancouver City Council finally passed a motion calling on the Province of British Columbia to prioritize funding for the school. Clearly, sometime got lost in translation over the last 10 years and neither the City, the School Board, or the Province prioritized this project until now. 

It seems like planners and local school boards are surprised that families want to live in urban environments like Downtown Vancouver. This should not be the case! Families are willing to trade in their cars and large homes to live in a walkable community that is well-serviced by amenities like parks, community centres, childcare, and yes, the most important thing is schools.

I have written article after article on how to build a family friendly city and profiled many families who have chosen this lifestyle. It is disappointing to say the least to find out that the City is not doing enough to help these families stay and thrive in Vancouver. 

We can do better. And we have to, if we want to create a livable city that includes families, children, workers like nurses, teachers, police officers and all those who provide essential services. Or, do we only aspire to be a retirement community for rich people?

Photo credit: 1) Crosstown Elementary: Unknown; 2) Brent Toderian: Megan Devlin/Daily Hive; 3) Olympic Village: Jason Payne/Province

14 Mar 05:03

Weeknotes: COVID-19 numbers in Datasette

COVID-19, the disease caused by the novel coronavirus, gets more terrifying every day. Johns Hopkins Center for Systems Science and Engineering (CSSE) have been collating data about the spread of the disease and publishing it as CSV files on GitHub.

This morning I used the pattern described in Deploying a data API using GitHub Actions and Cloud Run to set up a scheduled task that grabs their data once an hour and publishes it to https://covid-19.datasettes.com/ as a table in Datasette.

If you're not yet concerned about COVID-19 you clearly haven't been paying atttention to what's been happening in Italy. Here's a query which shows a graph of the number of confirmed cases in Italy over the past few weeks (using datasette-vega):

COVID-19 confirmed cases in Italy, spiking up to 10,149

155 cases 17 days ago to 10,149 cases today is really frightening. And the USA still doesn't have robust testing in place, so the numbers here are likely to really shock people once they start to become more apparent.

If you're going to use the data in covid-19.datasettes.com for anything please be responsible with it and read the warnings in the README file in detail: it's important to fully understand the sources of the data and how it is being processed before you use it to make any assertions about the spread of COVID-10.

My favourite resource to understand Coronavirus and what we should be doing about it is flattenthecurve.com, compiled by Julie McMurry, an assistant professor at Oregon State University College of Public Health. I strongly recommend checking it out.

Other projects

I've worked on a bunch of other projects this week, some of which were inspired by my time at NICAR.

  • fec-to-sqlite is a script for saving FEC campaign finance filings to a SQLite database. Since those filings are pulled in via HTTP and can get pretty big, it uses a neat trick to generate a progress bar with the tqdm library - it initiates a progress bar with the Content-Length of the incoming file, then as it iterates over the lines coming in over HTTP it uses the length of each line to update that bar.
  • datasette-search-all is a new plugin that enables search across multiple FTS-enabled SQLite tables at once. I wrote more about that in this blog post on Monday.
  • datasette-column-inspect is an extremely experimental plugin that tries out a "column inspector" tool for Datasette tables - click on a column heading and theh plugin shows you interesting facts about that column, such as the min/mean/max/stdev, any outlying values, the most common values and the least common values. Screenshot below. This prototype came about as part of a JSK team project for the Designing Machine Learning course at Stanford - we were thinking about ways in which machine learning could help journalists find stories in large datasets. The prototype doesn't have any machine learning in it - just some simple statistics to identify outliers - but it's meant to illustrate how a tool that exposes machine learning insights against tabular data might work.
  • github-to-sqlite grew a new sub-command: github-to-sqlite commits github.db simonw/datasette - which imports information about commits to a repository (just the author and commit message, not the body of the commit itself). I'm running a private version of this against all of my projects, which is really useful for seeing what I worked on over the past week when writing my weeknotes.

Here are two screenshots of datasette-column-inspect in action. You can try out a live demo of the plugin over here.

Outliers in number of appearences in the Avengers: Iron Man, Captain America, Spider Man and Wolverine

Column summary for states in actions_under_antiquities_act

14 Mar 05:03

Some Questions To Ask Before You Get Started

by Richard Millington

Typically we begin a new client project with an on-site workshop where we aim to get all the key people in the same room (and then on the same page).

Some useful questions to ask:

  • What does each person hope to get from the community and what do they fear about it?
  • What are the problems the community can solve, why haven’t they already been solved, how else could they be solved, and why can the community solve them better than any other method?
  • What problems will the community tackle in 0 – 3 months, 3 to 9 months, 9 to 18 months, and 18 months+?
  • What does success look like? What is an ‘ok’ outcome, a ‘good’ outcome, and a ‘fantastic’ outcome from the community?
  • What metrics matter to each stakeholder, what will we track, and what is the best means of sharing success?
  • What are the behaviors of our audience we need to change and what currently influences or drives those behaviors?
  • What are the most valuable behaviors each segment of our audience (visitors, lurkers, irregulars, and top members) can perform for us to achieve our goals?

Add and remove the questions as needed. But if you can get your colleagues aligned on these answers, together you will be doing far better than most.

14 Mar 05:03

BIKE MINDS March 2020: Bikes and Limits

by dandy

Story & photos by Robert Zaichkowski

The BIKE MINDS storytelling series has covered a wide variety of bike related stories since its inception in 2018 involving themes such as belonging, travel, career, and growth. The storytelling event even launched in Ottawa this past January with a second Ottawa event currently in the works. The theme for the Tuesday, March 3, 2020, event at Curbside Cycle was “Bikes and Limits” - illustrating how the power of bikes can be pushed to the limits, though there was no limit to the energy of the event’s emcee Janet Joy Wilson.

Jun Nogami – a University of Toronto engineering professor and author of the Biking in a Big City blog (and a regular dandyhorse contributor) – kicked things off in the fast lane by talking about the World Human Powered Speed Challenge, for which he is the U of T team’s faculty advisor and a chief timing official. The challenge is held annually at Battle Mountain, Nevada, on a straight section of State Road 305 known as the world’s fastest track. Riders have five miles (8 km) to accelerate and a 200 metre window where their speed is recorded before slowing down for one mile (1.6 km). The bikes are essentially recumbents with an aerodynamic shell. While earlier bikes such as the one Canadian Sam Whittingham used to reach 82.82 MPH (133.26 km/h) had clear windshields to see through, newer “camera bikes” and their aerodynamic improvements broke the record by at least 3 MPH (4.8 km/h) when first introduced in 2015 with the current record standing at 89.59 MPH (144.17 km/h).

Anne Fleming’s story was about a kayaking and mountain biking trip she and her husband took in Newfoundland 20 years ago, which she started with a brief aviation history about Gander. The town of 11,000 people welcomed over 6,000 during the September 11 attacks and was an important refuelling stop for early transatlantic flights since the airport was built in 1935 including during World War II. The area was known for wood, bogs, lakes, and rivers while a long hill – which Fleming likened to Poplar Plains – tested her limits. When the “tick mist” cleared, she saw the remains of a DC-4 plane crash from 1946 which she was not prepared to see, but her guide told an inspiring story. The crash prompted the largest rescue effort at the time which helicopters had to be de-assembled before being delivered to Newfoundland. 18 survivors were rescued over two to three days and the crash was featured in a CBC Land and Sea episode in 1992.

Mark Franklin is the founder of Career Cycles and spoke at a past BIKE MINDS event in 2018. His talk focused on his experiences as a trip leader for Backroads Active Travel based in Berkeley, California in which he led 25 people on a ride from Banff to Jasper. The job posting for trip leaders called for those serving others, hard working, sophisticated conversationalists, and problem solvers to meet the demands of an upper middle class clientele. After the application and interview, candidates then had to go to California for a weekend event where 60 people would compete for ten spots. They would respond to situations such as a suitcase being left behind, trying to raise and lower helium sticks, and go through tests involving problem solving, teamwork, mechanics, and public speaking. Franklin later designed his own trips including career counselling and left the audience with a new word – liminal – which refers to a transitional stage.

Najia Zewari moved from Afghanistan to Canada six-and-a-half years ago and co-founded the ck out of the Gateway Bike Hub in Thorncliffe Park. She reflected on how she felt depressed upon moving to Canada given she couldn’t connect with her surroundings. Through the Afghan Women’s Organization, she was among 15 women who took up CultureLink’s Bike Host program and Evergreen Brickworks also helped with bike training. Despite only being aware of her surroundings in 2017, Zewari’s experiences led her to learn bike mechanics through the Gateway bike hub, a group ride to Open Streets, and work with Markham Cycles before the Women’s Cycling Network started in October 2019. Zewari noted how the bike was a tool for empowerment and helped connect her with more communities.

Michael McMahon is a self professed “web geek” and talked about riding the BT700 with Melanie Chambers in which BT stands for butter tart. The BT700 is a 770 kilometre bike loop in Southwestern Ontario which starts and ends at St. Jacobs – a Mennonite community north of Waterloo – and passes through Owen Sound, Collingwood, and Orangeville. The ride is held at the end of the season with trail apples, tastings of beer, wine, and cider, crazy elevation gains, and close friendships made from such intense experiences. While there were several highlights, McMahon also mentioned some challenges such as how bringing a backpack was not a good idea, his tires were too narrow to handle some of the rougher terrain (40 mm tires were recommended), and some of the nights were as cold as 1’C in Mono Cliffs.

The final BIKE MINDS Toronto event for 2020 called “Bikes and Boundaries” will take place at the Ontario Bike Summit on Sunday, April 5 with tickets being available on March 20. The emcee left us all with a final message: We all can be “spokes” people for change, one story at a time.

14 Mar 05:03

Appropriating OpenLearn Content and Republishing Edited Versions Of It Via a “Simple” Automated Text Blogging Workflow

by Tony Hirst

I had intended on using my (unpaid) strike days to catch up with some books and harp practice, and maybe even the garden, and keep away from the keyboard; or failing that, to have a push on my rally data tinkering and get another LeanPub book started to try to reboot the £50 a quarter or so my previous publication (Wrangling F1 Data With R) generated, which keeps things like recurring Dropbox and Flickr etc etc charges covered (no-one has ever bought me a KoFi, as far as I can tell…).

And I was determined not to do any of the mounting workload associated with the day job, no matter how much fun some it is likely to be (like getting Ev3devSim working as an ipywidget in Jupyter notebooks).

Whilst I did manage to stick to the determined not to path, I never even really started down the intended one, instead spending hours and hours in front of keyboard trying to hack something together around my OpenLearn publishing workflow.

So here’s what I’ve come up with…

An OpenLearn Unit Text Publishing Thing

Firstly, it’s a thing that lets you grab the “source” content of an OpenLearn unit (at least, some of it; I still haven’t got round to grabbing things like video files or audio files, or scraping PDFs etc.) and churn it into a simple text format, markdown, which looks like this:

Headers are prefixed with a #, you can emphasise things by wrapping it in * characters, eg *italics* -> italics, or **double them up** for strong emphasis. Embedding links — [link text](link/path/file.html) — and images — ![Alt text](path/to/image.file) — is also pretty easy when you get the hang of it.

So how do you get started? First, you need a Github account (sign up here; just get one: you’re not going to have to do any hard Github stuff, you’re just making use of their free hosting). Get one, and sign in.

Second, visit my demo repo — psychemedia/openlearn-publish-test — (the URL will change at some point, but I’ll archive the original and link the new address from it…) and grab a copy of your own repo from mine by clicking the big green Use this template button:

You’ll be presented with a form:

Give your repo a name (no spaces). Optionally add a description. Keep the repo public. And click the big green Create repository from template button.

Things will churn for a moment or two:

And then you’ll have your own repo, containing a copy of the files in mine:

Behind the scenes, there is work going on…

Click the Actions tab on your copy of the repo to see what…

At first, it may look like nothing… but wait a moment or two and refresh the page:

A couple of actions will start running to initialise, and customise, your repo for you.

When the actions are done, you’ll be informed… (you shouldn’t have to refresh the page, the status indicators should update when things are done…):

If you go back to your repo homepage, you’ll see it’s been updated with a new README that’s slightly different to the original copy from my repo, and that has been personalised to yours:

So… now you can grab some OpenLearn content into your repo.

Click on the SET_UP.md file link in your repo:

You will be presented with a list of units on OpenLearn.

Find one you like the look of and click the Grab Unit into this repo link:

This will open a new issue for you in the Issues tab of your repo, and prepopulate it with a title that will tell a Github Action you want to grab some OpenLearn content, and an issue body that tells the action where the unit can be found.

Click the Submit new issue button to get things started.

Back in the Actions tab, you can see the helper elves have started doing their thing again…

If you click on a running Action, you can check its progress in more detail:

Click through on the actual job name to see what’s happening inside:

You can expand a step by clicking the arrow to see what each step is doing or has already done…

If you read through the steps, you’ll see several things are done: for example, we grab some OUXML (the OpenLearn content), convert to markdown, build some HTML files (these are what gets published), and deploy them, then build a LaTeX version of the material (which is used to generate a PDF), and an ePub ebook. (The LaTeX step takes some time; I should perhaps simplify things so that only the HTML build is done by default.)

When the Actions are green circled / green ticked and done (which may take a few minutes…), or at least, when the Deploy HTML to gh-pages step has run, go back to the repo home page, where you should see a new commit has been made to your repo:

If you click into the content folder you’ll see one or more session folders:

If you click into a session folder, you see some markdown files:

If you click on one of those, you’ll see some scraped and converted OpenLearn content:

So the content has been grabbed from OpenLearn and saved to your repo.

But that’s not all.

If you scroll down on your README page (I really should make this link more prominent in the README…) you’ll see a link to a github.io site published from your repo:

Click it…

If you see a “404”, page not found, don’t panic

On the repo home page, select the Settings tab:

and scroll down to the Github Pages area:

Change the Source from gh-pages branch to master branch:

And then, select the master branch:

And set the Source back to the gh-pages branch:

When you see something like this, you knows all good to go:

Note that cacheing of a previous build of the site may last for up to 10 minues, so grab yourself a cup of tea, or perhaps look through the markdown files in the content directory, or even go back to the Actions tab and, if the actions have completed.

If the Actions have completed, select the OpenLearnXML2 (or a completed nbsphinx publisher action if you have committed your own changes to the markdown files) and you should see and the availability of an Artifacts download.

Down load and unzip the artifacts file. If the build process has been able to build a PDF file and/or an ePub file from the content, it will be found in the unzipped downloaded directory.

Right… time to try your github.io site link again:

An OpenLearn Editing Thing

This will have to be in a part two to this post… I’ve run out of time for now and need to get back to the day job…

If you are itching to get started, this may work, if I’ve got my autopublishing things fixed…

In the content folder (on the default master branch of the repo),  find the markdown file you want to edit, and click on the pencil icon to open the editor:

my-oer_Part_00_02_md_at_master_·_psychemedia_my-oer

Edit the file / make the changes you want, and commit it (you may want to set a meaningful commit message title summarising the chnages, and perhaps even a longer description about the motivation for the changes, but both are optional…):

Editing_my-oer_Part_00_02_md_at_master_·_psychemedia_my-oer

Click the big green Commit changes button to commit the changes. If you look in the Actions tab, you should see that an nbsphinx publisher action has started that should publish your changes to your site.

Actions_·_psychemedia_my-oer6

Note that even when the publishing action has generated and pushed updated site pages  to where they need to be, the site may take a few minutes to update because of page cacheing on the Github site.

The Future

One of the spinoffs of this for me was the realisation that I could use Github Actions to run arbitrary code in response to particular events, such as.. commits or issue postings. The current machinery uses a Sphinx / nbSphinx publishing route, but I’ve also started exploring a recipe for Jekyll based Jupyter Book publishing. (Next on the to do list will be an Executable Book project / MyST workflow](https://ebp.jupyterbook.org/en/latest/); I also need to split out the workflows into actions of their own, but I haven’t figured out how to do that for myself yet.) It strikes me that I could bundle all these in the same repo with some way of flagging which build process I want to use. This would allow the user to then republish their material using the publishing tool, and its various peculiarities, customisationa and affordances, of their choice.

Immediate to dos, that may not happen because I’m the only user, I know it’s possible, and I’m not that interested, are to: make the Github Pages / pubished site link more prominent in the README; get movie and audio downloads and embeds working. Also a way of handling PDFs linked from the OpenLearn materials, and perhaps extracting text from those, even, to support republishing…

On the publish side, it would be useful to be able to publis to HTML only by default, with some optional way of invoking the PDF and ePub builds. The ePub build also needs things like title and author setting. The PDF build sometimes breaks, eg due to the inability to detect a bounding box size round a gif image. I maybe need to use another PDF generator, eg some hints here.

I also need to refactor the code, two ways: firstly, a simplification, that uses the bare minimum of packages and just churns the markdown direct from XML in one simple step. Secondly, fixing the current workflow, which stages the XML in a SQLite database, so that the database can properly handly content from multiple unitis and I can reliably churn the md from the database for any single unit. At the moment, I think things pretty much assume there’s content from just a single unit in the database. Putting the md into the db might be useful too… Then I could imagine a datasette powered publishing route too…

As a recent tweet from Martin Hawksey reveals, he’s been blogging about how we can turn Google’s App Script to our own purposes as a hosted code runner, and I think Github now provides a similar opportunity for anyone who wants to appropriate it to that end…

14 Mar 05:03

"History likes to keep things simple."

“History likes to keep things simple.” - | Ross Douthat, Joe Biden Beat Bernie Sanders....
14 Mar 05:03

A significant jump (32%) in total confirmed Dut...

by Ton Zijlstra

A significant jump (32%) in total confirmed Dutch Covid19 cases today, as reported by the RIVM, the national institute for health and environment. We’re still on an exponential curve. Although the last two days showed lower increases, today’s report partly compensates for them due to wider testing than the preceding days. Comparing the curve to Italy’s curve, we’re about where Italy was 2 weeks ago.
The percentages of increase of the past three days are all well below 40% however (22%, 19%, 32%), which is the rate at which the total number of positive tests doubles every other day. We’re now doubling total number of cases every 3 to 4 days. I don’t know whether that is a reflection of still limited testing, or of current containment measures, or something else. Slower is better (as long as it isn’t caused by a cap on testing) as it allows the healthcare facilities to handle the required efforts better as it gets spread out over (slightly) more time.


Source: numbers taken from daily bulletins of the RIVM

It would be interesting to see how many tests are actually carried out, but the RIVM doesn’t seem to share those numbers. [UPDATE after 20mins] The RIVM indicated that if e.g. household members of confirmed patients develop symptoms they do not get tested and are automatically assumed to have Covid19. They are not included in the numbers therefore, and the actual number of infected people is higher than the numbers suggest. This to save the available tests for tracking the spread of the infection along less obvious paths. [UPDATE after 6 hours: The current total of tests done is around 10000, so a positive rate of about 5% now.]

14 Mar 05:01

You’re sharing essential content. Here’s how to get colleagues to read it.

by Josh Bernoff

As I work with companies on internal communication, I see amazing and pervasive levels of inefficiency and waste. These groups work hard, generate valuable content, and then despair when it has less impact than they had hoped. This is fixable. I’ll share six ways you can change what and how you write to properly value … Continued

The post You’re sharing essential content. Here’s how to get colleagues to read it. appeared first on without bullshit.

10 Mar 01:28

Covid-19 :: Warum wir jetzt mit Einschränkungen leben sollen

by Volker Weber

exponentielleswachstum20200309a.jpg

Die Nachrichten zu dem neuen Virus überschlagen sich und ich will dem nichts hinzufügen. Aber ich will einmal ganz einfach erklären, warum wir unseren Alltag massiv ändern müssen. Es geht darum, ein exponentielles Wachstum zu verlangsamen. Alle Daten, die wir bisher haben, deuten Stand 9.3.2020 auf eine Verdoppelung der Fallzahlen alle fünf Tage (außerhalb von China) hin. Und das können wir uns einfach nicht vorstellen.

Wir kennen alle die Sage von dem Erfinder des Schachspiels, der als Entlohnung ein Reiskorn auf das erste Feld, die doppelte Menge auf das nächste, usw. haben wollte. Und das sieht so wenig aus. Aber auf das letzte Feld müssten dann 2^63 Reiskörner, dazu noch mal so viele auf allen vorherigen, minus ein Reiskorn. Macht 2^64 Reiskörner, oder 18.446.744.073.709.600.000 Stück. 18 Trillionen oder 1,8e19, wie der Ingenieur sagt. Das ist exponentielles Wachstum, das aus 64 mal schnell 18 Trillionen macht.

Aber zurück zu unserem exponentiellen Wachstum. Aktuell sind wir Deutschen bei 2^10 also ca. 1000 Infizierten. Wenn wir weiterhin alle fünf Tage verdoppeln, sind wir in 50 Tagen bei eine Million Infizierten. Und fünf Tage später bei zwei Millionen. Und das wollen wir verhindern, um Zeit zu gewinnen. Etwa 20 Prozent der Erkrankten müssen betreut werden und das können wir schnell nicht mehr leisten.

Die Chinesen haben es mit drastischen Maßnahmen geschafft, die Verdoppelung auf 19 Tage zu strecken und sind erst bei 2^16. Diese Maßnahmen sind in der Form aber hier nicht durchsetzbar.

Aber jeder kann selbst was tun. Hände waschen, nicht ins Gesicht fassen, einen Meter Abstand von anderen Menschen einhalten. Und wo man das nicht einhalten kann, einfach nicht hingehen.

Wer dieses Jahr übrigens kein Home-Office machen darf, wird es nächstes Jahr dürfen. Bis dahin haben wir das geübt.

PS: Bei der Homöopathie geht es übrigens andersrum. D20 ist eine 1/10^20-fache Verdünnung, also 0,00000000000000000001 Gramm Wirkstoff pro Gramm Lösung. Das kann sich wieder keiner vorstellen, aber 10^20 ist ungefähr 70% von 2^66. Das Schachspiel hätte also zwei Felder mehr und wir müssten noch zweimal verdoppeln. Und dann ein schwarzes Reiskorn (Wirkstoff) in 100 Trillionen weißen suchen. C30 ist noch lustiger. Das ist 100^30, oder 10^60. Das Schachspiel hätte dann 200 Felder.

10 Mar 01:27

The unexpected Google wide domain check bypass

The unexpected Google wide domain check bypass

Fantastic story of discovering a devious security vulnerability in a bunch of Google products stemming from a single exploitable regular expression in the Google closure JavaScript library.

Via Hacker News

10 Mar 01:27

New Artificial Intelligence (AI) blog series

by Troy Angrignon

New series on AI and Machine learning focused on how it can be applied to solve business problems and who the key players are in the industry.

The post New Artificial Intelligence (AI) blog series appeared first on Troy Angrignon.

10 Mar 01:25

Links of the Quarter: March 2020

by Dave Pollard

The mass of mankind is ruled not by its own intermittent moral sensations, still less by self-interest, but by the needs of the moment. It seems fated to wreck the balance of life on Earth — and thereby to be the agent of its own destruction. What could be more hopeless than placing the Earth in the charge of this exceptionally destructive species?

Homo rapiens is only one of very many species, and not obviously worth preserving. Later or sooner, it will become extinct. When it is gone Earth will recover. Long after the last traces of the human animal have disappeared, many of the species it is bent on destroying will still be around, along with others that have yet to spring up. The Earth will forget mankind. The play of life will go on.

— John Gray


PREPARING FOR CIVILIZATION’S END

How long we’ve been around: From paleobiologist David Hickey on the super-volcanoes that almost rendered the human species extinct practically before it got started: “An eruption occurred 39k years ago where Naples, Italy is now located. It probably had its greatest effect in Europe. It apparently resulted in regional extinctions of both the resident human culture and that of neanderthals. Those replacing them were from a different culture. Earlier, about 70k years ago the Mt. Toba, Sumatra eruption was even larger and apparently had world-wide consequences. Geneticists estimate human population [in the following millennium was reduced to between 2,000 and 20,000].” There have likely been tens of thousands of such cataclysmic events in the timeline depicted above, and our species barely survived the only two it’s faced. Our current collapse, the sixth great extinction, is almost certain to dwarf the super-volcanoes in its ecological impact.

Coming unglued: Brutus describes the collective madness that seems to be unfolding worldwide as hope and illusion yield to despair, fear and anger:

That’s where we are now, retreating into magical thinking we supposedly left behind in the wake of the Enlightenment. Call it the Counter-Enlightenment (or Un-Enlightenment). We’re on this track for a variety of reasons but primarily because the bounties of the closing Age of Abundance have been gobbled up by a few plutocrats. Most of the rest of population, formerly living frankly precarious lives (thus, the precariat), have now become decidedly unnecessary (thus, the unnecessariat). The masses know that they have been poorly served by their own social, political, and cultural institutions, which have been systematically hijacked and diverted into service of the obscenely, absurdly rich.

Dark knowledge: Catherine Ingram has written a long essay on collapse and our emotional reaction to it, and where we go from here. Her list of suggested responses at the end of the essay is quite similar to my Being Adaptable reminder list. Excerpt:

For decades, I had sensed that things were dramatically worsening, the rate of destruction increasing.  We are not facing merely a Black Swan event. We are facing a sea of black swans. As a journalist from 1982 to 1994, I specialized in social and environmental issues. I had written about global warming, the phrase we used in those days, numerous times in the 1980s, but because it seemed a far-off threat, we could intellectually discuss it without fear of it affecting our own lives in terribly significant ways. As time marched on, I began to awaken to how fast the climate was changing and how negative its impacts.  It became a strange relief to read and listen to the truth of the situation from people who were studying the hard data as it affirmed my instincts and threw a light on what had been shadowy forebodings, dancing like ghosts in my awareness.  It is an ongoing study that has taken me through a powerful internal process–emotional and cathartic–one that I felt might be helpful to share with those who have woken to this dark knowledge or are in the process of waking to it, just as I, over time, found comfort in the reflections of the small yet increasing number of comrades with whom I share this journey.

Because the subject is so tragic and because it can scare or anger people, this is not an essay I ever wanted to write; it is one I would have wanted to read along the way.  But the words on these pages are meant only for those who are ready for them. I offer no hope or solutions for our continuation, only companionship and empathy to you, the reader, who either knows or suspects that there is no hope or solution to be found. What we now need to find is courage.


LIVING BETTER

cartoon by Michael Leunig; thanks to NTHELove for the link

What’s the best milk?: There’s no perfect answer for your health and for the environment, but, bottom line, anything is better than dairy milk, and organic soy and oat milk have a slight edge over the alternatives.

How many stars are there?: Uncountable. Unknowable. Unimaginable.

Eating better: The USDA in the US is again reviewing its dietary guidance this year, with input from ‘interested parties’. Like in Canada, that process is overwhelmingly dominated by corporate interests in Big Ag, which depends on misinformation on the direct connection between the industrial diet and most of the diseases that sicken and kill Americans. Public health is consistently sacrificed. It was different last time around in Canada, though the Conservatives have vowed to scrap the new health guidelines and invite industrial agriculture to rewrite them to their specifications again. The current US guidelines are industry gold: they encourage consumption of dairy, meat, poultry, seafood and eggs — a payoff to all the Big Ag sectors. Their sample ideal meal includes cow’s milk, and their sample vegetarian ideal diet includes significant amounts of dairy and eggs. They also encourage exercise, but only up to the pathetic 2008 Physical Activity Guidelines (10 minutes a day of vigorous exercise or 20 minutes a day walking), ostensibly because they patronizingly think Americans will balk if asked to do any effective level of exercise. But there are competent, unbiased doctors and dieticians out there trying to improve the recommendations, such as Michael Greger and Brooke Goldner.


POLITICS AND ECONOMICS AS USUAL

Thomas Piketty on the demise of capitalism: Unregulated capitalism inevitably creates massive inequality of wealth and power, which leads to unsustainable and dysfunctional behaviours — in politics, in business, in (de-)regulation, and in economic and fiscal policy-making. We need to let this dangerous religion go.

Ezra Klein on the polarization of politics: It’s not that we’re getting more polarized. It’s that the parties, and the media, now oversimplify the choices we face in ways that reflect and reinforce what we say we think we want to hear, and push all the subtleties of every issue under the rug.

Oily economics: The application for the vast Tech Frontier Alberta tar sands bitumen sludge mine has been withdrawn, and while the official market-sensitive reason is politics (progressive Canadians are going to make it increasingly difficult for petrochemical corporations to destroy the planet), the real reason is a different sort of politics (all Canadians are beginning to catch on to the colossally expensive, massive government subsidies needed to enable ruinous petrochemical development to continue). The sludge only nets about $15/bbl in markets, way below the slumping price for ‘real’ oil, and its production cost is vastly higher than that. But Alberta, and some think Canada, will collapse economically without this ‘economic engine’ (traders in its currency, ludicrously, buy the lie, so the CA$ trades as a petrocurrency). Governments have always been the first to blink, but finally some oil companies are starting to do so too.

The Hatchet Job on Bernie Sanders: The NYT and the rest of the corporate media must be breathing a sigh of relief now that their campaign of lies and ‘guest’ editorials to defeat Bernie Sanders appears to have succeeded, again.


FUN AND INSPIRATION


cartoon by Will McPhail in the New Yorker

The story of Abbey Road side 2: A fascinating study of what many think is the finest composition in the history of rock music, and which John Lennon called “junk”.

Talking about trees: The Tyee has an educational and entertaining 5-part interview with renowned tree scientist Diana Beresford-Kroeger.

An animal that doesn’t breathe: Scientists have discovered a distant relative of the jellyfish that has no respiratory system to get oxygen.

A user manual for ‘me’: Cassie Robinson has created a ‘user manual’ explaining her personal operating style, needs, most effective working conditions, and other information to help co-workers ‘get the best’ out of her. It’s a fascinating model, and might work just as well as a self-description to help personal partners get along better too. Kind of related is a ‘relationship smorgasbord’ that invites those in new relationship to explicitly identify what’s ‘included in’ and ‘excluded from’ their relationship, from a menu with different categories for the different aspects of relationship. Thanks to Tree Bressen for the links.


RADICAL NON-DUALITY STUFF

Is the absence of ‘internal monologue’ and a ‘mind’s eye’ analogous to not having a ‘self’?: Many people are shocked to discover that most others often have ‘internal discussions’ in their heads, and can visualize things in their minds — when they themselves have neither capacity. These incapacities don’t seem problematic or even noticeable to the rest of us, but the sudden realization that one has them, and never knew it, can be unnerving. I wonder whether this is analogous to not having a ‘self’ — a sense that one is a real, separate person resident in a body with choice and control over what the person does. I’ve now met dozens of people who have lost their sense of self, and while that loss was definitely noticeable to ‘them’, it apparently had little effect on how others, even family members, perceived them, or how the apparent character ‘left behind’ after the loss continues to function. So perhaps it’s possible that there are millions out there who have never had a sense of self, a sense that they and things around them are real and separate — and that absence has never been missed.


THOUGHTS OF THE QUARTER


new self-portrait by my friend Ron Woodall

Excerpts and quotes by others from John Green’s podcast Capacity for Wonder and Sunsets, and his novel Turtles All the Way Down:

who are you,little i

(five or six years old)
peering from some high

window;at the gold

of november sunset

(and feeling:that if day
has to become night

this is a beautiful way)      [EE Cummings]

“Color is a fiction of light.” [Tacita Dean]

At some point in life, the world’s beauty becomes enough. You don’t need to photograph, paint, or even remember it. It is enough. [Toni Morrison]

“The greatest weapon against stress is our ability to choose one thought over another,” William James said. I don’t know what superpower William James enjoyed, but I can no more choose my thoughts than choose my name. The way [ I experience thoughts is] not as a choice but as a destiny. Not a catalog of my consciousness, but a refutation of it.

The really scary thing is not turning and turning in the widening gyre; it’s turning and turning in the tightening gyre. It’s getting getting sucked into a whirlpool that shrinks and shrinks and shrinks your world until you’re just spinning without moving, stuck inside a prison cell that is exactly the size of you, until eventually you realize that you’re not actually in a prison cell. You are the prison cell.

From an interview in the WSJ:

Several years ago, while sea kayaking off Washington state, author Darcey Steinke saw what a guide told her was the oldest-known killer whale on Earth, an orca named Granny believed by some experts to be 105 when she died in 2016. Ms. Steinke still thinks about this matriarch and others like her. (Killer whales and humans are the only known species whose females experience menopause.) “These whales lead their pods,” she said. “Nobody offers them hormone therapy. They just lead.”

From Bob Dylan’s A Hard Rain’s Gonna Fall:

I’m goin’ back out ‘fore the rain starts a-fallin’
I’ll walk to the depths of the deepest black forest
Where the people are many and their hands are all empty
Where the pellets of poison are flooding their waters
Where the home in the valley meets the damp dirty prison
Where the executioner’s face is always well-hidden
Where hunger is ugly, where souls are forgotten
Where black is the color, where none is the number
And I’ll tell it and think it and speak it and breathe it
And reflect it from the mountain so all souls can see it
Then I’ll stand on the ocean until I start sinkin’
But I’ll know my song well before I start singin’
And it’s a hard, it’s a hard, it’s a hard, it’s a hard
It’s a hard rain’s gonna fall

From Matilda Joslyn Gage — women’s suffragist, Native American rights activist, abolitionist, free thinker, and author, in 1887: “Oh, rebellious woman, to you the world looks in hope.” (Thanks to Kate Coffey for the link.)

10 Mar 01:24

Apple says you can safely use disinfecting wipes on iPhone screens

by Patrick O'Rourke
iPhone

Apple has confirmed that it is safe to use disinfecting wipes to keep your iPhone clean and free from potentially deadly viruses.

The tech giant recently updated its device cleaning tutorial page with a paragraph stating that it’s safe to use 70 percent isopropyl or Clorox wipes on Apple’s devices, including the iPhone, iPad, iMac and MacBook (the full paragraph can be found below).

Apple previously didn’t recommend using these cleaning products on any of its devices because they could scratch the display, according to The Wall Street Journal.

Apple’s help page mentions explicitly that is safe to use 70 percent isopropyl or clorox wipes on “nonporous surfaces” like the “display, keyboard or other exterior services.”

Of course, Apple’s pricey Pro Display XDR requires a specific cleaning kit.

Given the rapid spread of the coronavirus and the fact that touching contaminated objects is a way you can catch the illness, it makes sense for Apple to be changing its tune. Still, it’s strange that the company ever stated that it wasn’t safe to use these products on its devices.

Via: The Wall Street Journal Source: Apple 

The post Apple says you can safely use disinfecting wipes on iPhone screens appeared first on MobileSyrup.