Rolandt

Despite the introduction of the M1 chip in the new iPad Pro models, making the tablet more closely resemble a Mac from the inside, Greg Joswiak, Apple’s senior vice president of worldwide marketing, maintains that the iPad and Mac will not merge anytime soon.

Apple’s new 11-inch iPad Pro and 12.9-inch iPad Pro with the M1 chip is now very close to being a Mac (at least, internally), while the new 24-inch iMac resembles an enormous iPad. This has added further fuel to rumours that the two product lines could merge in the future — a claim that has always been refuted by Apple.

“There’s two conflicting stories people like to tell about the iPad and Mac,” says Joswiak, in an interview with The Independent. “On the one hand, people say that they are in conflict with each other. That somebody has to decide whether they want a Mac, or they want an iPad. Or people say that we’re merging them into one: that there’s really this grand conspiracy we have, to eliminate the two categories and make them one. And the reality is neither is true. We’re quite proud of the fact that we work really, really hard to create the best products in their respective category.”

In the same interview with The Independent, Apple hardware chief John Ternus reiterated Joswiak’s claim. “We’re pushing to make the best Mac we can make; we’re pushing to make the best iPad we can make,” Ternus said.

While the new M1-powered iPads likely feature the necessary system requirements needed to mimic a Mac and macOS, it isn’t likely we’ll see a hybrid of the two anytime soon.

For more on the new iPad Pro, check out our story focused on how it compares to last year’s model.

Source: Independent Via: Apple Insider

The post Existence of a Mac-iPad hybrid not likely to ever happen, says Apple appeared first on MobileSyrup.

Security researchers found a flaw with Apple’s AirDrop software that could expose personal information, but claim Apple hasn’t addressed the problem despite knowing about it since 2019.

Researchers at TU Darmstadt discovered that the process AirDrop uses to verify users can also expose personal information. AirDrop, if you haven’t used it, is a wireless communication technology used by Apple devices like iPhones and MacBooks to share files between devices.

AirDrop has three main ‘discovery’ settings that determine who can AirDrop you. Users can set AirDrop receiving to ‘off’ if they don’t want to receive files, ‘Everyone’ if they want to allow anyone nearby to send them files, or ‘Contacts only,’ which restricts AirDrop to functionality to people in your address book. That last mode is the default setting and also the one with the security flaw.

The researchers found that the verification mechanism that checks to see if both the sender and receiver are in each other’s address books could expose information like phone numbers and email addresses.

“The discovered problems are rooted in Apple’s use of hash functions for ‘obfuscating’ the exchanged phone numbers and email addresses during the discovery process,” the researchers explained in a blog post.

“However, researchers from TU Darmstadt already showed that hashing fails to provide privacy-preserving contact discovery as so-called hash values can be quickly reversed using simple techniques such as brute-force attacks.”

In other words, AirDrop uses a ‘hash function’ to disguise a user’s phone number and email address and then sends it to another device to verify if the information is in that device’s address book. But, that process can be easily reversed to reveal the phone number and email address.

Further, the researchers claim that an attacker only needs a few things to obtain this information: a Wi-Fi capable device, physical proximity to the target and for the target to initiate the discovery process by opening the sharing pane on an iOS or macOS device.

Worse, the researchers say they notified Apple about the vulnerability in May 2019 via responsible disclosure, but that Apple has not yet acknowledged the problem are indicated they’re working on a solution. Additionally, the researchers shared that they created a version of AirDrop dubbed ‘PrivateDrop’ that fixes the issue without significantly impacting AirDrop performance.

If you’re concerned about your information, the researchers note that you can protect yourself by disabling AirDrop (Settings > General >AirDrop > Select ‘Receiving Off’) or by avoiding opening the iOS share menu.

Source: TU Darmstadt Via: MacRumors

The post Apple’s AirDrop has a security flaw that could expose phone numbers, emails appeared first on MobileSyrup.

I come home from an afternoon of picture-taking with a memory card full of photos. I upload them, maybe wait a day or two, then take a look.

Many people keep all their photos, but through trial and error, I’ve found discarding the photos I don’t like to be more useful. I use the editing process to cull the photos that do not reflect my style, my point of view. I’m very critical of the photos, but not of myself, and that’s crucial—to judge the work, not the person.

Which photos make me happiest? Which photos have me nodding my head, exclaiming, “This! This is what I was after!”? Those are the ones to keep. If there are photos I’m on the fence about, I’ll keep them around for another month or two or however long it takes to resolve my opinion. But anything that feels off, like it could be done better, or feels generic, like anyone could have taken this photo? Those are the ones that I discard. And I discard a LOT.

Because it’s not just about taking pleasing photos. It’s about taking photos that I recognize as having come from my unique perspective.

And this editing down, not only does it mean more room on the external hard drive, it also helps me hone my point of view. Doing this with every shoot, I become more and more conscious of my preferences, my visual language.

“Why am I hesitant to delete this image?” That is an opportunity to become more in touch with my artistic sensibilities.

“Why didn’t this photo work?” is a chance to correct an error in our technique.

I use the editing process to question myself, to become more conscious of what I’m after, and where I go wrong.

This shot of Cheeno, for instance. I liked it… but it felt… not quite right. So I let it sit, came back to it maybe three months later. What do I like about the shot… her expression, her face. The background was… fine, her pose was…fine, but her face, framed by her hair, that calm expression, taking in the sunlight, that’s where my interest was, that’s what my eye kept being drawn to. The rest was distracting me from what I really wanted to focus on.

How to bring the viewer’s eye to this, to the exclusion of all else? I should’ve been closer, next time I’ll get closer, but… what if I just cropped it all away, leaving just the part I liked? And now we have a shot that I feel good about. Now it’s doing exactly what I want it to. The viewer sees it the way I saw it.

It’s not just sorting the good photos from the bad. It’s a process that I use towards greater self-awareness.

The more we understand our own work, the more we understand ourselves. The more we understand our own work, the better our photos will be the next time we go shooting.

And when put together with Communicating about what we want from a photo, repeating techniques until they are second nature, developing a sense of artistic competition as positive motivation, I (and now you) have a robust system, a set of tools that will inevitably lead us to more unique photos, photos that make us feel satisfied, photos that bear the stamp of our personal POV.

And that’s where the fun really starts.

How do you deal with self-critique? Let us know on Flickr Social!

There's a pretty good article about this in MIT Technology Review, but it's behind a paywall so I can't link to it (I have a full text version in my RSS feed reader; also see the comments here). The concept Hinton describes is called GLOM (derived from the slang "glom together"). The idea is simple but the tech is complex. Here's the idea: “Similarities of big vectors explain how neural networks do intuitive analogical reasoning.” A vector is an array of numbers that encodes information, for example, the xyz coordinates of a point. Any given perception can be represented as a really long vector - or as sets of multiple (and multidimensional) vectors. These subsets are similar to previously experienced vectors, allowing the neural net to extract parts from the whole. These are "islands of agreement". Why do I think this is a good idea? Because I had the same intuition in 1993 (note that I am in no way claiming to have discovered this; it's a very different thing to have an intuition and to flesh it out as a fully formed idea).

Ticks are vile little creatures that can transmit diseases like Lyme, Rocky Mountain Spotted Fever, and Powassan. According to the Center for Disease Control, these diseases are on the rise as ticks expand their geographic rage. “The number of counties in the northeastern and upper midwestern United States that are considered high-risk for Lyme disease increased by more than 300% between 1993 and 2012,“ they write. Lyme, the most prevalent tick-borne disease in America, is potentially debilitating and difficult to diagnose. We believe it is critical to protect yourselves from ticks. As we say in our bug repellent guide, a 20 percent concentration of picaridin repellent works well, but to really give yourself the highest level of protection we recommend using it in conjunction with permethrin-treated clothing.

Permethrin is an insecticide, not a repellent, so it will actually kill ticks and not just send them packing. It’s also different in that it is sprayed on your clothing rather than your skin. Once a piece of clothing is properly treated, the permethrin remains effective for weeks, if not months or years, depending on how it is applied.

What you can do now

You have a few ways to use permethrin. You can buy a spray and treat clothes you already own, buy already treated clothes, or send your clothes to a service for treatment.

Buying a permethrin spray is the easiest way to quickly take action. The most important thing is that the spray has a 0.5 % concentration of permethrin. There are a number of brands available, but we like the Sawyer brand, because of the range of sizes available and the easy spray nozzle.

Some permethrin sprays are made for a yard or for agricultural use. Since those formulas are meant to be sprayed on plants, they won’t stick to your clothes as well, according to Thomas Mather, aka The Tick Guy, director of the University of Rhode Island’s Center for Vector-Borne Disease and the TickEncounter Resource Center. Depending on what brand you end up with, treating clothes yourself means that the permethrin will last anywhere from four to six washings. Even if you don’t wash your treated shoes, Mather recommends re-treating them about once a month.

Buying a permethrin spray is the easiest way to quickly take action against ticks.

Sprays are a great everyday fix, but if you’re a dedicated outdoorsperson, looking into pretreated clothes might be worth your time (and remember, they also keep mosquitoes from biting you). If you want to buy pretreated garments, you have a ton of options: Amazon, Insect Shield, L.L.Bean, and REI all sell permethrin-treated clothes; Insect Shield also sells clothes just for kids (and a few things for dogs). These treatments last for about 70 washes.

Insect Shield can also treat clothes you already own. This is the same treatment the company applies to clothes it sells, both under its own brand and others (such as ExOfficio). Insect Shield charges per item of clothing but offers bulk discounts. Getting your treated clothes back takes about two weeks.

Pretreated is the way to go from a cost effectiveness standpoint, Mather told us. “Seventy washes probably gets closer to the life of the clothing for the most part. Four or five doesn't.” And it can be hard to remember to treat clothing yourself every month, he added. Buying it pretreated solves the upkeep problem.

Where to spray it and when

Ticks are less likely to bite if you’re wearing permethrin-treated clothes, and by far the most important thing to treat is what you’re wearing on your feet. Although studies have found that wearing a treated shirt or shorts makes ticks about two to four times less likely to bite, if you treat just your shoes and socks, you’re about 74 times less likely to be bitten by a tick than if you’re wearing untreated footwear, which is a pretty big deal.

“We've done tests with clothing, and we can watch the ticks fall off and die. So there is good scientific evidence that this works and it actually works pretty well,” said Mather. However, you have to be strategic about what treated clothes you wear and when.

In the spring and summer, it’s best to treat your socks and shoes. That could make you about 74 times less likely to be bitten by a tick, according to one study.

For the spring and summer, it’s best to treat your socks and shoes, Mather said. The immature, or nymph-stage, ticks are in the leaf litter at that time of year, and they’re most likely to get on your shoes and crawl up. They’re also so small that they can actually crawl through the weave of your socks, Mather said. “So spray your shoes the first of May, the first of June, first of July, first of August, and that will help you against the ticks that you can't see.”

In the fall, the adult-stage black-legged ticks come out. They tend to crawl up on plants and get on your body higher up, usually around your shins or knees, Mather said. “Then you would like to have treated pants, and you'd like to tuck your shirt tail in so that the ticks stay on the outside of your clothing longer.” If you’re still wearing shorts at that time, make sure to spray them both inside and out if they’re not pretreated.

In the winter, black-legged ticks can still be active as long as they're not frozen on the ground, Mather told us. “These are the first to emerge in the very early spring, followed fairly quickly by the American dog tick adults and the Lone Star tick, both nymph and adults.” So put those permethrin-treated pants back on when the snow melts.

Safety concerns

Some people worry that since permethrin is an insecticide, it will harm them. This is pretty unlikely. It kills ticks by interfering with how neurons fire in bugs’ little brains, causing them to spasm and die. But how our neurons fire is slightly different, plus we’re much larger and can metabolize permethrin before it can get to our nervous system (permethrin is over 2,250 times more toxic to ticks than to humans). Obviously, you shouldn’t eat it, but even if you’re exposed to a lot of permethrin, it’s unlikely to hurt you. According to the TickEncounter Resource Center permethrin fact sheet, a 140-pound person would have no adverse health effects even if exposed to 32 grams of permethrin in a day, and a bottle of clothing treatment has less than 1 gram of permethrin. (If you’re pregnant, know that animal studies have found no evidence that permethrin is harmful. The government gives it a Category B rating since there haven’t been meaningful permethrin studies with pregnant women.)

Permethrin can potentially harm bees, fish, and aquatic invertebrates. And oddly, cats—but only when it’s wet. When the permethrin spray dries, it’s okay for your cat to be around. This goes for fish too—if you step in a stream wearing permethrin-treated shoes, it won’t wash off and hurt the fish, Mather said. “Once it's dried onto the fiber, it doesn't come off very well. That's why it can go through the wash five or 70 times, depending on the mode of application, because it's stuck.”

But will it come off in the laundry and pollute the environment? That’s a much bigger question. Permethrin is a type of molecule known as a pyrethroid. These substances are pretty widely used: They’re in more than 3,500 registered products, including those used on pets and in treated clothing, in mosquito control, and in agriculture, according to the EPA. It’s also the main ingredient in some over-the-counter lice treatments. Permethrin spray for clothing is designed to stick to fiber and comes off only minimally in the wash. Still, research into pyrethroid pollution is ongoing.

Our favorite tick-check method

If you do get a tick, it’s important to remove the insect relatively quickly. Once one crawls onto your body, it can be hours until the tick attaches its horrible sawlike mouth onto you, and then another 12 hours until it starts transmitting disease—except for deer tick disease, which took only 15 minutes in a study on mice (yikes). Remember, black-legged ticks are tiny when they’re nymphs, about the size of a poppy seed (thank you for ruining all poppy seed things forever, CDC), so they can be hard to see.

Because ticks often attach in areas you tend not to pay much attention to—which makes finding them harder—Mather suggests doing tick checks fairly regularly, perhaps during one of life’s most intimate moments. “I noticed that I could see a lot if I just paid attention while I was sitting on the toilet,” Mather said. “I can see down the inside of both of my legs and behind my knees by doing a little twisting. I can push my junk to the left and to the right, and I can kind of check it out to see if I see any ticks there.” He did note that it’s hard to see your own butt, and you might still need help in that area. “But if I do that once or twice a day, I'm doing a pretty reasonable tick check just while I'm multitasking.”

If you do find a tick, don’t panic. First, get some fine-tipped tweezers. Grab the tick as close as you can to your skin, and pull it straight out. Rub the bite with alcohol and wash your hands. Take deep breaths. Eat a poppy seed muffin. Mather said you should try to ID the tick before doing away with it. The TickSpotters program can help identify the type of tick and give you a risk assessment if you send in a clear picture. “If the tick is a risky tick, attached long enough to transmit any germ they might be carrying, then we suggest that people might want to have the tick tested, for peace of mind, and to have more information to pass along to their primary care or veterinary care provider,” Mather said. If you come down with a fever a few weeks later, call your doctor. But remember that if the tick is on you for less than 24 hours, your chances of getting a tick-borne disease are small. And if you remember to wear your summer socks, you have a good chance of killing ticks before they attack.

Honda released a new North American pledge today to get closer to being carbon neutral and zero traffic collision fatalities by 2040.

On its way to phasing out gas-powered cars, the automotive company plans to offer 40 percent zero-emission vehicles by 2030, 80 percent by 2035 and 100 percent by 2040. On top of this, the company wants to be fully carbon neutral by 2050.

The automaker also announced a new electric vehicle (EV) platform called ‘e:Architecture’ that it says it will start releasing vehicles on in the second half of this decade. This is likely to be similar to GM’s Ultium battery or Volkswagens MEB platform.

The press release says that the brand wants to use all-solid-state batteries in its cars in the second half of the 2020s. This a newer form of battery tech that will supposedly help increase range and is safer than traditional Lithium-ion cells. Honda says that its tests are going well under lab conditions and it’s hoping to move forward with the project.

Until it can get e:Architecture off the ground, Honda says it will build two SUVs on GM’s platform. One is set to be sold through Acura and the other will be Honda-branded.

In North America specifically, the company also plans to supply its operations with 100 percent renewable energy. This includes North American manufacturing plants. The company has already bought carbon offsets and installed solar and wind turbines, but it still has a long way to go before it hits carbon neutral.

Source: Honda

The post Honda plans to fully embrace EVs in Canada by 2040 appeared first on MobileSyrup.

I want one of these. I’ve been looking at the option of building myself one, but would prefer to at least have a kit with the printed parts done in something sturdier (as I only have PLA and my printer is getting finicky).

As a direct side effect of having tracked this since it was announced on Twitter, I’ve also gone down the glorious rabbit hole at r/MechanicalKeyboards, but sadly there doesn’t seem to be anyone selling a 40% Planck layout with both Bluetooth and low-profile, silent switches.

The Apple Magic Keyboard is so great in that regard that it has spoiled me somewhat, and the two Logitech K380s I have are nearly as good…

Guten Sound in Clubhouse zu bringen war wirklich schwierig. Ich hatte zuletzt ein aufwändiges Setup mit einem Behringer FLOW 8, der in Deutschland mittlerweile ausverkauft war. Wir haben wohl zu viel Werbung dafür gemacht. Nun habe ich abgerüstet, nicht unbedingt finanziell, aber in jedem Fall, was die Anzahl der erforderlichen Kabel angeht. Und außerdem ist vieles einfacher geworden.

• Alles beginnt mit dem PodTrak P4. Der kostet aktuell 218 Euro. Das ist ein mobiler Podcast-Recorder, der vier Mikrofone mit 48 V Speisespannung versorgen kann und auch Anschlüsse für vier Kopfhörer hat. Der PodTrack hängt bei mir an einem USB-C Kabel, kann aber auch durch zwei AA-Batterien oder Akkus versorgt werden.
• An Kanal 1 des PodTrak habe ich ein Shure MV7 Mikrofon über ein nur 50 cm langes XLR-Kabel angeschlossen (4 Euro). Wer sich kein Shure-Mikro leisten will, kommt auch mit einem Tonor-Set für 65 Euro zu sehr ordentlichem Klang. Mikro, Ständer, Spinne, Poppschutz, Kabel, alles drin.
• Als Kopfhörer habe ich passend meinen Shure AONIC 50 angeschlossen, weil ich bei unserer täglichen Soundchecks exakt hören will,
• Das iPhone steht in einem Apple-Dock, das einen Ausgang für 3,5 mm TRRS-Klinke hat, alternativ nimmt man einen robusten Adapter für 20 Euro. Wichtig: Das Kabel muss beidseitig vierpolig sein. Ein Winkelstecker an einem Ende ist vorteilhaft, wie bei diesem Kabel. Vorteil PodTrak: Man muss nicht kompliziert zwei Wege splitten, sondern verbindet das iPhone mit einem einzigen Kabel, das an der rechten Seite eingesteckt wird und den Kanal 3 belegt.
• Das links eingesteckte USB-Kabel ist bei mir mit dem PC verbunden, wo ich ad-hoc aufnehme und Aufnahmen zurückspiele. Ich kann von dort aus auch Musik einspielen. Auf dem PodTrak belegt der PC den Kanal 4.

PodTrack, Mikro, Kopfhörer, PC, iPhone. Fünf Geräte, vier Kabel, einfacher geht es nicht. Mit den vier Reglern oben pegelt man die angeschlossenen Geräte, mit den Reglern unten die Kopfhörerausgänge, die immer den Main Mix wiedergeben. Den Kanal 2 habe ich noch frei für ein weiteres Gerät. Der PodTrack kann selbst alle vier Quellen aufzeichnen. Dazu gibt es für jeden Kanal eine Mute-Taste und vier Soundpads, die man mit eigenen Jingles programmieren kann.

Für den iPhone-Kanal und den USB-Kanal kann man noch festlegen, dass sie ein Mix-Minus rausschicken. Das brauche ich für Clubhouse, weil ich das, was ich von dort auf Kanal 3 höre, nicht wieder rausschicken will. Auf Kanal 4 könnte ich sogar andere Teilnehmer, zum Beispiel aus einem Teams-Call einspeisen.

Das eröffnet weitere Möglichkeiten. Das ganze Setup funktioniert ohne Änderung mit Zoom- oder Teams-Konferenzen, egal, ob sie auf dem iPhone oder dem PC laufen, inklusive Aufzeichnung auf dem PodTrack.

Over the past few months, the 10h-a-day pace that became a constant during the pandemic started creeping up as I took up a new role¹ and needed more and more time to get work done after a day’s worth of meetings. I ended up working the last couple of weekeends as well, so this one was almost exclusively devoted to downtime.

That meant reading a few books (am on my third one), lounging about the couch and staying away from most computers except my iPad while I soaked in some sunlight through the windows and mulled what has been going on…

To cut a long story short, I’ve been working around 12h a day of late due to my having taken on even more responsibility, and things reached a ludicrous level when one of our internal productivity tools gleefully reminded me that I was going to have 27 meetings the next day.

(And yes, it was 100% correct, with triple, quadruple and even quintuple conflicts on that calendar, many booked on top of slots I had explicitly marked as out-of-office.)

I then decided to start saying no to things, which, in a corporate culture (and role) where your prime goal (and focus) is to get into the thick of things and lead ongoing projects around (or over) any blockers can be tricky to do, to say the least.

After two or three “nos”, I was able to get back on track, and am now back at… 10 hours a day. Which remains tiresome, but a downward trend I will try to honor.

While I’m not unhappy about work (even if I were, the years have afforded me the ability to carry on through most things), the grind has been getting to me somewhat.

Getting interrupted on an hourly basis and having zero time to relax between context switches wears you down, and doubly so when your plans for the next few hours shift constantly and you can’t even follow up on previous commitments.

Toss in a couple of completely unrealistic requests and at least one particularly “zigzagging” initiative that has been all over the place, and I decided I need to take a serious break. If not immediately, then gradually so.

(I am also starting to have neck and back trouble from overwork–which my standing desk has helped mitigate, but not completely keep at bay.)

The fact that I can work like this (and achieve measurable outcomes while doing entire days’ worth of 30m context switches) doesn’t mean I like it, but after a while you need to question why and what and be extremely selective about what you actually do.

Under pressure is fine. Short on time is fine. We’ve all been there. Chaotic, however, is not fine, and my short term goal is to wrap up as many things as humanly possible and move on to a more manageable, saner subset.

I’m not mad, or sad, or depressed, or even annoyed at things, but merely tired of this messy, tiresome way to work and in need of a long(ish) break, preferably spent someplace I can take long walks, read voraciously, and maybe even work on something different (which, at this point, means anything not involving documents, presentations and endless meetings).

The most annoying thing for me this weekend was that even though I’ve been reading technical and musical history books, spending time coding or playing music was just not something I actually wanted to do, which is completely unlike me.

After a fair amount of pondering (and some fabulous ice cream) I realized that when your office has all of your gear in it and you spend all your waking hours there, even doing the dishes is a welcome change of scenery.

What happened was, I was fooling around with zero-knowledge proof ideas and needed to post public keys on the Internet in textual form. I picked ed25519 keys (elliptic-curve, also known as EdDSA) so I asked the Internet “How do you turn ed25519 keys into short text strings?” The answer took quite a bit of work to find and, after I posted it, provoked a discussion about whether I was doing the right thing. So today’s question is: Should these things be encoded with the traditional PKIX/PEM serialization, or should developers just blast the key-bits into base64 and ship that?

Previously

• Blog: How to Interchange Ed25519 Keys

• Code in Java and Go

• YCombinator discussion.

Old-school key wrapping

Traditionally, as described in the blog piece linked above, the public key, which might be a nontrivial data structure, is serialized into a byte blob which includes not just the key bits but metadata concerning which algorithm applies, bit lengths, and hash functions.

When I say “old-school” I mean really old, because the technologies involved in the process (ASN.1, PKIX, PEM) date back to the Eighties. They’re complicated, crufty, hard to understand, and not otherwise used in any modern applications I’ve ever heard of.

Having said all that, with a couple of days of digging and then help from YCombinator commentators, the Go and Java code linked above is short and reasonably straightforward and pretty fast, judging from my unit testing, which round-trips a thousand keys to text and back in a tiny fraction of a second.

Since the key serialization includes metadata, this buys you “Algorithm Agility”, meaning that if the flavor of key you’re using (or its supporting hash or whatever) became compromised and untrustworthy, you can change flavors and the code will still work. Which sounds like a valuable thing.

There is, after all, the prospect of quantum computing, which assuming that they can ever get the hardware to do anything useful, could crack lots of modern crypto notably including ed25519. I know very smart people who are betting on quantum being right around the corner, and others, equally smart, who think it’ll never work. Or that if it does, it won’t scale.

The simpler way

Multiple commentators pointed out that ed25519 keys and signatures aren’t data structures, just byte arrays. Further, that there are no options concerning bit length or hash algorithm or anything else. Thus, arguably, all the apparatus in the section just above adds no value. In fact, by introducing all the PKIX-related libraries, you increase the attack surface and arguably damage your security profile.

Furthermore, they argue, ed25519 is not likely to fail fast; if the algorithms start creeping up on it, there’ll be plenty of time to upgrade the software. I can testify that I learned of multiple in-flight projects that are going in EdDSA-and-nothing-else. And, to muddy the waters, another that’s invented its own serialization with “a 2-3 byte prefix to future proof things.”

Existence proof

I’m working on a zero-knowledge proof where there are two or more different public posts with different nonces, the same public key, and signatures. The private key is discarded after the nonces are signed and the posts are generated, and keypairs aren’t allowed to be re-used. In this particular case it’s really hard to imagine a scenario where I’d feel a need to switch algorithms.

Conclusions?

The question mark is because none of these are all that conclusive.

1. Algorithm agility is known to work, happens every time anyone sets up an HTTPS connection. It solves real problems.

   [Update: Maybe not. From Thomas Ptasek: Algorithm agility is bad… the essentially nonexistent vulnerabilities algorithm agility has mitigated over TLS's lifetime… ]

2. Whether or not we think it’s reasonable for people to build non-agile software that’s hardwired to a particular algorithm in general or EdDSA in particular, people are doing it.

3. I think it might be beneficial for someone to write a very short three-page RFC saying that, for those people, just do the simplest-possible Base64-ification of the bytes. It’d be a basis for interoperability. This would have the potential to spiral into a multi-year IETF bikeshed nightmare, though.

4. There might be a case for building a somewhat less convoluted and crufty agility architecture for current and future public-key-based applications. This might be based on COSE? This would definitely be a multi-year IETF slog, but I dunno, it does seem wrong that to get agility we have to import forty-year-old technologies that few understand and fewer like.

The current implementation

It’s sort of the worst of both worlds. Since it uses the PKIX voodoo, it has algorithm agility in principle, but in practice the code refuses to process any key that’s not ed25519. There’s an argument that, to be consistent, I should either go to brute-force base64 or wire in real algorithm agility.

Having said that, if you do need to do the PKIX dance with ed25519, those code snippets are probably useful because they’re simple and (I think) minimal.

And another thing. If I’m going to post something on the Internet with the purpose of having someone else consume it, I think it should be in a format that is described by an IETF RFC or W3C Rec or other stable open specification. I really believe that pretty strongly. So for now I’ll leave it the way it is.

There is a Japanese saying, (悪因悪果) that “from bad beginnings come bad endings”. Holding the Tokyo 2020 Olympics in the midst of a pandemic will not end well.

The Tokyo 2020 Olympics begin with a bribe and lie. That lie was told to the world when Prime Minister Abe assured them that Tokyo 2020 Olympics would be safe because the nuclear disaster at Fukushima was under control. It wasn’t under control then and it isn’t now. Deadly radioactive waste is spilling from 8000 corroded containers on site, the company running the operation, Tokyo Electric Power Company Holdings, will dump the tons of radioactive water on the site into the ocean in two years. They will keep dumping the water for years after, because sea water has to be pumped into the core to keep cooling the remains of the reactor. 

The Tokyo 2020 Olympics also begin with a bribe. That bride was given from the government through the Japan Olympic Committee (JOC) and channeled  by Dentsu, the largest advertising firm in Japan, to former members of the IOC (International Olympic Commission) to make sure that Japan won the bid. The French authorities investigated and the head of Japan’s Olympic commission resigned in disgrace. No one at the IOC or the JOC gives a fuck.

It bears repeating, if Fukushima nuclear disaster was really under control when Prime Minister Abe made that lie in 2014, the Japanese government wouldn’t be unilaterally deciding to dump nuclear contaminated waste into the ocean two years from now.

Now Japan claims that it has the pandemic under control. 

“Come to Tokyo! It’s perfectly safe!” 

It’s so safe that the government has banned attendance at all sporting events starting today—and plans to hold the world’s largest sporting event in three months. The safety protocols in place are underwhelming.

The 2020  Olympics which are very likely to be a catalyst for creating new and more terrible variance of the coronavirus, looks like a biological nuclear disaster waiting to happen. But just as Japan ignored warnings and coverups that led to the 2011 deadly disaster which displaced 160,000 people and will pollute the world for years to come, they are ignoring all sensible arguments to postpone or cancel the Olympics this year

The Tokyo Olympics are not something that the Japanese people want, they are something that a few old men in power want to hold desperately so they have something to add to their retirement scrapbooks. The majority of the Japanese people, nearly 80% do not want the Olympics to be held this year or want it be canceled. That is wise. Japan is in the middle of a state of emergency as coronavirus number surge here again, and anemic and poorly thought out countermeasures failed to stop the spread of the disease. People are dying and their dying faster than they have before. It took one year for the first 4000 people to die (January 16 2020–January 6 2021). The next 4000 died in less than two months. Today, 10,000 people will have died from COVID19 here. Neighboring countries in Asia have done much better.

At one point in time Japan’s Minister of Finance, bragged that Japan was able to handle the coronavirus without lockdowns or other stringent measures because of the superiority of the Japanese people. He can’t make that claim now. If you compare Japan to the United States or other countries in Europe, it seems to be doing very well, at least in terms of mortality. However if you could compare Japan to its Asian neighbors, it’s the worst kid on the block. Japan’s per 1000 people testing ratio is worse than Kazakhstan. It has refused to follow the successful examples of other countries in the region. Now there is a bit of a mystery as to why the death toll in Asia is so low, with theories that the genes are different or that an earlier less virulent form of the disease is already given people immunity, are that the BCG vaccine which was widely used in Asia especially the so-called Tokyo strain, gave those who received it what is called trained immunity. No one knows the answer. But here’s how it shakes out

Taiwan which has 1/5 the population of Japan, was the first country to warn the world of the, deadly virus, originating in China. Taiwan, thanks to strong leadership and a swift response, has done a remarkable job of containing the virus, without vaccines, so the people there are now living more or less a normal life. Korea, Vietnam, Thailand, Singapore, have all dealt with the virus better than Japan, if you count the number of deaths as a bear meter.

Singapore, Thailand, Vietnam, Hong Kong each country has had less than 100 deaths. Taiwan has only had 12 deaths. Even when adjusting the numbers of deaths to the population of each country, Japan has done a dismal job.

Japan now has 10,000 people dead from the coronavirus and more to come. Why has Japan done such a dismal job of protecting its own people from this virus?

Because time and time again the insane desire to put on the Olympics no matter what, has encouraged the country to take half ass measures to pretend that everything is all right, to squander opportunities to get the disease under control, and to put saving face before saving lives.

Tokyo is now in its 3rd State of Emergency. It will be lifted when the head of the IOC comes to visit the country. “We can’t have the IOC visiting Tokyo during a state of emergency, can we?”

Ask yourself, are the Tokyo 2020 Olympics worth holding if even one person dies as a result? How many deaths are acceptable?

Japan has wanted to save face over saving lives from the first reports of the deadly virus being issued from Taiwan—-the same day, January 16th, 2020 Japan had its first COVID19 case. When the infected cruise ship, The Diamond Princess, arrived on the shores of Japan, this nation refused to let the passengers be taken off board and treated at hospitals, because they didn’t want the numbers of infected and dead to be counted as Japan’s number. That wouldn’t look good for the Olympic Committee. So they kept them on board, effectively turning the ship into a giant floating Petri dish.

Then the government let the Japanese passengers leave the ship after insufficient testing and despite warnings that passengers not showing symptoms might still be carrying the disease. They went home by public transport—spreading the disease nationwide. Several turned out to be infected—the total number hasn’t been made public.

It also became clear that health care workers who had been aboard the Diamond Princess and staff from the Ministry of Health had become infected. However, at first the Japanese government refused to test them. Refused. And when they did test them, sure enough, there were infections.

Japan’s first cluster of coronovirus cases off the Diamond Princess was the Ministry of Health. It has been a clusterfuck ever since. The Olympics obsessed Abe government as well as Tokyo Governor Yuriko Koike seemed unconcerned about the coronavirus for weeks. Abe wined and dined the media while the virus was spreading. Koike barely mentioned the word until—-the Olympics were officially postponed on March 23rd 2020. The next day, Koike sounded the alarm bells, calling for a lockdown and the number of reported coronavirus cases miraculously surged. What a coincidence!

Japan has ignored the successful examples of other nations and steadfastly refused to test widely or test wisely. In the midst of the pandemic, the Japanese government ran a domestic tourism campaign, Go To Travel, which ensured that there was nowhere safe in the country from the coronavirus. Misguided efforts to prioritize the Olympics, to make Japan appear safer than it is, have delayed serious countermeasures and as a result, people have died. The mismanagement is so great that it is equivalent to professional negligence resulting in death and injury.

You could and you will argue that Japan has done so much better than the US or England. The relatives and loved ones of the 10,000 dead will tell you this irrelevant.

The Tokyo Olympics have already killed hundreds of people. They have been killed because priorities were screwed. If the Olympics continues, more people will die. Is even one death acceptable to hold what are, once you take away the hyperbole, simply games? Even one of Japan’s top athletes was brave enough to say what should be said, that human lives were more important than an international competition.

We know that the IOC has no moral compass. They have no qualms about hosting Olympics in China which is committing general genocide against a minority of its people. The only reason the IOC is not holding the Olympics in North Korea is the hermit Kingdom just doesn’t have enough money.

We should change the name of the IOC to stand for the International Oligarch Club, because that’s whom they appear to be serving.

Almost every media outlet in Japan is a sponsor of the Olympics, and having become a sponsor they have also become an accomplice in promoting the Olympics above public safety, and they should be ashamed of themselves. They aren’t.

Japan’s Olympic Committee will turn a blind eye to corruption, to bribes, to yakuza influence, to the real possibility that athletes die from heatstroke amidst Japan’s notoriously brutal summers.

Maybe I’m naïve, but if Japan and or rather the government of Japan, and the IOC actually gave a damn about the ideals espoused in the Olympics, they would suck up their losses and postpone the games to next year. And they hold them in the autumn (as they did in 1964) so fewer people die, or cancel the damn things altogether.

Japan’s Finance Minister Taro Aso also who asserted so stupidly that Japan’s so-called victory over the novel coronavirus was due to Japanese superior was right about one thing. He called the the Tokyo 2020 Olympics “cursed”

He’s right. For the sake of all the nations participating in the optics, and all the people living in Japan, it’s time to end that curse you. We only need two magic words, “Postpone” or “Cancel”.

Let’s see if the greedy clowns running the Tokyo 2020 Olympics have any decency and do the right thing, but I suspect if compassion were a Olympic event, the organizers wouldn’t even win a bronze medal.

The Tokyo Olympics (#Toxic2020) are a terrible idea in the middle of a pandemic—and were bought with a bribe and won with a lie. They do indeed seem to be cursed. If they are held as planned, it is likely to spread new and deadly variants of COVID19 to the public in Japan, and participants here may take back, along with their medals, new and deadly variants from Japan.

Time to end the curse. Let the IOC and the JOC know how you feel, before it’s too late.

After seeing Alison Roman’s recipe for key lime pie, I had nothing but Alison Roman’s key lime pie on the brain for days. The only way to engineer my way out of this was to actually make Alison Roman’s key lime pie. So I did.

In doing so I had the opportunity to use Dairy Isle Sweetened Condensed Milk for the first time; it’s made in ADL’s Perfection plant, just a few blocks away (Oliver and passed it every day on our walk to Birchwood Intermediate School). I used Purity heavy cream for the topping, also made in the neighbourhood.

Roman makes a big to-do about the difficulty of squeezing limes to make the required cup of lime juice, but I found it easy to do, if time-consuming, with a wooden reamer.

We had the pie for dessert tonight. It was everything I’d hoped it would be, the perfect mix of sweet and tart. I feel very capable.

This one is for the textbooks.

From The Hive:

… temporary free parking that began last year as a COVID-19 pandemic measure will be reduced   … pay parking will return …

Granville Island management had expected parking to be abundant due to travel restrictions, so they could afford to give drivers an incentive to come.  And drivers came – but not to support Granville Island businesses.  They came for the free parking.

“From employees of surrounding neighbourhood businesses to multiple groups parking to only ride their bikes along the seawall, and boaters gone for days and weeks at a time.

This, unfortunately, has resulted in a lack of parking near the Public Market for shoppers, the docks for kayak, paddleboard rentals and patios, and the artisans, restaurants and shops across the Island.”

In other words, the subsidy of free parking actually hurt the businesses it was meant to help.

However … I wonder what would happen if Granville Island tried out some experiments in active transportation – like cycling – that people actually drove there to experience.

Since the late 70s when the Island opened, the rights-of-way were primarily designed for vehicles (and for pedestrians to walk among them).  There are arrows to point out the primacy of the car:

From the hostile Bridge Street entrance to the one-way road system, it’s unsafe, confusing and frustrating for cyclists.  They cannot go where they obviously should be able to go.

When coming on to the Island, most bikes are heading for the Market.  Either of those roads would be direct routes. But it’s not legal to use them heading north, against the one-way traffic flow.  There are no separated lanes to allow for two-way cycling.

It’s been that way for so long, despite these transparent problems, that it’s fair to conclude Granville Island doesn’t really think it’s worth it to make changes, even experimentally, that would disrupt the assumptions of vehicle priority, and once again make for unhappy stakeholders.  Despite the conversations, nothing much changes.

Granville Island was a great experiment when it opened, allowing for a certain anarchy in the rights-of-way.  So maybe it’s time now for a little more anarchy.  Unless there is some accommodation, ignore the arrows.

Once this goes GA, most questions about whether or not Windows is a good development environment for UNIX folk will be moot (not just due to graphics support, but because the scaffolding for the entire thing will be battle-tested).

What I found really impressive, though, was having audio as well. Linux audio subsystems are such a pain that it feels borderline heroic to have it working already.

This is short story about a student finding something helpful in class and making my day, preceded by a long-ish back story.

In my programming languages course yesterday, I did a session on optimization. It's a topic of some importance, and students are usually interested in what it means for an interpreter or compiler to "optimize" code. I like to show students a concrete example that demonstrates the value of an optimization. Given where we are in the course and the curriculum, though, it would be difficult to do that with a full-featured language such as Python or Java, or even Racket. On the other end of the spectrum, the little languages they have been implementing and using all semester are too simple to benefit from meaningful optimization.

I found a sweet spot in between these extremes with BF. (Language alert!) I suppose it is more accurate to say that Eli Bendersky found the sweet spot, and I found Bendersky's work. Back in 2017, he wrote a series of blog posts on how to write just-in-time compilers, using BF as his playground. The first article in that series inspired me to implement something similar in Python and to adapt it for use with my students.

BF is well-suited for my purposes. It is very simple language, consisting of only eight low-level operators. It is possible to write a small interpreter for BF that students with only a background in data structures can understand. Even so, the language is Turing complete, which means that we can write interesting and arbitrarily complex programs.

The low-level simplicity of BF combines with its Turing completeness to create programs that are horribly inefficient if they are interpreted in a naive manner. There are many simple ways to optimize BF programs, including creating a jump table to speed up loops and parsing runs of identical opcodes (moves, increments, and decrements) as more efficient higher-level operators. Even better, the code to implement these optimizations is also understandable to a student with only data structures and a little background in programming languages.

My session is built around a pair of interpreters, one written in a naive fashion and the other implementing an optimization. This semester, we preprocessed BF programs to compute a table that makes jumping to the beginning or end of a loop an O(1) operation just like BF's other six primitives. The speed-up on big BF programs, such as factoring large numbers or computing a Mandelbrot set, is impressive.

Now to the story.

At the end of class, I talk a bit about esoteric languages more broadly as a way for programmers to test the boundaries of programming language design, or simply to have fun. I get to tell students a story about a four-hour flight back from OOPSLA one year during which I decided to roll a quick interpreter for Ook in Scheme. (What can I say; programming is fun.)

To illustrate some of the fun and show that programmers can be artists, too, I demo programs in the language Piet, which is named for the Dutch abstract painter Piet Mondrian. He created paintings that look like this:

That is not a Mondrian, but it is a legal program in the Piet language. It prints 'Piet'. Here is another legal Piet program:

It prints "Hello, World". Here's another:

That program reads an integer from standard input, determines whether it is prime or not, and prints 'Y' or 'N'. Finally, how about this:

If you are a certain age, you may notice something special about this image: It is made up exclusively of Tetris pieces. The program prints... "Tetris". Programming truly is an art!

One of my students was inspired. While reviewing the session notes, he searched for more information about Piet online and found this interactive editor. He then used it to create a Piet program in honor of a friend of his who passed away earlier this semester. It prints the Xbox gamertag of his late friend. In his email to me, he said that writing this program was therapeutic.

I'm not sure one of my class sessions has ever had a more important outcome. I'm also not sure that I have ever been happier to receive email from a student.

This has been a tough year for most everyone, and especially for students who are struggling with isolation and countermeasures against a nasty virus. I'm so glad that programming gave one student a little solace, at least for an evening. I'm also glad he shared his story with me.

photo by Noah Berger for AP

There is a perverse tendency in human nature for people who distrust and hate government (many of them self-described “libertarians”) to embrace fascism and other forms off totalitarianism and authoritarianism. First, there is an embrace of misanthropy and a rejection of democracy — as in “(This group of citizens) are too stupid or dangerous to be entitled to vote or otherwise participate in the political process”. “This group of citizens” may be defined by ethnic, racial, cultural, geographic, gender, caste, economic or social distinctions.

So the first sign of fascism is an attempt at selective disenfranchisement. That includes making registering and voting harder for the selected “enemy” group, the use of gerrymandering, transferring major powers and major government services to, and offering major no-bid contracts to, unelected “party faithful” members, and selling off public assets, properties and institutions to private interests known to be friends of the fascists.

Understandably, such attempts are going to be resisted and protested. So the second step in the slide into fascism is the selective suppression of dissent. The idea is to write oppressive laws in a weaselly way that allows enforcement agencies to apply them only against the fascists’ perceived “enemies”, while using code words to make it clear to fellow fascists that these laws, while theoretically non-discriminatory, will in fact only be used against opponents and not friends of the fascist parties. You know: “If you’ve done nothing wrong, you have nothing to be worried about…”.

The third step, which we have seen in places like Hungary and Poland of late, is the suppression, take-over, or shut-down of mainstream media that are not supportive of the fascists’ actions. It is hard, but not impossible, to get to the point that large swathes of the population are unable to hear in public places any opposition to the fascists’ talking points. Once the media are controlled, the time-worn adage that if you tell people a lie often enough they’ll believe it’s the truth, comes into play.

So the fourth step is to begin using these media as the mouthpieces for a steady barrage of misinformation, disinformation, hate-mongering, and other forms of fascist propaganda. Fear is sown so deeply and broadly this way that even moderates begin to doubt that what they’d always believed is true. Gaslighting is brought into play, both top-down, and, more importantly, at the local level — at the municipal, county and regional government levels that control most law enforcement.

And then the fifth step is to ensure that law enforcement, from local police to the military and the courts, are recruited to the fascist cause. That is often an easy step, since many of those recruited to law enforcement are those who have been traumatized by situations where brutality and atrocity was tolerated and hence are predisposed to favour harsh repression of, and low tolerance for, anyone perceived to be a threat to “personal or public safety”. Fascism, with its simplistic us-vs-them polarizing tactics, is pretty easy to sell to them, in a recipe of which the main ingredient is fear. And stacking the courts isn’t that hard once you’ve disenfranchised the opposition, and in some jurisdictions court appointments are for life.

And the sixth step is to develop, encourage and reward local fascist vigilante groups and “militias” to reinforce the police/military/court actions and “normalize” hate-crimes against the opponents of fascism, make it appear like “the average guy in the street” is on board.

Of course, the fascists will never use the term fascism to describe the nature, goals and effects of these actions. They will use terms like “fighting fraud and criminality”, “cracking down on anarchy”, “making the streets safe again”, returning to an idealized non-existent past, or even draw on old standbys like “eradicating communism” where that plays well.

These six steps pave the way for large-scale arrests, incarceration without charges, and “disappearances” of the identified enemies of the fascist regime, the cancellation of elections, the elimination of “term limits”, and the building of secret prisons, detention centres and “reeducation centres”. And, if that is successful, they may be inclined to invade other countries to eliminate “enemies” there too. Nothing at all new in any of this.

At this point, finally, it starts to twig to right-libertarians who have supported the fascist cause thus far, because they thought it was on the “right side”, that the regime has instituted the antithesis of libertarianism, and what they have helped bring about could easily be used against them. “First they came for…”.

But by this time it is too late. Once an initial easy-target enemy (often either a specific ethnic minority that is already envied for their success, or feared for their “different” behaviours, or just “immigrants” and “illegal aliens” in general) have been eliminated, the same formula can then be applied to other groups, until all opposition has been eliminated.

Germany and Italy in the 1930s (20 million killed), Russia under Stalin (60 million killed), China under Mao (80 million killed), most of the fascist regimes in so-called “third world countries”, Rwanda, Yugoslavia, you can see the same patterns so often before fascist and other totalitarian and authoritarian regimes became established. The pattern continues today, notably in some Latin American and Eastern European countries where democracy has never been more than an untested, theoretical ideal.

The US Republican Party has now fully embraced the principles of fascism and is moving rapidly to institute steps two through five, from the bottom up at least until the midterms. Dozens of draconian voter disenfranchisement and protest suppression initiatives have been passed or introduced in Republican-dominated states. Many of them are quite blatantly racist. You will soon, in some states, be exempt from prosecution if you run down protesters in your vehicle. This is how it starts.

US citizens have already been conditioned to accept gerrymandering and other voter suppression methods as “normal”. Republicans have been transferring government services (including prisons and military programs) to private friends since Reagan. The sell-off of public properties, assets, and institutions has accelerated with each Republican president and congress, in the interest of “balancing the budget”.

Republican conglomerates dominate the mainstream media in almost every area of the country, notably the execrable Faux News propaganda arm of the party. The attack on the few remaining moderate media outlets (“the liberal media”) has been incessant, and there are calls to disenfranchise NPR and PBS. Even the moderate outlets are faithfully repeating the fear-mongering and war-mongering anti-Chinese, anti-Russian, anti-Iranian talking points fed to them by the fascist-controlled American intelligence agencies, supported by the current faux-moderate administration.

The degree to which local police, even in large cities, and the military are breeding grounds for racism, and acts of oppression and even murder, has been brought into stark relief by recent trials. But essentially nothing has been or will be done about it. And the courts at every level have been packed with judges sympathetic to Republican suppression and oppression programs.

While the vigilantes and “militias” are in short-term retreat, that is mainly because they are not yet well-controlled and briefed on how to make their conduct politically and socially palatable. Expect to see them back, more disciplined, and better dressed, soon.

Some of the more severe fascist actions have been well-tested on small scales under both Republican and Democratic administrations. Secret prisons with waterboarding and other tortures were tested in several countries in the Middle East, and Guantanamo goes on and on. Homeland Security, the CIA, NSA and ICE are clearly already out of control of any elected government and are ripe for use by the next fascist regime, as is the Pentagon’s staggering and completely unaccounted-for budget. The FBI are known to use extortion and arbitrary detention on citizens and visitors to advance the US’s “security” authorities’ domestic and international spying and destabilization objectives.

So why? Why is a country that ostensibly stands for freedom slowly sliding into fascism?

My sense is it’s about fear. The evidence is everywhere that collapse is upon us — that the easy affluence (for some castes anyway) of the last 50 years is yielding to an era of scarcity, precarity, cascading crises, and widespread suffering. Denial is increasingly difficult to sustain. We feel it in our bones. The promise of a better future has evaporated, just as it did in many nations just before fascism took hold. Exhaustion and hopelessness are setting in. Someone, some group, must be held responsible, must be blamed. Something, anything, must be done to restore the dream.

There is a visceral buildup and outpouring of anger happening, which is so often a mask for fear. Talk radio, social media, barroom conversations, are rife with it.

I have long hoped that we would approach our fast-evolving multiple planetary emergencies the way we approached the Great Depression — with a renewed sense that we’re all in this together and have to work together to deal with it.

But now I fear we have been too long sold on the promise of endless “progress”, growth and prosperity, and its falsity is proving a bitter pill to swallow. Our capacity to function as communities has been lost, as communities have fragmented into anonymous, isolated “neighbourhoods”, where we don’t know and hence can’t trust our neighbours. And with the globalized economy we have lost the capacities, knowledge and skills needed to function as self-sufficient communities, and become almost infantilized — utterly dependent on huge, powerful, uncaring, profit-obsessed corporate oligarchies and soulless institutions, even for our essential goods and services. No wonder we’re scared!

And as the utter failure of the myth of perpetual growth and progress, and the devastation we have wrought that future generations will inherit, sinks in, the anger is exacerbated by a deep sense of shame, and of course denial. How could we have fucked up the world for our kids so badly? It just can’t be!

The small towns and isolated cities where the Republican fascist agenda is now entrenched, hear no other, more reasoned voices, only the endless drumbeat of Republican fascist propaganda, and the opprobrium and ridicule of faraway elitists calling them “deplorables”.

In the US, as a result, a civil war has begun, starting as a war of words and new, repressive laws. In much of the country, the war for the citizens’ hearts and minds has already been lost to the fascists.

This cannot end well. Instead of dealing with urgent, growing, global economic and ecological emergencies — the first wave of civilizational collapse — in a logical, coordinated, and consensual manner, it increasingly looks as if its next manifestation, at least in the US, will be the escalation of its civil war to physical violence, incarceration, oppression, and isolationism.

The lessons of history suggest that pulling back from the brink of fascism is unlikely — there is no deus ex machina, including Biden’s massive CoVid-19 bailout and infrastructure plans, that will change the situation on the ground for the majority of Americans who are giving up, or have already given up, on a peaceful means of coping with the deepening, intractable and seemingly hopeless situation that is their grim, unending day-to-day existence. It is now likely to be either civil war at home or a distracting and perilous foreign war against one or more of Biden’s (and the security apparatus’) identified overseas whipping boys — China, Russia, Iran, Syria, Venezuela, Cuba etc.

It’s hard to conceive which would be worse. The US might even face both simultaneously, like the Romans. Sadly, collapse is now inevitable in any case, but it would be nice to have the history books show that we at least did our best to address it, and to change course. When you speak with people who have survived a fascist regime, they always say they didn’t expect it to happen there. They always thought reason and good sense would prevail. I’m sure that twenty or thirty years from now they’ll be saying that about the US as well.

But it can’t be helped. Our species has shown neither the propensity nor the skill to learn the lessons of history, and that history is relentlessly violent and disappointing for believers in the myth of progress and human enlightenment.

America, despite its delusions of exceptionalism, has throughout its history often repeated the errors and atrocities of previous empires, and is doing so again now, on an unprecedented scale.

And the situation in many, perhaps most of the nations in the world, doesn’t look all that different, or too far behind America’s slide. The nations that smugly believe “It would never happen here” have likewise not read their history.

Still, we’ve done our best, all of us. It’s a shame that our naïveté, our incapacity to see and face the truth, and our massive collective trauma, have brought us to this ignominious point. We probably thought we’d at least last as long as the dinosaurs. No matter. After us, the dragons.

Once this goes GA, most questions about whether or not Windows is a good development environment for UNIX folk will be moot (not just due to graphics support, but because the scaffolding for the entire thing will be battle-tested).

What I found really impressive, though, was having audio as well. Linux audio subsystems are such a pain that it feels borderline heroic to have it working already.

Vaccines

The National Advisory Council on Immunizations today advised that AZ can be given to people as young as 30. They did also say, however, that mRNA vaccines were preferred.

I have talked before about how we are going to have SO. MUCH. VAX in the next year. Today, the famous high-tech powerhouse of Kazakhstan, announced that they have started delivering their own vaccine to their populace. We are going to have SO. MUCH. VAX.

Part of me wonders what is going to happen with the glut of vaccine suppliers. I think what’s going to happen is that we are going to enter a golden age of vaccines. We’ll get large-scale production of the new malaria vaccine and all kinds of other diseases — Zika and rhinoviruses and Lyme and Marburg and anthrax dengue fever and Trypanosomosis. Many countries are seeing the potential of mRNA technology; I expect to see mRNA used for vaccines for influenza and RSV and pneumonia and HIV and rabies and different kinds of cancers.

Speaking of gluts, the Canadian federal government has accepted paperwork for the developed-in-Canada vaccine Medicago to do a rolling acceptance, yay! (Alas, 90% is manufactured in the US right now. Medicago is building a plant in QC, but it won’t be ready until 2023.)

Vaccinations

One unknown in the Canadian vaccination drive is how many people will end up not taking a vaccine. For comparison, while USA seems to be slowing down a little with 40% having at least one dose, and the UK with 48% having at least one dose, Israel has fully vaccinated 80% of people over 16 (54% with two doses)!

North Dakota is vaccinating Canadian cross-border truckers. It’s probably more because they are afraid of our scary scary P.1 than the goodness of their hearts, but hey, I’ll take it. Thank you, North Dakota!

The world is going to hit one billion (billion with a B) COVID-19 vaccines delivered this weekend. Everyone all around the world likes to bitch about how ducked up the vax rollout in their country has been (well, except maybe Israel and Bhutan), but omg 1 billion doses only SIX MONTHS after the first trials finished? One year and 43 days after WHO declared a pandemic? That’s pretty amazing.

mkalus shared this story from Das Kraftfuttermischwerk.

Eine rbb-Doku über illegale Raves in Berlin und Brandenburg, während nicht nur Clubs gänzlich geschlossen sind. Ich tue mich schwer damit, das wertfrei zu sehen. Ich versuche wirklich, mich an die aktuellen Vorgaben zu halten, weiß dass das nicht immer einfach ist und erwische mich halt auch mal dabei, Dinge zu tun, von denen ich weiß, dass diese aktuell nicht richtig sind. Aber: ich würde halt gerade auf keinen Rave gehen, so sehr mir das auch fehlen mag.

Die Pandemie sorgt dafür, dass ausgerechnet die Subkultur verschwindet, die Berlin wie keine zweite nach außen repräsentiert: die elektronische Clubszene. Normalerweise dienen 140 Clubs als Anlaufstellen für feierwütige Berliner und Berlinerinnen, Läden wie das „Berghain“ oder das „Sisyphos“ genießen einen weltweit einzigartigen Ruf.

Doch seit März 2020 haben diese Clubs geschlossen – und die elektronische Szene verlagert sich auch in den Untergrund. Eine Schlüsselfunktion nimmt dabei die App Telegram ein. Sie bietet offenbar eine digitale Infrastruktur, die Feiernde vernetzt, Drogenlieferungen vereinfacht und die Arbeit der Polizei erschwert.

(Danke, Jan!)

Now that we’re starting to get widespread COVID vaccination “vaccine passports” have started to become more relevant. The idea behind a vaccine passport is that you would have some kind of credential that you could use to prove that you had been vaccinated against COVID; various entities (airlines, clubs, employers, etc.) might require such a passport as proof of vaccination. Right now deployment of this kind of mechanism is fairly limited: Israel has one called the green pass and the State of New York is using something called the Excelsior Pass based on some IBM tech.

Like just about everything surrounding COVID, there has been a huge amount of controversy around vaccine passports (see, for instance, this EFF post, ACLU post, or this NYT article).

There two seem to be four major sets of complaints:

1. Requiring vaccination is inherently a threat to people’s freedom
2. Because vaccine distribution has been unfair, with a number of communities having trouble getting vaccines, a requirement to get vaccinated increases inequity and vaccine passports enable that.
3. Vaccine passports might be implemented in a way that is inaccessible for people without access to technology (especially to smartphones).
4. Vaccine passports might be implemented in a way that is a threat to user privacy and security.

I don’t have anything particularly new to say about the first two questions, which aren’t really about technology but rather about ethics and political science, so, I don’t think it’s that helpful to weigh in on them, except to observe that vaccination requirements are nothing new: it’s routine to require children to be vaccinate to go to school, people to be vaccinated to enter certain countries, etc. That isn’t to say that this practice is without problems but merely that it’s already quite widespread, so we have a bunch of prior art here. On the other hand, the questions of how to design a vaccine passport system are squarely technical; the rest of this post will be about that.

What are we trying to accomplish?

As usual, we want to start by asking what we’re trying to accomplish At a high level, we have a system in which a vaccinated person (VP) needs to demonstrate to some entity (the Relying Party (RP)) that they have been vaccinated within some relevant time period. This brings with it some security requirements”

1. Unforgeability: It should not be possible for an unvaccinated person to persuade the RP that they have been vaccinated.
2. Information minimization: The RP should learn as little as possible about the VP, consistent with unforgeability.
3. Untraceability: Nobody but the VP and RP should know which RPs the VP has proven their status to.

I want to note at this point that there has been a huge amount of emphasis on the unforgeability property, but it’s fairly unclear — at least to me — how important it really is. We’ve had trivially forgeable paper-based vaccination records for years and I’m not aware of any evidence of widespread fraud. However, this seems to be something people are really concerned about — perhaps due to how polarized the questions of vaccination and masks have become — and we have already heard some reports of sales of fake vaccine cards, so perhaps we really do need to worry about cheating. It’s certainly true that people are talking about requiring proof of COVID vaccination in many more settings than, for instance, proof of measles vaccination, so there is somewhat more incentive to cheat. In any case, the privacy requirements are a real concern.

In addition, we have some functional requirements/desiderata:

1. The system should be cheap to bring up and operate.
2. It should be easy for VPs to get whatever credential they need and to replace it if it is lost or destroyed.
3. VPs should not be required to have some sort of device (e.g., a smartphone).

The Current State

In the US, most people who are getting vaccinated are getting paper vaccination cards that look like this:

This card is a useful record that you’ve been vaccinated, with which vaccine, and when you have to come back, but it’s also trivially forgeable. Given that they’re made of paper with effectively no anti-counterfeiting measures (not even the ones that are in currency), it would be easy to make one yourself, and there are already people selling them online. As I said above, it’s not clear entirely how much we ought to worry about fraud, but if we do, these cards aren’t up to the task. In any case, they also have suboptimal information minimization properties: it’s not necessary to know how old you are or which vaccine you got in order to know whether you were vaccinated.

The cards are pretty good on the traceability front: nobody but you and the RP learns anything, and they’re cheap to make and use, without requiring any kind of device on the user’s side. They’re not that convenient if you lose them, but given how cheap they are to make, it’s not the worst thing in the world if the place you got vaccinated has to mail you a new one.

Improving The Situation

A good place to start is to ask how to improve the paper design to address the concerns above.

The data minimization issue is actually fairly easy to address: just don’t put unnecessary information on the card: as I said, there’s no reason to have your DOB or the vaccine type on the piece of paper you use for proof.

However, it’s actually not straightforward to remove your name. The reason for this is that the RP needs to be able to determine that the credential actually applies to you rather than to someone else. Even if we assume that the credential is tamper-resistant (see below), that doesn’t mean it belongs to you. There are really two main ways to address this:

1. Have the VP’s name (or some ID number) on the credential and require them to provide a biometric credential (i.e., a photo ID) that proves they are the right person.
2. Embed a biometric directly into the credential.

This should all be fairly familiar because it’s exactly the same as other situations where you prove your identity. For instance, when you get on a plane, TSA or the airline reads your boarding pass, which has your name, and then uses your photo ID to compare that to your face and decide if it’s really you (this is option 1). By contrast, when you want to prove you are licensed to drive, you present a credential that has your biometrics directly embedded (i.e., a drivers license).

This leaves us with the question of how to make the credential tamper-resistant. There are two major approaches here:

1. Make the credential physically tamper-resistant
2. Make the credential digitally tamper-resistant

Physically Tamper-Resistant Credentials

A physically tamper-resistant credential is just one which is hard to change or for unauthorized people to manufacture. This usually includes features like holograms, tamper-evident sealing (so that you can’t disassemble it without leaving traces) etc. Most of us have lot of experience with physically tamper-resistant credentials such as passports, drivers licenses, etc. These generally aren’t completely impossible to forge, but they’re designed to be somewhat difficult. From a threat model perspective, this is probably fine; after all we’re not trying to make it impossible to pretend to be vaccinated, just difficult enough that most people won’t try.

In principal, this kind of credential has excellent privacy because it’s read by a human RP rather than some machine. Of course, one could take a photo of it, but there’s no need to. As an analogy, if you go to a bar and show your driver’s license to prove you are over 21, that doesn’t necessarily create a digital record. Unfortunately for privacy, increasingly those kinds of previously analog admissions processes are actually done by scanning the credential (which usually has some machine readable data), thus significantly reducing the privacy benefit.

The main problem with a physically tamper-resistant credential is that it’s expensive to make and that by necessity you need to limit the number of people who can make it: if it’s cheap to buy the equipment to make the credential then it will also be cheap to forge. This is inconsistent with rapidly issuing credentials concurrently with vaccinating people: when I got vaccinated there were probably 25 staff checking people in and each one had a stack of cards. It’s hard to see how you would scale the production of tamper-resistant plastic cards to an operation like this, let alone to one that happens at doctors offices and pharmacies all over the country. It’s potentially possible that they could report people’s names to some central authority which then makes the cards, but even then we have scaling issues, especially if you want the cards to be available 2 weeks after vaccination. A related problem is that if you lose the card, it’s hard to replace because you have the same issuing problem.^[1]

Digitally Tamper-Resistant Credentials

The major alternative here is to design a digitally tamper-resistant system. Effectively what this means is that the issuing authority digitally signs a credential. This provides cryptographically strong authentication of the data in the credential in such a way that anyone can verify it as long as they have the right software. The credential just needs to contain the same information as would be on the paper credential: the fact that you were vaccinated (and potentially a validity date) plus either your name (so you can show your photo id) or your identity (so the RP can directly match it against you).

This design has a number of nice properties. First, it’s cheap to manufacture: you can do the signing on a smartphone app.^[2] It doesn’t need any special machinery from the RP: you can encode the credential as a 2-D bar code which the VP can show on their phone or print out. And they can make as many copies as they want, just like your airline boarding pass.

The major drawback of this design is that it requires special software on the RP side to read the 2D bar code, verify the digital signature, and verify the result. However, this software is relatively straightforward to write and can run on any smartphone, using the camera to read the bar code.^[3] So, while this is somewhat of a pain, it’s not that big a deal.

This design also has generally good privacy properties: the information encoded in credential is (or at least can be) the minimal set needed to validate that you are you and that you are vaccinated, and because the credential can be locally verified, there’s no central authority which learns where you go. Or, at least, it’s not necessary for there to be a central authority: nothing stops the RP from reporting that you were present back to some central location, but that’s just inherent in them getting your name and picture. As far as I know, there’s no way to prevent that, though if the credential just contains your picture rather than an identifier, it’s somewhat better (though the code itself is still unique, so you can be tracked) especially because the RP can always capture your picture anyway.^[4]

By this point you should be getting the impression that signed credentials are a pretty good design, and it’s no surprise that this seems to be the design that WHO has in mind for their smart vaccination certificate. They seem to envision encoding quite a bit more information than is strictly required for a “yes/no” decision and then having a “selective disclosure” feature that would just have that information and can be encoded in a bar code.

What about Green Pass, Excelsior Pass, etc?

So what are people actually rolling out in the field? The Israeli Green Pass seems to be basically this: a signed credential. It’s got a QR code which you read with an app and the app then displays the ID number and an expiration data. You then compare the ID number to the user’s ID to verify that they are the right person.

I’ve had a lot of trouble figuring out what the Excelsior Pass does. Based on the NY Excelsior Pass FAQ, which says that “you can print a paper Pass, take a screen shot of your Pass, or save it to the Excelsior Pass Wallet mobile app”, it sounds like it’s the same kind of thing as Green Pass, but that’s hardly definitive. I’ve been trying to get a copy of the specification for this technology and will report back if I manage to learn more.

What About the Blockchain?

Something that keeps coming up here is the use of blockchain for vaccine passports. You’ll notice that my description above doesn’t have anything about the blockchain but, for instance, the Excelsior Pass says it is built on IBM’s digital health pass which is apparently “built on IBM blockchain technology” and says “Protects user data so that it remains private when generating credentials. Blockchain and cryptography provide credentials that are tamper-proof and trusted.” As another example, in this webinar on the Linux Foundation’s COVID-19 Credentials Initiative, Kaliya Young answers a question on blockchain by saying that the root keys for the signers would be stored in the blockchain.

To be honest, I find this all kind of puzzling; as far as I can tell there’s no useful role for the blockchain here. To oversimplify, the major purpose of a blockchain is to arrange for global consensus about some set of facts (for instance, the set of financial transactions that has happened) but that’s not necessary in this case: the structure of a vaccine credential is that some health authority asserts that a given person have been vaccinated. We do need relying parties to know the set of health authorities, but we have existing solutions for that (at a high level, you just build the root keys into the verifying apps).^[5] If anyone has more details on why a blockchain^[6] is useful for this application I’d be interested in hearing them.

Is this stuff any good?

It’s hard to tell. As discussed above, some of these designs seem to be superficially sensible, but even if the overall design is sensible, there are lots of ways to implement it incorrectly. It’s quite concerning not to have published specifications for the exact structure of the credentials. Without having a detailed specification, it’s not possible to determine that it has the claimed security and privacy properties. The protocols that run the Web and the Internet are open which not only allows anyone to implement them, but also to verify their security and privacy properties. If we’re going to have vaccine passports, they should be open as well.

Updated: 2021-04-02 10:10 AM to point to Mozilla’s previous work on blockchain and identity.

1. Of course, you could be issued multiple cards, as they’re not transferable. ↩︎
2. There are some logistical issues around exactly who can sign: you probably don’t want everyone at the clinic to have a signing key, but you can have some central signer. ↩︎
3. Indeed, in Santa Clara County, where I got vaccinated, your appointment confirmation is a 2D bar code which you print out and they scan onsite. ↩︎
4. If you’re familiar with TLS, this is going to sound a lot like a digital certificate, and you might wonder whether revocation is a privacy issue the way that it is with WebPKI and OCSP. The answer is more or less “no”. There’s no real reason to revoke individual credentials and so the only real problem is revoking signing certificates. That’s likely to happen quite infrequently, so we can either ignore it, disseminate a certificate revocation list, or have central status checking just for them. ↩︎
5. Obviously, you won’t be signing every credential with the root keys, but you use those to sign some other keys, building a chain of trust down to keys which you can use to sign the user credentials. ↩︎

6. Because of the large amount of interest in blockchain technologies, there’s a tendency to try to sprinkle it in places it doesn’t help, especially in the identity space For that reason, it’s really important to ask what benefits it’s bringing. ↩︎

The post Notes on Implementing Vaccine Passports appeared first on The Mozilla Blog.

by Igor

Greg wanted to have us build up a Neutrino Mini-Velo for commuting, travel, and a bit of fun on gravel roads. 

He opted for Tourist Bars to keep the position upright and comfortable. At 570cm in width, the Tourist is narrow by todays standards, but the sweep, rise, and grip length makes for a magnificent cockpit position. It's a personal favorite of ours.

Wheels are our Disc Hubs laced to Velocity Cliffhanger Rims in 32h drilling. And the rubber is Maxxis Grifter in 20"x2.3" (406 bsd). 

Drivetrain is Sram with a Rival 1 rear derailleur, Apex 1 crankset, 11-32 cassette, and Apex 11 shifter.

Braking is handled by TRP Spyres connected to our Grand Cru Brake Levers in Silver.

We also put on a Flat Pack Rack for general portage. 

All-in-all, this is a very fun bike. I'm actually thinking of rebuilding mine with a few cues from this one. 

As far as a status of Neutrino frames, we sold out of the S and L sizes basically overnight. I honestly thought they would have lasted longer, but they were basically in and out to places all around the world. We're getting more in the Summer, so keep your eyes peeled!

If you want the complete build list, here you go! https://velo-orange.com/pages/neutrino-build-list-commuter-traveler-with-flat-pack-rack

After the fatal Tesla Model S crash in the U.S. last week, Consumer Reports confirms that you can trick a Model Y into driving on Autopilot without anyone in the driver’s seat.

It should be noted that Tesla claims that Autopilot wasn’t enabled when the crash that killed two people took place in Texas, but local authorities claim that no one was sitting in the driver’s seat. Tesla CEO Elon Musk also tweeted that Autopilot couldn’t have been engaged on the road with the crash because there were no painted road lines. That being said, Tesla owners have taken to Twitter to show that this can also be circumvented.

Consumer Reports also clarifies that this is extremely dangerous to try, and it only did so on a closed test track.

The publication says the car drove along fine in its tests, but it failed to make sure the driver was paying attention and even notice that the driver’s seat was empty. This actually puts Tesla at a serious disadvantage since GM and Ford each use a camera to make sure the driver is looking at the road.

To get this to work, it needed to start driving with someone sitting in the driver’s seat on top of the buckled seatbelt (it needs to be buckled to engage Autopilot). Then, once the car started driving along, they lowered the speed down to zero, attached a weight to the steering wheel to simulate the weight of a human hand and hopped over to the passenger seat without opening the doors (that disengages Autopilot).

While this seems like a bit of a hassle, it does prove that it’s possible. All of this is to say that if you have a Tesla, keep your eye on the road before you kill someone or yourself.

Source: Consumer Reports 

The post Consumer Reports says you can trick a Tesla to drive by itself appeared first on MobileSyrup.

You may recall that MLA Hannah Bell tabled Motion 30, Social Assistance Vision Care, just under a month ago, in the Legislative Assembly of PEI.

From my own District 12 MLA, Karla Bernard, I learned that when motions get called for debate is managed by the party introducing them: each party is allotted, by negotiation, a certain amount of time, and budgeting that time is a complex dance involving other legislative business, the relative importance of various bills and motions, and other factors.

Motion 30’s time to shine finally arrived yesterday, and here is the video of the debate that ensued (original source here):

If you watched until the end, you will know that the bill passed unanimously, and that government, through the Minister of Social Development and Housing, committed to increasing the rates for eye examinations, lenses, and frames once the budget, currently being debated, passes.

There’s many a slip between the cup and the lip, so we don’t know right now the what and the when and the how of this, but the unanimous vote, with support from government members, the opposition, and the third party, must surely help offer momentum.

In conversations with optometrists and opticians in recent weeks, I know how transformative increasing support for vision care will be for many Islanders: the rates have languished for decades at their current level, leaving many with substandard vision.

Kudos to Hannah Bell, seconder Trish Altass, Minister of Social Development and Housing Brad Trivers, and all members of the Legislative Assembly for moving this forward.

How we rebuilt a 20-year-old system that lets New York Times photo editors search through our vast store of photo outtakes.

By Kate Brenner

On an assignment, a New York Times photographer might take hundreds of photos but only file a handful of their best shots. These best photos, often referred to as “selects,” are typically uploaded directly from the photographer’s camera to our CMS for a photo editor to access. It is a workflow that is geared towards efficiency, and to get the news out as quickly as possible.

But just because a few photos have been chosen to publish doesn’t mean that the outtakes are unusable or irrelevant. Many of these photos do in fact get published, sometimes days or even years later. Over time, The Times has collected these outtakes, which number more than 150 million, and has stored them in Amazon Web Services (AWS) storage gateways.

For Times photo editors to access the outtakes, an in-house team built a system called Scrounger, which was written as a series of Perl scripts that ran on an AWS server. Scrounger had a search UI that editors could use to find photos, but only if they knew the numerical assignment ID that is generated with each news assignment.

If an engineer needed to access the photos on the back-end, they had to connect to a remote desktop and sift through the storage gateways rendered as network drives. While this was a challenge in itself, there was another major flaw: no descriptive information was stored in a database, which meant there was no efficient way to query this massive collection of images. The only searchable information was in a photo’s file path, which included the date of the assignment, the assignment ID number and the file name, which looked something like this: 20210401/12345678A/test_photo_1.jpg.

In the past few years, the Times Photo Team has been systematically updating our photo management software. After updating the search algorithm for our main search tool for selects, we set our sights on building a system for outtakes — a Scrounger 2.0.

Making the images accessible

Some of the Photo team’s work has included consolidating our systems onto a single cloud provider — we use Google Cloud Platform (GCP). Our first step in creating Scrounger 2.0 was to transfer all of Scrounger’s photos to Google Cloud Storage (GCS) to make them easily accessible to our system.

This was more complex than we anticipated. Because the photos were stored in AWS storage gateways, they first had to be synched to an Amazon S3 bucket before they could be transferred to GCS using Google’s Storage Transfer Service.

Luckily, we were able to preserve the file path for each photo. This was a huge win because most of the information we had about these photos was in the file path.

Architecting a new system

Once the photos were in GCS, we had to determine the best way to efficiently ingest and store them. We built a database to keep relevant information about each photo, and we created a flexible schema that could be updated as needed.

We built four tables in our new system.

1. The Metastore table, which was the raw dump of all the information we had about an incoming photo and included all image metadata stored as JSON. This table was set up with a trigger to send each row to the second table.
2. The Event Queue table, in which rows were stored temporarily until they were processed and inserted into the third table.
3. The Assets table, which contained columns for metadata fields, such as description or credit, that were relevant to the photo. If we need to update the schema of the Assets table, we can “replay” ingestion and resend all rows from the Metastore table to the Assets table, without duplicating any records.
4. The Search table combined several columns from the Assets table, such as the identifier, creator, description, filename and assignment ID as a tsvector, so we could apply a full-text search on these fields. We could have our API query for assets on many different query parameters.

We built an application called Gateway as the initial point of ingestion for all past and future outtakes. Gateway listens for notifications of incoming images and runs exiftool on each image to combine its image metadata with GCS metadata and store all of that information in the Metastore table. An application called Megaphone listens for database events and publishes them to two more applications, Materializer and Enhancer. Materializer scans and externalizes specific metadata fields, such as photographer and caption, and stores them in the Assets table. Enhancer updates the Assets table with additional information from The Times’s assignment database, such as credit or photographer.

Combining the information from the original file path, externalizing several different forms of metadata, using information gathered from the assignments table, and storing all of this information in our new schema enabled us to make these photos searchable.

We wrote an API in python to interact with our database, and we created a UI for Scrounger 2.0 in React.

Importing the images into the new system

Once the new system was built, we ran the migration and kicked off ingestion for all 150 million photos. We created a temporary manifest from the image dump from the old system. For each photo in the GCS bucket, we extracted and parsed its original storage path to obtain the assignment ID, assignment date, file name, file size and file extension. We then put all that information into the manifest so we could perform queries and learn more about the photos in the image dump.

From there, we set up ingestion into the new system. We wrote a script that iterated through the manifest and applied each image’s metadata to the corresponding GCS object. Our GCS bucket sent notifications to the Gateway service with each metadata update.

To verify that all 150 million records had indeed been transferred over, we wrote a verifier script that iterated through the metastore table. For each Scrounger 1.0 entry, the script ensured that every row had been ingested into Scrounger 2.0 and it updated the temporary manifest with a status of “verified.”

Because the old system stored multiple versions of each image — such as high resolution, medium resolution and thumbnail — we needed to deduplicate them. Storing all resolutions from the old system would take up too much space in the Assets table, and it would create noisy search results in the UI. We didn’t want to permanently delete any photos, so we added a special metadata field that leaves the files in the Metastore table but deletes them from the Assets table. Since we still need different crops for different views in the UI, we created an API that can cut and cache renditions on the fly without taking up unnecessary storage space.

Final Touches

Since we launched Scrounger 2.0, our team has implemented enhancements around search, such as sorting and filtering capabilities. Now, in addition to assignment ID, Times editors can search by a variety of terms, such as photographer name or keywords that might be found in a photo’s description.

There is plenty more work to be done on this system. Not every outtake was stored in Scrounger 1.0; some outtakes were stored on DVDs in the Times newsroom. Now that our new system is up, we hope to finish transferring all of these images to help preserve the historical record these photos contain.

Kate Brenner is a software engineer on the Photo Team at The New York Times. When she’s not coding, she enjoys running, going on adventures with her dog and exploring food that can be used as a vehicle for ketchup.

The Photo Team is: Farah Abbasi, Frank Borell, Kate Brenner, Chris Frank, Chris Grillo, Sherman Hewitt, Jenny Hottle, Michael Laing, Michael Moffitt, Shonta’ Singleton and Sharon Tartarone. Special thanks to Photo Team alum Suman Roy.

Over 150 Million Photos, Some Perl Scripts and a Creaky User Interface was originally published in NYT Open on Medium, where people are continuing the conversation by highlighting and responding to this story.