Rolandt

The London School of Hygiene & Tropical Medicine’s COVID-19 vaccine tracker lists 292 vaccines in development as of 2021-01-26, and there is at least one not on that list yet. At least 69 vaccine candidates are currently in clinical trials.

Why are there so many? Surely, you say, we don’t need so many candidates. Why did so many companies try? Why don’t most of them quit now. Why don’t the losers just manufacture the winners’ vaccines?

There are several reasons why there are so many.

1. There are nearly 8 billion people out there. That huge market means a huge opportunity to make money, so many people wanted to invest.
2. There are lots of different market segments out there, and not all vaccines are appropriate for all markets. For example:
   • Wealthy countries with good infrastructure are able to cope with the refrigeration demands of the mRNA vaccines, while poor countries might not be able to.
   • Wealthy countries can buy expensive vaccines, while poor countries might not be able to afford it.
   • Some vaccines do not work well in certain populations. For example, Sanofi/GSK’s first vaccine candidate’s formulation did not give good results in the elderly.
3. Countries have a financial incentive to fund domestic development of vaccines. COVID-19 costs a huge, huge amount in lives, money, and social well-being, for every single day that the pandemic continues. Compared to being in a pandemic longer, a vaccine development program is cheap.
4. Countries have domestic security reasons to develop their own vaccines:
   • Countries would rather not have to inject their citizens with liquids coming from their political rivals. For example, Taiwan might not want not trust vaccines from China.
   • There are (IMHO legitimate) concerns that other countries might slap export controls on vaccines developed in their home countries, insisting that those vaccines go to their own citizens first. Countries have no control over when they can have access to foreign vaccines, but might be able to exert some control over domestic providers’ priority and sequencing choices.
5. Players — both countries and companies — have an incentive to invest in vaccine technology to ensure long-term competitiveness, especially for mRNA vaccines. The mRNA vaccines are so good that being able to make them domestically is a huge strength, both in terms of being able to make your citizens healthier in the future and in terms of increasing your country’s economic might.

As for the question of why the losers don’t just manufacture the “winners'” vaccines, in addition to #5 above:

• The “losers” might not have accepted yet that they have “lost”. Even if the pandemic eases in the developed countries in the next year, there’s still going to be billions of people who still need the vaccine. So even if the “loser” vaccine doesn’t get to market for a year, there’s still a lot of time to make money on the vaccine after that.
• The “winners” might not feel comfortable licensing their manufacturing technology to their rivals. Why should Moderna show Providence Therapeutics how to make mRNA vaccines, when Providence might turn into a competitor later on?
  • It would make more sense for the “winners” to flat-out buy their competitors. The winners presumably are making money right now, so they ought to be able to afford it.
• The “winners” are busy right now. They are making (and selling!) vaccines as fast as they can at the moment. Technology transfer deals — or outright purchases — take time and attention, and companies like Moderna probably do not have any attention to spare at the moment. Look for deals in a year or two. (Right now, Pfizer is doing an internal upgrade to its factory to boost production, causing a short-term drop in supply, causing people to totally lose their collective shit enormous consternation among their customers.)

At the 2015 Postgres conference, the great IT philosopher Robert r0ml Lefkowitz delivered a talk entitled The Image of Postgres. Here’s the blurb.

How do you think about what a database is? If you think of a database as only a place to store your data, then perhaps it does not really matter what the internals of that database are; all you really need is a home for your data to be managed, nothing more.

If you think of a database as a place where you develop applications, then your expectations of your database software change. No longer do you only need data management capabilities, but you require processing functions, the ability to load in additional libraries, interface with other databases, and perhaps even additional language support.

If your database is just for storage, there are plenty of options. If your database is your development framework, you need Postgres.

Why? Well, let’s get philosophical.

For over a year, I’ve been using Postgres as a development framework. In addition to the core Postgres server that stores all the Hypothesis user, group, and annotation data, there’s now also a separate Postgres server that provides an interpretive layer on top of the raw data. It synthesizes and caches product- and business-relevant views, using a combination of PL/pgSQL and PL/Python. Data and business logic share a common environment. Although I didn’t make the connection until I watched r0ml’s talk, this setup harkens back to the 1980s when Smalltalk (and Lisp, and APL) were programming environments with built-in persistence. The “image” in r0ml’s title refers to the Smalltalk image, i.e. the contents of the Smalltalk virtual machine. It may also connote reputation, in the sense that our image of Postgres isn’t that of a Smalltalk-like environment, though r0ml thinks it should be, and my experience so far leads me to agree.

I started writing a book to document what I’ve learned and done with this idea. It’s been a struggle to find motivation because, well, being the patron saint of trailing-edge technologies is often lonely and unrewarding. A book on this particular topic is likely to appeal to very few people. Stored procedures? So last century! Yes, Python provides a modern hook, but I can almost guarantee that one of the comments on my first book — “has a vision, but too weird” — would come back around.

I’m tempted not to bother. Maybe I should just focus on completing and polishing the things the book would describe.

And yet, it’s hard to let go. This isn’t just a compelling idea, it’s delivering excellent results. I rewatched r0ml’s talk today and got fired up again. Does it resonate for you? Would you like to see the ideas developed? If you watch the talk, please let me know.

Here are some excerpts to pique your interest.

On databases vs file systems:

I submit that the difference between the database and a file system is that database is a platform for enforcing your business rules.

On ontology:

client: The business guys are always upset because they want to know how many customers we have and we can’t tell them.

r0ml: That doesn’t sound like a very hard problem. SELECT * from the customer table, right?

client: No you don’t understand the problem.

r0ml: OK, what’s the problem?

client: It depends what you mean by customer because if you’re selling cell phone insurance, is the customer the person who has the cell phone account? What if they have two handsets and they’re both insured? What if it’s a family account and there are kids on the plan, do they count as customers? What if it’s a business account and you have 1000 people covered but only 700 using?

r0ml: How my customers you have, that’s a business rule.

So figuring out what your schema is, and figuring out how you organize the stuff and what do you do in the database, that’s all part of enforcing your business rules.

You have to decide what these things mean.

It’s an ontological problem.

You have to classify your knowledge and then enforce your business rules.

On n-tier architecture:

Let us think about the typical web application architecture. This architecture is called the three-tier architecture because it has four tiers. You have your browser, your web server, the thing that runs Python or PHP or JavaScript or Ruby or Java code, and then the database. And that’s always how you do it. And why do you do it that way? Well because that’s how everybody does it.

On Smalltalk and friends:

This is the BYTE magazine cover from August of 1981. In the 70s and the 80s, programming languages had this sort of unique perspective that’s completely lost to history. The way it worked: a programming environment was a virtual machine image, it was a complete copy of your entire virtual machine memory and that was called the image. And then you loaded that up and it had all your functions and your data in it, and then you ran that for a while until you were sort of done and then you saved it out. And this wasn’t just Smalltalk, Lisp worked that way, APL worked that way, it was kind of like Docker only it wasn’t a separate thing because everything worked that way and so you didn’t worry very much about persistence because it was implied. If you had a programming environment it saved everything that you were doing in the programming environment, you didn’t have to separate that part out. A programming environment was a place where you kept all your data and business logic forever.

So then Postgres is kind of like Smalltalk only different.

What’s the difference? Well we took the UI out of Smalltalk and put it in the browser. The rest of it is the same, so really Postgres is an application delivery platform, just like we had back in the 80s.

mkalus shared this story from Ian Welsh.

Good question.

Oxford University surprised and pleased advocates of overhauling the vaccine business in April by promising to donate the rights to its promising coronavirus vaccine to any drugmaker…

…A few weeks later, Oxford—urged on by the Bill & Melinda Gates Foundation—reversed course. It signed an exclusive vaccine deal with AstraZeneca that gave the pharmaceutical giant sole rights and no guarantee of low prices

Seems pretty straight forward.

Then there’s a second issue. The mRNA vaccines, which so far seem to be the most effective, are supply capped: they can’t manufacture them fast enough. Unrelated to Gates (so far as I know) the manufacturers of those vaccines, like Moderna, are refusing to share the manufacturing process and other information. Vastly more doses could be produced and many lives saved, but IP is sacred, not your grandmother’s life.

Moderna’s entire costs for the vaccine were covered by government, yet the manufacturing process is kept secret, and people die.

It would be simple enough to just pay Moderna and Pfizer their expected profits plus some, and break the patents: this is something governments can do. It was suggested and vetoed.

So, if someone you know dies because of no vaccine, remember that government and Bill Gates wanted that, because the holy grail of restricting information so that the rich could profit off it was more important than people living or dying.

This is always who Gates has been, by the way. He’s white-washed his reputation over the last few decades, but he’s always been a monopolist who is willing to have other suffer and his foundation is mostly about his own interests. The Gates Foundation interference in education was a vast disaster, as well.

All the content here is free, but subscriptions and donations do help, a lot.

mkalus shared this story from bellingcat.

• In our previous investigation, we disclosed the existence of a clandestine unit within the FSB’s Criminalistics Institute, members of which had shadowed opposition leader Alexey Navalny for nearly five years. Members of the unit with medical and chemical-weapons background, travelling in groups of two or three, had followed Navalny on more than 30 flights during his 2017 presidential election campaign. Three members of this squad had been near him during a suspected poisoning of his wife in July 2020, and during his near-fatal poisoning in August 2020. 
• We also disclosed that this unit was supervised by Col. Stanislav Makshakov, deputy director of the Criminalistics Institute and a scientist involved with Russia’s military chemical weapons program, which according to public sources had developed over 20 highly toxic materials including organophosphates of the Novichok type. Before Navalny’s poisoning with Novichok, Col. Makshakov was in frequent communication with scientists from the Signal Institute, which a previous joint investigation linked to Russia’s surreptitiously renewed chemical weapons program. 
• A follow-up report presented additional validation of the role of the FSB’s Criminalistic Institute in the poisoning of Alexey Navalny obtained through an inadvertent admission by one of the unit’s chemical weapons specialists – Konstantin Kudryavtsev – who had been dispatched to Omsk following the poisoning to dispose of any traces of the toxic substance on Alexey Navalny’s personal items.

Our initial report disclosed that the FSB poison squad traveled in clusters of two or three people to many more destinations that can be explained with their now-known efforts to murder Alexey Navalny. We continued exploring this unit’s extensive travel data (see this article on our methodology, as previously seen with our Navalny investigation) seeking to link their itineraries to previously unexplained deaths of political activists, as well as poisonings of prominent figures. To this end, we crowdsourced the research by making public the itineraries of 10 of the known members of the FSB squad.

We received more than 500 contributions from volunteers, journalists and other media, some of which pointed to previously unknown coincidences between squad members’ presence and unexplained deaths. We analysed all input and applied strict criteria in selecting potential leads for further investigations. For example, overlapping trips had to number no fewer than two with the presence of at least two squad members during each trip; or that a group of no less than three squad members needed to be present at a location where a suspicious unexplained death or severe poisoning of a public figure had taken place. These criteria were selected based on probabilistic assessment of the risk of false positives.

On the basis of these criteria, and further investigation, we were able to match the movements of the FSB’s poison squad with the suspicious deaths of three public figures between 2014 and 2019.

Two of the cases, which are presented below, conform to our self-imposed criteria, while another case does not have sufficient data points to fit the criteria but bears other striking similarities to one of these two cases. 

Based on the sheer volume of the squad’s travel data, this list is likely not exhaustive, and our investigation into other potential missions will continue. Furthermore, this list of victims uses travel data as a point of departure, and is therefore limited only to operations that took place outside of Moscow.

The Journalist From Nalchik: Timur Kuashev

Timur Kuashev, source: his Facebook

At about 6:30 pm on 31 July 2014, Timur Kuashev, a 26-year-old citizen journalist from Nalchik, the capital of Russia’s small North Caucasus republic of Kabardino-Balkaria, walked out of his home. He lived with his mother who was an actress at the local theater; she had a premiere at 7 pm that evening and he had requested two tickets. After he offered the second ticket to a friend who said she was busy that evening, Kuashev took a shower, put on dress trousers and a shirt, and walked the short way towards the Rodina theater. A witness later said they saw him talk to some people in plain-clothes at a bus stop who came out of a car.

Timur Kuashev’s lifeless body was found the next day on a road into a forest in the nearby suburb of Khasanya, more than 15 kilometres away from his home. He was lying face-down in the dirt, and his body and face had bruises and hematomas. The official coroner’s report said he died of heart failure triggered by an acute viral infection. The report said the forensic analysis “did not discover exterior symptoms of violence or signs of self-defense”.

However, this is clearly contradicted by a photograph of Kuashev’s face taken by a friend just before his funeral, and now shared with us. The photo shows clearly visible symmetrical bruises on his upper cheeks, possibly from fingers clutching his head from behind, and a hematoma on his left eyelid. Crucially, a follow-up coroner’s report identified traces of an injection in Kuashev’s left armpit area that had been applied six to 10 hours before the body was found.

Thanks to lobbying by his father – a retired senior criminal investigator – a murder investigation was opened into Kuashev’s death. Following a local laboratory analysis of blood samples that found no traces of narcotics or known toxins, the initial investigation hypothesis was that Kuashev had been poisoned with an unknown substance. Samples were sent for additional forensic analysis to Moscow, but no trace of poison was found. The final prosecution report from September 2016 closing the criminal investigation found no clear link between the injection and Kuashev’s death, and chalked it off to coronary failure. 

Timur Kuashev’s family did not consider the official investigation to be conclusive or valid. In comments to the news site Caucasian Knot, Khambi Kuashev pointed to the lack of any investigation into his son’s phone records from the date of his disappearance (there were two text messages from a service phone number that the investigators refused to pursue, due to – as the prosecution’s decision read – “risks of disturbance to the mobile operator’s systems”), as well as into the threats his son had reported in the months before. He also pointed to Timur Kuashev’s lack of any previously diagnosed medical issues. Additionally, the father, who spoke to our investigative team, says he had pointed to the fact that a USB drive was missing from his son’s computer, and that the keys to the apartment had not been on him when he was found – suggesting that someone might have entered his house after his disappearance.

Indeed, Timur Kuashev had alerted his friends – and even the authorities – of a string of anonymous threats he had received online as a result of his work as a journalist. As a blogger on LiveJournal – a popular blogging platform in Russia that had launched the careers of many researchers and journalists, including Alexey Navalny – Kuashev had relentlessly covered local politics in Kabardino-Balkaria, appearing to have irritated both local law enforcement and the FSB. Living in an autonomous republic which at times saw a spillover of violence from Chechnya, he had written articles on torture and persecution by security services for the popular regional news sites Caucasian Knot, Kavpolit and the magazine Dosh. He had also organized rallies against torture by police and was collaborating with national and local human rights organizations, and was an active member of the Russian opposition party Yabloko. According to his friends, relayed both in public statements and to our investigative team, Kuashev had been repeatedly summoned by both police and the FSB for questioning in connection with his work. Kuashev, an ethnic Circassian, had also been detained earlier that year at a rally marking the 150th anniversary of the last battle of the Caucasus War (a war that resulted in the incorporation of much of the North Caucasus into the Russian Empire and the forceful expulsion of hundreds of thousands of Circassians from their homeland). Two special agents from the police’s anti-extremism department had cautioned him about an article he had just written – which in turn caused Kuashev to file a complaint against the officers over their infringement of his right to free speech.

Kuashev also relentlessly covered a high-profile criminal case against 58 locals accused of involvement in the 2005 raid on Nalchik that was coordinated by Chechen rebel commanders, but which Russian security services claimed was instigated by foreign Islamist terrorist groups. The case was monitored by the Memorial human rights organization which criticized the extraction of dubious confessions via torture that were contradicted by other witness testimonies. The Russian journalist and political activist Maxim Shevchenko, editor of Kavpolit, told Bellingcat that Kuashev was the only local journalist who covered the court proceedings and repeatedly published damning critiques of the inconsistencies in the prosecution’s case. Shevchenko is convinced that Kuashev was murdered as a result of his interference with the Kremlin’s efforts to stage a smooth show trial. This theory is supported by other journalists such as Nadezhda Kevorkova, who, together with Shevchenko, conducted a journalistic investigation into Kuashev’s death and concluded that he was murdered. 

Timur Kuashev had also been openly critical of Russia’s military intervention in Ukraine, including the 2014 annexation of Crimea. He had been outspoken in his criticism of the Kremlin and of Russian president Vladimir Putin, stating that the country needed a “a new revolution for change of government”. Just a few hours before he was killed, he posted a comment on Facebook in which he wrote that Putin’s enemies were dying “as if on schedule”. 

Another source who cooperated with Kuashev and who requested not to be named due to safety concerns believed that during one of his many interrogations by the FSB, Kuashev had been coerced into cooperating with the security agency, but later reneged on the agreement. The source believes this to be the most likely reason for placing Timur Kuashev on the FSB’s blacklist, but we have been unable to confirm if this is true.

Facebook messenger exchange between Kuashev and a human rights activist from the North Caucasus republic of Dagestan from April 2014 in which Kuashev confirms he received threats. Screenshot provided to Bellingcat by the activist.

It is not clear precisely which of Timur Kuashev’s journalistic or activist endeavors may have contributed to him becoming a target of assassination. However, an analysis of travel data of FSB operatives from the poison squad of the Criminalistics Institute, as well as confidential documents from official prosecutorial investigation provided by an insider source, strongly suggests that his death was the result of a targeted poisoning operation by the same core FSB team that poisoned Alexey Navalny in 2020. 

Reconstruction of the FSB operation

Based on partial airline data for 2014, we can see that the same FSB squad previously identified by Bellingcat in the context of the Navalny poisoning investigation were conducting an ongoing operation in the Nalchik area at least as early as 13 July 2014. On that date Konstantin Kudryavtsev, one of the unit’s chemical weapons specialists, traveled to Nalchik. There is no data about his return date. 

Nine days later, on 22 July 2014, Ivan Osipov – a decorated FSB officer and one of the senior members of the FSB squad implicated in the poisoning of Alexey Navalny – flew from Moscow to Mineralnye Vody, north-west of Nalchik. He had originally bought a return ticket for 30 July, but on that day he changed it for the following morning, 31 July. However he didn’t take that flight either, and changed it one more time – to a flight on 1 August 2014 at 2:05 pm. That flight back to Moscow departed several hours after Timur Kuashev’s body was found outside Nalchik, an hour and a half’s drive away.

In Moscow’s citizen database, Osipov’s name is listed as a a person entitled to special benefits status. This marking is typically reserved for people with a high military distinction]

FSB agents: Medical expert Ivan Osipov (left) and field commander Alexey Alexandrov (right)

Osipov was not alone in or near Nalchik on the eve of and at the time of Kuashev’s death. In addition to Kudryavtsev, at least three other FSB operatives had arrived in the region in the days prior. A critical second presence was that of Dr. Alexey Alexandrov, the other member of the FSB poison squad whose phone records placed him near Alexey Navalny’s hotel in the hours before he fell into a coma in Tomsk. 

Unlike his colleague Osipov, Alexandrov flew to another nearby airport, Vladikavkaz, situated 100 km to the south-east of Nalchik. As disclosed in the inadvertent telephone admission by Konstantin Kudryavtsev, operatives of the FSB squad – even members of the same “brigade” – often travel on different flights to and from different airports to avoid detection. This modus operandi is corroborated by hundreds of flights of the FSB squad reviewed by Bellingcat. Alexandrov arrived one week after Osipov, on 29 July. Like Osipov, he initially had a return booking for 31 July but did not use it. Instead he bought a last-minute ticket from Vladikavkaz to Moscow on 2 August 2014, the day after Kuashev’s body was found. 

Also on 29 July, Denis Machikin and Roman Matyushin, both Moscow-based FSB officers likely working at the Department for the Protection of the Constitution, flew into Vladikavkaz. The two had purchased return tickets to Moscow on 30 July 2019. However, they didn’t use them – and changed the tickets to 31 July 2014, the day when Kuashev disappeared. The partial airline booking data we have reviewed does not show conclusively if they flew on that day or, like Ospiov, rebooked the once more for the following day.

We know that these two men are FSB officers because they travel on joint airline bookings with other known FSB officers, including Alexey Alexandrov. We believe they are most likely from the Department of Protection of the Constitution because members of the FSB poison squad work with and travel only with operatives of this unit. Furthermore, we have tracked parking and traffic fines of these people to near Vernadskogo 12 in Moscow, where this unit is headquartered.]

Denis Machikin, born 1985, a photograph from a speed camera. Open source traffic violation data shows Machikin has more than 60 unpaid fines in the last two years alone, totaling over Euro 2000. Russian security operatives routinely do not pay traffic violation fines.

A comparison with a cropped photo of Denis Machikin found on the VK social network page of his wife.

Following Kuashev’s disappearance, potentially crucial evidence in the case vanished, in a way that appears consistent with the modus operandi of the FSB squad. As Konstantin Kudryavtsev stated in the phone call with Navalny, the unit ensures that all security cameras are taken offline during their operations.

In Kuashev’s case, all security cameras from the square in front of the Rodina theater, where he had gone to watch his mother’s play, had been offline during the evening of his disappearance due to a technical error, or as described in the below document a “commutator malfunction”.

A letter from the traffic police department to investigators explaining that the security cameras on the Rodina square had been offline during the period 21:00 on 31 July 2014 until 10:00 on 1 August 2014 due to a technical malfunction

In response to your request, we write to inform you that video recordings from the video cameras located on Rodina Square from the period between 21:00 on 31.07.2014 to 10:00 on 01.08.2014 do not exist, due to a communication failure from the [network] switch. Archival footage from video cameras located at the entrance to the city of Nalchik in the direction of Khasanya village between 21:00 on 31.07.2014 and 10:00 on 01.08/2014 has been saved and developed. The storage period for recordings is seven days from the date of their request. The recording in question will be provided to your co-worker during the storage period, in DVD format.

Director, R. Karezhev

On 2 August, a day after Kuashev’s body was found, his friends and colleagues were able to obtain a full extract of call records from his mobile operator for the week beginning 25 July until 2 August. The call records showed zero activity – neither phone, text, nor internet – prior to 2 August 2014. The complete absence of communication through 1 August is contradicted by the activity for 2 August 2014 which shows eight incoming spam messages (a relatively regular occurrence on Russian mobile phones). The alternative hypothesis that this phone had been switched off, and only turned on again on 2 August 2014, is contradicted by phone data of other telephone numbers, including of a human rights activist from Dagestan who told our investigative team that they spoke and texted with Kuashev on that number in the days prior to his death. 

If a cleansing of Kuashev’s call activity had been undertaken by the mobile operator, this could not have reasonably happened without the intervention of the FSB who have overarching control over telecommunication operators. Russian mobile operators are obliged to preserve complete call records and provide them regularly to an FSB-supervised repository.

The Perfect Crime

Following the death of Timur Kuashev, tissue and blood samples were initially tested for toxins locally in Nalchik. However, the local hospitals’ laboratories do not have the capability to perform mass-spectrography, the best-practices method to identify toxins and narcotic compounds in blood and urine samples, or other complex analysis. Tissue samples were sent to Moscow for a follow-up analysis. 

The main expertise report conducted in Moscow confirmed the absence of any toxic substances in traces of Timur Kuashev’s blood, which had leaked on to his shirt around the injection area. This served as a key factor in closing the criminal investigation into his death. 

However, the analysis was performed by none other than the FSB’s Criminalistics Institute: the same unit that employs the clandestine poison squad, including Dr. Alexandrov, Osipov and Kudryavtsev. The expert report was signed by Vasiliy Kalashnikov: the same FSB expert who – as admitted by Kudryavtev in his tell-all phone call – was supervising the evidence-purging efforts in Omsk after the poisoning of Alexey Navalny. As we have previously determined, the poisoners of Alexey Navalny worked under the guise of this exact institute. In the case of Kuashev, this would have allowed the poisoners to investigate, and ultimately cover up their own assassination operation.

CENTRE FOR CRIMINAL TECHNOLOGY OF THE INSTITUTE OF CRIMINALISTICS

Expert testimony
Drafted 2 June 2015, city of Moscow

No 9/8/33

We, employees of the Institute of Criminalistics of the FSB’s Centre for Special Technologies, Vasily Anatolyevich Kalashnikov, Andrey Vyacheslavovich Bordulyak, Elena Borisovna Guseva and Ilya Viktorovich Etov, in connection with instructions from the director of the expertise department to carry out an expert analysis on materials relating to criminal case No. 71/94-14, have been forewarned of the criminal liability for knowingly providing false testimony under article 307 of the criminal code of the Russian Federation. In accordance with Article 199 of the Code of Criminal Procedure, the rights and responsibilities of those providing expert testimony, provided for under Article 57 of the Code of Criminal Procedure, have been explained to us.

A leading global expert on chemical weapons who requested to remain unnamed reviewed the full expertise (available here) and opined that:

“The analysis performed was rather basic. GC-MS and LC-MS with standard searches against standard libraries and no further specified in-house database“. Obviously, if the compound used was intentionally developed to avoid detection, the analysis would have missed such compound because it would not be in any of the “official” libraries”

The Lezgin Activist from Dagestan

On 24 March 2015, in the town of Kaspiysk – a coastal suburb of Makhachkala, the capital of Russia’s southernmost autonomous Republic of Dagestan – local residents found the body of Ruslan Magomedragimov, an activist of the local Sadval (Unity) movement. The Sadval movement had been created in 1990 and had as its ultimate mission the “creation of national statehood for the Lezgian nation”. Lezgins are an ethnic group of just under a million people, who have their own language and are dispersed between the Russian region of Dagestan and neighbouring Azerbaijan. 

Magomedragimov had phoned his friend at noon and agreed to meet him at a festival that celebrates the Lezgian language later that day. He did not make it to the meeting – his body was found just after 2 pm. His car was parked in a courtyard about 400 meters from where his body was found. According to Magomedragimov’s friend, Islam Klichkhanov, the dash cam was removed from Magomedragimov’s car, and two of the three phones he had on him were missing. Investigators who arrived at the crime scene found another person’s fresh shoe prints in the car.

The initial conclusion by the investigators was that Magomedragimov died of heart failure. However, a coroner who conducted a post-mortem on the victim reportedly challenged this conclusion, saying Magomedragimov had a perfectly healthy heart and that the death appeared to have been the result of asphyxiation. At the same time, there were no signs of violence on the victim’s neck or body. The preliminary investigation however ignored this input and concluded that Magomedragimov had died of heart failure. 

At the same time, Magomedragimov’s relatives claimed that after receiving the body for burial, they noticed two dots resembling the traces of a syringe needle on his neck. On the insistence of relatives, tissue samples were sent for analysis for the presence of toxins in his body. We could find no public reports of the results of this analysis and we have been unable to reach members of Magomedragimov’s family to obtain details. However, the leader of the Sadval movement publicly stated that he was convinced that Magomedragimov had been killed as a result of his activism. Regional media drew parallels between the deaths – including the traces of injections – of Magomedragimov and of Timur Kuashev a year earlier.

The similarities between Magomedragimov’s death and that of Kuashev also extended to the overlapping presence of members of the FSB poison squad. 

Like in the case of Kuashev, poison squad members Ivan Osipov and Konstantin Kudravtsyev traveled to Makhachkala in the period before Magomedragimov’s death: Osipov visited the town twice in January 2015, and Kudryavtsev traveled to Makhachkala two weeks before the death – just as happened in the 2014 Kuashev case – staying in the region from 11 to 16 March. 

Just four days before Magomedragimov’s death, on 24 March 2015, Ivan Osipov flew into Vladikavkaz – a four-hour drive from Kaspiysk. He stayed in the region for six days and left back to Moscow two days after the presumed poisoning.

While the datapoints and travel overlaps in Magomedriagimov’s case are fewer than those in Kuashev’s, they are not sufficient for a standalone conclusive assessment that he was killed by the FSB poison squad. Yet the strikingly similar circumstances surrounding the deaths of these two activists from the North Caucasus, including claims around the use of injections to induce death broadly mis-attributable to heart failure, are notable.

As a Sadval activist. Magomedragimov defended the rights of the Lezgin people and supported the idea of an autonomous “Lezgistan” based on unification of Russian and Azerbaijani Lezgins. Despite the pacifist nature and innocuous tactics of this organization, it is conceivable that it was seen as a separatist movement that carried long-term disruptive risks. Exactly a year after Magomedragimov’s death, the leader of the Sadval movement, Nazim Gadjiev, was found killed at his home in Makhachkala with knife wounds on his body. No suspects in the murder were named.

The Kremlin-Friendly Anti-Corruption Fighter

On 16 November 2019, Nikita Isaev posted a selfie on VK as he sat inside a train that was about to depart from Tambov, a city roughly 500 km south-east of Moscow. Isaev was leaving for home after another day of hard work: meeting with local residents disenchanted with local corrupt politicians and regional mini-oligarchs.

He had been doing this for just over а year as head of his self-styled “New Russia“ movement, and had always managed to tread the fine line between criticizing regional or second-tier Moscow politicians and never angering the federal government. While positioning his movement as independent and anti-establishment, Isaev continued to support the Kremlin’s main policies and President Putin personally, and to attack Russia’s foes, foreign and domestic, including Alexey Navalny. The latter was ironic given that some domestic observers called Isaev “the new Navalny”, alluding to the fact that he appeared to copy Navalny’s regional corruption-bashing style, and even bore a visible resemblance to him. Unlike Navalny, Isaev had unimpeded access to national TV channels and was a frequent commentator on topics ranging from consumer protection, US politics and the war in Ukraine. Many believed that Isaev had been a “spoiler” political project developed by his former mentor Vladislav Surkov, former aide to Putin and the architect of Russia’s political system of “sovereign democracy”. Isaev’s role, it was supposed, was to splinter and weaken Alexey Navalny’s political following, while attacking Surkov’s political enemies inside the Kremlin. He had even been made advisor on regional development for A Just Russia – one of the “systemic” opposition parties in the Russian Duma that routinely supports the government’s policies.

This made his sudden death on 16 Novemeber during the nine-hour train ride to Moscow even more puzzling. In the description of his former aide and love interest Alina Zestovskaya, Nikita Isaev woke up just after 1 am and went to the toilet. He was gone for a long time, and when he came back, he was slouching and holding his abdomen, as if someone had punched him. The last words he could utter before he collapsed to the floor in convulsions was “I got poisoned, I think”.

Zhestovskaya called the train conductor, but he said there was no point in stopping the train – it was speeding through endless forests, and the next possible stop didn’t even have a paramedic’s facility. By the time the train got to the next station – more than an hour later – Isaev had lost his pulse and his fingers and lips had turned blue. The doctors who came on board at the train station declared him dead on site.

Upon arrival in Moscow, blood samples from Nikita Isaev were sent for laboratory tests. Before the results came in, he was cremated – as he had wished, his family told our investigative team. The laboratory analysis showed no traces of toxins in his blood – and his death was declared the result of a heart attack. Few of his followers and sympathizers believed this explanation, as evidenced by the hundreds of incredulous phone calls to a radio show with Alina Zhestovskaya – after all, he was a healthy, fit 41-year-old, and he himself had said he believed he had been poisoned. Despite all that, Zhestovskaya told our investigative team she believes he died of a heart attack, and believes people who claim he – or Alexey Navalny – were poisoned are simply spreading conspiracies.

Nevertheless, our interviews with family members and friends of Isaev suggest that in the months before his death he was restless. A friend of his living abroad, who requested anonymity, told us that in June 2019, Isaev discussed with him possibilities to move his family out of Russia. Isaev did not inform him of any specific threats, but, in his words, during their June meeting “was not his normal self”. Two people close to Isaev, who also requested anonymity, told us that shortly before his death he began suspecting that Alina Zhestovskaya had been “assigned to watch over him”, and that he was intending to get rid of her. Zhestovskaya told us she initially met Isaev accidentally on a train and worked for him on a voluntary basis due to shared beliefs in fighting corruption. She was playing an increasing role in his movement, and after his death briefly took over his role in the organization.

Reconstruction of the FSB Operation

As a result of crowdsourced analysis of the travels of 10 of the known operatives from the FSB poison squad, contributors noted the overlap of two 2019 trips by the FSB’s Dr. Alexey Alexandrov and Nikita Isaev – whose campaign trips were diligently detailed on his VK account. These overlapping trips were to the remote city of Magnitogorsk near the border with Kazakhstan, in May 2019, and to Nizhny Novgorod in October 2019, a month before Isaev’s death.

These two overlaps were statistically significant but not sufficiently convincing for a hypothesis of FSB’s involvement in his death. However, these two trips provided us with a lead to identify other co-passengers of Alexandrov based on an analysis of flight and train passenger manifests. Those new three names in turn provided us with a larger set of trips which we could then compare once more to Isaev’s travel data. We also obtained additional data for Isaev’s travels from several offline (incomplete) ticketing databases, which broadened his own travel dataset beyond those trips known about from his VK account. Based on this, we again tried matching his trips and the broader set of known FSB operatives.

The result was a picture of at least five FSB operatives linked to the Criminalistic Institute’s poison squad, tailing Nikita Isaev for nearly a year before he died.

The table above shows that FSB operatives tailed Isaev to and from seven destinations around Russia, on a total of 13 flight or railway journeys. The actual number of overlapping trips is likely higher due to the fact that Isaev’s travel data is incomplete and, on at least some of the trips, the FSB operatives may have traveled on as yet unknown undercover identities. Furthermore, we have not yet identified all members of this FSB unit. However, even with only these seven (with thirteen segments to the trips) overlaps to destinations strewn across a gigantic territory – and in at least one case, tailing him between two destinations in the Russian far east – the statistical probability of a false positive is close to zero.

Map showing the wide range of locations where the FSB team trailed Isaev, from the far southwest of Russia in Sochi to the Pacific Ocean in Vladivostok. Source: Google Earth

The additional three FSB operatives identified through this operation – Alexander Samofal, Victor Kravchenko and Mikhail Tikhonov – flew during this period or earlier on jointly made bookings with either Alexey Alexandrov or other known members of the poison squad. Thus it can be excluded that they are random, accidental co-travelers. We have not yet determined if the three are operatives of the Institute of Criminalistics or of FSB’s Department for Protection of the Constitution, which, as discussed previously, tend to work in tandem with the poison squad.

Extract from a flight booking showing Alexandr Samofal flew on a joint booking with Konstantin Kudryavtsev and another operative

2013 photo of Kravchenko from a family member’s social media post.

Screenshot from reverse-phone-search app showing Viktor Kravchenko’s number listed with the heading “FSBb”. Kravchenko also traveled on joint bookings with Alexandrov.

It is important to note that the existing travel databases we have consulted do not show that any of these five operatives traveled under their known identities to Tambov during the days Isaev died. This could potentially be explained either by data purging after his death, or by the fact the officers traveled under different identities yet unknown to us. There is no conflicting travel data for these identities – i.e. existing travel data does not suggest they were in other places at the time of the Tambov trip. 

Possible Motives

None of the people close to Isaev have a plausible explanation of why a pro-Kremlin, patriotic-minded activist may have crossed the FSB. His conflicts with regional government officials – part of his activism – were deemed as unlikely motives: Isaev’s friends and family are convinced that if the Kremlin authorities were unhappy with any part of his activity, a simple hint would have sufficed for him to change course.

One source familiar with Isaev’s work who requested anonymity suggested that he may have been targeted by FSB due to the fact that he was a Surkov protégé. Vladislav Surkov, who fell in and out of grace with the Kremlin over the past decade and resigned from his position as advisor to President Putin in early 2020, is known to have had a fraught relationship with the FSB. However, to this source it appeared unlikely that the FSB would resort to murder in the course of its ongoing confrontation with Surkov. Thus far, known motives for assassinations through poisoning have been limited to charges of terrorism, treason, or serious threats to the regime such as in the case with Alexey Navalny.

A possible explanation might lie in Isaev’s frequent trips overseas that accelerated in the year prior to his death. Isaev traveled abroad frequently, with multiple trips to Germany, Spain, Italy, the Netherlands, Israel, Turkey, Latvia and the United States in 2019 alone. In fact, Nikita Isaev had previously purchased tickets to fly with his family to Miami on 5 December 2019, just three weeks after his demise. It is conceivable that the FSB may have suspected him of planning to defect; a suspicion that may have been fueled by his discussion with contacts about moving his family out of Russia.

Conclusion

The three cases described in this report presents further evidence that the clandestine FSB unit that tailed and, later self-admittedly, poisoned Alexey Navalny in August 2020 is in fact an effective state assassination program, and not simply a political intimidation tool.

The findings also shed light on the broad arsenal and application of techniques that this unit employs, as well as on its apparently selective approach to using seemingly simpler and fast-acting poisons on less public targets (such as an injected poison with Kuashev) versus use of more sophisticated toxic substances with deferred onset of symptoms on likely higher-value targets (such as Novichok with Navalny) who are subject to greater public scrutiny.

These case studies along with the previously known Navalny assassination attempt also provide a sample of the breadth of possible motives that determine a person’s placement on the FSB’s presumed kill list. It appears that in addition to the presumed involvement of this unit in the poisoning of perceived threats (such as the insurgent leader Doku Umarov, along with political figures who pose perceived threats to the central regime, such as Alexey Navalny) this FSB unit also appears to preemptively target opinion leaders and human rights activists in volatile regions.

While the case with Isaev requires more information before a motive can be identified, it does open the possibility for either the weaponization of the FSB’s poison squad for the resolution of conflicts inside Russia’s political elite, or – as is already known from the poisoning of Sergey Skripal – the use of security services to assassinate perceived traitors.

The post Navalny Poison Squad Implicated in Murders of Three Russian Activists appeared first on bellingcat.

Very excited to share the news that Revue is joining Twitter. I’m a huge fan of the idea of better newsletters and Automattic was the largest investor in Revue. I’m looking forward to seeing what the very talented team will do as part of the Twitter network. Also many thanks to Kevin Kelly and Om for introducing me to Revue early on.

13,000 regular expressions seems like a great solution to this particular problem - how the Guardian built an automated style guide checker

Apple’s HomePod mini has finally received the long-awaited ultra-wideband (UWB) handoff feature with the launch of iOS 14.4.

The feature essentially allows an iPhone 12 or iPhone 11 to transfer calls and music between the two devices seamlessly.

Although the tech giant’s previous speakers have supported the ability to hand off music, the new HomePod mini features a U1 ultra-wideband chip that makes it easier to transfer content. It also adds additional features such as new visual and haptic feedback.

Further, the device will also give users personalized listening suggestions and playback controls on their iPhone when they bring it near the speaker.

Apple showcased this feature when it first announced the HomePod Mini and said the functionality would be available sometime in late 2020. Although the tech giant slightly missed its deadline, it’s nice that the feature is here.

To use the new functionality, you need to ensure that your HomePod mini has been updated to the latest software and that your iPhone 11 or iPhone 12 is running iOS 14.4. It’s worth noting the feature isn’t available on the original HomePod.

If you want to learn more about the device you can check out MobileSyrup’s review here.

Via: The Verge 

The post Apple’s HomePod mini finally gets its ultra-wideband handoff feature appeared first on MobileSyrup.

The first thing that attracted me to the Sennheiser CX 400BT was the price.

The wireless earbuds retail for $180 in Canada, but right now, you can even find them on sale for as low as $129, a price that offers incredible value.

Sennheiser is known for its high-quality audio products like the recently announced IE 300 or the Momentum 2 Wireless earbuds, which both cost around $400 in Canada. This is good since the company can apply its high degree of sound knowledge to lower-cost earbuds. In practice, that shift in strategy has paid off.

Sennheiser’s support page even says that the CX 400BTs the same audio tech as the Momentum 2, giving these earbuds a soundstage well above their weight class.

How do they sound?

When it comes to sound quality, the 7mm speaker drivers in these earbuds hold their own. The soundstage is decently wide, offering a sense of balance to music that sounds great.

That said, the earbuds’ sound depth is slightly limited, with not as much separation between the lows and highs, leading to a sometimes compressed sound. Having said that, vocals are rich and songs with straightforward melodies come across extremely well.

The overall sound quality isn’t as impressive as some high-end headphones, but it’s still superb for the price, making the CX 400BTs a great deal for people looking for a good set of budget-friendly basic earbuds.

Other notable specs include Bluetooth 5.1 and support for popular high-end codecs like AAC and aptX, allowing for a premium listening experience on either iOS or Android devices.

What the connected app offers

As with most wireless headphones, except Apple’s AirPods, you can adjust the EQ within their app to tweak your listening experience. You can even save these EQ presets, so switching back to them later is easy.

You can also use the app to customize the earbuds’ controls. This is nice since I don’t often trigger my smart assistant on the earbuds, so being able to remap that to another play/pause button is convenient.

There may not be a lot in the app, but it thankfully includes a lot of the features people will likely use the most.

Low-cost, but loud and proud

When it comes down to it, I think that these earbuds are a real bargain. While they may not offer high-end features like active noise-cancelling (ANC) or a water resistance rating, they feature excellent sound quality and battery life.

On average, the buds last over seven hours without needing to be charged. With the charging case, Sennheiser says that users should get 30 hours of listening. You can charge them completely in about an hour and a half or get an extra hour of listening with a ten-minute charge.

While this isn’t industry-leading battery life, it’s more than enough time for me, and as long as I remember to plug the CX 400BTs in a couple of times a week, I have no issues.

I also found these earbuds fit comfortably in my ears with a little twist that locks them in place. I would even feel comfortable working out while wearing the X 400BTs if I was doing something where I didn’t want to hear the outside world. Concerning the MobileSyrup touque test, these buds remained snug in my ears, leaving them with a 5/5 Touque score.

When it comes to sound quality, the CX 400BT are a pretty stellar pair of under-the-radar earbuds.

You can pick up a pair at Best Buy until February 4th for $129 CAD, but regularly they’re sold for $179.

The post Sennheiser CX 400BT: Setting a bassline appeared first on MobileSyrup.

Electronic Arts has formed a new studio in Vancouver called Full Circle that will develop the next game in the Skate franchise.

Daniel McCulloch, who formerly led Microsoft’s Xbox Live division, has been appointed as Full Circle’s general manager. Additionally, Full Circle has brought back former Skate creative leads Deran Chung and Cuz Parry to spearhead the skateboarding franchise’s latest installment.

To help develop the currently untitled Skate game (informally referred to as “Skate 4“), EA is now hiring for several positions, including senior character artist, AI engineer and gameplay mechanics designer.

Full Circle is EA’s third studio in Vancouver, after EA Vancouver (FIFA and NHL) and a secondary Respawn Entertainment studio (Apex Legends). EA also has studios in Montreal (Star Wars Squadrons maker Motive) and Edmonton (Mass Effect and Dragon Age creator BioWare).

Skate 3, EA’s last Skate game, released in 2010. After years of fans calling for Skate 4, EA finally confirmed last summer that a new Skate is in the works.

EA hasn’t provided a release window for Full Circle’s Skate game.

Image credit: EA

Source: EA Via: VentureBeat

The post EA opens new studio in Vancouver to develop ‘Skate 4’ appeared first on MobileSyrup.

In the tech giant’s fiscal Q1 2021 earnings report, Apple revealed that it crossed the $100 billion USD (about $128 billion CAD) in quarterly revenue mark for the first time in the company’s history by bringing in $111.4 billion USD (roughly $142 billion CAD) in total, and $1.68 USD (about $2.15 CAD) per share.

This number also surpasses the company’s expected revenue, which was predicted to hit $103.12 billion USD ($132 billion CAD).

It’s important to note that quarterly earnings report includes iPhone 12, iPhone 12 mini, iPhone 12 Pro and iPhone 12 Pro Max sales, which amounted to $65 billion USD (approximately $83 billion CAD) in revenue, another all-time record for the company. The previous highest iPhone revenue quarter was $61.58 billion USD (roughly $78 billion CAD) in the first quarter of 2018.

It will be interesting to see if the predicted “super cycle” of upgrading from older iPhones like the iPhone X series to the iPhone 12, which features 5G and the most substantial redesign to the look of the smartphone in years, continues later into 2021.

“This quarter for Apple wouldn’t have been possible without the tireless and innovative work of every Apple team member worldwide,” said Tim Cook, Apple’s CEO, in a recent press release.

“We’re gratified by the enthusiastic customer response to the unmatched line of cutting-edge products that we delivered across a historic holiday season. We are also focused on how we can help the communities we’re a part of build back strongly and equitably, through efforts like our Racial Equity and Justice Initiative as well as our multi-year commitment to invest $350 billion throughout the United States.”

Apple’s Mac sales also surged this quarter to $8.68 billion USD (roughly $11.1 billion CAD), likely as a result of more people working from home amid the ongoing COVID-19 pandemic. In Q1 2021 Apple revealed a new MacBook Air, MacBook Pro and Mac mini powered by its own M1 processor. The company is expected to release a new version of its desktop iMac that features the M1, likely bolstering Mac sales even further.

On the services side of things, an area where Apple continues to shift into growth, the company earned $15.76 billion (about $20 billion) this past quarter. Services include platforms like Apple Music, Apple Arcade, Apple TV Plus, Apple News+ and most recently Apple Fitness+, a fitness subscription service.

Finally, sales from Apple’s wearables, home and accessories category, which includes the companies popular AirPods and AirPods Pro, hit $12.97 billion USD ($roughly $16.6 billion CAD) this quarter.

The post Apple earns over $100 billion in quarterly revenue for the first time ever appeared first on MobileSyrup.

This is the second in a series of posts about my new Library Map. You probably should read the first post if you're interested in why I made the map and why it maps the particular things that it does. I expected this to be a two part series but it looks like I might make a third post about automation. The first post was about why I made the map. This one is about how.

The tech stack

The map is built with a stack of (roughly in order):

• original Shape (SHP) and GeoJSON files
• QGIS
• geojson
• a bunch of csv files
• a tiny python script
• topojson
• some HTML, CSS and JavaScript
• leafletjs and leaflet plugins
• Map Box tile service

Boundary files

Since I primarily wanted to map things about library services rather than individual library buildings, the first thing I looked for was geodata boundary files. In Australia public libraries are usually run by local government, so the best place to start was with local government boundaries.

This is reasonably straightforward to get - either directly from data.gov.au or one of the state equivalents, or more typically by starting there and eventually getting to the website of the state department that deals with geodata. Usually the relevant file is provided as Shapefile, which is not exactly what we need, but is a vector format, which is a good start. I gradually added each state and data about it before moving on to the next one, but the process would basically have been the same even if I'd had all of the relevant files at the same time. There are two slight oddities at this point that may (or may not 😂) be of interest.

Australian geography interlude

The first is that more or less alone of all jurisdictions, Queensland provides local government (LGA) boundaries for coastal municipalities with large blocks covering the coastal waters and any islands. Other states draw boundaries around outlying islands and include the island — as an island — with the LGA that it is part of (if it's not "unincorporated", which is often the case in Victoria for example). As a result, the national map looks a bit odd when you get to Queensland, because the overlay bulges out slightly away from the coast. I'm not sure whether this is something to do with the LGA jurisdictions in Queensland, perhaps due to the Great Barrier Reef, or whether their cartography team just couldn't be bothered drawing lines around every little island.

Secondly, when I got to Western Australia I discovered two things:

1. The Cocos (Keeling) Islands are an Overseas Territory of Australia; and
2. Cocos and Christmas Islands have some kind of jurisdictional relationship with Western Australia, and are included in the Western Australia LGA files.

I hadn't really considered including overseas territories, but since they were right there in the file, I figured I may as well. Later this led to a question about why Norfolk Island was missing, so I hunted around and found a Shapefile for overseas territories, which also included Cocos and Christmas Islands.

Shapefiles are a pretty standard format, but I wanted to use leafletjs, and for that we need the data to be in JSON format. I also needed to both stitch together all the different state LGA files, and merge boundaries where local councils have formed regional library services. This seems to be more common in Victoria (which has Regional Library Corporations) than other states, but it was required in Victoria, New South Wales, and Western Australia. Lastly, it turns out there are significant parts of Australia that are not actually covered by any local government at all. Some of these areas are the confusingly named national parks that are actually governed directly by States. Others are simply 'unincorporated' — the two largest areas being the Unincorporated Far West Region of New South Wales (slightly larger than Hungary), and the Pastoral Unincorporated Area that consists of almost 60% of the landmass of South Australia (slightly smaller than France).

I had no idea these two enormous areas of Australia had this special status. There's also a pretty large section of the south of the Northern Territory that contains no libraries at all, and hence has no library service. If you're wondering why there is a large section of inland Australia with no overlays on the Library Map, now you know.

QGIS and GeoJSON

So, anyway, I had to munge all these files — mostly Shape but also GeoJSON — and turn them into a single GeoJSON file. I've subsequently discovered mapshaper which I might have used for this, but I didn't know about it at the time, so I used QGIS. I find the number of possibilities presented by QGIS quite overwhelming, but there's no doubt it's a powerful tool for manipulating GIS data. I added each Shapefile as a layer, merged local government areas that needed to be merged, either deleted or dissolved (into the surrounding area) the unincorporated areas, and then merged the layers. Finally, I exported the new merged layer as GeoJSON, which is exactly what it sounds like: ordinary JSON, for geodata.

CSV data

At this point I had boundaries, but not other data. I mean, this is not actually true, because I needed information about library services in order to know which LGAs collectively operate a single library service, but in terms of the files, all I had was a polygon and a name for each area. I also had a bunch of location data for the actual library branches in a variety of formats originally, but ultimately in comma separate values (CSV) format. I also had a CSV file for information about each library service. The question at this point was how to associate the information I was mapping with each area. There was no way I was going to manually update 400+ rows in QGIS. Luckily, CSV and JSON are two of the most common open file formats, and they're basically just text.

Python script

I'd had a similar problem in a previous, abandoned mapping project, and had a pretty scrappy Python script lying around. With a bit more Python experience behind me, I was able to make it more flexible and simpler. If we match on the name of the library service, it's fairly straightforward to add properties to each GeoJSON feature (the features being each library service boundaries area, and the properties being metadata about that feature). This is so because the value of properties within each feature is itself simply a JSON object:

{"type": "FeatureCollection",
"name": "library_services",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:EPSG::3857" } },
"features": 
[{ "type": "Feature", "properties" : {"name": "Bulloo Shire"}
 "geometry": { "type": "MultiPolygon", "coordinates": [ [ [ [143.78691062,-28.99912088],[143.78483624,-28.99912073] ... ]]}

The python script uses Python's inbuilt json and csv modules to read both the geojson and the csv file, then basically merge the data. I won't re-publish the whole thing, but the guts of it is:

# for each geojson feature, if a field in the json matches a field in the csv, add new properties to the json
for feature in json_data['features']:
  with open(csv_file, newline='') as f:
    # use DictReader so we can use the header names
    reader = csv.DictReader(f)
    for row in reader:
      # look for match
      if row[csv_match] == feature['properties'][geojson_match]:
        # create new properties in geojson
        for k in row:
          feature['properties'][k] = row[k]

The whole thing is fewer than 40 lines long. This saved me heaps of time, but as you'll discover in my future post on automation, I later worked out how to automate the whole process every time the CSV file is updated!

TopoJSON

GeoJSON is pretty cool — it's specifically designed for web applications to read and write GIS files in a native web format. Unfortunately, GeoJSON can also get very big, especially with a project like mine where there are lots of boundaries over a large area. The final file was about 130MB — far too big for anyone to reasonably wait for it to load in their browser (and Chrome just refused to load it altogether). Because of the way I originally wrote the Python script, it actually became nearly three times the size, because I put in a two-space indent out of habit. This created literally hundreds of megabytes of empty spaces. "Pretty printing" JSON is helpful if a human needs to read it, but rather unhelpful if you want to keep the file size down.

Enter TopoJSON. To be honest I don't really understand the mathematics behind it, but TopoJSON allows you to represent the same information as GeoJSON but in a much, much smaller file. I reduced a 362MB GeoJSON file (admittedly, about 200MB being blank spaces) to 2.6MB simply by converting it to TopoJSON! By "quantising" it (essentially, making it less accurate), the file size can be reduced even further, rendering the current file of about 2.2MB - definitely small enough to load in a browser without too much of a wait, albeit not lightning fast.

Good old HTML/CSS/JavaScript

At this point we're ready to start putting together the website to display the map. For this I used plain, vanilla HTML, CSS, and JavaScript. The web is awash with projects, frameworks and blog posts explaining how to use them to create your SPA (Single Page App)™️, but we really don't need any of that. The leaflet docs have a pretty good example of a minimal project, and my map is really not much more complex than that.

Something that did stump me for a while was how to bring the TopoJSON and CSV files into the JavaScript file as variables. I'm a self-taught JavaScript coder, and I learned it back to front: initially as a backend scripting language (i.e. nodejs) and then as the front-end browser scripting language it was originally made to be. So sometimes something a front-end developer would consider pretty basic: "How do I import a text file into my JavaScript and assign it to a variable?" takes me a while to work out. Initially I just opened the files in a text editor and copy-pasted the contents between two quote marks, made it the value of a javascript variable, and saved the whole thing as a .js file. But it was obvious even to me that couldn't possibly be the correct way to do it, even though it worked. In nodejs I would use fs.readFile() but the only thing that looked vaguely similar for front end JavaScript was FileReader — which is for reading files on a client, not a server. Finally I did a bit of research and found that the answer is to forget that the file is sitting right there in the same directory as all your JavaScript and HTML files, and just use AJAX like it's a remote file. The modern way to do this is with fetch, so instead of doing this:

// index.html
<script src="./boundaries.js" type="text/javascript"></script>
<script src="./branchesCsv.js" type="text/javascript"></script>
<script src="./ikcCsv.js" type="text/javascript"></script>
<script src="./mechanics.js" type="text/javascript"></script>
<script src="./nslaBranches.js" type="text/javascript"></script>
<script src="./load-map.js" type="text/javascript"></script>

// boundaries.js
const boundaries = `{"contents": "gigantic JSON string"}`
// branchesCsv.js
const branchesCsv = `lat,lng,town,address,phone
-35.5574374,138.6107874,Victor Harbor Public Library Service, 1 Bay Road, 08 8551 0730
... etc`
// ikcCsv.js
const ikcCsv = `lat,lng,town,address,phone
-10.159918,142.166344,Badu Island Indigenous Knowledge Centre,Nona Street ,07 4083 2100
...etc`
// mechanics.js
const mechanics = `lat,lng,town,address,phone
-37.562362,143.858541,Ballaarat Mechanics Institute,117 Sturt Street,03 5331 3042
..etc`
// nslaBranches.js
const nslaBranches = `lat,lng,town,address,phone
-37.809815,144.96513,State Library of Victoria,"328 Swanston Street, Melbourne",03 8664 7000
... etc`

// load-map.js
	// boundaries and the other constants are now globals
	const loanPeriod = new L.TopoJSON(boundaries, options)

...we do this:

// index.html
<script src="./load-map.js" type="text/javascript"></script>

// load-map.js
const boundaries = fetch('data/boundaries.topo.json')
.then( response => response.json())

const branchesCsv = fetch('data/public_library_locations.csv')
.then( response => response.text());

const ikcCsv = fetch('data/indigenous_knowledge_centre_locations.csv')
.then( response => response.text());

const mechanics = fetch('data/mechanics_institute_locations.csv')
.then( response => response.text());

const nslaBranches = fetch('data/nsla_library_locations.csv')
.then( response => response.text());

// fetch returns a promise so we have to let them all 'settle' before we can use the returned value
Promise.all([boundaries, branchesCsv, ikcCsv, mechanics, nslaBranches])
.then( data => {
	// data is an array with the settled values of the fetch() promises
	const loanPeriod = new L.TopoJSON(data[0], options)
}

In the code this doesn't necessarily look much simpler, but in terms of workflow it's a huge improvement that cuts out manually copy-pasting every time a CSV or TopoJSON file is updated, and reduces duplication and the total number of files.

So now the site consists of:

• the original data as CSV and TopoJSON files
• an index.html file to display the map
• a single CSS file for basic styling
• a single JavaScript file to load the map

Leaflet and friends

Finally it's time to actually put all of this stuff into a map using Leaflet. This is a really great JavaScript library, with pretty good documentation. Leaflet allows us to plot shapes onto a map, and using JavaScript to make them interactive - including adding popups, zoom to features when they're clicked, and add interactive overlays.

I won't try to replicate the Leaflet docs here and explain the exact steps to making my map, but I do want to highlight how two Leaflet plugins really helped with making the map work nicely. Leaflet has a fairly strong plugin collection, and they allow the base library to be fairly lightweight whilst the entire system is still quite flexible and fully featured.

I knew from the beginning it would require the whole library community to keep the map up to date over time. There are hundreds of library services across Australia, and they don't set their rules or their procurement decisions in stone forever. So it needed to be relatively simple to update the data as it changes. As we've discussed, GeoJSON also takes up a lot of space. Ideally, I could store as much data in CSV files as possible, and use them directly as data feeding the map. Turns out there's a plugin for that - Leaflet.geoCSV. This allows us to load CSV files directly (for library building locations), and it's converted to GeoJSON on the fly. Since CSV files are much smaller than the equivalent data in JSON, this is not only easier to maintain, but also loads faster.

The second plugin that really helped was Leaflet.pattern. The problem this helped me to solve was how to show both the fines layer and the loan period layer at the same time. Typically for a chloropleth map, different colours or shades indicate certain values. But if you add a second overlay on top of the first one, the colours no longer necessarily make much sense and combinations can be difficult or impossible to discern. Thinking about this, I figured if I could make one layer semi-transparent colours, and the second layer patterns like differently angled stripes or dots, that might do the trick. Leaflet.pattern to the rescue! After some alpha-testing by my go-to volunteer Quality Assurance tester, I worked out how to make the layers always appear in the same order, regardless of which order they were added or removed, making the combination always look consistent:

Tile service

Once all of that's complete, we can load the map. But there's a problem: all we have is a bunch of vector points and lines, there's no underlying geography. For this we need a Map Tile service. We can use one of several options provided by OpenStreetMap, but I ended up using the commercial Map Box service on a free plan (or at least, it will be free as long as thousands of people don't suddenly start using the map all at the same time). Their dark and light map styles really suited what I was trying to do, with minimal detail in terms of the underlying geography, but with roads and towns marked at the appropriate zoom level.

So that's it! It took a while to work it all out, but most of the complexity is in getting the data together rather than displaying the map. Once I had that done (though there is still a fair bit of information missing), I was able to pay more attention to maintaining the map into the future. That led me to look into some options for automating the merging of data from the library services CSV file (when it's updated) into the TopoJSON file, and also automatically refreshing the data on the actual map when the GitHub repository is updated. In my next post I'll explain how that works. While you're waiting for that, you can help me find missing data and make the map more accurate 😀.

Yesterday I had the great privilege of attending the GO GLAM miniconf, held under the auspices of the Linux Australia conference. Hosted by the fabulous Bonnie Wildie and the indefatigable Hugh Rundle, GO GLAM brought the power and promise of open-source software to the GLAM sector. This miniconf has been run a couple of times before but this was my first visit. It was pretty damn good. I’m glad I started scribbling down notes, otherwise it would all be this massive blend of exhausted awesome.

The day began with an opening keynote by some guy called Cory Doctorow, but he wasn’t very interesting so I didn’t pay much attention. He did talk a lot about self-determination, and he did use the phrase ‘seizing the means of computation’ that I definitely want on a t-shirt, but there was a big ethics-of-care-sized gap at the centre of his keynote. I found myself wishing someone would use the words ‘self-determination’ and ‘social responsibility’ in the same talk.

Good tech platforms can exist, if we care enough to build them. As it happened, GO GLAM’s first speakers, a group of five mostly francophone and mostly Indigenous artists and coders from what is now eastern Canada, wound up doing almost exactly this. Natakanu, meaning ‘visit each other’ in the Innu language, is an ‘Indigenous-led, open source, peer to peer software project’, enabling First Nations communities to share art, data, files and stories without state surveillance, invasive tech platforms or an internet connection. I can’t express how brilliant this project is. I’m still so deeply awed and impressed by what this team have built.

Two things leapt out at me during this electrifying talk—that Natakanu is thoughtful, and that it is valuable. It consciously reflects First Nations knowledge cultures, echoing traditions of oral history, and exemplifying an ‘approach of de-colonized cyberspace’. Files are shared with ‘circles’, where everyone in a circle is assumed to be a trusted party, but each member of that circle can choose (or not) to share something further. Building a collective memory is made easier with Natakanu, but the responsibility of doing so continues to rest with those who use it.

Natakanu embodies—and makes space for—First Nations sovereignties, values and ethics of care. It’s technology by people, for people. It’s a precious thing, because our communities are precious, too. The Natakanu platform reflects what these communities care about. Western tech platforms care about other things, like shouting at the tops of your lungs to ten billion other people in an agora and algorithmically distorting individuals’ sense of reality. We implicitly accept these values by continuing to use these platforms. Our tech doesn’t care about us. We could build better tech, if we knew how, and we chose to. (There’s a reason I’ve been consciously trying to spend less time on Twitter and more time on Mastodon.) But more on computational literacy a little later.

A few people mentioned in the Q&A afterwards how they’d love to bring Natakanu to Indigenous Australian communities. I don’t doubt their intentions are good (and Hugh touched on this in the recap at the end of the day), but in my (white-ass) view the better thing is to empower communities here to build their own things that work for them. A key aspect of Reconciliation in this country is developing a sense of cultural humility, to recognise when your whitefella expertise might be valuable and to offer it, when to quietly get out of the way, and which decisions are actually yours to make. Or, as speaker Mauve Signweaver put it, ‘instead of telling them “tell us what you need and we’ll make it for you”, saying “Tell us what you need and we’ll help you make it”‘.

I can’t wait to rewatch this talk and catch up on some parts I know I missed. It was absolutely the highlight of the entire miniconf. I couldn’t believe they were first-time speakers! Can they do the keynote next year?

Metadata and systems might not last forever, but we can still try. I think it’s safe to say many attendees were very taken with Arkisto, the ‘open-source, standards-based framework for digital preservation’ presented by Mike Lynch. It’s a philosophical yet pragmatic solution to describing, packaging and contextualising research data. Arkisto’s framework appears particularly useful for rescuing and re-housing data from abandoned or obsolete platforms (such as an Omeka instance where the grant money has run out and the site is at risk of deletion).

Arkisto describes objects with RO-Crate (Research Object Crate, a derivative of Schema.org) and stores them in the Oxford Common File Layout, a filesystem that brings content and metadata together. It’s actively not a software platform and it’s not a replacement for traditional digipres activities like checksums. It’s a bit like applying the philosophy of static site generators to research data management; it’s a minimalist, long-term, sustainably-minded approach that manages data in line with the FAIR principles. It also recognises that researchers have short-term incentives not to adequately describe or contextualise their research data (no matter how much librarians exhort them to) and tries to make it easier for them.

The new PARADISEC catalogue includes Arkisto and an associated web interface, Oni, as part of its tech stack. I was very taken with the catalogue’s principle of ‘graceful degradation’—even if the search function ceases to operate, browsing and viewing items will still work. As a former web archivist I was heartened to see them holding this more limited functionality in mind, an astute recognition that all heritage, be it virtual, environmental or built, will eventually decay. So much of my web archiving work involved desperately patching dynamic websites into something that bore a passing resemblance to what they had once been. We might not always be able to save the infrastructure, but one hopes we can more often save the art, the data, the files, the stories. (Which reminds me, I’ve had Curated Decay on my to-read shelf for far too long.)

I shouldn’t have needed reminding of this, but sometimes I forget that metadata doesn’t begin and end with the library sector. It was a thrill to hear someone in a related field speaking my language! I wanna hang out with these people more often now.

Generosity resides in all of us. My first impressions of Hugh Rundle’s talk were somewhat unfavourable—he only spent a couple of minutes talking about the bones of his project, a Library Map of every public library in Australia, and instead dedicated the bulk of his time to complaining about the poor quality of open datasets. Despite having had several sneak previews I was rather hoping to see more of the map itself, including its ‘white fragility mode’ and the prevalence of fine-free libraries across the country. Instead I felt a bit deflated by the persistent snark. Hugh was the only speaker to explicitly reference the miniconf’s fuller title of ‘Generous and Open GLAM’. But this felt like an ungenerous talk. Why did it bother me?

Perhaps it’s because Hugh is a close friend of mine, and I expected him to be as kind and generous about the failings of Data Vic as he is about my own. I’m not sure I held other speakers to that high a standard, but I don’t think anyone else was quite as mean about their data sources. I also hadn’t eaten a proper breakfast, so maybe I was just hangry, and I ought to give Hugh the benefit of the doubt. After all, he had a lot on his plate. I intend to rewatch this talk when the recordings come out, to see if I feel the same way about it on a full stomach. I hope I feel differently. The Library Map really is a great piece of software, and I don’t think Hugh quite did it justice.

Omg pull requests make sense now. Liz Stokes is absolutely delightful, and her talk ‘Once more, with feeling!’ was no exception. Her trademark cheerfulness, gentleness and generosity shone in this talk, where she explored what makes a comfortable learning environment for tech newbies, and demonstrated just such an environment by teaching us how GitHub pull requests worked. How did she know that I desperately needed to know this?! Pull requests had just never made sense to me—until that afternoon. You ‘fork’ a repository by copying it to your space, then make changes you think the original repo would benefit from, then leave a little note explaining what you did and ‘request’ that the original owner ‘pull’ your changes back into the repo. A pull request! Amazing! A spotlight shone upon my brain and angels trumpeted from the heavens. This made my whole day. Hark, the gift of knowledge!

Liz also touched on the value of learning how to ‘think computationally’, a skill I have come to deeply appreciate as I progress in my technical library career. I’ve attended multiple VALA Tech Camps (including as a presenter), I’ve done all sorts of workshops and webinars, I’ve tried learning to code umpteen times (and just the other day bought Julia Evans’ SQL zine Become a SELECT Star! because I think I’ll shortly need it for work), but nowhere did I ever formally learn the basics of computational thinking. Computers don’t think like humans do, and in order to tell computers what we want, we have to learn to speak their language. But so much learn-to-code instruction attempts to teach the language without the grammar.

I don’t have a computer science background—I have an undergraduate degree in classics, and am suddenly reminded of the innovative Traditional Grammar course that I took at ANU many years ago. Most students come to Classical Studies with little knowledge of grammar in any language; instead of throwing them headfirst into the intricacies of the ancient languages, they learn about the grammars of English, Latin and Ancient Greek first and together. This gives students a solid grounding of the mechanics of language, setting them up for success in future years. Programming languages need a course like Traditional Grammar. Just as classicists learn to think like Romans, prospective coders need to be explicitly taught how to think like computers. A kind of basic computational literacy course.

(Of all the things I thought I’d get out of the day, I didn’t expect a newfound admiration of Professor Elizabeth Minchin to be one of them.)

Online confs are awesome! Being somewhat late to the online conference party, GO GLAM was my first experience of an exclusively online conference. I’ve watched a handful of livestreams before, but it just isn’t the same. A bit like reading a photocopied book. I don’t think I had any particular expectations of LCA, but I figured I’ve sat in on enough zoom webinars, it’d be a bit like that, right? Wrong. The LCA audio-visual and conference tech stack was an absolute thing of beauty. Everything looked a million bucks, everything was simple and easy to use. It was a far more active watching experience than simply tucking into a livestream—the chat box on the right-hand side, plus the breakout Q&A areas, helped me feel as if I were truly part of the action. I didn’t usually have a lot to say past ‘That was awesome!’ but it was far less intimidating than raising my hand at an in-person Q&A or cold-tweeting a speaker after the fact.

As someone who is deeply introverted, probably neurodivergent and extremely online, virtual conferences like GO GLAM are so much more accessible than their real-life counterparts. I didn’t have to travel, get up early, put on my People Face, spend hours in a bright and noisy conference hall, eat mediocre food, make painful small talk, take awkward pictures of slides and furiously live-tweet at the same time, massively exhaust myself and make a mad dash for the exit. Instead I could have a nap, grab another pot of tea, turn the lights down, share links in the chat, clap with emojis, watch people make great connections, take neat and tidy screenshots of slides, squeeze in a spot of Hammock Time and still be feeling excited by it all at the end of the day.

I’m sure people will want to return to some form of physical conferencing in the fullness of time, but I fervently hope that online conferencing becomes the new norm. This infrastructure exists, it costs a lot less than you think (certainly less than venue hire and catering), and it makes conferences accessible to people for whom the old normal just wasn’t working. Please don’t leave us behind when the world comes back.

factoryjoe posted a photo:

www.amazon.com/gp/help/contact/contact.html

Unexpectedly pleased with tonight’s dinner, which I call Balti Chicken with Noodles.

For a little over a year I’ve been a monthly supporter of the Internet Archive. I’m doing this because the Internet Archive is a very useful resource to me. Both to find back things I’ve linked to in the past, since taken offline but preserved in the Archive, and as a neutral way of providing proof something was said somewhere online at some point. You may have noticed that Wikipedia increasingly routes all their internal links through the Archive, because it preserves the linked site as it was referenced, thus maintaining the integrity and the verifiability of that reference.

This month I received a thank you mail from the Internet Archive, and in part it said….

Wait, what? They have only 4000 monthly contributors? For a global service like that? There must be more people out there able to make a monthly donation, that can help build a solid fundament underneath the Internet Archive. Especially if you use the Archive regularly as a tool, it is worth considering if you would like to make a recurring donation.

I see my own donation as a subscription rather, part of rearranging where I spent my money and changing some spending habits. In the past months I stopped spending money with Amazon if I can help it. Part of that flows to independent book sellers, part of it flows to independent online services I use, such as the Internet Archive. More deliberately ‘voting with my spending’ of sorts.

This is fun (don’t miss the video at the bottom). If the RP2040 is powerful enough to drive a 16-voice polyphonic synthesizer, there is certainly a lot more like this to come.

I fear I may indeed need more room for playing around with music hardware in the future.

What are the major questions in software engineering that researchers should be trying to answer?

A high level question whose answer is likely to involve life, the universe, and everything is: What is the most cost-effective way to build software systems?

Viewing software engineering research as an attempt to find the answer to a big question mirrors physicists quest for a Grand Unified Theory of how the Universe works.

Physicists have the luxury of studying the Universe at their own convenience, the Universe does not need their input to do a better job.

Software engineering is not like physics. Once a software system has been built, the resources have been invested, and there is no reason to recreate it using a more cost-effective approach (the zero cost of software duplication means that manufacturing cost is the cost of the first version).

Designing and researching new ways of building software systems may be great fun, but the time and money needed to run the realistic experiments needed to evaluate their effectiveness is such that they are unlikely to be run. Searching for more cost-effective software development techniques by paying to run the realistic experiments needed to evaluate them, and waiting for the results to become available, is going to be expensive and time-consuming. A theory is proposed, experiments are run, results are analysed; rinse and repeat until a good-enough cost-effective technique is found. One iteration will take many years, and this iterative process is likely to take many decades.

Very many software systems are being built and maintained, and each of these is an experiment. Data from these ‘experiments’ provides a cost-effective approach to improving existing software engineering practices by studying the existing practices to figure out how they work (or don’t work).

Given the volume of ongoing software development, most of the payback from any research investment is likely to occur in the near future, not decades from now; the evidence shows that source code has a short and lonely existence. Investing for a payback that might occur 30-years from now makes no sense; researchers I talk to often use this time-frame when I ask them about the benefits of their research, i.e., just before they are about to retire. Investing in software engineering research only makes economic sense when it is focused on questions that are expected to start providing payback in, say, 3-5 years.

Who is going to base their research on existing industry practices?

Researching existing practices often involves dealing with people issues, and many researchers in computing departments are not that interested in the people side of software engineering, or rather they are more interested in the computer side.

Algorithm oriented is how I would describe researchers who claim to be studying software engineering. I am frequently told about the potential for huge benefits from the discovery of more efficient algorithms. For many applications, algorithms are now commodities, i.e., they are good enough. Those with a career commitment to studying algorithms have a blinkered view of the likely benefits of their work (most of those I have seen are doing studying incremental improvements, and are very unlikely to make a major break through).

The number of researchers studying what professional developers do, with an aim to improving it, is very small (I am excluding the growing number of fake researchers doing surveys). While I hope there will be a significant growth in numbers, I’m not holding my breadth (at least in the short term; as for the long term, Planck’s experience with quantum mechanics was: “Science advances one funeral at a time”).

The previous post introduced the topic of how to compile Swift code for use in R using a useless, toy example. This one goes a bit further and makes a case for why one might want to do this by showing how to use one of Apple’s machine learning libraries, specifically the Natural Language one, focusing on extracting parts of speech from text.

I made a parts-of-speech directory to keep the code self-contained. In it are two files. The first is partsofspeech.swift (swiftc seems to dislike dashes in names of library code and I dislike underscores):

import NaturalLanguage
import CoreML

extension Array where Element == String {
  var SEXP: SEXP? {
    let charVec = Rf_protect(Rf_allocVector(SEXPTYPE(STRSXP), count))
    defer { Rf_unprotect(1) }
    for (idx, elem) in enumerated() { SET_STRING_ELT(charVec, idx, Rf_mkChar(elem)) }
    return(charVec)
  }
}

@_cdecl ("part_of_speech")
public func part_of_speech(_ x: SEXP) -> SEXP {

  let text = String(cString: R_CHAR(STRING_ELT(x, 0)))
  let tagger = NLTagger(tagSchemes: [.lexicalClass])

  tagger.string = text

  let options: NLTagger.Options = [.omitPunctuation, .omitWhitespace]

  var txts = [String]()
  var tags = [String]()

  tagger.enumerateTags(in: text.startIndex..<text.endIndex, unit: .word, scheme: .lexicalClass, options: options) { tag, tokenRange in
    if let tag = tag {
      txts.append("\(text[tokenRange])")
      tags.append("\(tag.rawValue)")
    }
    return true
  }

  let out = Rf_protect(Rf_allocVector(SEXPTYPE(VECSXP), 2))
  SET_VECTOR_ELT(out, 0, txts.SEXP)
  SET_VECTOR_ELT(out, 1, tags.SEXP)
  Rf_unprotect(1)

  return(out!)
}

The other is bridge code that seems to be the same for every one of these (or could be) so I’ve just named it swift-r-glue.h (it’s the same as the bridge code in the previous post):

#define USE_RINTERNALS

#include <R.h>
#include <Rinternals.h>

const char* R_CHAR(SEXP x);

Let’s walk through the Swift code.

We need to two imports:

import NaturalLanguage
import CoreML

to make use of the NLP functionality provided by Apple.

The following extension to the String Array class:

extension Array where Element == String {
  var SEXP: SEXP? {
    let charVec = Rf_protect(Rf_allocVector(SEXPTYPE(STRSXP), count))
    defer { Rf_unprotect(1) }
    for (idx, elem) in enumerated() { SET_STRING_ELT(charVec, idx, Rf_mkChar(elem)) }
    return(charVec)
  }
}

will reduce the amount of code we need to type later on to turn Swift String Arrays to R character vectors.

The start of the function:

@_cdecl ("part_of_speech")
public func part_of_speech(_ x: SEXP) -> SEXP {

tells swiftc to make this a C-compatible call and notes that the function takes one parameter (in this case, it’s expecting a length 1 character vector) and returns an R-compatible value (which will be a list that we’ll turn into a data.frame in R just for brevity).

The following sets up our inputs and outputs:

  let text = String(cString: R_CHAR(STRING_ELT(x, 0)))
  let tagger = NLTagger(tagSchemes: [.lexicalClass])

  tagger.string = text

  let options: NLTagger.Options = [.omitPunctuation, .omitWhitespace]

  var txts = [String]()
  var tags = [String]()

We convert the passed-in parameter to a Swift String, initialize the NLP tagger, and setup two arrays to hold the results (sentence component in txts and the part of speech that component is in tags).

The following code is mostly straight from Apple and (inefficiently) populates the previous two arrays:

  tagger.enumerateTags(in: text.startIndex..<text.endIndex, unit: .word, scheme: .lexicalClass, options: options) { tag, tokenRange in
    if let tag = tag {
      txts.append("\(text[tokenRange])")
      tags.append("\(tag.rawValue)")
    }
    return true
  }

Finally, we use the Swift-R bridge to make a list much like one would in C:

  let out = Rf_protect(Rf_allocVector(SEXPTYPE(VECSXP), 2))
  SET_VECTOR_ELT(out, 0, txts.SEXP)
  SET_VECTOR_ELT(out, 1, tags.SEXP)
  Rf_unprotect(1)

  return(out!)

To get a shared library we can use from R, we just need to compile this like last time:

swiftc \
  -I /Library/Frameworks/R.framework/Headers \
  -F/Library/Frameworks \
  -framework R \
  -import-objc-header swift-r-glue.h \
  -emit-library \
  partsofspeech.swift

Let’s run that on some text! First, we’ll load the new shared library into R:

dyn.load("libpartsofspeech.dylib")

Next, we’ll make a wrapper function to avoid messy .Call(…)s and to make a data.frame:

parts_of_speech <- function(x) {
  res <- .Call("part_of_speech", x)  
  as.data.frame(stats::setNames(res, c("name", "tag")))
}

Finally, let’s try this on some text!

tibble::as_tibble(
  parts_of_speech(paste0(c(
"The comm wasn't working. Feeling increasingly ridiculous, he pushed",
"the button for the 1MC channel several more times. Nothing. He opened",
"his eyes and saw that all the lights on the panel were out. Then he",
"turned around and saw that the lights on the refrigerator and the",
"ovens were out. It wasn’t just the coffeemaker; the entire galley was",
"in open revolt. Holden looked at the ship name, Rocinante, newly",
"stenciled onto the galley wall, and said, Baby, why do you hurt me",
"when I love you so much?"
  ), collapse = " "))
)
## # A tibble: 92 x 2
##    name         tag
##    <chr>        <chr>
##  1 The          Determiner
##  2 comm         Noun
##  3 was          Verb
##  4 n't          Adverb
##  5 working      Verb
##  6 Feeling      Verb
##  7 increasingly Adverb
##  8 ridiculous   Adjective
##  9 he           Pronoun
## 10 pushed       Verb
## # … with 82 more rows

FIN

If you’re playing along at home, try adding a function to this Swift file that uses Apple’s entity tagger.

The next installment of this topic will be how to wrap all this into a package (then all these examples get tweaked and go into the tome.

A few months ago I posted about being aware of what of your surroundings you could reach within 60 or 90 minutes by car or public transport. Towards the end of that posting I posted a map of my reach from home for 60 and 90 minutes. It was a bit of work to find a service that could make such isochrone maps for me.

Today Open Street Map volunteer Rory pointed to CommuteTimeMap which provides isochrone maps for any location in the world, based on Open Street Map. That’s very cool.

Of course I immediately compared CommuteTimeMap with the maps I had made before. What I used before didn’t allow for doing this for public transport (just walking and cars iirc), and CommuteTimeMap does. However the underlying data about public transport may be incomplete (just buses perhaps), as the map for 60 mins of public transport shows a very limited range, where the actual range is more or less the full size of the image (Zwolle, Amsterdam, Utrecht, Apeldoorn, Ede all within range). Or it simply isn’t set up for multimodal transport, and it assumes I’d take public transport from the bus stop nearest to me, where in reality I would cycle the 7 minutes it takes to the nearest railway station. Taking the bus to the railway station would cost significantly more time.

my 60 min public transport range, according to OSM, more pessimistic than in reality

On the other hand, the map for my reach by car in 60 minutes seems a little bit optimistic, covering most of what on my previously made map is shown for 90 minutes of driving. In general it provides a similar contour though, and a lot more detail (such as excluding car free national parks).

my 60 min driving range, according to OSM

my 60 and 90 min driving range, according to what I previously used

I’m definitely using this from now on.

Our Saturday walk took us to the industrial waterfront views of New Brighton Park.

Plant life everywhere was crunchy with frost.

Dr. Brad J. Cox Ph.D. 1944 - 2021:

Dr. Brad Cox, Ph.D of Manassas, Virginia, died on January 2, 2021 at his residence. Dr. Cox was a computer scientist known mostly for creating the Objective – C programming language with his business partner, Tom Love, and for his work in software engineering (specifically software reuse) and software componentry.

I had always struggled building apps for the Mac, first in Classic MacOS and then in Mac OS X. I used Java, I used REALbasic, and I even had a professional license for Metrowerk's CodeWarrior so I could write apps in C or C++. But there was always some sort of mental block, or maybe the way my brain worked never really lined up with the way these programming languages did. Out of these I was most efficient with Java, but the UI toolkit was pretty lacking.

Then I saw these cool apps coming out for Mac OS X, and they were proudly written in Objective-C.

I took a couple of runs at it. The brackets and pointers threw me off, but I kept on trying because I knew I needed to learn this language to build the apps I wanted to see in this world.

And then one day, I believe when plugging away on my little internet search utility VoodooNetKey, everything clicked.

Pointers, the brackets, selectors, messages. Oh my god, I get it now — the brackets! It's not a function call, it's something way better!

Everything just sort of aligned in my brain. Previously I found myself struggling to express the ideas I had in my head, and it was a frustrating experience. Now I was struggling to type fast enough and come up with new ideas that I could express in Objective-C. I felt like I could do anything I wanted with it.

And loadable bundles, and input managers- I can't express to you about how much fun it was being able to inject my Objective-C code into Safari or other applications to add functionality at runtime. Or using class-dump against the system frameworks to explore private methods. NSObject's performSelector: was just so great when you needed to quickly hack around something.

Objective-C was brilliant, powerful, and elegant.

All my applications are written in Objective-C even to this day. I play around in other languages from time to time, but I haven't found one that fits me like Objective-C does.

I realize it's not for everyone, and that Objective-C requires discipline, but it's been amazing for me. I owe my career in part to Brad Cox's creation, and I am forever grateful for it.

This Zoom forum thread about being able to story gallery view provides a fascinating look into the many people, from teachers to dancers to actors to game developers, for whom this feature would be a big improvement.

I noticed that, after a recent update, my Zoom did, indeed, allow me to drag people around gallery view; what I didn’t know is that if the host does this, then the order can be deployed for everyone:

Your custom order will be seen only by you, or the host can deploy their custom view to all participants. This order can be released and the order will revert to the default.

Cucumbers were really the only food that Catherine truly abhorred. As a result, I lived in a cucumber-free household for 28 years; it was as if cucumbers went extinct in 1992.

This week, as part of my weekly online Charlottetown Farmers’ Market order, I added an Atlantic Grown Organics cucumber, and I’ve built cucumber into our last two suppers. Wow. Fantastic.

All hail the cucumber, back from extinction.

The week was grueling and I’m effectively working through the weekend (at least a few hours a day) to catch up, but I thought a short update was in order regarding my last post on this topic, since a number of things have come to pass.

This post is the latest in my increasingly long series on the COVID-19 pandemic as viewed from Portugal, which began 50 days after the start of the pandemic and has had irregular updates 120, 200-ish, 250-ish, 300-ish and 320-ish days later.

This Week

…was a mess, in many ways:

• Schools were closed this Thursday, with the haphazard justification that the English variant of COVID-19 is quickly rising to prevalence (and semi-denial that they would contribute to community spreading regardless of common sense).
• We’ve surpassed every single indicator (cases/day/million, deaths/day/million, etc. worldwide). Portugal is now officially the country that is hardest hit by the pandemic (and, in my rather biased opinion, where it’s been most mis-managed).
• All manner of mobility indicators show that despite the seriousness of the situation, there are roughly twice as many people out and about than in last year’s lockdown. People just don’t care (even down to walking empty dog leashes and exploring all manner of other loopholes).
• Emergency services are overwhelmed in various aspects (beds and gear are not quite lacking but can’t be set up quickly enough, and we’re running out of carers, with people being drafted and from all kinds of medical staff and fast-tracked through training). Ambulances are scarce for any kind of patients right now.
• We’re still going to have the presidential election tomorrow, where I expect absenteeism to be at least 70% (if not up to 90%) and our local right-wing Trump wannabe (who, incidentally, has flouted his contempt of sanitary rules more than once) to garner enough votes to become more than a nuisance.

This last one is partly because our Constitution does not allow for postponing elections come hell or high water, and because there seems to be zero political common sense and will to amend it, in much the same way we stumble forward every fortnight and have long and protracted bi-weekly decisions because emergency measures are limited to 15 days’ duration, and our politicians can’t think (or act) faster than the pandemic.

New Data

Meanwhile, all curves on my dashboard are still going exponential:

…and I’ve started looking at some correlations, since with a few hundred data points I can actually trust their accuracy somewhat:

Decision Paralysis

Amidst the overbearing certainty that this could have been averted to a massive degree by taking action much earlier (instead of endless political dithering) and having done much more in terms of preparation–like reinforcing staff and hospitals during early Autumn and starting the lockdown before Christmas, instead of putting up a facade of normality and loosening restrictions during the holiday season–there are some things that I find utterly inexcusable and thoroughly irresponsible.

One of them is that the Ministry of Education, in a gloriously incompetent way worthy of being prominently featured as a textbook example of governing according to the least common denominator, has decreed that the next two weeks are an “interruption” (i.e., vacation, to be compensated later) and has stated no schools are to move to online learning, flamboyantly ignoring the fact that even though it is clear that there are thousands of schools (and kids, and most especially teachers) that are not prepared (or have the means) for it, the Government did nothing to prepare for this situation since last year, and would rather burn these two weeks instead of allowing many schools that had forethought to decide for themselves.

Oh, and it was pretty obvious this was going to happen. Announcing it this late, in this way and without any forward-looking plan borders on criminal incompetence, and I fully expect at least a couple of rushed “digitizations” of public schools to emerge as media success stories soon in an attempt to save face.

How It Affects Us

My kids’ school issued a contrite note stating (and I paraphrase) “we were fully prepared to move back to full remote schooling this Friday, but the government won’t allow it.”

My kids were expecting (and ready) to resume online classes, and are now miserable because they can’t do that. And yes, I know we’re privileged in that regard, and I can only imagine what it must be like for the majority of people (especially those with smaller kids), but having them around the house doing nothing for two weeks means a fair amount of supervision that, given my current workload, I would rather do without…

Also, online shopping is getting a little wonky again (not to the same degree as last year, but cooking balanced meals for 4 people on a daily basis–some with dietary restrictions–is going to be another challenge).

What really gets to me is the brazen way the Portuguese government gets away with the roaring abyss of inaction it leaves behind us, and that we seem to be ruled by a pack of medieval peasants who would rather have us all roll in the muck rather than stand up for ourselves whenever possible, let alone planning ahead…

If you’ve read this far, apologies for the rant. I have far too much on my mind at this point to hold everything back, and, at the same time, I’m trying to lay down enough reference points that I can look back to these times later and write something further in-depth.

I’ve been on a Swift + R bender for a while now, but have been envious of the pure macOS/iOS (et al) folks who get to use Apple’s seriously ++good machine learning libraries, which are even more robust on the new M1 hardware (it’s cool having hardware components dedicated to improving the performance of built models).

Sure, it’s pretty straightforward to make a command-line utility that can take data input, run them through models, then haul the data back into R, but I figured it was about time that Swift got the “Rust” and “Go” treatment in terms of letting R call compiled Swift code directly. Thankfully, none of this involves using Xcode since it’s one of the world’s worst IDEs.

To play along at home you’ll need macOS and at least the command line tools installed (I don’t think this requires a full Xcode install, but y’all can let me know if it does in the comments). If you can enter swiftc at a terminal prompt and get back <unknown>:0: error: no input files then you’re good-to-go.

Hello, Swift!

To keep this post short (since I’ll be adding this entire concept to the SwiftR tome), we’ll be super-focused and just build a shared library we can dynamically load into R. That library will have one function which will be to let us say hello to the planet with a customized greeting.

Make a new directory for this effort (I called mine greetings) and create a greetings.swift file with the following contents:

All this code is also in this gist.

@_cdecl ("greetings_from")
public func greetings_from(_ who: SEXP) -> SEXP {
  print("Greetings, 🌎, it's \(String(cString: R_CHAR(STRING_ELT(who, 0))))!")
  return(R_NilValue)
}

Before I explain what’s going on there, also create a geetings.h file with the following contents:

#define USE_RINTERNALS

#include <R.h>
#include <Rinternals.h>

const char* R_CHAR(SEXP x);

In the Swift file, there’s a single function that takes an R SEXP and converts it into a Swift String which is then routed to stdout (not a “great” R idiom, but benign enough for an intro example). Swift functions aren’t C functions and on their own do not adhere to C calling conventions. Unfortunately R’s ability to work with dynamic library code requires such a contract to be in place. Thankfully, the Swift Language Overlords provided us with the ability to instruct the compiler to create library code that will force the calling conventions to be C-like (that’s what the ＠cdecl is for).

We’re using SEXP, some R C-interface functions, and even the C version of NULL in the Swift code, but we haven’t done anything in the Swift file to tell Swift about the existence of these elements. That’s what the C header file is for (I added the R_CHAR declaration since complex C macros don’t work in Swift).

Now, all we need to do is make sure the compiler knows about the header file (which is a “bridge” between C and Swift), where the R framework is, and that we want to generate a library vs a binary executable file as we compile the code. Make sure you’re in the same directory as both the .swift and .h file and execute the following at a terminal prompt:

swiftc \
  -I /Library/Frameworks/R.framework/Headers \ # where the R headers are
  -F/Library/Frameworks \                      # where the R.framework lives
  -framework R \                               # we want to link against the R framework
  -import-objc-header greetings.h \            # this is our bridging header which will make R C things available to Swift
  -emit-library \                              # we want a library, not an exe
  greetings.swift                              # our file!

If all goes well, you should have a libgreetings.dylib shared library in that directory.

Now, fire up a R console session in that directory and do:

greetings_lib <- dyn.load("libgreetings.dylib")

If there are no errors, the shared library has been loaded into your R session and we can use the function we just made! Let’s wrap it in an R function so we’re not constantly typing .Call(…):

greetings_from <- function(who = "me") {
  invisible(.Call("greetings_from", as.character(who[1])))
}

I also took the opportunity to make sure we are sending a length-1 character vector to the C/Swift function.

Now, say hello!

greetings_from("hrbrmstr")

And you should see:

Greetings, 🌎, it's hrbrmstr!

FIN

We’ll stop there for now, but hopefully this small introduction has shown how straightforward it can be to bridge Swift & R in the other direction.

I’ll have another post that shows how to extend this toy example to use one of Apple’s natural language processing libraries, and may even do one more on how to put all this into a package before I shunt all the individual posts into a few book chapters.