A set of simple guidelines to write consistent unix tools.

Those outside the Colonies may not know that Payless—a national chain that made footwear affordable for millions of ‘Muricans who can’t spare $100.00 USD for a pair of shoes their 7 year old will outgrow in a year— is closing. CNBC also had a story that featured a choropleth with a tiny button at the bottom that indicated one could get the data:

I should have known this would turn out to be a chore since they used Tableau—the platform of choice when you want to take advantage of all the free software libraries they use to power their premier platform which, in turn, locks up all the data for you so others can’t adopt, adapt and improve. Go. Egregious. Predatory. Capitalism.

Anyway.

I wanted the data to do some real analysis vs produce a fairly unhelpful visualization (TLDR: layer in Census data for areas impacted, estimate job losses, compute nearest similar Payless stores to see impact on transportation-challenged homes, etc. Y’now, citizen data journalism-y things) so I pressed the button and watched for the URL in Chrome (aye, for those that remember I moved to Firefox et al in 2018, I switched back; more on that in March) and copied it to try to make this post actually reproducible (a novel concept for Tableau fanbois):

library(tibble)
library(readr)

# https://www.cnbc.com/2019/02/19/heres-a-map-of-where-payless-shoesource-is-closing-2500-stores.html

tfil <- "~/Data/Sheet_3_data.csv"

download.file(
  "https://public.tableau.com/vizql/w/PAYLESSSTORECLOSINGS/v/Dashboard2/vud/sessions/6A678928620645FF99C7EF6353426CE8-0:0/views/10625182665948828489_7202546092381496425?csv=true&showall=true",
  tfil
)
## trying URL 'https://public.tableau.com/vizql/w/PAYLESSSTORECLOSINGS/v/Dashboard2/vud/sessions/6A678928620645FF99C7EF6353426CE8-0:0/views/10625182665948828489_7202546092381496425?csv=true&showall=true'
## Error in download.file("https://public.tableau.com/vizql/w/PAYLESSSTORECLOSINGS/v/Dashboard2/vud/sessions/6A678928620645FF99C7EF6353426CE8-0:0/views/10625182665948828489_7202546092381496425?csv=true&showall=true",  : 
##   cannot open URL 'https://public.tableau.com/vizql/w/PAYLESSSTORECLOSINGS/v/Dashboard2/vud/sessions/6A678928620645FF99C7EF6353426CE8-0:0/views/10625182665948828489_7202546092381496425?csv=true&showall=true'
## In addition: Warning message:
## In download.file("https://public.tableau.com/vizql/w/PAYLESSSTORECLOSINGS/v/Dashboard2/vud/sessions/6A678928620645FF99C7EF6353426CE8-0:0/views/10625182665948828489_7202546092381496425?csv=true&showall=true",  :
##   cannot open URL 'https://public.tableau.com/vizql/w/PAYLESSSTORECLOSINGS/v/Dashboard2/vud/sessions/6A678928620645FF99C7EF6353426CE8-0:0/views/10625182665948828489_7202546092381496425?csv=true&showall=true': HTTP status was '410 Gone'

WAT

Truth be told I expected a time-boxed URL of some sort (prior experience FTW). Selenium or Splash were potential alternatives but I didn’t want to research the legality of more forceful scraping (I just wanted the data) so I manually downloaded the file (*the horror*) and proceeded to read it in. Well, try to read it in:

read_csv(tfil)
## Parsed with column specification:
## cols(
##   A = col_logical()
## )
## Warning: 2092 parsing failures.
## row col           expected actual                      file
##   1   A 1/0/T/F/TRUE/FALSE        '~/Data/Sheet_3_data.csv'
##   2   A 1/0/T/F/TRUE/FALSE        '~/Data/Sheet_3_data.csv'
##   3   A 1/0/T/F/TRUE/FALSE        '~/Data/Sheet_3_data.csv'
##   4   A 1/0/T/F/TRUE/FALSE        '~/Data/Sheet_3_data.csv'
##   5   A 1/0/T/F/TRUE/FALSE        '~/Data/Sheet_3_data.csv'
## ... ... .................. ...... .........................
## See problems(...) for more details.
## 
## # A tibble: 2,090 x 1
##    A    
##    <lgl>
##  1 NA   
##  2 NA   
##  3 NA   
##  4 NA   
##  5 NA   
##  6 NA   
##  7 NA   
##  8 NA   
##  9 NA   
## 10 NA   
## # … with 2,080 more rows

WAT

Getting a single column back from readr::read_[ct]sv() is (generally) a tell-tale sign that the file format is amiss. Before donning a deerstalker (I just wanted the data!) I tried to just use good ol’ read.csv():

read.csv(tfil, stringsAsFactors=FALSE)
## Error in make.names(col.names, unique = TRUE) : 
##   invalid multibyte string at '<ff><fe>A'
## In addition: Warning messages:
## 1: In read.table(file = file, header = header, sep = sep, quote = quote,  :
##   line 1 appears to contain embedded nulls
## 2: In read.table(file = file, header = header, sep = sep, quote = quote,  :
##   line 2 appears to contain embedded nulls
## 3: In read.table(file = file, header = header, sep = sep, quote = quote,  :
##   line 3 appears to contain embedded nulls
## 4: In read.table(file = file, header = header, sep = sep, quote = quote,  :
##   line 4 appears to contain embedded nulls
## 5: In read.table(file = file, header = header, sep = sep, quote = quote,  :
##   line 5 appears to contain embedded nulls

WAT

Actually the “WAT” isn’t really warranted since read.csv() gave us some super-valuable info via invalid multibyte string at '<ff><fe>A'. FF FE is a big signal^{1 2} we’re working with a file in another encoding as that’s a common “magic” sequence at the start of such files.

But, I didn’t want to delve into my Columbo persona… I. Just. Wanted. The. Data. So, I tried the mind-bendingly fast and flexible helper from data.table:

data.table::fread(tfil)
## Error in data.table::fread(tfil) : 
##   File is encoded in UTF-16, this encoding is not supported by fread(). Please recode the file to UTF-8.

AHA. UTF-16 (maybe). Let’s poke at the raw file:

x <- readBin(tfil, "raw", file.size(tfil)) ## also: read_file_raw(tfil)

x[1:100]
##   [1] ff fe 41 00 64 00 64 00 72 00 65 00 73 00 73 00 09 00 43 00
##  [21] 69 00 74 00 79 00 09 00 43 00 6f 00 75 00 6e 00 74 00 72 00
##  [41] 79 00 09 00 49 00 6e 00 64 00 65 00 78 00 09 00 4c 00 61 00
##  [61] 62 00 65 00 6c 00 09 00 4c 00 61 00 74 00 69 00 74 00 75 00
##  [81] 64 00 65 00 09 00 4c 00 6f 00 6e 00 67 00 69 00 74 00 75 00

There’s our ff fe (which is the beginning of the possibility it’s UTF-16) but that 41 00 harkens back to UTF-16’s older sibling UCS-2. The 0x00‘s are embedded nuls (likely to get bytes aligned). And, there are alot of 09s. Y’know what they are? They’re <tab>s. That’s right. Tableau named file full of TSV records in an unnecessary elaborate encoding as CSV. Perhaps they broke the “T” on all their keyboards typing their product name so much.

Living A Boy’s [Data] Adventure Tale

At this point we have:

• no way to support an automated, reproducible workflow
• an ill-named file for what it contains
• an overly-encoded file for what it contains
• many wasted minutes (which is likely by design to have us give up and just use Tableau. No. Way.)

At this point I’m in full-on Rockford Files (pun intended) mode and delved down to the command line to use a old, trusted sidekick enca:

$ enca -L none Sheet_3_data.csv
## Universal character set 2 bytes; UCS-2; BMP
##   LF line terminators
##   Byte order reversed in pairs (1,2 -> 2,1)

Now, all we have to do is specify the encoding!

read_tsv(tfil, locale = locale(encoding = "UCS-2LE"))
## Error in guess_header_(datasource, tokenizer, locale) : 
##   Incomplete multibyte sequence

WAT

Unlike the other 99% of the time (mebbe 99.9%) you use it, the tidyverse doesn’t have your back in this situation (but it does have your backlog in that it’s on the TODO).

Y’know who does have your back? Base R!:

read.csv(tfil, sep="\t", fileEncoding = "UCS-2LE", stringsAsFactors=FALSE) %>% 
  as_tibble()
## # A tibble: 2,089 x 14
##    Address City  Country Index Label Latitude Longitude
##    <chr>   <chr> <chr>   <int> <chr>    <dbl>     <dbl>
##  1 1627 O… Aubu… United…     1 Payl…     32.6     -85.4
##  2 900 Co… Doth… United…     2 Payl…     31.3     -85.4
##  3 301 Co… Flor… United…     3 Payl…     34.8     -87.6
##  4 304 Ox… Home… United…     4 Payl…     33.5     -86.8
##  5 2000 R… Hoov… United…     5 Payl…     33.4     -86.8
##  6 6140 U… Hunt… United…     6 Payl…     34.7     -86.7
##  7 312 Sc… Mobi… United…     7 Payl…     30.7     -88.2
##  8 3402 B… Mobi… United…     8 Payl…     30.7     -88.1
##  9 5300 H… Mobi… United…     9 Payl…     30.6     -88.2
## 10 6641 A… Mont… United…    10 Payl…     32.4     -86.2
## # … with 2,079 more rows, and 7 more variables:
## #   Number.of.Records <int>, State <chr>, Store.Number <int>,
## #   Store.count <int>, Zip.code <chr>, State.Usps <chr>,
## #   statename <chr>

WAT WOOT!

Note that read.csv(tfil, sep="\t", fileEncoding = "UTF-16LE", stringsAsFactors=FALSE) would have worked equally as well.

The Road Not [Originally] Taken

Since this activity decimated productivity, for giggles I turned to another trusted R sidekick, the stringi package, to see what it said:

library(stringi)

stri_enc_detect(x)
## [[1]]
##      Encoding Language Confidence
## 1    UTF-16LE                1.00
## 2  ISO-8859-1       pt       0.61
## 3  ISO-8859-2       cs       0.39
## 4    UTF-16BE                0.10
## 5   Shift_JIS       ja       0.10
## 6     GB18030       zh       0.10
## 7      EUC-JP       ja       0.10
## 8      EUC-KR       ko       0.10
## 9        Big5       zh       0.10
## 10 ISO-8859-9       tr       0.01

And, just so it’s primed in the Google caches for future searchers, another way to get this data (and other data that’s even gnarlier but similar in form) into R would have been:

stri_read_lines(tfil) %>% 
  paste0(collapse="\n") %>% 
  read.csv(text=., sep="\t", stringsAsFactors=FALSE) %>% 
  as_tibble()
## # A tibble: 2,089 x 14
##    Address City  Country Index Label Latitude Longitude
##    <chr>   <chr> <chr>   <dbl> <chr>    <dbl>     <dbl>
##  1 1627 O… Aubu… United…     1 Payl…     32.6     -85.4
##  2 900 Co… Doth… United…     2 Payl…     31.3     -85.4
##  3 301 Co… Flor… United…     3 Payl…     34.8     -87.6
##  4 304 Ox… Home… United…     4 Payl…     33.5     -86.8
##  5 2000 R… Hoov… United…     5 Payl…     33.4     -86.8
##  6 6140 U… Hunt… United…     6 Payl…     34.7     -86.7
##  7 312 Sc… Mobi… United…     7 Payl…     30.7     -88.2
##  8 3402 B… Mobi… United…     8 Payl…     30.7     -88.1
##  9 5300 H… Mobi… United…     9 Payl…     30.6     -88.2
## 10 6641 A… Mont… United…    10 Payl…     32.4     -86.2
## # … with 2,079 more rows, and 7 more variables: `Number of
## #   Records` <dbl>, State <chr>, `Store Number` <dbl>, `Store
## #   count` <dbl>, `Zip code` <chr>, `State Usps` <chr>,
## #   statename <chr>

(with similar dances to use read_csv() or fread()).

FIN

The night’s quest to do some real work with the data was DoS’d by what I’ll brazenly call a deliberate attempt to dissuade doing exactly that in anything but a commercial program. But, understanding the impact of yet-another massive retail store closing is super-important and it looks like it may be up to us (since the media is too distracted by incompetent leaders and inexperienced junior NY representatives) to do the work so it is left for another eve.

Folks who’d like to do the same can grab the UTF-8 encoded actual CSV from this site which has also been run through janitor::clean_names() so there’s proper column types and names to work with.

Speaking of which, here’s the cols spec for that CSV:

cols(
  address = col_character(),
  city = col_character(),
  country = col_character(),
  index = col_double(),
  label = col_character(),
  latitude = col_double(),
  longitude = col_double(),
  number_of_records = col_double(),
  state = col_character(),
  store_number = col_double(),
  store_count = col_double(),
  zip_code = col_character(),
  state_usps = col_character(),
  statename = col_character()
)

If you do anything with the data blog about it and post a link in the comments so I and others can learn from what you’ve discovered! It’s already kinda scary that one doesn’t even need a basemap to see just how much a part of ‘Murica Payless was:

The integrated community hub is a seductive idea.

Imagine it! A single destination which combines blogs, forums, groups, events, training, ideation, advocacy, webinars, case studies, documentation. Better yet, it’s integrated with your customer database. You will know exactly what kind of activities your customers participate in.

Every member can access everything through a single login.

The problem is this often works better in theory than in practice.

Part of the problem is members don’t see it this way. Members who have loved visiting the same forum, scanning activity, and reading the discussions that interest them now have to trawl through much more clutter.

Often the single, secure, login automatically logs members out every week (or, sometimes, every day). Which drives members nuts. No-one wants to log in every time they want to participate. Sometimes members simply have no interest in doing all the other things.

A bigger problem is the benefits aren’t as clear as you might think. Migrations like these are costly in both time and resources. They put all other community projects on hold for months, even years. And few organizations genuinely do much with all the extra data they have on their customers/community participation.

Which doesn’t mean you shouldn’t do it. But before you do it you should be clear about the problem you’re trying to solve.

If your current web platform will soon be unsupported, has security issues, or may go out of business, that’s a good reason.

If members are complaining about the limited functionality, that’s also a good reason.

If staff can clearly articulate what extra data they need and what they will do with the data, that too might be a good reason.

Remember the goal isn’t to make a better experience for you managing the community, but for members experiencing the community. This means you need members involved and guiding the process. If you can’t do that, you should probably stick with what you have.

AZ Central: Grand Canyon tourists exposed for years to radiation in museum building, safety manager says.

“For nearly two decades at the Grand Canyon, tourists, employees, and children on tours passed by three paint buckets stored in the National Park’s museum collection building, unaware that they were being exposed to radiation.

Although federal officials learned last year that the 5-gallon containers were brimming with uranium ore, then removed the radioactive specimens, the park’s safety director alleges nothing was done to warn park workers or the public that they might have been exposed to unsafe levels of radiation.

In a rogue email sent to all Park Service employees on Feb. 4, Elston “Swede” Stephenson — the safety, health and wellness manager — described the alleged cover-up as “a top management failure” and warned of possible health consequences.
[…]
In his letter to colleagues, Stephenson apologized for the untimely notice. He stressed that exposure may not be severe depending on how close individuals got to the source, how long they were exposed, what they were wearing, and other factors. He also emphasized that employees will not necessarily suffer health consequences, but should consider receiving a medical screening.

“Of particular concern are 1000s of children attending ‘shows’ in very close proximity to the uranium,” he wrote. Those presentations lasted a half hour or more, he said, yet radiation dosages could have exceeded federal safety standards within seconds.
“

Link via MetaFilter.

Last Wednesday, Vancouver City Council approved funding for a permanent makeover to Vancouver’s only central public square. Earlier this month, there was concerns that funding for the 800-block Robson Square plaza might be reallocated. With this vote, the funding is secured and the project is moving forward.

I have been following the story of public space around the Vancouver Art Gallery for years, from my sharing my concerns about the design of the North Plaza, to advocating for the permanent closure of the road that runs through Robson Square. Since then, the North Plaza has been renovated, the 800 block of Robson Street has been permanently closed. The only missing piece is renovating Robson Square, which is now slated to be completed this year!

The $5.4 million new plaza will include a continuous flat level of concrete and pavers consistent with the rest of Robson Square, landscaping, movable furniture, curved wooden benches around the glass domes, permanent lighting, and public art.

There will be bollards on both ends of the plaza and trees will be planted on the plaza’s Hornby Street end. The pedestrian crossings on each side of the plaza will become Vancouver’s first pedestrian scrambles, which stop traffic and allow pedestrians to cross from all sides of an intersection.

Give thanks to the team at the Vancouver Public Space Network (VPSN), who have been advocating for the expansion and enhancement of Robson Square since they launched their organization over 12 years ago. According to the VPSN:

“Robson Square (and the 800-block) is the public space heart of the city, widely used and loved by people of all backgrounds, circumstances, and walks of life. Today’s decision is a big city-building move, the benefits of which will be felt for years to come.”

Over the years I’ve noticed a trend in consultative facilitations that goes something like this: a client calls wanting to consult with the community about something. Sometimes this takes the form of a leader wanting to engage employees. The request is usually to design an event where we can hear from people without them being dominated by more powerful voices. At some point the client says something like “we’d like to have our people there as observers or table hosts or mixed in as silent listeners.”

Often this looks like elected officials not wanting to dominate citizen meetings, government or agency staff not wanting to dominate community meetings, or executive teams not wanting to dominate the lower level employees.

My response to this over the years has been to push back hard against that idea, despite how noble it seems. Often it comes from a good place: that those with power want to create space for people without power to speak and have their ideas taken seriously. I get that, and I honour it, but truthfully the best way to do engagement is to, well, engage. It’s entirely possible to design engagement to maximize what you want and minimize what you don;t want all the while not create

Let’s get a few things out of the way

• Groups of people are never free of power and dominating behaviours. It doesn’t matter if you are using a well conducted circle process or a self-organizing process, or placing limits on who can speak and who cannot. It is impossible to build a group process that is free from these behaviours. So the challenge is to mitigate them.
• In truly participatory processes, observers are indeed influential. Have you ever been somewhere and there are people there not participating, just watching from the sides silently and taking notes? Does it feel like this kind of set up lessens power in any way or builds trust?
• If you are consulting because you don’t know the answer to a question, being absent from the conversations does not help you learn. The trickiest challenges we face aren’t solved by listening quietly to someone else in the hopes that they will provide you the answer you are looking for. They are addressed by diving in together and looking for ways to tackle problems in new ways.

If you are facing a truly sticky issue and you have no answers, getting as many people as possible fully engaged in exploring it is critical. So here are a few bits of advice I find myself giving out time after time, in no particular order.

Use a process like Open Space or World Cafe that allows participants to set their own agendas.  These processes, and many others, place the onus of discovery, creativity and action on the participants. They operate from the assumptions that the ways forward are there to be discovered together, from the creative spaces between people. Furthermore they are founded on good dialogic principles, which you can point to and practice, such as, speak from your experience, listen to learn and be aware of your impact. Inviting a group into these practices helps them focus on each other as as potential experts.

Use small groups and break them up.  I’ve never understood the aversion to small groups, but trust me when I say that you can do very little rapid creative work in groups larger than five. If you want to learn more about my approach to group sizes, here’s a post summing up what I know, and here’s a quick video my friend Nancy White made. Making and breaking up small groups is an important complex facilitation technique that allows for people to create without getting entrained and therefore sinking into domination patterns are or other kinds of bias.

Trust your people.  There is an undercurrent to the base worry that clients share with me, and it’s worth addressing with them. I find that when we probe deeper, we discover that often the client has a deeper issue about either trusting their own people to behave well, or trusting a group of “lesser powered” folks to be resilient enough to speak. This is actually easily remedied by designing the session well, but it sometimes helps to have an offline conversation about the way the client feels about participants.

Have truly open questions. If you want your meeting to be truly participatory and engaging, you have to ask a group a questions you are stuck on. The questions need to be open and honest, and the group you assemble needs to be the people best suited to explore the question and create actions around it. Never bring a pre-determined answer to a participatory process, and give people the illusion that they are creating something new together. It’s unethical. Beyond that, truly open questions make it easy to encourage people to listen to one another and they de-centre expertise, meaning that the group itself can truly become the experts. If we can separate those in power from those with answers, we get a truly rich dialogue and learning experience.

Commit to supporting what you start. In my practice of chaordic design, I call this the Architecture of Implementation. You have to know what you are willing to commit to ensure that whatever happens at the meeting will have an effect. This doesn’t always mean money. It could also mean that time, space, power, connections, and many other resources can be put at the behest of the group to move to action. It could also be that you let people know that “nothing will come of this meeting beyond the learning that happens in the meeting itself. It doesn’t matter to me what the architecture is, but it does matter to the group. Being honest helps people to show up in a trusting way, and helps them to know how much time and energy to spend on your initiative.

Invite authentically.  If you have designed with all of the above in mind, you can authentically invite the right people to your gathering with very little fear that there will be catastrophic domination. And authentic invitation brings people into the room ready to work on a problem that they are needed for. That is a powerful call.

I’m sure lots of experienced facilitators out there have other wisdom to add about how to address this concern. What have you got to add?

In the latest episode of The Omni Show, I am the guest. How does this even work? Do I interview myself? What’s going on here?

PS I like being on podcasts, and I’d be happy to be on yours, to talk about Omni stuff or my own stuff or both. I could probably even convince Ken Case to be a guest on your show if you want to talk about Omni apps and developing for Mac and iOS in 2019.

(My email address is no secret, and every spammer everywhere already has it, so I’ll just post it here: it’s brent@ranchero.com.)

NYT4, the fourth and longest living version of nytimes.com died in March, 2018 after serving New York Times content for 17 years.

NYT4 is survived by Jeff Damens and Oliver Karlin, the engineers who invented Context, a programming language that powered the site, and dozens of other engineers who worked on the site over the years.

Context was born in 2001 in the office of Scott Meyer, then general manager of nytimes.com. Meyer wanted to make the New York Times website more personal by adding readers’ usernames to the top of the homepage. At the time, nytimes.com was a static site that could not be customized for each person, but Meyer wanted that to change.

Damens and Karlin could have hacked something together just to add the username. They could have used PHP and CGI, which were common technologies in 2001. But, as Karlin pointed out, “You didn’t want to CGI-up the homepage to put one dynamic text in there.”

The developers had concerns that CGI wouldn’t scale to handle traffic on the homepage, which even then was growing quickly. They were also skeptical that Meyer’s vision for personalizing nytimes.com would end with the addition of a username.

Instead, they created Context, a lightweight language that was compiled to run on The New York Times’s servers and optimized for speed. Damens said they were able to get speeds a hundred times faster than CGI in early tests.

But they didn’t stop at creating a language.

Anticipating that editors would one day want to add more dynamic sections to the site, Damens and Karlin designed context to support newsroom workflows.

Editors were familiar with adding tags to their page layouts to change elements, such as the weather, for different regional editions. Rather than asking editors to learn a programming language, the developers tied the language into the macros the editors used to send formatting instructions to the printing press.

What started as a request for a username turned into a programming language, a compiler, a pre-processor that scanned every page on the site and converted it into a context program, and a virtual machine to run those context programs on The Times’s servers.

“We probably over-engineered it, I guess,” Damens said.

But Context wasn’t just an elaborate username-printing system for long. Soon, the system would be used to add a weather widget to the homepage. Then it was used to streamline the process for publishing movie showtimes online. Within a few months, a Context program was put into service to paginate articles, which was a major driver of page views. Context quickly became the technology powering the bulk of nytimes.com.

While it had a reputation for being fast and reliable, Context was also controversial. Engineers bristled at having to learn a programming language only used at The Times. Only a few learned enough to help add new features or extend its abilities. Recruiting new engineers was sometimes a challenge because candidates had to accept that they were joining an organization with a homegrown programming language and build system. Looking back, Damens and Karlin regret not picking an existing syntax of a language, like PHP, to make it easier to adopt.

Context probably would have been phased out quickly were it not for another personalization challenge that required its speed. This time the problem was ads. Later in 2001, Zip-code targeted ads were slowing down the site and by this time Context had credibility as a fast and reliable solution. Damens and Karlin were called in to help.

They used Context to create a fast ad server called AdX, which used regular expressions to match ad slots according to targeting parameters for each ad campaign. The regular expressions ran against metadata in the page (such as a reader’s zip code and what section of the paper they were visiting) to target ads to specific users. AdX also kept track of how frequently ads were shown so it could manage overall ad inventory.

All of this was able to run on the existing servers of The Times.

By the end of 2001, Context was powering nearly all pages and advertising for The Times. It went on to serve more than a decade in this capacity, seeing the transition from a supplement to the printed newspaper, to the way most readers consume The Times. It survived traffic surges during elections and major news events. AdX expanded to serve increasingly sophisticated targeting, including marketing messages on the site.

With all that Context did at The Times, it never ventured outside the walls of the building; it was never open sourced or sold. The years that Context was the most creative and powerful as a system were long before The Times had a culture that embraced open source.

But perhaps that isolation helped keep it alive until earlier last year. With only a handful of contributors, the language was spared being pulled in different design directions. Fewer than a dozen engineers knew how the build system worked. Maybe a hundred engineers learned to ever write Context code. Without a large community of engineers, Context was spared from the drag that new features can add to a system, which meant it stayed small, nimble and able to scale to meet the growing traffic of The Times for a decade after its creation.

As the last files of nytimes.com were rewritten using newer technologies, Context disappeared entirely from The Times. While the language could have grown into something larger had it been open sourced, it instead served to help The Times find its footing on the internet. Damens and Karlin were right: The Times news offerings have expanded from a static website to include apps and more personalized features.

But one thing has stayed the same: readers can still find a link to their username on the homepage.

---

Remembering a Programming Language that Helped Shape the Digital New York Times was originally published in Times Open on Medium, where people are continuing the conversation by highlighting and responding to this story.

Photo by freestocks.org on Pexels.com

The following is from an email that Rick sent around to members of TransportAction BC this week and is presented here for your information and in the hopes that we can attract a few more members locally.

Ride Hailing:

Another report has linked declining US transit ridership to ride hailing services. University of Kentucky researchers state that observed ridership declines are not just the results of service reductions, fare increases and cheaper gas. The engineers used a random-effects model accounting for 10 different variables. They conclude that transit ridership declines by 1.3% (heavy rail) and 1.7% (bus) for each year after Transportation Network Companies (ride hailing) enter a market. Potential solutions to the decline will require more that just increasing service, an expensive option not feasible for many cash-strapped agencies. Congestion pricing, re-allocation of street space to sustainable transportation modes and transit prioritisation are potential options for policy makers’ consideration.

An overview of the research is at https://www.businessinsider.com/uber-lyft-having-devastating-effect-on-public-transportation-study-2019-1 and the full report is at http://usa.streetsblog.org/wp-content/uploads/sites/5/2019/01/19-04931-Transit-Trends.pdf.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

    City of Vancouver staff presented a report to Council (https://council.vancouver.ca/20190115/documents/rr1.pdf) on implementation principles and issues/concerns related to ride hailing service in Vancouver. The report is a response to the province’s request for input into its proposed ride hailing legislation. The provincial request limits input to 4 areas:  1) Criteria for establishing boundaries; 2) How to balance supply of service with consumer demand; 3) Criteria for establishing a price and fare regime; and 4) Driver’s licence class requirement. The City has established 6 principles to guide its input to the province: 1) Regional co-ordination; 2) Passenger safety; 3) Enhanced mobility; 4 ) Enhanced accessibility; 5) Reduced GHG emissions; and 6) Economic viability.

Within this framework, the City has concerns with the proposed provincial legislation. The legislation changes the Vancouver Charter to: 1) prevent the City from limiting the number of ride hailing vehicles operating in the City; 2) prevent the City from banning ride hailing services; and 3) prevent the City from setting ride hailing charges. Also of concern is that the legislation does not require the PTB (provincial ride hailing regulator) to consider regional issues in determining how many ride hailing licences it issue; removes the City’s ability to manage congestion arising from ride hailing services; and municipalities are not given access to trip data the PTB will collect from ride hailing services.

The City will bring these concerns forward in ongoing discussion with the province. Included in the discussion will be future opportunities to introduce congestion or per-trip fees to manage congestion and other municipal concerns such as curbside traffic management.

The Vancouver Sun‘s story (https://vancouversun.com/news/politics/dan-fumano-vancouver-wants-to-charge-uber-and-lyft-users-a-congestion-fee) on the report emphasized the possibility of congestion fees and discussed how other cities have dealt with charges to ride hailing services. The Uber/Lyft supported Ridesharing Now for BC group is not on board, though, stating “the no. 1 topic … is affordability”, although this is not mentioned on its web site. An Uber spokesman also brought up the affordability concern, which is disingenuous, given its surge pricing policies..

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Island Corridor Rail Service:

Capital Region mayors (https://www.timescolonist.com/news/local/1.23626082)  asked the provincial government  get passenger rail service back on the E&N between Langford and Victoria. They have also requested funding for rapid bus lanes along Highway 1. The Mayors tout the benefits of congestion relief, reduced GHG emissions, economic development and more time with families (Victoria Mayor Lisa Helps).

Transport Minister Trevina stated that safety and seismic assessments are necessary to ensure current standards can be met. Apparently this work is being planned  (https://www.timescolonist.com/news/local/island-corridor-foundation-e-n-rail-line-is-not-dead-yet-1.23570436?fbclid=IwAR3Iy-dkI5vmDknBZmEXWCyHirSCYN1v82Km9uCCcPiA1NY98MxGvSurl_s) and Minister Trevina expects it to take six months.

However, the previous Liberal government also made this promise and there appears to be little to show for it.  And no mention of who would be performing the assessment – an independent agency or the BC Safety Authority, which was accused of having a too close relationship with Southern Rail (2018-11-17 TABC:FYI:http://focusonline.ca/node/808?fbclid=IwAR1Y0UInpk_8a92mJmmOWIl8yeJBJJvRdNMpfD33Uv3Q3F1X8Nu_DtY6T5I)

Trevina also said First Nations issues are a crucial part of any decision on bringing service back to the E&N.

She did not address the Mayors’ suggested Highway 1 bus lanes but some Mayors pointed out that a recently opened 2.3 Km bus lane on Douglas Street in Victoria was saving peak hour transit users 10 minutes per trip.

As an a side, the Douglas Street bus lane project was delayed for several years because of cost issues (https://www.cbc.ca/news/canada/british-columbia/victoria-bus-lanes-delayed-1.4169563 ) but finally opened in Nov. 2018. Of interest to Vancouver transit users is the fact that segments of the bus lanes are 24/7 (https://bctransit.com/victoria/transit-future/victoria-bus-lane-douglas-hwy-1), unlike most of Vancouver’s transit only lanes.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

CN Rail Vancouver Double-tracking:

The federal government has given CN Rail permission to double-track its line from the Vancouver waterfront, through Strathcona and the Grandview Cut, to Nanaimo Street. No time line is given for the project but CN increased traffic on the route 2 years ago. The traffic increase causes delays at level crossings and has led to noise and pollution complaints. However, those issues seem far more serious on Clark Drive, which parallels part of the rail route, with its heavy and constant truck traffic.

https://vancouversun.com/news/local-news/rail-expansion-through-port-of-vancouver-aimed-at-hiking-imports-from-asia?utm_source=Sailthru&utm_medium=email&utm_campaign=Vancouver%20Sun%20Daily%20Headlines%202019-02-17&utm_term=VS_HeadlineNews

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Autonomous / Driverless Vehicles:

An opinion piece in the Globe and Mail argues that as vehicles become increasingly automated, we need to start considering the impact of this automation on drivers’ abilities and how driving skills will atrophy with increasing automation. How do we test and license vehicle operators in a world where most are tested once at a young age and are never tested again until old age sets in?

He uses the example of aircraft where much of a flight’s operation is handled by autopilot. The pilot’s abilities are increasingly marginal to to a flight, until something goes wrong. At that point the pilot’s skills are absolutely critical for a safe resolution of the incident.. The author calls this “the paradox of automation” – as a system becomes more automated and human input decreases, the importance of the human input increases – when it is needed.

Similarly, for vehicle automation, a driver’s abilities are not constantly being tested and improved by vehicle operation, as they are by non-automated vehicle operation. When a problem occurs in a [semi-]automated vehicle, the driver must be fully alert and cognizant of what efforts are needed to resolve the problem. How can society ensure that drivers maintain the skills needed for emergency situations?

The author’s solution is to consider airline pilot training methodologies for vehicle drivers. Regular simulation testing and licensing based on successful completion of the simulations. Failing a simulation means losing one’s license.

https://www.theglobeandmail.com/opinion/article-the-long-dangerous-road-to-the-world-of-driverless-cars/

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

---

---

Thank you Chairman Chau and committee members.

My name is Todd Weaver, and I think you’ll find I’m an unusual witness here today, while I may be sitting side-by-side with impressive privacy protection groups, I am here as the CEO of a rapidly growing technology company based in California.

I am here calling for much stronger consumer privacy protections – starting with giving consumers the power to opt IN rather than opt OUT of sharing their personal data.

I am here to tell you it’s time for California’s extraordinary tech industry to stop harvesting and “sharing” our most personal private data without our meaningful consent and knowledge.

I am not here to tell you AB 375 (or stronger) protections are tough to implement, history is filled with wrongdoers complaining that doing right will put them out of business only to comply and thrive later. Incidentally, this same tech industry complained about Europe’s GDPR that certainly did not put them out of business.

I am here to tell you the new law (or stronger) is easy to technically comply with – if we companies simply begin to honor our customer’s privacy rights and design our services to be privacy-protecting rather than privacy-exploiting.

I started Purism, when I came to realize that my two young daughters, like all children, need convenient products and services that protect them, rather than exploit them.

As a technologist, I understand painfully well how much the technology sector can exploit my kids with ease.

That’s why I started this social purpose company founded on privacy-protection principles. Purism is already manufacturing in California and assembling these laptops shown right here, including the operating system, applications, and bundled services.

We will also this year be manufacturing a privacy-designed phone with bundled services that comply with AB 375 and go even further with opt-in by default for all offered services.

We have been growing by triple digits annually, reflecting the huge built up consumer demand from parents and professionals and enterprises just like you who simply want to keep their and their children’s lives private and secure.

This is done by a simple approach: privacy by design.

As AB 375 seeks to make clear, privacy is a right, and your every location and every communication and every webpage and every search stored permanently should not be exploited to use needed services online.

I strongly suggest the time has come for Californians to take back their constitutional right of privacy on the Internet, and urge you to substantially strengthen the privacy protections afforded by AB 375.

For your and my childrens’ sake, Thank you.

The post Purism’s CEO Todd Weaver Testifies at California Congressional Privacy Commission appeared first on Purism.

Highlights

• The need for speed! Using some handy scripts from Florian, mconley was able to do some frame recordings on the 2018 reference hardware to measure start-up performance.
  • Firefox Nightly is running in the 4 frames on the left, Google Chrome is running in the 4 on the right.
  • Firefox Nightly is consistently faster to get to first paint, and faster to paint the browser toolbars.
  • The primary deficit appears to be presenting about:home, which we’ll be focusing on next.

• Hot on the heels of the last update, there’s a new Firefox Front-end Performance Update. Read all about it!
• Touchbar support has been enabled in Nightly!

• The team behind Firefox Lockbox, our password manager for Android and iOS, is working with the password manager team on a WebExtension for Firefox Desktop.
• A new version of Firefox Screenshots landed in Nightly. This version disables screenshot uploads, and has migration instructions if you’ve uploaded screenshots before. There’s also a new Ctrl + Shift + S keyboard shortcut.
• We’ve landed lots of Dev Tools improvements!
  • Logpoints landed in the Debugger, letting you add and edit console logs everywhere, without ever touching the files – thanks to Bomsy!
  •  📣 Call to Action: ✨ Use All The Breakpoints ✨ in the new debugger. Keep an eye out for issues. If you see something, say something.
• The first Project Fission Newsletter is out!

Friends of the Firefox team

Fixed more than one bug

• Abdoulaye O. LY
• Avery Berninger
• Special shout-out to Manish [:manishkk] for fixing 10 bugs in the last two weeks! 🎉
• Martin Koroknay
• Oriol Brufau [:Oriol]
• Shivam Singhal [ :championshuttler ]
• Tim Nguyen :ntim

New contributors

• 🌟 Avery Berninger migrated about:tabcrashed to Fluent, and also updated browser_all_files_referenced.js so that it also looks for Fluent .ftl files!
• 🌟 Brian Chen migrated the “reset profile” strings and about:rights to Fluent, and then made browser_misused_characters_in_strings.js check Fluent files.
• edward.i.wu made it possible to theme selection text and background colours from a WebExtension theme.
• Garvit Khatri [:garvitdelhi] fixed a styling glitch in our DevTools Font Inspector.
• 🌟 Martin Koroknay fixed a bug where add-ons could get stuck checking for updates, and made it so that we always show the filter toolbar buttons for the console.
• 🌟 Sakshaat Choyikandi improved our DevTools documentation.
• 🌟 SresthaSrivastava [:srestha] removed some unused attributes from aboutNetError.xhtml.
• 🌟 Varun [:vdey he/him IST] cleaned up some of our Telemetry tests.

Project Updates

Add-ons / Web Extensions

• Support for update.rdf was removed. 🎉 🎉 🎉 🎉
• Themes got the ability to handle selection colors.
• Moving an unloaded tab no longer loads it.
• Work on extensions in private browsing mode continues.
• The first bits of a new about:addons written in HTML landed.

Applications

Screenshots

• A grab bag of improvements!
  • Show the context menu when there’s a text selection.
  • Enable pageAction in Reader Mode.
  • Add additional requestIdleCallback wait around migration server check.
  • Metrics improvements:
    • Stop sending metrics if the server isn’t available.
    • Add logic to not send timing information if the server is unavailable.
    • Only send analytics for 10% of users.
  • Make pressing the “Enter” key download a shot.
  • Use incognito: "spanning" in the extension manifest.
  • Update first slide text to remove references to uploading.
  • Query user status from server.
  • Remove My Shots links for non-server-users.
  • Pop open server page for people with indefinite shots.
  • Remove My Shots test, since My Shots is no longer available.

Lockbox

• This past sprint covered foundational work on the WebExtension:
  • lorchard made it so that we disable prompts to save logins while on extension pages.
  • meandave updated the styles in the doorhanger list view.
  • lorchard added integration tests for item deletion & modification.
  • 6a68 made it so that we filter out Sync logins in event handlers.
  • 6a68 made it so that we derive entry titles from saved login URL.
  • Relequestual updated the Readme.

Services (Firefox Accounts / Sync / Push)

• We’re continuing to port Places to Rust for Fenix and Firefox for iOS!
  • Thom is looking at optimizing our frecency algorithm and history expiration. 🕒
  • Thom also has a proposal for syncing key-value shaped data in a generic way. 📦
  • History sync is largely complete (and ready to test in the Android Reference Browser!), with new APIs for clearing history landing soon. 🕵️‍♀️
  • Mark landed an initial cut of the bookmarks schema. We’ll be implementing syncing and merging next. 📚
  • Our bookmark sync merging algorithm lives in a separate Rust crate now, called Dogear. 🐶
  • Lina integrated Dogear into Desktop, using the Rust XPCOM bindings. 🦀
• Edouard implemented Web Push crypto in Rust, to support Send Tab (and, eventually, Web Push!) in Android Components. 🔐
• JR and Phil are working on a Push component for Rust. 📣

Browser Architecture

• We now have more Fluent strings than DTDs!

• Documents in system add-ons can use Fluent now.
• The Fluent XUL cache, which will allow us to use Fluent on the startup path, is in review.

Developer Tools

• The new scrollable badge got enabled on Nightly by Patrick Brosset, letting you quickly find the element causing unwanted scrollbars.

• The Debugger team is working across many fronts to provide you the Breakpoints quality that you deserve!
  • Brian Hackett is cleaning up the Debugger’s sources/script handling, which not only fixes long-standing issues with inline, console, evaled & gc’d scripts but also sets the foundation for shipping “windowless” worker debugging.
  • Landing: Logan Smyth unified the pause location logic in the Debugger Server & JS Engine. Expect rock solid & fast pausing/stepping!
    • Sneak preview: Having proper locations support will also unblock the new Column Breakpoints feature and Pretty Print improvements.
• Razvan landed the Copy support for the new Changes panel, so you can easily export the style modifications done the Inspector to share or apply in your IDE.

• UX Contributor Florens Verschelde continues his icon update work with more and more batches.
• Nicolas making it easier to copy and share traces from the Console by improving the copied formatting
• Micah added a Copy Stylesheet URL to the Style Editor to make compat debugging easier.
• Contributor Anthony X. added syntax highlighting to the Debugger’s conditional breakpoint input.
• Contributor Jarim added the method filter to XHR Breakpoints.

Fission

• Fission now has milestones in Bugzilla!
  • Milestone 1 (M1)
    • Render an OOP iframe
    • Targeted for end of February
  • Milestone 2 (M2)
    • Interactive OOP iframes
    • Targeting for +1 or 2 months, still TBD
  • Milestone 3 (M3)
    • Still being formulated
  • Please nominate bugs with Fission Milestone -> M1/M2/M3 if you think it applies to Fission
• Groundwork for frame script replacement is landing at a fast pace.
• Some docs for the front-end are on the wiki.

Lint

• Standard8 is investigating centralising the configuration for test directories so that we don’t need a .eslintrc.js in every test directory.
• ESLint is discussing a big change to how they manage configurations.

Password Manager

• MattN reports that project areas are coming together and should be prioritized by the end of the week! 🔐🙈👍👍
• Completed
  • The whole team has landed a lot of test fixes to prepare for new development.
  • prathiksha added an Allow/Disallow autofill password option in the management UI.
  • prathiksha also made it so that we now provide visual feedback when a password field is autofilled, bringing login fields in line with autofilled address form fields.
  • sfoster made it so that you now have the option to save passwords in Private Browsing mode.
• In Progress
  • sfoster is working on delaying autofill on background tabs until selected.
  • prathiksha is adding a “View Saved Logins” footer to the password manager autocomplete popup.
  • jaws is fixing a bug where logins imported from Chrome (on Windows) have the wrong character encoding.
  • MattN is fixing a bug where the selection of previously entered form history (and username) data is broken in Nightly when Form Autofill is enabled.

Performance

• dthayer’s research into a warm-up service seems to indicate that we can perhaps shave off 1s of net time required to start Firefox from cold OS boot.
  • esmyth is running a Heartbeat survey to get some user sentiment on the various trade-offs here.
• dthayer is going to collect Telemetry on startup cache hits and misses.
• felipe’s new tab animations have been baking in Nightly behind the browser.tabs.newanimations pref, and plawless has been working with UX to free up epang to help figure out the remaining animations
• felipe has also resurrected an older patch to make creating the ever-mysterious Hidden Window lazy to load on Windows and Linux.
  • This has resulted in some nice clean-ups, since felipe has discovered quite a bit of unused code.
• Gijs landed a patch to make it easier to experiment with the low-end hardware browser adjustment on beta. He’s going to try shifting the technique to lower the frame rate by having the RefreshDriver skip every other vsync, rather than lowering the vsync rate.
  • We believe this will allow scrolling and video to still play at 60fps, but that painting of web and chrome content will happen less frequently per second.
• Gijs also made it so that we don’t invalidate the start-up cache when installing or uninstalling a WebExtension
• mconley landed a probe that measures the time from process start to painting the top sites in the initial tab’s about:home
• mconley enabled the process priority manager by default on Windows on Nightly.
  • Got our first fallout report, and just today landed some patches to increase the priority of background tabs playing audio.

Policy Engine

• New policies for turning off captive portal detection via CaptivePortal, disabling prefetch via NetworkPrediction, and toggling ExtensionUpdate.
• Making progress on the extension allowlist and fixing blocking add-on bugs along the way.
• Started investigating the Chrome Legacy Browser Extension.

Search and Navigation

Bookmarks & History

• Marco fixed an issue with favicons where we would lose them in some circumstances if you clear history on shutdown.
• Mark fixed an issue where importing bookmarks could unnecessarily fail earlier if invalid entries resulted in an empty section.

Search

• Mike de Boer landed a patch to make the Search Service load asynchronously, to prepare for loading WebExtension-based engines.

Quantum Bar

• Lots of bug fixes and code cleanups, list too long.
• Search one-off buttons have been added to the display, but currently can’t be selected/clicked on.
• Now ported the majority of tests in the urlbar/ directory to work with QuantumBar. Likely to start on those outside the urlbar/ directory soon.
• Working on getting the list of bugs remaining for Nightly down, will likely be looking at starting wider testing soon.

User Experience

• After the implementation of Contextual Feature Recommender (CFR) for Extensions, the goal is to extend recommendations to include Firefox features, like Pinned Tabs. For example, if you open multiple tabs and repeatedly use these tabs, we may offer you a feature called “Pin Tabs” and explain how it works. Firefox curates the suggested features and notifies you in an appropriate context.
  • After several rounds of user testing, we’ve finalized the design and will run CFR Pinned Tab in a shield study. Engineering work is scheduled for 67.
  • CFR Pinned Tabs Bug
  • CFR – Pinned Tabs Spec

There is a cute little research trick that semi-clever operators use to con gullible rubes. I will give you a small, silly example of it which I hope will make it more understandable on a large, global scale. It goes like this.

Let's say you want to open a strip club in a residential neighborhood. Obviously, no one in the community in their right mind wants a strip club in their neighborhood. But as the potential owner of the strip club you have to make a case to the city council to try to get your permit.

You do a survey in your community. What you don't ask is a clear, direct question, "Do you want a strip club in your community?" because you'll get a resounding no and a few solid blows to the golden globes.

Instead, you ask a question that sounds kinda like a suitable question: "Do you think the residents of Smallville would benefit from more recreational and entertainment opportunities?" This question has a lot of benefits.

• Who is going to say no to the vague notion of "more recreational and entertainment opportunities?"
• The so-called "recreational and entertainment opportunities" are not defined
• The social ramifications (cost/benefit relationship) of the so-called "recreational and entertainment opportunities" are not described

Once the survey is completed you go to the city council and show them your pitch slides:

• 88% of people in our community are in favor of "more recreational and entertainment opportunities." That's what we provide!
• If approved, revenue from our company will contribute over $1 million annually to the tax base in the community.
• We understand that not everyone will be in favor of our business, but enjoying our shows is entirely voluntary and no one is forced to patronize our establishment.

Even a city council isn't dumb enough to swallow this bullshit. Even a city council isn't dumb enough to not understand when they're being conned. That's how they're different from us.

I would submit to you that this is exactly the type of specious rationale that underpins the entire online ad industry. The con goes like this: the reason that tracking and spyware are necessary is that consumers want "more relevant advertising." This claim is put forth virtually every time the spy masters are asked to justify their practices.

To quote a semi-clever operator named Zuckerberg, “People consistently tell us that if they’re going to see ads, they want them to be relevant.”

Yeah, right. People are out in the streets marching for more relevant advertising.

A recent New York Times piece by a communications professor and a law professor exposed this bullshit for what it is. They reported on two large studies they did. Here are some of the results...

"Sixty-one percent of respondents said no, they did not want tailored ads for products and services, 56 percent said no to tailored news, 86 percent said no to tailored political ads, and 46 percent said no to tailored discounts. But when we added in the results of the second set of questions about tracking people (emphasis mine - BH) on that firm’s website, other websites and offline, the percentage that in the end decided they didn’t want tailoring ranged from 89 percent to 93 percent with political ads, 68 percent to 84 percent for commercial ads, 53 percent to 77 percent for discounts, and 64 percent to 83 percent for news."

By posing questions in manipulative ways that don't actually describe the issues in question, it is possible to use research to distort the truth. If you ask someone "do you prefer ads that are relevant?" of course they're going to say yes. Just like if you ask if they want more entertainment opportunities.

But if you're asking the appropriate question -- "Are you willing to trade private, personal information about yourself and your family, and have your movements tracked and catalogued both online and offline, and have your emails and texts read and archived, and have files about you sold to anyone who wants to buy them, in order to get more relevant advertising?"-- I don't think you need to be a Harvard-billionaire-semi-clever-operator to know that you better be wearing a cup.

Many consumer items have specific labelling requirements associated with them: children’s products, food, tobacco products, for example.

Manufactured products, including but in now way limited to toys, electrical products nad telecommunications equipment, also need to be safe (Product safety for manufacturers).

The labels typically identify things that can cause harm or act as a way to allow the consumer to manage risk. Labels may also be used to try to regulate a means of production through the market (e..g certification schemes, “free range eggs”, etc).

So do our new electronic devices also need to labelled with the environmental sensors they incorporate based on the potential for privacy breaking harms?

This post is a stub for examples where such sensors have not been clearly identified (please let me know via the comments if you come across further examples):

• Nest Secure had a secret microphone, can now be a Google Assistant

So what sensors should be identified?

• cameras
• microphones

Obviously…

Location sensing would be harder to label, because this may be done by the device itself (eg GPS), with help from other services (eg looking up location by cell tower or wifi hotspot localisation), or identified from your device by a remote service (eg IP address based localisation).

So maybe radios should also be clearly identified on the label (i.e. any wireless means by which a device can connect to a communications network). (Radios already have to comply with regulations around electromagnetic interference.)

Is this something the Lords Communications Committee, perhaps, or the Commons Science and Technology Committee has looked at, perhaps?

Tangentially related: Surveillance Camera Commissioner Guidance on the use of domestic CCTV on the one hand, and domestic video surveillance on the other.

Another thought: what starts out as spyware, eg USB cable with hidden microphone may also end up being commoditised as security devices: light bulbs with cameras/microphones for example; then just everyday: of course your music speaker needs to have a microphone in it, and your telly must have a microphone AND a camera, because, well, obvs…

PS  an example UK privacy opinions survey.

PPS internet harms, eg fake news? House of Commons Digital, Culture, Media and Sport Committee — Disinformation and ‘fake news’: Final Report

You know Henry David Thoreau, author, transcendentalist, author of Walden Pond, a celebrant of the simple life lived in nature? He seems an unlikely candidate for a technophile, but often the least likely among us are susceptible to the allure of technology. His diary entries in 1851 present quite a poetic view of the newest technology to come to New England: the telegraph:

1851, Sept. 3.  As I went under the new telegraph wire, I heard it vibrating like a harp high overhead. It was as the sound of a far-off glorious life, a supernal life, which came down to us and vibrated in the lattice-work of this life of ours.

1851, Sept. 22. I put my ear to one of the posts and it seemed to me as if every pore of the wood was filled with music, labored with the strain–as if every fibre was affected and being seasoned or timed, rearranged according to a new and more harmonious law. Every cell and change or inflection of the tree pervaded and seemed to proceed from the wood, the divine tree or wood! How much the ancients would have made of it! To have a harp on so great a scale, girdling the very earth, and played on by the winds of every latitude and longitude, and that were, as it were, a manifest blessing from heaven on a work of man’s! Shall we not add a tenth muse to the immortal Nine? And that the invention this divinely honored and distinguished–on which the Muse has condescended to smile–is this magic medium of communication for mankind!

I felt the same way about the internet when I first encountered it–a magic medium of communication for humankind! It’s often difficult for us to “see” this kind of magic anymore because we now know where it has ended up. Power lines, telephone lines–these are not a thing of great beauty, to us. They don’t look like harps to us. This has been beautifully illustrated by Robert Crumb in a drawing titled “A Short History of America”:

Thoreau lived in frame 3, and we live in frame 12. We can see beauty and nature disappearing, and see that, maybe it would have been better to put those lines underground. And that it is up to us to make frames 13, 14, 15. Can we improve it? What will frame 24 look like? 

Not only the built environment, but the inner life has been changed by what Thoreau sees as the  “magic medium of communication for mankind”. This is what I first loved about the internet:  it connected us to each other. We love to connect!

But we’re not communicating any more. We went past Dunbar’s number, beyond the number of people we can meaningfully know, which makes our relationships brittle and thin. Fake news, platitudes, bias, and not seeing our friends anymore– just reading their updates–is what it’s come to. 

This passage from Thoreau tells me three things: One, we should not forget the wonder of being able to communicate with one another across great distances. All the wonders of the internet are still there: we should see it again with it’s magic. Two: we should pay attention to the past to learn for the present. And three, living as we do in Thoreau’s future, where we can see the future outcomes of those telegraph wires, we should think deeply about the future we ourselves are creating and guide it to a better, more beautiful, future.

---

 

 Walden and Civil Disobedience by Henry David Thoreau. Many people now say that Thoreau’s stay at Walden Pond was more like an early example of performance art then any commitment to living permanently in simplicity.  Still, the virtues of living simply resonate in our overstimulated, trivia-filled lives. And this edition includes Civil Disobedience, his great ode to freedom, which inspired non-violent protest everywhere is a must-read for all of us. Libertarians and liberals alike have marched beneath its banner, and the fact that it can encompass so many diverse viewpoints is a testament to its depth and power.

America by Robert Crumb. Have Thoreau and Crumb ever shared a page? This may be a first.  Both are deeply American. Thoreau is easy for me to like, but I have a love-hate relationship with Robert Crumb. If you haven’t seen the fantastic Terry Zwigoff documentary about him, Crumb, you must, and it will help you understand where he’s coming from. But I have to work hard to get past the pornography, misogyny, racism and scab-picking ugliness of all he does, in order to appreciate the great things he’d done, like that comic above, and his nasty (NSFW) 1989 comic about Donald Trump.

---

 

 

As Benjamin Franklin once said (probably), software users who trade privacy for functionality deserve neither.

Authors love to complain. (Well, everybody loves to complain, but the complaints of authors are more erudite and literate.) From my post as editor and coach, I get to hear it all. And I take your whines seriously, friends. So here’s a call-and-response that may help you cope. I cannot believe how much work it is to be an author. This is so hard. Art is hard. Being a smartass … Continued

The post In response to the finest whines of nonfiction book authors appeared first on without bullshit.

White power assholes are not exactly the smartest people on the Internet. State-sponsored manipulation operations have better skills and can use the domestic guys as human sock puppets.

Pedophiles aren't the smartest people on the Internet either. Even the "highly technical dark web" pedo networks are using off-the-shelf tricks that are far behind what even the most basic adfraud operation can pull off.

So I'm writing to explain why I'm going to move your long-form think piece about the "power" of the Internet "duopoly" to the probably-never-going-to-get-to-it end of my to-do list. Let's have a look at just the last 24 hours:

• Google is getting its ass kicked by pedophiles: On YouTube, a network of paedophiles is hiding in plain sight | WIRED UK

• Facebook is getting its ass kicked by white power assholes: Facebook decided which users are interested in Nazis — and let advertisers target them directly - Los Angeles Times

(Come back in 24 hours for more.) You're asking me to be interested in reading your ever-so-thoughtful essay about the awesome power of two companies that have to be like the fifth most influential people on the Internet, max. Even the Facebook ad integrity guy is down to asking for free reports of scam ads, on Twitter.Dude, if the "get everybody else to do your QA for you" strategy worked, then we'd all be running desktop Linux.

The "powerful platforms" are a box on the Internet cable between terrible marketing decisions on one end and criminals and terrorists on the other. A box maintained by vaguely creepy but not especially interesting IT staff. Yes, let's write about CMOs who attach their brands to heinous shit—what's up with that? Yes, let's write about the criminals who end up with the money—that can't be good. But the companies in the middle are not the story.

Bonus links

Oracle Data Cloud Companies Expose ‘DrainerBot’ App Fraud Scheme

Disney reportedly pulls ads from YouTube following child exploitation controversy

Google says the built-in microphone it never told Nest users about was 'never supposed to be a secret' (GOOG, GOOGL)

Disinformation and ‘fake news’: Final Report published - News from Parliament - UK Parliament

Legit Brand Creative Is Getting Hijacked – And Advertisers Need To Start Paying Attention

Culture secretary commissions study into 'opaque' world of online advertising

If Facebook wants to stop the spread of anti-vaxxers, it could start by not taking their ad dollars

Leslie Roberts uses paint to encode text as colors and geometry:

My paintings translate words into visual language. These panels with texts and accompanying abstract structures might be called illuminated manuscripts of the everyday.

Written in these recent paintings are collections of ambient found language: fragments from street signs, junk mail, end user licensing agreements, email, labels, subway ads, receipts, newspapers, and instruction manuals. Transcripts of fine print from the relentless flow of information surrounding us are used to derive a personal abstract vernacular.

Tags: paint, text

"How do I make my data come alive? I want it to sing. I want it to dance." Here are some ways to achieve that. Read More

If you want a basic light bulb that looks great and is clearly cheaper long term, there is no reason not to choose LED bulbs—they look as good as incandescents and are far more energy efficient. We recommend Cree’s Exceptional Light Quality line of A19 bulbs. For general-purpose lighting, look for soft white LED bulbs, available in 60 W and 40 W versions, and for a whiter, cooler light for working, get Cree’s daylight LED bulbs, also in 60 W and 40 W versions. The Cree bulbs also work great with dimmer switches, neither flickering nor buzzing, and their good color accuracy brings out the best in your decor, furniture, and food, improving the ambiance of your entire home.

Keeping up with our promise of 2 years of Android updates, Nokia 6 (2017) is now officially running on Android 9, Pie 💪! Nokia phones get smarter over time. pic.twitter.com/50aLqColh7

— Juho Sarvikas (@sarvikas) February 20, 2019

Nougat, Oreo, Pie. Das dritte volle Android Release auf diesem Einsteigergerät. Bin sehr froh, dass meine Empfehlung aufgeht.

Eins habe ich gestern noch vergessen: Das Voyager Neckband von Plantronics.

Meine ersten Eindrücke waren richtig. Das Headset hat sich bewährt. Ich finde es etwas lästiger, einen Ohrstöpsel zu entfernen, als einfach das Headset abzunehmen. Aber es gibt Situationen, wo man ein ANC-Headset haben will und gleichzeitig die Freiheit haben will, sich irgendwo anzulehnen. Das Headset konkurriert direkt mit dem Bose QC30, ist aber besser zum Telefonieren geeignet. Der Listenpreis ist wie immer bei Plantronics völlig überzogen. Statt 336 € muss man aktuell nur noch 184 € bezahlen.

Was man nicht vergessen sollte: Alle Voyager sind ziemlich clever. Man kann sich die Namen von Anrufern vorlesen lassen und die Rufannahme per Sprache steuern. Während man telefoniert, kann das Headset einen zweiten Anruf signalisieren und der Kragen kann vibrieren, um leise über Anrufe zu informieren. Während des Telefonats kann das Headset die eigene Stimme auf die Ohrstöpsel in drei verschiedenen Lautstärken zuspielen, damit man sich selbst besser hört. Die Voyagers haben außerdem einen Schutz gegen zu laute Töne. Es soll ja Leute geben, die versuchen den Angerufenen mit der Trillerpfeife zu verletzen. All diese Erkenntnisse aus dem professionellen Einsatz fließen bei Plantronics in die Software ein.

Was ich bei diesem Headset besonders mag: Der Bügel enthält eine dicke Batterie, die den ganzen Tag hält. Gleichzeitig kann man einfach die Ohrstöpsel baumeln lassen, wenn man gerade nicht spricht oder keine Musik hören will. Wie bei allen ECH ist es sehr wichtig, die richtige Größe der Ohrstöpsel zu wählen. Hat man eine zu kleine Größe gewählt, klingen sie zu dünn und isolieren den Umgebungsschall nicht richtig. Auch das ANC funktioniert nur richtig, wenn die Ohren "dicht" sind.

More >

Samsung has had a good day today. And that was not only the Unpacked Event.

• The Galaxy S8 is getting Android Pie with One UI. That means that Samsung now has the 2017 S8, the 2018 S9 and the new S10 on the same software.
• Speaking of One UI, this time Samsung software does not look so bad actually. They are bringing everything down on those huge screens to make it more reachable, and it's not gimmicky at all. It's a good move, but of course it all ends when you add non-Samsung apps.
• The new S10 looks quite amazing. New triple camera with three focal lengths, an ultrasonic in-screen fingerprint reader, and an even larger screen with a camera cutout instead of a bezel.
• Bixby is available in three more languages: German, Italian and Spanish. That means the Bixby button might finally be useful.

All of this was framed by two devices you probably will not be buying: a Galaxy 5G and the foldable Galaxy Fold, which turns from a smartphone into a small tablet. You should not be buying the 5G phone because there won't be a 5G network for starters. And the Galaxy Fold is so expensive you can actually buy an iPhone XR and an iPad Pro for the same price.

What you probably should be buying if you are in the market for a new Samsung phone is last year's S9. It is now well below 500 €.

Als DJ Koh gestern den Preis für das Galaxy Fold ankündigte, wurde es kurz sehr still im Saal. 1980 US-Dollar - das macht jedermann gleich klar, das wird nicht sein nächstes Smartphone. Große Stückzahlen sind aber auch gar nicht das Ziel dieses Gerätes. So viele kann Samsung gar nicht herstellen. Ich kann mir sogar vorstellen, dass sie trotz des hohen Preises noch drauflegen. Galaxy Fold ist ein Leuchtturm, der zeigt, was die Firma alles bauen kann.

Ohnehin ist es ziemlicher Blödsinn, sich dieses Gerät anzuschaffen. Schon bei den Apps für größere Tablets sieht es bei Android ganz bitter aus. Ich habe von 7 bis 13 Zoll alles ausprobiert, und die allermeisten Apps skalieren einfach hoch, ohne die größere Fläche sinnvoll zu nutzen. Was genau soll man mit einem etwa quadratischen Display machen? Drei Apps parallel, so wie es Samsung zeigte?

Was man statt eines Galaxy Fold so alles kaufen könnte. pic.twitter.com/8EE6OoIKVP

— Volker Weber (@vowe) February 20, 2019

Lampe verschenken, Öl verkaufen, werden sie bei Amazon denken. Anders kann ich mir solche Preise nicht erklären. HD für 25 Euro, 4K für 35. Meine Güte! Dabei machen die nicht nur Prime Video sondern auch YouTube, Netflix und die Mediatheken der Öffis.

More >

Jason Snell's podcasting setup is similar to mine – he wants to hear his own voice, record his local audio track, and have a conversation with multiple people on Skype, who also need to hear his voice coming from an external microphone. And he wants to use one computer to do it all. Now he's figured out how to podcast from an iPad Pro with the help of an additional USB interface:

In the past, I’ve done something similar using the Audio-Technica ATR2100-USB, a microphone that can output a digital signal using USB and an analog signal via an XLR cord simultaneously. The problem is that the last time I tried to use the ATR2100-USB with my iPad Pro, it didn’t return my own voice into my ears, making me unable to judge the sound quality of my own microphone. After years of having my own voice return to me, I strongly prefer not to record unable to hear my own voice. (I use in-ear headphones that largely shut out audio from the outside world, so the experience of speaking while not hearing yourself is even more profoundly weird than it would be with leaky earbuds.)

This time I wanted it all, or at least as close to all as I’m able to get with iOS in the mix: A pristine recording of my own voice, that same high-quality microphone audio also flowing across digitally to my podcast guests via Skype, and the ability to hear both my guests and myself at the same time.

The takeaway from the story isn't that Snell wanted to prove a point to spite Mac users – it's that he was able to travel with one computer instead of two (he would have used most of the same audio gear with a Mac too) and that he found an expensive, but real workaround to professional podcast recording on iPad Pro.

I don't currently have a USB audio interface like Snell's USBPre 2, but I may have to buy one before the summer so I can record podcasts from our beach house using only the iPad Pro. (That is, assuming the iOS 13 beta I'll have installed at that point doesn't have meaningful improvements for audio workflows.)

→ Source: sixcolors.com

A week ago, Apple sent an email to developers announcing that it would require two-factor authentication for all developer accounts beginning February 27, 2019. The message linked to an Apple two-factor authentication support page that applies to all Apple IDs. The trouble was, the support page didn’t answer many of the developer-specific questions that were immediately raised.

The concern I’ve heard voiced most often by developers is whether someone who uses one Apple ID to log into their developer account would be able to do so using an Apple device that is logged in using a different Apple ID. Today, Apple published a new support page answering this and many other questions. Specifically with respect to the two-Apple ID scenario, Apple’s FAQ-style support page says:

Will I need a trusted device dedicated to my Apple Developer account if I enable two-factor authentication?

No. You’ll need to use a trusted device to enable two-factor authentication for the first time. However, you can use the same trusted device for multiple Apple IDs that are enabled for two-factor authentication. Additionally, if you do not have access to your trusted device, you can get your verification code via SMS or phone call. When possible, you should use a trusted device to increase security and streamline the process.

The document covers many other situations as well including:

• How to check if you have two-factor authentication enabled
• Configuring an iOS device or Mac to accept authentication codes for multiple Apple IDs
• Enabling multiple trusted phone numbers that can receive authentication codes

The support page concludes with a link to a contact form for Apple’s developer team to raise any other circumstances that prevent a developer from enabling two-factor authentication.

Although it would have been better if this level of detail was published when Apple’s initial email went out to developers last week, the company has clearly heard the concerns raised by the developer community and has put together a thorough explanation that should address most situations. By answering the most common questions, Apple Developer Relations will hopefully be freed up to deal with any outlier issues that aren’t addressed in its support documentation.

---

Support MacStories Directly

Club MacStories offers exclusive access to extra MacStories content, delivered every week; it's also a way to support us directly.

Club MacStories will help you discover the best apps for your devices and get the most out of your iPhone, iPad, and Mac. Plus, it's made in Italy.

Join Now

The boys dive into a sea of rumors after Federico explores San Jose's municipal websites, Myke gives everyone a gift and Stephen returns from a journey.

On this week's episode of Connected, we discuss the latest Marzipan rumors and consider the implications of a 6K display made by Apple. You can listen here.

Sponsored by:

• Pingdom: Start monitoring your websites and servers today. Use offer code CONNECTED to get 30% off.
• Luna Display: The only hardware solution that turns your iPad into a wireless display for your Mac. Use promo code CONNECTED at checkout for 10% off.
• Care/Of: Personalized supplements delivered to your door.

→ Source: relay.fm

A Better Path to Universal Health Care | Jamie Daw suggests we look to Germany’s health...