Rolandt

It feels like Christmas Eve around here. I am sitting at home on Bowen Island and our house is full of friends and colleagues Amanda Fenton and Kelly Poirier who have now retired to bed. Along with Caitlin, we have completed a long and productive day of planning and design for what will be the 17th annual Art of Hosting on Bowen Island. This evening I am sitting by my fire, finishing a dram of Laphroaig and remembering the first one in 2003 when Toke Moeller and I sat by this same fireplace discussing teaching and learning and what this practice is really all about.

Back then the Bowen Island gatherings were hosted at Rivendell, a beautiful contemplative retreat centre on a small mountain above the village of Snug Cove on Bowen Island. That first one in 2003 was hosted by Myriam Laberge, Brenda Chaddock, Toke, Tenneson Woolf and Teresa Posakony (if I recall correctly) and supported by Marks and Marg McAvity, who we (and still are) stewards of Rivendell. That was the first Art of Hosting for me, and it was really a coming home.

For years I had been working as a facilitator specializing in large group participatory methods and I had a strong sense that there was a leadership practice in the way we hosted Open Space and World Cafe, but I couldn’t put my finger on it. Luckily Toke and his partner Monica Nissen and friends Jan Hein Nielsen and Finn Voldtofte and others had done the ahrd thinking and realized that great participatory meeting had four characteristics: people were present, they were all participating, they were being hosted and they were co-creating something. The Danes postulated that increasing these patterns would bring more engagement, more dignity and more emergence in conversations and so they articulated the four-fold practice of theArt of Hosting, which are the four simple touchstones of presence, participation, hosting and co-creation.

In 2003 I came home to this and was invited the next year to come as an alumni and then the following year where I was invited to be on the hosting team . Every year since 2005 I have been pleased to welcome people to our island, known as Nex?wle?lex?wm in the Squamish language, to experience the Art of Hosting. SInce that time I have been privileged to be on nearly 100 hosting teams for Art of Hosting gatherings around the globe in places as diverse and far flung as Japan, South Africa, Estonia, Ireland, Turkey, and all over Canada and the US. I have worked with dozens of stewards of this practice, and thousands of practitioners, learning every day more and more about how to create social processes that truly affirm human dignity, invite folks into all kinds of storywork, and help people listen to each other in a way that makes it easier and maybe a little more possible for them to co-create the futures they need.

A couple of years ago my friend Scott Macklin caught the spirit of our gathering in a short film. It reflects the kind of pace and deep learning that characterizes the Bowen Island gathering, and is a beautiful record of our 2017 team. Have a watch:

So, as I get ready for bed tonight, I’m feeling deep gratitude for my teachers, especially Toke Moeller and Monica Nissen who guided me onto this path of my life’s work, and who have supported me over these 15 years with love and care. And I’m looking forward to meeting these folks that are coming, each of them like a little Christmas gift, full of surprise and delight and curiosity and possibility.

Ich fand diese Keynote sehr informativ.

My friend and collaborator Brenda Whiteway is delivering the November Island Studies Lecture at the University of PEI on November 19, 2019 at 7:00 p.m. in the Faculty Lounge in Main Building:

The passage of time, transitions in nature, and shifting patterns of life have been recurrent themes in Brenda Whiteway’s creative work. The White Sands PEI Shoreline Project is a further exploration of these themes through paintings, drawings, photographs, and mixed-media of a small coastal area in White Sands, PEI, which has personal significance but resonates on a universal scale. The site originally belonged to her maternal grandparents who had a subsistence farm and fished off the Northumberland Strait close to Pictou Island. Brenda has been observing, recording, and researching the area’s weather patterns, light, tidal shifts, flora and fauna, history, and cultural ecology. For her, this is a metaphoric petri dish through which cultural and climactic shifts may be viewed on an intimate scale and expressed creatively. Capturing the elusive qualities of the seasons through various art mediums while plants bloom and fade, tides shift, earth freezes and thaws, and the sky reveals its many moods is an attempt to capture the genius loci of an area the artist holds dear.

There are fewer more thoughtful, creative people than Brenda, and you should plan to attend.

My friend Elmine, in her post Leer weer langzaam denken, passes on a piece of advice from Frank:

Belangrijk nog om te melden is de tip die ik van Frank kreeg: ook met vijftien minuten per dag kom je verder in een boek. Die tip heeft me in beweging gekregen om aan een eerste boek te beginnen. En wat gebeurd er vervolgens? Ongemerkt lees ik nu zo weer een uur of langer achter elkaar.

Or, machine-translated:

Another important thing to report is the tip I got from Frank: even with fifteen minutes a day you continue in a book. That tip got me moving to start a first book. And what happens next? Unnoticed, I now read for an hour or more in succession.

I’ve found that if I simply make an effort to scatter books around my home and office, I’ll be more likely to pick them up and read them.

And if I pick them up and read them I’m more likely to keep reading them, for far longer than I’d set out to.

This proves a far more effective “I should read more” strategy than trying to conjure up a perfect reading nook and booking out specific reading time.

Put another way: you can’t read a book that’s not in front of you.

My friend Elmine, in her post Leer weer langzaam denken, passes on a piece of advice from Frank:

Belangrijk nog om te melden is de tip die ik van Frank kreeg: ook met vijftien minuten per dag kom je verder in een boek. Die tip heeft me in beweging gekregen om aan een eerste boek te beginnen. En wat gebeurd er vervolgens? Ongemerkt lees ik nu zo weer een uur of langer achter elkaar.

Or, machine-translated:

Another important thing to report is the tip I got from Frank: even with fifteen minutes a day you continue in a book. That tip got me moving to start a first book. And what happens next? Unnoticed, I now read for an hour or more in succession.

I’ve found that if I simply make an effort to scatter books around my home and office, I’ll be more likely to pick them up and read them.

And if I pick them up and read them I’m more likely to keep reading them, for far longer than I’d set out to.

This proves a far more effective “I should read more” strategy than trying to conjure up a perfect reading nook and booking out specific reading time.

Put another way: you can’t read a book that’s not in front of you.

Seeing this a lot.

This is the fifth and final in a series of posts examining some of the most common and most problematic problems we need to consider when looking to scale research in organisations. You can start with the first post in this series here.

Here are five common dysfunctions that we are contending with.

1. Teams are incentivised to move quickly and ship, care less about reliable and valid research
2. Researching within our silos leads to false positives
3. Research as a weapon (validate or die)
4. Quantitative fallacies
5. Stunted capability

In this post, we’re looking at what happens when the research practice in an organisation fails to mature.

A great first step

Testing one user is 100 percent better than testing none – Steve Krug, Don’t Make Me Think

Many organisations get started doing research with customers and users off the back of encouragement from people like Steve Krug and his classic book ‘Don’t Make Me Think’. In this and other books Steve makes simple usability testing accessible and achievable to almost anyone.

Steve and others like him are evangelists reaching out to those companies who are afraid to engage with their customers to understand opportunities for them to improve. This is important work. Their message is usually that talking to customers is not hard or scary, and that we’ll be better for doing a bit of it, even not perfectly, than not doing it at all.

The first step can be scary

And they are right. Having anyone in the company talking to just one user (and hopefully some more) is a fabulous first step. But it is intended to be just that – a first step. An encouragement to realise the benefits of involving people outside our offices in the process of designing and developing products and services. And help to overcome the fear of engaging with customers and users and an opportunity to experience how beneficial this can be.

For those of us who work with research participants on a regular basis, it may be hard to recall exactly how terrifying those first few research sessions felt. Even trained and experienced researchers continue to experience some background fear (or exhilaration?)  of all the things that could go wrong in the research study – and there are plenty!

The thing about first steps, though, is that they are usually intended to be followed by second steps. Once we break through the fear (or in some cases, just lack of awareness), the idea is that we continue to increase the maturity of our practice.

And this is where many organisations seem to hit a roadblock. More and more people in the organisation might be out and eagerly involving customers in the process of shaping their products, but they often don’t invest in either improving their own skills in research or investing in hiring people who have training and experience doing research. 

Talking to users is not research

One important realisation we need to have on the path to maturity is recognising that ‘talking to customers’ is actually not the same thing as doing research. Talking to customers or watching customers use our products and services has many benefits – in particular it can increase our empathy for our customers and users, it can help expose us to scenarios of use that are dramatically different to our own and what we would expect, and it can provide clues as to where the biggest problems may like. All of these are good outcomes.

If we want to use research as evidence for decision making – either for product strategy or design decisions – then we need to be able to do more to ensure that the insights we are gleaning are sufficiently reliable and valid.

Research doesn’t need to be ‘perfect’, just valid and reliable.

 ‘I don’t need the research to be perfect, I just need enough to help me make a decision’.

Often this is said in response to the suggestion that the research we should be doing will take longer or be more difficult and expensive than our speaker would like. In this situation, there is often a pre-existing ‘hunch’ and they are looking to users for validation. Or perhaps they are stuck between two options and seeks a tie breaker.

Any specialist researcher has almost certainly had their recommended approach discredited as ‘too academic’, and sometimes it is true. Sometimes the research methodology is overdone for the question the business is seeking to answer. But what often follows is a bit of a race to the bottom where considered sample design and appropriate methodology are quickly discarded in favour of whatever is fastest and easiest.

Without the right experience and training, all too often interviewers ‘cut to the chase’ so we get more or less directly to the topic at hand. Somewhere in the world right now a product manager under pressure to make a decision is asking questions like these in a customer interview:

‘here’s what we’re thinking of making, what do you think about it?’

or, perhaps worse…

‘if we made this, would you pay for it’

It can be easy and tempting – so much faster and often quantitative – to mistake the research question for the interview question.

Even with training, it seems that the urge to be able to say that 10 out of 12 people said they would pay for it is almost irresistible. ‘Beating around the bush’ to get the question answered seems like a waste of everyone’s time, in this time where the bias to action and desire to ship at velocity is most valued. 

(It shouldn’t really be a surprise that lack of research capability maturity exposes us to the previous four dysfunctions).

Matching methodology to risk

Whilst we should have plenty of sympathy for this desire for lightweight research and simplicity, it is important to ensure that the methods employed are matched to the risk involved in the decision, rather than the most compressed timeframe.

As our organisations grow, the decisions we take using evidence from our customers can become more and more substantial – the gains of getting it right are greater and the risks of getting it wrong get uglier.

In the same way, our research maturity needs to continue to grow so that we can continue to match the size of the risk of getting it wrong.

This is not to say that mature organisations only ever do serious, time consuming research. Rather, that we invest where the risk is highest.

Investment might look like hiring trained researchers who can design and recruit the right sample and conduct the research in a way that reduces bias. Or investment might look like iterative research with an every increasing number of increasingly diverse participants, sprint after sprint – allowing the team to continue to learn, This can work beautifully when the team is able to be responsive to that learning over time.

Investing too much

Conversely, there are situations where the investment in research is far too high for the decision being made. This often happens where the organisations design process has broken down, or where designers have entirely lost confidence in being able to make relatively conventional design decisions. In these situation we design complex studies to ‘validate’ one micro design treatment over another. In this case,  the mismatch in risk to research investment can result in large quantities of what I would consider to be wasteful and often unreliable research. 

Beware Dunning Kruger

User Research is particularly susceptible to Dunning Kruger syndrome, wherein a relatively small amount of knowledge can result in an excess of confidence. Many people claim a ‘background in research’ when they could mean they watched someone else do a bunch of usability studies in their last job, or they did a research based degree at university.

Many designers and product managers are entirely happy with the outcomes they get from research and how it enables their practice – and often loudly object to the suggestion that anyone could get a better result from the research than they do.

Yet, at the same time, the harsh reality is that the work that is done is often resulting in misleading outcomes that can put their product and their organisation at risk. 

It also undermines the reputation of research in the organisation when people claim when a ‘researched’ product goes into the world and doesn’t succeed as expected. ‘We did research before and it didn’t work’.

In the same way that often both design and product management capabilities require an engineering led organisation to move through the stages from unconscious incompetence through to conscious competence ,  the very same is true for the research capability.

Achieving research maturity

And so, at the end of our five dysfunctions, what can be done to help provoke an organisation to not only involve users in the process of creating products and services but to start and continue to grow their ability to do so by revealing the important insights that are both reliable and valid?

Here are some things that have worked for me.

Perhaps through improving business fluency. By talking less about empathy and more about the risk to the business of getting it wrong. Talking less about customer obsession and more about the reliability and validity of the different types of evidence we can use to make decisions. And by running an open research practice – getting out of the black box, removing any mystery about our work, showing our workings and involving others in the process.

Make use of existing momentum – bringing new shape and substance to whatever your organisation is using to bring its attention to its customers – whether its an NPS survey, a customer convention, a feedback form, or a guerilla research practice – start by shaping the existing connections into something more insightful, more reliable and valid.

Be brave, but be patient and we’ll get there.

Fri. 11.08.19 – More and more, due to software algorithms, it is hard to get a phone’s camera to achieve a satisfying blur or nightscape. All the recent camera phones want to make dark scenes light and blurred scenes sharp. Thus, when I am able to get the Samsung in a quick snapshot to achieve... Read more »

I finally got around to implementing dark mode on my website. It was easier than I feared it would be, thanks to CSS variables.

Given that the forecast was for a decent amount of snow early next week, along with some cold weather, it was time to pull the winter beater out of the garage.

I also had to have a new bottom bracket installed by the good folks at Hoopdriver Bicycles.

Look, small patches of black ice seen yesterday. Easy enough to avoid, but my agreement with my family is that if the temperatures are below zero, I ride the bike with studded tires. This is because of one evening several years ago that started well above zero, but ended with freezing rain, and I went over hard on my pink commuter and broke my collarbone.

I did get some comments from the LBS about the spectacular amount of rust on my fenders, which were described as “leperous”.

They also said that the crank bolts had corroded and were difficult to remove.

They also replaced my stainless chain with a galvanized one. We’ll see how long this one lasts, but I pledged to wipe the salt off of the bike at least once in a while.

I was also inspired to swap back the winter tires on the Haul a Day.

All ready for winter now, although it looks like I put the front tire the wrong way around.

With the end of DST, my evening commutes are now dark, and so I’ve switched to my T2 helmet with the built in lights. With earflaps, of course.

Still happy with it although it looks like the LA based company that made it is no more. The helmet still appears to be available in Europe for the moment. However, I wasn’t too pleased to find out that this fellow uses the same helmet.

I apparently have another helmet with built in lights in bound from a much delayed kickstarter, and I’ll report on it if and when it finally arrives.

Here’s a common situation: you have a folder full of similarly-formatted CSV or otherwise structured text files that you want to get into R quickly and easily. Reading data into R is one of those tasks that can be a real source of frustration for beginners, so I like collecting real-life examples of the many ways it’s become much easier.

This week in class I was working with country-level historical mortality rate estimates. These are available from mortality.org, a fabulous resource. They have a variety of data available but I was interested in the 1x1 year estimates of mortality for all available countries. By “1x1” I mean that the tables show (for men, women, and in total) age-specific morality rate estimates for yearly ages from 0 to 110 and above, for every available historical year (e.g. from 1850 to 2016 or what have you). So you can have an estimate of the mortality rate for, say, 28 year olds in France in 1935.

Downloading this data gives me a folder of text files, one for each country. (Or rather, country-like unit: there are separate series for, e.g. East Germany, West Germany, and Germany as a whole, for example, along with some countries where sub-populations are broken out historically.) The names of the files are consistently formatted, as is the data inside them, and they have a .txt extension. What I wanted to do was get each one of these files into R, ideally putting them all into a single big table that could be the jumping-off point for subsetting and further analysis.

I know from the documentation provided by mortality.org that the files all have the same basic format, which of course makes things much easier. The data is already clean. It’s just a matter of loading it all in efficiently, or “ingesting” it, to use the charming image that seems to be preferred at present.

Here we go. First, some libraries.

1
2
3
4

library(tidyverse)
library(janitor)
library(here)

1
2

## here() starts at /Users/kjhealy/Source/demog

We get a list of the filenames in our raw data folder, along with their full paths. Then we take a look at them.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

filenames <- dir(path = here("rawdata"),
                 pattern = "*.txt",
                 full.names = TRUE)

filenames

##  [1] "/Users/kjhealy/Source/demog/rawdata/AUS.Mx_1x1.txt"    
##  [2] "/Users/kjhealy/Source/demog/rawdata/AUT.Mx_1x1.txt"    
##  [3] "/Users/kjhealy/Source/demog/rawdata/BEL.Mx_1x1.txt"    
##  [4] "/Users/kjhealy/Source/demog/rawdata/BGR.Mx_1x1.txt"    
##  [5] "/Users/kjhealy/Source/demog/rawdata/BLR.Mx_1x1.txt"    
##  [6] "/Users/kjhealy/Source/demog/rawdata/CAN.Mx_1x1.txt"    
##  [7] "/Users/kjhealy/Source/demog/rawdata/CHE.Mx_1x1.txt"    
##  [8] "/Users/kjhealy/Source/demog/rawdata/CHL.Mx_1x1.txt"    
##  [9] "/Users/kjhealy/Source/demog/rawdata/CZE.Mx_1x1.txt"    
## [10] "/Users/kjhealy/Source/demog/rawdata/DEUTE.Mx_1x1.txt"  
## [11] "/Users/kjhealy/Source/demog/rawdata/DEUTNP.Mx_1x1.txt" 
## [12] "/Users/kjhealy/Source/demog/rawdata/DEUTW.Mx_1x1.txt"  
## [13] "/Users/kjhealy/Source/demog/rawdata/DNK.Mx_1x1.txt"    
## [14] "/Users/kjhealy/Source/demog/rawdata/ESP.Mx_1x1.txt"    
## [15] "/Users/kjhealy/Source/demog/rawdata/EST.Mx_1x1.txt"    
## [16] "/Users/kjhealy/Source/demog/rawdata/FIN.Mx_1x1.txt"    
## [17] "/Users/kjhealy/Source/demog/rawdata/FRACNP.Mx_1x1.txt" 
## [18] "/Users/kjhealy/Source/demog/rawdata/FRATNP.Mx_1x1.txt" 
## [19] "/Users/kjhealy/Source/demog/rawdata/GBR_NIR.Mx_1x1.txt"
## [20] "/Users/kjhealy/Source/demog/rawdata/GBR_NP.Mx_1x1.txt" 
## [21] "/Users/kjhealy/Source/demog/rawdata/GBR_SCO.Mx_1x1.txt"
## [22] "/Users/kjhealy/Source/demog/rawdata/GBRCENW.Mx_1x1.txt"
## [23] "/Users/kjhealy/Source/demog/rawdata/GBRTENW.Mx_1x1.txt"
## [24] "/Users/kjhealy/Source/demog/rawdata/GRC.Mx_1x1.txt"    
## [25] "/Users/kjhealy/Source/demog/rawdata/HRV.Mx_1x1.txt"    
## [26] "/Users/kjhealy/Source/demog/rawdata/HUN.Mx_1x1.txt"    
## [27] "/Users/kjhealy/Source/demog/rawdata/IRL.Mx_1x1.txt"    
## [28] "/Users/kjhealy/Source/demog/rawdata/ISL.Mx_1x1.txt"    
## [29] "/Users/kjhealy/Source/demog/rawdata/ISR.Mx_1x1.txt"    
## [30] "/Users/kjhealy/Source/demog/rawdata/ITA.Mx_1x1.txt"    
## [31] "/Users/kjhealy/Source/demog/rawdata/JPN.Mx_1x1.txt"    
## [32] "/Users/kjhealy/Source/demog/rawdata/KOR.Mx_1x1.txt"    
## [33] "/Users/kjhealy/Source/demog/rawdata/LTU.Mx_1x1.txt"    
## [34] "/Users/kjhealy/Source/demog/rawdata/LUX.Mx_1x1.txt"    
## [35] "/Users/kjhealy/Source/demog/rawdata/LVA.Mx_1x1.txt"    
## [36] "/Users/kjhealy/Source/demog/rawdata/NLD.Mx_1x1.txt"    
## [37] "/Users/kjhealy/Source/demog/rawdata/NOR.Mx_1x1.txt"    
## [38] "/Users/kjhealy/Source/demog/rawdata/NZL_MA.Mx_1x1.txt" 
## [39] "/Users/kjhealy/Source/demog/rawdata/NZL_NM.Mx_1x1.txt" 
## [40] "/Users/kjhealy/Source/demog/rawdata/NZL_NP.Mx_1x1.txt" 
## [41] "/Users/kjhealy/Source/demog/rawdata/POL.Mx_1x1.txt"    
## [42] "/Users/kjhealy/Source/demog/rawdata/PRT.Mx_1x1.txt"    
## [43] "/Users/kjhealy/Source/demog/rawdata/RUS.Mx_1x1.txt"    
## [44] "/Users/kjhealy/Source/demog/rawdata/SVK.Mx_1x1.txt"    
## [45] "/Users/kjhealy/Source/demog/rawdata/SVN.Mx_1x1.txt"    
## [46] "/Users/kjhealy/Source/demog/rawdata/SWE.Mx_1x1.txt"    
## [47] "/Users/kjhealy/Source/demog/rawdata/TWN.Mx_1x1.txt"    
## [48] "/Users/kjhealy/Source/demog/rawdata/UKR.Mx_1x1.txt"    
## [49] "/Users/kjhealy/Source/demog/rawdata/USA.Mx_1x1.txt"

What does each of these files look like? Let’s take a look at the first one, using read_lines() to show us the top of the file.

1
2
3
4
5
6
7
8
9

read_lines(filenames[1], n_max = 5)

## [1] "Australia, Death rates (period 1x1), \tLast modified: 26 Sep 2017;  Methods Protocol: v6 (2017)"
## [2] ""                                                                                               
## [3] "  Year          Age             Female            Male           Total"                         
## [4] "  1921           0             0.059987        0.076533        0.068444"                        
## [5] "  1921           1             0.012064        0.014339        0.013225"

All the files have a header section like this. When we read the data in we’ll want to ignore that and go straight to the data. But seeing as it’s there, we can make use of it to grab the name of the country. It saves us typing it ourselves. Let’s say we’d also like to have a code-friendly version of those names (i.e., in lower-case with underscores instead of spaces). And finally—while we’re at it—let’s grab those all-caps country codes used in the file names, too. We write three functions:

• get_country_name() grabs the first word or words on the first line of each file, up to the first comma. That’s our country name.
• shorten_name() makes the names lower-case and replaces spaces with underscores, and also shortens “The United States of America” to “USA”.
• make_ccode() wraps a regular expression that finds and extracts the capitalized country codes in the file names.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

get_country_name <- function(x){
    read_lines(x, n_max = 1) %>%
        str_extract(".+?,") %>%
        str_remove(",")
}

shorten_name <- function(x){
    str_replace_all(x, " -- ", " ") %>%
        str_replace("The United States of America", "USA") %>%
        snakecase::to_any_case()
}

make_ccode <- function(x){
    str_extract(x, "[:upper:]+((?=\\.))")
}

Now we create a tibble of summary information by mapping the functions to the filenames.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

countries <- tibble(country = map_chr(filenames, get_country_name),
                        cname = map_chr(country, shorten_name),
                        ccode = map_chr(filenames, make_ccode),
                        path = filenames)

countries

## # A tibble: 49 x 4
##    country     cname       ccode path                                      
##    <chr>       <chr>       <chr> <chr>                                     
##  1 Australia   australia   AUS   /Users/kjhealy/Source/demog/rawdata/AUS.M…
##  2 Austria     austria     AUT   /Users/kjhealy/Source/demog/rawdata/AUT.M…
##  3 Belgium     belgium     BEL   /Users/kjhealy/Source/demog/rawdata/BEL.M…
##  4 Bulgaria    bulgaria    BGR   /Users/kjhealy/Source/demog/rawdata/BGR.M…
##  5 Belarus     belarus     BLR   /Users/kjhealy/Source/demog/rawdata/BLR.M…
##  6 Canada      canada      CAN   /Users/kjhealy/Source/demog/rawdata/CAN.M…
##  7 Switzerland switzerland CHE   /Users/kjhealy/Source/demog/rawdata/CHE.M…
##  8 Chile       chile       CHL   /Users/kjhealy/Source/demog/rawdata/CHL.M…
##  9 Czechia     czechia     CZE   /Users/kjhealy/Source/demog/rawdata/CZE.M…
## 10 East Germa… east_germa… DEUTE /Users/kjhealy/Source/demog/rawdata/DEUTE…
## # … with 39 more rows

Nice. We could have written each of those operations as anonymous functions directly inside of map_chr(). This would have been more compact. But often it can be useful to break out the steps as shown here, for clarity—especially if map() operations have a tendency to break your brain, as they do mine.

We still haven’t touched the actual data files, of course. But now we can just use this countries table as the basis for reading in, I mean ingesting, everything in the files. We’re going to just add a list column named data to the end of the table and put the data for each country in it. We’ll temporarily unnest it to clean the column names and recode the age variable, then drop the file paths column and nest the data again.

The hard work is done by the map() call. This time we will use ~ formula notation inside map() to write what we want to do. We’re going to feed every filename in path to read_table(), one at a time. We tell read_table() to skip the first two lines of every file it reads, and also tell it that in these files missing data are represented by a . character. Everything read in ends up in a new list column named data.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

mortality <- countries %>%
    mutate(data = map(path,
                      ~ read_table(., skip = 2, na = "."))) %>%
    unnest(cols = c(data)) %>%
    clean_names() %>%
    mutate(age = as.integer(recode(age, "110+" = "110"))) %>%
    select(-path) %>%
    nest(data = c(year:total))

mortality


## # A tibble: 49 x 4
##    country      cname        ccode           data
##    <chr>        <chr>        <chr> <list<df[,5]>>
##  1 Australia    australia    AUS     [10,434 × 5]
##  2 Austria      austria      AUT      [7,881 × 5]
##  3 Belgium      belgium      BEL     [19,425 × 5]
##  4 Bulgaria     bulgaria     BGR      [7,104 × 5]
##  5 Belarus      belarus      BLR      [6,438 × 5]
##  6 Canada       canada       CAN     [10,101 × 5]
##  7 Switzerland  switzerland  CHE     [15,651 × 5]
##  8 Chile        chile        CHL      [1,887 × 5]
##  9 Czechia      czechia      CZE      [7,437 × 5]
## 10 East Germany east_germany DEUTE    [6,660 × 5]
## # … with 39 more rows

And we’re done. Forty nine tables of data smoothly imported and bundled together. Each of the country-level data tables is a row in data that we can take a look at as we like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

mortality %>% 
  filter(country == "Austria") %>% 
  unnest(cols = c(data))


## # A tibble: 7,881 x 8
##    country cname   ccode  year   age   female     male    total
##    <chr>   <chr>   <chr> <dbl> <int>    <dbl>    <dbl>    <dbl>
##  1 Austria austria AUT    1947     0 0.0798   0.0994   0.0899  
##  2 Austria austria AUT    1947     1 0.00657  0.00845  0.00753 
##  3 Austria austria AUT    1947     2 0.00425  0.00469  0.00447 
##  4 Austria austria AUT    1947     3 0.00337  0.00340  0.00339 
##  5 Austria austria AUT    1947     4 0.00235  0.00270  0.00253 
##  6 Austria austria AUT    1947     5 0.00174  0.00195  0.00184 
##  7 Austria austria AUT    1947     6 0.00131  0.00152  0.00142 
##  8 Austria austria AUT    1947     7 0.00132  0.00169  0.00151 
##  9 Austria austria AUT    1947     8 0.00115  0.00149  0.00132 
## 10 Austria austria AUT    1947     9 0.000836 0.000997 0.000918
## # … with 7,871 more rows

Now you can get on with the actual analysis.

There isn’t anything especially unusual in the steps shown here. It’s just a pretty common operation that’s worth knowing how to do cleanly. One nice thing about this approach is that it’s immediately applicable to, say, a folder containing the 5-year mortality estimates rather than the 1 year estimates. You don’t have to do anything new, and there’s no mucking around with manually naming files and so on.

mkalus shared this story from Das Kraftfuttermischwerk.

In Berlin Kreuzberg hat die Bezirksverordnetenversammlung Friedrichshain-Kreuzberg zu einer öffentlichen Diskussionsveranstaltung eingeladen und dabei unter anderem vorgeschlagen, den Heinrichplatz zum Rio-Reiser-Platz umzubenennen. Und das ganz zu Recht, wie ich finde. Wir brauchen viel mehr Rio-Reiser-Plätze. Überall.

Ob es dazu tatsächlich kommen sollte, entscheidet demnächst die Berliner Bezirksverordnetenversammlung.

Kreuzberg hat gesprochen: Der Heinrichplatz wird wahrscheinlich bald Rio-Reiser-Platz heißen. Am Donnerstagabend hatte die Bezirksverordnetenversammlung Friedrichshain-Kreuzberg zu einer öffentlichen Diskussionsveranstaltung eingeladen. Wie soll Kreuzberg an den verstorbenen Sänger erinnern? So lautete die Frage. Mit knapp 70 Gästen war der Veranstaltungssaal Aquarium am Kottbusser Tor sehr gut besucht.

Gemeinsam mit den Gästen diskutierten die Fürsprecher*innen die verschiedenen Vorschläge. Neben dem Vorschlag zum Heinrichplatz war eine Idee, den Südzipfel des Mariannenplatzes umzubenennen, der in Reisers berühmten »Rauch-Haus-Song« erwähnt wird. Auch die Umbenennung des Uferwegs an der Lohmühleninsel, eine Gedenkinstallation oder der Vorschlag, die Grünanlage vor dem Kunsthaus Bethanien zum Rio-Reiser-Park zu erklären, standen zur Debatte. Neben der Grünen-Politikerin und Kulturstadträtin von Friedrichshain-Kreuzberg Clara Herrmann hatten sich auch Gert C. Möbius, der Bruder von Rio Reiser, und Kai Sichtermann, Bassist und Gründungsmitglied der Band Ton Steine Scherben, mit Redebeiträgen beteiligt.

As you build your professional network it won’t be uncommon for someone you know to ask you for an introduction. A decade ago, Fred Wilson asked for the adoption of the double opt-in introduction. Today these are standard as they are compassionate way to connect people. This post describes the emails involved in the introduction dance.

Helping others expand their network contributes to compounding the value of your own. But it’s important to act with compassion toward those involved. Your colleague Bob may ask you to introduce you to your friend Alice. You don’t know if Alice has the interest or personal bandwidth to make the connection unless you ask her.

Never begin an introduction with an email cc’ing both Alice and Bob saying “Hi Alice, I’d like to introduce you to Bob”.

Instead, use a double opt-in introduction. First, send a short note to Alice.

Subject: Introduction Request

Hi Alice. I’d like to introduce you to Bob who is a [personal friend / person I’ve been mentoring / former colleague].

Bob would like to connect with you so he can learn more about [thing that interests Bob]. It may be valuable for you to connect as he [thing that Bob knows or does that Alice cares about].

May I have your permission to connect the two of you?

Keep the email short and concise. And don’t sell Bob short. Include the reason it would be valuable for Alice to accept the request.

If you don’t receive permission, send a short note back to Bob with regrets.

Subject: Introduction to Alice — Regrets

Hi Bob. Alice sends regrets that she is unable to make the intro.

As with invitations, regrets are the best simple and short. You need not give a reason. Bob is a compassionate and empathetic person so he doesn’t take the notice of regrets personally. He knows that people are busy and he’ll be able to find other means of solving his particular problem.

Having received permission, send an email cc’ing both Alice and Bob. I’ve used an introduction Trevor O recently made on my behalf to construct this template. It’s the best example of a double opt-in introduction email I’ve seen. I’m using it here with his permission.

Subject: Alice <> Bob

Hi Alice,

As mentioned, I’d like to introduce you to Bob. He and I know each other from [place you and Bob know each other from]. He’s recently been [doing thing that interests Alice]. And I mentioned what you have been doing with [thing that interests Bob].

Bob,

I’d like you to meet Alice. Alice has tons of experience with [thing that interests Bob]. I respect how she [thing that you respect about Alice].

I’ll let you two take it from here.

- Shawn

The subject “NAME <> NAME” is the standard double opt-in introduction subject. Some people receive many introductions a week so it goes a long way to keep things consistent.

Begin by addressing the person granting the introduction. Introduce Alice to Bob with some brief info on how you know Bob and why it would be valuable for Alice to connect. Also, make it clear it’s a double opt-in introduction by stating “As mentioned …”. This lets Bob know you have already spoken to Alice.

Next, address the person who requested the introduction. In this case, introduce Bob to Alice and explain what she is doing in terms that will appeal to Bob.

Finally, sign off allowing both Alice and Bob to continue the conversation. Bob will reply to the email to follow up with Alice, removing you from the thread.

The double opt-in introduction is the compassionate way to make introductions. Using this system will grow the value of your network and your reputation.

Originally published at shawnprice.com

Elizabeth Warren has big plans, and they would cost a lot with a big shift in government spending. The New York Times breaks it down.

I realize the topic here is important, but NYT’s bubble game is on point in this piece. Check out those transitions as the bubbles funnel into the screen from the top and how the pie-like segments rotate as each segment is highlighted.

Force-directed graphs, for the win, amirite.

Tags: Elizabeth Warren, New York Times, spending

An interview is a significant time investment. If it’s not personal, it’s useless. How would you respond to this pitch? Subject: Invitation @ Interview Series Hi Josh, My name is Olga and I’m the Project Marketing Manager at Brief. Apple has featured us as [Apps We Love] in 2018 and 2019. We’ve been working on … Continued

The post Never send a generic invitation appeared first on without bullshit.

My friend and collaborator Brenda Whiteway is delivering the November Island Studies Lecture at the University of PEI on November 19, 2019 at 7:00 p.m. in the Faculty Lounge in Main Building:

The passage of time, transitions in nature, and shifting patterns of life have been recurrent themes in Brenda Whiteway’s creative work. The White Sands PEI Shoreline Project is a further exploration of these themes through paintings, drawings, photographs, and mixed-media of a small coastal area in White Sands, PEI, which has personal significance but resonates on a universal scale. The site originally belonged to her maternal grandparents who had a subsistence farm and fished off the Northumberland Strait close to Pictou Island. Brenda has been observing, recording, and researching the area’s weather patterns, light, tidal shifts, flora and fauna, history, and cultural ecology. For her, this is a metaphoric petri dish through which cultural and climactic shifts may be viewed on an intimate scale and expressed creatively. Capturing the elusive qualities of the seasons through various art mediums while plants bloom and fade, tides shift, earth freezes and thaws, and the sky reveals its many moods is an attempt to capture the genius loci of an area the artist holds dear.

There are fewer more thoughtful, creative people than Brenda, and you should plan to attend.