What impact might the release of my evidence-based software engineering book have on software engineering in 2021?

Lots of people have seen the book. The release triggered a quarter of a million downloads, or rather it getting linked to on Twitter and Hacker News resulted in this quantity of downloads. Looking at the some of the comments on Hacker News, I suspect that many ‘readers’ did not progress much further than looking at the cover. Some have scanned through it expecting to find answers to a question that interests them, but all they found was disconnected results from a scattering of studies, i.e., the current state of the field.

The evidence that source code has a short and lonely existence is a gift to those seeking to save time/money by employing a quick and dirty approach to software development. Yes, there are some applications where a quick and dirty iterative approach is not a good idea (iterative as in, if we make enough money there will be a version 2), the software controlling aircraft landing wheels being an obvious example (if the wheels don’t deploy, telling the pilot to fly to another airport to see if they work there is not really an option).

There will be a few researchers who pick up an idea from something in the book, and run with it; I have had a couple of emails along this line, mostly from just starting out PhD students. It would be naive to think that lots of researchers will make any significant changes to their existing views on software engineering. Planck was correct to say that science advances one funeral at a time.

I’m hoping that the book will produce a significant improvement in the primitive statistical techniques currently used by many software researchers. At the moment some form of Wilcoxon test, invented in 1945, is the level of statistical sophistication wielded in most software engineering papers (that do any data analysis).

Software engineering research has the feeling of being a disjoint collection of results, and I’m hoping that a few people will be interested in starting to join the dots, i.e., making connections between findings from different studies. There are likely to be a limited number of major dot joinings, and so only a few dedicated people are needed to make it happen. Why hasn’t this happened yet? I think that many academics in computing departments are lifestyle researchers, moving from one project to the next, enjoying the lifestyle, with little interest in any research results once the grant money runs out (apart from trying to get others to cite it). Why do I think this? I have emailed many researchers information about the patterns I have found in the data they sent me, and a common response is almost completely disinterest (some were interested) in any connections to other work.

What impact do you think ‘all’ the evidence presented will have?

A familiar story last week.

Someone tried to start a community of practice within a niche field.

He reached out to prospective members. They all agreed it was a good idea and seemed keen to be involved. He created a group on an existing social network, initiated a few discussions, and invited them to join.

Then nothing much really happened. Few people replied. Discussions never really took off. Within 2 weeks the community was pretty much dead.

On paper, the founder had done almost everything right.

But the community was missing a spark. It was missing the oomph.

It was missing the thing that was going to make this community more exciting than anything else they can do at the time. It’s very hard to build a community if you’re going through the motions.

Sometimes the oomph is just the overwhelming passion and belief of the founder. Some people are simply magnetic and make communities happen. Sometimes it’s daring to do something new for the first time, maybe something that might not work. Sometimes it’s an especially exciting, audacious, goal. Sometimes it’s a feeling of having a secret and flying under the radar. etc…

I’d love to be able to list how to put the oomph into your community. But it doesn’t really work like that. You have to spend enough time with your audience that you notice things no-one else does. Maybe an unmet desire, or a cluster of people forming, or a desire for change.

Then you have to have desire and belief to think you can make it happen.

One of the things I've enjoyed about lockdown has been the 'casualisation' of news and media.

Technical issues, bad sound, wonky camerawork are laughed off and forgiven when everyone's struggling to even get on air. And the presenters seem more chatty, more willing to break the fourth wall, when everyone's broadcasting from their cupboard or from under a quilt.

I like that they've stopped pretending that there are no cameras and microphones involved. Conference organisers had a phase where they tried to hide the laptops too, but that seemed to pass eventually.

And it means that a wider, more diverse range of better people have started turning up on podcasts and online conferences. Because they don't have to travel. They don't have to take so much time off work. They don't have to arrange child care.

The sound gets slightly worse but the content gets hugely better. That's worth it.

When you live on an island like ours, Nex?wle?lex?xwm/Bowen Island there are rhythms that are like breathing. They come and go over time on cycles as short as an hour or as long as geological epochs.

Most mornings I begin my day on my covered porch, drinking a coffee, reading a meditation, spending some time in silence and contemplation. At this time of year the mornings are dark and, more often than not, wet. This morning we are in day four of what is called an atmospheric river, a massive steady plume of rain that extends from the Hawaiian Islands north-east to our coast. Such a pattern is like a long exhale of moisture, a plume of breath from the tropics that brings warm air and rain and sometimes fronts with gale force winds, which we call the Pineapple Express.

From my morning perch I can see the ferry coming and going, every hour or so, our connection to “the continent.” The early ferries – 6:20 and 7:30 – are commuter runs, with workers heading to the city on the earlier boat and high school students off to school on the later one. As the ferry approaches, the intensity of traffic on my road increases, and the closer we get to sailing time, the higher the speed of cars racing to make sure they don’t miss the boat, or to deliver a sleepy bus-missing teenager to the dock. There is a period of stillness and then the flow reverses and the labourers from the city who have made an early start travel in their work vans and pick-up trucks through the arteries and capillaries of our island road system. As the ferry leaves, things become still and quiet again.

It is very much like the tide that comes and goes twice a day, sometimes bringing as much as 3.5 meters of water to our shoreline, lifting the logs off the beach and floating them on the currents and eddies of Atl’ka7tsem/Howe Sound, the inlet in which we live. This time of year we are coming into our highest tides, and the beaches will be cleared of the boom logs and torrent debris that has cascaded off the mountains into the sea during the past few months of rain and erosion.

And there are the longer period rhythms here as well. The world breathes birds on to our island all in season. Right now there are the winter residents having their run of the place, and with a mild and possibly snowless winter at this altitude near sea level, the towhees, juncos, song sparrows, chickadees, nuthatches, and wrens are spoiled for choice. large flocks of siskins and kinglets swirl in the grey air and occasionally at night you can hear the calls of snow geese flying high above the coast line in search of their estuary winter feeding grounds south of here at the mouth of the Sto:lo/Fraser River.

Winter is short here, and the new year brings with it a month of rain and grey, with gradually brightening skies and then the first hints of spring weather in early February, in line with the Irish seasons more so than the Gregorian ones. But of course there is already a calendar here, the Squamish calendar,that relates to the seasons of food and harvest. The land never really rests in warm winters like this, with the forest extracting as much as it can from the dim light but the mycelial networks in the forest floor working overtime to breakdown nutrients and keep everything fed and flowing in the moist and nutrient rich humus. The forest itself breaths a rhythm of feed from the sky and the earth, continuously growing the giant trees for which our coast is known.

Everything is geared around natural rhythms here, and they care little for the smoothing out of human life brought about by a pandemic. Our community rhythms have become a faint signal in the past 10 months, the peaks and troughs of gatherings, festivals and commemorative events flattened into mere bumps and barely acknowledged remembrances. In this sense it feels very much like our village has been holding its breath, but I also have a small worry that with another year of lock down we will become severed from the rhythms of community life. It only means that we will have to create new ones, or resurrect the former ones in new forms. But it does remind one of how easy it is to break the fabric of community life and set people adrift with one another, a dynamic that was sued against the indigenous population here over generations, through the pandemics of smallpox and colonization, which ravaged community life and stole even the waiting breath.

And in my own life, a turn has been taken as my youngest child has moved out, into an apartment in the city where his 20 year old life is also on hold. He has a job and will start a new set of university classes online this winter, but being 20 and living in a city for the first time is supposed to be a time of socializing, living life fully and enjoying oneself, and that’s just not possible at the moment. Back on the island, Caitlin and I have become empty nesters, and have just spent a couple of weeks in quiet and still recalibration of our lives in a shared space that, after 23 years of parenting, is once again just the two of us. Another exhalation, a deeper one, and an intake of breath for what the next third of life will hold.

Happy New Year to you all. May you continue to breathe and find life in the rhythms of breath that surround you.

I enjoyed reading Vitalik's 2020 endnotes. Vitalik is one of the founders of Ethereum, and one of the most interesting people in the world to follow right now.

Like Vitalik, I'm interested in economic systems, multi-stakeholder coordination, and public good governance and sustainability. How do we create Open Source communities that will thrive for hundreds of years to come? How do we make sure the Open Web is still thriving in a thousand years? These are some of the questions I think about.

While I think about these things, Vitalik is making it happen. The blockchain world is experimenting with new funding models (e.g. ICOs, DAICOs or quadratic funding), decision-making models (e.g. quadratic voting), organizational models (e.g. DAOs), and architectural innovation (e.g. Filecoin or Blockstack).

The blockchain allows these concepts to be implemented in a robust and secure way. Eventually, they could be used to help sustain and govern public goods like Open Source projects and the Open Web.

But it's not just Open Source or the Open Web that should be considered. Some of the very biggest problems in the world (for example, climate change) are multi-stakeholder problems that require better funding, coordination and decision-making models too.

So, yes, these are important developments to pay attention to!

Stephan fragte mich, welches VPN er verwenden sollte. Ich bin dafür nicht so leicht zu haben, weil man dann sämtlichen Verkehr über diesen Tunnel abwickelt. Ich bin allerdings sehr dafür zu haben, meinen DNS-Traffic zu tunneln. Die Fritzbox macht das einfach, siehe oben. Man trägt einfach vier Adressen ein:

185.95.218.42
185.95.218.43
2a05:fc84::42
2a05:fc84::43

Dann schaltet man DNS über TLS (DoT) an und im letzten Feld noch dns.digitale-gesellschaft.ch - das war's. Ab dann werden alle DNS-Anfragen verschlüsselt und an einen vertrauenswürdigen Server geschickt. Damit wird Überwachung nicht unmöglich, aber schwieriger.

More >

PS: Ich selbst verwende übrigens Quad9 mit

9.9.9.9
149.112.112.112
2620:fe::fe
2620:fe::9
dns.quad9.net

Warum? Das ist in meinen Benchmarks der schnellste Server und er blacklisted böse Buben.

Everything You Always Wanted To Know About GitHub (But Were Afraid To Ask)

ClickHouse by Yandex is an open source column-oriented data warehouse, designed to run analytical queries against TBs of data. They've loaded the full GitHub Archive of events since 2011 into a public instance, which is a great way of both exploring GitHub activity and trying out ClickHouse. Here's a query I just ran that shows number of watch events per year, for example:

SELECT toYear(created_at) as yyyy, count() FROM github_events WHERE event_type = 'WatchEvent' group by yyyy

Via A Hacker News comment

Over Christmas break I teased some screencaps:

A more refined #rstats #swift "SwiftR" example. Simple Image view + some text views, a color picker and a button that runs

R-in-Swift code (like {reticulate} does for Python in R)

Note no ssd/hd storage round-trip for the plot.

Code snippet: https://t.co/fWaHnztUgd pic.twitter.com/y5m1I16tCB

— Caliban's War (@hrbrmstr) December 28, 2020

Final teaser for #rstats R-in-Swift (SwiftR) since I _think_ I've 'd all memory leaks and refined the example project. The displayed 10,… numbers come right from sample()'d R vector and the fill color live refreshes fast b/c {magick} is magical. Shld have repo up Thu. pic.twitter.com/boGa5ej9FZ

— Caliban's War (@hrbrmstr) December 29, 2020

of some almost-natural “R” looking code (this is a snippet):

Button("Run") {
  do { // calls to R can fail so there are lots of "try"s; poking at less ugly alternatives

    // handling dots in named calls is a WIP
    _  = try R.evalParse("options(tidyverse.quiet = TRUE )")

    // in practice this wld be called once in a model
    try R.library("ggplot2")
    try R.library("hrbrthemes")
    try R.library("magick")

    // can mix initialiation of an R list with Swift and R objects
    let mvals: RObject = [
      "month": [ "Jan", "Feb", "Mar", "Apr", "May", "Jun" ],
      "value": try R.sample(100, 6)
    ]

    // ggplot2! `mvals` is above, `col.hexValue` comes from the color picker
    // can't do R.as.data.frame b/c "dots" so this is a deliberately exposed alternate call
    let gg = try R.ggplot(R.as_data_frame(mvals)) +
      R.geom_col(R.aes_string("month", "value"), fill: col.hexValue) + // supports both [un]named
      R.scale_y_comma() +
      R.labs(
        x: rNULL, y: "# things",
        title: "Monthly Bars"
      ) +
      R.theme_ipsum_gs(grid: "Y")

    // an alternative to {magick} could be getting raw SVG from {svglite} device
    // we get Image view width/height and pass that to {magick}
    // either beats disk/ssd round-trip
    let fig = try R.image_graph(
      width: Double(imageRect.width), 
      height: Double(imageRect.height), 
      res: 144
    )

    try R.print(gg)
    _ = R.dev_off() // can't do R.dev.off b/c "dots" so this is a deliberately exposed alternate call

    let res = try R.image_write(fig, path: rNULL, format: "png")

    imgData = Data(res) // "imgData" is a reactive SwiftUI bound object; when it changes Image does too

  } catch {
  }

}

that works in Swift as part of a SwiftUI app that displays a ggplot2 plot inside of a macOS application.

It doesn’t shell out to R, but uses Swift 5’s native abilities to interface with R’s C interface.

I’m not ready to reveal that SwiftR code/library just yet (break’s over and the core bits still need some tweaking) but I can provide some interim resources with an online book about working with R’s C interface from Swift on macOS. It is uninspiringly called SwiftR — Using R from Swift.

There are, at present, six chapters that introduce the Swift+R concepts via command line apps. These aren’t terribly useful (shebanged R scripts work just fine, #tyvm) in and of themselves, but command line machinations are a much lower barrier to entry than starting right in with SwiftUI (that starts in chapter seven).

FIN

If you’ve wanted a reason to burn ~20GB of drive space with an Xcode installation and start to learn Swift (or learn more about Swift) then this is a resource for you.

The topics in the chapters are also a fairly decent (albeit incomplete) overview of R’s C interface and also how to work with C code from Swift in general.

So, take advantage of the remaining pandemic time and give it a .

Feedback is welcome in the comments or the book code repo (book source repo is in progress).

Hope everyone has a safe and strong new year!

A while back we began looking at how many active members a community should have.

We looked at the number of active members of a few dozen communities and began comparing them by platform, size, maturity, revenue, and a bunch of other characteristics.

Few had a statistically significant impact. But the biggest predictor, by far, was search traffic for the brand.

The more people search for the brand, the more active members a community tends to have.

There is an obvious reason why this is a far bigger predictor than company size, customers, revenue, or any other attribute; it reflects how many people need help.

A thousand customers using a complicated product are likely to have far more questions than the millions of customers eating cornflakes for breakfast every morning.

Other factors play a role too. Where else can the audience get help? Will a community add value these other channels can’t? How big is the ecosystem etc…

But we’ve found search traffic for the brand to be the best predictor by far for active members.

What this also shows is some very large communities are significantly over-performing and some are significantly underperforming.

If you’re looking for benchmarks to compare yourself to others, the relative search traffic to active members of your brand vs. others is a good place to start.

Via Chris Wiegman’s post on leaving big tech, I learned about Hetzner.com’s hosted Nextcloud service.

Most of my work team and many other colleagues are east of me, time zone-wise. This means my mornings are full and I have to make time to get up and move around.

It’s cold and rainy, and nearby Woodland Park is filled with seagulls as I loop around to get some steps in.

Autumm Caines considers the question of 'Zoom fatigue' and traces it to the unrelenting stare of the camera using the analogy of the 'Zoom gaze', a concept that incorporates the idea of being treated as an object, subject to inspection, and required to consider (and adjust) the details of your appearance, behaviour and environment to meet the expectations of others. She borrows from the concepts of Laura Mulvey's “male gaze” and Toni Morrison's “white gaze” to arrive at this metaphor. As she runs through a long list of things Zoom users must take into account while online, two things struck me: first, that no shorter a list of things must be taken into account in personal face-to-face interactions (perhaps even more!), and second, that her list made me think of all the things that I (and probably other introverts) constantly review and think about in any face-to-face encounter (which is why face-to-face leaves us exhausted). So I suggest that maybe Caines isn't describing properties of Zoom videoconferencing, but rather, the properties of (some) Zoom users, and specifically, (what I'll call) 'Zoom introverts'.

The Duke of Data at Simon Fraser University’s City Program Andy Yan suggested it first: if you have Christmas lights up on your condo railing or your abode, why not keep them up longer this year to get through the dark, dull, rainy part of each Metro Vancouver winter. No one will judge you, especially this year.

This idea of bringing in more light in the darkest part of winter is feasible too with the energy efficient outdoor lighting now widely used. And the idea of keeping Christmas lights up (or jazzing them up with colours that are not so directly Christmas festive) has been adopted elsewhere.

The City of Kitchener Ontario’s mayor Berry Vrbanovic is encouraging people to keep their Christmas lights up through January stating: “Seeing our neighbourhoods lit up with lights and decorations has been a wonderful way to feel connected as a community – I love the idea of stretching that festive atmosphere into the new year as we continue to get outside for safe neighbourhood walks and physical activity.”

And in Salem Virginia, residents are urged to keep Christmas lights up to honour the front line healthcare workers through January. This is part of a national campaign urging municipalities and organizations across the United States to keep Christmas lights up until January 31, and to spread the word on social media with the hashtag ##LightsUp4Heroes. In Colorado,the initiative is being embraced state wide.

There is also a historical precedent~in Great Britain, the feasting, decorations, Christmas cakes and puddings used to continue for a much longer period than what we typically do today, in tucking everything away by early January.

The holiday swag would come out for St. Martin’s Day in early November and continue on until Candlemas Day, which is February 2. This was done before central heating and the development of electric light and modern comforts, when the importance of bright decorations and hearty food fuelled people through dark winters.

In a winter where our typical community connections have radically changed, being lit longer may be an idea that has come full circle.

The  YouTube video below from the BBC comedy show  QI (which stands for Quite Interesting) discusses the culture and history of when to take down Christmas decorations in the British historical perspective. That’s actress Carrie Fisher along with host Stephen Fry in the clip.

A new year and time to remind the Province that some initiatives are simple to make movement around places more palatable for more people during the pandemic and after. There is an increase in people wanting to walk, roll and bike and get outside. Regional parks in Metro Vancouver had a 61 percent increase in visits in June of 2020 compared to June of 2019. Sadly access to most regional parks requires a vehicle.

We saw nimble work in some surprising places with  repurposing roads for all street users, with Winnipeg and Calgary leading the way. The City of Vancouver was a little slower in their rollout of “Slow Streets” which referred to streets where walking and cycling were supposedly encouraged by signage for slower vehicle driver  movement.

Sadly the barriers of choice for Slow Streets were very moveable rather light plastic jersey barriers, which of course were scuttled to the side of many of the designated  streets by vehicle drivers, much the way a spent beer can is kicked to the curb when there’s no deposit on the return.

But  look at what Brussels in Belgium has achieved~they have a metro population of 2.5 million (Vancouver’s metro area is 2.46.)

With the extraordinary statement that motorists “should simply no longer feel welcome” the Transportation Minister for Brussels announced that  in the inner city a new 30 km/h (20 mph) speed limit covers the entire city centre. This applies to all streets except for ring roads and some traffic arteries.

In Brussels fifty people annually  die or are seriously maimed by speeding drivers. The intent is to have more rail use and less vehicular traffic in the inner city for air quality purposes , and to reduce vehicular traffic by 33 percent. The goal within ten years is to have the entire city consisting of traffic calmed zones, with more right of way space dedicated to pedestrians, open spaces and bike lanes.

Helga Schmidt in Taggesschau.de points out that enforcement will be achieved by the increased use of speed cameras set up throughout the city. By announcing the intended  measures last year, the agency in charge of mobility in the Brussels-Capital area already has undertaken consultations with transportation businesses, transit operators, police and the public.

Extra funding has been allocated for addressing streets that are still dangerous to vulnerable users, as well as for assistance in processing fines for speeding drivers.

Contrast this with the request of the Union of British Columbia Municipalities  (UBCM) in 2019 who unanimously approved the motion to ask the Province to allow municipalities to be able to make 30 kilometer per hour neighbourhood zones.  UBCM wants to make it easier for municipalities in B.C. to follow the leads of other jurisdictions  in slowing driver speed limits to increase livability in neighbourhoods.

Without the Province giving their assent, cities choosing to have 30 km/h zones have to sign every street in every direction for the reduced speed. Not only is this costly, it also is quite frankly silly, when a simple Provincially approved undertaking for neighbourhood wide 30 km/h speed limits and perimeter signage describes the purpose and does the job.

The Province’s answer? During a pandemic when more people are trying to walk and get outside for mental and physical health? When people are working from home more and need to get outside more and change their patterns? When we are trying to reduce automobile usage and make streets safer? When we could simply do a demonstration project with this because it is the right thing to do?

Nothing. No response from the Province. Crickets.

Meanwhile in Brussels where the city’s road network  already had 60 percent of the network at 30 km/h, the new edict vastly reduces automobile emissions,promotes walking, rolling and cycling and allows the street to be used differently. As Brussels’ Transportation Minister Elke Van der Brandt bluntly put it, “Drivers have to adjust to a new normal” that is inclusive of other users on the streets.

In British Columbia we are still waiting for Provincial permission for cities to even trial this at a neighbourhood level.

You can take a look at the new roll-out for 30 kilometer streets in Brussels in the YouTube video below.

Image:BrusselsTimes

Robin Sloan last month wrote about how newsletters should have seasons like tv shows. Peter Rukavina refers to that in the context of maybe closing up his online shop for letterpress artefacts for a while, something other than a newsletter entirely.

It made me muse about the general application of ‘seasons’ to any type of creative output. Newsletters, knowledge work in general, creation of artefacts, expression. It reminds me of the phases used to describe artist’s lives and work. “She was nearing the end of her blue phase when she met fellow painter X and started experimenting with a new work form.” Van Gogh’s work is described in the ‘Dutch phase’, ‘Impressionist phase’, ‘Arles phase’ and ‘Late phase’, spanning just a decade.

The word season has a rounded pleasant feel to it. Much better than the word phase, which in the context of projects evokes the notions of pre-planned milestones and stress before deadlines. Seasons has a much better fit with things like the natural flow of one’s interests, of (digital) gardening, where there’s a rhythmic change to your activities.

There are internal reasons and external reasons for thinking in terms of seasons for creative production.

Internal ones are about

• building in rest, and treating rest as a fundamental part of your production process (which fits well with my notion of knowledge work as artisanal work).
• an opportunity to reflect (mentioned by Sloan), to step back from the work in progress and take a look at the bigger whole in which it fits
• avoiding the relentlessness that is buried within ‘weekly’, ‘daily’ and other preconceived rhythms, and which always after a while if conceived as ‘endless’ or having an end which is still far away becomes a burden. There is of course the juxtaposed notion of ‘not breaking the chain’. The latter is aimed more at getting the mental satisfaction of keeping up a streak, when the underlying tasks are more of a chore and not likely to provide that satisfaction. With creative production the satisfaction is likely more in the output itself, and then forcing the streak to continue may be counter productive, causing a rut that decreases the fun and satisfaction of production.

External ones

• a sense of progress (mentioned by Sloan), of exploration. An exploration is always a temporary thing, before it morphs into something else again.
• an opportunity to alter course (mentioned by Sloan), e.g. because your list of current interests, or current questions you hold has changed
• a way to change the form of expression, which can bring new inspiration also if themes remain the same. Switching from writing haiku’s to photography, from consultancy to on-line training modules.
• to embrace a natural end point or evolution, providing the ability to let go gracefully not as ‘I’ve quit doing/exploring that’, but ‘I moved to doing/exploring this’. ‘Seasons’ lend themselves well to weaving them into your or other’s narrative.

Those last three fit well with combinational creativity, in all its three varieties of problem driven, similarity driven and inspiration driven approaches.

Seasons by Alphonse Mucha, public domain image, shared by Robson Epsig as CC-BY

For a few people, NetNewsWire for Mac crashes on startup — and the crash log erroneously blames an invalid code signature.

If it were truly invalid, the app wouldn’t launch for anybody. But it launches fine for almost everybody.

This started with macOS 10.15.4 and continues in macOS 11. We’ve posted more details on our issue tracker at GitHub.

Does anybody have any ideas for how to work around this?

The period of time I spent at the end of December consciously not working is one of the first where I wasn’t either (a) explicitly in competition with others, or (b) implicitly in competition with myself.

Competition can be good. It can be motivational and help us strive to be better / faster / stronger. But, too often, it can be damaging and cause us to act in ways that aren’t beneficial to ourselves or those around us.

I’ve been a gamer all my life and so the idea of beating myself (as a kind of ghost car) has always appealed to me. But, having reached the age at which almost every elite athlete has retired, I need to stop kidding myself that I’ll ever run a sub-20 minute 5k. That’s OK.

In addition, I’ve come to understand the approach my mother took to family board games when I was a child. She refused to play to win, instead making sure (as far as she could) that my sister and I never finished last. As a parent, I get that now.

A competitive approach to life is often justified by talking about “preparing young people for the real world”. It’ as if the so-called real world is red in tooth and claw. In my experience that’s not the case; the ‘real world’ is more focused on collaboration than competition.

So, perhaps we’ve got things backwards. Maybe the reason adult life involves competition is not because of the nature of the ‘real world’ but because capitalism demands competition, and so we bake it into childhood.

All of this has made me realise that while competition still has a role in my life, it’s a diminished one. I need to put it back in the box where it belongs, to be taken out where appropriate.

The rest of the time, I should be collaborating, helping bring attention to those who deserve it. That’s instead of (and it pains me to admit it) seeking the reassurance of “doing better” than others. We’re all in this together, after all.

This post is Day 82 of my #100DaysToOffload challenge. Want to get involved? Find out more at 100daystooffload.com.

After the basic installation of Docker and getting a WordPress blog up and running in Docker containers in part 1, this part of my Docker series will take a look at how to add two important features to the setup:

• TLS certificates and LetsEncrypt with auto-renewal
• How to host several websites on one server that are accessible on the same ports (80 for http and 443 for https)

I decided to extend the initial WordPress setup for two reasons. The early example in part 1 is not really suitable for live network deployment as it was missing HTTPs access. And secondly, the power of Docker is to run many independent containers on a single server. On the other hand, especially for web servers, it is important that they are all reachable from the outside on the standard web ports (80 and 443).

Serving Several Web Sites From One Server – The Classic Approach

In a non-Dockerized approach, serving different web sites on the same ports is usually done by using the ‘virtual hosts‘ function on the single web server that hosts all web sites. If three independent WordPress blogs are hosted on that server, the (single) web server would, based on the domain name given during connection establishment, decide from which sub-directory files are served and server side code is executed.

Reverse Proxies

As the Docker approach to web services is to have many independent containers, i.e. one or more containers for each WordPress blog, another approach has to be taken. One way to do this is to use a reverse (web) proxy (in a container) as a front end for port 80 and 443 and then to forward incoming requests to the containers which serve the individual websites. In this setup, 4 web servers are involved: The web server used as reverse proxy and one web server for each WordPress container setup. Without any bells and whistles, that already involves 7 containers: One for the reverse proxy and two each for the WordPress blog (web server and database).

Other Uses of Reverse Proxies

Before going on, here are a few more thoughts on reverse proxies. Content distribution networks and large websites use reverse proxies extensively for a number of tasks. For web sites with lots of traffic, web proxies can receive requests and distribute them to back end web servers where answers are generated. This makes particular sense when generating response pages is computationally intensive. While a single web proxy can forward the requests with little effort, the web servers in the back can do the heavy lifting in a work sharing fashion. Also, the reverse proxy can serve static pages straight away, which further reduces the workload of the backend servers. Another reason for using reverse proxies is to have a single entry point that can be much easier protected against outside attacks than hardening the servers in the background. Also, reverse proxies are used by Content Distribution Networks such as Cloudflare to fend off DDoS attacks to web services by putting themselves ‘in the line of fire’. The downside is, however, that the connection between the reverse proxy and the web server is not encrypted, so the CDN network provider can see all content. For some services that is quite an issue. For our project, not having encryption between the reverse proxy and the backend web servers is less of a problem, as we run the reverse proxy ourselves. So experimenting with reverse proxies is worthwhile not only for understanding Docker setups but to also get an insight into their use for other purposes.

A Reverse Proxy For Serving Different Web Sites With Docker

Let’s get back to our Docker application of serving several web sites through a single front end. This project of Evert Ramos on Github creates such a setup and links to a number of sub-projects that automate the creation of containers for WordPress and other web applications that attach to the reverse-http proxy. When you look closer you’ll see that the reverse proxy does not only consist of 1 but of 3 containers. The first container is for the reverse proxy. The other two ‘companion’ containers manage the automatic creation of new virtual host configurations for the reverse proxy web server and the automatic generation of Letsencrypt certificates when a new dockerized application becomes part of the Docker setup. In other words, a setup with 3 independent WordPress instances involves 9 containers (3 for the reverse proxy and 2 for each WordPress blog).

Trying This Out In The Public Cloud

Unlike for the previous examples I had to go and ‘rent’ a virtual server on the public cloud. This is because I already use ports 80 and 443 of my Internet connection at home for other services so I can’t use them for this setup. Using other ports is not possible, because requesting LetsEncrypt certificates triggers a request from Letsencrypt to port 80 of the server to get a proof of ownership for the domain name for which a certificate is requested. But that’s not much of a problem, as a small Ubuntu server for this purpose and a public IP address can be rented for less than 3 euros a month, for example from Hetzner.

The other thing that is required are of course domain names mapped to the IP address of the server. Fortunately, I already own a number of domains such as wirelessmoves.com, and my DNS provider makes it easy to create new subdomains and link them to an IP address with a few clicks.

Once the server and a few subdomains are in place, installing the reverse proxy via docker-compose with Evert Ramos’ project just takes a few commands:

# Install Docker and Docker-compose as described in the first part 

# Install git
apt install git

# Now clone the github repository
git clone https://github.com/evertramos/docker-compose-letsencrypt-nginx-proxy-companion.git

Once git finishes a new directory is present that contains, among other things, a .env file with many variables that can be modified. The only variable that needs to be changed to get a working installation, however, is the IP address of the server. Therefore, don’t bother to look at all the potential options in the file but just modify the following line:

cd docker-compose-letsencrypt-nginx-proxy-companion/

# copy the file
cp .env.sample .env

# now edit the following line in .env
IP=0.0.0.0

After that, run the script provided by the project to set everything up!

./start.sh

And that’s already it! When you run ‘docker ps’, you will see 3 Docker containers running.

O.k., so that’s the reverse proxy. To be useful, we need at least one service that uses it. So let’s use the WordPress tie-in of the project to create a WordPress instance that is accessible via the reverse proxy:

cd ~
git clone https://github.com/evertramos/docker-wordpress.git

mv docker-wordpress docker-wordpress-1
cd docker-wordpress-1

cp .env.example .env

# Change the following lines in the .env file
COMPOSE_PROJECT_NAME=new-site
CONTAINER_DB_NAME=new-site-db
CONTAINER_SITE_NAME=new-site-site
DOMAINS=domain.com,www.domain.com  # For the rev. proxy and LetsEncrypt!!!

# Note: For experimenting you can leave the DB usernames/pwds as they
# are. For production. You should obviously change them...

# Now pull all images, configure containers and start them
docker-compose up -d

And again, that’s it, the WordPress blog is up and running after half a minute or so! It takes a few seconds because the first time around the LetsEncrypt container of the reverse proxy has to request a certificate for the domain name given above.

You can observe progress by following the logs of the Letsencrypt container as follows:

docker logs nginx-letsencrypt --follow

The Second WordPress Blog Behind the Reverse Proxy

O.k. that’s nice, we have 5 running containers now and the WordPress instance can be reached over an https connection via its domain name. So now let’s be bold and bring up another WordPress blog with a different domain name (that was also tied previously to the IP address of the server). Just repeat the steps above but with different names for the folder, container and domain names and run docker-compose up -d in the other folder.

‘docker ps‘ now shows 7 containers. The WordPress instances are reachable only on the inside on port 80 and only the nginx container for the reverse proxy is reachable from the outside via port 80 and port 443.

Two Database Containers To Choose From…

And one more thing to have a look at in this post: Each WordPress project has its own database container with a separate IP address. In both database containers the database is reachable on port 3306. So how does each WordPress instance know to which (container) IP address to talk to? The answer lies in the docker-compose.yml file in the project directory. Here, the WORDPRESS_DB_HOST environment variable points to the container name of one of the database containers which we configured differently in the .env file CONTAINER_DB_NAME variable for each WordPress installation.

Summary

By now we have a setup with 7 containers to serve two WordPress Blogs with domain names and LetsEncrypt certificates. Installed with just a few commands and setting some variables, we didn’t have to deal with any kind of web server or proxy configuration at all. Also, we had to have no background information of how to install WordPress. Adding more WordPress sites takes only little additional effort and shows the power of the containerized approach for deploying many similar or identical services.

But the story is far from over. In the next part of this series, I’ll have a look at the following things:

• Move the WordPress installation from part 1 from my local server to this public cloud server with the reverse proxy setup. This demonstrates the flexibility of containers.
• How does the reverse proxy part of the setup notice that a new internal server is started and for which domain name a Letsencrypt certificate should be requested?
• What is the chain of trust for the remote proxy setup, i.e. whom do I have to trust to keep this installation secure (in addition to the folks at nginx, WordPress and Docker)?

Im Zusammenhang mit dem Fernunterricht und den damit auftretenden Schwierigkeiten will ich mal aus 10.000 Metern Höhe zeigen, wo eigentlich das Problem liegt. Ich vereinfache, damit jeder mitkommt, auch wenn er es nicht so mit der IT hat.

Fangen wir mal dort an, wo jeder versteht, um was es geht. Man kann sich einen Server hinstellen, ein Linux-Image drauf ballern und "gegen Hacker" härten. Darauf packt man dann die erforderliche Software. Oder man installiert einfach ein Image, wo das alles vorinstalliert ist. Wenn der Server zu klein ist, dann macht man mehr Speicher rein oder einen schnellen Prozessor. Das nennt man Scale-Up - in die Höhe wachsen.

Das scheitert irgendwann, dann stellt man einen zweiten daneben. Das nennt man Scale-Out - in die Breite wachsen. Sowas lässt sich weiter ausbauen, so lange man genug Geld hat, immer neue Hardware auf das Problem zu werfen. Aber man bekommt ein neues Problem: Man muss diese ganzen Server koordinieren, so dass sie auf gemeinsame Daten zugreifen und man muss sie außerdem wie einen aussehen lassen. Ohne irgendeinen technischen Einblick würde ich bei unseren von den Ländern bereitgestellten Lernplattformen genau dort den Hund begraben vermuten. Das geht nicht mit allen Schülern gleichzeitig und immer. Egal wieviel Hardware dazu gebaut wird.

Server zu beschaffen kann Monate dauern, besonders wenn sie größer (Scale-Up) sind. Das sind nicht einfach PCs in einem Schrank. In der Regel wird sowas auf Bestellung produziert.

Diesen Ärger kann man sich vom Hals schaffen, indem man die Kapazitäten mietet, so wie man sie braucht. Infrastructure-as-a-Service (IaaS) nennt man das. Mit der Hardware hat man nichts mehr zu tun. Dickster Anbieter ist Amazon Web Services.

Es bleibt das Koordinierungsproblem: Wenn ich zum Beispiel eine Lernumgebung hosten will, dann kann der Engpass irgendwo liegen. Im Frontend kann man Proxy-Server und Web Server dazu schalten, aber im Backend mit den Daten wird das schwierig bis unmöglich. Dann muss ein Software-Architekt ran, einen neuen Plan machen, der dann umgesetzt und immer wieder angepasst wird.

Den Ausweg aus dieser Misere bekommt man mit Platform-as-a-Service (PaaS). Das wird sofort deutlich teurer, denn irgendjemand muss ja nun mehr machen als einfach Kisten an den Strom und das Netz zu hängen. Bei PaaS gehört dir nur noch die Anwendungssoftware, alles andere macht der Anbieter. Und damit sieht es nur dem ersten Anschein nach teurer aus, weil der Anbieter das besser kann als du. Ich überspringe das Thema mal, weil PaaS unser Problem nicht löst.

Die Königsdisziplin ist Software-as-a-Service (SaaS). Beispielsweise Microsoft Teams. Du konfigurierst gar nichts. Du meldest "einfach" 1 Millionen User an und lässt Microsoft alle anderen Probleme lösen. Kein Scale-Up oder Scale-Out für dich. Microsoft nutzt sich dabei selbst, in dem sie SaaS mit ihrem eigenen PaaS realisieren, das wiederum auf ihrem IaaS läuft. Von alldem musst du nichts verstehen. Du gehst in dein Teams, machst ein Meeting auf und gut ist.

Ich habe das jetzt mal bewusst einfach gehalten. Teams ist selbst auch wieder eine Plattform, auf die man mehr bauen kann, etwa die Education-Angebote. Und Google hat dasselbe auch, nur in anderer Geschmacksrichtung.

Man darf sich das nicht so vorstellen wie den Server in der Arztpraxis oder im Rechnerraum der kleinen Firma, wo ein Admin mit Systemrechten Software installiert, Backup-Datenträger handhabt, kurz schalten und walten kann, wie er will. Das ist eine vollautomatisierte Fabrik, die niemals stillsteht und in der jede kleinste Handhabung protokolliert wird. Das muss revisionssicher sein, so dass alles nachvollziehbar ist, es muss zertifiziert werden, immer wieder geprüft und abgenommen. Alles ist mehrfach vorhanden, über mehrere Rechenzentren verteilt und wenn irgendwo was umfällt, dann darf das keinen Einfluss auf die Fabrik haben.

Aber, aber, die bösen Amis ... kann man das nicht auch selbst so machen? Ja, kann man. Jeder kann eine Autofabrik bauen, wenn er sich alles kauft, was man dazu braucht. Es dauert nur ein Jahrzehnt. Und das ist ungefähr die Zeit, die wir zwischen 2010 und 2020 verloren haben. Es reicht nicht, eine Idee zu haben. Ideen sind billig. Die Umsetzung ist schwierig. Und es dauert sehr, sehr lange, bis man die Kosten optimiert kriegt. Und selbst wenn man einen Morgan Sportwagen über Jahrzehnte baut, dann sind die noch immer teuer und man muss den Motor irgendwo einkaufen.

Die mit Abstand teuerste Methode, sowas zu bauen, ist klein-klein. Und wenn es dann mal steht, dann muss es dauernd repariert werden. Einmal hinrotzen reicht nicht.

Those of us that have scripts periodically querying sites like fccid.io and safetykorea.kr may have noticed new FCC registrations from Microsoft that popped up within the last 24 hours. The registrations only include photos of unsurprising modular radio components for LTE (Quectel EM12GPA-512) and Wi-Fi 6 (Intel AX201). Given last year’s FCC registrations, I expect these to be from the upcoming Surface Pro 8:

What is interesting is that Microsoft goofed on the FCC registrations by posting Wi-Fi photos for the LTE registration and LTE photos for the Wi-Fi registration. With the FCC rubber stamp process, its not surprising this error slipped through. Also, unlike prior LTE-enabled models like the Pro X and Pro 5, the LTE module is a B-keyed M.2 2242 form factor. Does this mean that non-LTE Pro 8 models will have an-populated M.2. slot that can be used for an extra SSD? Given the limited bandwidth requirements of LTE, only USB 2.0/3.x is necessary and is what the Quectel devices use. So we’ll have to wait to find out if the Pro 8 motherboard routes 2x PCIe lanes or SATA to the socket. Given Microsoft’s push towards modularity in recent launches, I’m hopeful.

I’m also hopeful for tremendous performance. Geekbench results of a design verification (DV) model were posted last month showing significant improvements compared to Pro 7 CPU/GPU performance with Intel’s RocketLake platform. With Windows 10 21H1 and device driver updates, hopefully we’ll see an even higher performance bump when the Pro 8 launches.

References:

• Surface Pro 8 and Laptop 4 coming soon per Korean certifications

I bought my Skipper Otto 2021 membership today. It’s a Community Supported Fishery (CSF) – you buy credits and order seafood through out the year.

I haven’t found the right CSA in Vancouver yet, but that’s another goal for the year. Suggestions?

I am making many mistakes this year, almost continuously. It’s a good thing. I am learning two new skills right now that I am bad at. And keep in mind, I am doing these things at age 62. First off, my family received the gift of a 100-year old grand piano from a family friend. … Continued

The post Making mistakes appeared first on without bullshit.

I think it's pretty certain that post-Covid classrooms will employ a lot more technology than they used to. Even if nothing else changes, digital content is set to become mainstream. But what about all technology? It won't be equally welcomed. Here, for example, is a call to record in-class sessions. Personally, I would think it's fine for most cases, but the same people who object to surveillance technology in learning will likely object to class recording, and for similar reasons. Michael Petrilli says, "no doubt some teachers won’t be thrilled to know that their administrators could also be watching them—and evaluating their performance—any time they like." The same applies to students, perhaps even more so, especially is analytics are applied to the live feed.

DALL·E: Creating Images from Text

"DALL·E is a 12-billion parameter version of GPT-3 trained to generate images from text descriptions, using a dataset of text–image pairs.". The examples in this paper are astonishing - "an illustration of a baby daikon radish in a tutu walking a dog" generates exactly that.

Via Hacker News

snbl schrieb mir:

hab deinen artikel über secure dns gelesen - fand ich gut! falls du eine idee für einen weiteren artikel dazu brauchst: unter iOS fehlt ja ein bequemer menüeintrag zum eintragen des dns wie unter android. es gibt/gab dazu zwar apps aber wenn man diesen nicht vertraut, kann man das auch selber machen.

Wozu? Man kann sicheres DNS über TLS oder HTTPS auch mit einem iPhone machen, wenn man unterwegs ist.

hab dazu exemplarisch zwei profile gebaut die bei manueller installation sowohl requests via wifi als auch cellular abdecken (bei einer mdm installation gilt das offenbar nur für managed wifi verbindungen). nach der installation kann via systemeinstellungen → general → vpn & network das gewünschte profil ausgewählt werden (automatic übernimmt die default dhcp/schnittstellen einstellungen).

Ich habe snbl vorgeschlagen, seine Beispielprofile auf github abzulegen, weil er sie dann weiterentwickeln kann. Da gehören sie hin und nicht auf meine Website.

Stellt sich raus, da sind sie schon.

Verbesserungsvorschlag: Wenn man im eigenen Netz ist, dann will man wahrscheinlich auch die lokale Namensauflösung nutzen. Da kann man die Profile noch weiterentwickeln.

More >

The latest earbud design coming out of Bose is a new take on the gadget that tries to balance exterior sound with your music.

The new earbuds are called ‘Bose Sport Open Earbuds,’ and they sort of hook around the rear of your ear and then rest just inside your ear lobe, leaving your ear canal unobstructed so you can hear the outside world and your music simultaneously.

While many headphones offer a passthrough mode that allows users to artificially hear the outside world through their headphones, the new option from Bose promises to be more natural sounding since both the music and ambient room noise are naturally picked up by your ears instead of being converted through mics and speakers.

Bose says that this setup works just as well as regular headphones and doesn’t compromise privacy, so people can’t hear what you’re listening to.

The Sport Open Earbuds have an eight-hour battery life, and the included case doesn’t charge them. Instead, you’ll need to take them home and plug them into an included charging dock to recharge them. It takes three hours to recharge these buds.

The earbuds cost $249 in Canada and you can pre-order them starting today. The Bose website says they’ll begin shipping on January 20th.

Bose is lucky that it priced these headphones so low since I can’t see people picking them up as their everyday headphones since they don’t come with a charging case. That said, for $250, they could be a nice splurge if you spend a lot of time working out or doing some other activity that requires you to hear your surroundings.

I can vouch for the concept of these headphones being useful. I do like the ability to hear my surroundings when I’m biking and I found the passthrough mode on the Galaxy Buds+ and the cool semi-in ear design of the Skullcandy Push Ultra to both be good options.

That said, if Bose has really stumbled onto a way to make the new SportOpen Earbuds sound fantastic, they might just blow the competition out of the water.

Source: Bose 

The post New Bose earbuds are tiny speakers that sit above your ear appeared first on MobileSyrup.