Last week, I shared my State of Drupal presentation at Drupalcon Nashville. In addition to sharing my slides, I wanted to provide more information on how you can participate in the various initiatives presented in my keynote, such as growing Drupal adoption or evolving our community values and principles.

Drupal 8 update

During the first portion of my presentation, I provided an overview of Drupal 8 updates. Last month, the Drupal community celebrated an important milestone with the successful release of Drupal 8.5, which ships with improved features for content creators, site builders, and developers.

Drupal 8 continues to gain momentum, as the number of Drupal 8 sites has grown 51 percent year-over-year:

Drupal 8's module ecosystem is also maturing quickly, as 81 percent more Drupal 8 modules have become stable in the past year:

As you can see from the Drupal 8 roadmap, improving the ease of use for content creators remains our top priority:

Four ways to grow Drupal adoption

Drupal 8 was released at the end of 2015, which means our community has had over two years of real-world experience with Drupal 8. It was time to take a step back and assess additional growth initiatives based on what we have learned so far.

In an effort to better understand the biggest hurdles facing Drupal adoption, we interviewed over 150 individuals around the world that hold different roles within the community. We talked to Drupal front-end and back-end developers, contributors, trainers, agency owners, vendors that sell Drupal to customers, end users, and more. Based on their feedback, we established four goals to help accelerate Drupal adoption.

Goal 1: Improve the technical evaluation process

Matthew Grasmick recently completed an exercise in which he assessed the technical evaluator experience of four different PHP frameworks, and discovered that Drupal required the most steps to install. Having a good technical evaluator experience is critical, as it has a direct impact on adoption rates.

To improve the Drupal evaluation process, we've proposed the following initiatives:

Initiative	Issue link	Stakeholders	Initiative coordinator	Status
Better discovery experience on Drupal.org	Drupal.org roadmap	Drupal Association	hestenet	Under active development
Better "getting started" documentation	#2956879	Documentation Working Group	grasmash	In planning
More modern administration experience	#2957457	Core contributors	ckrina and yoroy	Under active development

To become involved with one of these initiatives, click on its "Issue link" in the table above. This will take you to Drupal.org, where you can contribute by sharing your ideas or lending your expertise to move an initiative forward.

Goal 2: Improve the content creator experience

Throughout the interview process, it became clear that ease of use is a feature now expected of all technology. For Drupal, this means improving the content creator experience through a modern administration user interface, drag-and-drop media management and page building, and improved site preview functionality.

The good news is that all of these features are already under development through the Media, Workflow, Layout and JavaScript Modernization initiatives.

Most of these initiative teams meet weekly on Drupal Slack (see the meetings calendar), which gives community members an opportunity to meet team members, receive information on current goals and priorities, and volunteer to contribute code, testing, design, communications, and more.

Goal 3: Improve the site builder experience

Our research also showed that to improve the site builder experience, we should focus on improving the three following areas:

• The configuration management capabilities in core need to support more common use cases out-of-the-box.
• Composer and Drupal core should be better integrated to empower site builders to manage dependencies and keep Drupal sites up-to-date.
• We should provide a longer grace period between required core updates so development teams have more time to prepare, test, and upgrade their Drupal sites after each new minor Drupal release.

We plan to make all of these aspects easier for site builders through the following initiatives:

Initiative	Issue link	Stakeholders	Initiative coordinator	Status
Composer & Core	#2958021	Core contributors + Drupal Association	Coordinator needed!	Proposed
Config Management 2.0	#2957423	Core contributors	Coordinator needed!	Proposed
Security LTS	2909665	Core committers + Drupal Security Team + Drupal Association	Core committers and Security team	Proposed, under discussion

Goal 4: Promote Drupal to non-technical decision makers

The fourth initiative is unique as it will help our community to better communicate the value of Drupal to the non-technical decision makers. Today, marketing executives and content creators often influence the decision behind what CMS an organization will use. However, many of these individuals are not familiar with Drupal or are discouraged by the misconception that Drupal is primarily for developers.

With these challenges in mind, the Drupal Association has launched the Promote Drupal Initiative. This initiative will include building stronger marketing and branding, demos, events, and public relations resources that digital agencies and local associations can use to promote Drupal. The Drupal Association has set a goal of fundraising $100,000 to support this initiative, including the hiring of a marketing coordinator.

Megan Sanicki and her team have already raised $54,000 from over 30 agencies and 5 individual sponsors in only 4 days. Clearly this initiative resonates with Drupal agencies. Please consider how you or your organization can contribute.

Fostering community with values and principles

This year at DrupalCon Nashville, over 3,000 people traveled to the Music City to collaborate, learn, and connect with one another. It's at events like DrupalCon where the impact of our community becomes tangible for many. It also serves as an important reminder that while Drupal has grown a great deal since the early days, the work needed to scale our community is never done.

Prompted by feedback from our community, I have spent the past five months trying to better establish the Drupal community's principles and values. I have shared an "alpha" version of Drupal's values and principles at https://www.drupal.org/about/values-and-principles. As a next step, I will be drafting a charter for a new working group that will be responsible for maintaining and improving our values and principles. In the meantime, I invite every community member to provide feedback in the issue queue of the Drupal governance project.

I believe that taking time to highlight community members that exemplify each principle can make the proposed framework more accessible. That is why it was very meaningful for me to spotlight three Drupal community members that demonstrate these principles.

Principle 1: Optimize for Impact - Rebecca Pilcher

Rebecca shares a remarkable story about Drupal's impact on her Type 1 diabetes diagnosis:

Principle 5: Everyone has something to contribute - Mike Lamb

Mike explains why Pfizer contributes millions to Drupal:

Principle 6: Choose to Lead - Mark Conroy

Mark tells the story of his own Drupal journey, and how his experience inspired him to help other community members:

Watch the keynote or download my slides

In addition to the community spotlights, you can also watch a recording of my keynote (starting at 19:25), or you can download a copy of my slides (164 MB).

Before I left for ISA 2018 and AAG 2018, I purchased a ton of books. I have been doing way more work on waste (not only human manure but also municipal garbage) and while my water library is spectacularly well populated, I didn’t have enough books on waste, so my poor credit card took a big hit and I started purchasing a ton of books that I thought I might need. Along the way, I found a few books on academic writing that were inexpensive and that I thought would make the shipping costs worth it.

Yes, I admit it: I buy academic books sometimes as “order padders” so when I pay for shipping I don’t feel as bad. So, anyway, I wanted to read William Germano’s book for new authors and thus I purchased it (”Getting It Published: A Guide for Scholars and Anyone Else Serious About Serious Books“). This book is not for authors who are PhD graduates and want to revise their dissertations as books. For that purpose, Germano wrote “From Dissertation to Book“, which I’ve also written about here on my blog.

This is my concluding tweet from a long-ish thread on Germano’s book. I think this is an endorsement if there’s ever one.

Bottom line: purchase both @WmGermano 's books, and do as I suggested: use one as reference (Getting It Published) and the other one to revise your PhD dissertation into a book (From Dissertation to Book) #AcWri #PhDChat #GetYourManuscriptOut </thread>

— Dr Raul Pacheco-Vega (@raulpacheco) April 14, 2018

I usually don’t endorse books, but I found William Germano’s books so useful I really learned A LOT from them. In my Twitter thread (which you can read in its entirety by clicking anywhere on the tweet shown below) I embedded recommendations of other academic writing books that I’ve read.

.@WmGermano 's first book and this one are both published by @UChicagoPress – Getting It Published is for NEW books, not revising PhD theses pic.twitter.com/J88jkpokTC

— Dr Raul Pacheco-Vega (@raulpacheco) April 14, 2018

This book is intended to demystify the book writing and publishing process. I find Germano's writing quite fluid and accessible. pic.twitter.com/9PS0WbqxeV

— Dr Raul Pacheco-Vega (@raulpacheco) April 14, 2018

Contrary to "From Dissertation to Book", in "Getting It Published" Germano offers a handbook. A guide. A reference volume to "get back to" pic.twitter.com/38eMxlXwYL

— Dr Raul Pacheco-Vega (@raulpacheco) April 14, 2018

As Miles says, cited by Germano, we should aim to explain the field to a broader audience – the first being scholars in other fields. pic.twitter.com/au7vbV9lJA

— Dr Raul Pacheco-Vega (@raulpacheco) April 14, 2018

Here is where you'll find all sorts of disagreement. Depending on the press, you will find that they will like more jargon, less jargon. pic.twitter.com/vXl3xAyZVG

— Dr Raul Pacheco-Vega (@raulpacheco) April 14, 2018

Germano's list of what editors look for totally makes sense to me and it's also the advice I've received from my editors (and friends) pic.twitter.com/LqtK5t39se

— Dr Raul Pacheco-Vega (@raulpacheco) April 14, 2018

These are 4 gems on editing a book (as an edited collection) and why you shouldn't do it. Regardless, I've edited two books, I'd do it again pic.twitter.com/46aZAHy4TH

— Dr Raul Pacheco-Vega (@raulpacheco) April 14, 2018

If you go by 3GPP and common sense it can be expected that most operators having an LTE network today will launch 5G as what is referred to as 5G EN-DC (eUTRAN New Radio – Dual Connectivity), a.k.a. 5G NR ‘option 3’ in 3GPP circles. In other words, mobile devices will still camp on the existing 4G LTE eNB base stations and 5G gNB resources will be added when required. This makes sense in many cases as 5G will often be deployed on higher frequency bands and also not everywhere at first. Hence, the idea is to use the LTE network as a coverage layer and add 5G to the connection when available. Also, this has the advantage that no 5G core network (5GC) is required at first. But where do we go from here as 5G coverage gets more widespread and operators start using a 5G core network in addition to the existing 4G EPC?

In addition to ‘option 3’, which is the first version of 5G specified in 3GPP Release 15, 3GPP is the in the process in specifying additional options to connect 4G and 5G radio networks to 4G and 5G core networks. This slide set gives a good overview of the different options.

All options (except option 3) shown in the slide set only focus on how 5G capable mobile devices will connect to the network in the future. But what about ‘legacy’ 4G LTE devices? How will they connect to the network once the RAN is connected to a 5G core network?

From my point of view the answer is that in practice, network operators will use several 5G options to connect the radio and core networks simultaneously, i.e. the LTE eNBs and the 5G gNBs are each connected to the 4G LTE EPC core network and also to the 5G core network (5GC). This interesting whitepaper from the GTI shows how such a multi-option configuration could look like in practice on page 12 in figure 8.

By connecting the 4G radio network to the 4G and 5G core and also the 5G radio network to the 4G and 5G core, it will be possible to support several 5G connectivity flavors at the same time. Here are a few examples:

Support of Option 3, i.e. 4G is the coverage layer, 5G added in Dual Connectivity Mode: This is done via the connection of the 4G eNB to the 4G core and the 5G gNB connection to the 4G eNB.

Support of Option 4, i.e. 5G provides the coverage layer and 4G is added via Dual Connectivity for capacity and speed. In this scenario the mobile device (UE) must not only be 5G EN-DC (option 3) capable but must also support native 5G connectivity and connect to the 5G gNB directly. The 5G gNB is connected to the 5G core network which has a different signaling protocol than the 4G core network so the UE must be capable of communicating with the 5G core. Early ‘option 3-only devices might not support this and might not be able to support option 4 deployments. Note the ‘might’ in the sentence as this is just a wild guess at this point. This configuration makes sense when 5G is deployed on a low frequency band, e.g. the 700 MHz band in Europe. This way, 5G could provide the coverage layer and 4G could be added to increase data rates (e.g. 1800, 2600 MHz) when available.

Support of Option 7, i.e. the 4G eNB is also connected to the 5G core network and the 5G gNB is added in Dual Connectivity mode to improve speed. This makes sense in areas where the LTE radio access network continues to be used for coverage on a lower band and 5G for speed on a higher band. From a system perspective this makes a difference compared to option 3 because it makes it easier for the system to hand-over a UE between 4G and 5G cells. This is because the UE always communicates to the 5G core instead of switching between the EPC and the 5GC. This also means less NAS signaling and fewer data path changes in the network.

And, not to forget, there’s also option 2, i.e. the 5G gNB RAN is standalone and a mobile device communicates only to the gNB, there is no dual connectivity to 4G. To me, this option only makes sense if there is 5G coverage in low and high bands to provide both coverage and capacity so the network is no longer required to add LTE resources to a UE that is being served from a 5G gNB. From my point of view this scenario is even further in the future than the other options.

4G/5G handovers: Once 5G UEs support option 2 and 4, i.e. they camp on the 5G gNB rather than the 4G eNB, it will also be necessary to support 5G to 4G and perhaps also 4G to 5G handovers on mobile devices and also in the network while 4G network coverage exceeds 5G coverage.

The important point in all of this is the following: A network operator is not restricted to only deploying one 5G option in its live network. An operator might start with option 3 and then, after some time, adds a 5G core network. After that, more and more spectrum will be moved from 4G LTE to 5G NR over time as the number of 5G capable devices grows. At the same time mobile devices will evolve and will not only be able to talk to a 4G core network and thus be limited to option 3 but will also implement the 5G core network protocol stack. As a result, it will then be able to make use of the all options which are deployed in a network simultaneously.

Theo in FaireChild RainCoat and Pants

I don't know if you've heard but it rains a lot in Vancouver (huge understatement). My kids live in rain gear in all seasons and I find most gear can't stand up to that much wear and it's often ugly. So I was thrilled to find a company that is making beautiful rain gear that is made to last and sustainably made, in Canada no less! In speaking with the team at Faire Child I feel like I've gotten an education in outerwear and the possibilities in sustainable clothing. Aesthetically speaking the rain pants are the cutest I've ever seen and the adjustable straps are genius. I had trouble deciding between the rain coat and the anorak as they are both equally adorable. Theo is going to get years out of the pants and coat and I don't doubt they will stand up to Mae wearing them for years after. 

Also the material is so light and soft which I didn't expect from waterproof clothing. Trevor grew up in Prince Rupert which sees more rain than Vancouver, so he is very specific about outdoor gear. We've often disagreed on outerwear as I want it to look nice and Trevor just wants it to function. This is the first time we have agreed on outerwear for the kids! Trevor was especially impressed by the sealed seams (something that I wouldn't have noticed). We spent a rainy day at the beach recently where I captured Theo playing in Faire Child. Theo is wearing the 5/6 Rain Coat and Pants. He is small for his age, still fitting size 3/4 clothes at 4.5 years old. 

The women behind Faire Child kindly offered to answer some questions below about their company. The care and attention to detail that has gone into the creation of this line is truly inspiring. Thank you to Beth, Alissa and Tabitha for sharing and creating something so functional and beautiful for our kids!

Theo on the wet sand in his Faire Child Rain Pants

I'm often complaining about the lack of beautiful, classic, well-made rain gear for kids. But instead of complaining about it, you solved the problem! How did you get the guts to start your own company?

Once I became a mother, I felt I had the guts to do anything! The act of giving birth was the most challenging and rewarding thing I have ever done. My now-toddler reminds me daily that there is no right or wrong way to do something. I admire her persistence and unending willingness to try new things. It altered my once crippling view of perfection and made me much more accepting of failure, thus allowing me to be more creative and free. Within a year of having my daughter, I was on the path to starting the business I had always wanted. 

Can you tell us a bit more about your unique fabric and manufacturing processes as I know this is a key component for your company? 

So glad you asked! We are huge champions for the fabric we use and for the innovative textiles being made by Sympatex. I discovered the fabric through the Sustainable Angle Fashion Expo in the UK. Sympatex has taken advantage of the unique properties of PET, commonly used in water bottles, and harnessed them into an amazing textile.

As far as the manufacturing process goes, after the PET bottles have been recycled, Sympatex processes them by crushing the bottles into small bits that are then reformed into a yarn. These yarns are woven into a textile that is 100% recycled, 100% waterproof, windproof, breathable, soft and lightweight and moisture wicking. There are also Bluesign and Oeko-tex certified which means no harmful chemicals have been used to make the fabric. Using rPET, or recycled polyester, as opposed to traditional virgin polyester, helps divert waste from landfills and reduces our dependency on petroleum as a raw material. It takes 19 average water bottles to make a Faire child rain coat in size 5/6.

The end result is a fabric with amazing properties. We are confident that children can spend the entire day outside, comfortable, in any season, in this fabric. The real kicker is that the fabric is also recyclable, so it never has to end up in a landfill. We have set up a take back program so that after many, many years of use the garment can be returned to us and we take responsibility for recycling it.

If you want to get into the real nitty gritty of the fabric’s eco-impact – The rPET (recycled polyester) fabric we use from Sympatex has a 32% reduction carbon emission, a 94% reduction in water use and a 50% reduction in energy use when compared to virgin polyesters.

If you’re super interested in even more details about our manufacturing process, please take a look at our website – https://fairechild.com/pages/about-the-fabric

 

How was starting your own Kickstarter campaign? Any advice for anyone considering doing the same?

Running a Kickstarter Campaign was definitely full of ups and downs! Going into it, we were a little naïve, but we were so overwhelmed by the amount of support we received. Designing a thoughtful campaign and having a good quality video was paramount for us in making sure we communicated our message effectively. As a new company, we wanted to ensure our first impression was a good one. 

If we were to give advice to anyone on the fence about doing Kickstarter – it is definitely a huge benefit to have strong networks around you and people who are willing to share your campaign. Especially having likeminded organizations and individuals around you who support and share your project. It can be feel awkward and presumptuous reaching out to people and asking for help, especially being so new. Having said that, the response was incredible, and we are so happy with the outcome.

One other thing to be mindful of – those Kickstarter fees and payment processing fees – yowza!

I loved a recent post you did about how to layer Faire Child with other clothing to make it work for all seasons, even winter! Living in a small space, I love the concept of having less gear that works all year round. Could you share more about that here? And share some of your favourite brands for layers? 

Yes! We are huge evangelists for layering. It really makes such a huge difference. By learning a few basics of the unique properties of various materials you can use them to your advantage. Our rain gear can be used year-round as an outer layer and by understanding a few layering principles you will know how to get the most use out of your garments.

We aren’t huge fans of traditional outerwear, specifically snow gear, because they don’t actually keep you warm and dry for that long. In a way, they are trying to be a three in one - the base, insulating and outer layer – and not really doing a great job at any of them.

In the fall and winter, we advocate for wearing a base and insulating layer. Fabrics like merino wool are breathable and wick away moisture. They are really great as a base layer in the winter. A fabric like cotton shouldn’t be worn as a base layer in colder temperature because it soaks up moisture, instead of wicking it away, and so when it gets wet it stays wet and makes you cold. Our favourite brands for base layers are Luv Mother, Simply Merino or Ruskovilla (ed. note yes, we love Simply Merino too! Theo is wearing their top in these photos).

Fleece, wool and down are great insulating layers for chilly weather. It’s best to pick sweaters that aren’t too bulky. We love the wool overalls made by Disana and Mini Mioche has great fleece insulating layers, too.

As temperatures get warmer in the spring and summer you can ditch that insulating layer and use cotton and linen for your base layer. Red Creek Kids has adorable linen garments and we love the organic cotton in the Knit Essentials Collection by Petits Vilains (ed. note: agreed! PV is a favourite for us too!). So thankful for so many amazing Canadian-made options!

I know we are fellow Canadians but I'm pretty sure the weather is a bit more extreme on the East Coast (correct me if I am wrong?). What are your best tips for getting outside with the kids despite the weather?

We definitely get our fair share of wet, cold and windy weather – and often all at once! Our winters and springs are so humid that you have to take extra precautions to keeping that chill away. And Nor’easters are definitely a thing. The saying ‘If you don’t like the weather, wait a few minutes’ truly applies to Nova Scotia!

Proper clothing – for yourself and your children -  really does make a huge difference for getting outside with the kids. Obviously, in that rainy weather, having waterproof outerwear with sealed seams is key. Sealed seams, while super common in high-end adult rainwear, are extremely rare when it comes to children’s outerwear. Likely because it is costly to manufacture, and kids tend to get new jackets every season. To that end, we’ve tried to create garments that can grow with your child and which are durable and unisex so they can be passed from child to child. Our outerwear can also be layered up or down, so it is useful for all seasons.

Between us at Faire Child we have a couple rambunctious toddlers who love to be outside. We have found that keeping it simple is key – it doesn’t have to be a huge trek or ordeal. It can be a neighborhood stroll, building a snowman in the backyard or walking to the closest park. It’s the fresh air and not rushing that feels so rejuvenating about the outdoors for us and our kiddos. Make it an adventure – you’ll get those imagination muscles working too!

We try to make sure to layer up properly and wear cozy hats, mitts, wool socks and waterproof booties. We’ve found that once those hands or feet get cold, it’s game over in about ~2 seconds.

What is the best part of running Faire Child?

The luxury and privilege it is to be able to run my own company. I’m super aware of how lucky I am to be able to do this and while it can definitely keep me up at night sometimes, it is the most rewarding opportunity.

What is the worst part ;) ?

It’s not the worst part, but definitely the most difficult part has been holding Faire Child to a very high standard in terms of sustainability. We wanted to make sure what we were creating for the next generation was truly not just less bad, but more good. Often seemingly small decisions become huge hurdles, like finding compostable shipping labels and poly bags. We’ve had a few setbacks as a company because it is just so important to us that when we put something out into the world, it all speaks the same language. The marine grade sliders on the backpack are recyclable, the snaps made from re-used brass and the elastic, thread, webbing, etc, have all been painstakingly sourced. Many sustainable options are brand new or still in the middle of research and development which can create delays - we are so excited for a day when these will be the only options available! 

Look for a giveaway with Faire Child on my Instagram this week! 

 

“And you know what? After a week or ten days or so, my facebook feed started giving me the same feeling as daytime TV. … I stopped watching TV years ago.” says Stephanie Booth. Very recognisable.

[Please note that the code in this blog post is now up on PyPI as part of the modutil library]

One of the new features in Python 3.7 is PEP 562 which adds support for __getattr__() and __dir__() on modules. Both open up some interesting possibilities. For instance, with __dir__() you can now have dir() only show what __all__ defines.

But being so immersed in Python's import system, my interest lies with __getattr__() and how it can be used to do lazy importing. Now I'm going to start this post off by stating that most people do not need lazy importing. Only when start-up costs are paramount should this come into play, e.g. CLI apps that have a short running time. For most people, the negatives to lazy loading are not worth it, e.g. knowing much later when an import fails instead of at application launch.

The old ways

Traditionally there have been two ways to do lazy/delayed importing. The oldest one is doing a local import instead of a global one (i.e. importing within your function instead of at the top of your module). While this does work to postpone importing until you run code that actually needs the module you're importing, it does have a detriment of having to write the same import statement over and over again. And if you only make some imports locals it become rather easy to forget which ones you were trying to avoid and then accidentally import the module globally. So this approach works, it just isn't ideal.

The other approach is using the lazy loader provided in importlib. Now various people like Google, Facebook, and Mercurial has successfully used this lazy loader. The first two love it to minimize overhead when running tests while the last one wants a fast start-up. One perk to the lazy loader over the local import is you can trigger a ModuleNotFoundError early as finding a module is done eagerly, it's just the loading that is postponed.

Most people also set it up so that everything is lazily loaded. Now that's a good and bad thing. It's good in that you have to do very little to implicitly make everything lazily load. It's bad in that when you make something implicit you can end up breaking expectations that code has (there is a reason that "explicit is better than implicit"). If a module expects to be loaded eagerly then it can break badly when loaded lazily. Mercurial actually developed a blacklist of modules to not load lazily to work around this, but they have to make sure to keep it updated so it isn't a perfect solution either.

The new way

In Python 3.7, modules can now have __getattr__() defined on them, allowing one to write a function which will import a module when it isn't available as an attribute on the module. This does have the drawback of making it a lazy import instead of a load and thus finding out very late if a ModuleNotFoundError will be raised. But it is explicit and still globally defined for your module, so it's easier to control.

The code itself is actually not that complicated:

import importlib


def lazy_import(importer_name, to_import):
    """Return the importing module and a callable for lazy importing.

    The module named by importer_name represents the module performing the
    import to help facilitate resolving relative imports.

    to_import is an iterable of the modules to be potentially imported (absolute
    or relative). The `as` form of importing is also supported,
    e.g. `pkg.mod as spam`.

    This function returns a tuple of two items. The first is the importer
    module for easy reference within itself. The second item is a callable to be
    set to `__getattr__`.
    """
    module = importlib.import_module(importer_name)
    import_mapping = {}
    for name in to_import:
        importing, _, binding = name.partition(' as ')
        if not binding:
            _, _, binding = importing.rpartition('.')
        import_mapping[binding] = importing

    def __getattr__(name):
        if name not in import_mapping:
            message = f'module {importer_name!r} has no attribute {name!r}'
            raise AttributeError(message)
        importing = import_mapping[name]
        # imortlib.import_module() implicitly sets submodules on this module as
        # appropriate for direct imports.
        imported = importlib.import_module(importing,
                                           module.__spec__.parent)
        setattr(module, name, imported)
        return imported

    return module, __getattr__

To use it, you can do the following:

# In pkg/__init__.py with a pkg/sub.py.
mod, __getattr__ = lazy_import(__name__, {'sys', '.sub as thingy'})

def test1():
    return mod.sys

def test2():
    return mod.thingy.answer

In designing this, the trickiest bit was how to simulate the import ... as ... syntax to avoid name clashes. I ended up accepting a string which closely resembles the import statement you would have written had you done a global import. I could have broken it out into a third argument which took a mapping, but I thought that was unnecessary and I preferred to have a more unified API.

Anyway, I'm always pleased when I can do something like this in only 20 lines of Python code and feel like it isn't a total hack. 😉

As a resident of Oakland, CA and a baseball fan, I have more than a passing interest in the health and welfare of the Oakland A's baseball team.

Like all sports franchises, the A's have had their ups and downs. But in recent years they have become one of the most hapless franchises in all of American sports.

For years the ownership of the A's have turned off fans by trading excellent players for "prospects," hinting that they were going to leave town, constantly whining about their predicament, and making one false start after another trying to build a new stadium. They have also had lousy teams.

But the end of last season was hopeful. Although they finished in last place in their division, the final month of the season they played .586 ball with a 17-12 record. This would have placed them in 2nd place in their division and earned them a playoff spot (a .586 winning percentage would have won the division in the American League East.) They had some good young players and showed promise for an exciting 2018.

With that as background I went to an A's game last week. It was very depressing. It was a night game in which parking was free (saving fans $30) and still the park was empty. There was one other person in my row. The A's announced attendance of about 7,000 which means there were probably fewer than 5,000 people really there. The stadium holds over 50,000. And this was the first week of the season when fans are at their most hopeful and interest is high. Something, I thought, is terribly wrong.

And then I read an article in the San Francisco Chronicle...

"For many years, the A’s had the best television ads in the game...This season, the A’s have moved their advertising in-house, and the TV spots are no more...The A’s have decided to focus on targeted marketing this season rather than mass advertising, and they’re segmenting their advertising campaigns to customized audiences..."

"advertising in-house...targeted marketing... customized audiences...?" This ol' boy doesn't need an interpreter to know what that bullshit means -- social media crap to millennials. It's the default advertising strategy for everyone who knows nothing about advertising.

So far in this early season the A's have missed every advertising and marketing opportunity they've had. In the first week they had potentially the most exciting player in a generation - Shohei Ohtani - come to town. Did they tell the market about it? No, they were too busy doing "targeted marketing to customized audiences."

They have a player, Khris Davis, who has more home runs than everyone in baseball except the much ballyhooed Giancarlo Stanton the past two years. Have the A's told the 5 million or so people in their market about him? No, they've been too busy doing "targeted marketing to customized audiences."

So I did a little research to see how well their new strategy is working.

All of last year the A's averaged 18,446 people per game. The first eight games of this year they averaged 15,212. A drop of almost 20%. And it's really a lot worse. Last year's attendance figures include the dog days of August. And this year's small sample include both Opening Day and Opening Night, often the biggest crowds of the season.

Which leads us to tonight. The A's are staging a generous, but potentially misguided marketing stunt. To celebrate the 50th anniversary of their first game in Oakland they have distributed 200,000 free tickets for tonight's game -  200,000 tickets and fewer than 60,000 seats. What could go wrong?

And what for? So they can have a meaningless PR claim -- "the biggest crowd ever to watch an A's baseball game." Which proves what? That if you give something away for nothing people will take it? This stunt has a marketing value of zero. In the best case scenario it will be forgotten in 48 hours.

The Oakland A's problems go way deeper than marketing incompetence. But when you're in the toilet the last thing you need is amateurs screwing around with the plumbing.

In a case of synchronicity I’ve read Cory Doctorow’s novel Walkaway when I was ill recently, just as Bryan Alexander scheduled it for his near future science fiction reading group. I loved reading the book, and in contrast to some other works of Doctorow the storyline kept working for me until the end.

Bryan amazingly has managed to get Doctorow to participate in a webcast as part of the Future Trends in learning series Bryan hosts. The session is planned for May 16th, and I marked my calendar for it.

In the comments Vanessa Vaile shares two worthwile links. One is an interesting recording from May last year at the New York public library in which Doctorow and Edward Snowden discuss some of the elements and underlying topics and dynamics of the Walkaway novel.

The other is a review in TOR.com, that resonates a lot with me. The reviewer writes how, in contrast with lots of other science fiction that takes one large idea or large change and extrapolates on that, Doctorow takes a number of smaller ideas and smaller changes, and then works out how those might interplay and weave new complexities, where the impact on “manufacturing, politics, the economy, wealth disparity, diversity, privilege, partying, music, sex, beer, drugs, information security, tech bubbles, law, and law enforcement” is all presented in one go.

It seems futuristic, until you realize that all of these things exist today.
….. most of it could start right now, if it’s the world we choose to create.

By not having any one idea jump too far from reality, Walkaway demonstrates how close we are, right now, to enormous promise and imminent peril.

That is precisely the effect reading Walkaway had on me, leading me to think how I could contribute to bringing some of the described effects about. And how some of those things I was/am already trying to create as part of my own work flow and information processes.

Last summer, we were hired by Eventbrite to work on their EventTribe community.

EventTribe is a really interesting customer acquisition community. The primary goal is to gather leads of significant value through the community. This meant the concept had to be about the topic (running successful events) and not about the product (the latter would only attract existing customers).

In this post, I want to share the process we went through.

(You can see the results for yourself here: www.eventtribe.com).

 

Background

EventTribe was an inception-stage community, our goal was to drive it to establishment and, eventually, maturity. This meant increasing the number of quality leads generated from the community while putting the site on the path to sustained growth.

To get started, we undertook three types of research, interviews, surveys, and analysis of community data.

1) Interviews with key stakeholders. The first step was to interview the key stakeholders. This is especially important to establish the value. For example, what qualifies as a lead? Is lead-scoring used? What is the process for passing a lead from the community to the sales team/process? Who are the best types of leads etc? These interviews also identified any areas of uncertainty among staff and the kind of training which would suit them best. This helped us focus our efforts in a few key areas.

2) Interviews with community members. We interviewed a range of community members and highlighted every possible useful point in the transcript. This revealed a range of challenges members faced, how they thought about the community at the moment (‘interesting, but many discussions weren’t relevant’), and opportunities the members might want to pursue. We dropped most of these ideas into the surveys (below) to validate it among the broader community.

3) Survey of the community. Click here to see the exact questions. We wanted to know who the active audience were, what type of events they ran, what topics most interested them, and what they wanted to see next. One of the key questions here is to let members rank which types of content they want to see and how they want to see it. The results broadly showed we had a big audience who run events for 101 to 500 members and want to learn event promotion, project management, and finding good vendors/venues. They also wanted this as quick tips, detailed guides, and interviews/AMA formats. The survey also revealed some other interesting challenges and information we would use later.

 

4) Analysis of the community data. The community data showed an increase in traffic (especially due to some paid social advertising), but a decrease in the number of members who were participating each month. We identified the exact areas where members were dropping out and the key challenge, relevancy.

Once the research was complete, we could begin making laser-focused interventions to improve the metrics we wanted to move.

 

Improving The Newcomer To Regular Conversion Ratio

The first challenge was to improve the newcomer to regular conversion ratio. This began by mapping out the current process. We reviewed every community touch point and developed broad recommendations to optimize each point. Then we prioritized them and decided which we had the resources to pursue.

(Aside, it’s often staggering how effective most of these ideas are, yet so few people talk about them).

Once complete, we pursued the process systematically.

 

Step 1: Increasing the number of visitors to the community

Remember the motivation model below?

Establishment-phase communities need to ramp up their awareness. This has to be done within the very structure of the community.

This began by looking at where members came from today and doubling down on the most successful channels. The data quite conclusively showed the Eventbrite site was the biggest driver of traffic (especially the blog).

We doubled down on this source of traffic at the expense of social. This included:

1) Inserting community-related messages in blog posts. This meant going through many of the old, but frequently visited, content published and adding simple links to the community.

2) Adding pop-up notifications during community webinars with top experts. These are very effective to drive traffic from a popular blog to a community and sourcing good questions.

3) Getting better at mentioning and promoting community activities on the blog in general. Going forward, we would collaborate better with the writers of the blog to mention and feature community activities where relevant.

As a result, traffic from the blog increased steadily and exploded with a monthly increase, so far, of nearly 230% (shown below).

We also undertook a detailed technical audit of the community. All SEO activities in a community are constrained by the platform (in this case Discourse), but the audit highlighted several opportunities. These included:

1) Shortening the title/banner of the community on Discourse. The current title tag and meta-description were too long, so we shortened this.

2) Reducing the size and content of the title of popular topic discussions. Same as the above, we had long category names which hurt our search traffic. So we reduced the size of the category titles.

3) Creating discussions around topics most likely to drive traffic (this included venues/vendors, AV needs etc…). Very specific topics (e.g. top event venues in London) seemed to be very popular for search (albeit less good for discussions).

4) Merging related discussions together. This is still a work in progress but will become increasingly important going forward.

5) Adding better meta-descriptions and copy to the category pages (e.g. the event planning page). Like most communities, EventTribe had category pages which were devoid of almost all SEO-optimized content.

These changes (almost certainly combined with the natural growth in long-tail search terms) increased traffic by around 37% (excluding Christmas period).

We also tried to build good relationships with partners and drive referral traffic. This proved to be a colossal failure. Generally speaking, partners weren’t as invested or interested in the community as we were.

However, overall the results were extremely positive. Traffic to the community has risen by 50% since September.

It’s definitely possible to increase this, but with limited resources we also need to ensure we can convert this traffic into engaged members of the community. This is where the real challenge begins.

 

Step 2: Increasing the number of visitors who register

Most communities, with some glaring exceptions in specific categories, have conversion (sign-up) rates that hover from 0.1% to 2%. By the time we had begun working on the visitor to registered members ratio, it had dropped to 1.77% (this often happens when you drive more traffic, the registration ratio declines).

 

Replacing the banner

The biggest problem was the design and layout of the community. At the time, it wasn’t great:

The background image was slightly jarring, the message was bland and contrasted badly with the background, and the sign up button was hidden in the top right corner.

The banner suffered from the same problem as most banners. It was dull, impossible to hide, and showed the same message to every member regardless of how engaged the member had been.

It was wasting the most valuable real-estate in the community.

Fortunately, because the community was on Discourse, we could revamp this to almost anything we want. We used some conditional logic rules to design a banner which had a clear call to action for new members. We also added a clear reason to join the community (i.e. what people get by joining). This came directly from the interviews we had undertaken.

This not only guided people to participate, but also highlighted the exact first steps we needed them to take. The ‘hide banner’ option in the top left was a useful touch for regular members.

 

Featured discussions at the top

We also worked harder to ensure fresh, engaging, discussions appeared at the top of the community. This meant people would genuinely want to join and participate in the discussion. The community is best for the ‘editors picks’ type of filter.

This was easy enough to do and was great for testing different things to see what people participated in.

These small tweaks doubled the number of visitors who registered to join the community from 1.77% to 3.43%.

These might sound like small figures, but consider this ensures our awareness efforts are now twice as effective. It means thousands of new members every year.

You can beat this if your community is brand new, by using pop-ups, and dangling incentives (e.g. sign up to get this free report), but we didn’t want to go down this path for various reasons.

 

Step 3: Increasing the number of registrants who participate.

The messaging on the conditional logic banners didn’t just ask members to sign-up and get started, it also guided them to introduce themselves to the community and ask their first question.

As they complete each task, a ‘strikethrough’ would appear (we might replace these with ticks soon) so they could follow the journey.

If they completed all three, they would be moved to the ‘second banner’ where they would get a different set of tasks to complete (more on that in a second).

More than anything else, the banners immediately increased both the percentage of new members who made their first contribution and encouraged many lurkers to make their first contribution too. The percentage of newcomers who made a contribution has doubled to around 32%

While the overall number of new contributors has risen by 220%

(note, it looks a lot bigger on this graph as community-analytics base-rate is set to the minimum value instead of zero).

I suspect it would be very difficult to increase these conversion metrics any higher. Beyond 30% there tends to be a law of diminishing returns.

 

Step 4: Increasing the number of registrants who participate.

The next step was to increase the number of members who participated overall (i.e. get members to stick around and participate more).

This required some content programming, direct engagement via @mention groups, more conditional logic tweaks, improving @mentions too.

 

AMA Sessions

From the survey we identified the key topics members wanted to learn about and found experts who wanted to talk about them. Some of these proved more successful than others, however, they also spiked the traffic.

We even designed a custom banner for each person where we can quickly tweak the wording/photo for each new expert.

It would be interesting to test having multiple panelists to discuss topics over the course of a week. We tend to run sessions once a month, but these can last longer.

Thus far they have been successful, but there is plenty of scope to improve these. We’re still testing ideas.

 

Direct engagement via @mention groups

An often cited problem was the relevance of the content. There is a huge difference between people working on a global festival and those hosting a look bookclub meetup. We needed to increase the relevancy of content.

Discourse allows you to drop people into groups and @mention the entire group at once. Most people aren’t using these features at all.

The Eventbrite community allows people to add the events they’re most interested in on their profiles. We created a SQL query using the data explorer plugin on Discourse to list all members (and then all new members) by event type and then use the ‘bulk add to group feature’ to drop them each into unique groups.

We now have almost 30 groups separated by biggest interest. We’ve only just begun testing this, but I suspect it will prove quite effective at @mentioning small groups of people into relevant discussions (you can also use this for all newcomers).

 

SQL Queries For Other Members

We also created a few other queries to identify members at specific times when a direct interaction with the community manager could prove most valuable. This included:

• New registrations previous 7 days.
• Top 10% of participants (over past 60 days)
• Members who were active 2 months ago but not the past month (i.e. people drifting away).
• First-time contributors in past 7 days (see below)
• Members who have joined but not participated.

These queries aren’t too complicated to create (or find someone to create) (see below) and they help the community manager build quick lists of people to reach out to with a specific message at a specific time.

 

Adding the badge to a banner

Another innovation we tried (we try new ideas on almost all client projects) was adding a badge to the banner below. This helped members see how they compared with other members and where they ranked overall based upon their user levels.

This needs a few tweaks (we might switch to ‘likes received’ rather than use levels), but the idea seems to be effective in driving more contributions as we will soon see:

Ongoing Conditional Logic in Banners

We also experimented with adding more conditional logic to the banners (so every option wouldn’t be struck out, as shown in the option above). The final list of conditional logic will include:

(note these all link to discussions/activities).

• Banner 1: Getting Started
  – Sign up and get started.
  – Introduce yourself to the group.
  – Share a challenge and let’s solve it together!
• Banner 2: Building the habit
  – Share your best event promotion tip.
  – Tell us what resources would help you run your events?
  – What’s the best event venue you’ve used?
• Banner 3: Becoming a top member
  – Help answer five questions (and get your badge!)
  – Suggest a potential AMA speaker/interviewee for us.
  – Become a community volunteer.

This is in addition to the temporary webinar banner we add to the community. All these banners are implemented with some custom CSS in the Discourse themes.

The results have proven fairly positive. The number of questions asked by members has increased by around 35% (with some variation over Christmas/Easter).

The number of posts from active community members also rose by 35% from an average of 1.7 to around 2.3 (these will be skewed by the extremes, the median is probably a little lower – note this excludes posts by the community managers):

The Ultimate Metric – Active Participants

Perhaps the ultimate metric of any community’s success has been the ongoing increase in posting members as seen below. Thus far it’s risen by almost 160%.

Key Lessons

There are still a lot more things we can do here. It’s still a fledgling community with a relatively small target audience (event professionals in the UK). We haven’t yet done enough with gamification, a top contributor program, community volunteers, and lead qualification, but it shows how much you can achieve without any of these things.

1) You can achieve major increases without major platform changes. As we’ve tried to show here, you can achieve sustainably great results simply by making a few small tweaks in the right places without investing a fortune in a new technology.

2) Invest a lot of time to understand members. The interviews, survey data, analysis of community data take a lot of time to undertake, but they reveal almost everything you need to optimize engagement. Set aside 4 to 6 weeks just for this phase.

3) Law of diminishing returns. It’s not about trying to optimize everything to the max, it’s about investing your limited resources to achieve the best results. Beyond a certain level, it’s not worth the time to spend more time trying to optimize things. Focus on things which have the biggest, long-term, impact upon the majority of community members.

4) Beware of external events skewing your stats. Try to use a three-month average as disruptions such as Christmas, Easter, and February’s shorter month can play havoc with the stats.

5) Some things aren’t worth quantifying. These are plenty of things above we’re fairly sure are working, but we aren’t sure how best to quantify them. Some things take a longer period of time to have an impact. We also added the terrific community team to our training courses and tried to be better in how/what discussions we responded to within the community.

6) Learn quickly from your hits and misses. Not every discussion or activity is a hit and many are outright misses. Over time we test new ideas and get a sense of what it / isn’t working. Don’t keep pursuing a tactic which clearly isn’t working.

Over the years, we’ve become increasingly confident at following a process to increase participation and outputs in almost any community. The process begins with getting a full picture of the community data and then making laser-focused interventions over a period of time.

There is some technical work involved, but nowhere near as much as you might imagine. The majority of the work is understanding members in an unbiased and empathetic way. This is harder than you might imagine.

None of the above took a prohibitively long amount of time, cost a huge amount of money, or was technically impossible to implement. I really hope you can borrow a lot of these ideas in your own community.

Good luck!

p.s. if you’re running any sort of events, I strongly recommend you join EventTribe.

The garden office set-up under the apple tree is fully functional, right on time for the first 20 degrees day. Table arrived this morning.

Das S in IoT steht übrigens für Sicherheit.

— Volker Weber (@vowe) November 16, 2016

Letztes Jahr legte das Mirai-Botnet das halbe Internet lahm, in dem es 100.000 unsichere IoT-Geräte zur Waffe machte. Und dieses Risiko wird erheblich größer werden, weil immer mehr Gerätschaften vernetzt werden. Man denke an Heizkörperthermostate und ähnliches. Alle diese Geräte werden von einer Gattung Prozessoren gesteuert, die man Micro Controller Units (MCU) nennt. Sie sind um den Faktor Hundert schwächer als Micro Processor Units (MPU). Derzeit werden solche Geräte einmal produziert und niemals wieder aktualisiert. Der kleinste Fehler multipliziert sich abermillionenfach. Man kann sich leicht ausmalen, was passiert, wenn man diese Geräte vernetzt.

Azure Sphere ist Microsofts Antwort auf dieses Problem. Azure Sphere OS, basierend auf einem modifizierten Linux-Kernel, bildet die Softwareseite ab. Sie läuft auf Azure Sphere zertifizierten Mikrocontrollern, der erste kommt von MediaTek noch dieses Jahr. Der schlüsselfertige Azure Sphere Security Service sichert diese Geräte mit zertifikatsbasierter Authentisierung und Software Updates ab.

Das alte Microsoft hätte versucht, Windows zu verkaufen. Für MPUs gibt es ja auch Windows 10 IoT. Das neue Microsoft geht dagegen neue Wege. Statt die MCUs, die zu schwach für Windows 10 sind, zu ignorieren, deckt Microsoft nun alles von den kleinsten bis zu den größten Systemen ab. Fünf Milliarden will das Unternehmen in den IoT-Markt investieren. Und hier sieht man einen ersten konkreten Schritt.

Ich kenne viele Leute, die mit dem Wort Cloud vor allem Unsicherheit assoziieren. Bei mir ist das Gegenteil der Fall. Die Angreifer sind vernetzt und wer sich ihnen alleine gegenüberstellt, wird einfach niedergemacht. Wir haben nur eine Chance, wenn wir die Verteidigung auch vernetzen. Das betrifft die gesamte Malware-Bedrohung auf unseren Endgeräten.

Microsoft ist schon in der Zukunft angekommen. Ich wünsche vielen IT-lern das Gleiche.

Stephanie Booth, a long time blogging connection, has been writing about reducing her Facebook usage and increasing her blogging. She says at one point

As the current “delete Facebook” wave hits, I wonder if there will be any kind of rolling back, at any time, to a less algorithmic way to access information, and people. Algorithms came to help us deal with scale. I’ve long said that the advantage of communication and connection in the digital world is scale. But how much is too much?

I very much still believe there’s no such thing as information overload, and fully agree with Stephanie that the possible scale of networks and connections is one of the key affordances of our digital world. My rss-based filtering, as described in 2005, worked better when dealing with more information, than with less. Our information strategies need to reflect and be part of the underlying complexity of our lives.

Algorithms can help us with that scale, just not the algorithms that FB uses around us. For algorithms to help, like any tool, they need to be ‘smaller’ than us, as I wrote in my networked agency manifesto. We need to be able to control its settings, tinker with it, deploy it and stop it as we see fit. The current application of algorithms, as they usually need lots of data to perform, sort of demands a centralised platform like FB to work. The algorithms that really will be helping us scale will be the ones we can use for our own particular scaling needs. For that the creation, maintenance and usage of algorithms needs to have a much lower threshold than now. I placed it in my ‘agency map‘ because of it.

Going back to a less algorithmic way of dealing with information isn’t an option, nor something to desire I think. But we do need algorithms that really serve us, perform to our information needs. We need less algorithms that purport to aid us in dealing with the daily river of newsy stuff, but really commodotise us at the back-end.

Katrina Schwartz, Mind/Shift, Apr 20, 2018

---

This is being posted to an NPR website as an example of good teaching, but my concern here is that this approach is not grounded in a proper understanding of critical thinking (which is why I recently wrote Critical Thinking for Educators). The non-standard approach is something called 'claim-evidence-reason' where the reason 'explains why' the evidence supports the claim. This misunderstanding of argument form makes it impossible for students to learn how arguments work. Moreover, the examples used are often poor, in some cases literally begging the question, and grammatically incoherent throughout. That these are being used for English language learning (ELL) only compounds the problem, because students are led to misunderstand the roles of different words in day-to-day use. If you're going to teach language and logic, you need to be somewhat proficient in it yourself.

Web: [Direct Link] [This Post]

After a longer than usual wait (six weeks, to be exact), Google has released the latest Android distribution statistics.

Despite the addition of two more weeks of data, there aren’t any surprises this month: Oreo and Nougat continue to grow their share of the Android pie, albeit at an almost glacial pace, while older versions of Google’s operating system continue to lose ground.

As of this month, Oreo 8.0 and 8.1 are installed on 4.6 percent of active Android devices. Compared to the same time year, Nougat devices accounted for 4.9 percent of the active Android device base. As such, users are adopting Oreo at a slightly slower pace than its predecessor, at least for the time being. With OEMs that ship Android P devices required to support Project Treble, the hope is that the newest version of Android will gain traction faster.

Elsewhere, Marshmallow 6.0, despite a 2.1 percentage point decrease in overall usage, continues to be the most widely available version of Android. As of Google’s latest distribution numbers, Android 6.0 is installed on 26 percent of Android devices that visited the Play Store.

Source: Google Via: Android Police

The post Oreo is now installed on 4.6% of Android devices appeared first on MobileSyrup.

Whether you’re packing chili for lunch or oatmeal to eat after your morning commute, a great food thermos will keep hot foods at safe temperatures for hours and won’t leak into your bag. After comparing 22 food thermoses and testing 10, we’re confident that the Zojirushi Stainless Steel Food Jar is the best for most people. It has excellent heat retention, is durable and easy to clean, and comes in three sizes ranging from 12 to 25 ounces. We like the 10-ounce Thermos Funtainer Food Jar for packing kids’ lunches or for people who prefer smaller portions.

As I’ve stated before, I’m a professor at a very small university with wonderful colleagues so I enjoy the privilege of having smaller class sizes and a lower teaching load. However, I feel the same pressures to publish, teach, do service as many others because the expectation in my institution is that we behave as though we are in an R1, which puts enormous levels of pressure on our performance.

I am currently chairing two searches for new tenure-track faculty and thus I can’t use my No Email Before Noon rule at least until the searches are completed, so I’ve been trying to find a way in which I can continue doing my research without letting down my institution and job candidates. To do this, I try to follow even more closely my “Accomplish Two Things Before Anything Else” approach. I usually do this on a regular basis, but now that I’m under more pressure to respond to emails in the morning and throughout the day, it’s become even more important that I do it.

The Accomplish Two Things Before Anything Else approach is quite simple. I set out to write SOMETHING and read SOMETHING every day. Even if it’s writing 25 words of a new memorandum, or just the first page or abstract of a new-to-me research article or book chapter, I try to write and read every day. Every. Single. Day. I don’t get out of the house nor do I open my email before I get *some* writing done and *some* reading done. That’s why I champion smaller goals. During crunch time, I can’t stay sitting at my computer until I crank 1,500 words. So, I set out to accomplish at least two little things. This approach liberates me from the guilt of “you’re not doing your research” and frees my mind to enable me to do the work that I need to do.

I tweeted about this approach so you can open my Twitter thread by clicking anywhere on the tweet below.

787 words. I felt stuck at about 87 and then BOOM, ideas started to flow. It's like @explorstyle says.. We write … pic.twitter.com/xVByuc35p1

— Dr Raul Pacheco-Vega (@raulpacheco) April 16, 2018

I have the same temptations that all y'all do: I wake up and want to check Twitter, and my email is there, tempting me. So, during crunch time (like now that I am chairing two searches for TT professors) when I can't say "sorry, no email before 12 noon", I stay with 2 goals.

— Dr Raul Pacheco-Vega (@raulpacheco) April 16, 2018

I'm going to read ONE article (AIC+CSED) and then BOOM off I go to campus to deal with the barrage of stuff that accumulated while I was at #ISA2018 and #AAG2018 My strategy is to NEVER leave my house before accomplishing TWO things, small goals (25-75 words, 1 AIC/CSED).

— Dr Raul Pacheco-Vega (@raulpacheco) April 16, 2018

And while today I was lucky to write 787 words, other days I write 65 words and that should be enough. Today I’ll do an AIC-CSED combo, because I have enough time, but I could just as easily simply read an article’s abstract because I don’t have any more time. It’s kind of a simplified version of my Quick Wins approach. Two Quick Wins: write a few words, read ONE article. Hopefully this approach works for other people!

Jason Kottke reminds us that blogging is most certainly not dead, and that there are great blogs out there.

My only objection is the use of the word “dead” to apply to things that aren’t alive. Even when you’re saying that something is not dead.

I’ve done it myself. It’s shorthand, yes, but it’s a broad binary take when something more nuanced and true would be warranted.

On the blues harp:

A diatonic harmonica is designed to ease playing in one diatonic scale…

Blues harp subverts the intention of this design with what is “perhaps the most striking example in all music of a thoroughly idiomatic technique that flatly contradicts everything that the instrument was designed for.”

Jason Kottke, kottke.org, Apr 19, 2018

---

This is mostly just a set of links to some odds and ends in the blogging world, but there are comments I want to highlight. The first is this: "I refuse to let social media take everything. Those shapeless, formless platforms haven’t earned it and don’t deserve it... When I log into Facebook, I see Facebook. When I visit your blog, I see you." And this: "People are increasingly souring on the surveillance state Skinner boxes like Facebook and Twitter. Decentralized media like blogs and newsletters are looking better and better these days." Kottke (to whom I still subscribe) has been around for 20 years. Image: Hoder. Realted: How to make 29 different shapes of pasta by hand.

Web: [Direct Link] [This Post]

David Hopkins, Technology Enhanced Learning Blog, Apr 19, 2018

---

Am I an instructional designer (ID) or a learning technologist (LT), asks David Hopkins. "The only difference is that the ID role requirements are for commercial/corporate employers, and the LT ones for universities." Excpet that it feels like there should a difference - doesn't it? " Does the title/name given to your role even matter? Perhaps the difference here is time … what was once two distinct roles have now merged in outlook and intention and can be seen as the same, depending on which title the organisation prefers?"

Web: [Direct Link] [This Post]

Adobe, Apr 19, 2018

---

Adobe has released a study (64 page PDF) that appears to be based on a survey of 'policymakes and influencers' and 'educators' (their terms) from the U.S., the U.K., Germany and Japan. The premise is to higjlight the importance of teaching creativity in schools, to suggest it's not happening as much as it should, and to identify the reasosn why (which are mostly related to policy, access to technology, and training). Maybe Adobe could consider lowering some prices to provide greater access to tools. Hm? The reports are lavishly over-illustrated PDF versions of PowerPoints. Via Campus Technology.

Web: [Direct Link] [This Post]

Natalie Afshar, Australian Education IT blog, Microsoft, Apr 19, 2018

---

OK, despite the title of this post, you are not going to create a chatbot in ten minutes, despite what the headline says (I followed through and found that you have to have a FAQ already written, in which case (presumably) the chat bot is simply selecting the most likely response from your FAQ to type or say. But I am linking to this item in order to think about what would happen if you truly could make a chat bot in 10 minutes. Suppose, for example, I fed it all the contents of this website, and then asked it to answer questions with the most relevant sentence or paragraph from my work. It wobbles the mind.

Web: [Direct Link] [This Post]

After researching wiper blades for more than 60 hours, scouring user reviews, talking to auto-service shops in such weather-challenged regions as Chicago and Portland, Oregon, and testing top competitors on a handful of cars, our findings show that the Bosch Icon is a good bet for most drivers, as long as it fits your car. Bosch wiper blades are recommended by the shops we interviewed more than any other brand, and the Icon is consistently among the highest-rated models by users on websites that sell a wide range of wipers. It’s also earned among the highest ratings of any top-selling blade on Amazon with relatively few complaints.

@mkjcktzn asked if one can access Feedly “Saved for Later” items via the API. The answer is “Yes!”, and it builds off of that previous post. You’ll need to read it and get your authentication key (still no package ) before continuing.

We’ll use most (I think “all”) of the code from the previous post, so let’s bring that over here:

library(httr)
library(tidyverse)

.pkgenv <- new.env(parent=emptyenv())
.pkgenv$token <- Sys.getenv("FEEDLY_ACCESS_TOKEN")

.feedly_token <- function() return(.pkgenv$token)

feedly_stream <- function(stream_id, ct=100L, continuation=NULL) {
  
  ct <- as.integer(ct)
  
  if (!is.null(continuation)) ct <- 1000L
  
  httr::GET(
    url = "https://cloud.feedly.com/v3/streams/contents",
    httr::add_headers(
      `Authorization` = sprintf("OAuth %s", .feedly_token())
    ),
    query = list(
      streamId = stream_id,
      count = ct,
      continuation = continuation
    )
  ) -> res
  
  httr::stop_for_status(res)
  
  res <- httr::content(res, as="text")
  res <- jsonlite::fromJSON(res)
  
  res
  
}

According to the Feedly API Overview there is a “global resource id” which is formatted like user/:userId/tag/global.saved and defined as “Users can save articles for later. Equivalent of starring articles in Google Reader.”.

The “Saved for Later” feature is quite handy and all we need to do to get access to it is substitute our user id for :userId. To do that, we’ll build a helper function:

feedly_profile <- function() {
  
  httr::GET(
    url = "https://cloud.feedly.com/v3/profile",
    httr::add_headers(
      `Authorization` = sprintf("OAuth %s", .feedly_token())
    )
  ) -> res
  
  httr::stop_for_status(res)
  
  res <- httr::content(res, as="text")
  res <- jsonlite::fromJSON(res)
  
  class(res) <- c("feedly_profile")
  
  res
  
}

When that function is called, it returns a ton of user profile information in a list, including the id that we need:

me <- feedly_profile()

str(me, 1)
## List of 46
##  $ id                          : chr "9b61e777-6ee2-476d-a158-03050694896a"
##  $ client                      : chr "feedly"
##  $ email                       : chr "...@example.com"
##  $ wave                        : chr "2013.26"
##  $ logins                      :'data.frame': 4 obs. of  6 variables:
##  $ product                     : chr "Feedly..."
##  $ picture                     : chr "https://..."
##  $ twitter                     : chr "hrbrmstr"
##  $ givenName                   : chr "..."
##  $ evernoteUserId              : chr "112233"
##  $ familyName                  : chr "..."
##  $ google                      : chr "1100199130101939"
##  $ gender                      : chr "..."
##  $ windowsLiveId               : chr "1020d010389281e3"
##  $ twitterUserId               : chr "99119939"
##  $ twitterProfileBannerImageUrl: chr "https://..."
##  $ evernoteStoreUrl            : chr "https://..."
##  $ evernoteWebApiPrefix        : chr "https://..."
##  $ evernotePartialOAuth        : logi ...
##  $ dropboxUid                  : chr "54555"
##  $ subscriptionPaymentProvider : chr "......"
##  $ productExpiration           : num 2.65e+12
##  $ subscriptionRenewalDate     : num 2.65e+12
##  $ subscriptionStatus          : chr "Active"
##  $ upgradeDate                 : num 2.5e+12
##  $ backupTags                  : logi TRUE
##  $ backupOpml                  : logi TRUE
##  $ dropboxConnected            : logi TRUE
##  $ twitterConnected            : logi TRUE
##  $ customGivenName             : chr "..."
##  $ customFamilyName            : chr "..."
##  $ customEmail                 : chr "...@example.com"
##  $ pocketUsername              : chr "...@example.com"
##  $ windowsLivePartialOAuth     : logi TRUE
##  $ facebookConnected           : logi FALSE
##  $ productRenewalAmount        : int 1111
##  $ evernoteConnected           : logi TRUE
##  $ pocketConnected             : logi TRUE
##  $ wordPressConnected          : logi FALSE
##  $ windowsLiveConnected        : logi TRUE
##  $ dropboxOpmlBackup           : logi TRUE
##  $ dropboxTagBackup            : logi TRUE
##  $ backupPageFormat            : chr "Html"
##  $ dropboxFormat               : chr "Html"
##  $ locale                      : chr "en_US"
##  $ fullName                    : chr "..."
##  - attr(*, "class")= chr "feedly_profile"

(You didn’t think I wouldn’t redact that, did you? Note that I made up a unique id as well.)

Now we can call our stream function and get the results:

entries <- feedly_stream(sprintf("user/%s/tag/global.saved", me$id))

str(entries$items, 1)
# output not shown as you don't really need to see what I've Saved for Later

The structure is the same as in the previous post.

Now, you can go to town and programmatically access your Feedly “Saved for Later” entries.

You an also find more “Resource Ids” and “Global Resource Ids” formats on the API Overview page.

Paul Surette, newly-appointed co-moderator of the Facebook group "Penny For Your Thoughts: Malden", is not pleased with me. Who could blame him?

Recently, he complained to the group about Mya and Deenaa Cook, two high school students from Malden who took a brave stand last year against their charter school’s discriminatory dress code. The Massachusetts Attorney General agreed with the Cook sisters; do did the Anti-Defamation League, the ACLU, the Washington Post and the Boston Globe. For some reason, though, Paul wants to take on these two high school girls yet again. (It's not fair; they're much more knowledgable.)

In the ensuing kerfuffle, I wrote:

Mark Bernstein: Amazing how these racists won’t let this go. Malden was already a national laughingstock for harboring the sort of racist school policies normally associated with Mississippi and Alabama. Racism lost; now you want to argue some more?

In response, Joe Kaplan denounced me in an interesting way:

Joe Kaplan: Paul....look who you are debating,. an indoctrinated self-loathing Jew.Save your brain cells.

Paul Surette: You're right, Joe

Now, I have plenty of faults, but self-loathing is not one I hear a lot about. I'm not sure what Joe think’s he’s thinking, but I imagine it all stems from a bunch of boys in middle school with a pilfered copy of Portnoy’s Complaint, trying to find the good parts before the third-period bell. Finding Philip Roth puzzling, perhaps they looked to the flap copy for an explanation. I bet they figured it out eventually; anyway, the phrase seems to have stuck.

“Penny For Your Thoughts” used to be a conduit for local political issues -- the sort of place where you heard about zoning changes for a new restaurant or candidates for school committee. Lately, though, it's changed. More fake news memes were posted and taken seriously, often from Russian or alt-right sites. A former city-councillor decided to catechize a local activist, is a private citizen who happens to be Muslim, challenging her to renounce Sharia. Paul Surette chimed in:

Paul Surette From what I've seen the last 6 years is that a 'moderate' muslim is one who says nothing about secular violence, but quietly cheers it on.

Paul Surette: My 'understanding' of moderate Muslims here is accurate as judged by Muslims I know who live here who know who the moderate Muslum is really about. Moderates live under the quise of being anonymous while quietly cheering secular violence.

The former city councillor joins in, this time ridiculing another private citizen for her religious beliefs.

Neil Kinnon: Some of us have not given up. Better to fight them now. Bruce Warren Lynch is a radical who epitomizes the idea of ”defining deviance down”. He and his significant other according to Nichole Mossalam were elected delegates at the Democratic city Committee out of Ward Two Edgeworth (not confirmed yet) Lynches girlfriend is a self declared Witch, excuse me Wiccan according to him and our lovely friend Nichole, see yesterday’s posts. This is what the local Democratic Party is being taken over by. Pagans who worship witchcraft. The last political group that were elected with widespread beliefs in the occult were the National Socialists in Germany... [emphasis mine]

Another participant lamented the fact that Malden is now “only 37% American”, by which she meant that a majority of Malden residents today are Asian-Americans, Hispanic-Americans, Caribbean-Americans, or African-Americans. (A number of Old Maldonians are scions of families that moved to Malden from South Boston to escape school integration in the 1970s, when Malden High was still essentially segregated. The integration of Malden schools may help explain why so many regulars in ‘Penny For Your Thoughts” no longer live in Malden and are so angry at Malden’s current residents.)

In another discussion, Muslims are collectively responsible for, well, basically everything.

Joe Howard …The hateful muslim group is a major problem worldwide, and they're the ones who are responsible for the overseas terror. Immigration sanctions are out of control.

The group’s posted rules call for "No name calling, threats, racism, sexism, or that sort of thing." Interestingly, all the above pass muster. Other posters falsely blamed Warsaw’s Jews for surrendering to gun control and failing to resist the Nazis, attributed credit for Victory in Europe to the German resistance(!), and claimed George Soros was bussing in demonstrators against the Boston “Free Speech” march for white supremacy. What’s behind this bile, in what Mayor Gary Christenson has celebrated as a diverse and welcoming city?

(I have left the spelling unchanged in the quoted posts. A number of these writers often misspell words they dislike — for example, “anti-Semete” for “anti-Semite.” I don't know whether this is meaningful — a dog whistle of some sort? — or merely random.)

- - -

Why not simply ignore white-supremacists and bigots on social media?

I’m a long-time student of new media; it's my primary research area. This year, I’ve been focussing on some dangerous asymmetries that facilitate malefactors on Twitter and Facebook. With Dr. Clare Hooper, I wrote “A Villain’s Guide To Social Media and Web Science” for the 2018 ACM Conference on Hypertext and Social Media. The conference was good enough to give us a prize! (If you enjoy funny academic papers and don’t mind 25 footnotes, happy to share a preprint. Email me.)

One important lesson we have learned is that, in social media, ignoring villainy today makes it harder to oppose villainy tomorrow. This is the lesson of the 1930s: if you close your shutters when the brownshirts are shouting in the street and don’t call the police, next year those same brownshirts may be the police. If you don’t oppose Vichy today, after the Liberation comes your friends and neighbors may look at you and see a collaborator.

A second vital lesson — one that was entirely unexpected — is that on social media it is easier to spread lies than to disseminate the truth.

Third, we have the core asymmetry: a single scurrilous word can do lasting harm that a thousand well-intentioned “likes” will not repair.

In better, safer times, we may safely ignore wretched hives and scum and villainy. This is not such a time.

Ideally MySQL include this in a future version of mysqldump, but here's a neat little utility to split extended inserts into one row per line. This allows you to use extended inserts (for much faster importing of large tables) while having the ability to grep and diff rows easily.

Add Ken Sim to the candidate list for the NPA’s Mayoral nomination in the October 20 civic election.  Stay tuned for the NPA’s selection on May 29.  And a fifth candidate may be close to filing papers.

Mike Howell writes in the Vancouver Courier:

. . .  it was clear from a news release he issued that making Vancouver more affordable was one of the main reasons he wants to become the NPA’s mayoral candidate.

“I grew up here and believe my four boys should have the opportunity to raise their families here also,” Sim said. “I believe this generation, and future generations, should be able to afford to live in this city.”

The entrepreneur wants to focus on job creation, saying he was “gravely concerned that countless Vancouverites have been forced to move to other cities for better opportunities.” The city’s ongoing social problems related to mental health, addiction and homelessness are also on Sim’s agenda.

Travis Lupick writes in the Straight:

Other issues that Sim mentions in the release as areas on which he would focus if elected mayor include “bureaucratic red tape” and some of Vancouver’s most-persistent challenges concentrated in the Downtown Eastside.

“I’ve spent years supporting causes related to the Downtown Eastside and am disappointed with the lack of progress on issues related to mental health, addiction and homelessness—despite the fact significant money has been spent,” he said. “I believe it is time we bring real change to help our vulnerable sisters and brothers living in this city.”

What I want to talk about is this issue of what is an action and what is a calculation in terms of timeliness, because we know that deep down in the computer, everything is an action. Every operation depends on what is that particular locations in memory at the time that the operation is run.

Transcript

My name is Eric Normand, and I’m writing a book called “A Theory of Functional Programming.” I want to talk about a certain topic from that book and I’m going to use the transcript of this video to help me write the book. Everything here might make it into the book. At least the topic is going to. Who knows what’s going to happen with the editing.

What I want to talk about is this issue of what is an action and what is a calculation in terms of timeliness, because we know that deep down in the computer, everything is an action. Every operation depends on what is that particular locations in memory at the time that the operation is run.

Even like addition, an addition in the mathematical sense is a calculation, it’s timeless, but when you run the add instruction, it reads two locations in memory which are mutable, and it writes to a location in memory, which is also mutable.

It is an action deep down and how the machines work, because our machines are based on the idea of a turing machine. It’s all actions all the way down.

"The ADD instruction is an Action deep down and how the machines work, because our machines are based on the idea of a Turing Machine. It’s all Actions all the way down."https://t.co/huyW8lHEg4

— Eric Normand (@ericnormand) December 14, 2018

How is it that we can claim that we actually have calculations and it’s just through discipline, the discipline of our compiler, or even just as programmers, we set it up so that those registers don’t have anything important in them so we can overwrite them and we’re going to pull the value right out of the register and put it on the stack or stored in a variable right away.

We’ve got these disciplines that allow us to do the operations in a way that doesn’t depend on when it’s run. Like I said, the disciplines are either enforced by the programmer or by the compiler, and both of them are fine. The compiler makes it a lot easier on the programmer and avoids a lot more mistakes.

“We’ve got disciplines that allow us to do the operations that don’t depend on when they are run. The disciplines are either enforced by the programmer or by the compiler. The compiler makes it a lot easier on the programmer and avoids a lot of mistakes.”https://t.co/VBkIZDgnhA

— Eric Normand (@ericnormand) June 16, 2018

In the sense, it’s all an illusion. The whole idea of timelessness of calculations, but it’s a very valuable illusion. We can make it pretty safe. Pretty reliable.

There’s another issue though which is that you have language like Haskell just as a really good example of a functional language. It has a very hard line that it draws between calculation and action. The calculations are function types. All of the functions are pure and so their calculations and I/O is reserved for actions.

The thing is that’s a great line. That is where it should be drawn as a language making this choice for the programmer. That is the right place to put it. There are some actions, let me put it this way, what if I am writing and reading to a file? That’s definitely an action.

You can’t really control the file system. It’s shared by all of the programs that are running on the computer at the same time. They can read and write to it as the same time as I do. It’s definitely a time evolving. It’s bound up in the time because I could write before they write or I could write after they write.

It really changes things. What if I were to add a lot of discipline to my program to lessen the chances almost to zero that someone else would be writing and reading at the same time as me? I would generate a new temporary file with a random name so that no one could guess it.

I read and write to it very quickly and then immediately delete it. Basically there’s near-zero chance that at any moment another program could guess the file, read from it, write to it, and mess up my code.

In the same way, the operating system will protect the memory, saying, “You can’t read that memory from this other program. You can’t write to the memory of the other program.” You’re enforcing a similar discipline. Would that still be considered an action?

Does it matter if it’s a near-zero probability that someone can mess with my code? I do it in such a way that it’s a pure calculation. I just needed a temporary file to hold some values for me. You can argue — and it’s the right way to argue — that that is not an action.

You’re using discipline to ensure that it doesn’t depend on when it is run, or how many times it is run. We can move that into a calculation. In fact, Haskell gives you a way to take something that is typically an I/O, meaning disk reading and writing — disk I/O — any kind of I//O but just in this example disk I/O is an I/O.

They give you a thing called “perform unsafe I/O.” What that does is it takes an I/O and turns it into a calculation. That’s great. Haskell knows that this is a thing that you’re going to run into, where sure it’s I/O in the general sense. In this particular case, I want a compiler that trusts me, that I know what I’m doing and it should be a calculation. That’s great.

“I want a compiler that trusts me, that I know what I’m doing and it should be a Calculation.”https://t.co/mahmnjKXDH

— Eric Normand (@ericnormand) August 10, 2018

Similarly, if you have a calculation…it’s a pure function. It’s like a statistical calculation, but it uses big data. It takes a long time to run. It’s a very complicated, convoluted algorithm, but it’s pure. If you run it twice, you get the same answer.

The issue is it takes 24 hours to run. In general, this would be considered a calculation because it’s pure, but when you start running it, you’re going to wish that you had started 24 hours ago. That’s yesterday. You’re not going to know the answer until tomorrow. It does matter when you run it.

All calculations take some time. Usually it’s so little time that we don’t care. 10 milliseconds on the human time scale, we don’t care. Hundred milliseconds, we don’t really care. A minute, maybe we start to care, but 24 hours, we definitely care. That’s starting to get into, “Oh, is this going to be done by the end of the week?” territory.

"All Calculations take some time. Usually it’s so little time that we don’t care. 10 milliseconds, A hundred milliseconds, we don’t care. But 24 hours, we definitely care. That’s starting to get into, 'Oh, is this going to be done by the end of the week?”"https://t.co/m4frXyiuDx

— Eric Normand (@ericnormand) November 26, 2018

What do we do? We should call that an action. It does matter when you run that. What I’m trying to get at is this principle that this is an illusion that we are building. In many cases, we can get away with it because everything takes time. Everything has a side effect… [coughs] Excuse me.

Everything has a side effect of using memory. It’s putting memory pressure on the garbage collector. It’s using one of the CPUs while it’s running. That’s affecting the scheduler, and what other things can run at the same time, how long other things take to run.

It all does have an effect. It’s generating heat. It’s using electricity. All those things. We ignore those. Like in a physics problem at school, we would ignore friction. Even though we know it’s there. We know that there must be friction. We just don’t count it. We know there’s air resistance. We just don’t count it. The answer you get is good enough without it even if you ignore it.

We do the same. We say this calculation is timeless enough. We don’t care about the time it takes.

[pause]

Eric: Some languages, like I’ve said, give you a little bit more support in the discipline. For instance, Haskell gives you a lot of support because its functions are all pure by themselves. Clojure gives you a lot of support because it has immutable data structures.

“Some languages, give you a little bit of support in the discipline. For instance, Haskell gives you a lot of support because its functions are all pure. Clojure gives you a lot of support because it has immutable Data structures.”https://t.co/eOWgQOtAfb

— Eric Normand (@ericnormand) May 16, 2018

I believe that functional programming is something you can do if you know what you’re doing. It is possible to do it with purely programmer discipline and not explicit support from the language. This is why, because it’s all a choice anyway. It’s all a system that’s built up by the compiler and the programmer anyway.

This has been me talking about the illusion that we’re creating when we’re doing functional programming. An illusion of timelessness, an illusion of timelessness of data as well. A lot languages don’t have immutable data structures. Everything is mutable. What’s popular now is having a variable that is unchangeable after it is set.

In JavaScript, you have const. Even if you use a const, if you assign an array or an object to that const, you can still mutate the array or the object. It’s not really immutable. It’s only one level of immutability. What you want is to guarantee that when you generate data it is an actual bit of data. It is a value. It is not a place that can change. It is not a place for storing stuff.

It is a record. A record means something that you keep forever. You’ve recorded and it’s stored forever. I don’t know if I’ve talked about that enough. Data is a recorded fact about an event that happened. This is the definition of data we talked about before. By record, it means that we want to keep it. You record something to keep it around later.

“A record means something that you keep forever. You’ve recorded and it’s stored forever. Data is a recorded fact about an event that happened. By record, it means that we want to keep it. You record something to keep it around later.”https://t.co/1hjcYS4dfZ

— Eric Normand (@ericnormand) June 12, 2018

We’ve lost that idea of record generally in computing, in software, mostly because we had a very limited amount of storage space. Hard drives were expensive. RAM is certainly expensive.

We’re starting to remember that in any other information systems outside of the computer, say in an accounting system or a medical records system, when you say I have a record, it means it’s permanent. You want to keep that record around forever. You don’t want to change it.

The typical threat at school — at least here in America — any absences will be marked on your permanent record. That’s exactly what we want. We want things to be stored forever. It’s permanent.

Of course, that permanence of storage doesn’t really count for stuff in RAM. If we turn off the computer, boom, you’ve lost it. We do want it as a transient place to store something before it gets recorded. Putting a record immutably in memory so that we don’t have to worry about it being overwritten, that is very useful.

As a discipline, functional programming moves toward this immutable data structure approach because it allows you to reason very locally. You don’t have to worry about who else has a pointer to this data structure if it’s immutable. You can just rely on it never changing.

Whereas if you have mutable data structures, you have to have a very strong trust, a very strong contract with the clients that are using you about ownership and what you’re allowed to mutate. The language will allow mutation. There’s a lot more non-local reasoning that you have to do.

You have to say, “Look, I’m going to give you this value. It’s not yours. I’m just letting you read it. Do not write on it. I’m letting you borrow it but return it in the same way you found it.” Whereas in an immutable thing that’s enforced by the run time.

Often what we’ll do is we will to help enforce the safety we will use a discipline called Copy on Write or something even stricter, which is Copy on Share. Every time someone asks for a value that is mutable in the language, say in an object in JavaScript or an array in JavaScript, every time someone asks for it, you make a copy and pass them the copy so they get a fresh one.

“Every time someone asks for a value that is mutable in the language, say in an object or an array in JavaScript, every time someone asks for it, you make a copy and pass them the copy so they get a fresh one.”https://t.co/NwKlZYFHMx

— Eric Normand (@ericnormand) September 21, 2018

You don’t have to trust them anymore. You can write all over this if you really want to, it’s not going to affect me.

The problem is that’s very wasteful in terms of memory and it can easily get out of hand. For instance, copying an array is a linear operation. If you don’t know that it’s copying, because this is part of the contract, you have to tell them, “Hey, this is now…Reading this value is now a linear operation.”

Now, it can become accidentally quadratic, because what if they ask for the value, not knowing that is linear? Let’s say they just assume it’s constant time, it becomes accidentally quadratic, because they’re asking for each element.

See? It complicates things to have mutable things, because if you want to enforce it through discipline, you have to have all these checks. You have to have a trust, you have to have a contract that’s better understood. Whereas a mutable is a very simple contract.

I’m going to give you something, it just can’t write to it. You can try, it’s not going to work. It’s easy. It’s very easy to reason about.

I’ve talked enough about this. Thank you so much for listening, tell the robots what you like, so they know how to share and recommend to other people. They don’t have emotions, they can’t tell what’s good. Let them know. Let them know what you like. Hit those buttons, mash them, subscribe.

Sorry about the audio for some previous videos. I really think it’s my headphones, not these. I switched back to these. These don’t work so well, but I think the microphone is fine. Sorry about the audio. I bought a new pair, they’re in the mail. We’ll see how they work.

See you later. Bye.

The post A Theory of Functional Programming 0006 appeared first on LispCast.

A new study has revealed that thousands of Android apps are improperly tracking and sharing data on children.

The report titled “Proceedings on Privacy Enhancing Technologies,” was compiled by researchers at the International Computer Science Institute at the University of California, Berkeley and looked at 5,855 child-directed apps that feature “several concerning violations and trends.”

Specifically, the researchers state that the apps that are improperly collecting and sharing data are all included in Google’s Designed for Families program.

Some of the most notable findings of the report include:

• 40 percent of apps shared children’s personal info insecurely
• 39 percent of apps violated Google’s terms
• 19 percent of apps shared private information with third-party services that aren’t supposed to be present in children’s apps
• 5 percent of apps collected children’s location or contact data without requesting parental consent

Furthermore, more than half of the apps were found to be in violation of Children’s Online Privacy Protection Act (COPPA), United States federal law.

However, the report does suspect that “many privacy violations are unintentional and caused by misunderstandings of third-party SDKs.”

The full report can be viewed here.

Via: The Verge 

The post Thousands of Android apps are improperly tracking children, says new report appeared first on MobileSyrup.

Defining success in data analysis has eluded me for quite some time now. About two years ago I tried to explore this question in my Dean’s Lecture, but ultimately I think I missed the mark. In that talk I tried to identify standards (I called them “aesthetics”) by which we could universally evaluate the quality of a data analysis and tried to make an analogy with music theory. It was a fun talk, in part because I got to play the end of Charles Ives’ Second Symphony.

Statisticians, in my experience, do not discuss this topic very much. It’s either because it’s so stupid that everyone has an (unspoken) understanding of it, or that everyone kind of has a slightly different understanding of it, or that no one understands it. Either way, in my close to twenty years as a statistician, I don’t think I’ve had many in-depth conversations with anyone about what makes a data analysis successful. The most that I’ve ever discussed this topic is on Not So Standard Deviations with Hilary Parker, where this is a frequent topic of conversation. Recently, Hilary gave a talk related to this topic (slides here), and so I was inspired to write something.

I think I’ve come around to the following definition of data analysis success, which is,

A data analysis is successful if the audience to which it is presented accepts the results.

There are a number of things to unpack here, so I will walk through them. Two key notions that I think are important are the notions of acceptance and the audience.

Acceptance

The first idea is the notion of acceptance. It’s tempting to confuse this with belief, but they are two different concepts that need to be kept separate (although that can be difficult at times). Acceptance of an analysis involves the analysis itself—the data and the methods applied to it, along with the narrative told to explain the results. Belief in the results depends on the analysis itself as well as many other things outside the analysis, including previous analyses, existing literature, and the state of the science (in purely Bayesian terms, your prior). A responsible audience can accept an analysis without necessarily believing its principal claims, but these two concepts are likely to be correlated.

For example, suppose a team at your company designs an experiment to collect data to determine if lowering the price of a widget will have an effect on profits for your widget-making company. During the data collection process, there was a problem which resulted in some of the data being missing in a potentially informative way. The data are then handed to you. You do your best to account for the missingness and the resulting uncertainty, perhaps through multiple imputation or other adjustment methods. At the end of the day, you show me the analysis and conclude that lowering the price of a widget will increase profits 3-fold. I may accept that you did the analysis correctly and trust that you did your best to account for the problems encountered during collection using state-of-the-art methods. But I may disagree with the conclusion, in part because of the problems introduced with the missing data (not your fault), but also in part because we had previously lowered prices on another product that we sell and there was no corresponding increase in profits. Given the immense cost of doing the experiment, I might ultimately decide that we should abandon trying to modify the price of widgets and leave things where they are (at least for now). The analysis was a success.

This simple example illustrates two things. First, acceptance of the analysis depends primarily on the details of the analysis and my willingness to trust what the analyst has done. Was the missing data accounted for? Was the uncertainty properly presented? Can I reason about the data and understand how the data influence the results? Second, my belief in the results depends in part on things outside the analysis, things that are primarily outside the analyst’s control. In this case, these are the presence of missing data during collection and a totally separate experience lowering prices for a different product. How I weigh these external things, in the presence of your analysis, is a personal preference.

Acceptance vs. Validity

In scientific contexts it is tempting to think about validity. Here, a data analysis is successful if the claims made are true. If I analyze data on smoking habits and mortality rates and conclude that smoking causes lung cancer, then my analysis is successful if that claim is true. This definition has the advantage that it removes the subjective element of acceptance, which depends on the audience to which an analysis is presented. But validity is an awfully high bar to meet for any given analysis. In this smoking example, initial analyses of smoking and mortality data could not be deemed successful or not until decades after they were done. Most scientific conclusions require multiple replications occurring over many years by independent investigators and analysts before the community believes or concludes that they are true. Leaving data analysts in limbo for such a long time seems impractical and, frankly, unfair. And ultimately, I don’t think we want to penalize data analysts for making conclusions that turn out to be false, as long as we believe they are doing good work. Whether those claims turn out to be true or not may depend on things outside their control.

A related standard for analyses is essentially a notion of intrinsic validity. Rather than wait until we can validate a claim made by an analysis (perhaps decades down the road), we can evaluate an analysis by whether the correct or best approach was done and the correct methods were applied. But there are at least two problems with this approach. In many scenarios it is not possible to know what is the best method, or what is the best combination of methods to apply, which would suggest that in many analyses, we are uncertain of success. This seems rather unsatisfying and ultimately impractical. Imagine hiring a data analyst and saying to them “In the vast majority of analyses that you do, we will not know if you are successful or not.” Second, even in the ideal scenarios, where we know what is correct or best, intrinsic validity is necessary but far from sufficient. This is because the context in which an analysis performed is critical in understanding what is appropriate. If the analyst is unaware of that context, they may make critical mistakes, both from an analytical and interpretative perspective. However, those same mistakes might be innocuous in a different context. It all depends, but the analyst needs to know the difference.

One story that comes to mind comes from the election victory of George W. Bush over Al Gore in the 2000 United States presidential election. That election hinged on votes counted in the state of Florida, where Bush and Gore were very close. Ultimately, lawsuits were filed and a trial was set to determine exactly how the vote counting should proceed. Statisticians were called to testify for both Bush and Gore. The statistician called to testify for the Gore team was Nicolas Hengartner, formerly of Yale University (he was my undergraduate advisor when I was there). Hengartner presented a thorough analysis of the data that was given to him by the Gore team and concluded there were differences in how the votes were being counted across Florida and that some ballots were undercounted. However, on cross-examination, the lawyer for Bush was able to catch Hengartner in “gotcha” moment which ultimately had to do with the manner in which the data were collected, about which Hengartner had been unaware. Was the analysis a success? It’s difficult to say without the having been directly involved. Nobody challenged the methodology that Hengartner used in the analysis, which was by all accounts a very simple analysis. Therefore, one could argue that it had intrinsic validity. However, one could also argue that he should have known about the issue with how the data were collected (and perhaps the broader context) and incorporated that into his analysis and presentation to the court. Hengartner’s analysis was only one piece in a collection of evidence presented and so it’s difficult to say what role it played in the ultimate outcome.

Audience

All data analyses have an audience, even if that audience is you. Ultimately, the audience may accept the results of an analysis or they may fail to accept it, in which case more analyses may need to be done. The fact that an analyst’s success may depend on a person different from the analyst may strike some as an uncomfortable feature. However, I think this is the reality of all data analyses. Success depends on human beings, unfortunately, and this is something analysts must be prepared to deal with. Recognizing that human nature plays a key role in determining the success of data analysis explains a number of key aspects of what we might consider to be good or bad analyses.

The Role of Narrative

Data analysis is supposed to be about the data, right? Just the facts? And for the most part it is, up until the point you need to communicate your findings to an audience. The problem is that in any data analysis that would be meaningful to others, there are simply too many results to present, and so choices must be made. Depending on who the audience is, or who the audience is composed of, you will need to tune your presentation in order to get the audience to accept the analysis. How is this done? Here are two extremes.

In the worst case scenario, it is done through trickery. Graphs with messed up axes, or tables that obscure key data; we all know the horror stories. A sophisticated audience might detect this kind of trickery and reject the analysis, but maybe not. That said, let’s assume we are pure of heart. How does one organize a presentation to be successful? We all know the other horror story, which is the data dump. Here, the analyst presents everything they have done and essentially shifts the burden of interpretation on to the audience. Rarely is this desired. In some cases the audience will just want the data to do their own analyses, but then there’s no need for the analyst to waste their time doing any analysis.

Ultimately, the analyst must choose what to present, and this can cause problems. The choices must be made to fit the analyst’s narrative of “what is going on with the data”. They will choose to include some plots and not others and some tables and not others. These choices are directed by a narrative and an interpretation of the data. When an audience is upset by a data analysis, and they are being honest, they are usually upset with the chosen narrative, not with the facts per se. They will be upset with the combination of data that the analyst chose to include and the data that the analyst chose to exclude. Why didn’t you include that data? Why is this narrative so focused on this or that aspect?

The Role of Creativity

On one extreme, it could be thought that a data analyst should be easily replaced by a machine: For various types of data and for various types of questions, there should be a deterministic approach to analysis that does not change. Presumably, this could be coded up into a computer program and the data could be fed into the program every time, with a result presented at the end. How is it that every data analysis is so different that a human being is needed to craft a solution? How can the words “creativity” and “data analysis” even appear in the same sentence?

Well, it’s not true that every analysis is literally different. Many power calculations, for example, are identical. However, exactly how those power calculations are used can vary quite a bit from project to project. Even the very same calculation for the same study design can be interpreted differently in different projects. The same is true for other kinds of analyses like regression modeling or other more fancy modeling. The reason creativity is needed in data analysis has to do fundamentally with things that we might traditionally think are “outside” the data.

The audience is a key factor that is “outside the data” and influences how we analyze the data and present the results. One useful approach is to think about what final products need to be produced and then work backwards from there to produce the result. For example, if the “audience” is another algorithm or procedure, then the exact nature of the output may not be important as along as it can be appropriately fed into the next part of the pipeline. In particular, interpretability may not weigh that heavily because no person will be looking at the output of this part. However, if a person will be looking at the results, then you may want to focus on a modeling approach that lets that person reason about the data and understand how the data inform the results. For example, you might want to make more plots of the data, or show detailed tables if the dataset is not that large.

In one extreme case, if the audience is another data analyst, you may want to do a relatively “light” analysis, but then prepare the data in such a way that it can be easily distributed to others to do their own analysis. This could be in the form of an R package or a CSV file or something else. Other analysts may not care about your fancy visualizations or models; they’d rather have the data for themselves and make their own results.

Creativity is needed in part because a data analyst must make a reasonable assessment of the audience’s needs, background, and preferences for receiving data analytic results. If the analyst has access to the audience, the analyst should ask questions about how best to present results. Otherwise, reasonable assumptions must be made or contingencies (e.g. backup slides, appendices) can be prepared for the presentation itself.

“Inconsistent” Results

Many times I’ve had the experience of giving the same presentation to two different audiences. One audience loves it while the other hates it. How can that be if the analyses and presentation were exactly the same in both cases? The truth is that an analysis can be accepted or rejected by different audiences depending on who they are and what their expectations are. A common scenario involves giving a presentation to “insiders” who are keenly familiar with the context and the standard practices in the field. Taking that presentation verbatim to an “outside” audience that is less familiar will often result in failure because they will not understand what is going on. If that outside audience expects a certain set of procedures be applied to the data, then they may demand that you do the same, and refuse to accept the analysis until you do so.

I vividly remember one experience that I had presenting the analysis of some air pollution and health data that I had done. In practice talks with my own group everything had gone well and I thought things were reasonably complete. When giving the same talk to an outside group, they refused to accept what I’d done (or even interpret the results) until I had also run a separate analysis using a different kind of spline model. It wasn’t an unreasonable idea, so I did the separate analysis and in a future event with the same group I presented both analyses side by side. They were not wild about the conclusions, but the debate no longer centered on the analyses themselves and instead focused on other scientific aspects. In retrospect, I give them credit for accepting the analyses even if they did not necessarily believe the conclusion.

Summary

I think my proposed definition of a successful data analysis is challenging (and perhaps unsettling) because it suggests that data analysts are responsible for things outside the data. In particular, they need to understand the context around which the data are collected and the audience to which results will be presented. I also think that’s why it took so long for me to come around to it. But I think this definition explains much more clearly why it is so difficult to be a good data analyst. When we consider data analysis using traditional criteria developed by statisticians, we struggle to explain why some people are better data analysts than others and why some analyses are better than others. However, when we consider that data analysts have to juggle a variety of factors both internal and external to the data in order to achieve success, we see more clearly why this is such a difficult job and why good people are hard to come by.

Another implication of this definition of data analysis success is that it suggests that human nature plays a big role and that much of successful data analysis is essentially a successful negotiation of human relations. Good communication with an audience can often play a much bigger role in success than whether you used a linear model or quadratic model. Trust between an analyst and audience is critical when an analyst must make choices about what to present and what to omit. Admitting that human nature plays a role in data analysis success is difficult because humans are highly subjective, inconsistent, and difficult to quantify. However, I think doing so gives us a better understanding about how to judge the quality of data analyses and how to improve them in the future.