The Thunderbird Project is hiring for a software engineer! We’re looking for an amazing developer to come on board to help make Thunderbird the best Email client on the planet! If you are interested you can apply via the link below, following the job description.

Here’s the job description:

Title: Thunderbird Software Engineer

About Thunderbird
Thunderbird is a email client depended on daily by 25 million people on
three platforms: Windows, Mac and Linux (and other *nix). It was developed by the Mozilla Corporation until 2014 when development was handed over to the community. The Mozilla Foundation is now the fiscal home of Thunderbird. The Thunderbird Council, who lead the community effort, has begun hiring contractors through Mozilla in support of this venture and to guarantee that all vital services are provided in a reliable fashion.

You will join the team that is leading Thunderbird into a bright future. As a software engineer you will be maintaining and improving the existing Gecko-based Thunderbird but also pave the way for its transition to being based on web technologies.

The Thunderbird team works openly using public bug trackers and repositories, providing you with a premier chance to showcase your work to the world.

About the Contract
The Thunderbird project is looking to hire software engineers to help maintain Thunderbird. You’ll be expected to work with community volunteers, the Thunderbird Council, and other employees to maintain and improve the Thunderbird product.

This is a remote, hourly 6-month contract. Hours will be up to 40 a week. You will be expected to have excellent written communication skills and coordinate your work over email, IRC, and Bugzilla.

As a software engineer for Thunderbird you will
* Fix bugs and regressions and address technical debt.
* In collaboration with Thunderbird’s Engineering Steering Committee,
replace/rewrite modules to prepare Thunderbird for the transition to a
new platform.
* Maintain and improve Thunderbird to ensure that both nightly builds
and releases are always possible.
* Follow improvements made by Mozilla engineers for the Firefox platform
process and implement those for Thunderbird.
* Be a self-starter. In a large code-base it’s inevitable that you
conduct your own research, investigation and debugging, although others
in the project will of course share their knowledge.
* Work with both volunteers and employees across the world to fix issues.
* Collaborate with QA, Security, Localization, and Release Engineering
for coordinated code releases.

Your Previous Experience
Since we are looking to fill one or more positions, we are interested to
hear from junior and senior candidates who can offer the following:
* Solid knowledge and experience developing a large software system (7+
million lines of code).
* Solid knowledge of C++ as well as JavaScript, HTML and CSS.
* Ideally exposure to the Mozilla platform as a voluntary contributor or
add-on author with knowledge of XPCOM, XUL, etc.
* Some experience using distributed version control systems (preferably
Mercurial, Git would be acceptable).
* Some prior exposure to Python and build systems (preferably make)
would be beneficial.
* Experience developing software cross-platform applications is a plus.
* B.S. in Computer Science would be lovely, but real-world experience is
preferred.

Check out the Thunderbird Source Code

Want to learn more about Thunderbird and get a sense of the project? You can find the source code and a short tutorial on getting started below:

Source: https://hg.mozilla.org/comm-central/

Getting Started: https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Source_Code/Getting_comm-central

Next Steps
If this position sounds like a good fit for you, please send us your resume with a cover letter to apply@mozillafoundation.org.

A cover letter is essential to your application, as it shows us how you envision Thunderbird’s technical future. Tell us about why you’re passionate about Thunderbird and this position. Also include samples of your work as a programmer, either directly or a link. If you contribute to any open source software, or maintain a blog we’d love to hear about it.

Please note that while the Thunderbird project is a group of individuals separate from the Mozilla Foundation that works to further the Thunderbird email client, the Mozilla Foundation is the Project’s fiscal home. The Thunderbird Council, separate from Mozilla, manages the Project and will direct the software engineer’s work.

The successful applicant will be hired as freelancer (independent contractor) through the Mozilla Foundation’s third-party service Upwork (www.upwork.com). By applying to this job, you are agreeing to have your applications reviewed by Thunderbird contractors and volunteers who are a part of the hiring committee as well as by staff members of the Mozilla Foundation.

Mozilla values diversity. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

Vancouver’s Mobi (by Shaw Go) has some interesting times ahead. Next-gen dockless  systems are edging nearer.  Costs march downwards as competition rears its head.

First is Ubike Technologies Inc (U-bicycle), who’ve put 160 bikes into Victoria, and opened an office in 2017 in Vancouver.  Promo clip HERE.

While it’s one thing to be the first to put bike-share into a city, it’s a new world when the second comes along.

It becomes harder for each to compete, due to revenue split. But the business should eventually drain to the low-cost entrant, as long as other things like coverage and ease-of-use are the same. In this case, U-bicycle charges $1 for 30 minutes, drawn from cash deposited in the rider’s account. Mobi has a much broader and more complex charge scheme based on daily, quarterly or yearly membership fees.  But you’d be hard-pressed to ride enough to get your per-ride Mobi costs down to $1 for 30 minutes.

For the rider, it becomes a choice:  both systems, or one.  A membership fee at one or a deposit at the other. One app or two. For the city, it is more complex.  Several hundred docks, or several thousand bikes at racks and lamp-posts within the coverage area. Stranding a supported system vs. no-cost entrant.  And who knows what lurks behind the scenes in the agreements and understandings with Shaw around their sponsorship.

The concept of integrating Mobi and U-bicycle looks like a non-starter.  Starting with the apps, then cost structure, money-handling, telecommunications, back-end logistics, customer service, fines and pick-up fees, revenue-splitting, cost allocation.  [Shudder: Two bosses].

Second is Dropbike, recently approved by city Council in Kelowna for up to 1,200 bikes in an 18-month no-cost pilot starting April, 2018. Interesting difference to U-bicycle is the Dropbike concept of “parking havens”, where riders are encouraged to leave their bikes when the ride is over. Cost is really low, at $1 per hour, with extra cost if you don’t drop the bike at a haven. Otherwise the two systems are similar, and so are the bikes. Dropbike won’t supply helmets.

Kelowna’s regional programs manager Jerry Dombowsky shows off one of the Dropbike cycles that will be used in the city’s new bike-share program while addressing council Monday.—Image: Alistair Waters/Capital News

Summary: Rich Hickey explained the design choices behind Clojure and made many statements about static typing along the way. I share an interesting perspective and some stories from my time as a Haskell programmer. I conclude with a design challenge for the statically typed world.

Rich Hickey’s Keynote at Clojure/conj 2017 has stirred up the embers of some old flame wars, particularly the static vs. dynamic typing debate. I happen to have an interesting perspective, having worked professionally in both Clojure and Haskell. Some people on each side of the debate seem to be confused about what he was getting at, so I thought I’d share my interpretation of what he meant.¹ I want to shed some light on the spirit of his argument, using some of my experiences.

Context

Clojure was designed to make a certain kind of software easier to write. That kind of software can be characterized as:

• solving a real-world problem => must use non-elegant models
• running all the time => must deal with state and time
• interacting with the world => must have effects and be affected
• everything is changing => must change in ways you can’t predict

Everything Rich talks about in his presentation is within this design context. I think that is lost in some of the discussions I’ve seen online, so I’m just highlighting it here.

He’s never really talking in the context of dynamic typing vs. static typing. It’s always Clojure vs. type systems available to him (though he never explicitly said that, it’s easily understood from his language). This is just a guess, but at the time he wrote Clojure, those type systems most likely were Java-style (Java, C++, C#) and Haskell. It’s not an attack on the theory of type systems or what they are capable of in the general sense. It is a pragmatic argument about whether any existing systems fit his design requirements. Although his language does not perfectly precisely address it, the spirit of his comments are answers to the question “Why invent Clojure when you could just use Haskell or Java?”

But more on this later.

JSON and ADTs

When I was working on Haskell-based web software, we dealt with a lot of JSON, as one does. Haskell had a really neat way of representing JSON as an Abstract Data Type (ADT). It’s nice because JSON is well-specified as a recursive type.

Any JSON value could be represented using that ADT. It was totally well-typed. Except that, even though you knew the type, you still knew nothing about the structure of that JSON. The type system couldn’t help you there.

What we wound up doing was nesting pattern matching to get at the value we wanted. The deeper the JSON nesting, the deeper our pattern matching. We wrote functions to wrap this up and make it easier. But in general, it was a slog. The tools Haskell gives you are great at some things, but dealing with arbitrarily nested ADTs is not one of them.

Some people say that this is the cost of having well-typed code elsewhere–dealing with the untyped stuff is terrible. But imagine if more and more of your code is dealing with JSON. It would be nice if there were some easier way.

Types, positional semantics, and coupling

If you have a procedure with 10 parameters, you probably missed some.

— Alan Perlis

Rich had some good points about positional semantics. When defining an ADT, positional semantics make it hard to read the code. If you’ve got seven arguments to your ADT’s constructor, you’ve probably got them in the wrong order and you’ve forgotten one.

So the answer is “don’t use positional constructors”. Haskell does have constructors with named arguments. And that’s fine, except in a real system, you start with a simple ADT that has two arguments–very easy to keep track of. You use it all over the place, using pattern matching. Its structure is now coupled all over the code.

Then, you need to add one more piece of information. You add a third argument. Then you follow down all of the compiler errors, adding that third argument to your pattern matching statements. Great! Everything is good now.

Except, over time, you add a fourth and a fifth. Pretty soon, you’ve got seven or eight of them. Good thing you have a type system to keep track of all of that. And don’t get me started about merge conflicts when two branches want to modify the ADT at the same time. That’s a digression.

The fact is, in real systems, these ADTs do accrete new arguments. We had a central type called Document that had a bunch. I don’t remember how many now. It has been a few years. Seven? Ten? It was a lot. And I remembered every single one of them because we processed documents, so most code was about Documents.

You may catch them early and give them names. But in some cases, you don’t. So you file a ticktetto change from positional to named, and then to rewrite pattern match expressions in almost every file. And when you prioritize that backlog, that task is probably not as important as adding a new feature. So it stays.

My recommendation to those of you writing Haskell style guides is to disallow positional constructors with more than three arguments. Put a linter on it.

Maybe String

Rich mentioned that Maybe wasn’t a good solution to lack of knowledge. He said “You either have it or you don’t.” This one is the most baffling to me. It’s baffling because it seems very right to me but I can’t say why. I also don’t have any stories handy to explain my experiences with Maybe. Unfortunately, he doesn’t go very deeply into it.

As some people online mentioned have mentioned, Maybe String means precisely “I may (or may not) have a String”. But that’s missing the point. Let me try to explain why.

Maybe is a very neat idea. It’s a type that represents having a value or not. It’s often used as a replacement for nullable fields. In a world where types, classes, or fixed records are used to hold information about an entity, using a Maybe for the optional fields is a way to represent that optionality.

In the kinds of systems Rich is talking about, any data could possibly be missing–if not now then at some point in the future when requirements change. At the limit, you would need to make everything Maybe. And if that’s the case, I would propose that maybe, for this context, you want to move that optionality into the information model, not the domain model. Your Person type is part of the domain model (it says: these are the bits of information we care about for people). The fact that any bit of information could be missing is part of the information model. The information model might contain other data, like when the information entered the system.

I’m reminded of my friend’s company. They do medical record software. The standards for medical records have hundreds of types. And each version that comes out changes things slightly. On top of that, you’re getting data from systems that don’t implement things properly. And on top of that, humans are entering the data incorrectly, which isn’t surprising since the spec is hundreds of pages long.

In a system like that, you can’t write correct types for every kind of entity. You’d never finish before the new spec came out. Instead, you need to think at an information level. One approach is to do your best with the information you need to work with and pass along the rest as-is. Regardless of how you decide to handle it, sprinkling Maybes around isn’t going to cut it. Nor are fixed entity types. You really need to take it up a level to build a robust information model.

Types as concretions

Rich talked about types, such as Java classes and Haskell ADTs, as concretions, not abstractions. I very much agree with him on this point, so much so that I didn’t know it wasn’t common sense.

But, on further consideration, I guess I’m not that surprised. People often talk about a Person class representing a person. But it doesn’t. It represents information about a person. A Person type, with certain fields of given types, is a concrete choice about what information you want to keep out of all of the possible choices of what information to track about a person. An abstraction would ignore the particulars and let you store any information about a person. And while you’re at it, it might as well let you store information about anything. There’s something deeper there, which is about having a higher-order notion of data.

A focus on composable information constructs

Sure, we get very few guarantees about the data we have in Clojure. We get that. But if we assume we’ve got some basic things right, like that we have a map when we think we have a map, we do get some nice guarantees. For instance, associng the same key and value is idempotent. That we can access the value for any key in constant time.

And Haskell has that for ADTs. But can Haskell merge two ADTs together as an associative operation, like we can with maps? Can Haskell select a subset of the keys? Can Haskell iterate through the key-value pairs?

Of course, you could build a type with all of those properties and more in Haskell. A universal information model, with a convenient library of functions for dealing with them. That universal information model would look something like the JSON datatype or a richer one like an EDN data type. And your library of functions for dealing with them would look something like Clojure’s standard library. But just like with the JSON type, you’d have very few static guarantees. At some point you’re just re-implementing Clojure.

Types are a design space

Okay, now for some pontification. A type system gives you a new design space. It lets you express things, like type constraints, that you simply don’t have to in untyped languages. That gives you a lot more choices to make. Choices can be good. But they can also be a distraction.

Rich Hickey mentioned puzzles as being addictive, implying that it’s fun to do stuff in the type system because it’s like a puzzle. It’s similar to the Object-Oriented practice of really puzzling out those isA relationships. It very much is like a puzzle: you’ve got some rules and an objective. Can you figure out a solution? Meanwhile, it gets you no closer to the goal.

I’ve definitely experienced this myself in Haskell, both as the puzzler and on onlooker. This may have changed in the four years since I’ve done Haskell professionally, but I often found Haskell libraries to be puzzle solutions. Someone wanted to figure out how to type some protocol to “make it safe”. They usually succeeded at getting a partial implementation before giving up. On many occasions, after looking through several attempted implementations of a simple protocol, no existing libraries fit our bill.

Like I said, this may have changed. But I’m confident in asserting that there’s a danger with getting carried away with the complexity of your types. Haskell gives you plenty of rope to do that.

Type systems are a design space

The language’s type system itself is a design space. In addition to the syntax and runtime semantics of the language, typed languages add on an additional space for exploration. This inherently makes them harder to design. They simply have more choices to make. I would say that Clojure and Haskell are both well-designed languages. And now, thanks to the talk, we know what Clojure was designed for. What was Haskell designed for?

What Haskell was designed for

I don’t think it’s any secret that Haskell was designed as a lingua franca for functional programming and type theory research. It famously “avoids success at all costs” by sticking to its principles of purity and typeability.

Of course, many people and companies are successful using Haskell. People have learned to wield the tremendous power of the type system. I think that points to something inherently valuable about Haskell and the other typed languages. Lots of parts of a system are inherently typeable–meaning they do have small, clear, and stable types. Those parts aren’t as chaotic as the outside world. They can be well understood and codified into precise types to great advantage. And there’s no reason the information model couldn’t be either. Of course it’s possible. But has it been done?

Design challenge

It’s easy to hear critiques of static typing and interpret them as “static typing is bad”. Rich Hickey was certainly not doing that. He had a particular problem to solve and was noting how static typing was not particularly well-suited to that problem. Maybe doesn’t solve the problem. Person and other entity types don’t solve the problem. They are useful for some things, but not for this particular problem.

I see the gist of his talk not as a condemnation of static typing, but instead as a design challenge to Haskellers and language designers. Many people online brought up Haskell extensions or row polymorphism, or some feature of Purescript, etc. These are all great. The challenge is to piece together an actual solution to the problem of “situated programs”, not just point to features that might address one issue. Perhaps you can do it in pure Haskell. Maybe you need some extensions. Maybe it’s an entirely new language. Or maybe you dismiss the challenge as unimportant–that his description of “situated programs” is not the right way to look at things.

But I don’t see people doing any of those things. What people are doing is saying “but of course that’s possible in principle, so the argument against static typing is invalid”. It’s not about what’s possible, it’s about the particular designs of the actual languages we have to choose from. He criticized particular features used for particular purposes. Maybe is not it. Pattern matching is not it. Entity types are not it. What is the static typing solution to the entire problem?

Conclusions

When I was working on the Haskell system, I really missed having a more flexible model of information. So much of our code was about taking form fields and making sense of them. That usually involved trying to fit them into a type so they could be processed by the system. Like Rich mentioned, the information-processing part dominated the codebase.

When I moved into a Clojure job, I felt such a sense of freedom. I could treat information as information and not have to prove anything to a type checker. However, I had the benefit of having traversed the gauntlet of a strict static type system. My mind had been seeded with the thought-pathways of type correctness. My Clojure code benefited and I still look in horror on some Clojure code that plays too much with types.

I sometimes wish I could tighten down some Clojure code with static type guarantees. These are in small, remote portions of the code that have intricate algorithms. Static types would be very useful there. But that’s the exception. In general, I like the flexible information model approach of Clojure better than the statically-typed approach of Haskell.

Elm’s design seems superbly crafted for its purpose of building rich, interactive web interfaces. Is it possible with our current understanding to build a statically-typed language that rivals Clojure for its intended purpose? What would such a language look like? What types would it have? Who will build it?

The post Clojure vs. The Static Typing World appeared first on LispCast.

Generative testing is great for testing pure functions, but it is also used to test the behavior of stateful systems that change over time. Those systems are often designed for highly concurrent usage, such as queues, databases, and storage. And then sometimes your code is also asynchronous, such as in JavaScript. Learn several patterns to find inconsistencies across platforms, race conditions, and corner cases. This talk assumes basic understanding of generative testing.

Slides

Download the slides

Transcript

Hi. What’s happening in our industry would have been unthinkable ten years ago. More people than ever–an amazing number of people–are happily employed writing Lisp and doing Functional Programming. I think that’s awesome.

More people than ever are asking “how do I develop a functional mindset?” And generative testing is a really good answer to that question. There’s nothing like generative testing for forcing you to think about the properties of your system as a whole. And there’s nothing like generative testing for honing functional thinking. And I am honored and privileged to teach it to you today.

The title of this talk is Testing Stateful, Concurrent, and Async Systems using test.check. My name is Eric Normand. I run a company called PurelyFunctional.tv. There’s a newsletter. It’s free. You should sign up! It’s meant to inspire Clojure programmers. It’s weekly. It’s about ten links every week.

I also have a members section with over thirty eight hours of Clojure instructional videos. So, if you’re interested in learning Clojure, or you need other people to learn Clojure at your company, check out my offering. Get in touch with me if you have any questions.

So, a map of the talk. First, we’re going to develop a few example-based tests that we can then pick on as a strawman. Then we’re going to use those same tests but rewrite them generating the data instead of hard-coding the data. Then we’re going to develop tests that test a sequential–basically, generate the whole test. And then we’re going to add parallelism to this so we can test for race conditions.

A show of hands. Who who tests their system? Okay, that’s just about everybody. A few people weren’t paying attention. Who uses generative testing? Alright, maybe a half half of you. Okay, cool! Well, I hope that the half of you that don’t use it start to use it by the end, and the half of you that do use it learn some new techniques for doing this.

Alright, so let’s start with our system that we’re testing. It’s a very simple system because I only have 40 minutes. It’s just a key-value database. This database might be on another server and you’re sending messages over the network. It could be on the same server using like RPC or something. And then it also could be in-memory. It doesn’t matter.

What’s important is the interface. It has these operations. You can create a new one empty. Then you can clear it. That just gets rid of all data in it. You could store a key-value, delete a key, fetch a key and get the value out, and then ask for the size. Okay so there’s five operations. Pretty simple. Does what you expect.

So let’s write some tests using clojure.test. We can define a test called store-contains. We want to store a key and see if the database will fetch it back out for us. Pretty simple test so we created database. Create a key and a value. Store it. Fetch it out and then assert that. It’s equal to the value that we expect.

Another test. This one is that the second store is going to overwrite the value for the first score. So we create a key and two values. Store them both in order and then when we fetch it out. We expect it to be equal to the second value. Make sense?

Okay, clear-empty. After I clear, there shouldn’t be anything in it. So I’m going to store something in it just to make sure that I do actually clear something. Then I’m going to clear. Check the size should be zero.

I don’t want you to feel bad about these tests. I write tests like this too but you should feel bad about these tests. And I know when I say feel bad with testing, there’s a lot of shaming going on with TDD. And like whether you’re a real programmer if you don’t do TDD. And that’s not what I mean. I don’t mean you should feel guilty. I mean you should just be scared these tests are not doing very much. They’re testing a very little bit of your system. Just to put this in perspective, you might think well it’s a very simple system. There’s only five operations possible and like it’s not that critical. Right?

Let’s compare it to a system like this. This is a cockpit of an airplane. A lot of switches and dials and gears and legs and you know, other things you can do. And what if I wrote tests like I did for the key-value database but for this?

So I say I’m going to press this button and then flip that switch. I’m going to test it. It did the right thing. Okay, now I’m going to flip this switch and turn this knob and tested. It did the right thing and then I’m going to turn to move this lever and put that switch and test if it does the right thing.

Would you fly in this airplane? If those were the only tests that I ran right and so this system looks complicated. It looks big. There’s a lot of possibilities here and it’s a very dangerous critical system. But what is the size of our system?

This database that we’re writing. Let’s just ask a few questions and think about so how many strings are there. But if my keys and values are strings, that’s a lot of different inputs that are possible. Infinity. How many unicode characters are there? Thousands? How many key-value pairs are there? A lot. Infinity. How many operations are there? Five. How many pairs of operations? Actually 25. Five squared.  How many triples of operations? 125. And I mean it goes up from there right?

So this is actually a really big system. But what is different is, compared to the cockpit, is that the description is small, right? It’s only five operations and you can kind of explain it in a few sentences. So we want to be able to capture that small description in our test–which we haven’t done if we just write all the examples. So I’ve had three tests. You could expand that out. 225 doing the pairs of operations. Still you wouldn’t come close to testing the whole description.

Alright, so let’s get into generative testing. First idea you have to know is generators are objects that generate random data of a given type. So the data we need are keys and values and we’ll just say their strength. So this is saying keys are random strengths and values of random strings.

We can sample our generator and get 20 values out into what it looks like. This is just random. You run it again, you’ll get different stuff but you noticed already. We’ve got way more interesting data than our dinky string with a character in it. We got an empty string that’s really interesting usually.

A good corner case we’ve got some characters that I don’t even know what they’re called or how to type them. You got this like over score one over here kind of like that one divide sign that’s nice.  Anyway, there’s much more interesting rich data here.

Okay, now we’re going to set up properties which are kind of like saying we can translate our tests. Our original tests into properties. Properties are just assertion on our system.

We’re using the defspec which hooks into the clojure.test. Test runner store contains the name of our test and we’re going to run a hundred tests or didn’t generate a hundred different types of this. So we’re using a prop for all macro. And what this is basically saying is select a key from that generator and a value from that generator. Create the database store. The key-in value and then fetch it out and compare it to the right. Saying it’s basically the same code as before. And notice it returns a Boolean that the Ariel if it returns false and the test fails, you don’t have to use the macro.

Alright! Next test is to overwrite this one. We’re going to set a queue and to be store them both and then we fetch it out and compare. It should be equal to v2 same code.

This one I’m not going to go through line by line but we’re storing the random kV and then fetching ID and clearing it and checking the size.  Alright, what is a failure look like?

Failure. You’ll get a map that reports what happens if we find a failing case. It’s got Kiev results false because we returned false when they eat. When they’re not equal. Store contains is the name of the var that’s interesting. Failing signs? There’s a bunch of stuff in here that I don’t really use that much.

I’m not going to go over it but this is the K and V that were generated for this failing test case. Notice it’s a really long crazy string that failed. I don’t know what to do with that. I can’t debug that. Why did that fail? I don’t know. But luckily, what test check does is it will shrink it for us. And what shrinking means is it takes that string. Say this K and it removed the character from it and sees and reruns ITAT. And says does it still fail. Okay. And if it does, then it removes another character and then runs the test again. And sees does it still fail and it keeps doing this until it can’t find any failing.

Another thing that we’ll do besides just removing characters is it will decrements the character code and try that string and so it keeps doing. That keep doing that until it won’t fail anymore. And then, it returns the smallest one. That still failed and notice how small it is. So with this, I can go in and debug and it turns out that there’s a bug with that Ashe character.

That was a special character and my code went through a code path that checked for that and crashed. And it gives you the feed so you can rerun the same test. I just checked the test bar. Figure out which one failed. And then the smallest to see if I can reproduce it. You can also take this smallest and turn it into an example based test so you don’t have a regression on that test.

Okay. But the thing is, we could sit there and write 25 tests like this generating random data. But we’re still not capturing the description of the system that I was talking about. That really small description. What we want to be able to do is describe the entire behavior and test it in one go.

Okay. So how do we do that? It’s actually quite easy. First, there are four steps. Okay. The first step is we’re going to build a model of our system. It turns out that a key value database—the properties that we’re interested in, is the same properties that hashmap has. Alright. Keep all your database stores keys and values hashing. Have source keys and values same overwrite properties, same clearing property, everything’s the same.

Okay. Step two. We’re going to take the operations and reapply them and what we want to do is run these operations on our database. Run them on a hashmap and then at the end, compare that they’re equivalent. That we got the same answer. So we’re going to make generators for all of our operations or turning them into data. We’re using the gen/return generator which just always returns a constant value and so we’re returning a tuple. That starts with the keyword clear with the same thing with size. It’s just a constant tuple with the size because there’s no parameters. So we just need to generate it.

Next thing is we have store. Store has two parameters so we’re making a tuple that starts with the keyword store and has a random peon value in it. Same with delete except we just need the key and then fetch tuple with the constant keyword. Fetch and a key then we can make a generator that generates a vector of these and we use the one of generator which chooses a generator from this collection. And then chooses a random value from that generator.

So in the end, we’re going to get a vector of operations. Let’s sample it to see what it looks like so we could have the empty vector. So no operation. We see one with clear another empty one. We see one with a fetch and a clear or clear a fetch. A clear fetch and a side. Side so we’re getting some really weird tests. We would never write these tests but that’s interesting right? We didn’t have to write them. They’re just generated for us automatically.

Alright. Now we need to make the runner to run these operations on the database and then once run them on the hashmap. Let’s start with the database. Take a database and a vector of operations. We’re going to iterate through these operations we’re destructuring. So we’re getting the name of the operation and the key and value if they exist. We’re going to dispatch on the name. It’s a clear. We’re going to call clear. If it’s size we call size and they’re all the same right? 

We’re just calling the corresponding function for our hashmap runner because it’s an immutable value. We’re going to use reduce. So notice where our initial value is. The database that we’re getting it’s a hashmap but we’re calling in the database and we’re reducing over the operations. You’re going to destructor it same thing but this time I have to say we’re only interested in the mutation properties of these operations right? So we’re going to compare them at the end. We don’t care if intermediate fetches are giving us the right value. We could care but in this example we’re not okay.

Alright. So clear. Just sets it to the empty map size does not mutate. So we just returned the hasmap we got. Store is like a socially disliked dystocia and fetch doesn’t modify it. So we’re just going to return the hashmap.

Alright, now we can define a property we’re calling it hash map equiv. We’re going to generate a sequence of operation, we’re going to run our ops on our hash map and create a refresh database then run our ops on the database and then compare them that they’re equivalent. We don’t have that function defined so let’s define it a quiz. What does that look like? Take the database in the hash map first, it compares the size of the hash map to the size of the database. Make sure they’re equal and then it’s going to go through every key value in the hash map and make sure that the equivalent can value are in the database.

Okay, there’s a problem with this though, I’m developing this incrementally. I’m writing the naive code and then making it better each time. The problem is when we’re generating keys, we’re generating random keys each time we generate keys so it’s a bit like so we have this set and we have a store and each one is getting a random key. It’s a bit like I asked 10 of you in the audience to choose a random star in the galaxy and expecting that some of you would choose the same star right? We want to test that if I do a store and then I do a fetch or I do a store and a delete with the same key that there’s some collision right, we’re not testing that they’re the same teeka sort of very unlikely to choose the same key.

So what do we need to do? What we could do is I could choose a solar system in the galaxy and then ask you now the 10 of you choose a planet out of that solar system and because the solar system is really small I I can guarantee you well there’s a higher probability of having key collisions. Alright, so we want to still possibility that there are no collisions but it’s much higher probability that there are.

So we want to encourage these key collisions. How do we do that? Really easy trick so these are the same, it’s just the same generators there’s no parameters but here with store we have keys we’re changing it from a generator to a function that returns a generator. So this function takes the set of keys. It’s like we’re passing in the solar system to choose from we generate a tuple and notice the keys here is selecting a random element from that Keys collection. We do the same trick with delete, we are selecting a random element from the keys collection that we sin same thing with fetch, now we have this helper function that takes the keys it generates this vector and notice we’re passing the same keys that we get an argument to the individual functions. So all of these are going to be selecting from a small set of keys. It encourages collisions and then we just here’s where we generate our solar system our small set of keys the non empty vector of keys and then we pass it to jenoff star and that will give us a sequence of operations with a good number of key collisions we can sample.

Now these won’t have collisions because they don’t have two operations with keys but this one has a collision notice.  Whether you loved twice, this one has a collision so I just wanted to double check that I’m getting some collisions in there so I get curious about like what am I actually testing how big is this system that I’ve got so just using the repple I’ve um made this query like how big is it if I generate a hundred sequences of operations what’s the biggest one how big am I getting here it turns out I got 91 when I generated 100 random samples of op sequences the biggest one was 91 long what about if I do a thousand I got 96 so I think there’s some like diminishing returns to like the size of what you’re doing I you know there’s a normal curve here and you get out further and further you’re just getting like these smaller and smaller outliers but just out of curiosity if I have five operations ninety one long how big is that system four times ten to the 63 there’s a really big system and I’m only generating country how does that right so it’s it’s not like I’m covering like the whole system here but what is interesting is when you get something that long you do have a lot of those pairwise interactions triple wise interactions going on right ninety one operations you’re getting some interesting interactions going on.

Oh what does the failure look like now? We rerun our test with these new generators. Here is a failure this is the operation sequence that we ran. It’s actually 49 long. This one crashed or broke then in detail to meet our expectations. But look at the smallest look at the shrunk size like that.

I can actually go in and debug, I can look, I’ll look store broke when I passed it this string and in fact that is the exact bug that I introduced so it’s really easy to debug. Okay, so we’ve we’re able to describe our system we’re basically saying it’s like a hashmap very short description but we live in a multi-core world this thing is going to be accessed in the network.

What about race conditions? How can we test for those generative testing can do it? So what we need to do is make a runner that runs in a new thread. So the top function here just starts a new thread and it calls DB run. Alright so it’s just running our thing in a new thread and then we can have a vector of sequences and we just run bang our running thread over all that so for every op-sequence in there. We’re making a new thread and running them all but it’s more complicated than that because we don’t know when the threads are going to finish so we can’t just compare them equivalent until all the threads are done.

So we need to do this trick where each thread creates a promise and then after it finishes, its DB run it will deliver on that promise and we return the promise from the function so then when we do our thread run, we have to remember that thing so we’re doing a map. Map is lazy so we have to do run it to start all the threads and then we’re going to run bang d ref on it which is going to block until all the promises are done.

Okay there’s another problem because if we have a lot of sequences, it could be that we start the first thread and it finishes before we even get to the end of the list and start the last thread. We want them all to start at the same time so there’s another trick it’s called a latch. Like a latch, you’re saying you know like a latch that it’s going to release the the gate that all the horses start at the same time right? You want all those racehorses to have the same chance of winning so we passing the latch up at the top and then right after we start our thread.

The first thing we do is we blow on the latch. Okay, so when we do thread run we create that latch we pass it here to all of the run and threads. We start all the threads then we deliver on that latch. They all unblock at the same time then we wait for them all finish.

Okay, now we have to write our property that uses this new thread runner. We generate two op sequences A and B. You can generate more but start with two. We’re going to combine them into one sequence and we run our hash map with that off sequence. We create a database do thread run with ops A and knob speak and then we check it they’re equivalent.

Okay this won’t work okay? We need a few more things we’ve got the same problem again where now we’re independently generating two op sequences and there’s no key collisions between them so we’ll never have a store in this thread and a delete in this thread of the same key right?

So we need to solve the same problem again which is really easy so we’ll make a function that will generate n sequences for us we’re going to generate that set of keys that solar system of possible keys then we make a tuple of the Gienapp star where we’re passing in the same keys to all of them

so now we use that gen op sequences to we’d be structure it into into two variables and now we run it and we have two collisions between threads.

Okay, the next problem with this it won’t work yet. I need to explain graphically so we have these two threads and let’s say that they’re just running two operations each how and we’re sending messages to the database so they’re getting queued up in a certain order. They’re received at the database in a certain order. What order they received, we don’t.  Could be that all of A is come before all B. Could also be that B’s happen for A or this one or this one

turns out that they’re six and it’s like a tree. So if you choose at any point A is going to happen next or B is going to happen next you can you know left and right. Make this binary tree and there are six possible interleavings of these things and you can’t know what interleaving happens or you can’t control which one will happen because that’s the nature of threads in on the JVM. So the best we can do is say that well, we know one of these happened and so is is the plus is what we got equivalent to one of these any of them we don’t know which one so we’re just going to say as long as it is equivalent to something that could have happened.

We’re good, this generates all possible interleavings. I’m not going to go over the code but it’s basically doing that tree building out the whole tree.

Okay, so instead of combining naively the two opt sequences, you think can cat like I did before which only puts A’s always before all the B’s. This is generating all possible interleavings. We create the database, do the thread run, and then we’re going to compare it to all possible ones running on the hash map and see if at least one is is valid. It is we pass.

okay, so this will work the problem is very often race conditions are about which interleaving. Actually, did happen and so you’ll run a test and it fails and you run it again in passes. That’s like a heisenbugs right?

So how do we suss out those heisenbugs the way that I found? That works and that is also in some of John Hughes papers. John Hughes’s is the creator of quick check (the original generative testing system) what they do is they run it ten times. They just run it ten times and they check that it passes every time. If it fails, once then the whole system fails for the whole test fail and that repeat, repetition happens during the shrinkage – so it turns out that in practice, ten is good enough to find the bugs that you need to find.

Okay, so now there’s this other problem and if this will work but when you get it you might get a failure like this that won’t shrink. This is the shrunk version, it’s so long you’re like what is it doing it’s doing like a fetch and the lead of something that isn’t even in there another dilly wide impact shrink away and you read John Hughes paper and you figured it out it’s a timing issue when you have something like this it’s a timing. It’s doing a whole bunch of operations just to get the timing right? Because it takes time to do a fetch, it takes time to do a delete and so thread B just needs to wait a little bit of time. So what do you do? Well, you need to make it so that this can shrink to like a weight.

So we need a new generator for a new operation called sleep and it’s a tuple with the sleep keyword at the beginning and then it just chooses a number of milliseconds between one and a hundred. Don’t make it too big or your Tesla Marisha stop. Right, so now we have to add sleep to this jenoff star so that it’s selected as one of the possible operations and another thing I did was, I reordered them because now we’re thinking about shrinking. I reordered them from least destructive to most destructive because it turns out that this one of generator will shrink to the beginning of the collection so from you know the right-hand side to the left-hand side and so now the most destructive thing is going to or the disruptive things are going to shrink to less destructive things. I don’t know if that really makes a difference but makes me feel better.

Okay, so we’re going to add thread the sleep to the DB runner and it’s actually going to do a thread sleep easy.

now in our hashmap runner. Sleep doesn’t even touch the database so it’s not going to modify it. So we just return that map.

and now when you run it and it shrinks, this is what you get, it’s shrunk to sleep sixty six inches will shrink to sleep sixty five they needed, sixty six milliseconds exactly and what it turns out is this bug was that if you store twice and then delete in another thread, it gets the wrong answer. So this sleep is enough time to do two stores in thread A less than that and it could get interleaved between the two stores or you know happen before the first store. So now we can see the problem right? It’s very clear because there’s not much noise.

Okay, so I wish I had more time to go into the async stuff. I know the title had a sink in it and I just don’t have time like this was already like a lot but there’s two questions remaining. How do you test the system where you don’t have an easy model like a hashmap that you can just use like already? And I bet a lot of you have that question. The other question is what about async stuff? There’s no async in here. Notice all the operations were synchronous. I did a delete and I just waited for it to return and then I could guarantee that it was done when you have multiple systems. Multiple you know, like over a network and they have latency and stuff you have an eventual consistency model. Maybe you have cache right? So I sent, I write something gets written to my cache and sometime later it goes to the server and then gets inked down to the other clients you have. It’s a much more complicated system but it turns out that the answer to both of those is the same which is to develop a richer model and it’s actually not that hard.

There’s papers about it. It’s this model is still very has a still a very small description and I would love afterwards to sit down with some pen and paper and sketch it out for you. Please, please do that but right here, what I want to show is  the left, we have 25 tests that are written in you know, regular example based testing style and they, each test the two operations and then some make some assertion and this is all the code we developed in this tal. Right, nothing till now so not only are we like way less code but we’ve also modelled a much more complete notion of what the system is supposed to do. We’ve captured its behavior and we can generate as many tests as we want. We can run this over night, we can have it running in our CI server for hours like all the time. Basically, we’ll really change them like the model of paying for CI servers. Right, because right now, they’re relying on us to write really small short tests that take like seconds to run and now we were saying well we can generate tests all all day long. Interesting! Okey, so if you want more talk to me but if you’re not here you’re on the if you’re on the recording go to this URL and you will get more talks about generative testing and about how to do the more async stuff and develop your own model. There was a talk at Clojure Conj of 2015, where Benjamin Pierce talks about doing this for dropbox so it’s good for existing systems and even black box systems where you don’t even have control over what it’s doing or any introspection into how it’s working so thank you very much.

The post Testing Stateful and Concurrent Systems Using test.check appeared first on LispCast.

mkalus shared this story from The Omni Group.

I could write about the recent changes that came to our Stencil Browser in OmniGraffle 7.6 for Mac, but Dan and Mark made a super casual, extremely informative video that covers all of the reasons why it’s so much more helpful to nearly all workflows.

Some of the highlights:

• The Stencil Browser can now be positioned in the left or right sidebar, or the two views you’re familiar with: popover and detached window.
• You can add new objects to any stencil by dragging them from the Canvas to the Stencil Browser! (Just hold Option while doing it.)
• Multiple Stencils can be selected for use at a time, rearranging or reorganizing by with Folders is a snap, and more.

You can read more about the release here.

We setup our second shared hosting server in Europe last week in Digital Ocean’s London-based data center. Originally it was in response to the poor performance we were having with our Kraftwerk server in Frankfurt. As fate would have it, Kraftwerk is running better than ever since we set this new server up, but we are still ready and willing to take any request to be moved off Kraftwerk onto, wait for it …. the Wire server. 

Our own correspondent is sorry to tell
Of an uneasy time that all is not well
On the borders there’s movement
In the hills there is trouble
From “Reuters”

Named after London’s punk pioneers that were eluded by mainstream success of bands like The Clash, Sex Pistols, and The Ramones, but had arguably as great an influence on everything from hardcore to post-punk to alternative music of the 80s, 90s and beyond. Their debut album Pink Flag has become a classic, and to steal a quote from the Wikipedia article:

Steve Huey of AllMusic opined that Pink Flag was “perhaps the most original debut album to come out of the first wave of British punk”

That’s something. It’s worth listening to all 22 songs, the shape and form of the album displays obvious influence on The Minutemen‘s Double Nickels on the Dime. It defies an simple definition of punk, hence its wide influence, and in many ways captures the spirit of musical exploration around the idea of punk before that word morphs into a genre-defining set of characteristics that come to dictate the form in the 1980s. My first exposure to Wire was through Minor Threat’s cover of their song “12XU:”

I remember listening to them based on the cover and thinking they’re not punk. That might provide a small, solipsistic sense of how alien they could seem only 8 or 9 years after releasing their debut album. But listening to them 30 years later they’re fresher than ever. So, in honor of timeless British punk, Europe’ second shared hosting server, and the UK’s first,* is named in their honor. 

Special thanks to Anne-Marie Scott who was willing to help us make sure this one worked by allowing us to migrate her sites before the announcement, and we can confirm no blog posts were lost during the transfer  

*Might be worth noting this is not our first server in Digital Ocean’s London data center, we also host Coventry University’s instance of Domain of One’s Own through this data center, but given that is not a shared hosting server it is fair to say Wire is the UK’s first shared hosting server. 

Apple will start selling the HomePod in select markets in early February: US, UK, Australia. That is obviously a language thing. You can talk to HomePod, and this is their first entry into farfield voice recognition. When it ships, the software isn't ready. It still runs on AirPlay, not AirPlay 2. It won't do multiroom or stereo pairs. A software update will bring these features later.

Although it is a very limited product — Apple only talks about Apple Music — I am still very excited about HomePod. Apple "gets" some important features like room calibration and beam forming. Btw, did you know that Sonos Playbar is using beam forming for five years now? I think, Apple is the only other player besides Sonos who gets these things. And they are not just copying like Google does with their speakers. I am looking forward to the HomePod, when it finally ships to Germany "in the spring".

DJI introduced another marvelous drone today. It's called Mavic Air. You probably read about it everywhere. But that's not what I want to talk about. These machines should appeal to me. I mean, look at them. Look at all the things they can do. How they can avoid crashing into things. How they can stabilize their cameras like a tripod. And still, I don't want one.

I think I would use it for a little. And then leave it at home. Videos and photos from higher altitudes are fascinating at first, but get boring rather quickly. And flying a drone is frowned upon. They are noisy, they are invasive, and they are being more regulated for a reason. I could shoot a few interesting videos like chasing the dog as it hunts birds, but quickly I would find out that the very moment this happens I cannot get it into the air quickly enough.

It's one of the things I can enjoy when my friends have one. I like to look and be happy for them, but I don't want to own one.

Intro

This post is an experiment I decided to attempt after conversations with Ben Lesh and some other folks. I will assume as little knowledge of Haskell as I possibly can here. Later we’ll talk about some tools we have in Haskell to make the pattern more conceptually compact.

I hope to make this accessible to as many people as possible, and I’d love to hear from you if you think there are things I could add or clarify in order to do so.

Intro to Haskell

If you can already read Haskell at least a little, go ahead and skip this section. I will annotate the code in the examples, but if you’ve never read Haskell at all, then this section may be helpful to you.

All data in Haskell is (statically) typed. Types may be concrete, such as Integer or Bool, but there are also type constructors¹. Type constructors must be applied to a type argument in order to become a concrete type and have concrete values – the same way a function would get applied to an argument and then evaluated. So, we have a type, called Maybe that looks like this:

data Maybe a = Nothing | Just a

This datatype says that a value of type Maybe a is constructed by applying Maybe to another type; a is a variable so it could be almost any other type that we apply it to. We could have a Maybe Integer or a Maybe String, for example. It also says that we have either a Nothing value, in the case where there was no a that we could construct a Maybe a value from, or (this is an exclusive disjunction, known as a sum type) a Just a value, where the a has to be the same type as the a of Maybe a.

If we’re constructing a Maybe String value then we can either return a Nothing (where there is no String) – a kind of null or error value – or a Just "string". We use this type very often in cases where a possibility of not having a value to return from some computation exists – a String might appear, on which we can perform some future computation, or it might not, in which case we have Nothing.

Next let’s look at case expressions, a common way of pattern matching on values to effect different outcomes based on the matched value. We’ll start with this one that can remind me how bloody old I am:

function xs =
  case (xs == "Julie") of
    --     ^ equality function returns a Bool value
    True -> (xs ++ " is 43.")
    False -> "How old are you?"

When this function is applied to an argument that is equal to the String “Julie”, it will match on the True (because == reduces to a Bool) and concatenate “Julie” with ” is 43.” Given any other String argument, it will match on the False.

Note this doesn’t include any means of printing any of our strings to the screen; if you want to play with it in the REPL, you can, as GHCi always runs an implicit print action. It’s at the top of the code file that goes with this post.

A case expression in general looks like this:

function =
  case exp of
    value1 -> result1
    value2 -> result2
    -- ... (the pattern matches should
    --       be exhaustive)

The values are the patterns we’re matching on. They must be of the same type, the same type as the result type of exp, as Bool is the result type of == so we had the values True and False in our previous example. They should cover all possible values of that type.

When such a function is called:

• exp is evaluated (ignoring Haskell’s actual evaluation strategy), which means it will be reduced to some value;
• the result value is matched against value1, value2, and so on down;
• the first value it matches is chosen and that branch is followed;
• the result of matching on that value is the result of the whole case expression.

A typical specimen

A consequence of the focus on taxonomies in 17th and 18th century was the creation of museums, which present the studied objects neatly organized according to the taxonomy. A computer scientist of such alternative way of thinking might follow similar methods. Rather than finding mathematical abstractions and presenting abstract mathematical structures, she would build (online and interactive?) museums to present typical specimen as they appear in interesting situations in the real-world. – Tomas Petricek, Thinking the Unthinkable

OK, let’s say we need to validate some passwords. We’ll start by picking some criteria for our users’ passwords: we’ll first strip off any leading whitespace, we’ll only allow alphabetic characters (no special characters, numbers, or spaces) and we’ll have a maximum length of 15 characters because we want our customers to choose unsafe passwords.

We’ll write each of our functions discretely so we can consider each problem separately. First, let’s strip any leading whitespace off the input:

stripSpacePwd :: String -> Maybe String
stripSpacePwd "" = Nothing
-- this first step gives us an "error"
-- if the input is an empty string
-- and also provides a base case for the
-- recursion in the case expression
stripSpacePwd (x:xs) =
  case (isSpace x) of
    True -> stripSpacePwd xs
    -- is recursive to strip off as many
    -- leading whitespaces as there are
    False -> Just (x:xs)

This (x:xs) construction is how we deconstruct lists (Strings, in this case) to pattern match on them element-by-element; the x refers to the head of the list and the xs to the rest of it. We test each x in the string to see if it is whitespace; if there is no leading whitespace, we return the entire string (wrapped in this Just constructor) – the head, x, consed onto the rest of the list, xs. If there is whitespace, we return the tail of the list only (xs) and call the function again on the tail, in case there is more than one leading whitespace. If you give it a string of all whitespace, it’ll hit that base case and return Nothing – we have no password to validate. Otherwise it will stop when it reaches a character that isn’t whitespace, follow the False branch, and return the password.

Next let’s make sure we have only alphabetic characters. This one is less complex because we don’t have to (manually) recurse, but otherwise the pattern is the same:

checkAlpha :: String -> Maybe String
checkAlpha "" = Nothing
checkAlpha xs =
  case (all isAlpha xs) of
    False -> Nothing
    True -> Just xs

We’re again returning a Maybe String so that we have the possibility of returning Nothing. (In a “real” program, that could allow us to match on the Nothing to return error statements to the user, for example. We’ll see other ways of handling this in later posts.) We used isAlpha which checks each character to see that it’s an alphabetic character and all which recursively checks each item in a list for us and returns a True only when it’s True for all the elements.

Finally, we’ll add a length checker:

validateLength :: String -> Maybe String
validateLength s =
  case (length s > 15) of
    True -> Nothing
    False -> Just s

We had decided on a maximum length of 15 characters, for purely evil reasons no doubt, so it takes the input string, checks to see if its length is longer than 15; if it is, we get a Nothing and if it’s not, we get our password.

Validation time!

Now what we need to do is compose these somehow so that all of them are applied to the same input string and a failure at any juncture gives us an overall failure.

We could write one long function that nests all the various case expressions that we’re using:

makePassword :: String -> Maybe String
makePassword xs =
  case stripSpacePwd xs of
    Nothing -> Nothing
    Just xs' ->
      case checkAlpha xs' of
        Nothing -> Nothing
        Just xs'' ->
          case validateLength xs'' of
            Nothing -> Nothing
            Just xs''' -> Just xs'''

This is valid Haskell, but these can get quite long and hard to read and think about, especially if we need to add more steps later (or remove some). And you sometimes have to rename arguments to avoid shadowing (that’s what xs' and xs'' are: new names).

We might initially be tempted to try just composing them in some way, perhaps:

makePasswd :: String -> Maybe String
makePasswd xs = (validateLength . checkAlpha . stripSpacePwd) xs

-- or

makePasswd :: String -> Maybe String
makePasswd xs = validateLength (checkAlpha (stripSpacePwd xs))

Unfortunately, the compiler will reject both of those and chastise you with intimidating type errors!

The reason is that each of those functions returns a Maybe String – not a String – but they each only accept String as their first argument.

At the risk of appearing quite not smart, I’ll admit that when I was first learning Haskell, I used to sometimes write this all out on paper to trace the flow of the types through the nested or composed function applications.

What we need is something that will allow to chain together functions that take a String and return a Maybe String.

In a bind

Conveniently, Haskell has an operator that does this: >>=. It’s so important and beloved by Haskellers, that it’s part of the Haskell logo. It’s called bind, and we can chain our validation functions together with it like this:

makePassword :: String -> Maybe String
makePassword xs = stripSpacePwd xs
                  >>= checkAlpha
                  >>= validateLength

The result of stripSpacePwd will affect the whole rest of the computation. If it’s a Nothing, nothing else will get evaluated. If it’s a Just String, then we will pass that Just String to the next function, even though it needs a String as the first argument, like magic.

(It’s not magic, though.)

Look at the types

If you’ve never looked at Haskell before, this part might be somewhat opaque for you, but explaining all this in detail requires explaining almost all of Haskell. We’re only here for the gist, not the crunchy details.

Let’s look at how >>= works. The (not quite complete) type signature for this operator looks like this:

(>>=) :: m a -> (a -> m b) -> m b
--         ^     ^
--          these
--       are the same

When the m type constructor that we’re talking about is Maybe, the type looks like this:

(>>=) @Maybe :: Maybe a -> (a -> Maybe b) -> Maybe b

-- you can do this in your REPL by turning on the
-- language extension TypeApplications

The complete type of >>= looks like this:

(>>=) :: Monad m => m a -> (a -> m b) -> m b
--       |________|
--       this part
-- tells us that whatever type `m` is,
-- it has to be a monad

Ahhh, the M word.

Our new friend >>= is the primary operation of the Monad typeclass, so very literally this constraint (Monad m =>) says that whatever type m is, it must be a type that is a monad: a type that has an implementation of this function >>= written for it.

But what is a monad?

A monad is a type constructor (a type like Maybe that can take a type argument, not a concrete type like Bool) together with a (valid, lawful) implementation of the >>= operation. So you’ll hear sentences like “Maybe is a monad” meaning it’s a type that has such an implementation of >>=. And this is why you hear people talk about wrapping things “in a monad” or about containers and burritos and whatnot – we even have a function that does nothing but wrap a value up so it can be used in such a computation (it’s called pure now and lives in the Applicative typeclass instead of in Monad but why is a long story).

Why do Haskellers care about this?

I’m not going to go too much into typeclasses here, how they work and how we leverage them to good effect in Haskell. So what else can we say about this? That by recognizing a common pattern and giving it a name, we can gain some intuition about other times we might see it and what to expect when we use them.

In particular, Haskellers like the opportunity to reason algebraically about things. Let’s talk about what that means for a moment.

Algebras

An algebra is a set together with some operation(s) that can be defined over that set. In this case, >>= is the symbol for an operation that can be defined over many (not all) sets – think of types as sets.

So, a monad is an algebra, or algebraic structure, that has at least two components:

• a set, or type, such as Maybe;
• a bind operation defined over it.

It also has some laws, or else it wouldn’t be a proper algebra, and we talk a lot about the monad laws in Haskell and we can (and should) property check our >>= implementations to make sure they behave lawfully. BUT the Haskell compiler doesn’t enforce laws, so a monad in Haskell is perhaps slightly less imposing than a monad in mathematics.

Thinking algebraically

Reasoning about code in terms of (types) and operations we can define over those sets without having to think too much about the details of each and every set that we could ever make. Can this type that we have here be used in sequential computations where the performance of the next computation depends in some way on the result of the one before it? Cool, we might have a monad then and recognizing that might give us some extra power to reason about and understand and predict what our code will do.

Typeclasses remove yet another layer of detail to think about and let us generalize even more. How well that works is a matter of some debate, but in part we do it for the same reasons that mathematicians talk about groups and sets and very general things like that: abstracting away some details allows us to focus on and consider only the parts we care about at a certain time, and sometimes allows us to see connections we never noticed before.

But isn’t it a monoid in the category of endofunctors, though?

It is, and understanding why can be helpful. But it’s not the right place to start understanding monads, unless you already understand some category theory, and while I admire people who do, they are distinctly not my intended audience for this post. I suspect they would ostracize me for being so hand-wavy about all this.

Someday, I’ll try to write a beginner friendly post about what it means that it’s a monoid in the category of endofunctors. If you know what a monoid is and know that “endofunctors” for Haskell purposes just means “functors” and understand that by “functors” we mean type constructors (not fmap itself), then perhaps the monoidness of >>= (and also the Applicative operation <*>) will begin to become apparent. Perhaps Ken’s Twitter thread will help, too. If you don’t already understand those things, then it might not, and that’s OK; it takes time to build up and internalize all the concepts.

If you’re trying to learn Haskell and don’t already know category theory, it is perfectly fine to use >>= when you need it (or do syntax, which is a syntactic sugar over this and looks more imperative²) and not worry any more about it. Since all user input and every main action in Haskell is handled with monads, you sort of have to to be able to use them without understanding them deeply for a while. If you have a series of computations that should be performed sequentially such that each new one depends on the successful result of the one before it, you may want >>= to chain them together (which requires them to be wrapped in a [monad] type constructor such as Maybe.)

Since you generally want side effects to be sequenced, and we use the IO type to constrain side-effecting code, IO is a sort of canonical monad. IO, which is the obligatory type constructor of all main actions and all side-effecting code, is a monad, so a lot of code you write that will do anything involving IO is already wrapped in such a constructor and, hence, monadic, but understanding the actual implementation of this is beyond unnecessary to writing real working programs in Haskell.

And what about do syntax?

I don’t use do syntax when I’m trying to teach monads, even though it is meant to allow the writing of monadic code in a nice imperative style. For teaching purposes, I don’t like the fact that it hides the composition-like piping of arguments between functions. I once said it “hides the functors”; it makes it harder for me to follow the flow of types through the function applications, and so I don’t particularly like it when I’m teaching people about functors and monads. It’s cool to start using do to effect monadic operations without understanding how >>= works, though; we’ve all been there.

A Note on Terminology

Monad can refer to a few things. One is a typeclass that (mostly) corresponds to an algebraic structure (a set plus some law-abiding operations defined for that set) of the same name; to form a complete algebra, in Haskell at least, though, you need three things:

• the class declaration, which defines the operation with maximal generality;
• a type that can implement that operation; and
• a typeclass instance that binds the type with the typeclass declaration and defines the operation(s) specifically for that type.

Usually, the phrase “X is a monad” tells you that X is a type constructor with an instance of Monad.

This is why I don’t prefer saying that Maybe is an instance of Monad but, rather, that it has an instance because an instance declaration is a specific piece of code that has to exist or else the type has no legitimate implementation of the function. If no instance exists, no function exists for that set so we have an incomplete algebra.

Incidentally, Haskellers do this with the names of (some, but not all) other typeclasses, too, so the type Maybe is a monoid, a functor, a monad, and so on, because it is a type (set) that has [monoidal, functorial, monadic] operations defined over it.

A Note on Learning Haskell

One of the reasons I hesitated so long to publish this post is that people who don’t have much interest in learning Haskell or who are just at the beginning of learning Haskell seem, contrary to the best advice on the internet, to always want to know straightaway what a monad is. It’s like monads have taken on some outsized mythical status. But the monad is really a sort of small thing. It’s a common enough programming task, chaining together sequences of functions that we want to behave in a predictable manner. Monad is a means of simplifying that (in some way; it doesn’t seem like a simplification when it’s new to you, but by giving us certain intuitions about how this pattern should behave – the infamous monad laws! – and being highly composable, they do remove some complexity, as the right abstraction should).

Everything we do in Haskell, even IO, can be done without monads, but not as easily or well. Monads let us do those things more easily, more consistently. I know when I was learning Haskell I had built it up in my mind that it would be this huge, difficult to understand thing, and it’s sort of anticlimactic when you find out what it really is: instead of nesting case expressions or something like that, we’ll just chain stuff together with an operator. Cool.

Further reading:

• I wrote a course that starts in just about the same place as this post but continues refactoring the password validation code with different types and ends up demonstrating one of the differences between Applicative and Monad. It’s not free, but it is available now.

• A Gentle Intro to Monads … Maybe? – code in this one is all JavaScript.

• A Guide to FP Lingo for JavaScripters – what the heck, it’s like the JavaScripters outnumber us Haskellers. :)

• It’s a bit like asking “What is a number?”

• Adjacency – There is an interesting point being made here about adjacency in Haskell vs adjacency in do-blocks, but he rather undersells the fact that monadic binding (and, thus, do syntax) isn’t just for IO.

1. For more on constructors, see here.↩︎

2. Examples of all this code but using do instead of >>= are in the code file.↩︎

A couple of years ago, I got one of the juiciest TV assignments ever: To interview NASA astronaut Scott Kelly, who was just about to lift off into space for a year aboard the International Space Station. That’s by far the longest any American has ever spent continuously in space. The idea was to study the effects of long-term zero-gravity living in space, in the name of preparing for a manned mission to Mars. (You can watch that CBS story here.)

Earlier, Kelly had already spent a six-month stint in space, and NASA already knew that bad things happen to you after awhile: you lose bone mass, you lose muscle, your immune system weakens, your eyesight suffers, and you get as much radiation each day as you’d get from 20 X-rays. Now, Kelly had more of that to look forward to. And remember: NASA stopped launching manned rockets in 2012 when it retired the Space Shuttle — so Kelly would ride to the Space Station aboard a Russian Soyuz rocket, which still uses decades-old technology.

“The good news is that it works — most of the time,” Kelly told me that day in 2015 as we climbed into a mockup of the tiny Soyuz capsule that would carry him into space. “They’ve had a couple accidents. But, you know, so have we, on the Space Shuttle. It’s risky. But you know, flying through space is a risky thing.”

Kelly made it to the Space Station safely, spent the year in space, conducted experiments, and entertained thousands of people with a steady stream of tweets (and stunning photos) from space. Last March, he made it back to Earth. (The Russians don’t splash down. Their capsule thuds down — on dry land.)

At the Consumer Electronics Show in Las Vegas a couple of weeks ago, I caught up with him again, the first time I’d had to chat with him since his return to Earth.

Tech in space

As a consumer tech guy, the first thing I wanted to know was how much gadgetry he had up in space. Did he have a phone?

“No, not like you’re thinking,” Kelly said. “We do have a capability to make calls, but it’s software on a laptop.” He had an iPad, but it couldn’t get online; if he wanted to send a text, he did it via email from his laptop.

“There’s some slow internet capability, kinda like dialup used to be,” he said. “I’ve bought stuff on Amazon. I bought airline tickets. We did some online banking with Chase.”

And, of course, he had cameras. “The cameras we use are almost exclusively Nikons —hence my being here for the CES at the Nikon booth.” (He used a Nikon D4.)

So, how did the year in space feel compared to his six-month gig? “It was a lot longer, clearly, but it felt a lot longer, too,” he says. “I mean, a year’s a long time to be in any one place.”

One of NASA’s goals was to compare his biological readings with his identical twin brother Mark (also a former NASA astronaut, and husband of former U.S. Representative Gabby Giffords). And sure enough, they found differences: “I’m smarter and more handsome than he is, they found,” Kelly cracks.

But seriously, folks…

“There were some changes in our gene expression, which was interesting to the scientists. Meaning, the genes inside of me were turning on and off much more rapidly than my brother’s on Earth. And gene turning on could be good, it could be bad — we don’t know. And they saw that my telomeres got better. Telomeres are things on the ends of your chromosomes, that are an indication of your physical age. It’s almost like I got a little bit younger. Clean living, I guess.”

Physical effects

Kelly also told me about the physical effects of returning to Earth after a year without gravity. It wasn’t pretty.

“The experience coming back was different than when I flew for six months,” Kelly says. “It was much more significant: soreness, muscle aches and joint pain, swelling in my legs when I would stand up — I could feel the blood rushing out of the top of my body, down into my legs, could see my leg swollen up.” For weeks after returning, his skin also exhibited rashes and hives wherever it came in contact with cloth — like clothing or bed sheets.

“I was fatigued, nauseous, dizzy — I mean, things that were much different after spending a year in space versus six months. So, at least for me, there’s symptomatically kind of a bend in the curve there somewhere between six and nine months where the negative effects seem to accelerate.” It took eight months, he says, before he felt normal again.

The anti-science movement

I couldn’t help asking Kelly about the weird anti-science wave that seems to be washing through America these days.

“I think about it a lot, actually,” he says. “It’s a dangerous place to be where you doubt scientists, and when you have our government questioning that 97% of scientists say that climate change is real. It’s something we need to be serious about. It’s going to have effects on our children and our grandchildren if we don’t do something about it. And then to have our country be the lone country that pulls out of the Paris climate accords!? We’re supposed to be the leader, you know? Not alone. It’s just crazy.”

Kelly says that sometimes, despite himself, he winds up in conversation with “the flat-Earth people,” who maintain that the Earth is not, in fact, round. “I think some of those folks don’t really believe the Earth is flat; they just do it as a goof,” he says. “And it’s fun for them. But the problem with that is, you’re discounting a pretty significant scientific fact that the Earth is round. And if you can call question into that fact, it’s much easier for people to believe that climate change is a hoax, too. It validates other conspiracy theories, which we don’t need to be doing. It’s just dangerous. We need to be the leader on science, not the guy sitting in the corner as the denier.”

“Endurance”

Kelly didn’t ask me to plug his new book, “Endurance: A Year in Space, a Lifetime of Discovery.” But after our interview, I downloaded it and dove in. Its chapters alternate between telling the story of his growing up as a poor, unmotivated and underachieving student, and the story of his year in space. It spent 10 weeks on the New York Times bestseller list.

It’s a fantasticread, especially the sections comparing NASA’s approach to space travel (by-the-book, careful, risk-averse) with Russia’s (fairly freewheeling by comparison). Whereas NASA keeps people three miles away from the launch pad, people teem all around the Soviet launch pad — some of them smoking — until just before takeoff. The Russians don’t bother with a countdown, either; when they’re ready, they just take off.

Finally, “on the space shuttle,” Kelly writes in the book, “I never knew whether I was really going to space that day until I felt the solid rocket boosters right under me; there were always more scrubs than launches. On Soyuz, there is no question. The Russians haven’t scrubbed a launch after the crew was strapped in since 1969.”

Next stops

So what’s next for Kelly? He’s no longer with NASA. For now, he’s writing books and making speaking appearances.

“I probably wouldn’t fly in space again,” he says. Or at least “probably not with NASA, but you never know. Somewhere else maybe.”

Dear SpaceX: Are you listening? Commander Kelly is ready to go.

David Pogue, tech columnist for Yahoo Finance, welcomes non-toxic comments in the Comments below. On the Web, he’s davidpogue.com. On Twitter, he’s @pogue. On email, he’s poguester@yahoo.com. You can sign up to get his stuff by email, here. 

Read more:

Exclusive: What Fitbit’s 6 billion nights of sleep data reveals about us

Tech that can help you keep your New Year’s resolutions

Pogue’s holiday picks: 8 cool, surprising tech gifts

Google’s Pixel Buds: Wireless earbuds for the extremely tolerant

Study finds you tend to break your old iPhone when a new one comes out

Rejoice: Sonos Speakers are finally voice-controllable

Follow Yahoo Finance on Facebook, Twitter, Instagram, and LinkedIn

The Age of Abundance

Mozilla was born out of, and remains a part of, the open source and free software movement. Through the Mozilla Open Source Support (MOSS) program, we recognize, celebrate, and support open source projects that contribute to our work and to the health of the Internet. That’s why in 2017 we invested $1,650,000 in supporting open source projects around the globe. Half a million of which we dispersed just since our last update in October.

One of the early MOSS recipients was Buildbot, a Python-based continuous integration server. We interviewed Bill Deegan, a lead developer, about his experience with the MOSS program.

Another project in the Python world is the Python Package Index. PyPI is at the core of the Python ecosystem, supporting the download of over 100 million packages every week. However, the software platform it runs on has been getting harder to modify and improve for a while. s a result, the Python community has begun developing a replacement – “Warehouse” – using modern frameworks and techniques. Mozilla was pleased to award that effort $170,000 to push it over the finish line to a place where it can replace the existing codebase.

Additional Awards

We have also supported a number of projects in the past quarter that we believe will advance a free and healthy Internet. Mozilla is directly enhancing the products and services we offer our users based on many of the technologies these grantees are developing.

• $60,000 to Harfbuzz, a text rendering and shaping engine, to improve their documentation;
• $65,000 to Zappa, a “server-less” application deployment system, to improve its scalability and build an ecosystem around microservices;
• $25,000 to Tatoeba, a platform for submitting and storing voice data, to integrate it with Common Voice and deal with some licensing-related issues;
• $160,000 to the Tor Project’s Open Observatory of Network Interference, which tracks network-level taps, modifications and outages around the world, to make their gathered data more accessible, and improve their client software.

Global Mission Partners

Additionally, the assessment of applications for the first round of “Global Mission Partners: India” led to a single award of $19,000, to the Commento project. Commento is a lightweight embeddable discussion system for websites.

We will be conducting a review of our processes relating to this round of work in India, before before initiating further rounds.

Secure Open Source

In this quarter, under the Secure Open Source arm of MOSS, we expanded the scope of the program by funding development of secure code projects. Some principal developers on the libav media library project are building rust-av, a similar library written in Rust, which can take advantage of that language’s type, memory, and data safety properties. We are providing $71,356 (€51,000) to help that group towards making a Minimum Viable Product and demonstrating the plausibility of their approach. Media libraries are common pieces of software which are exposed to malicious input and which have a large attack surface (every codec enabled adds more); many bugs leading to system compromise have historically been found in them. Having an option with a Rust core can help mitigate that security risk.

Apply Online

Applications for “Foundational Technology” and “Mission Partners” remain open, with the next batch deadline being the end of January 2018 – only eight days away! Please consider whether a project you know of could benefit from a MOSS award. Encourage them to apply! You can also submit a suggestion for a project which might benefit from an SOS audit.

The post MOSS Q4: Supporting the Python Ecosystem appeared first on The Mozilla Blog.

A community’s ‘type’ is similar to a movie’s ‘genre’. It should provide you with a set of rules which should focus your community building efforts.

Community work varies greatly by the type of community you’re developing. Building a Q&A community for support is very different from building a community where your members proactively share their best ideas.

In this post, I want to highlight the three common types of community, how to build each type, and the constraints of each type.

If you completely understand what type of community you’re building, you can align everything you do to match.

Three Broad Types Of Brand Communities

Many people fall into the mistake of trying to build a general community about the topic. This usually happens when the company avoids making tough choices and tries to cover every possible use case for the community. Don’t do this.

General communities have weak concepts and tend to struggle to sustain much activity.

The most successful branded communities today usually fall within three core community types.

These are:

1. Q&A (or support) communities.
2. Idea-sharing (or education) communities.
3. Peer groups (or exclusive) communities.

Each has positive and negative attributes. We’ll go through each in turn:

Type 1: Q&A / Support Communities

Most of the successful brand communities are based around questions and answers (Q&A). The most common of these are customer support communities. Customers bring their problems and get solutions from top staff/other members.

The key aim of a support community is to remove the frustration that brought the member to the community in the first place. It’s not enough just to provide an answer, you need to provide an answer with the speed, clarity, and sentiment that helps members feel less frustrated.

If interacting with the community makes people feel unhappy, you haven’t really solved the problem.

Benefits of A Q&A/Support Community

There are three main benefits of a Q&A / Support community:

1) Direct contribution to value. Whereas other types of communities are often several layers removed from value, support communities are fantastic for demonstrating a reduction in support costs, reducing satisfaction of disgruntled customers, and identifying/resolving potential problems early. They are also often used to solicit feedback.

2) Easier to launch. If you have a lot of customers with a lot of questions, you can usually make a support community work quite easily. Most traffic to a community initially comes via the website and email links. As you build up a base of answers, SEO traffic will usually become the prime source of traffic.

3) Member familiarity. Related to both of the above, members are familiar with the idea of asking a question and getting responses from others online. It’s a behavior similar to what we already do and doesn’t require much explanation.

All of the above explains why most successful brand communities are based around customer support and why support communities tend to have the most success.

Downsides Of A Q&A/Support Community

However, there are some inbuilt major problems with managing a Q&A/Support community. These usually include:

• You need a large base of members to succeed. Companies with less than 100k customers usually shouldn’t try to launch a support community. They struggle to attract the critical mass necessary to attract the superuser group and can’t deflect enough tickets to justify the investment.
• Negative tone of voice. Because most people only visit when they’re frustrated, the tone of voice skews more negative than other types of communities. This frustration can often turn on other members or community staff who can find themselves victims of very personal online attacks.
• Most members only visit once. Most people only visit when they have a problem. It’s hard to build any real sense of community among people who don’t want to be in the community at all.
• Static participation levels. The level of activity and participation is often driven by factors beyond your control (e.g. new product launches, changes to search algorithms, placement on the website). This can make it difficult to move the needle in many areas.
• Costly to run. It’s possible to do support on cheaper forum-based platforms like Vanilla/Discourse, but the standard for a large company is typically a premium platform with the security, functionality, and analytics they provide. You want to be able to add common answers to a knowledge base, create levels for superuser programs etc…

None of these are fatal and most aren’t avoidable, but they’re likely to be an ongoing problem with the job.

How To Improve A Support Community

If you’re managing or optimizing a support community, you will probably spend your time working across four dimensions. These are speed, accuracy, sentiment, and integration. Specifically, this means:

1) Decreasing the time to get a solution. You want the majority of people to find the solution without having to ask a question. This means recruiting and nurturing a top contributor program to provide quicker responses. This may take up the bulk of your time. You also need to ensure questions are well categorized, tagged, and have an accepted solution where possible. You need trending or topical questions to appear high up the page.

2) Increase the accuracy and clarity of the response. You want the quality of responses to be extremely strong. This means ensuring responses are easy to read and understand. Using video walkthroughs, screenshots, and bullet points usually helps (and training your team to do the same). You also want to frequently update the top 20% of answers responsible for 80% of traffic (especially after major product updates).

3) Improve the sentiment of the responses. You need to carefully consider how you personally engage and respond to discussions within the community. You need to deeply understand the psychology of your audience (p.s. I’d strongly recommend anyone working on a support community take this program). Your answers need to be personalized, friendly, empathetic, as well as accurate.

4) Using the questions and solutions throughout the organization. You should also be spending a lot of time escalating issues internally, ensuring questions are incorporated into product decisions, and helping your company take notice of the key trends within the community.

If you’re working on a support community, most of your time should be spent in the above areas. You can’t expect your members to be happy, but you should expect to be driving really incredible results for your company.

Type 2: Idea/Education Communities

Many of the most popular communities today are based around the idea of members proactively sharing resources, tips, and links. These are not solicited.

This can range from full-fledged articles (Medium), sharing resources (ProjectManagement.com) to simple link sharing (Reddit).

These kinds of communities come with some incredible benefits and equally challenging downsides.

Benefits of An Idea/Education Community

There are three main benefits of an idea/education community.

1) Growing the business. These are the best kinds of communities to improve customer satisfaction/retention (by helping people use the products better), attract new business (via search traffic), and drive innovation. The very best of these communities become the hub of their field.

2) Positive tone of voice. These communities usually have a positive tone of voice. It’s communities filled with people sharing what they’re doing and learning from one another. People don’t visit when they have a problem, they visit to get better at what they do.

3) In-built participation habits. These communities have in-built variable-reward mechanisms. Every time you visit, there might be a great new idea you can use. This is a lurker’s paradise and are environments ripe for forming habits.

If I were to add a third, it would be these communities typically explore the cutting edge of any field or topic. This is an exciting/motivating type of community to build.

Downsides Of An Idea/Education Community

Very few brands try to build an idea/education community. There are many reasons for this, but the biggest include:

• Very difficult to get started. By far the biggest problem is getting started. You need one group to attract the other. The success rate of these communities is far lower as a result. You might have better luck turning an existing, general, community into this type of community. But you need a large group of smart people willing to share great links first.
• About the topic, not the brand. With a few exceptions, these communities are better at serving broad topic areas (e.g. inbound.org) than specific brands (e.g. HubSpot). If you try to build the community about you, you’re going to find it harder to attract a high-quality audience. People want to talk about the broader topic than just a brand. However, there are plenty of exceptions here.
• Can be overwhelmed with spam. Once you encourage everyone to share their content, they often do. This quickly descends into poor-quality, promotional, content which drives everyone else away. It’s hard to fight this and maintain high-quality content. This leads into the next problem.
• Customized platform requirements. While there are a handful of idea/education-based communities on forums, the vast majority are not. Forums are better designed to support communities than education communities. Education communities tend to use a custom-built platform designed to solicit these specific recommendations. These range from templates/resource sites, news/link aggregation, pinterest-style boards, etc…

While the benefits of building an idea/education community might be higher, the costs and risks are usually much higher too.

Optimizing An Idea/Education Community

Based upon the above, it should be relatively clear how best to optimize a support community. This will include:

1) Recruiting and helping members to share interesting things. This is obviously critical. You need to find ways to identify smart/motivated people and get them to share great stuff. In the beginning this will usually be you and your team finding the best ideas. As you grow, you should be able to gradually bring more people into the fold.

2) Developing and improving your filter for high-quality content. You need great filters to separate the good from the bad. This usually means a combination of editor’s picks, tagging, upvoting, trending items, and (less often) algorithms. You need to work on ensuring members see the best stuff as quickly as possible. Technical competence is important when building this type of community.

3) Promoting the community. These communities benefit most from traditional publicity tactics. This means getting publicity on relevant blogs, influencer outreach efforts, and doing interesting things that attract a lot of attention.

4) Turning interest into results. You also need to turn the community interest in value for the business. This might be through lead generation, ‘sponsored’ posts, etc…

As you grow, you may also need to focus on how you build sub-groups within this community for related topics or subtopics.

Type 3: Peer Groups/Exclusive Communities

The easiest type of community to create is an exclusive community. The people who join are those who meet a high criteria based upon demographics, habits, or psychographics. In these peer groups, members usually share a strong, shared, identity with other members. The connections tend to run deeper than other types of communities.

This occurs every day in your private WhatsApp and messenger groups. You also see it on platforms like Nextdoor and Meetup, as well as companies like socialmedia.org.

Benefits of Peer Groups / Exclusive Groups

The key benefits of peer groups/exclusive communities are:

1) ‘Lock-in’ key audiences. Building an exclusive peer group among some of the top people in your field can be a great way to ‘lock in’ a key audience. This works well for companies in the B2B space, those looking to charge for membership, and those building platforms for peer groups to thrive.

2) Easier to launch. If you don’t have a large, existing, audience, the easiest way to start a community is to keep it exclusive and targeted only at some of the top people in the field. This is motivating for those people in the field to join and participate. Many communities begin exclusively before expanding to a broader audience once they have established their reputation.

3) Members connect with each other on a deep, personal, level. These groups whether via working-out-loud or simply providing each other with emotional support can be life-changing for participants. An exclusive community effort tries to bring together a group of people with a very strong shared identity and create a sense of belonging among them. These communities tend to have a lot of off-topic discussions and real-world meetings.

The Downsides Of An Exclusive Community

There are also some common disadvantages to creating and managing an exclusive community.

• Internal disputes. Exclusive communities tend to be hypersensitive to petty disputes between members. Given the small size and close relationships of the groups, these disputes can rip audiences apart.
• Building credibility with top people in your field. You need to have relationships with high-calibre people to get the community started. If you don’t, you need to invest the time to build and maintain these relationships before you can build the community.
• Limited growth. The very nature of having a high-barrier to entry ensures the community size is always limited to a degree. Any expansion is a threat to the close ties of the group itself. This means you have to gain the maximum benefit from the members you have.
• A high barrier to entry. You don’t need to make it impossible, but there should be a very clear calibre of people who are allowed to join the community. These reasons should be very public.

Optimizing A Peer Groups/Exclusive Community

These kinds of communities tend to have the most flexibility in your daily work. A small peer group diverges significantly from a larger, exclusive, community. The focus of your work however will usually be along the lines of:

• Programming content. You need to host events or create content that supports the community. Simply being exclusive isn’t enough unless you offer a clear benefit beyond this exclusivity. This should offer members something which they cannot get access to elsewhere.
• Attracting and keeping the right members. You will need to invest more time to interact personally with each member and ensure they are happy and engaged within your community. This is often very difficult to do. It might mean breaking the bigger group into subgroups so members can engage and interact with each other.
• Building a sense of community. You work hard here to build a strong sense of community among the group. This includes plenty of rituals, emotive discussions, and roles for each member of the community.

Summary

If you’re not sure what type of community you’re building, you’re probably not building a very strong community.

If you are sure, make sure you focus your time and effort in the areas which are going to have the biggest, possible, impact for your type of community.

Almost every organization we’ve worked with can really improve their community by better understanding what type of community they’re building in the first place.

Good luck!

I’m not very fond of having to install closed source software on a Linux systems I administrate and try to avoid if at all possible. Unfortunately I was very close at having to do just this recently when I needed to connect to a network behind a Cisco IPSec VPN gateway. Cisco has a software package for Linux for this but apart from it being not open source the installation process is far from confidence inspiring. But then I noticed that there is actually an open source Ubuntu NetworkManager plugin that can be installed straight from the Debian repository: OpenConnect!

Installation is (almost) as straight forward as it could be. Ubuntu 16.04 requires the following package to be installed:

sudo apt-get install network-manager-openconnect-gnome

After the installation a new VPN-plugin becomes available in the network manager as shown in the screenshot at the top. For my setup I then needed to do the following things:

• Supply the VPN-server’s domain name in the main configuration dialog box.
• Enter the username and password during the first tunnel establishment.

That’s all there was to it and I got a connection to the VPN server the first time I tried. For some strange reason, however, the DNS server was not configured correctly during the connection establishment and I’m not sure if that’s a problem with the VPN connector or the VPN server setup on the other side. In any case this can easily be fixed by clicking on the IPv4 tab of the VPN configuration in the NetworkManager and choosing ‘Automatic (VPN) addresses only’ as configuration method and then supplying the DNS server’s IP address manually in the provided input field for DNS server addresses. Since I wasn’t aware of the DNS server’s address on the other end I used Google’s DNS server (8.8.8.8) which worked just fine because the network on the other side provided Internet access.

Population data typically comes in the context of boundaries. City data. County data. Country data. With their Population Estimate Service, NASA provides data at higher granularity. You can request estimated population in the context of a world grid.

Here’s an interactive map to demonstrate the API. Click and drag a shape across any region in the world and get an estimate of the population within that shape. [via kottke]

Tags: NASA, population

Anytime someone asks me how to turn their weird tech project into reality, my immediate instinct is to recommend the Raspberry Pi. This $35 computer, the size of a deck of cards, is as capable as it is cheap. With just a bit of know-how and curiosity, you can use it to make a retro-gaming console, a robot brain, a smart-home sensor, or even a fully functional Alexa-compatible speaker.

Over the past three years, we’ve tested 144 sets of headphones, and we’re positive that the Jabra Elite Active 65t is the best set of wireless workout headphones. They sound great for music and calls, they fit comfortably, and because they have no wires, they stay secure and completely out of your way during rigorous workouts. They should withstand abuse, sweat, and moisture when used properly, plus they’re backed by a two-year warranty.

I normally do all my computer work on a MacBook Pro; the convenience of being able take it anywhere with me is worth the cramped feeling of having “just” 1TB of disk and a small screen. Unfortunately, my MacBook Pro, which is about 3½ years old, went south yesterday, and now and it freezes up after a minute or three. I spent all day trying stuff to fix it, but it seems to be a hardware issue, So I called up Apple Support to arrange repair.

While creating my repair case, the guy asked whether it was covered by AppleCare, their extended warranty. I'd had the three years of AppleCare, but it expired a few months ago; I replied “Apple products never break down while covered by AppleCare”.

His reaction was priceless: “I can't tell whether that's a compliment or an insult.”

Hah. I guess it's both.

I'm writing this post on an Apple desktop machine from 2009, which is still humming along just fine. But Apple's cutting edge products like their phones and laptops squeeze just a bit more out of the space available than anyone thought possible until they actually demonstrate it's possible by releasing the product, and that kind of cutting edge stuff comes with this kind of cost of ownership... it breaks down a lost more than a sturdy old-tech product. In my experience, that's often right after the warranty expires.

Anyway, the delivery service just picked it up. Last time I did this (~8 years ago?) I had it back quickly... maybe three days. I hope it's the same this time....

Update:

Well, that was quick. It was picked up by the courier service on Wednesday night, and at 8am this morning (Friday) I got an email from Apple that said they have received my computer. I expected to eventually get a message offering an estimate for the repair and asking whether I wanted to proceed, but less than seven hours later, at 2:45pm, I got a message informing me that “my repair request is complete” and that it was already shipped back.

There's no information as to whether they actually did anything, and if so, how much they'll charge me, but I suspect that either they did nothing and are returning my broken computer, or it'll cost the maximum fixed fee of about 80,000 yen (~$730), which had already been reserved on my credit card.

In any case, I'll find out tomorrow... it's schedule to arrive before noon.

NBC News: Tourism to U.S. under Trump is down, costing $4.6B and 40,000 jobs.

“International tourism to the U.S. began to wane after Trump took office, leading to a so-called Trump slump. Experts say that Trump’s proposed travel bans and anti-immigration language have had a negative impact on the U.S.’s attraction for foreign visitors, in addition to a weaker dollar and heightened security measures.

“It’s not a reach to say the rhetoric and policies of this administration are affecting sentiment around the world, creating antipathy toward the U.S. and affecting travel behavior,” Adam Sacks, the president of Tourism Economics, told The New York Times.”

Link via MetaFilter.

From Time Out in London comes this unfailingly forward story about Brenda Puech. Brenda believes in sustainability, and she had a great idea. She went to buy an annual parking permit from her local council, but the council refused to give one to her because she did not have a car, and was not going to get one. So left with a parking space in front of her house, Brenda “decided to take the initiative and  “convert a parking space directly outside my home into a garden”.

Brenda noted that in her area of London there are twice as many households that don’t own cars as there are ones that do. The space she chose for her garden was “usually vacant, so I knew that using it would cause minimal inconvenience to my car-owning neighbours. On May 26, at the start of the summer, I officially launched the People Parking Bay: a patch of artificial grass the size of a car, with flowerpots, a bench and table, a bright red umbrella and a large sign that read “You’re welcome to park yourself on the bench.”

Brenda’s parking spot as public park with bench went viral. “People used the Parking Bay as a resting point on the way back from shopping or cycling; mums used it to feed their babies; locals watered the plants. One couple had their first date there. Some people left books and it became a mini-library. People even used the bay as a community noticeboard. I’m proud of how it became a focal point, where you’d see complete strangers smiling and talking to each other”. 

Brenda even left a visitor book for people to sign on the table. Five books were filled with comments within a month. Most comments were about the brilliance of the idea. But the local Council caught wind of the parking spot as public park, and an eviction notice arrived. Not to be thwarted, Brenda simply moved her parking spot as public park to differing locations in the neighbourhood with stealth like precision. But when the officials finally caught up to the moving park, it was game over.

As Brenda observes:  There’s no mini-garden now: I’ve had to bring it into my front yard, and sadly it’s not open to the public. When a parklet like mine has been so transformational for a local community, it’s such a shame the council insists on disposing of it. But if I hadn’t set up the People Parking Bay, people wouldn’t have realised it was even a possibility. Sometimes you have to take matters in your hands”.

You can check out Brenda’s website at the link available here.

Vancouver startup Mobile Reality has launched the world’s first mobile virtual reality arcade, a customized trailer that can travel to public spaces like fairs, festivals and city streets.

With eight networked virtual reality gaming stations, Mobile Reality says it can offer daily racing and flight simulations to hundreds of customers every day.

“We are excited to be rolling out the Mobile Reality Titan Trailer,” said co-owner Dave Rice in a press statement. “Our experience gives people of any skill level the thrill of premium seated VR, networked together so that groups play together, at the same time in the same game.”

To celebrate the launch, Mobile Reality is offering free sessions — which typically last around 25 minutes — from January 25th to January 27th at LaSalle College Vancouver on 2665 Renfrew Street.

On Mobile Reality’s website, customers can claim free tickets using the discount code “MOBILEVR.” Sessions will otherwise cost $20 CAD.

“The soft launch helps us finesse our practices and get the word out about our brand,” said co-owner Scott Belyea in a press statement. “We aim to be the most efficient VR service available, a sit-down-and-go type of experience for all ages, while delivering the highest possible quality product. There’s enough second-rate VR out there posing as authentic and we want people to know us as the difference.”

Outside of arcades, opportunities to demo virtual reality can be few and far in between, so Mobile Reality’s offering is certainly a novel concept.

More information on Mobile Reality can be found here.

Source: Canada Newswire

The post Vancouver startup creates the world’s first virtual reality arcade on wheels appeared first on MobileSyrup.