Jean-Yves

Le monde du test automatisé est en plein essor. Avec un marché mondial totalisant 20.70 milliards d’euros en 2021¹ et une croissance annualisée estimée à 19% d’ici 2030, le déploiement de tests automatisés s’accélère pour aider les entreprises à se transformer.

Les apports majeurs de l’automatisation de tests remontés dans le World Quality Report 2022-23² sont de contribuer à la mise en place d’une démarche de CI/CD (55%²) et l’amélioration de la valeur des tests tout en réduisant leur nombre (53%²).

La meilleure valeur des tests automatisés est d’ailleurs captée par les tests fonctionnels (46%²) et pour les tests d’intégrations (45%²). Ces typologies de tests sont précisément ceux supportés par Cerberus Testing, la plateforme de tests 100% open-source.

La nouvelle release 4.15 de Cerberus Testing propose une série de nouveautés et améliorations pour permettre au plus grand nombre d’accélérer avec la qualité en améliorant l’accessibilité et la facilité de mise en place de tests automatisés.

Des parcours utilisateurs plus intuitifs

La philosophie de Cerberus Testing est d’animer la collaboration transverse sur les tests automatisés autour d’une interface partagée. La prise en main a été améliorée dans cette version en particulier pour les nouveaux utilisateurs de la plateforme.

La vision de l’expérience est celle d’un “1-click amazon” permettant de « créer, exécuter, visualiser un test automatisé en moins de 30 secondes. » à tout utilisateur, qu’il soit familier ou non avec la solution, en supprimant toute friction sur les parcours d’automatisation.

Figure 1: Le parcours de test de Cerberus Testing devient moins technique et plus intuitif.

Cette nouvelle version réalise cette promesse pour les tests d’applications web grâce à la refonte du parcours et une librairie reformulée fonctionnellement. Les prochaines améliorations supporteront des cas d’usages plus avancés et les tests mobiles et APIs.

Le succès d’une démarche d’automatisation de tests dépendant de sa facilité d’utilisation (39%²) et des compétences de l’équipe (34%²), ces améliorations vont permettre au plus grand nombre d’automatiser des tests sans forcément être technique ou devoir programmer. 

Le connecteur JIRA/Xray maintenant disponible

Xray est un plugin nativement intégré à JIRA pour supporter la collaboration des équipes sur les exigences et les critères d’acceptances. Ce plugin largement déployé supporte la définition de cas de tests pour des exécutions manuels ou automatiques en format Gherkin.

Le connecteur permet de déclencher l’exécution des tests Cerberus Testing depuis Xray et d’en récolter les résultats de manière centralisée. Cette approche permet de maintenir chaque outil focalisé sur sa valeur ajoutée et de faciliter le déploiement de tests automatisés.

Figure 2: L’intégration de Cerberus Testing avec JIRA/Xray fournit un reporting intégré.

L’intégration entre les deux solutions ne nécessite qu’une configuration minimale pour que les plateformes puissent collaborer. Les deux versions « Cloud » et « DC » (Datacenter) sont supportés pour permettre au plus grand nombre d’en bénéficier.

Ce connecteur s’ajoute à la famille des intégrations disponibles dans Cerberus Testing dans la catégorie du Test Management comme XStudio, laissant aux équipes la flexibilité d’ajouter des solutions en fonction de leur maturité.

Le support d’Avro pour Apache Kafka

Le besoin de transformation toujours plus rapides et de traitement massif de données accélère le déploiement d’architectures dîtes « event-driven ». L’asynchronisme propre à ce paradigme change la manière dont les tests doivent être conçus et mis en place.

Dans cet écosystème, Apache Kafka® est l’une des plateformes majeures supportant une connectivité massive de producteurs et de consommateurs, échangeant des messages sous un format libre ou défini via des schémas « Avro ».

Figure 3: L’intégration de Cerberus Testing avec Apache Kafka et Avro.

Depuis plusieurs années Cerberus Testing supporte le test de composants « event-driven » et « microservices » avec Apache Kafka, étant également open-source. Cette nouvelle version apporte en principale nouveauté le support pour les schémas Avro.

D’autres améliorations sont également disponibles comme la gestion centralisée de la configuration Kafka pour en faciliter la maintenabilité, ou encore la possibilité de recherche et de test d’éléments présents sur les « headers » des messages.

Améliorations multiples de cas d’usages avancés

On entend souvent que « le diable est dans le détail ». C’est un adage valable pour l’automatisation de tests oú les cas d’usages rencontrés en entreprise sont souvent loin des cas nominaux et relativement standards rencontrés dans les tutoriels.

Tests de reconnaissance d’image, de manipulation de fichiers, ou de validation à critères multiples sont autant d’exemples complexes auxquels nombre de testeurs doivent faire face dans leurs entreprises, et savoir en assurer une mise en place pérenne.

Figure 4: Les améliorations apportées au niveau de Sikuli.

La reconnaissance d’image disponible dans Cerberus Testing permet maintenant de cibler un offset spécifique sur l’écran, activer un délai pour les actions de type « keypress », ou faciliter la reconnaissance de caractères avec espacement.

On retrouve également des améliorations pour l’upload et le download de fichiers, de nouveaux contrôles pour les listes de valeurs numériques, et la relance d’exécution groupée depuis l’interface – maintenant le mantra « Stop Coding, Start Testing ».

Une nouvelle version, mais aussi un nouveau site

Cette nouvelle version suit une refonte complète du site web de Cerberus Testing pour une meilleure lisibilité et accès aux différentes ressources de la plateforme. Les offres de formations et les partenaires sont également visibles et identifiés sur le site.

Figure 5: Le nouveau site web de Cerberus Testing.

Cette nouvelle version s’inscrit dans la continuité de la philosophie de fournir une plateforme d’automatisation à l’échelle supportant les démarches agiles et DevOps dans laquelle la collaboration transverse des acteurs autour de la qualité permet d’accélérer.

La communauté utilisateur Cerberus Testing se rejoint régulièrement pour partager les nouveautés et met à disposition les précédents événements en replay sur youtube oú tous les utilisateurs, en phase de découverte ou d’expertise sont les bienvenus.

Les prochaines versions viendront complémenter les parcours utilisateurs pour faciliter et accélérer le déploiement de tests automatisés pour des équipes en quête continue d’amélioration de leur qualité et rapidité de livraison.

L’équipe reste à disposition pour aider la communauté à accélérer.

À propos de l’auteur: Antoine Craske

Fondateur de la QE Unit, la communauté de Quality Engineering. Co-fondateur de Cerberus Testing. Passionné par l’architecture, la transformation et les technologies. Convaincu que la qualité logicielle permet d’accélérer et qu’elle doit s’adresser en transverse, et que l’open source nous permettent de trouver des solutions plus efficaces à des problèmes communs.

Références

¹ Polaris Market Research (2021), Automation Testing Market Share, Size, Trends, Industry Analysis Report, By Testing Type

² The World Quality Report 2022-23 (2022), Sogeti.

Lately, we have seen a lot of people struggling with CSS, from beginners to seasoned developers. Some of them don’t like the way it works, and wonder if replacing CSS with a different language would be better—CSS processors emerged from this thinking. Some use CSS frameworks in the hopes that they will have to write less code (we have seen in a previous article why this is usually not the case). Some are starting to ditch CSS altogether and use JavaScript to apply styles.

But you don’t always need to include a CSS processor in your work pipeline. You don’t need to include a bloated framework as the default starting point for any project. And using JavaScript to do the things CSS is meant for is just a terrible idea.

In this article we will see some tips and recommendation to write better, easier-to-maintain CSS code, so your stylesheets are shorter and have fewer rules. CSS can feel like a handy tool instead of a burden.

The “minimum viable selector”

CSS is a declarative language, in which you specify rules that will style elements in the DOM. In this language, some rules take precedence over others in the order they are applied, like inline styles overriding some previous rules.

For instance, if we have this HTML and CSS code:

<button class="button-warning">

.button-warning {
  background: red;
}

button, input[type=submit] {
  background: gray;
}

Despite .button-warning being defined before the button, input[type=submit] rule, it will override the latter background property. Why? What is the criteria to decide which rule will override the styles of another one?

Specificity.

Some selectors are considered to be more specific than others: for instance an #id selector will override a .class selector.

What happens if we use a selector that is more specific than it really needs to be? If we later want to override those styles, we need an even more specific selector. And if we later need to override this more specific selector, we will need… yes, it’s a snowball growing larger and larger and will eventually become very difficult to maintain.

So, whenever you are writing your selectors, ask yourself: is this the least specific selector that can do the job here?

All of the specificity rules are officially defined in the W3C CSS Selectors specification, which is the way to find out every single detail about CSS Selectors. For something easier to understand, read this article on CSS specificity.

Don’t throw new rules at bugs

Let’s imagine this typical situation: there is a bug in your CSS and you locate which DOM element has the wrong style. And you realise it’s somehow inheriting a property that it shouldn’t have.

Don’t just throw more CSS at it. If you do, your code base will grow a bit larger, and locating future bugs will be a bit harder.

Instead, stop, step back, and use the developer tools in your browser to inspect the element and see the whole cascade. Identify exactly which rule is applying the style you don’t want. And modify that existing rule so that it doesn’t have the unintended consequence.

In Firefox you can debug the cascade by right-clicking on a element in a page and selecting Inspect element.

Look at that cascade in all its glory. Here you can see all the rules applied to an element, in the order in which they are applied. The top entries are the ones with more specificity and can override previous styles. You can see that some rules have some properties struck out: that means that a more specific rule is overriding that property.

And you can not only see the rules, but you can actually switch them on and off, or change them on the fly and observe the results. It’s very useful for bug fixing!

The needed fix may be a rule change or it may be a rule change at a different point in the cascade. The fix may require a new rule. At least you will know it was the right call and something that your code base needed.

This is also a good time to look for refactoring opportunities. Although CSS is not a programming language, it is source code and you should give it the same consideration that you give to your JavaScript or Python: it should be clean, readable and be refactored when needed.

Don’t !important things

This is implied in the previous recommendations, but since it’s crucial I want to stress it: Don’t use !important in your code.

!important is a feature in CSS that allows you to break the cascade. CSS stands for “Cascading Style Sheets,” this is a hint.

!important is often used when you are rushing to fix a bug and you don’t have the time or the will to fix your cascade. It is also used a lot when you are including a CSS framework with very specific rules and it’s just too hard to override them.

When you add !important to a property, the browser will ignore other rules with higher specificity. You know you are really in trouble when you !important a rule to override another rule that was marked as !important as well.

There is one legitimate use of !important —and it’s while using  the developer tools to debug something. Sometimes you need to find which values for a property will fix your bug. Using !important in the developer tools and modifying a CSS rule on the fly lets you find these values while you ignore the cascade.

Once you know which bits of CSS will work, you can go back to your code, and look at which point of the cascade you want to include those bits of CSS.

There’s life beyond px and %

Working with px (pixels) and % (percentages) units is quite intuitive, so we will focus here on less-known or less intuitive units.

Em and rem

The most well-known relative unit is em. 1em is equivalent to the font size of that element.

Let’s imagine the following HTML bit:

<article>
  <h1>Title</h1>
  <p>One Ring to bring them all and in the darkness bind the.</p>
</article>

And a stylesheet with just this rule:

article {
  font-size: 1.25em;
}

Most browsers apply a base font size of 16 pixels to the root element by default (by the way, this is overridable—and a nice accessibility feature—by the user). So the paragraph text of that article element will probably get rendered with a font-size of 20 pixels (16 * 1.25).

What about the h1? To understand better what will happen, let’s add this other CSS rule to the stylesheet:

h1 {
  font-size: 1.25em;
}

Even though it’s also 1.25em, the same as article, we have to take into account that em units compound. Meaning, that an h1 being a direct child of a body, for instance, would have a font-size of 20 pixels (16 * 1.25). However, our h1 is inside an element that has a font-size that is different than the root (our article). In this case, the 1.25 refers to the font-size we are given by the cascade, so the h1 will be rendered with a font-size of 25 pixels (16 * 1.25 * 1.25).

By the way, instead of doing all of these multiplication chains in your head, you can just use the Computed tab in the Inspector, which displays the actual, final value in pixels:

em units are really versatile and make it really easy to change—even dynamically—all the sizes of a page (not just font-size, but other properties like line-height, or width).

If you like the “relative to base size” part of em but don’t like the compounding part, you can use rem units. rem units are like em‘s that ignore compounding and just take the root element size.

So if we take our previous CSS and change em units for rem in the h1:

article { font-size: 1.25em; }
h1 { font-size: 1.25rem; }

All h1 elements would have a computed font-size of 20 pixels (assuming a 16px base size), regardless of them being inside an article or not.

vw and vh

vw and vh are viewport units. 1vw is 1% of the viewport width, whereas 1vh is 1% of the viewport height.

They’re incredibly useful if you need a UI element that needs to occupy the whole screen (like the typical semi-transparent dark background of a modal), which is not always related to the actual body size.

Other units

There are other units that might be not as common or versatile, but you will inevitably stumble upon them. You can learn more about them at the MDN.

Use flexbox

We have talked about this in a previous article about CSS frameworks, but the flexbox module simplifies the task of crafting layouts and/or aligning things. If you are new to flexbox, check out this introductory guide.

And yes, you can use flexbox today. Unless you really need to support ancient browsers for business reasons. The current support for flexbox in browsers is above 94%. So you can stop writing all of those floating div‘s which are hard to debug and maintain.

Also, keep an eye open for the upcoming Grid module, which will make implementing layouts a breeze.

When using a CSS processor…

CSS compilers like Sass or Less are very popular in the front-end development world. They are powerful tools, and—when put in good use—can allow us to work more efficiently with CSS.

Don’t abuse selector nesting

A common feature in these processors or “compilers” is selector nesting. So, for instance, this Less code:

a {
  text-decoration: none;
  color: blue;

  &.important {
    font-weight: bold;
  }
}

Would get translated to the following CSS rules:

a {
  text-decoration: none;
  color: blue;
}

a.important {
  font-weight: bold;
}

This feature allows us to write less code and to group rules that affect elements which are usually together in the DOM tree. This is handy for debugging.

However, it is also common to abuse this feature and end up replicating the whole DOM in the CSS selectors. So, if we have the following HTML:

<article class="post">
  <header>
    <!-- … -->
    <p>Tags: <a href="..." class="tag">irrelevant</a></p>
  </header>
  <!-- … -->
</article>

We might find this in the CSS stylesheet:

article.post {
  // ... other styling here
  header {
    // ...
    p {
      // ...
      a.tag {
        background: #ff0;
      }
    }
  }
}

The main drawback is that these CSS rules have extremely specific selectors. We have already seen that this is something we should avoid. There are other disadvantages as well to over-nesting, which I have talked about in another article.

In short: do not let nesting generate CSS rules you wouldn’t type yourself.

Include vs extend

Another useful feature of CSS processors is mixins, which are re-usable chunks of CSS. For instance, let’s say that we want to style buttons, and most of them have some basic CSS properties. We could create a mixin like this one in Less:

.button-base() {
  padding: 1em;
  border: 0;
}

And then create a rule like this:

.button-primary {
  .button-base();
  background: blue;
}

This would generate the following CSS:

.button-primary {
  padding: 1em;
  border: 0;
  background: blue;
}

As you can see, very handy to refactor common code!

Besides “including” a mixin, there is also the option of “extending” or “inheriting” it (the exact terminology differs from tool to tool). What this does is to combine multiple selectors in the same rule.

Let’s see an example using the previous .button-base mixin:

.button-primary {
  &:extend(.button-base)
  background: blue;
}

.button-danger {
  &:extend(.button-base)
  background: red;
}

That would be translated to:

.button-primary, .button-danger {
  padding: 1em;
  border: 0;
}

.button-primary { background: blue; }
.button-danger { background: red; }

Some articles online tell us to only use “include”, whereas some tell us to only use “extend”. The fact is that they produce different CSS, none of them is inherently wrong, and depending on your actual scenario it would be better to use one or the other.

How to choose between them? Again, the “would I write this by hand?” rule of thumb applies.

I hope this can help you to reflect on your CSS code and write better rules. Remember what we said before: CSS is code, and as such, worthy of the same level of attention and care as the rest of your code base. If you give it some love, you will reap the rewards.

(French translation)

A couple of years ago I drew this picture and started using it in various presentations about agile and lean development:

Since then the drawing has gone viral! Shows up all over the place, in articles and presentations, even in a book (Jeff Patton’s “User Story Mapping”  – an excellent read by the way). Many tell me the drawing really captures the essence of iterative & incremental development, lean startup, MVP (minimum viable product), and what not. However, some misinterpret it, which is quite natural when you take a picture out of it’s original context. Some criticize it for oversimplifying things, which is true. The picture is a metaphor. It is not about actual car development, it is about product development in general, using a car as a metaphor.

Anyway, with all this buzz, I figured it’s time to explain the thinking behind it.

First example – Not Like This

The top row illustrates a common misconception about iterative, incremental product development (a.k.a Agile).

Many projects fail badly because they do Big Bang delivery (build the thing until 100% done and deliver at the end). I’ve lost count of the number of failed projects I’ve seen because of this (scroll down for some examples). However, when Agile is presented as an alternative people sometimes balk at the idea of delivering an unfinished product – who wants half of a car?. Imagine this:

“Here sir, here’s our first iteration, a front tire. What do you think?”

Customer is like “Why the heck are you delivering a tire to me? I ordered a CAR! What am I supposed to do with this?”

(By the way, I’m using the term “customer” here as a generic term to represent people like product managers, product owners, and early adopter users).

With each delivery the product gets closer to done, but the customer is still angry because he can’t actually use the product. It’s still just a partial car.

And finally, when the product is done, the customer is like “Thanks! Finally! Why didn’t you just deliver this in the first place, and skip all the other useless deliveries?”.

In this example he’s happy with the final product, because it’s what he ordered. In reality, that’s usually not true. A lot of time has passed without any actual user testing, so the product is most likely riddled with design flaws based on incorrect assumptions about what people need.  So that smiley face at the end is pretty idealistic.

Anyway, the first row represents “bastardized agile”. Technically it might be incremental and iterative delivery, but the absence of an actual feedback loop makes it very risky – and definitely not agile.

Hence the “Not Like This” heading.

Second example – Like this!

Now for the second row.

Here we take a very different approach. We start with the same context – the customer ordered a car. But this time we don’t just build a car. Instead we focus on the underlying need the customer wants fulfilled. Turns out that his underlying need is “I need to get from A to B faster”, and Car is just one possible solution to that. Remember, car is just a metaphor, think any kind of customized product development situation.

So the team delivers the smallest thing they can think of that will get the customer testing things and giving us feedback. Some might call it an MVP (Minimum Viable Product), but I prefer to call it Earliest Testable Product (more on that further down).

Call it what you like (some even call their first release the “the skateboard version” of the product, based on this metaphor….).

The customer is unlikely to be happy with this. This is nowhere near the car he ordered. But that’s OK! Here’s the kicker – we’re not trying to make the customer happy at this point. We might make a few early adopters happy (or pragmatists in pain), but our main goal at this point is just to learn. Ideally, the team explains this clearly to the customer in advance, so he isn’t too disappointed.

However, as opposed to the front wheel in the first scenario, the skateboard is actually a usable product that helps the customer get from A to B. Not great, but a tiny bit better than nothing. So we tell the customer “don’t worry, the project is not finished, this was just the first of many iterations. We’re still aiming to build a car, but in the meantime please try this and give us feedback“. Think big, but deliver in small functionally viable increments.

We might learn some really surprising things. Suppose the customer says he hates the skateboard, we ask why, and he says “I hate the color”. We’re like “uh…. the color? That’s all?”. And the customer says “Yeah, make it blue! Other than that, it’s fine!”. You just saved *alot* of money not building the car! Not likely, but who knows?

The key question is “What is the cheapest and fastest way we can start learning?” Can we deliver something even earlier than a skateboard? How about a bus ticket?

Will this help solve the customer’s problem? Maybe, maybe not, but we’ll definitely learn something by putting this into the hands of real users. Lean Startup offers a great model based on listing your actual hypotheses about the users and then working systematically to validate or invalidate them.

You don’t need to test the product on all users, and you don’t need to build a product to test something. Testing a prototype on even a single user will teach you more than nothing.

But OK, back to the skateboard example.

After playing around with it in the office, the customer says “OK, kind of fun, and it does get me to the coffee machine faster. But it’s unstable. I fall off too easily”.

So the next iteration we try to solve that problem, or at least learn more about it.

Customer can now get around the office without falling off!

Happy? Not really, he still kind of wants that car. But in the meantime he is actually using this product, and giving us feedback. His biggest complaint is that it’s hard to travel longer distances, like between buildings, due to the small wheels and lack of breaks. So, next release the product morphs into something like a bicycle.

Now the customer can zoom around campus. Yiihaaa!

We learn some things along the way: The customer likes the feeling of fresh air on his face. The customer is on a campus, and transportation is mostly about getting around between buildings.

The bicycle may turn out to be a much better product than the car originally envisioned. In fact, while testing out this product we may learn that the paths are too narrow for a car anyway. We just saved the customer tons of time and money, and gave him a better product in less time!

Now you may be thinking  “but shouldn’t we already have known that. via up-front analysis of the customer’s context and needs?” Good point. But in most real-life product development scenarios I’ve seen, no matter how much up-front analysis you do, you’re still surprised when you put the first real release into the hands of a real user, and many of your assumptions turn out to be way off.

So yes, do some up-front analysis, discover as much as you can before starting development. But don’t spend too much time on it and don’t trust the analysis too much – start prototyping and releasing instead, that’s when the real learning happens.

Anyway, back to the story. Perhaps the customer wants more. Sometimes he needs to travel to another city, and the bike ride is too slow and sweaty. So next iteration we add an engine.

This model is especially suitable for software, since software is, well, Soft. You can “morph” the product as you go, as opposed to hardware where you essentially have to rebuild every time. However, even in hardware projects there is a huge benefit to delivering prototypes to observe and learn from how the customer uses your product. It’s just that the iterations tend to be a bit longer (months instead of weeks). Even actual car companies like Toyota and Tesla do a lot of prototyping (sketches, 3d models, full-scale clay models, etc) before developing a new car model.

So now what? Again, maybe the customer is happy with the motorcycle. We could end the project earlier than planned. Most products are riddled with complexity and features that nobody uses. The iterative approach is really a way of delivering less, or finding the simplest and cheapest way to solve the customer’s problem. Minimize the distance to Awesome. Very Zen.

Or, again, the customer could choose to continue, with or without modifications to the requirements. We may in fact end up with the exact same car as originally envisioned. However it is much more likely that we gain vital insights along the way and end up with something slightly different. Like this:

The customer is overjoyed! Why? Because we learned along the way that he appreciates fresh air in his face, so we ended up with a convertible. He did get a car after all – but a better car than originally planned!

So let’s take a step back.

What’s your skateboard?

The top scenario (delivering a front tire) sucks because we keep delivering stuff that the customer can’t use at all. If you know what you’re doing – your product has very little complexity and risk, perhaps you’ve built that type of thing hundreds of times before – then go ahead and just do big bang. Build the thing and deliver it when done.

However, most product development efforts I’ve seen are much too complex and risky for that, and the big bang approach all too often leads to huge expensive failures. So the key question is What’s your skateboard?

In product development, one of the first things you should do (after describing what problem you are trying to solve for whom) is to identify your skateboard-equivalent. Think of the skateboard as a metaphor for the smallest thing you can put in the hands of real users, and get real feedback. Or use “bus ticket” if that metaphor works better.

This will give you the vitally needed feedback loop, and will give both you and the customer control over the project – you can learn and make changes, instead of just following the plan and hoping for the best.

Let’s take at some real-life examples.

Example 1: Spotify music player

“With over 75 million users, it’s hard to remember a time without Spotify. But there was. A time when we were all mulling the aisles of Target for new CDs. A time in our lives where we all became thieves on Napster. A time when iTunes forced us to buy songs for $2/piece. And then came Spotify.” –Tech Crunch

Spotify is a pretty fancy product now. But it didn’t start that way. I was lucky to be involved pretty early in this amazing journey (and still am).

As a startup in 2006, Spotify was founded on some key assumptions – that people are happy to stream (rather than own) music, that labels and artists are willing to let people do so legally, and that fast and stable streaming is technically feasible. Remember, this was 2006 when music streaming (like Real Player) was a pretty horrible experience, and pirate-copied music was pretty much the norm. The technical part of the challenge was: “Is it even possible to make a client that streams music instantly when you hit the Play button? Is it possible to get rid of that pesky ‘Buffering’ progress bar?”

Starting small doesn’t mean you can’t think big. Here’s one of the early sketches of what they had in mind:

But instead of spending years building the whole product, and then finding out if the assumptions hold, the developers basically sat down and hacked up a technical prototype, put in whatever ripped music they had on their laptops, and started experimenting wildly to find ways to make playback fast and stable. The driving metric was “how many milliseconds does it take from when I press Play to when I hear the music?”. It should play pretty much instantly, and continue playing smoothly without any stuttering!

“We spent an insane amount of time focusing on latency, when no one cared, because we were hell bent on making it feel like you had all the world’s music on your hard drive. Obsessing over small details can sometimes make all the difference. That’s what I believe is the biggest misunderstanding about the minimum viable product concept. That is the V in the MVP.” -Daniel Ek, co-founder and CEO

Once they had something decent, they started testing on themselves, their family, and friends.

The initial version couldn’t be released to a wider audience, it was totally unpolished and had basically no features except the ability to find and play a few hard-coded songs, and there was no legal agreement or economic model. It was their skateboard.

But they shamelessly put the skateboard in the hands of real users – friends and family – and they quickly got the answers they needed. Yes, it was technically possible. And yes, people absolutely loved the product (or more like, what the product can become)! The hypotheses were validated! This running prototype helped convince music labels and investors and, well, the rest is history.

Example 2: Minecraft

Minecraft is one of the most successful games in the history of game development, especially if you take development cost into consideration. Minecraft is also one of the most extreme examples of the release-early-and-often mindset. The first public release was made after just 6 days of coding, by one guy ! You couldn’t do much in the first version – it was basically an ugly blocky 3d-landscape where you can dig up blocks and place them elsewhere to build crude structures.

That was the skateboard.

The users were super-engaged though (most developer-user communication happened via Twitter, pretty funny). Among the early users were me and my four kids.  Over hundred releases were made during the first year. Game development is all about finding the fun (some game companies I’ve worked with use the term “Definition of Fun” instead of “Definition of Done”), and the best way to do that is by having real people actually play that game – in this cases thousands of real people who had actually paid to try the early access version and therefore had a personal incentive to help improve the game.

Gradually a small development team was formed around the game (mostly 2 guys actually), and the game became a smash hit all over the world. I don’t think I’ve met any kid anywhere who doesn’t play Minecraft. And last year the game (well, the company that was formed around the game) was sold to Microsoft for $2.5 billion. Quite amazing.

Example 3: Big Government Project

Around 2010 the Swedish Police started a big initiative to enable police to spend more time in the field and less time at the station – PUST (Polisens Utrednings STöd). A fascinating project, I was involved as coach and wrote a book about what we did and what we learned (Lean from the Trenches).

The idea was to put laptops in the cars, and customized software to give police access to the systems they need in real-time, for example while interrogating a suspect (this was the pre-tablet age).

They had tried to build similar systems in the past and failed miserably, mainly because of Big Bang thinking. They told me that one of their previous attempts took 7 years from inception to first release. SEVEN YEARS! By then of course everything had changed and the project was a total failure. So this time they wanted to do it differently.

The 60-person project (later referred to as “PUST Java”) succeeded surprisingly well, especially for being a big government project (it even came second in CIO Awards “Project of the Year”). One of the main success factors was that they didn’t try to build the whole thing at once – they split the elephant along two dimensions:

• By Region. We don’t need to release to ALL of Sweden at once, we could start by releasing to just one region.
• By Crime type. We don’t need to support ALL crime types initially, we could start by just supporting 1-2 crime types.

The first version, 1.0, was their skateboard.

It was a small system, supporting just a couple of crime types, and it was field-tested on a handful of cops in Östergötland (a region in Sweden). Other crime types had to be dealt with the old way – drive to the station and do paperwork. They knew they were guinea pigs, and that the product was nowhere near finished. But they were happy to test it, because they knew the alternative. They had seen what kind of crappy systems come out of processes that lack early user feedback, and now they finally had a chance to influence a system while it was being built!

Their feedback was harsh and honest. Many of our assumptions flew out the window, and one of the big dilemmas was what to do with all the carefully crafted Use Case specifications that were getting less and less relevant as the real user feedback came in (this was an organization with a waterfall history and a habit of doing big upfront analysis).

Anyway, long story short, the early feedback was channeled into product improvements and, gradually, as the those cops in Östergötland started liking the product, we could add more crime types and spread it to more regions. By the time we got to the big release (1.4), with nationwide rollout and training of 12000 police, we weren’t all that worried. We had done so many releases, so much user testing, that we slept well on the night of the nationwide release.

Unfortunately the victory was short-lived. A follow-up project (PUST Siebel) botched it and went back to waterfall-thinking, probably due to old habit. 2 years of analysis and testing without any releases or user-testing, followed by a big-bang release of the “next generation” of the product to all 12,000 police at once. It was an absolute disaster, and after half a year of hemorrhaging they shut the whole thing down. The development cost was about €20 million, but Internal studies estimate that the cost to Swedish society (because the police were handicapped by the horrible system) was on the order of €1 Billion!

Pretty expensive way to learn!

Example 4: Lego

I’m currently working at Lego, and I’m amazed by their ability to deliver new smash-hits, year after year without fail. I hear lots of interesting stories about how they do this, and the common theme is prototyping and early user testing! I often see groups of kids in the office, and designers collaborate with local kindergartens and schools and families to field-test the latest product ideas.

Here’s a recent example – Nexo Knights (released Jan 2016):

When they first started exploring this concept, they did paper prototypes and brought to small kids. The kids’ first reaction was “hey, who are the bad guys? I can’t see who’s good and who’s bad!”. Oops. So the designers kept iterating and testing until they found a design that worked with the kids. I bet even you can see who’s good and who’s evil in the picture above…

Not sure exactly where the skateboard is in this story, but you get the idea – early feedback from real users! Don’t just design the product and build the whole thing. Imagine if they had built the product based on their original design assumptions, and learned about the problem after delivering thousands of boxes to stores all over the world!

Lego also has it’s share of hard-earned failures. One example is Lego Universe, a massively multiplayer online Lego world. Sounds fun huh? Problem is, they got overambitious and ended up trying to build the whole thing to perfection before releasing to the world.

About 250 people worked for 4-5 years (because of constant scope creep due to perfectionism), and when the release finally came the reception was… lukewarm. The finished game was beautiful, but not as fun as expected, so the product was shut down after 2 years.

There was no skateboard!

Why not? Because skateboards aren’t Awesome (at least not if you’re expecting a car), and Lego’s culture is all about delivering Awesome experiences! If you work at Lego HQ in Billund, Denmark you walk past this huge mural every day:

Translates roughly to “Only the best is good enough”. It has been Lego’s guiding principle ever since the company started 80+ years ago, and has helped them become one of the most successful companies in the world. But in this case the principle was misapplied. The hunt for perfection delayed vital feedback, which meant mistaken assumptions about what the users like and don’t like. The exact opposite of Minecraft.

Interestingly enough the Lego Universe teams were actually using Scrum and iterating heavily – just like the Minecraft guys did. But the releases were only internal. So there was most likely a skateboard, and a bicycle, and so on, but those products never reached real users. That’s not how Scrum is intended to be used.

It was an expensive failure, but Lego learned from it and they are constantly getting better at early testing and user feedback.

Improving on “MVP”

And that (deep breath…) brings me to the topic of MVP – Minimum Viable Product.

The underlying idea is great, but the term itself causes a lot of confusion and angst. I’ve met many customers that are like “No Way do I want an MVP delivery –  that’s the last delivery I’ll get!” All too often teams deliver the so-called Minimum Viable Product, and then quickly get whisked away to the next project, leaving the customer with a buggy, unfinished product. For some customers, MVP = MRC (Minimum Releasable Crap).

I know, I know, this comes down to bad management rather than the term MVP, but still… the term invites misunderstanding. “Minimum” and “Viable” mean different things to different people, and that causes problems.

So here’s an alternative.

First of all, replace the word “Minimum” with “Early”. The whole idea behind releasing an MVP is to get early feedback – by delivering a minimum product rather than a complete product, we can get the feedback earlier.

Few customers want “minimum” but most customers want “early”! So that’s our first change:

Minimum => Earliest

Next, remove the word “Viable” since it’s too vague. Your “viable” is my “horrible”. Some people think Viable means “something I can test and get feedback from”, others think it means “something the customer can actually use”. So let’s be more explicit and split it into three different things:

Earliest Testable Product is the skateboard or bus ticket – the first release that customers can actually do something with. Might not solve their problem, but it generates at least some kind of feedback. We make it very clear that learning is the main purpose of this release, and that any actual customer value will be a bonus.

Earliest Usable Product is perhaps the bicycle. The first release that early adopters will actually use, willingly. It is far from done, and it might not be very likeable. But it does put your customers in a better position than before.

Earliest Lovable Product is perhaps the motorcycle. The first release that customers will love, tell their friends about, and be willing to pay for. There’s still lots to improve, and we may still end up with a convertible, or a plane, or something else. But we’ve reached the point where we have a truly marketable product.

I considered adding an even earlier step “Earliest Feedbackable Product”, which is basically the paper prototype or equivalent that you use to get your first feedback from the customer. But four steps seems too many, and the word Feedbackable….. ugh. But nevertheless, that is also an important step. Some would call a paper prototype the Earliest Testable Product, but I guess that comes down to how you define Testable. Check out Martin’s MVP Guide to learn more – he’s got plenty of super-concrete examples of how to get early feedback with minimum investment.

Of course people can still misinterpret Earliest Testable/Usable/Lovable, but it’s at least one step more explicit than the nebulous Minimum Viable Product.

Takeaway points

OK time to wrap it up. Never thought it would get this long, thanks for sticking around! Key takeaways:

• Avoid Big Bang delivery for complex, innovative product development. Do it iteratively and incrementally. You knew that already. But are you actually doing it?
• Start by identifying your skateboard – the earliest testable product. Aim for the clouds, but swallow your pride and start by delivering the skateboard.
• Avoid the term MVP. Be more explicit about what you’re actually talking about. Earliest testable/usable/lovable is just one example, use whatever terms are least confusing to your stakeholders..

And remember – the skateboard/car drawing is just a metaphor. Don’t take it too literally :o)

PS – here’s a fun story about how my kids and I used these principles to win a Robot Sumo competition :o)

Thanks Mary Poppendieck, Jeff Patton, Alistair Cockburn, Anders Haugeto, Sophia, colleagues from Crisp, Spotify and Lego, and everyone else who gave tons of useful feedback.

• lien n°1 : Site web de Calaos
• lien n°2 : Documentation du projet
• lien n°3 : Forum de la comunauté
• lien n°4 : Téléchargement
• lien n°5 : Code source sur github
• lien n°6 : Capture vidéos de l'interface

Lire les commentaires

J’utilise depuis quelques mois une solution domotique qui me donne pleinement satisfaction. Cette solution relativement économique repose sur l’utilisation d’un RaspberryPi avec le logiciel Domoticz ainsi que d’un « modem » radio 433MHz RFXCom

Je ne vais pas détailler l’installation de Domoticz, tout est décrit sur leur site et l’opération reste très simple.

Mes besoins essentiels concernent les consommations de l’habitation ( eau, gaz, électricité ) et les relevés de température et humidité dans certaines pièces. De plus, je pilote quelques lumières ainsi qu’un radiateur de type sèche serviettes. Depuis quelques temps j’ai ajouté des capteurs d’ouverture sur les Vélux et les portes. Je peux ainsi savoir par une notification sur le téléphone si l’on quitte la maison avec un Vélux oublié ouvert…

La solution à base de technologie radio en 433MHz présente quelques avantages:

• Le prix
• Le nombre de modules disponibles sur le marché
• La possibilité de créer facilement ses propres capteurs avec un arduino par exemple

Le gros inconvénient c’est que contrairement au Zwave, il n’y a pas de retour d’état. Pour piloter des éléments « critiques » comme le chauffage, c’est un peu gênant mais on peut y remédier d’une façon assez satisfaisante à l’aide d’un script.

Les modules que j’utilise sont disponibles facilement dans les grandes surfaces de bricolage ou sur internet.

Le raspberry équipé d’une carte RFM12Pi en 868MHz pour un usage futur.

Le modem RFXCom. On pourrait en créer un avec un arduino pour beaucoup moins cher mais il faut décoder les différents protocoles ce qui m’aurait demandé trop de temps. J’ai donc préféré acheter un module « tout fait » et consacrer plus de temps sur les autres parties.

Des sondes de température Oregon sont disposées aux endroits stratégiques ( Salon, Salle de bain, extérieur, chambres…)

Le carillon de la porte d’entrée est lui aussi compatible. Il peut donc servir d’alerte sonore si besoin, d’autant qu’il dispose de 3 mélodies utilisables avec des codes différents.

Pour les Vélux, de simples capteurs d’ouverture

Pareil pour la porte du garage

Il y a aussi un capteur de mouvement pour le déclenchement d’un éclairage et d’une notification sur smartphone

Les lumières sont pilotables par ces petits relais.

J’utilise aussi les modules d’éclairage pour piloter le sèche serviette. Attention, il faut le piloter au travers du fil pilote. Il y a une petite astuce très bien décrite sur le site d’Alexandre.

La consommation électrique est relevée par un OWL180. Une petite pince ampèremétrique branchée sur le compteur transmet la consommation en temps réel.

Un petit écran déporté affiche la consommation en temps réel

Pour le gaz, c’est un peu plus compliqué mais aussi plus amusant. Les modules du commerce sont très chers, le DIY vient à la rescousse.

C’est simple, un arduino relève les impulsions générées par le compteur de gaz (un aimant est placé sur la roue du compteur) et les transmet à Domoticz par radio. On peut capter les impulsions d’un compteur de gaz avec un interrupteur reed.

Par la même occasion je relève la température et l’humidité du garage avec un DHT22.

Je vais aussi ajouter la consommation d’eau un peu plus tard.

Et le circuit vraiment très simple

Le schéma et le code pour programmer l’arduino sont sur Github ici. C’est une première version mais ça marche très bien.

Pour la transmission des informations, j’ai utilisé une librairie permettant de simuler un capteur de type RFXMeter et RFXSensor. Du coup domoticz détecte le capteur tout seul comme un grand. Pour les informations de température et d’humidité, j’ai plutôt utilisé le format Oregon V2 gentiment mis à disposition par Olivier ici. Le nombre d’impulsions comptées est stocké dans l’EEPROM de l’arduino toutes les heures afin de ne pas perdre les informations en cas de coupure de courant.

Un seul arduino peut donc émuler toutes les sondes possibles et imaginables. D’autant qu’il ne faut qu’un arduino et un petit transmetteur 433MHz qui coûte 2-3€ ! Pour les arduino, j’utilise des Mini Pro en 5v ou en 3.3v suivant le type d’alimentation.

Après les informations concernant le matériel, voilà un petit tour de ce que ça donne dans Domoticz.

Des scripts lua sont aussi possibles pour faire ce qui n’est pas possible avec l’interface web. Ici par exemple, on envoie une notification sur mobile si la porte du garage est restée ouverte. Pratique avec des enfants

print(‘Porte de garage…’)

t1 = os.time()
s = otherdevices_lastupdate[‘Porte de garage’]
— returns a date time like 2013-07-11 17:23:12

year = string.sub(s, 1, 4)
month = string.sub(s, 6, 7)
day = string.sub(s, 9, 10)
hour = string.sub(s, 12, 13)
minutes = string.sub(s, 15, 16)
seconds = string.sub(s, 18, 19)

commandArray = {}

t2 = os.time{year=year, month=month, day=day, hour=hour, min=minutes, sec=seconds}
difference = (os.difftime (t1, t2))
if (otherdevices[‘Porte de garage’] == ‘Open’ and difference > 300 and difference < 400) then
commandArray[‘SendNotification’]=’La porte de garage est ouverte depuis plus de 5 minutes!’

end

return commandArray

Il y a aussi des applications android / iphone pour piloter le tout avec son mobile.

Voilà un bref aperçu de ce que l’on peut faire. Tout ou presque est possible ! Si vous avez des questions sur cette solution, n’hésitez pas à laisser un commentaire

For the introduction of Elasticsearch #BYODemo environment, please check out our announcement blog post.

Background

“…today in New York, approximately 4,000 New Yorkers are seriously injured and more than 250 are killed each year in traffic crashes. Being struck by a vehicle is the leading cause of injury-related death for children under 14, and the second leading cause for seniors. On average, vehicles seriously injure or kill a New Yorker every two hours.

This status quo is unacceptable. The City of New York must no longer regard traffic crashes as mere ‘accidents,’ but rather as preventable incidents that can be systematically addressed. No level of fatality on city streets is inevitable or acceptable. This Vision Zero Action Plan is the City’s foundation for ending traffic deaths and injuries on our streets.

The City will use every tool at its disposal to improve the safety of our streets.”

From NYC.gov’s Vision Zero site

Before making major public policy decisions based on anecdotal information that may cost significant taxpayer money based on anecdotal information, are there tools available to analyze data to identify patterns and trends then drill down into details in motor vehicle collisions? Can we get answers to questions such as:

• Are there noticeable trends in the number and types of accidents occurring over time?
• Are there certain areas that experience more accidents than others? Are there particular types of accidents more likely to occur in these areas?
• Are there intersections within my borough that experience more accidents involving pedestrians than others? What are the leading causes of accidents in those areas?

In less than an hour, we can set up Logstash, Elasticsearch, and Kibana (ELK) to start asking these questions. With our search capabilities, powerful APIs, and visualizations, it’s possible to calculate totals, counts, averages and have a very interactive experience with your data. And if you don’t know what to ask, you can use aggregations to discover aspects and anomalies of the data.

Getting started

After following these instructions, you should see the following dashboard in your browser.

By clicking on the rows LIVES LOST and PEOPLE INJURED, several stats panels will be opened, displaying statistics for deaths and injuries at a glance.

The row REASONS FOR LOSS OF LIFE accommodates a hits panel displaying the varying reasons lives are lost in combination with an automobile accident.

In the row REASONS FOR INJURIES you will find the same for injuries.

We have setup a number of different panels to show the many different ways you can visualize and interact with the data in Kibana. We encourage you to expand each row and learn more about the data and how to visualize it in Kibana.

Discovering the uncommonly common

By clicking the row ALL ACCIDENTS OVER TIME, a histogram for all accident types over the time will be displayed.

There are several spikes indicating some unusual behavior in our data. But which events and accident types are responsible for these? Can we find the reason among our all-time top 5 accident types? Let us open the row TOP 5 ACCIDENTS OVER THE TIME and see if we can answer this question.

By looking at the the top 5 accident types: Driver Inattention/Distraction, Failure to Yield Right-of-Way, Fatigued/Drowsy, Backing Unsafely, Lost Consciousness in the histogram TOP 5 EVENTS OVER THE TIME it becomes clear that these events are not necessarily the major factors for most of the spikes. How can we find out what the top contributing factors for these time periods are?

Let us zoom into the time around March 1, 2013 on the histogram ALL ACCIDENTS OVER TIME (just span a rectangle with the mouse around the spike).

Now let us see the top accidents types for the selected time range. In doing so, we can refer to the terms panel TOP ACCIDENT TYPES in the row ALL ACCIDENT TYPES AND THEIR DISTRIBUTIONS.

There is a change of order. Suddenly, a new accident type is appearing among the top 5 – Slippery. Is this the accident type responsible for the spike in accidents reported? If yes, we would see some correlation of spikes between all events and this particular accident type. To confirm this, we are going to modify the histogram ALL EVENTS OVER TIME. By clicking on the cog symbol in the top right corner, a setting dialog will be displayed.

Please open the tab called Queries and activate Slippery by clicking on it. Now click on the Save button. The histogram should look like this.

The correlation of all events and accidents of the type Slippery at the February 9, 2013 looks like this.

Let’s have a look at this eye-catching behavior. After selecting the time frame around this spike and removing all events query, we get something like this.

Since we are looking at the query for Slippery, we assume that the reason for this unusual event is due to some changed weather conditions.

By doing some research, we discover that on February 9, 2013, the storm unofficially called “Winter Storm Nemo” or “Blizzard of 2013” developed strong activity over NYC and other parts of the East Coast. It was snowing, freezing cold, and streets got slippery, no wonder there was a spike in accidents caused by slippery pavement!

Image of the the nor’easter on February, 2013 (source: Wikipedia)

By the way:

• We could improve our demo by using some weather data directly in Kibana. We leave it to you as an exercise.
• Elasticsearch version 1.0 introduced aggregations as a new way to explore data in multiple dimensions. To getting started with aggregations using this dataset, use the sample query by clicking on the link. Please make sure that Marvel is up and running. The sample query will be loaded into Marvel Sense console.

Are there places that are more dangerous than others?

Let’s see if there are any areas of NYC that are more likely to have slippery streets (on the January 21st, 2014 – another date with a spike in activity). The pie chart DISTRIBUTION OF ACCIDENTS OVER BOROUGHS suggests that the most accidents happened in Queens followed by Manhattan, Brooklyn, the Bronx, and Staten Island.

Let us have a closer look at Queens by clicking on the corresponding part of the pie chart and analyze the distribution of the accidents on the map LOCATION OF ACCIDENTS. The map suggests that all events are almost evenly distributed across Queens and the limited slice of the data we’re looking at prevents us from making any clear conclusions on specific danger areas.

We can broaden our analysis by expanding the search and removing all filters to look at the entire data set for Queens. Now, some locations in Queens indicate multiple accidents. In the example below, we can observe seven accidents at the junction of Queens Blvd. and Skillman Avenue. With this information, the city may want to consider implementing safety measures at that particular location.

Understanding where and why fatal accidents occur

It is in the interest of every modern city to make roads safer. One way to achieve this goal is to understand why and where accidents happen.

Let us have a look at the overall statistics for deaths over all accidents types. By clicking at the row PEOPLE KILLED, some stats panels will be displayed exhibiting the number of killed people by category.

Let us now analyze geographic distribution of accidents with deadly outcome. For doing so, we will modify the query All accidents. Please unpin the query.

After unpinning, an input field will be displayed.

In the input field, an arbitrary query can be entered and issued. To restrict the dataset to the events with a deadly outcome, please enter number_of_pedestrians_killed:[1 TO *] OR number_of_persons_killed:[1 TO *] OR number_of_motorists_killed:[1 TO *] OR number_of_cyclists_killed:[1 TO *].

Let us go back to to the pie chart DISTRIBUTION OF ACCIDENTS OVER BOROUGHS. It should look like this, suggesting that the most accidents with deadly outcome happen to be in Brooklyn followed by Queens, Manhattan, Bronx and Staten Island.

Let us check the distribution on a map and see if there are any “hot spots”.

It looks like that the accidents are evenly distributed and there is no particular hot spot. Another observation you can make by zooming into the map, is that the many accidents happen on a road junction. Checking the contribution factors and if ignoring the numbers for unspecified events, it it gets evident that Trafic Control Disregarded is the top contribution factor followed by Failure to Yield Right-of-Way and Driver Inattention/Distraction.

By the way, Failure to Yield Right-of-Way is among the all-time top three factors for all accidents. It seems that NYC needs to address this issue in particular.

Summary

It is said that a picture is worth a thousand words. We believe this is true. That is the one of the reasons why we’re proud of what the ELK stack (Elasticsearch, Logstash, Kibana) can do.

This New York City traffic incidents demo is the first of a series of demos we hope to publish in the coming months to show how easy it is to perform meaningful analysis on data using Elasticsearch, Logstash and Kibana. In this post, we presented you with some ideas on how to get started with experimenting with this data set, but we encourage you to explore further — brainstorm to generate interesting questions you want to ask of the data and experiment with the various features in Kibana to start getting answers to those questions. If you find anything interesting or have feedback you’d like to share with us, we’d love to hear it either via email or on Twitter – just make sure to use the hashtag #BYODemos.

This blog and associated demo was a collaborative effort and certainly would not have been possible with out help from my colleagues Steve Mayzak, Peter Kim, and Christian Dahlqvist.

The post #BYODemos: New York City Traffic Incidents appeared first on Elasticsearch.org.

En attendant avec impatience le cru 2014 de la conférence Scala.IO, j’ai récemment regardé la présentation ‘Trouble with Types’, de Martin Odersky. Elle a eu lieu lors de la conférence Strange Loop l’année dernière. L’inventeur du langage Scala propose de parler de théorie des langages et surtout, des systèmes de type. Car finalement, pourquoi a-t-on besoin de type?

Petit rappel

Il existe un premier découpage des systèmes de type, dynamique ou statique. Il nous est rappelé les points forts des deux écoles.

Typage statique

• Plus efficace à l’exécution
• Outillage de développement plus précis
• Moins de tests nécessaires, des contraintes sont vérifiées à la compilation
• Le typage sert de documentation
• Permet du refactoring pour la maintenance

Typage dynamique

• Le langage en lui-même est plus simple
• Plus expressif et moins verbeux
• Pas de problème de compilation
• Plus adapté à l’exploration

Qu’est-ce qu’un bon design?

Martin enchaîne rapidement sur cette épineuse question. Pour lui, il doit être :

• clair
• correct
• minimal

On retrouve ici les principes des designs évolutifs avec les deux acronymes YAGNI (You Ain’t Gonna Need It) et KISS (Keep It Simple, Stupid).

La recette de Martin pour construire un bon design? Patterns & Constraints.

Comment un langage peut-il permettre de faire un bon design? Tout simplement en permettant de le découvrir. Petite citation du jour : « Great designs are often discovered, not invented ». Pour lui, les « patterns » s’expriment au travers des « abstractions », les « contraintes » par les « types ».

Des types pour exprimer des contraintes

Et donc voilà pourquoi on a besoin de types ! Pour y exprimer des contraintes métier. La valeur de ma transaction financière est un réel, pas une chaine de caractère, mon nom de famille est composé de caractères, pas de chiffres… Et cela fait bien partie de notre travail de développeur, traduire les contraintes de problèmes réels dans un langage compréhensible par la machine.

Ce qui va différer entre les deux approches, c’est le niveau de sécurité que l’on veut avoir et à quel moment.

Dans un langage au typage dynamique, ces contraintes sont vérifiées à l’exécution, c’est à dire au plus tard dans le cycle de vie d’un programme. C’est une approche très pragmatique. Il permet des cycles de développement cours et est très adapté à l’exploration. Sa philosophie : What You See Is What You May Get. En effet, sans tests automatisés, impossible d’avoir la sécurité que les différents modules s’intègrent bien les uns avec les autres.

Au contraire, avec un typage statique, le développeur va s’assurer que son programme s’intègre bien au travers de contrats. Ces contraintes sont vérifiées avant même que le programme puisse se lancer. Sa philosophie : What You See Is What You Try To Get. En effet, où placer une contrainte métier : à la compilation dans le type ou à l’exécution par validation? C’est peut-être là LA différence principale entre un développeur habitué à l’orienté objet et un plus familiarisé avec la programmation fonctionnelle.

Avec la programmation fonctionnelle, le typage se veut souvent statique

Issue majoritairement du milieu universitaire, la culture mathématique a induit la volonté de placer un maximum de contraintes dans le type des données et ses abstractions. Les niveaux d’abstractions sont parfois si élevés qu’un développeur habitué à résoudre des problèmes plus « concrets » peut être perdu.

Imaginons la méthode suivante sur un Double.

public Double divisePar(double diviseur){
 if(diviseur == 0){
  throw new IllegalArgumentException("Le diviseur ne peut être 0");
 }

 return this / diviseur;
}

Pourquoi utiliser un double pour le diviseur ? Cette fonction n’est pas définie pour l’ensemble des valeurs de son paramètre en entrée. Je suis obligé de faire une vérification de la valeur à l’exécution, qui se traduit par une exception. Pourquoi ne pas créer un type de donnée similaire au double mais sans 0 ?

Imaginons la méthode suivante sur un Double.

//NonZero est difficile à exprimer en Java
//NonZero un = NonZero(1);
//NonZero(0); //throw new IllegalArgumentException
 
public Double divisePar(NonZero diviseur){
 return this / diviseur;
}

À l’exécution, ces deux propositions sont identiques, une exception sera lancée. Mais avec la seconde solution, la méthode annonce clairement son contrat. Si l’on désire l’utiliser, c’est à l’utilisateur de cette méthode de s’ assurer que le diviseur est différent de zéro.

C’est ce type de réflexion que la programmation sur les types nous amène à nous poser, et il faut avouer que c’est plutôt déroutant.
Le type d’une donnée joue aussi le rôle de documentation. Si je sais qu’en retour d’une méthode, je peux ne rien retourner, pourquoi utiliser null ? Il vaut mieux utiliser le système de type pour traduire cette contrainte. C’est le but de la « classe » Option. J’inscris dans mon contrat, via un type, que je peux ne rien retourner. C’est vérifié à la compilation et l’utilisateur de mon API doit gérer le cas explicitement.

C’est de là que vient toute la problématique avec les types. Nous en avons besoin si nous souhaitons construire des systèmes simples, efficaces et durables. Mais bien le faire est difficile. Il existe des debuggeurs pour vérifier les contraintes de notre programme à l’exécution. Cependant, il n’y a pas de debuggeur quand le compilateur n’arrive pas à faire son travail de vérification des types. Petit extrait d’une exception du compilateur Scala sur un exemple bien connu :

5862.scala:36: error: type mismatch;
found : scala.collection.mutable.Iterable[_ >: (MapReduceJob.this.DataSource, scala.collection.mutable.Set[test.TaggedMapper[_, _, _]]) with test.TaggedMapper[_$1,_$2,_$3] forSome { type _$1; type _$2; type _$3 } <: Object] with scala.collection.mutable.Builder[(MapReduceJob.this.DataSource, scala.collection.mutable.Set[test.TaggedMapper[_, _, _]]) with test.TaggedMapper[_$1,_$2,_$3] forSome { type _$1; type _$2; type _$3 },scala.collection.mutable.Iterable[_ >: (MapReduceJob.this.DataSource, scala.collection.mutable.Set[test.TaggedMapper[_, _, _]]) with test.TaggedMapper[_$1,_$2,_$3] forSome { type _$1; type _$2; type _$3 } <: Object] with scala.collection.mutable.Builder[(MapReduceJob.this.DataSource, scala.collection.mutable.Set[test.TaggedMapper[_, _, _]]) with test.TaggedMapper[_$1,_$2,_$3] forSome { type _$1; type _$2; type _$3 },scala.collection.mutable.Iterable[_ >: (MapReduceJob.this.DataSource, scala.collection.mutable.Set[test.TaggedMapper[_, _, _]]) with test.TaggedMapper[_$1,_$2,_$3] forSome { type _$1; type _$2; type _$3 } <: Object] with scala.collection.mutable.Builder[(MapReduceJob.this.DataSource, scala.collection.mutable.Set[test.TaggedMapper[_, _, _]]) with test.TaggedMapper[_$1,_$2,_$3] forSome { type _$1; type _$2; type _$3 },scala.collection.mutable.Iterable[_ >: (MapReduceJob.this.DataSource, scala.collection.mutable.Set[test.TaggedMapper[_, _, _]]) with test.TaggedMapper[_$1,_$2,_$3] forSome { type _$1; type _$2; type _$3 } <: Object] with scala.collection.mutable.Builder[(MapReduceJob.this.DataSource, scala.collection.mutable.Set[test.TaggedMapper[_, _, _]])

et cela continue sur 200 lignes ;)

Martin le précise dans son discours. Programmer dans le système de type est séduisant, mais reste très peu outillé, complexe car abstrait.

Et JavaScript et Clojure alors ?

Ces deux langages marquent clairement la démarcation entre programmation fonctionnelle et typage statique. Ces langages sont fonctionnels car la fonction est une primitive du langage. Par contre, ils sont dynamiquement typés. Ces langages sont puissants et flexibles. Clojure est d’ailleurs un des langages préférés par la communauté DDD (Domain Driven Design). Tout est immutable et l’on se focalise sur les transformations des données, les fonctions.

Cependant, l’absence de types statiques en fait souvent la cible de critiques de la part des partisans du camp opposé. De la même manière, les temps de compilation de Scala sont tout aussi souvent critiqués.

Jusqu’à récemment, je n’avais que peu d’expérience avec ces langages dynamiques, étant plutôt adepte de langages comme Java, Scala ou Haskell. Notre Hackathon chez Xebia m’a permis de travailler avec Node.JS en JavaScript. Débuter a été une expérience déroutante. Fini ce fil d’Ariane qui me permet de savoir ce que je manipule, ce que je fait. Je présume. À mon étonnement, cela fonctionne plutôt bien.

Conclusion

Le typage statique, c’est majoritairement le choix de la pérennité. Cela permet de transposer un maximum de contraintes métiers dans les types. Le compilateur se charge pour nous de les vérifier avant l’exécution. Cela nous évite une batterie de tests d’intégrations des modules. Le refactoring est aussi plus sûr. En contrepartie, il est souvent plus fastidieux d’écrire du code expressif. Le raisonnement n’est pas aisé et peu induire beaucoup d’erreurs.

À l’opposé, ce n’est pas un hasard si la majorité des startups développent leurs produits dans des langages au typage dynamique comme Ruby avec la plateforme Rails ou Javascript avec Node.js. D’ailleurs, la majorité des projets réalisés lors du Hackathon a choisi Node.JS pour serveur. Cela permet souvent d’aller plus vite et se concentrer sur le produit. Ces langages sont aussi d’excellents supports pour créer des DSL de tests. Mais il s’ensuit quasiment systématiquement une migration vers un langage statiquement typé pour des problématiques de performance et de maintenance. Twitter en est un bon exemple.

Petit bonus de la présentation, Martin nous présente rapidement Dotty, son nouveau langage. Il prototype un nouveau système de typage qui sera certainement l’avenir de Scala. Il tend à simplifier les différentes abstractions du système de typage. Scala fait le pari de lier l’expressivité et la puissance des langages dynamiques et la sécurité d’un système de type statique, au prix d’une compilation au coût qui peut sembler exorbitant. Mais finalement, qu’est-ce qu’une compilation sinon qu’une suite de tests, lancée avant l’exécution, qui s’assure du bon fonctionnement de l’application ? Vous préférez attendre la phase de tests ou le lancement d’Infinitest ?

En écrivant cet article, je pense à une dernière citation. Pour moi, un système de types dynamiques permet de construire un programme qui n’est pas cassé. Avec un système de types statiques, le programme est incassable. Que l’on choisisse de travailler encore plus avec le compilateur ou de s’en passer, produire un design qui sera clair et minimal, cela demande de la maîtrise et beaucoup, beaucoup, beaucoup de pratique.

Et demain, allez-vous typer un peu, beaucoup, à la folie, ou pas du tout ?

Pour aller plus loin

Vous souhaitez faire votre choix, nous vous conseillons les liens suivants.

• Trouble with types: http://www.infoq.com/presentations/data-types-issues
• Un exemple de programmation avancée par typage: http://proseand.co.nz/2014/02/17/type-programming-shifting-from-values-to-types/
• Une présentation intéressante pour le typage dynamique: http://www.infoq.com/presentations/dynamic-static-typing
• Une discussion pour le typage statique : http://www.quora.com/Is-static-type-checking-overrated
• Et pour aller vraiment plus loin : Propositions as types. J’avoue ne pas avoir tout saisie sur ce dernier lien.

Up until now, developing web apps on mobile has been a little tricky.

After spending the time developing your app, getting your users to install it is difficult, especially when the concept of “installing a web app” is not very well defined.

The most popular method is synonymous with adding a shortcut to the homescreen. This is problematic on a number of fronts, not least because the management of web apps – especially around launching, switching between and uninstalling web apps – differs significantly from that of native apps.

The web app “exists” only on the homescreen, not in the app drawer.

When it’s running, it is not clearly marked in the Recent Apps list.

Even once you get something like a smooth user-flow of installing your app onto the user’s phone’s homescreen, you often find that your app is running in a degraded or out-of-date web view, missing out on compatibility or speed optimizations of a desktop class browser.

What we as developers would like is a modern, fast web runtime, which is kept up-to-date on our devices.

Wouldn’t it also be nice for our users to launch and manage their web apps in the same way as native apps?

Introducing APK Factory

We have been working on making web apps be real on the desktop for some time. On the desktop, if you install a web app, Firefox will repackage the app as a desktop app so that it will integrate perfectly with the rest of your system – as outlined in more detail in Progress report on cross-platform Open Web Apps.

That means being in the Start menu on Windows, or in the Launch Control screen on Mac OS X.

From Firefox 29, that will apply to Android too.

This means that as a web developer, you can rely on a modern, up-to-date web runtime on Android to run your web apps. Even better, that web runtime is provided by an ordinary Android app, which means it will stay modern and up-to-date, and you can finally say goodbye to the Android Browser.

A web app, called ShotClock. Notice its icon in the top right of the screen.

The user will experience your web app as if it is a real native Android app:

• The app appears in the App Drawer, the Recent Apps list, with its own names and icons,
• The app can be installed and uninstalled just like a native Android app,
• The app can be updated just like a native Android app.

In the App Drawer

In the Recent Apps list: all these apps are web apps

Installed with certain permissions

Best yet, is that we make these changes without the developer needing to do anything. As a developer, you get to write awesome web apps, and not worry about different packaging needed to deliver the web app to your users.

So if you’re already making first-class apps for Firefox OS, you’re already making first-class apps for Android.

The Technical details

On Firefox, you can install an app using the window.navigator.mozApps.install(manifestUrl) method call. Any website can use this API, so any website can become an app store.

The manifestUrl is the URL of a manifest.json document which describes the app to your phone without actually loading the app:

• The app’s name and description, translated into any number of languages.
• The app’s icon, in various sizes for different pixel densities.
• The permissions that the app needs to run.
• The WebActivities that the app wants to register.
• For packaged apps only, this provides a URL to the zip file containing the app’s code and resources.

On Firefox for Android, we implement this method by sending the URL to a Mozilla-managed service which builds an Android APK specifically for the app.

APKs created by the Factory use Android’s excellent Resource framework so that the correct icon and translation is displayed to the user, respecting the user’s locale or phone screen.

Web app permissions are rendered as Android permissions, so the user will have a completely native experience of installing your app.

For packaged apps, the APK also includes a copy of the packaged zip file, so that no extra networking is required once the app is downloaded.

For hosted apps, the first time the app is launched, the resources listed in its appcache are downloaded, so that subsequent launches can happen as quickly as possible, without requiring a network connection.

And if you want to detect if the app is running in a web app versus in a webpage, checking the getSelf() method call will help you:

if (window.navigator.mozApps) {
  // We're on a platform that supports the apps API.
  window.navigator.mozApps.getSelf().onsuccess = function() {
    if (this.result) {
      // We're running in an installed web app.
    } else {
      // We're running in an webpage.
      // Perhaps we should offer an install button.
    }
  };
}

Keeping your apps up-to-date

For hosted apps, you update your apps as usual: just change your app on your server, and your users will pick up those changes the next time they run your app. You can change as much as you want, and your users will get the latest version of the app each time they launch and connect to your servers.

For anything needing changes to the app’s manifest, your users will get an updated APK sent to them to update their existing installation.

For example, if you want to change the app’s icon, or even name, changing the app’s manifest will cause the APK Factory service to regenerate the app’s APK, and notify your users that there is a new version available for install.

For packaged apps, the same mechanism applies: change the app’s package zip file, then update the version number in the app’s manifest file, and the APK Factory will pick up those changes and notify your users that an updated app is available. Your users will get an notified that there’s a new APK to install. Simples.

Is that it?

This is an exciting project. It has very little web developer involvement, and no extra API for developers to use: however, it should represent a step forward in usability of web apps on Android.

Now that we have taken the step to generate APKs for web apps, this gives us a platform for further blurring the lines between web apps and apps. Check it out for yourself and help us improve the feature: get Firefox Beta from the Google Play Store, install apps from the Firefox Marketplace, and let us know what you think!

WebRTC is now on Firefox for Android as well as Firefox Desktop! Firefox 24 for Android now supports mozGetUserMedia, mozRTCPeerConnection, and DataChannels by default. mozGetUserMedia has been in desktop releases since Firefox 20, and mozPeerConnection and DataChannels since Firefox 22, and we’re excited that Android is now joining Desktop releases in supporting these cool new features!

What you can do

With WebRTC enabled, developers can:

• Capture camera or microphone streams directly from Firefox Android using only JavaScript (a feature we know developers have been wanting for a while!),
• Make browser to browser calls (audio and/or video) which you can test with sites like appspot.apprtc.com, and
• Share data (no server in the middle) to enable peer-to-peer apps (e.g. text chat, gaming, image sharing especially during calls)

We’re eager to see the ideas developers come up with!

For early adopters and feedback

Our support is still largely intended for developers and for early adopters at this stage to give us feedback. The working group specs are not complete, and we still have more features to implement and quality improvements to make. We are also primarily focused now on making 1:1 (person-to-person) calling solid — in contrast to multi-person calling, which we’ll focus on later. We welcome your testing and experimentation. Please give us feedback, file bug reports and start building new applications based on these new abilities.

If you’re not sure where to start, please start by reading some of the WebRTC articles on Hacks that have already been published. In particular, please check out WebRTC and the Early API, The Making of Face to GIF, and PeerSquared as well as An AR Game (which won our getUserMedia Dev Derby) and WebRTC Experiments & Demos.

An example of simple video frame capture (which will capture new images at approximately 15fps):

navigator.getUserMedia({video: true, audio: false}, yes, no);
video.src = URL.createObjectURL(stream);
 
setInterval(function () {
  context.drawImage(video, 0,0, width,height);
  frames.push(context.getImageData(0,0, width,height));
}, 67);

Snippet of code taken from “Multi-person video chat” on nightly-gupshup (you can try it in the WebRTC Test Landing Page — full code is on GitHub)

function acceptCall(offer) {
    log("Incoming call with offer " + offer);
 
    navigator.mozGetUserMedia({video:true, audio:true}, function(stream) {
    document.getElementById("localvideo").mozSrcObject = stream;
    document.getElementById("localvideo").play();
    document.getElementById("localvideo").muted = true;
 
    var pc = new mozRTCPeerConnection();
    pc.addStream(stream);
 
    pc.onaddstream = function(obj) {
      document.getElementById("remotevideo").mozSrcObject = obj.stream;
      document.getElementById("remotevideo").play();
    };
 
    pc.setRemoteDescription(new mozRTCSessionDescription(JSON.parse(offer.offer)), function() {
      log("setRemoteDescription, creating answer");
      pc.createAnswer(function(answer) {
        pc.setLocalDescription(answer, function() {
          // Send answer to remote end.
          log("created Answer and setLocalDescription " + JSON.stringify(answer));
          peerc = pc;
          jQuery.post(
            "answer", {
              to: offer.from,
              from: offer.to,
              answer: JSON.stringify(answer)
            },
            function() { console.log("Answer sent!"); }
          ).error(error);
        }, error);
      }, error);
    }, error);
  }, error);
}
 
function initiateCall(user) {
    navigator.mozGetUserMedia({video:true, audio:true}, function(stream) {
    document.getElementById("localvideo").mozSrcObject = stream;
    document.getElementById("localvideo").play();
    document.getElementById("localvideo").muted = true;
 
    var pc = new mozRTCPeerConnection();
    pc.addStream(stream);
 
    pc.onaddstream = function(obj) {
      log("Got onaddstream of type " + obj.type);
      document.getElementById("remotevideo").mozSrcObject = obj.stream;
      document.getElementById("remotevideo").play();
    };
 
    pc.createOffer(function(offer) {
      log("Created offer" + JSON.stringify(offer));
      pc.setLocalDescription(offer, function() {
        // Send offer to remote end.
        log("setLocalDescription, sending to remote");
        peerc = pc;
        jQuery.post(
          "offer", {
            to: user,
            from: document.getElementById("user").innerHTML,
            offer: JSON.stringify(offer)
          },
          function() { console.log("Offer sent!"); }
        ).error(error);
      }, error);
    }, error);
  }, error);
}

Any code that runs on Desktop should run on Android. (Ah, the beauty of HTML5!) However, you may want to optimize for Android knowing that it could now be used on a smaller screen device and even rotated.

This is still a hard-hat area, especially for mobile. We’ve tested our Android support of 1:1 calling with a number of major WebRTC sites, including talky.io, apprtc.appspot.com, and codeshare.io.

Known issues

• Echo cancellation needs improvement; for calls we suggest a headset (Bug 916331)
• Occasionally there are audio/video sync issues or excessive audio delay. We already have a fix in Firefox 25 that will improve delay (Bug 884365).
• On some devices there are intermittent video-capture crashes; we’re actively investigating (Bug 902431).
• Lower-end devices or devices with poor connectivity may have problems decoding or sending higher-resolution video at good frame rates.

Please help us bring real-time communications to the web: build your apps, give us your feedback, report bugs, and help us test and develop. With your help, your ideas, and your enthusiasm, we will rock the web to a whole new level.

• lien n°1 : Le post original sur le blog de Michael Meeks
• lien n°2 : Le site de LibreOffice
• lien n°3 : Faire un don pour LibreOffice

Lire les commentaires