(Have a head cold today. Medicated but wanted to get some thoughts down inspired by Dylan's writings.)

Donald Trump is now President of the United States. As far fetched as this statement seemed 8 months ago (or even 3 months ago), this is reality today. I have been stunned and saddened by many of President Trumps actions. It's not so much that I differ from him greatly on many issues. It's more about the rise of incivility and the disrespect for people who are minorities / marginalized in society. 

The global Women's March inspired a lot of thinking about takeaways post the inauguration and the march.

My thoughtful friend Dylan Houlihan recently wrote on the things he would like to do to be even more welcoming in his own life. (It's great to see Dylan is blogging). I think he has written a really good list. But, I wonder if it's enough. Dylan concentrates his list mostly on what I could call progressive issues and causes. I also support these wholeheartedly. I also wonder if we need to include an appreciation for issues and causes more associated with conservatism. Issues like government regulation and taxation, free trade, and crime / safety?

Taxes is a good example here. As Dylan notes, citizen is a much more expansive (and better, in my opinion) term than taxpayer. But people are concerned about taxes and the ability to pay. People talk to me consistently about high taxes. This is a very authentic and legitimate concern. It doesn't at all mean that these folks don't care about making the community better. It will always be a balancing act.

We don't have to agree with someone to be empathetic. A lot of people who voted for Trump were people who felt left out / disengaged from the economic and political system. In my view, its important to listen across ideologies and viewpoints and to learn from each other. To be in relationship with as many people as possible. If we are truly try to break down some of the partisan divides, we need to embrace the diversity in our communities and countries. We need to do this in a respectful manner.

So, what I would likely add to Dylan's list is the desire to show empathy, learn, and dialogue with others who have very different views. 

Federico’s back, Stephen is the Mac mini Spokesperson and Myke needs to buy some stuff.

On this week's Connected, we also had some interesting discussions about home automation (which Myke likes) and sleep tracking (which Myke doesn't like – yet). You can listen here.

Sponsored by:

• Smile: The PDFpen Family of apps lets you take control of PDFs on all you devices.
• Blue Apron: A better way to cook. Get three free meals with free shipping.
• Squarespace: Make your next move. Enter offer code WORLD at checkout to get 10% off your first purchase.

→ Source: relay.fm

January 2017 has been an exceptionally cold and wet month in the Northern Hemisphere. The US has seen storms stir destruction from coast to coast and the cold wave across Europe dipped below minus-22 Fahrenheit in cities like Moscow.

While this damp and cold kick can get you down, we’d like to look on the brighter side of things. Flickr has put together a gallery of natural and cultural landmarks from around the world covered in this season’s snow. Travel around the world with us to celebrate the beauty of winter and the uniqueness of some of the world’s most popular historical sites. We’ve collected some gorgeous photos of European castles, American National Parks and Asian temples frosted with wintery vibes.

Washington Monument, Washington DC, US. – January 8th, 2017

Portland Head Light – Delano Park, Cape Elizabeth, Maine – December 30th, 2016

New Palace in Park Sanssouci – Potsdam, Germany – January 15th, 2017

Kurama Temple, Kurama-dera, Kyoto, Japan – January 14th, 2017

Neuschwanstein Castle, Southwest Bavaria, Germany – January 15th, 2017

The Great Wall of China at Mutianyu near Beijing – January 15th, 2017

Did you travel somewhere historic this winter and have a photo you’d like to add? Post it in the comments below the gallery using the format [FLICKR LINK] to display the photo with others from around the world.

A little birdie — okay, it was Sol-meister K. — tells me that CocoaConf Yosemite is close to selling out.

Don’t miss out! It’s so beautiful. This will be my third trip — because I miss it when I’m not there, and because the people are awesome.

Every single post is an opportunity to deepen engagement.

This isn’t the same as activity.

Activity is physical, engagement is mental.

Activity is easy to see and measure, engagement is in our minds and hard to prove.

These replies you’re going to write to members today, are you going to bring your A-game to deepen engagement with each member?

Will your response create better mutual understanding, build a bridge for stronger relationships in the future, and leave the member in an emotionally better state than before your response?

(That last one sounds obvious, but you might be surprised how many interactions leave a member feeling more peeved)?

Are you trying to understand and clarify their position before explaining your own? Are you showing empathy for their frustration? Are you using positive language and highlighting opportunities for the future?

Treat each interaction like the incredible opportunity it is. You can’t turn a furious member into a delighted supporter overnight, but perhaps you can make them a little less furious today?

An extra minute or two in each post can make a big difference.

Editor’s note: A version of this post first appeared on  Jing Daily the leading digital publication on luxury consumer trends in China. 

Luxury brands have long been releasing their one-off Zodiac-themed collections targeted at the Chinese market during the Spring Festival shopping season, but in many cases, the designs simply don’t click with consumers. This year, Chinese blogger and handbag guru Mr. Bags teamed up with Scotland-based label Strathberry to create a capsule collection for his followers that he knew they would love.

Mr. Bags, whose real name is Tao Liang, visited Strathberry’s factories in Spain to work with the artisans, ultimately coming up with an exclusive series of pink handbags, called “Sweet Pink Fantasy,” which feature interchangeable red leather bag tags with illustrated characters. Tao said his goal was to “create designs that can really appeal to youthful female shoppers, creating a whimsical dream come true.”

“I reached out to the brand, and we just sat down and talked about what kind of colors and bags Chinese consumers love, and then we came up with the idea that maybe we could work on a collaboration exclusively for my followers,” Tao said. “I figured that a lot of Chinese people love exclusive collections instead of normal, classic colors right now.”

Electric bikes are hitting their stride, with more people than ever riding electric bicycles throughout the world. If you're in the market for an eBike, you have many options to choose from. Yet in order to find the right electric bike, there are a number of things you should ask before making the final purchase. We've outlined some of the main things you need to ask.  

Ask if it’s Pedal Assist, Power on Demand or Speed Pedelecs

Pedal Assist electric bikes, sometimes referred to as “pedelecs,” are equipped with electric motors that provide additional power when the  bike is being pedaled.  A sensor detects when the bike is being used, and provides the selected level of assistance.

Power on demand allows riders to activate and control the motor with a throttle, button, or trigger located on the handlebars, regardless of whether or not they’re actually pedaling.

Some electric bikes combine pedal assist and a throttle, providing the best of both worlds.

Speed pedelecs are the fastest e-bikes, allowing riders to go up to 28 mph. However, their use on bike paths and trails may be restricted. In some jurisdictions, speed pedelecs may not be equipped with a throttle.

Ask about the battery

An electric bike’s battery is one of the most important components of the bike. Be sure to ask who the manufacturer of the battery is. Panasonic and Samsung make some great batteries for electric bikes. If you don’t hear a name you recognize, ask about the battery and make sure it is coming from a reputable brand that will offer support and replacements.

You also want to ask about the capacity of the battery. A very common rating is 36 volts and 10 Ah. On this type of battery rating, you can get 20 to 40 miles per charge, depending on your riding style and how much you pedal. That’s a very good baseline to start with.

When comparing e-bikes, you may want to compare battery specifications side by side, including cell manufacturer, voltage and amperage. This will provide an objective comparison as opposed to manufacturer’s subjective range claims.

Ask about the motor

When purchasing an electric bicycle, ask the vendor about the motor’s wattage rating. The number you typically see on your screen when you are doing internet research is the continuous wattage rating. For example; you might see 350W, that means that’s the wattage that the motor is able to put out on a continuous level for the duration of the charge. The peak wattage rating is what the motor is capable of on heavy load situations. Climbing a hill for example naturally draws more current from the battery. It is good to know both numbers, not just the continuous wattage number.

Does it fit?

Bikes are not really “one size fits all”, so you want to find a bike that matches how you want to ride it. Ask yourself whether you’d rather sit in a relaxed, upright posture or lean forward and focus on going fast. Look at where the handlebars are relative to the seat to get an idea how you will sit on it- the high they are, the more relaxed the riding posture.

Also, consider whether your prefer a traditional frame or a step-through frame. Traditional frames require you to swing your leg over the bike to get on and off. Step-through frames allow you to step right on and off the bike through the middle of the frame. Step-throughs were previously referred to as “ladies’ bikes”, but people of all genders can benefit, especially beginner riders, riders with limited leg mobility, and riders carrying significant amounts of cargo.

Ask about the bike components

Be sure to ask what kind of components are included with the electric bike, especially the shifting system and the braking systems. Reputable companies include Shimano Tektro, and SRAM. However, each manufacturer sells a wide range of components of varying quality and price, which also all depend on proper adjustment for smooth operation.

Test Ride!

Assuming you have dealers near you, this is the fun part! After some research has narrowed things down to a few appealing bikes, test ride them if you can. This will tell you a lot about the electric performance the bikes provide, how comfortable the fit is, the quality of the components, and how well they were assembled and adjusted.

In the previous post I’ve had a closer at, among other things, the spectrum use of ADSL vs. ADSL2+. The change in uplink and downlink throughput when the line was updated was quite significant. The main difference was made by using the lower spectrum that was previously used for voice telephony as additional spectrum for the DSL uplink and by doubling spectrum use from 1.1 to 2.2 MHz, i.e. from 256 tones (carriers) to 512. Let’s have a look at how ADSL2+ spectrum use compares with VDSL2, which I have at my home.

The screenshot on the left can be directly compared to the ADSL and ADSL2+ screenshots in the previous post. The upper yellow graph shows the signal to noise (SNR) ratio of the portions of the spectrum that are used for downlink data transmission. In the lower graph, the number of bits per carrier (tone) is shown. Blue are downlink channels, green are uplink channels. Compared to the ADSL charts in the previous post the following major difference become immediately apparent:

• Spectrum use was significantly expanded from 1.1/2.2 MHz (ADSL / ADSL2+) to 17.6 MHz. Instead of 512 tones, 4096 tones (carriers) are now used.
• To increase uplink data rates to 10 Mbit/s (in my example) several parts of the spectrum are used.

It’s also quite interesting to observe how the signal to noise ratio slowly gets worse as the frequency increases. One thing that would be interesting to know is why the SNR decreases so significantly between tone 256 and 512 and then suddenly significantly improves again. Does anyone with layer 1 expertise have an idea why this is so!?

Also interesting to see is how the number of bits per tone slowly decreases as a result of the worsening SNR.

One of the advantages of reading Hacker News regularly is being exposed to the blogs of pretty technical people. Naturally, they’re the kind of people who are likely to be the first to implement new technologies.

Recently, I came across a blog that had a pop-up from the address bar. It asked me if I’d like to turn on ‘push notifications’ for new posts. I’m used to Google Calendar, Slack, etc. asking for these kinds of permissions, but it was a first for a blog.

After a bit of investigation, it would seem that implementing this myself in a manual way would involve more than just a half-hour tinker. It was then that I came across PushCrew, a service that offers a WordPress plugin. Configuration couldn’t have been simpler.

For the last couple of weeks, visitors to this blog have seen the following notification:

So far, 29 people have opted-in. Given it’s likely the first time most visitors have seen this kind of thing, I’d expect these kinds of numbers.

Hopefully, this is a useful development for people. I’m happy to experiment with it for a while, and gain your feedback. It’s free for up to 500 subscribers, so it’s not costing me anything for the foreseeable future

To me, it’s a half-way house for people who, with the best will in the world, are never going to subscribe via RSS, don’t want blog post emails  cluttering up their inbox, and who might miss updates via social media. It’s also cross-platform, and built on web standards.

Let me know if you think this is useful (and if you’re thinking of adding it to your own blog!)

Image CC0 Frank McKenna

TL;DR

This blog post says the opposite of its lazy and deliberately provocative title. I have become a huge fan of ReactiveUI. I just want to ramble about the path I took to get here.

Listening to Paul Betts

I first heard about ReactiveUI at a conference presentation by Paul Betts. I think it was at Xamarin Evolve. Mostly I remember feeling dumb. He said a lot of things that I didn't understand.

I went to that session without much real experience in Model-View-ViewModel (MVVM) development. Conceptually, I understood the idea of a ViewModel. But Paul mostly talked about how ReactiveUI avoids certain problems. And since I had not experienced those problems, his words didn't sink in.

Talking to teenagers about risk

Each time one of my kids was approaching adolescence, I sat down and explained the risks associated with certain choices. Laws and moral judgements aside, the simple fact is that many choices involve risks, and I thought it would be helpful to pass along that bit of information.

And in each case, my child said, "Thanks Dad", and proceeded to always make wise and low-risk choices from that point on.

Well, actually, no.

Teenagers simply do not learn that way. They process risk very differently from people who are more mature. Tell a 16-year-old that "if you drive too fast you might get a ticket". The adolescent will immediately begin driving too fast, and, in all likelihood, will not get ticket. This is how teenagers realize they are smarter than their parents.

Tangent #1: It is almost certainly a good thing that young people are more brave. It would be Very Bad if everybody started out with the same level of risk aversion as the average 65-year-old. Go watch the "Tapestry" episode of Star Trek TNG.

Tangent #2: I really should claim no expertise in parenting, but if somebody forced me to write a book on parenting a teenager, I would say this: Let your kid suffer from their own choices. That said, it is worth the effort to try and help them avoid the really bad mistakes, the ones with consequences that last for decades. But they do have to learn to make their own choices. Realize this as early as you can. The path to frustration starts with making everything all about you.

How we learn new technologies

My metaphor has many problems. For starters, Paul Betts is not my Dad.

Also, the element of adolescent rebellion was not present. I didn't hear Paul's wisdom and run in the opposite direction because of my deep need to separate my identity from his. In fact, I started devouring everything I could find on MVVM and IObservable. I really wanted to understand what he was saying.

But the metaphor works in one significant way: Like a teenager, I had to learn by doing. Nobody's words made much of a difference. None of that reading helped me become a a user of ReactiveUI. I went down another path.

Actually, I went down several other paths.

Maybe it's just me

I observe that most developers want content that explains how to get something done. "If your objective is to do X, then do the following steps." The most popular books and articles tend to follow this pattern. Questions of this form are the ones that do well on StackOverflow.

But this is almost never what I want.

I much prefer content that explains how things work. Once I understand that, I can figure out the steps myself.

When I am developing software, I always, ALWAYS do better when I understand what is going on "under hood", when I can see through the leaky abstractions.

And as I mentioned, I am apparently in the very small minority on this. If 90% of the world disagrees with me, does that put me in the top 10% ? Or does it mean my approach is somehow defective? Modesty aside, my history contains enough successes to allow me some confidence in believing that my approach is better.

I also observe that my approach is just a different spelling for the old adage, "Give a man a fish and you feed him for a day. Teach a man to fish and he eats for a lifetime."

If you tell a software developer what to type and where to click, you can help them complete today's task. But if you instead teach them how things work, they will be able to apply that understanding on other days too.

Hmmm. I'm talking myself into this. I don't know why most people prefer shallow recipes, but I really do think deep understanding is better.

Still, I like to stay open-minded about things. I've got a lot of failures too.

The truth is that my approach has tradeoffs. The need to understand everything tends to slow me down during the early stages. I usually gain some of that back in the fourth quarter of the game, where deeper understanding is often helpful in diagnosing tricky problems.

But again, in the decision making around software development, absolutes are rare. I'll admit that sometimes a simple set of steps without depth are exactly what is needed.

Maybe the ReactiveUI docs are just bad?

I don't know. Maybe. I've read the docs plenty. They don't seem bad to me. I also see nothing there that makes me want to defend them as the best docs ever.

Suppose that I regret not choosing ReactiveUI sooner. Further suppose that I wanted to blame somebody else for my choices. I guess I could find something to complain about. But I also don't tend to find that criticizing somebody else's work is helpful.

And remember, I started this journey sitting in front of an audience, listening to Paul Betts, and feeling dumb. To be clear, in that kind of context, I *like* feeling dumb. It's an opporunity to learn.

So why did I not choose ReactiveUI sooner?

I guess I don't really know. But I'm pretty sure that nothing has made me appreciate ReactiveUI more than the suffering that comes from not using it.

And that remark isn't very helpful, is it? I'd like to try and do better. Let's see...

"Son, it's just basic statistics. If you're going to always drive 15 MPH over the speed limit, you will eventually get caught. Suppose you roll the dice 20 times in a row without getting a 12. You still might get a 12 on the next roll, right?"

Oh, wait, wrong topic. Let me try again.

Why is ReactiveUI awesome?

In some software development situations, like mobile apps, if you take a step back and look at the forest instead of the trees, you will see that most of your code is reacting to something that changed.

There are lots of tools you can use to approach this kind of app. You can use C# events and callbacks and switch statements and delegates and lambdas and observables and notifications and bindings and more.

For simple apps, none of these approaches are much better than any other. But as your app gets more complicated, some approaches cope more gracefully than others.

Most cars drive pretty smooth at 30 MPH. But at 75 MPH, some vehicles are still giving a smooth ride, while others are shaking.

Let's try a conceptual example or two. Suppose you have a button, and you want something to happen when the user presses that button. This is pretty simple. All reasonable solutions to this problem are about the same.

On the other hand, let's say you have a list of items. The items in that list come from a SQL query. That query has 4 inputs, each of which comes from a UI control. Every time one of those controls changes its value, the query needs to be re-run and the list needs to be updated. A couple of those controls need to be disabled under certain circumstances.

These UI elements have a complicated relationship. We still have plenty of choices in how to express that relationship in code, but this situation is complicated enough that we start to see differences between those approaches. Some of the ones that worked out really well in the simple case seem kinda tedious for this case.

If my driveway has half an inch of snow, all methods of clearing it are about the same. But if my driveway has 15 inches of snow, a shovel is decidedly inferior to a tractor.

Why do I like ReactiveUI? Because I have found that it copes gracefully as the situation gets more complicated.

Why is this? Much of the credit goes to the "reactive" foundation on which ReactiveUI is built. Reactive Extensions. Rx. IObservable. These building blocks are particularly adept at expressing the relationship between a group of things that are changing. ReactiveUI adds another layer (or two) on top of these things to make that expressiveness more convenient when implementing user interfaces.

To be honest, I fudged a little bit when I said that all solutions are roughly equivalent when the problem is simple. That's not quite true. For simple situations, I'd have to admit that ReactiveUI might be a little worse. There is a learning curve.

If I am writing a grocery list, I could use a word processor, but a pencil and paper is actually simpler. But if I am writing a novel, the word processor is the clear winner.

I'm claiming that the effort to learn Rx and ReactiveUI is worth the trouble. My claim is based on this notion that ReactiveUI shines as complexity increases, but also on my belief that most people underestimate the complexity of their app.

If you disagreed with me above when I said that "most of your code is reacting to something that changed", you might be underestimating the complexity of your app. It is in fact very common to start implementing under the assumption that something will not change and then later realize that you need notifications or events. Or an observable.

Hmmmm.

Would the paragraphs above have changed my course earlier?

I don't know. Probably not.

I didn't start this believing that I could write the best ReactiveUI advocacy ever. Looking at it now, I can't believe I wrote it with no code in it. The canonical ReactiveUI evangelism pamphlet has gotta have WhenAnyValue() in it somewhere.

I just think it's interesting that despite my best efforts, I was unable to really understand the benefits of ReactiveUI until I tried using its alternatives. My current project is late. If I had chosen ReactiveUI earlier, maybe it would be, er, less late? There are questions here worth asking.

But am I 100% certain that it is always better to spare yourself the learning experience of using less effective approaches? No.

Can I credibly claim that everyone should choose ReactiveUI in every situation? Certainly not.

Maybe all I can say is that I am currently having a great experience with ReactiveUI.

Maybe that means the rest of this blog post is useless.

But you should have known that when you saw the cheesy title.

 

I’ve been following the data thing for a few years now, and it’s been interesting to see how data related roles have evolved over that time.

For my own part, I’m really excited to have got the chance to work with the Parliamentary Digital Service [PDS] [blog] for a few days this coming year. Over the next few weeks, I hope to be starting to nose around Parliament and the Parliamentary libraries getting a feel for the The Life of Data there, as well as getting in touch with users of Parliamentary data more widely (if you are one, or aspire to be one, say hello in the commentsto see if we can start to generate more opportunities for coffee…:-)

I’m also keen to see what the Bureau of Investigative Journalism’s Local Data Lab, headed up by Megan Lucero, starts to get up to. There’s still a chance to apply for starting role there if you’re a “a journalist who uses computational method to find stories, an investigative or local journalist who regularly uses data, a tech or computer science person who is interested in local journalism or a civic tech person keen to get involved”, and the gig looks like it could be a fun one:

• We will take on datasets that have yet to be broken down to a local level, investigate and reveal stories not yet told and bring this to local journalists.
• We will be mobile, agile and innovative. The team will travel around the country to hear the ideas and challenges of regional reporters. We will listen, respond and learn so to provide evidence-based solutions.
• We will participate in all parts of the process. Every member will contribute to story ideas, data wrangling, problem solving, building and storytelling.
• We will participate in open journalism. We will publish public interest stories that throw light on local and national issues. We will open our data and code and document our shortcomings and learnings. We will push for greater transparency. We will foster collaboration between reporters and put power into regional journalism.

I’m really hoping they start a fellowship model too, so I can find some way of getting involved and maybe try to scale some of the data wrangling I will be doing around Isle of Wight data this year to wider use. (I wonder if they’d be interested in a slackbot/datawire experiment or two?!) After all, having split the data out for one local area, it’s often trivial to change the area code and do the same for another:

(It’ll also be interesting to see how the Local Data Lab might complement things like the BBC Local Reporting Scheme,  or feed leads into the C4CJ led “representative network for community journalism”.)

Data journalism job ads are still appearing, too. A recent call for a Senior Broadcast Journalist (Data), BBC Look North suggests the applicant should be able:

• To generate ideas for data-driven stories and for how they might be developed and visualized.
• To explore those ideas using statistical tools – and present them to wider stakeholders from a non-statistical background.
• To report on and analyse data in a way that contributes to telling compelling stories on an array of news platforms.
• To collaborate with reporters, editors, designers and developers to bring those stories to publication.
• To use statistical tools to identify significant data trends.

The ad suggests that required skills include good knowledge of Microsoft Excel, a strong grasp of how to clean, parse and query data as well as database management*, [and] demonstrable experience of visualising data and using visualisation tools such as SPSS, SAS, Tableau, Refine and Fusion Tables.

* I’m intrigued as to what this might mean. As an entry level, I like to think this is getting data into something like SQLite and then running SQL queries over it? It’s also worth remembering that Google Sheets also exposes an SQL like interface that you can query (example, about).

When I started pottering around in the muddy shores of “data journalism” as it became a thing, Google Sheets, Fusion Tables and Open (then Google) Refine were the tools I tried to promote because I saw them as a relatively easy way in to working with data. But particularly with the advent of accessible working environments like RStudio and Jupyter notebooks, I have moved very much more towards the code side. This is perceived as a much harder sell – it requires learning to code – but it’s amazing what you can do with a single line of code, and in many cases someone has already written that line, so all you have to do is copy it; environments like Jupyter notebooks also provide a nicer (simpler) environment for trying out code than scary IDEs (even the acronym is impenetrable;-). As a consequence of spending more time in code, it’s also made me think far more about reproducible and transparent research (indeed, “reproducible data journalism”), as well as the idea of literate programming, where code, text and, particularly in research workflows, code outputs, together form a (linear) narrative that make it easier to see and understand what’s going on…

As well as the data journalism front, I’ve also kept half an eye on how academic libraries have been engaging with data issues, particularly from an “IT skills” development perspective. Generally, they haven’t, although libraries are often tasked with supporting research data management projects, as this job ad posted recently by the University of Michigan (via @lorcanD) for a data workflows specialist shows:

The Research Data Workflows Specialist will advance the library’s mission to create and sustain data services for the c­ampus that support the mission of the University of Michigan researchers through Research Data Services (RDS), a new and growing initiative that will build the next generation of data curation systems. A key focus of this position will be to understand and connect with the various disciplinary workflows on campus in order to inform the development of our technical infrastructure and data services.

I suspect this is very much associated with research data management. It seems to me that there’s still a hole when it comes to helping people put together their own reproducible research toolchains and technology stacks together (as well as working out what sort of toolchain/stack are actually required…).

Finally, I note that NotWestminster is back later next month in Huddersfield (I managed to grab a ticket last night). I have no idea what to expect from the event, but it may generate a few ideas for what I can usefully do with Island data this year…

PS just spotted another job opportunity in a related opendata area: Data Labs and Learning Manager, 360 Giving.

Securing modern web applications effectively is a complex process. However there are many straightforward security controls such as HTTP security headers which are very effective at blocking web common attacks.

At Mozilla we provide Security Guidelines as well as a Checklist of security controls for the developers of Firefox Services. Last year, we introduced the Mozilla Observatory, a hosted scanner to evaluate the security of websites and services. In this blog post, we present the ZAP Baseline scan designed to test the security controls of web applications in Continuous Integration and Continuous Deployment (CI/CD).

Verifying that the correct controls are in place across all of our applications can be challenging, especially in a CI/CD environment. We run full vulnerability scans against our services on a regular basis, but these can take a long time to run and are not really adapted to fast release cycles.

This is why we have introduced a ‘baseline’ scan which runs very quickly and on every release of a service, but still gives us crucial feedback about the key security controls that we are concerned about. The baseline scans are included in the CI/CD pipelines of Firefox services to inform developers of potential issues before they reach production environments. We also run it against all of those services every day to generate a dashboard of the overall security status of our services.

This blog post presents the techniques we use to implement baseline scans in our infrastructure.

The ZAP Baseline Scan

Mozilla invests heavily in the development and support of security tools. The author of this blog post leads the OWASP Zed Attack Proxy (ZAP) project,  which we use to run baseline and vulnerability scans.

The ZAP baseline scan is a quick, easy and highly configurable way to test the security controls you care about. The tests are non intrusive so they are safe to run against production applications.
You don’t need to have ZAP installed – the zap-baseline.py script uses Docker and is included in the 2 ZAP Docker images:

• https://hub.docker.com/r/owasp/zap2docker-stable/
• https://hub.docker.com/r/owasp/zap2docker-weekly/

For our baseline scans we use the weekly docker image which has more options available – you can run the script with the -h flag to see all of them.

The script zap-baseline.py uses the ZAP spider to explore the application, by default for just one minute. Spidering the application is important to verify that all pages, and not only the top one, implement the required security controls. This is particularly useful when web frameworks will handle some of the pages automatically without setting the headers.
The script will then report all of the potential issues found.

The baseline scan can be run against an application by just specifying its URL using the -t flag:

docker run owasp/zap2docker-weekly zap-baseline.py -t https://www.example.com

This will produce output like: 

Total of 3 URLs
PASS: Cookie No HttpOnly Flag [10010]
PASS: Cookie Without Secure Flag [10011]
PASS: Password Autocomplete in Browser [10012]
PASS: Cross-Domain JavaScript Source File Inclusion [10017]
PASS: Content-Type Header Missing [10019]
PASS: Information Disclosure - Debug Error Messages [10023]
PASS: Information Disclosure - Sensitive Informations in URL [10024]
PASS: Information Disclosure - Sensitive Information in HTTP Referrer Header [10025]
PASS: HTTP Parameter Override [10026]
PASS: Information Disclosure - Suspicious Comments [10027]
PASS: Viewstate Scanner [10032]
PASS: Secure Pages Include Mixed Content [10040]
PASS: Weak Authentication Method [10105]
PASS: Absence of Anti-CSRF Tokens [10202]
PASS: Private IP Disclosure [2]
PASS: Session ID in URL Rewrite [3]
PASS: Script Passive Scan Rules [50001]
PASS: Insecure JSF ViewState [90001]
PASS: Charset Mismatch [90011]
PASS: Application Error Disclosure [90022]
PASS: WSDL File Passive Scanner [90030]
PASS: Loosely Scoped Cookie [90033]
WARN: Incomplete or No Cache-control and Pragma HTTP Header Set [10015] x 1
    https://www.example.com
WARN: Web Browser XSS Protection Not Enabled [10016] x 3
    https://www.example.com
    https://www.example.com/robots.txt
    https://www.example.com/sitemap.xml
WARN: X-Frame-Options Header Not Set [10020] x 1
    https://www.example.com
WARN: X-Content-Type-Options Header Missing [10021] x 1
    https://www.example.com
FAIL-NEW: 0    FAIL-INPROG: 0    WARN-NEW: 4    WARN-INPROG: 0    INFO: 0    IGNORE: 0    PASS: 22

By default the output lists all of the passive scan rules applied and whether they passed or failed.

You can change how the baseline handles different errors by specifying a rule configuration file via either the -c flag (for a local file) or the -u flag for a remote URL.
You can also generate a default file using the ‘g’ option: https://github.com/zaproxy/community-scripts/blob/master/api/mass-baseline/mass-baseline-default.conf
As specified in the generated file header you can change any of the “WARN”s to “IGNORE” or “FAIL”.

The script will exit with a 0 if there are no issues, 1 if there are any failures or 2 if there are just warnings. The return value can therefore be used in CI tools like Jenkins, CircleCI, TravisCI, etc. to fail a build step.

For example, the configuration below shows how the baseline scan can run in CircleCI with every pull request:

test:
  override:
    # build and run an application container
    - docker build -t myrepo/myapp
    - docker run myrepo/myapp &
    # retrieve the ZAP container
    - docker pull owasp/zap2docker-weekly
    # run the baseline scan against the application
    - |
      docker run -t owasp/zap2docker-weekly zap-baseline.py \
      -t http://172.17.0.2:8080/ -m 3 -i

Scanning Multiple Sites

The baseline scan is a great way to check that a single site meets your base security requirements.

In order to run the ZAP Baseline scan against a large number of websites, we have written a set of wrapper scripts specific to Mozilla. You can find  generic versions of these scripts in the ZAP community-scripts repository.

You will need to customize these scripts as detailed in the README:

• Change the sites listed in mass-baseline.sh
• Change the relevant user and repo details in mass-baseline.sh
• Build a docker image
• Run the docker image, setting the credentials for your user

These scripts will then generate a summary dashboard in your repo wiki:

The ‘Status’ badge is a link to a page containing the latest baseline results for the relevant application and the ‘History’ date links to a page which show all of the previous scans.
Example pages are included on the community scripts wiki: https://github.com/zaproxy/community-scripts/wiki/Baseline-Summary

Tuning

The baseline scan is highly configurable and allows you to fine tune the scanning to handle your applications more effectively.
You can do things like:

• Increase the time spent spidering your application
• Use the Ajax Spider in addition to the standard ZAP spider to handle applications that make heavy use of JavaScript
• Include alpha passive scan rules as well as the beta and release quality ones used by default
• Ignore specific URLs or even ignore specific issues on those pages
• Link known issues to a bugtracker URL
• Specify any of the options supported on the ZAP command line

For more details see the ZAP wiki: https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan

Conclusion

The baseline scan gives us immediate feedback about the security controls in place across all of our web applications. The scans run on every commit so that we are immediately aware if there has been any regression. The dashboard allows us to track the state of our applications and the CI integration provides the ability to block a deployment if the baseline is not met.

Integrating baseline scanning in CI/CD helps us work more closely with developers and operators. We don’t force our security tools onto DevOps processes, we integrate security into DevOps. The net effect is better collaboration between teams, and faster turnaround on fixing security issues.

The post Setting a Baseline for Web Security Controls appeared first on Mozilla Security Blog.

Each year, Ken Case of The Omni Group takes a look back at the past year and ahead to what’s coming next for the company’s products. It was a good year for Omni, which released a major update of OmniGraffle for Mac as well as updates to OmniFocus for Mac and iOS and OmniPlan for iOS. The year concluded with OmniGraffle for Mac and OmniPlan for iOS making Apple’s ‘Best of 2016’ list.

In addition to planned updates to OmniGraffle for iOS, OmniOutliner for Mac, and OmniFocus, Ken Case outlined ambitious plans to bring new automation features to its iOS apps with some help from Sal Soghoian, who has been reviewing Omni’s efforts:

In 2016 we scratched the surface with URL automation on iOS, but in 2017 we plan to roll out user automation on iOS in a big way across all our apps with a much richer set of capabilities. This automation support won’t be limited to a simple set of URL primitives; instead, we’re adding support for running JavaScript code: code that has the same level of deep support for manipulating the data in our apps as we’ve previously exposed to AppleScript.

…

We’re also adding support for background scripts which can automatically respond to document edits.

Omni is going further with URL schemes too:

Oh, and did I mention that we’re including support for calling out to other apps by their URL handler? So you’ll be able to tie into the Workflow app and its already great ecosystem of automation. In OmniFocus, imagine the possibilities that open up when you can trigger a workflow just by checking something off!

The prospect of desktop-class automation in The Omni Group’s iOS apps is exciting and right in line with its original vision for the iPad.

→ Source: omnigroup.com

Best use of the atrium space at Pacific Centre I’ve seen:

This feels genuinely Vancouver.

For all of us, Chinese Lunar New Year seems like almost-spring festival – a time when it feels about right to think of renewal and plans for the future.

Gung Hay Fat Choy …

When you need to turn your Mongo database into a RESTFul API, RESTHeart can get you up-and-running quickly. In this article, we'll explore using RESTHeart to expose a RESTFUL API directly from a Mongo database on Compose.

Setup

Before we get started, we'll need to set up a few basic services. You can create a new MongoDB deployment on Compose by following the rough guide to composing. You'll also need to install Docker for your platform. If you're using macOS or Windows, the best way to get Docker is by installing the Docker Toolbox. On Linux, all major distributions have a package available in their package management systems.

Once you have a new MongoDB deployment on Compose, you'll need to add a database to your deployment to serve your RestHEART API. Create a new database called restheart through the MongoDB browser.

Then, create a user for the restheart database by clicking on the database in the Compose browser, selecting the Users menu and clicking the Add User button.

Running RESTHeart in Docker

RESTHeart can be run in a number of ways, but the easiest is to get use the Docker container. You can get started by pulling the RESTHeart container from the Docker Hub central container repository using the docker pull command:

docker pull softinstigate/restheart  

RestHEART uses YAML files to store configuration information so we'll need to create one to point to our new Mongo database. The default configuration file is stored in a file called restheart.yml in the /opt/restheart/etc directory of the container and assumes that you're running MongoDB in another docker container on your local machine with the name of mongodb. We'll need to update the configuration to point to our Compose MongoDB deployment. The easiest way to update those is to mount our own configuration in place of the default by using Docker Volumes.

First, let's create a new directory for our RESTHeart project. Then, create a directory within it called etc.

mkdir restheart  
cd restheart  
mkdir etc  
cd etc  

Next, copy the sample configuration file into the etc folder in a new file called restheart.yml. There are a few key differences between our sample file and the default configuration file in the docker container. Namely, we're going to point our database to Mongo on compose rather than a local Mongo URI. Search the restheart.yml file for the mongo-uri key and replace the value with the MongoDB connection string from Compose:

mongo-uri: mongodb://dbuser:secret@aws-us-east-1-portal.8.dblayer.com:15234,aws-us-east-1-portal.7.dblayer.com:15234/?authSource=restheart  

The ?authSource at the end of the URI is important: it allows us to connect to MongoDB with the user credentials from our newly created database (in this case, our restheart database). This also has the effect of scoping RestHEART so it only has access to a single database that we allow. While you can configure RESTHeart to have root access to your database, it's not recommended.

Next, we'll need to run the Docker container and tell RestHEART to use our updated configuration file. Since we can't pass in the configuration file to the docker container, we'll instead use the volume mounting capabilities of Docker to mount our configuration file (or, in our case, the entire configuration folder) in the same location as the default one. RESTHeart will automatically load our new configuration file from this location. We'll do this by running the container with a local read-only volume pointing to our new etc folder using the following command:

docker run -d -p 80:8080 --name restheart -v $PWD/etc:/opt/restheart/etc:ro softinstigate/restheart:3.0.0  

Note that this command assumes you're in root directory of your project. You can reference the container later using the name restheart. For example, to stop the container, use the following command:

docker stop restheart  

Once the container is running, you can access it by navigating to http://localhost/browser in your web browser. RESTHeart comes pre-installed with the HAL data browser, which makes verifying the installation easier.

Basic Operations

Once we have a running container with RestHEART, we can now test our new RESTful API by making an HTTP GET call to localhost.

$ curl http://localhost
{"http status code":403,"http status description":"Forbidden","message":"The MongoDB user does not have enough permissions to execute this operation."}

Initially, you will get a response with a 403 Forbidden status code. This is the desired result right now - it means that we have not given RESTHeart credentials that work on the root of our database deployment. Since we want to scope access to a specific database, we'll include the database name at the end of the URI.

$ curl http://localhost/restheart
{"_embedded":[],"_id":"restheart","_size":0,"_total_pages":0,"_returned":0}

You should now get a message back indicating that the query was executed and that there were no results. That's because our database is currently empty. Let's try adding some data to a collection in the restheart database.

Adding a Collection

Now that we have RESTHeart running successfully, let's put a document in our restheart database. RESTHeart uses the POST HTTP method to create new documents in the database. Let's try this out by creating a movies collection and adding some of our favorite movies to it.

First, we'll create the movies collection inside our restheart database. All operations in RESTHeart are based on RESTful API calls, so creating a new document, collection, or even new databases will all consist of making HTTP calls. RESTHeart exposes the interface for adding new collections through the PUT method on the collection:

$ curl -i -H "Content-Type: application/json" -X PUT http://localhost/restheart/movies -d '{"desc": "These are some of my favorite movies"}'

RESTHeart expects to receive data in JSON format, so we first set the Content-Type to application/json. Then, we specify the PUT method and send along the collection we wish to add. Note that the URL format for our RESTHeart instance takes the following form:

http://localhost/<database>/<collection>

where <database> is the name of the database you want to operate on and <collection> is the name of the collection.

This structure can be used to retrieve data from specific locations as well. For example, to retrieve all of the collections in our restheart database, let's make a GET call to the database URL:

$ curl http://localhost/restheart
{"_embedded":[{"_id":"movies","desc":"These are some of my favorite movies","_etag":{"$oid":"588734a946e0fb000a6e8827"}}],"_id":"restheart","_size":1,"_total_pages":1,"_returned":1}

Notice how this output is different from the first time we make this call. The newly added movies collection can now be seen in the list of collections.

Using POST to Create a New Document

Following this logic, we can add a new movie to our movies collection by using the same URL scheme. To create a new document in RESTHeart, we'll send an HTTP POST call to RESTHeart and specify which collection to put the data in by using the URL. In our case, the URL looks like the following:

http://localhost/restheart/movies  

So if we wanted to add "Gone with the Wind" to our movies collection, the request would look something like the following:

$ curl -i -H "Content-Type: application/json" -X POST http://localhost/restheart/movies -d '{"title":"Gone With the Wind", "year": "MongoDB"}'

We can test that our insertion worked by sending a GET request to the same URL to list all of the documents in a collection:

$ curl http://localhost/restheart/movies
{"_embedded":[{"_id":{"$oid":"5887875ed4cf043f4cdcdf8b"},"_etag":{"$oid":"588736fc46e0fb000a6e8829"},"title":"Gone With the Wind","year":"MongoDB"}],"_id":"movies","desc":"These are some of my favorite movies","_etag":{"$oid":"588734a946e0fb000a6e8827"},"_returned":1}

Using PUT to Update a Document

Updating a document is similar to inserting a new document, and adds one more layer of specificity by adding the document ID to the URL. To update our favorite movie with a record of Academy Awards, we can use the following:

$ curl -i -H "Content-Type: application/json" -X PUT http://localhost/restheart/movies/5887875ed4cf043f4cdcdf8b -d '{"academy_awards": 10}'

You can also update all the documents in the entire collection by omitting the document id from your URL:

$ curl -i -H "Content-Type: application/json" -X PUT http://localhost/restheart/movies/ -d '{"visible": true}'

Do a GET request again to see the latest changes:

$ curl http://localhost/restheart/movies
{"_embedded":[{"_id":{"$oid":"5887875ed4cf043f4cdcdf8b"},"academy_awards":12,"_etag":{"$oid":"5887387d46e0fb000a6e882b"}}],"_id":"movies","visible":true,"_etag":{"$oid":"588738fd46e0fb000a6e882c"},"_returned":1}

More Operations

RESTHeart has a comprehensive API, but this should get you started with the basics. You can try out more API calls using the CURL utility or gain an in-depth understanding of the RESTHeart API by browsing through the latest documentation.

Conclusion

RESTHeart can be a great way to quickly expose your Mongo database directly as RESTFul API resources. Since the recommended configuration for RESTHeart uses docker containers, it can be especially convenient for teams that use Docker for deploying their applications.

As with any API, you should never expose a running instance of RESTHeart to the open Internet without strong authentication and end-to-end encryption. RESTHeart comes with several ways to configure HTTPS and authentication, and in a future article we'll show you how to secure your RESTHeart API for production. In the meantime, you can find more information on Configuring and Enabling Security in RESTHeart in the official documentation.

---

If you have any feedback about this or any other Compose article, drop the Compose Articles team a line at articles@compose.com. We're happy to hear from you.

It’s a big day in Google Voice land.

Over 3.5 million people use this brilliant, useful, free Google (GOOG, GOOGL) service—and everybody else is missing out.

And yet it’s no wonder most people haven’t heard of Google Voice; its mother ship has been spectacularly negligent. Google bought the service in 2007, released it as a Google product in 2009—and basically ignored it ever since. Most fans feared that Google planned to kill off Google Voice.

But they’re not. Today, in fact, Google has begun rolling out a new version of Google Voice (for Android, iOS, and web). It’s been attractively redesigned, and it adds two crucial features it’s always lacked.

Before we get to what Google Voice has become, though, you might not even know what Google Voice is.

Here’s what I wrote in my New York Times review in 2010:

You pick out a new phone number. Anytime your friends call that number, all your phones ring at once: cell, home, work. One number to rule them all. People don’t have to chase you down anymore. (“I left a message for you on your cell—I’ll try you at home…”)

And now you have a single voicemail checkpoint: a private Web page, accessible from anywhere. Better yet, your voicemails are transcribed into text, which get sent to you by e-mail and/or text messages.

Then there are all the cool control features: “Don’t ring this phone on weekends.” Record a different voicemail greeting for each caller in your Address Book. And neither you nor your callers ever, ever hear 15 seconds of prerecorded instructions on using voicemail (“When you have finished recording, you make hang up… to page this person, press 5….”).

And all of it’s free. No ads, no fees, nothing.

And by the way: You can also place free phone calls from the Google Voice app. To any phone number in America.

What you couldn’t do—at least until today’s update, at least without involving Google hangouts—is receive photos sent in text messages. It was only text. And you also couldn’t send group texts.

Now you can. At long last, people can send photos to your Google Voice number, and you get it on all of your phones.

Google says that its days of Google Voice neglect are over. More features are on the way—including RCS (Rich Communication Suite), which is the next generation text-message format after SMS. (RCS eliminates the 160-character length limit and makes it easier to include photos and videos in your texts.)

It’s a great day for Google Voice. And if you haven’t yet tried GV yourself, then this is a great day for you to do so.

David Pogue, tech columnist for Yahoo Finance, welcomes non-toxic comments in the Comments below. On the Web, he’s davidpogue.com. On Twitter, he’s @pogue. On email, he’s poguester@yahoo.com. You can read all his articles here, or you can sign up to get his columns by email (http://j.mp/P4Qgnh). 

Nice change to Apple.com currently propagating across various international versions. Here's a comparison of the same webpage before and after the switch to San Francisco. I've never been a fan of the mixture of San Francisco and Myriad on the Apple website, and I dig the refreshed look. Looks like the transition to San Francisco is finally complete.

→ Source: twitter.com

Today’s photo investigation.

The big “story” now is that the Women’s March left a big mess and that’s awful, and they should have cleaned it up. Here’s the image — it’s shocking!

Well, this is almost too easy. There’s two ways to do this. If you search Snopes for the term “Women’s March signs snopes” you’ll find an article that debunks the right-hand photo, at least partially:

What the Snopes article says about that righthand photo is it is signs left by the Women’s March, but these particular signs were left at the Trump International Hotel in D.C. as part of the protest. That’s why they are clustered together like that. Someone does have to clean them up, but it wasn’t routine littering. Additionally, the Parks Service has remarked the protest was tidier than previous events. While the Snopes article gives no single word ruling, their presentation is close to what they usually call “Mixed” — partially true, but misleading in presentation.

Speaking of cleaning up, what about the photo on the left-hand side:

Well, you see that “alamy” watermark by the guy’s waist? I’m guessing this is stock photography. And stock photography, in general, is not released after the day of an event, so I’m thinking this was taken long ago.

We’d like to right-click the photo and search by image, but my guess is that the two images pushed together won’t match anything. So let’s use the snipping tool.

Windows: Call up the “snipping tool“:

Use it to capture the piece of the photo you want to search for. Save it to somewhere you’ll remember:

Mac: Hold the “Command, Shift and 4” together and then select what you want to do a screen shot of. Save it to somewhere you’ll remember.

Now go to Google search and upload it, the way we’ve done with past photos.

Any of these results is probably good to click on, but I pick door number three, partially because it is so specific.

And when we do that, we have good luck. We get to the stock photo purchase page, where there is a full description:

We even get the date and location. It was shot seven years ago. So no, this was not from the march.

And… we’re done. Fake-a-rooni.

The semester is well under way now, and the deluge of support over the last two days has been ample evidence of that. But we added a bit to our plates this months when we decided to decommission two shared hosting servers, namely Ramones (which has been going strong for over 3 years!) and Saints. We’ve been slowly migrating our infrastructure from Reliable Site over to Digital Ocean, and this month saw our first two shared hosting servers on Digital Ocean.* Once they were up and running smoothly, we decided to migrate all remaining accounts on Ramones and Saints over. That’s now done, and while these moves always require some clean-up post facto, everything’s over cleanly and all of our servers are now less than 3 years old.

The two new shared hosting servers we’re running on Digital Ocean have been named in honor of two ground breaking punk bands from the 80s: Minor Threat and L7. We were a bit hesitant to name a web server “minorthreat” given it might be immediately flagged as….well, a threat, so we opted to name it after their groundbreaking 1983 Straight Edge hardcore album Out of Step. This server is also dedicated to Peter Rowan who has been regularly hounding Reclaim about out glaring oversight of memorializing Minor Threat’s contributions to hardcore punk in the form of a shared hosting server. I couldn’t agree with him more. In fact, Ian MacKaye’s career between Minor Threat, Fugazi, and Dischord Records may be the most impressive of just about any punk rock figure. And all the while he embodied a vocal insistence on DIY, affordable shows, all ages access, experimentation, and a socially responsible ethos. He has walked the walk his entire career, and this short-lived band galvanized an entire sub-genre of punk that I grew up with in NYC. Straight Edge hardcore bands like Youth of Today, Judge, Uniform Choice, Gorilla Biscuits, Slapshot and many more all owe some debt to Minor Threat. And while MacKaye has always been ambivalent about the idea of straight edge as a doctrine or a movement, there is no question Minor Threat articulated the vision quite early with songs like “Out of Step,” “Straight Edge,” and “Bottled Violence.”  So, if you find yourself on the Outofstep server, draw a big black X on your hand and refrain from all sex, drugs, and rock and roll

The second new shared hosting server is another ground breaking 80s punk band L7. Their in-your-face punk rock coupled with an aggressive attack on patriarchy made them a pre-cursor and influence for riot grrrl bands like Bikini Kill, Bratmobile, and Huggy Bear to name a few. L7 songs like Shitlist, Shove, Wargasm, and Andres epitomize their hard, grinding style that helped define grunge rock in the early 90s. At the same time their songs were irreverent and their shows often filled with controversial surprises. The band formed Rock for Choice in 1991 to help support the Pro-Choice movement. Their songs are as fun and raucous as they are serious and political, a hard balance to master. That said, L7 is not for squares…get it? L7, squares….

So, that’s our two latest servers, and they are filling up fast, so we have a third in the works. It’s good to be a Reclaimer, just don’t make our shit list!

---

*We have finally been able to setup shared hosting on Digital Ocean thanks to the relatively recent addition of block storage.

If you’re having trouble taking advantage of Netflix’s new offline streaming feature due to limited phone storage, the company is coming to your rescue — at least for Android users.

Netflix is now allowing Android phone owners to download applicable offline content to a memory card. The company is unable, of course, to provide such a solution to iPhone users, who do not have access to MicroSD card slots on their devices.

Android​ Users – You can now download to your SD card so there's always room for more. pic.twitter.com/iQvnJL6AXH

— Netflix Canada (@Netflix_CA) January 24, 2017

It should be noted, however, that the update description on Netflix’s Google Play Store page states the feature is “not available on all devices,” which Neowin points out could refer either to devices without microSD card slots or smartphones that have those slots but still remain unsupported.

Offline playback came to Canadian Netflix on November 30th, 2016, with a selection of both Netflix originals and other TV and movie content, including Orange is The New Black, Narcos, The Crown, Scarface and The Office.

Source: Twitter Via: Neowin

Have you ever felt that someone's behavior was hostile or aggressive towards you, formed a negative opinion about them and then taken protective action(s) against them?

If so, you've likely participated in a Cycle of Mistrust.

In their must-read book, Driving Fear Out of the Workplace: Creating the High-Trust, High-Performance Organization, Kathleen Ryan and Daniel Oestreich described Cycles of Mistrust as a kind of organizational flu.

The cycle looks like this:

The cycle may begin at any one of the six starting points above.

The authors describe each step as follows:

• Negative assumptions: A manager adopts negative beliefs about an employee's intentions, style, or behavior.

• Self-protective behavior: The manager behaves in ways that are self-protective, acting to defend against the employee's potential to harm or hinder the manager.

• Observed aggressive, confusing, frustrating, or irritating behavior: The employee sees the manager's self-protective behavior and interprets it as intended to harm or block the employee in some way.

• Negative assumptions: Based on this interpretation, the employee adopts negative beliefs about the manager's intentions, style, or behavior.

• Self-protective behavior: The employee acts in self-protective ways in order to defend against the manager's potential to harm or block the employee.

• Negative assumptions: A manager adopts negative beliefs about an employee's intentions, style, or behavior.

Observed aggressive, confusing, frustrating, or irritating behavior: The manager sees the employee's self-protective behavior and interprets it as intended to harm or hinder the manager in some way.

A Real Example

Let's look at a real-world example, beginning with a supervisor forming a "Negative Assumption" and continuing clockwise around the Cycle of Mistrust:

Cycle One:

1. Negative assumption: Supervisor thinks that an employee is lazy and unconcerned with company finances because he doesn't fill out his timesheet.

2. Self-protective behavior: Supervisor sends a note to the employee's manager.

3. Observed aggressive behavior: Employee is confronted by their manager.

4. Negative assumption: Employee doesn't like "getting in trouble" for useless administrivia when he is working so hard for the company.

5. Self-protective behavior: Employee fills out timesheet using a quick & dirty approach, making sure to document all of the hours he worked, which is often beyond 40 hours per week.

6. Observed aggressive behavior: Supervisor observes that timesheet categories aren't used correctly and more than 40 hours are submitted, which causes problems for accounting and forces her to make time-consuming adjustments.

We've only made one trip around the cycle of mistrust. Unfortunately, the cycle tends to continue, getting worse with each rotation...

Cycle Two:

1. Negative assumption: Supervisor thinks the employee is a slacker who doesn't care about all the work he is making for her.

2. Self-protective behavior: Supervisor calls employee's manager again to discuss the problem.

3. Observed aggressive behavior: Employee's manager has another discussion with them about the supervisor's problem with time sheets.

4. Negative assumption: Employee feels as though they are being micromanaged because they are not trusted.

5. Self-protective behavior: Employee fills out timesheet using correct categories but still includes hours over 40 to document all of the time spent on work.

6. Observed aggressive behavior: Supervisor assumes that the employee just doesn't care about all of the re-work they are creating.

And so on...

So what is happening in these cycles of mistrust?

Not a whole lot of real communication or understanding!

In this particular example, the employee doesn't understand what the supervisor does with the time sheet data or the fact that late or inaccurate time sheets lead to late invoices, which lead to late payments by clients, which lead to late payments to staff, which lead to stress and unhappy staff who may consider getting jobs with different employers. Similarly, the supervisor doesn't understand how little the employee knows about the purpose of time sheets, the significance of the work categories used in time sheets, the irrelevance of any hours documented over 40, etc.

What we have here is a failure to communicate. And when that occurs, cycles of mistrust grow like weeds.

Clearly what is needed here is not more aggressive behavior but a company-wide dialogue about time sheets that highlights what they are for, why they are used, how they are meant to be used, etc. Such a dialogue could in fact lead to productive changes to the process, making everyone happier.

In fact, the above example resembles an issue within Industrial Logic. New employees had been inheriting a time tracking system without an explanation for why it existed or what is what truly meant to achieve. A cycle of mistrust developed because communication and trust were sorely lacking.

We broke that cycle of mistrust by having a company-wide discussion about time sheets, openly discussing issues with time sheets and modifying the process to be far simpler.

Breaking Cycles of Mistrust

The first step in breaking cycles of mistrust is to become aware of them. There are many ways to approach that. One way is for employees and supervisors to privately fill out what they believe is happening during the six steps of the model. Then, if it feels safe, they can discuss their perspectives, perhaps with a trust facilitator.

In my experience, many cycles of mistrust are born out of poor communications, assumptions and an environment that is lacking in collaboration and shared understanding.

I will conclude this post with some wise words from the authors of Driving Fear out of the Workplace:

The post Cycles of Mistrust appeared first on Industrial Logic.

Alongside beta versions of iOS, macOS, and tvOS, Apple today announced the release of the first beta of watchOS 3.2. The beta has yet to appear on Apple's developer portal, but it should be available soon. Besides the standard bug fixes and performance improvements, this update includes a couple new features, one of which is called Theater Mode. From Apple's developer release notes:

Theater Mode lets users quickly mute the sound on their Apple Watch and avoid waking the screen on wrist raise. Users still receive notifications (including haptics) while in Theater Mode, which they can view by tapping the screen or pressing the Digital Crown.

This sounds like an interesting new option that could be useful in scenarios besides being at the movie theater. Personally, I'm likely to use Theater Mode when I wear my Apple Watch overnight for sleep tracking. My normal practice is to turn off Raise to Wake in the Settings app before going to bed, but this could prove an easier method.

Besides Theater Mode, the most significant update in 3.2 is enhancements to Siri. Last year iOS 10 improved Siri by enabling it to handle queries from third-party apps that fit into specific categories:

• Messaging
• Payments
• Ride booking
• Workouts
• Calling
• Searching photos

Though all of those areas could be handled by Siri on iOS 10, Siri on Apple Watch was previously only able to direct you to your iPhone to perform those actions. But with watchOS 3.2, that is longer the case, as Siri on the Watch is now able to perform these third-party requests.

watchOS 3.2 will likely see a public release this spring, after a couple months of beta testing is complete.

---

Support MacStories Directly

Club MacStories offers exclusive access to extra MacStories content, delivered every week; it’s also a way to support us directly.

Club MacStories will help you discover the best apps for your devices and get the most out of your iPhone, iPad, and Mac. Plus, it’s made in Italy.

Join Now

In one of the most detailed and extensive reports on the Galaxy S8 to date, The Guardian has revealed several new rumoured details about Samsung’s upcoming smartphone.

Let’s start with the basics. Corroborating a previous report from Bloomberg, The Guardian reports that the S8 will come in two sizes and will feature a bezel-less, edge-to-edge “infinity” display. The publication then goes on to note three interesting things related to the S8’s display.

Design and display

The first is that the “infinity” screen takes up most of the front of the device, leaving little space for other design elements, including Samsung’s logo, which is reportedly not present on the phone’s front side.

Also missing from the front of the device is a fingerprint scanner. In one of the most interesting revelations from this report, The Guardian says that rather than integrate a fingerprint sensor into the S8’s screen, Samsung has allegedly chosen to place the sensor on the back of the device. The in-screen fingerprint scanner is something less realistic reports have claimed Samsung planned to build into the S8.

Third, The Guardian notes, according to its sources, that the screen protector shared by SamMobile and Twitter user @dfordesign last week is an accurate representation of the device.

Rollover features

Returning features from past Samsung devices include the Note 7’s iris authentication technology, USB-C connectivity, expandable storage via a SD card slot and, in what’s likely to come as good news to many, Samsung hasn’t excised the venerable 3.5mm headphone jack from the device. The phone will feature Qualcomm’s new Snapdragon 835 processor — at the expense of other smartphone OEMs.

When it comes to the S7’s excellent back-facing camera, the S8 will only feature incremental improvements upon its predecessor, with low light performance, speed and image quality seeing upgrades. The Guardian says it was able to confirm that the smartphone’s camera will integrate with Samsung’s unreleased Bixby personal assistant to enable image recognition functionality.

One other piece of good news is for anyone that ends up deciding to buy the S8 is that all base models will ship with 64GB of internal storage.

New accessories

Samsung will also launch new versions of its Gear VR headset and Gear 360 camera alongside the device, as well as a dock that turns the smartphone into a replacement desktop machine, again corroborating a past rumour. The Guardian doesn’t provide too many details on how this dock will work, merely noting that the feature will be called “DeX.”

The Guardian claims Samsung will start selling the S8 on April 21st, which verifies speculation that Samsung made a decision to delay the S8’s launch following its troubles with the Note 7. It’s important to keep in mind that launch date could be specific to the U.K., as The Guardian is based in the U.K.

Source: The Guardian

mkalus shared this story from TorrentFreak.

---

Over the years, adult image publisher Perfect 10 developed a reputation for making a business out of suing Internet services for alleged copyright infringement.

The company targeted Google, Amazon, MasterCard and Visa, even hosting providers such as LeaseWeb and OVH. After securing several private settlements in earlier actions, the company sued Usenet provider Giganews after Perfect 10 images appeared on Giganews servers. Things didn’t go well.

In November 2014, the U.S. District Court for the Central District of California found that Giganews was not liable for the infringing activities of its users. Perfect 10 was subsequently ordered to pay Giganews $5.6m in attorney’s fees and costs.

With Perfect 10 not quite done the case went to appeal, but in an opinion just handed down by the Court of Appeals for the Ninth Circuit, the adult publisher has received a crushing defeat. The panel held that to be held liable for direct copyright infringement, Giganews must have committed some voluntary act that caused the infringement to occur. The requirements for such “volitional conduct” were not met.

“The panel concluded that the evidence showed only that Giganews’ actions were akin to passively storing material at the direction of users in order to make that material available to other users upon request, or automatically copying, storing, and transmitting materials upon instigation by others,” the ruling reads.

The panel also found that Giganews was not liable for contributory infringement after Perfect 10 failed to show that Giganews “materially contributed to or induced infringement.”

On Perfect 10’s claim for vicarious infringement, the panel upheld the district court’s summary judgment in Giganews’ favor, noting that Perfect 10 failed to show a “causal link between the infringing activities and a financial benefit to Giganews.”

Ron Yokubaitis, Co-CEO of Giganews, said that his company’s decision not to give in to Perfect 10 had resulted in a long and hard-fought battle, but the end result meant it had been worth it.

“We decided that it would be important to stand up to Perfect 10 and not be bullied by its abusive litigation tactics.  We were not going to settle this case just to avoid the risk of potentially catastrophic statutory damages in today’s crazy copyright world, a threat that unscrupulous plaintiffs like Perfect 10 use to extract unjust settlements from more timid companies,” he said.

“We took a stand for Usenet, for technology and online platforms, for the public, and for ultimate benefit of rational copyright law.  We were not just battling Perfect 10:  standing behind Perfect 10 – and even sharing in its oral argument at the court of appeals – was the Recording Industry Association of America(RIAA), which tried to argue that it was voicing the interests of small copyright holders.”

Giganews went on to thank several groups that gave it support during its battle with Perfect 10, including the Internet Infrastructure Coalition, EFF, and Public Knowledge. While Giganews will continue in the Usenet business, Perfect 10’s efforts to extract billions in damages from the provider have essentially developed into a suicide mission.

“With this decision, Perfect 10’s days as a copyright troll masquerading as a porn company are now finished,” Giganews said.

“The case now moves to its final stage to collect attorney’s fees from Perfect 10.  Giganews is seeking the appointment of a receiver to take charge of all of Perfect 10’s copyrights, trademarks, and domain names and to liquidate them in partial satisfaction of Giganews’ judgment against Perfect 10.”

The only area where Giganews failed to convince the court was in its request to add Perfect 10 founder Norman Zada to the verdict. The district court already denied that request and the panel at the court of appeal upheld that decision.

The full ruling is available here

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.

I like Incredibox.   A lot.   Because it’s fun.   How much fun?   Just watch and see.

The ultimate in road widening has to be Avenida 9 de Julio (Argentina’s Independence Day) – often stated to be the widest avenue in the world.  (Is that true?  Anyone?)

It runs for a kilometre between two railway stations – the original rationale for its expansion in 1912.  But the history of public works and private companies is a particularly messy one in Buenos Aires, and nothing happened for decades.

When a particularly determined mayor was able to get a big-buck bond bill through city council during the world-wide Depression, the basic construction project of five blocks and a new plaza took only three months or so in 1937.  But by then the proposal had gone from 33 metres wide to 100 (it would relieve congestion!) – though the complete right-of-way, building to building, seems to be closer to 140 meters.  That meant taking out literally blocks of buildings without a lot of notice, and the cost, well over estimate, had a ruinous impact on the city budget.

I didn’t expect to like it one whit.  I expected it would be hostile, difficult to cross on foot and altogether unpleasant.  But, as it turned out, not so much.

The avenue is broken into about nine or ten different sections, from sidewalk to median to busway, and like elsewhere in BA it’s heavily treed and, in parts, very pleasant.

Best of all, though, is the recently installed rapidbus corridor in the centre, removing at least four lanes of car traffic and outfitted with a well-designed series of stations and platforms that also allow pedestrians to walk comfortably down the centre of the avenida.

Oh, that big white building with the radio tower and a shilouette of, yes, Eva Peron – it’s the Ministry of Health building, too big to tear down when the avenue was being extended in the 1960s.

There are two silhouettes in this art piece by Marmo installed in 2011 for her 50th anniversary.  The one in the second image from the top is Peron in a passionate speech (haranguing, it’s said, the city’s wealthy oligarchs in Barrio Norte.)  While facing south, Evita is smiling down on the city’s poor.

Karl Kapp, Kapp Notes, Jan 27, 2017

---

Papert called it "hard fun". We;ll, it has been fun but I've been swearing a lot recently. Getting to understand technologies like Docker and Vagrant - and all the associated middleware and protocols and conventions - has been a lot to undertake. I have a simple objective to start - create an image of gRSShopper and serve it from AWS. Why do it it this way? Why not just study what I need? Because if I want to truly understand it I have to build it. That's the point of Karl Kapp's argument in this post. "When you really want learners to understand content or concepts, force them to struggle with the concept or the idea. The act of struggling and manipulating and engaging with content will make it more meaningful and more memorable." And if I want to be credible when I talk about these technologies, I have to know them, inside and out.

[Link] [Comment]

David Raths, Campus Technology, Jan 27, 2017

---

I think this vision of the future of educational technology is fundamentally correct, though the description in this article is lacking. "We are using Canvas as a thin layer and laying apps on top of it. For instance, we needed a better way to record video, so we developed an app to record video on an iPhone or iPad. Once you upload it, it automatically gets bounced into your Canvas account. We are using Canvas as the core glue to hold together a bunch of other things."

[Link] [Comment]

ABSTRACT

The difference between two sequences A and B can be compactly stored using COPY/INSERT operations. The greedy algorithm for finding these operations relies on an efficient way of finding the longest matching part of A of any given position in B. This article describes how to use a suffix array to find the optimal sequence of operations in time proportional to the length of the input sequences. As a preprocessing step, we find and store the longest match in A for every position in B in two passes over the suffix array that has been enhanced with longest common prefix information (LCP).

INTRODUCTION

The days of losing work due to a power outage are over. In modern applications, users expect their work to be saved immediately. They also make mistakes, such as accidentally deleting large sections, and they expect to be able to restore their work to a prior state. To facilitate this, we need a way to retrieve or reconstruct every version of a document. We can take advantage of the similarities between the versions to minimize the space they consume.

INSERT/DELETE ALGORITHMS

Given two sequences of items A and B, we can compute a set of operations that will transform A into B. That way, only these operations need be stored. Two main classes of differencing algorithms are commonly used. Version control tools that deal with source code are often based on the longest common subsequence. For example, given the text:

A: The quick brown fox jumped over the lazy dog.

B: The lazy dog jumped over the quick brown fox.

The longest common subsequence is:

(The )(jumped over the )(.)

Although the text is written out here for clarity, only the position and length of each matching block is used. Using this information, one can directly compute the ranges where the text changed and thus derive INSERT / DELETE operations.

The example above can be encoded:

AT 5 DELETE "quick brown fox "
AT 5 INSERT "lazy dog"
AT 30 DELETE "lazy dog"
AT 30 INSERT "quick brown fox "

While INSERT/DELETE changes are easy to see visually in side-by-side comparison tools, they are suboptimal for file storage because they cannot exploit out-of-sequence similarities.

We can fix this by adding a MOVE operations. It is possible to change pairs of INSERT/DELETE commands to MOVE as a post-processing step. However, a more pressing issue is that any algorithm based on the longest common subsequence has a O(N^2) runtime in its worst case. Modern differencing tools often use Meyer's O(ND) algorithm, which is proportional to the length of the strings and the number of differences between them. Of course, when the two texts share little similarity, that will take a very long time.

THE COPY/INSERT ALTERNATIVE

Tichy describes system based only on COPY/INSERT. Starting with an empty string, it is possible to recreate a B by copying from various positions of A. Portions that do not exist in A can be created using the INSERT command. It can also be used when the encoding of the insert command would be smaller than the equivalent copy.

A: The quick brown fox jumped over the lazy dog.

COPY (The )
COPY (lazy dog)
COPY (jumped over the )
COPY (quick brown fox)
INSERT (.)

Finding COPY/INSERT operations is much simpler than LCS based algorithms.

# the position of the string.
q = 0
while q < len(B):
    find p and l such that (p.q.J) is a maximal block move 
    position, length = findLongestMatch(q)
    if length > 0:
        If insertFrom >= 0:
             outputInsertCommand(insertFrom, q - insertFrom)
             insertFrom = -1
        outputCopyCommand(position, length)
        q += length
    elif insertFrom == -1:
        insertFrom = q

if insertFrom >= 0:
     outputInsertCommand(insertFrom, q - insertFrom)

The runtime of the algorithm is dependent on the findLongestMatch function. Given a position in B, it finds the position in A with the longest matching sequence.

The brute force solution, which simply compares each possible position of A each time, performs surprisingly well when the sequences are mostly similar. It is not called very often, because the matches it finds are long. For sequences that are different, it again devolves into an O(N^2) running time.

The algorithm presented by MacDonald (2000) for use in the XDFS file system uses a preprocessing step to achieve O(N) operation. At each offset in A, the next 16 characters are placed into a lookup table and mapped to that position. FindLongestMatch is then:

code = next 16 characters in A
If code exists in the table,
    Return the length of the longest match at A[table[code]:] and B[index:]
Else
   No match at this position.

This algorithm is very fast and reasonably thorough. However, it is often not optimal. To guarantee O(N) time, the algorithm makes a tradeoff. When the table of positions is built, existing entries are "clobbered" by later ones. Only one position for each code is stored. If all of the positions were stored, then the algorithm would have to check each one, which would make the overall runtime O(N^2) with certain combinations of inputs.

THE SUFFIX ARRAY

A suffix array is an ordered list of positions in the string. Modern suffix array construction algorithms will bucket sort certain positions in the string, and then use the information to "induce" the positions of the other characters. With this clever trick, a suffix array can be created from a sequence in O(N) time where N is the length of the sequence.

It is often useful to build an enhanced suffix array. In addition to the suffix array of length N, the longest common prefix (LCP) array of length N-1 is built. IT contains the length of the longest common prefix between each entry and the next. By exploiting the commonalities in the sequence, these prefixes can also be computed in O(N) time, either during the suffix array creation, or as a separate step.

To find commonalities between two different sequences, they are appended together, separated by a character not found in either string, and a suffix array is constructed. An example is here:

A: "mississippi" + "u0001"

B: "sips and misses" + "u0000"

String   Index   Lcp  15 characters
   B       15     0 |"u0000"
A          11     0 |"u0001sips and missesu0000"
   B        4     1 |" and missesu0000"
   B        8     0 |" missesu0000"
   B        5     0 |"and missesu0000"
   B        7     0 |"d missesu0000"
   B       13     0 |"esu0000"
A          10     1 |"iu0001sips and missesu0000"
A           7     2 |"ippu0001sips and misses"
   B        1     1 |"ips and missesu0000"
   B       10     3 |"issesu0000"
A           4     4 |"issippiu0001sips and mis"
A           1     0 |"ississippiu0001sips and "
   B        9     4 |"missesu0000"
A           0     0 |"mississippiu0001sips and"
   B        6     0 |"nd missesu0000"
A           9     1 |"piu0001sips and missesu0000"
A           8     1 |"ppiu0001sips and missesu0000"
   B        2     0 |"ps and missesu0000"
   B       14     1 |"su0000"
   B        3     1 |"s and missesu0000"
   B       12     1 |"sesu0000"
A           6     3 |"sippiu0001sips and misse"
   B        0     2 |"sips and missesu0000"
A           3     1 |"sissippiu0001sips and mi"
   B       11     2 |"ssesu0000"
A           5     3 |"ssippiu0001sips and miss"
A           2     0 |"ssissippiu0001sips and m"

Note 1. The position of sequence 1 and 2 can be easily determined by its offset into the concatenated string.

Note 2. The LCP at index i efers to the commonality with position i+1 in the suffix array.

Note 3. The LCP between any two entries in the suffix array is the minimum of the LCP of all adjacent entries between them.

The common parts of the combined string AB are near each other in the the suffix array. However the common parts of A and B are not necessarily adjacent. If either string has commonalities with itself, then there will be two or more adjacent entries in the suffix array belonging to the same string.

Examining the suffix array above, we see that the location "sips and misses" in B has below it a match in string A of length 2. However, above it is a match of length 3. Our algorithm must consider both possibilities.

We do this in two passes. In the forward pass, we find the longest common prefixes between B and any part of A previous to it in the suffix array. In the reverse pass, we find the LCPs between B and any part of a that is after it in the suffix array.

def longestMatches(self):
        # returns, for every position in B, a tuple with the longest matching 
        # position in A and the length of that match.
        result = [None] * self.length2

        # forward pass
        lcp = 0
        aIndex = 0
        for i in range(len(self.sa)):
            if self.sa[i] < self.length1:
                # string in A
                lcp = self.lcp[i]
                aIndex = self.sa[i]
            else:
                # string in B.
                result[self.sa[i] - self.length1] = (aIndex, lcp)
                lcp = min(lcp, self.lcp[i])

        # reverse pass
        lcp = 0
        aIndex = 0
        for i in range(len(self.sa)-1, -1, -1):
            if self.sa[i] < self.length1:
                # string in A
                aIndex = self.sa[i]
                if i > 0:
                    lcp = self.lcp[i-1]
            else:
                # string in B.
                lcp = min(lcp, self.lcp[i])
                bIndex = self.sa[i] - self.length1
                oldAIndex, oldLcp = result[bIndex]
                if lcp > oldLcp:
                    result[bIndex] = (aIndex, lcp)

        return result

ANALYSIS

Even though building the suffix array is an O(N) algorithm, it will always be slower compared to the XDFS method because it requires several passes through the data. If optimal encoding is required, and the algorithm must be O(N) for all inputs, then the suffix array method may be considered. Preprocessing operations, such as removing the common prefix and suffix from the input, are important to reduce the problem size.

RESOURCES

Here is the Python source code that I used to test this technique. It contains a transcription of the SAIS suffix array construction algorithm from C into Python.

• barcamp (comic)
• 0, 1, Many, a Zillion
• The strange man reading a novel in the meeting room
• Optimizing Ubuntu to run from a USB key or SD card
• Why Perforce is more scalable than Git
• Finding Bieber: On removing duplicates from a set of documents
• Fast and Easy Levenshtein distance using a Trie