Rolandt

For the true geek, especially one who values the planet we all exist on, the Fairphone 2 is perhaps the ultimate smartphone. That it’s demonstrably not the best ‘bang per buck’ or indeed the prettiest phone out there, is almost irrelevant in light of the game changing Unique Selling Point here – you can take the Fairphone apart. Officially. Continue reading →

Pray, The Sequel

(Update: switched to BlockAdBlock.)

Doc Searls writes,

To have a deal, both parties need to come to the table with terms the other can understand and accept. For example, we could come with a term that says, Just show me ads that aren’t based on tracking me. (In other words, Just show me the kind of advertising we’ve always had in the offline world — and in the online one before the surveillance-based “interactive” kind gave brain cancer to Madison Avenue.)

Read the whole thing.

"Just show me ads that aren’t based on tracking me" is a message that you can send and receive today. You can build C.H.E.D.D.A.R. ads if you go to the right ad network, or install the right ad server. You can run a C.H.E.D.D.A.R. browser today, if you install (for example) Privacy Badger and Self-Destructing Cookies on Firefox.

All the pieces of C.H.E.D.D.A.R. exist, but they're just not integrated, branded, or made easy to install everywhere.

Are you already running a C.H.E.D.D.A.R. browser? Let's find out.

Detecting...

So, now all the JavaScript programmers have done a "View Source" on this page, and you're all like, wtf, that's it? A tracking protection detector and a check for a first-party ad element?

Yes, that's it. You can always write more refined versions of these, but the point is that you can do C.H.E.D.D.A.R. on the client without waiting for any new code on the server side, and you can do C.H.E.D.D.A.R. on the server without waiting for any new code on the client side.

Wouldn't C.H.E.D.D.A.R. be better if we added an extra layer of protocol, or a special HTTP header, or something? No, because no server can tell how a client is configured. If a browser or extension sends some future new intent message, it doesn't reliably tell the site if the user is also running a conventional ad blocker. Considering that ad blockers are the most popular browser extensions, it's likely that many people who install a "we welcome ads not based on tracking" extension will also have tried an ad blocker, and might not even remember they left it on, or not know they have to turn it off.

Actually testing for the delivery of a legit ad, or fake ad element, is necessary. Combine that with DNT and tracking detection, and you get a reliable "Just show me ads that aren’t based on tracking me" message.

That doesn't mean that C.H.E.D.D.A.R. is anywhere near done. If you want to build software around it, there are a lot of potential projects.

• Better tracking protection and tracking detection

• New ways to test that a DNT-respecting ad has been delivered to a human user

• DNT-respecting, fraud-resistant web analytics

• A "Just show me ads that aren’t based on tracking me" button in privacy tools and ad blockers.

• C.H.E.D.D.A.R. detection built into web content management systems

The problem with web ads, legit and otherwise, is much like the problem of opt-in email newsletters and email spam. Somehow the idea of a "spam-free replacement for SMTP" never really caught on. Instead, we got:

• Legit email is the kind of email that makes it through existing spam filters.

• A good spam filter is the kind of filter that lets legit email through (but blocks "spam")

The first spam filters got started before legit email senders had to be concerned about deliverability—but because of spam filters, deliverability is big business today.

If you write a new spam filter, or set up an email service, you have to let through the mail that people agree is legit. If you start a new service that sends mail, you have to pass the existing spam filters.

Different services have different ToSs, but we can send and receive email as ToSs change, because they all reflect a common set of norms around what is and isn't spam. And we never actually have to agree on a common definition of "spam".

We'll never get the web advertising problem nailed down in precise legal and technical terms. There will always be a mix of old and new clients and servers, a variety of laws and norms, and new inventions and business models. Whatever we come up with will have to be messy, imprecise, and resilient in order to stand a chance.

I find myself tasked with polishing and publishing a little custom JSON-encoded language. It’s harder than it ought to be.

This didn’t start with the language, it started with prototype software this guy wrote, that did something old and familiar in a new and dramatically better way. He replaced a bunch of gnarly old code with a few JSON templates to save time. Now, in the rearview, the JSON looks like an important part of an important product.

And there’s a lesson in that: All the good markup vocabularies are discovered by coders trying to get shit done, not cooked up in committee rooms in advance of software. For example, TimBL just needed to tag his hypertexts. Not that this is that big.

Q: Why JSON?

If it looks like a document, use XML. If it looks like an object, use JSON. It’s that simple. The essential difference isn’t simplicity/complexity or compact/verbose or typed/text, it’s ordered-by-default or not.

This particular thing I’m working on is a lot like objects and not at all like documents. Case closed.

By the way, it’s amusing that this century hasn’t yet offered a plausible new markup alternative to the last one’s mouldy leftovers. Also pleasing to one who has left fingerprints on both leftovers.

Q: How to author?

With pain. By default, I write things in Emacs, which unaccountably doesn’t have a JSON mode that knows where to put the fucking commas.

I’ve also tried editing JSON in IntelliJ and it’s not terrible but not remotely good enough. I’m writing what you’re now reading in XML in an Emacs mode that’s a fantastically-productive finely-tuned machine, so the thing should be possible.

There’s another solution: make JSON easier. Hjson (“the Human JSON”) addresses this problem and looks quite cleanly thought out. There’s also JSON5 (“JSON for the ES5 era”), but I stopped reading at “Unicode characters and escape sequences aren’t yet supported”.

Meh. Fewer markup languages are better; fix the editors, I say. Maybe Amazon would be OK with me dropping all this cloud stuff and working on JSON authoring tools for a few months? Not holding my breath.

Q: How to specify?

If you want to ship your own language, you need to specify it. The most important part of any language spec is the human-readable prose that describes the constructs, says what they mean, and offers advice on how to use them.

Next most important is examples; lots of ’em, well-chosen, and downloadable from somewhere.

(Actually, if you ever find yourself tasked with specifying something, go read Mark Pilgrim’s Why specs matter, which clarifies your task: Turning morons into experts.)

Next most important, an open-source validator, so people can check their efforts; it should have helpful error messages which include line and column numbers.

Now, to write the validator, a schema will help, and you should write one anyhow; if you’re like most people, writing down the formalisms will probably shake out some bugs and sloppy thinking in your mental model of your language.

Having a schema is a lousy way to explain a language, and it only solves part of the validation problem, because any nontrivial language will have semantic constraints that you can’t capture declaratively.

But like I said, you should write one anyhow. Which means that if your language is JSON-based, it sucks to be you, because JSON Schema is a big fat pool of pain.

JSON Schema, sigh

If you find yourself needing to use it, run don’t walk over to Understanding JSON Schema, which has lots of examples and reasonably human-readable narrative; a nice piece of work.

In its first paragraph, it says “learning to use [JSON Schema] by reading its specification is like learning to drive a car by looking at its blueprints.” They’re being tactful; the JSON Schema spec is really not very good at all. I don’t think this is a controversial thing to say, but let me offer some evidence anyhow.

Most obvious: There are multiple pieces of software out there that claim to implement JSON Schema, and their behavior is really inconsistent, in my experience.

One area where I observe inconsistencies is in the handling of the “$ref” construct. Irritated, I decided to go check the official spec. “$ref” is not defined (but is used) in JSON Schema Core. Same for JSSON Schema Validation. Same for JSON Hyper-Schema. Same for the Core/Validation Meta-Schema and the Hyper Meta-Schema

Actually, I could be wrong; the spec is really hard to read; and I say that as one with much more experience in spec-reading and schemas than most.

When I’m defining a language, I need to work on the schema and the validator and the examples all together, iteratively, validating things multiple times per minute, so I need a tool that I can run from the command line that is:

1. Fast,

2. consistent,

3. complete, and

4. reports errors with line and column numbers.

So far, I haven’t found one. JSONLint (npm) isn’t up to date with the latest schema spec draft. json-schema and json_schema (ruby; see, hyphen vs underscore, isn’t that quaint?) are inconsistent, particularly in their handling of “$ref”, and neither of them reports line/column, and neither of them respond sanely to JSON-level rather than schema-level errors.

Then there’s json-schema-validator, but that’s in Java, which means it’s probably too slow for command-line use, and also I’m not smart enough to run Java from the command line without hand-holding from an IDE or a really good build system that knows dependencies.

Feaugh.

What I’m actually doing

First, I’m keeping my schema in one great big file; cross-file “$ref” handling is irremediably broken, near as I can tell.

Second, I’m using “json-schema” (hyphen not underbar) in a little ruby script, which first of all runs “jsonlint” to check for basic JSON sanity. “jsonlint” at least does line numbers, yay. ♫ Ruby and Node, sittin’ in a tree… ♫

Because I distrust the tools I’m building a Java testbed in IntelliJ that will let me double-check with “json-schema-validator” from time to time.

JSON specification future?

I notice that the most recent JSON-Schema IETF draft expired in 2013, and that a couple of the tools have “looking for maintainer” signs posted on them.

Now that I’ve mostly figured out JSON Schema, I neither love it nor hate it. So far, I’ve been able to make it do what I wanted. But it sure feels lame compared to RELAX NG, in terms of polish, documentation, and tooling.

This is a space that could use some work.

Last year I decided I wanted a smaller interchangeable lens camera to use as a “take everywhere” second camera to my Olympus OM-D E-M1. The E-M1 is a fairly small package compared to my former Canon 60D, but it’s still not so small that I take it with me everywhere. It’s certainly not pocketable in normal coat/jacket pockets or slacks/shorts pockets. And that was what I was hoping to get, a pocketable camera that still would work with all my micro four thirds lenses. I had heard about the recently announced Panasonic GM-5 (released late fall 2014) that is a micro4/3 format camera. I considered it compared to the Olympus PEN E-PL6, which, in online comparisons didn’t seem that much larger. I ended up deciding on the E-PL6 because of price ($299 compared to $899!) and the fact that I was already familiar with Olympus menu system.

I enjoyed using the E-PL6 for a little over a year and it was reasonably pocketable…in coats. However, a few weeks ago I began wishing I had a really pocketable, yet comprehensive, camera so I began looking at options again. I kept coming back to the Panasonic GM-5 and, so, watched videos again on it as well as reading more about the camera. I discovered the price had dropped to $485 through some suppliers and I became more convinced that the GM-5 was, in fact, quite a bit smaller than my E-PL6. I ordered it and got it a week or so ago. I sold my like new E-PL6 on eBay.

Wow! The Panasonic GM-5 dimensions don’t make it seem that much smaller in the abstract, but when you see and hold it, trust me, it’s smaller! And that suits me fine. It is, in fact, just what I have been wanting! I bought mine with its “made for the GM-1 & 5” kit lens, the Panasonic 12-32mm, f/4-5.6. The whole “package” is tiny in your hands but don’t let that fool you. Even though the camera and lens are plasticy, the build quality appears to be very good. The camera has the standard micro4/3 16MP sensor (2x crop factor) and excellent quality components and the 12-32mm lens is quite capable. In fact, the other night I was blown away by how sharp a blow up was of the head of one person in a picture I took at 32mm from probably 70 feet (23m) away in a dark theater.

I don’t intend this to be an in-depth review of the camera and lens, as there are many of those available online by people who report on all facets of the camera from both a technical and use perspective. These are, instead, my everyday impressions of the Panasonic GM-5 with its lens as a regular photographer who shot Canon for many years and Olympus micro4/3 for the past three years. I expect that some of what I’ll write are things that others may find trivial or unimportant. And, I kind of hope that’s the case, because I want to discuss the things that I look for when using a camera or things that surprise me in a positive way because I didn’t expect them. So, the first thing is, as I have said, that this is one seriously small camera! However, even so, it is quite usable. Even with 12-32mm lens, it is possible to put the camera in pants pocket (not jeans!) and certainly a coat pocket or a purse. I wrote the other day about using a body cap lens with the GM-5 so that the camera really is then quite pocketable.

The Panasonic GM-5 has a hot shoe so I can use, for example, my Nissin i40 with it or, for that matter, any of my available hot shoe flashes. Panasonic does supply a small pop-on flash. This included small flash can be used to wirelessly fire other, off-camera flash units as well. The GM-5 added a small EVF (electronic viewfinder) which the previous GM-1 did not. It is small and does get some getting used to if one is accustomed to a larger EVF, but it is still very handy to have when shooting in bright, sunny situations or if you just don’t like using the LCD as your viewfinder.

Another reason I wanted to replace my E-PL6 besides size was that it did not have built-in wi-fi. The Panasonic GM-5 does and, in fact, will connect to a router to connect to the Internet for direct posting of images. The wi-fi connection to the camera is done through a free mobile app provided by Panasonic and works quite well. My only complaint is that one can only select up to thirty images at a time to transfer to the connected smartphone or tablet. This is, of course, not really a big deal, but surprised me. One can also control the Panasonic GM-5 remotely using the wi-if connection. Both of these are features I have gotten very used to with my E-M1, so I am glad to have them on both of my cameras now. Connecting to the Internet via a router, however, is not so easy. That is, I have not yet been able to do it. You must set up an account with a free, Panasonic cloud service and then connect to your wireless router to access it. The Panasonic service, apparently, then transfers the image or images to the social network you have chosen. However, I’m still stuck at setting up the free account! More on this as I figure it out, but if anyone reading this piece knows some tricks, please do share!

The Panasonic GM-5’s LCD is a touchscreen, so many of the settings can be done using your fingers on the screen. This is something I’m not used to so I am still getting used to using this capability. I like that the camera has a mode dial position (only one) labeled “C” for custom. One can, however, create up to three custom camera configurations. With the mode dial in the custom position, one then selects which of the custom camera configurations to activate. This feature is much like the MySets system I have on my Olympus OM-D E-M1 but it is nice to have a specific position on the mode dial to use with the feature. There are two physical (+ record button) and five touch screen buttons that can be customized which gives you a quick way to get to features you find yourself using often but have to dive into menus to activate.

There are multiple screen display options accessible by pressing the DISP (display) button on the lower right corner of the camera back. One of these is a Quick Menu Screen that displays many settings of the camera. Each of the controls on the screen can be accessed by a tap of the finger or you can press the “Q.Menu” button (with trash can icon) to activate a control selection and change it using the four-way button set on the middle right back of camera. I like this as it is much like the Super Control Panel display on Olympus cameras and the Quick Screen on my old Canon 60D.

Finally, my last favorite feature of the Panasonic GM-5 is that is it comes with a cordless battery charger!! I love this type of charger and don’t understand why Olympus doesn’t follow suit here. And that brings me to the things that are not so positive. The first, naturally, is battery life. The GM-5 takes a Lithium Ion battery and the supplied battery is rated at only 680mAh. Panasonic says this will allow you to shoot up to 200 images per charge. I have gotten accustomed to short battery life with these all-electronic, micro4/3 cameras, so I was not surprised at the short battery life. I always buy at least two extra batteries for my micro4/3 cameras. Interestingly, some third party batteries for the GM-5 are rated as high as 1050mAh, although the ones I got are only 1000mAh. One other negative for the camera is that it only supports a flash sync speed of 1/50 second. However, if one isn’t going to use flash with the camera that much, this is not a big deal.

Overall, I am extremely pleased with my purchase of the Panasonic GM-5. I like that there is a dedicated dial just above the LCD to change shutter speed and aperture settings when shooting in Shutter-Priority or Aperture-Priority modes or Manual mode. It is small and I am still getting used to controlling it, but it is nicer than having to use the 4-button control set as I did on my E-PL6. A feature I wish the GM-5 had is an articulating LCD screen, even if – like the E-PL6 – it only articulated up and down. Maybe a newer version (perhaps “GM-6?”) will be released and Panasonic will add that feature. Who knows?

If you would be interested in virtual tutoring sessions using either Skype or Google Hangouts, please drop me and e-mail or leave a private message to me on Facebook or Google+.

Please leave me questions or comments on my Facebook page (https://www.facebook.com/billbooz) or on Google+ (http://plus.google.com/+BillBooz). I look forward to interacting with you!

Please follow my Photography Learning Center videos on my YouTube Channel!

Check out my free eBook on Mastering Your Camera’s Priority Modes: http://www.infotor.com/ebooks. Download and enjoy!

I am available to do one-on-one tutoring or small group lessons designed to meet YOUR needs and what you want to learn in the area of photography, using flashes, or the use of Apple products and software. Give yourself the gift of learning: http://www.infotor.com/photoclasses! And check out my tutoring bundle: http://www.infotor.com/photoclasses/tutoringbundle.php as well. A PDF of my brochure is at http://www.infotor.com/photoclasses/brochures/WHBI_2015-16Brochure.pdf.

Jamie Beard's talk on Stressing The Elements looks in detail at how Lego bricks are meant (and not meant) to fit together.

After years of trying, Intel is finally ready to give up on the smartphone market. The company has sold its Atom chips to OEMs at very low prices, or in some cases, even paid the OEMs to use the chips in their smartphones.  Continue reading →

Image by Jeff Carson from dandyhorse July 2015

VISION ZERO – An update from Bloor Street

Public Works logic: Could support for Vision Zero mean zero bike lanes on Bloor?

by Albert Koehl

When Toronto’s Public Works and Infrastructure Committee (PWIC) met last week to debate a pilot bike lane on Bloor St., it was no surprise that many residents and businesses attended to express their support. The surprise was that Council’s PWIC chair, who is also spearheading Toronto’s road safety strategy, modelled on Vision Zero, voted against the pilot despite its modest 2.4 km length (on a 5,500-km road system) and its safety objectives. As a result, the pilot has now been punted to the full council for debate and decision this upcoming Wednesday, May 4, 2016.

In the week straddling the public works committee meeting, six people, including two cyclists, were killed or gravely injured on city roads. Toronto is on pace to exceed even last year’s number of fatalities. In February 2015, the city’s high road death toll spurred current PWIC chair, councillor Jaye Robinson, to announce the development of a comprehensive road safety strategy, comparable to Vision Zero initiatives in other jurisdictions that seek to reduce road deaths to zero. “As a city, we can and must do better,” she said at the time, “particularly for pedestrians and cyclists, our most vulnerable road users.”

The goal of the Bloor pilot, according to Transportation Services Manager Steven Buckley, is to provide a safe, designated space for cyclists while assessing the impacts on all road users. Although Bloor is a popular cycling route, it has a high number of collisions involving cyclists. In the proposed pilot stretch of Bloor alone, there are 22 cycling collisions each year.

Bike lanes are an obvious safety measure while a pilot is a valuable planning tool that provides real world observations prior to a final decision. In rejecting the pilot Robinson explained that, “I support bike lanes but only where and when they make sense.” The councillor ignored pleas to approve the pilot from more than two dozen speakers, including merchants, students, parents, and leaders of cultural organizations and residents’ associations. (A video presented to PWIC, along with a great number of emails, delivered the same message.) Both the Annex and Koreatown Business Improvement Areas (BIAs) support the pilot and have already begun collecting data for a detailed study of the economic costs and benefits.

Parkdale Collegiate student Dale Atlas-Jones said that, “having a bike lane on Bloor St. would give peace of mind to both parents and students who want to have a safer, more efficient way to get to school.” Ron Koperdraad of the Bloor Hot Docs Cinema, who suspects he loses business because some cyclists by-pass Bloor for safer roads, welcomes bike lanes for his patrons. And Michelle Stafford of Secrets from Your Sister “passionately supports” bike lanes on Bloor for the safety of her cycling customers and staff.

The lone business naysayer at the public meeting scoffed at the notion that cyclists contribute to his enterprise. He was quickly corrected on social media by his many cycling patrons, as well as by PWIC member, councillor Mary-Margaret McMahon, a cyclist and shopper at his store. The owner later issued a clarification saying he welcomed the pilot and would embrace bike lanes if the pilot succeeded.

But if Bloor isn’t safe, why do cyclists continue to ride there? The answer is simple … just ask a motorist. Bloor, which connects with Danforth Ave., is a convenient, streetcar-free, east-west route that provides access to jobs and other downtown destinations. Bloor is itself an attractive destination with many coffee shops, grocery stores, restaurants, gyms, institutions like the Royal Ontario Museum, and schools, including the University of Toronto.

Bloor has among the highest cycling numbers in the city, despite the absence of dedicated space for cyclists. The potential for growth in cycling is therefore substantial - an increase that may well come from motorists, many of whom would love to cycle but for the danger. A significant percentage of car trips in Toronto are actually short enough to be cycled.

From a safety perspective, the key difference between cyclists and motorists is that the former, like pedestrians, are more vulnerable in collisions because they are not protected by bumpers, air bags, and in-car safety technologies.

Robinson attempted to portray her opposition to the pilot on the need for more information even though Buckley assured her the pilot was the most comprehensive of any such cycling project ever undertaken by the city.

Robinson alone can’t be blamed for the result at PWIC. Councillor Stephen Holyday, who considers cyclists a “minority compared to the mass volumes of people who need to move,” also voted against the pilot, ensuring the 2-2 stalemate. Holyday was preoccupied with whether there would be any increase in travel times, which, as one speaker informed him, was precisely the purpose of the pilot.

The vote by councillor Robinson (and Holyday) does not bode well for residents who are hoping for on-the-ground changes to make roads safe. Fortunately, other city councillors still have an opportunity at this week’s Council meeting to prove that protecting cyclists actually matters, and that the safety of “our most vulnerable road users” is a real priority in Toronto.

Albert Koehl is an environmental lawyer, road safety advocate, and a founder of Bells on Bloor.  He was an expert member of the Ontario Chief Coroner’s panel for road safety in 2012.

Related on the dandyBLOG:

Another small step forward for bike lanes on Bloor

Bloor bike lane another step forward, hurdles still ahead

Trading Ambition for Implementation

Bike Spotting: Do you Shop by Bike?

Bike lanes on Bloor Deja Vu

Bike lanes on Bloor on the back burner again?

Bike Spotting at Bloor and St. George: Do we need bike lanes on Bloor?

Bike lanes on Bloor: an update

Bikes Are Good For Business

My latest writing project is "Introduction to Cognitive Computing".

Most of the examples in the book involve text processing using the open source TensorFlow library for machine intelligence and the OpenNLP projects. I decided to concentrate on what I do now and only supply references to other areas like image and speech recognition, etc. that I don't work in anymore. I spent years working on digital signal processing for handling images and some time using time delay neural networks for speech recognition but now, my interests are mostly in intelligently processing and understanding natural language text.

I feel like I am entering the third stage of my work in natural language processing, starting in the 1980s with symbolic processing, then machine learning approaches like the Stanford NLP library and OpenNLP, and now in deep learning.

While the book specifically covers setting up a laboratory on the Google Cloud Platform, much of the DevOps material is also applicable to AWS and Azure.

I am developing the book examples in a private git repository now and they will eventually be in this github repository.

Reykjavík Grapevine @rvkgrapevine
A little Icelandic girl named Ripley meets her alien godmother, Sigourney Weaver. grapevine.is/news/2016/04/3… #cute pic.twitter.com/Jo5RmXoaUA

Tim Bray @timbray
So much we know to be true really isn’t: fivethirtyeight.com/features/who-w…

The mobile phone industry has had two waves - first voice and SMS and then the smartphone.  The voice wave has taken it from zero to 5 billion people on earth with a mobile phone, and now close to 2 billion mobile phones are sold every year. In parallel, starting 9 years ago, the smartphone wave converted a larger and larger percentage of those phone sales to smartphones. 

And since smartphones could be sold for higher average prices than feature phones, revenue grew even faster than unit sales. This was a great multiplier for the right companies - smartphones were a growing percentage of growing phone sales at growing prices. 

All of this is now reaching an end - the wave is almost over. 

By 2020 there'll be 6bn adults on earth and more than 5bn people with phones, and the last billion are necessarily the slowest and hardest to reach.  Phone sales are a function of the install base and the replacement rate - the install base hasn’t got much more growth and the replacement rate is also starting to lengthen (or at least not shorten). So phone sales will slow. Then, most phone sales now are already smartphones (as seen in the chart above), so the conversion of phone sales to smartphone sales also hasn’t got much further to grow. The smartphone install base does have a lot of room to grow, but that's a function of replacement at close to existing volumes, and even that will be largely done in a few more years. Hence: smartphone sales growth is slowing down. 

On one level this is just classic saturation - no industry can grow forever. But what happens next? 

At the level of the consumer internet, it’s been clear for some time that Apple and Google won the platform war and that the important questions have moved up the stack - how far can Google and Facebook capture attention and intent, what other interaction models will emerge, how far Android and iOS can shape interaction and consumer behaviour, and so on. 

For the hardware companies themselves, though (and that includes Apple), when you’re selling to everyone on earth (something the tech industry has never really done before), what do you do next? TV, once thought of as the next phase after PCs, turned to be an accessory to smartphones, and so are watches and (to some extent) even tablets. VR and AR are some time away with unclear market size, though I think AR could in principle be the next ecosystem after the smartphone. 

The obvious next market is cars, which in aggregate are much larger in revenue terms, and where a large part of the supply chain will be fundamentally remade by the shift to electric and (in due course) to autonomy.  Cars are a Big Deal for the tech industry. 

But it’s also interesting to think about the phone market itself, which isn’t going away any time soon (though AR may affect that in the next decade).

I spent some time chatting to Condor at MWC this spring. It's a subsidiary of an Algerian family-owned conglomerate, which began in the cement business and expanded into white goods - fridges and washing machines - and then televisions ('brown goods'). It built a nation-wide network of 150 stores to support that business. Then it got into the phone business, and last year it sold 3m phones, of which 2.8m were Android smartphones. The best-selling model retails for $80. It expects to reach sales of $1bn this year, and has around a third of the Algerian market. 

Condor is possible because mobile phone technology became something that you could buy off the shelf - if you can make a TV, you can make a mobile phone or a smartphone, without needing deep understanding of how cellular technology works anymore, or writing your own OS. In parallel, the manufacturing base of the industry moved from factories you own yourselves to outsourced contracting. So, you can make phones, or get someone else to make them for you, or some combination of the two, with much lower barriers to entry. And if you come from the cement business, your idea of a great margin looks rather different to Sony's. 

However, there's a big difference between making a phone and selling it. It’s all very well to put it in a shipping container in Shenzhen, but what happens after that? A lot of Apple's sales growth since 2007 has actually been about expanding distribution through mobile operators (which sell far more iPhones than Apple retail does), with the really big additions being Verizon Wireless and China Mobile. Indeed, the fact that it has now signed up all the operators that matter is one reason sales growth has slowed. In parallel, distribution was a big part of the Samsung story. It has effectively cloned Nokia: it offers every technology, frequency and specification, at every price point, for every operator, through every sales channel, and spends billions of dollars on sales and distribution to support that, of which a very large part will be sales commissions. 

That is, with the tech available off the shelf, the barrier to entry has moved from the creation and manufacture of the phones themselves to sales, distribution, marketing and support, and a lot of the innovation in the handset business now is around how to address that. Which part of the value chain do you start from and try to leverage, and which parts you outsource? Someone has to make it, someone has to import it, someone has to put into shops, or market it for online sales, and (especially in developed markets) someone has to provide support if you smash the screen. But all of those are being disassembled and reassembled in different combinations. 

Hence, at one end of the spectrum are Chinese companies that are just looking for distribution deals overseas, and will sell you a few thousand or tens of thousand with the brand of your choice printed on the back, and what happens after the shipping container (or suitcase) leaves Shenzhen is up to you. The next step along are those trying to create a brand of their own, often in parallel with selling phones under other people's brands. So I've met several companies that have a slick new consumer brand of their own with nicely designed handsets and a decent Android skin, and are thinking about how to take that abroad - what that sales and distribution might look like, and where it should be. One interesting illustration of this is Wiko, which has a double-digit share of the French market and is expanding in south-east Asia. The back of the phone says ‘Designed in France, Assembled in China’, but in fact Wiko seems to be majority owned by a Chinese company, Shenzhen Tinno Mobile Technology Company Ltd. 

Sitting right next to Wiko are ever more companies starting from the other end - building brand, distribution and marketing locally, adding some design, and outsourcing the manufacturing. Wiley Fox in the UK comes from people with a background in mobile operators, selling a premium design at a mid-range price with a lightly skinned version of Android. BQ in Spain originates in ereaders, amongst other things. Blu in Latin America has built a huge business on distribution. And of course Google sells its own ’Nexus’ line, using a rarely-encountered custom build of Android and adding a small amount of marketing and distribution. 

As the price for a good Android experience moves from $600 to $150 or $250, these companies can increasingly pass up operator subsidies, with burdens of inventory etc. that this imposes, and move straight to selling unsubsidised and online. The poster-child for this model is of course Xiaomi, which has pioneered an online-only flash sales model, backed by an attempt to build a passionate community around the brand and software experience. This has worked well in China but it's not clear how well it can be made to work elsewhere, and whether it can be built once and scaled globally or whether you need to do it one country at a time from scratch. 

Then, coming from the other end of the spectrum, mobile operators are increasing buying in a selection of low-end smartphones than they sell (generally unsubsidised on prepay) under their own brand. Sometimes these have operator apps preloaded (if they've not given up on that yet), sometimes not. One could argue that the value being added here is really only distribution, and so one might see other companies with distribution getting into this, such as mass-market retailers. Some of these have already experimented with Android tablets, with mixed results (as of course they did with MVNOs). 

This is all rather like the PC clone market of the 1980s - hundreds of undifferentiated companies fighting it out to sell commodity computers built with commodity components running a commodity operating system (though those companies mainly made the PCs themselves, where many phone brands do not). That world in due course led to companies like Dell - people who embraced the volume, low-margin commodity model and found an angle of their own. We’re starting to see equivalent model-creation now. 

Fri 04.29.16 – Happy 13th Birthday to this blog a few days late! While I worked most of the evening of April 24th in 2003 to set this blog up on Movable Type, I was able to publish the first post just a few minutes after midnight. Photo by Ms. Jen on Tues. 04.26.16 of... Read more »

Internet speed is a very contentious issue, especially in a country as big and spread out as Canada. While mobile internet quality has been a major concern for many Canadians, fewer people talk about the inequalities between cities and regions when it comes to home internet.

A new report from the Canadian Internet Registration Authority is seeking to identify any problem areas and highlight any improvements that might make Canadians’ home internet experiences better. The report sourced data from 126,000 internet users across Canada, and shows that there’s still lots of work to do to get Canadians across the country the best internet service possible.

Among the important findings from the report, the group says that different regions of Canada have vastly different average internet speeds, including bright areas in Ontario, Montreal, and the Atlantic provinces. On the other hand, Western and Northern Canada suffered from the slowest internet, with the territories’ speeds specifically lagging behind the national average by a whole 25 percent, in addition to worse service overall.

The report also highlighted some concerns even in cities where average internet speeds remain high, in that city cores will have high internet speeds, while suburbs will receive significantly slower internet.

Another point in the report is that while download speeds have been most important to users for surfing the web and streaming video, upload speeds suffer in comparison. This hasn’t been much of a concern until recently, but as social media and video calling are on the rise, outbound traffic is also increasing fast. Online storage and backup like Google Drive and Dropbox are also increasingly important, and those services also require significant upload speeds.

According to Byron Holland, the president and CEO of the Canadian Internet Registration Authority, the goal of the report is “…to see is a better online Canada. Where Canada not only competes with, but tops the global leaders in access speed, quality, and data sovereignty. CIRA launched the Internet performance test and is beginning a series of stories to educate Canadians, policy makers, and organizations on how the Internet really works so they can make informed decisions and improvements.”

This is certainly a noble goal, and interested users are encouraged to take a connection test at performance.cira.ca to help the group collect more data about Canadian internet speeds.

From today, UK consumers who use their mobiles across Europe should see a reduction in their bills. The good news is that further price reductions when roaming are planned in all European Union countries from June next year, at which time all such charges will be abolished.

For the moment however, there is an immediate cap on how much surcharge operators can levy as follows:

• €0.05 extra per minute for calls
• €0.02 extra per SMS sent
• €0.05 extra per MB of data.

According to the UK government, data, texting and phone calls will all be included in the reduced charges and should collectively save us millions of pounds.

“Roughly a million Brits stay the night in Europe every day, and they spend around £350m a year on roaming charges”, said Ed Vaizey, the UK’s minister for the digital economy, adding that “by realising these changes, we’re going to save British consumers millions of pounds a year”.

At the moment, when roaming abroad roaming charges are applied when you connect to another operator, besides the call cost, which typically means a hefty additional charge.

Unsurprisingly the biggest changes will be for those people downloading content such as email, videos and pictures. Such roaming charges will reportedly be reduce by almost three quarters, with call costs being reduced by a similar rate. Texts will end up costing two thirds less than today.

Looking at the figures, that means placing a call in Europe will fall to around 4 pence from more than 16 pence. Likewise, data downloads will cost a maximum of 4 pence from 17.4 pence.

In practical terms, the government claims the average cost of phone calls when roaming will fall by more than 50 percent. However, not everyone will notice the new reduced costs, as some of the UK’s operators already offer special deals and packages that provide cheap calls and downloads when abroad.

O2 claims that many customers will notice an immediate reduction in prices, whereas EE says most will not be affected. Three is one of the operators who offer free roaming in a selection of European countries, with its Roam Like Home package.

Greg Eh @gregeh
If you can't be more imaginative with your graffiti, you deserve a tower you don't want. pic.twitter.com/KzghLgPV8J

Karen Quinn Fung 馮皓珍 @counti8
@CitizenYan I've never been on anything except TekSavvy since 2008. Me and @sillygwailo love it.

Parveen Kaler @kaler
@dbarefoot it’s just that all the flair has been smashed out of the English announcers. Remember Jim Hughson when he started?

My first book on electronics was Getting Started with Electronics; to this day, I still imagine electrons as oval-shaped particles with happy faces because of its illustrations. So naturally, I was thrilled to find that the book’s author, Forrest Mims III, and my good friend Star Simpson joined forces to sell kit versions of classic circuits straight off the pages of Getting Started with Electronics. This re-interpretation of a classic as an interactive kit is perfect for today’s STEM curriculum, and I hope it will inspire another generation of engineers and hackers.

I’m very lucky that Star sent me a couple early prototypes to play with. Today was a rainy Saturday afternoon, so I loaded a few tracks from Information Society’s Greatest Hits album (I am most definitely a child of the 80’s) and fired up my soldering iron for a walk down memory lane. I remembered how my dad taught me to bend the leads of resistors with pliers, to get that nice square look. I remembered how I learned to use masking tape and bent leads to hold parts in place, so I could flip the board over for soldering. I remembered doodling circuits on scraps of paper after school while watching Scooby-Doo cartoons on a massive CRT TV that took several minutes to warm up. Things were so much simpler back then …

I couldn’t help but embellish a little bit. I added a socket for the chip on my Bargraph Voltage Indicator (when I see chips in sockets, I hear a little voice in my head whispering “hack me!” “fix me!” “reuse me!”), and swapped out the red LEDs for some high-efficiency white LEDs I happened to have on the shelf.

I appreciated Star’s use of elongated pads on the DIP components, a feature not necessary for automated assembly but of great assistance to hand soldering.

It works! Here I am testing the bargraph voltage indicator with a 3V coin cell on my (very messy) keyboard desk.

Voilà! My rendition of a circuit classic. I think the photo looks kind of neat in inverse color.

I really appreciate seeing a schematic printed on a circuit board next to its circuit. It reminds me that before Open Hardware, hardware was open. Schematics like these taught me that circuits were knowable; unlike the mysteries of quantum physics and molecular biology, virtually every circuit is a product of human imagination. That another engineer designed it, means any other engineer could understand it, given sufficient documentation. As a youth, I didn’t understand what these symbols and squiggles meant; but just knowing that a map existed set me on a path toward greater comprehension.

Whether a walk down nostalgia lane or just getting started in electronics, Circuit Classics are a perfect activity for both young and old. If you want to learn more, check out Star Simpson’s crowdfunding campaign on Crowd Supply!

I’m very honored and proud to have one of my books offered as part of the Hacking Humble Bundle. Presented by No Starch Press, the Hacking Humble Bundle is offering several eBook titles for a “pay-what-you-feel” price, including my “Hacking the Xbox”, along with “Automate the Boring Stuff with Python”, “The Linux Command Line” and “The Smart Girl’s Guide to Privacy”. Of course, you can already download Hacking the Xbox for free, but if you opt to pay at least $15 you can get 9 more fantastic titles — check out all of them at the Humble Bundle page.

One of the best parts about a humble bundle is you have a say in where your money goes.

If you click on “Choose where your money goes” near checkout area, you’re presented with a set of sliders that let you pick how much money goes to charity, how much to the publisher, and how much as a tip to the Humble Bundle. For the Hacking Humble Bundle, the default charity is the EFF (you’re free to pick others if you want). For the record, I don’t get any proceeds from the Humble Bundle; I’m in it to support the EFF and No Starch.

If you enjoyed Hacking the Xbox, this is a perfect opportunity to give back to a charitable organization that was instrumental in making it happen. Without the EFF’s counsel, I wouldn’t have known my rights. Knowledge is power, and their support gave me the courage I needed to stand up and assert my right to hack, despite imposing adversaries. To this day, the EFF continues to fight for our rights on the digital frontier, and we need their help more than ever. No Starch has also been a stalwart supporter of hackers; their founder, Bill Pollock, and his “Damn the Torpedoes, Full Speed Ahead” attitude toward publishing potentially controversial topics has enabled hackers to educate the world about relevant but edgy technical topics.

If hacking interests you, it’s probably worth the time to check out the Hacking Humble Bundle and give a thought about what it’s worth to you. After all, you can “pay what you feel” and still get eBooks in return.

I’ve been thinking about storage capacity lately. I had a couple of IO errors on my aging Synology NAS drive that caused a minor panic at the thought of losing all of my redundancy and archived media. At 6TB of RAID5 backing up all our photos, ripped DVDs and iTunes libraries, it’s a daunting prospect to imagine replacing it all.

For the curious, I upgraded to a Synology DS416 and 4x4TB Seagate NAS drives, rated for 3 years of warranty. After RAIDing, that’s 10-12TB of effective storage which can theoretically survive a drive failure, if not an all-out zombie apocalypse. This should serve as personal cloud storage for the next 6-10 years.

It’s been nearly 10 years since I last looked at how much storage I’m carrying when going nomad – something I’m going to have to deal with very soon. Back in 2007, I was toting around 338GB of capacity. That’s almost 4 times what Johnny Mnemonic was working with in the 1995 movie based on the William Gibson short story. That much seems decidedly lotek these days and that movie was set in a fictional 2021. (still no monomolecular filament whips either)

Let’s add it up!

• 2012 Macbook Pro (I’m writing this on it!): 512GB (SSD)
• 1TB Thunderbolt/USB3 drive, camera+video backup
• 160GB iPod classic
• 16GB Nexus 5
• 64GB iPad Air
• 64GB SD Card for my OM-D E-M1
• 2x32GB SD Card backup
• 64GB flash drive

The total: 1968GB.

2TB. Two. That’s not including any of the RAM in each of these devices, which probably adds up to another 20-30GB.  Nor does it include the many terabytes of cloud storage I have access to in varying drip-sized portions.

Also notable is how much of that is in purely solid state memory. Half of it – the slow half – is a spinning platter. There’s a tiny spinning harddrive in the iPod classic too. The rest, nearly 1TB of it is in solid state flash memory.

That’s roughly the amount of storage you’d need for a complete academic research library. It fits in my backpack. It’s amazing how fast you can fill it up with GoPro and HD video footage and stills from my camera, and I’m not even trying to deal with 4K. If VR takes off in any meaningful way, the storage requirements are going to be monstrous.

Regular commenter MD has some good observations following up on this post – “Transforming Laneways.” 

 

This reminds me of the gardens at First and Fir on the former CPR right-of-way that started with one planting bed built on a postage stamp piece of derelict land and mostly funded by one aspiring gardener who had to pull the wisdom teeth of the board members of the adjacent co-op to cover a part of the cost of materials.  That one bed, bursting with flowers, brought many thanks and appreciative comments from the condo-dwelling neighbours.

.

Back then the CPR still ran one train a day to Molson’s, and the engineers and brakemen always greeted the gardener, so unlike the more recent PR shenanigans pulled by the CPR further up the tracks deeper into Creme territory where gardens were demolished in front of the cameras as a land negotiating tactic with the city. The lone gardener moved up the hill in the late 90s but, after a few years of neglect, another co-op member resurrected the idea and made it into a showpiece that inspired a small army of gardeners to help from various nearby buildings.

Chris articulated the potential of lanes, which is vast compared to their standard utilitarian use today. I hope the city can recognize the volunteer effort of residents and follow up any policy initiative on lanes with a small program not unlike Green Streets that allows citizens some flexibility. You can do a lot in even just a 0.6m setback.

There was a lane redesign program out there at one time but I understand some of the design techniques failed, such as permeable paving and infiltration channels which quickly plugged up. Cost was also a factor.  (See more here on “The Country Lane in the City.”)

.

I suggest the program should be given a new life but without experimentation that is at risk of failure. Go with the tried and true, like solid paving with cobbles and setts or even cheaper unit pavers used in channels, and rated for garbage truck weights. All it takes is imagination and the ability to learn from experience.

[Plus, check out this issue of the old Price Tags as pdf file: “The Lane Issue,” based on an essay by Lisa Brideau.]

 1. Low call volume

By Rose Behar

Despite the 10’s amazing sound performance for listening to music, the sound quality for calling is poor. In any public or outdoor area, it’s impossible to carry on a conversation without plugging in headphones or putting the phone on speaker.

The weak front-facing speaker at the top of the phone seems to be the culprit. It’s possible HTC didn’t focus on the quality of call audio since most users would tend to listen through headphones or on speakerphone, but its an irritating handicap nonetheless.

2. The design is amazing 

The HTC is arguably the best looking phone currently on the market, rivalling even the iPhone 6s and S7/S7 edge. The crisp chamfered edges on the brushed aluminum back have all the elegance of James Bond’s Aston Martin DB5, and lend a maturity to HTC’s flagship that was lacking in its forerunner, the M9. The logo is simple, understated and the camera is unobtrusive.

It would be a shame to put a case on this phone, something that can’t be said of most other Android phones. Its weight also gives off the impression of durability – though in reality we have no idea how rugged it is (unlike our other stories, we didn’t put this to the test by lighting it on fire). But even with a case, the 10 would still be an aesthetic standout with its sleek, understated black screen and elongated oval fingerprint scanner in place of a physical home button.

3. It still gets really hot (just like the M9) 

While the vast majority of reviewers have commented on the capacity and speed of the Snapdragon 820 processor, there is a downside to the 10’s impressive horsepower.

Much like the M9, the phone gets uncomfortably hot after extended periods of use, particularly with applications that require intensive graphics processing. After about ten minutes of using Snapchat, for instance, the 10 became so hot that it was unpleasant to use and had to be turned off for a while.

4. Freestyle layouts are cool, yo

In addition to a beautiful exterior design, the 10 offers a completely customizable and unique UI experience with freestyle layouts. With this feature, you can make your home screen in to a themed scene, replacing normal app icons with “stickers.” One of our favourites is a pixelated space theme where your camera icon becomes a space ship and your browser becomes a planet.

To turn this mode on, you simply have to press and hold on the home screen and choose “Freestyle layout,” under “Change home screen layout.” There are currently eight themes to choose from, ranging from the childishly adorable to the slightly risqué.

5. Stock Android is good Android

By Igor Bonifacic

Beyond the stunning hardware, one of the most pleasant surprises the HTC 10 brings with it is its software experience. After HTC opened the Pandora’s box that was Android skinning with the introduction of Sense, the company has at long last toned many of its past excess in favour of simplicity and a strong adherence to Google’s own design principles.

There are fun flourishes here and there (as you can see above), but for the most part — with the exception of perhaps a Motorola phone — this is the closest you’ll get to stock Android without purchasing a smartphone from Google directly.

HTC has also replaced many of its own apps in favour of Google mainstays. Gone, for instance, is the company’s gallery app to be replaced by Google Photos. Moreover, the HTC apps that do remain have been redesigned to adhere to Google’s Material Design guidelines, which results in a more coherent experience across the device’s entire software library.

Whether this means updates to the device will be faster to arrive in the future remains to be seen, but for now the 10 is one of the best ways to experience the latest version of Google’s operating system.

6. You’ll want a great set of headphones to go along with it

Sound quality has long been a focus of HTC devices. This is a trend that continues with the 10, though perhaps not in the way most previous HTC device owners might expect.

Following the lead of the A9, the 10 abandons the BoomSound speakers that came to define so many of the company’s previous smartphones. Instead, it focuses on the device’s headphone sound experience. I don’t know about you, but I almost exclusively use my smartphone with a pair of good headphones, and the HTC 10 — more so than almost any other smartphone I’ve used in the past year — makes the most of a good in-ear setup.

Part of it is its more powerful internal sound hardware compared to other smartphones on the market; the other part is a nifty piece of pre-installed software that adjusts the sonic output of the phone based on the user’s hearing.

Using this software, the user can tune the output of their device by either answering a couple of quick questions about their listening habits, or by going through an EQ test.

Going through the EQ test, which takes a couple of minutes at most, is the preferred way to adjust the 10 sonic output. Basically, it involves listening to a bunch of tones, once with each ear, and turning down the volume until they’re barely audible. Once adjusted, there is a noticeable difference (I also found out my hearing is worse in my left ear).

For those that to listen to music and podcasts on their smartphone daily, the HTC 10, paired with a quality set of headphones, is a joy.

7. The ultimate selfie taker

While the HTC 10 has a rear-facing camera that is comparable with many of the latest flagship smartphones on the market, it is perhaps this phone’s front-facing camera that is its most standout feature from a photography standpoint. For the first time on a smartphone, a manufacturer has added optical image stabilization (OIS) to a front-facing camera. In almost all shooting conditions, the addition of OIS will result in sharper images, but where OIS particularly stands out as a great addition to a smartphone camera is in low light conditions.

In those situations, the tech allows the HTC 10’s front-facing camera to take a longer exposure (that is, keep its shutter open longer to allow more light to filter into the camera sensor), leading to night time shots that aren’t a noisy mess.

With so much of modern smartphone use centred around a device’s front-facing camera, it’s surprising this feature didn’t make it to other devices before the 10. For those who love taking selfies, rest assured the HTC 10 is a selfie machine.

8. Super slippery

By Patrick O’Rourke

Thankfully, despite how slippery the 10’s aluminum unibody is, I haven’t managed to drop it on rock hard pavement. The phone has, however, slid out of my hands at home on a number of occasions, landing on the faux hardwood floor of my apartment.

Even with its sloped edges HTC likes to refer to as “chamfered,” the 10 is difficult to get a firm grip on, though it is a joy to look at, especially in bright sunlight. Natural light bounces off the phones sleek design in a unique way that I haven’t seen with a smartphone before.

So if you’re thinking of picking up the HTC 10 and you’re like me, always wanting to keep your mobile device in pristine condition, opting for a case is a must. While we haven’t gone hands on with it yet, early impressions of the 10’s Ice View case seem to be very positive. It’s a little pricey though, coming in at $49 in Canada.

9. Capacitive buttons are frustrating

Capacitive buttons make sense in theory, but in practice, I’ve always found them a little frustrating. Whether I’m using the S7, S7 edge or HTC 10, I always find myself accidentally pressing them with my palm, or another part of my hand.

This occurs even more frequently when using the phone on an off-kilter angle, for example, while laying in bed browsing the internet. And while I have experienced this problem with past flagships featuring capacitive buttons, it occurs significantly more frequently with the 10, leading me to believe that its navigations buttons are perhaps a little too sensitive.

While far from a deal breaker, this is an issue I have a feeling many people will run into with the 10.

10. Battery life is superb

The HTC 10’s battery life is extremely impressive. I’ve actually managed to make it through an entire day and more thanks to its beefy 3,000 mAh size.

I’m not entirely sure why the 10’s battery seems to have lasted considerably longer than other high-end Android smartphones I’ve reviewed this year with similar milliampere-hour size, but it really does. In once instance, I actually forgot to charge the phone in the evening and somehow the battery lasted until about noon the following day, and this is as with moderate use.

The 10 also supports Qualcomm’s Quick Charge 3.0 technology – only the HTC 10, HP Elite x3, LG G5 and HTC One A9 utilize this tech – allowing the phone to charge to 100 percent in approximately an hour. While I personally prefer my smartphone’s battery to last longer instead of charge more quickly, the two features don’t need to be mutually exclusive. So those interested in charging their smartphone quickly, will definitely be pleased with Qualcomm’s Quick Charge 3.0 tech being supported by the 10.

Related reading: HTC 10 review: Back to basics

In a course team accessibility briefing last week, Richard Walker briefly mentioned a tool for automatically generating text descriptions of Statistics Canada charts to support accessibility. On further probing, the tool, created by Leo Ferres, turned out to be called iGraph-Lite:

… an extensible system that generates natural language descriptions of statistical graphs, particularly those created for Statistics Canada online publication, “The Daily”. iGraph-Lite reads a graph from a set of graphing programs, builds intermediate representations of it in several formats (XML, CouchDB and OWL) and outputs a description using NLG algorithms over a “string template” mechanism.

The tool is a C# application compiled as a Windows application, with the code available (unchanged now for several years) on Github.

The iGraph-Lite testing page gives a wide range of example outputs:

Increasingly, online charts are labeled with tooltip data that allows a mouseover or tab action to pop-up or play out text or audio descriptions of particular elements of the chart. A variety of “good chart design” principles designed to support clearer, more informative graphics (for example, sorting categorical variable bars in a bar chart by value rather than using an otherwise arbitrary alphabetic ordering) not only improves the quality of the graphic, it also makes a tabbed through audio description of the chart more useful. For more tips on writing clear chart and table descriptions, see Almost Everything You Wanted to Know About Making Tables and Figures.

The descriptions are quite formulaic, and to a certain extent represent a literal reading of the chart, along with some elements of interpretation and feature detection/description.

Here are a couple of examples – first, for a line chart:

This is a line graph. The title of the chart is “New motor vehicle sales surge in January”. There are in total 36 categories in the horizontal axis. The vertical axis starts at 110.0 and ends at 160.0, with ticks every 5.0 points. There are 2 series in this graph. The vertical axis is Note: The last few points could be subject to revisions when more data are added. This is indicated by the dashed line.. The units of the horizontal axis are months by year, ranging from February, 2005 to January, 2007. The title of series 1 is “Seasonally adjusted” and it is a line series. The minimum value is 126.047 occuring in September, 2005. The maximum value is 153.231 occuring in January, 2007. The title of series 2 is “Trend” and it is a line series. The minimum value is 133.88 occuring in February, 2005. The maximum value is 146.989 occuring in January, 2007.

And then for a bar chart:

This is a vertical bar graph, so categories are on the horizontal axis and values on the vertical axis. The title of the chart is “Growth in operating revenue slowed in 2005 and 2006”. There are in total 6 categories in the horizontal axis. The vertical axis starts at 0.0 and ends at 15.0, with ticks every 5.0 points. There is only one series in this graph. The vertical axis is % annual change. The units of the horizontal axis are years, ranging from 2001 to 2006. The title of series 1 is “Wholesale” and it is a series of bars. The minimum value is 2.1 occuring in 2001. The maximum value is 9.1 occuring in 2004.

One of the things I’ve pondered before was the question of textualising charts in R using a Grammar of Graphics approach such as that implemented by ggplot2. As well as literal textualisation of grammatical components, a small amount of feature detection or analysis could also be used to pull out some meaningful points. (It also occurs to me that the same feature detection elements could also then be used to drive a graphical highlighting layer back in the visual plane.)

Prompted by my quick look at the iGraph documentation, I idly wondered how easy it would be to pick apart an R ggplot2 chart object and generate a textual description of various parts of it.

A quick search around suggested two promising sources of structured data associated with the chart objects directly: firstly, we can interrogate the object return from calling ggplot() and it’s associated functions directly; and ggplot_build(g), which allows us to interrogate a construction generated from that object.

Here’s an example from the quickest of plays around this idea:

library(ggplot2)
g=ggplot(economics_long, aes(date, value01, colour = variable))
g = g + geom_line() + ggtitle('dummy title')

#The label values may not be the actual axis limits
txt=paste('The chart titled"', g$labels$title,'";',
          'with x-axis', g$labels$x,'labeled from',
          ggplot_build(g)$panel$ranges[[1]]$x.labels[1], 'to',
          tail(ggplot_build(g)$panel$ranges[[1]]$x.labels, n=1),
          'and y-axis', g$labels$y,' labeled from',
          ggplot_build(g)$panel$ranges[[1]]$y.labels[1], 'to',
          tail(ggplot_build(g)$panel$ranges[[1]]$y.labels,n=1), sep=' ')
if ('colour' %in% attributes(g$labels)$names){
  txt=paste(txt, '\nColour is used to represent', g$labels$colour)

  if ( class(g$data[[g$labels$colour]]) =='factor') {
    txt=paste(txt,', a factor with levels: ',
              paste(levels(g$data[[g$labels$colour]]), collapse=', '), '.', sep='')
  }
}

txt

#The chart titled "dummy title" with x-axis date labeled from 1970 to 2010 and y-axis value01 labeled from 0.00 to 1.00
#Colour is used to represent variable, a factor with levels: pce, pop, psavert, uempmed, unemploy."

So for a five minute hack, I’d rate that approach as plausible, perhaps?!

I guess an alternative approach might also be to add an additional textualise=True property to the ggplot() call that forces each ggplot component to return a text description of its actions, and/or features that might be visually obvious from the action of the component as well as, or instead of, the graphical elements. Hmm… maybe I should use the same syntax as ggplot2 and try to piece together something along the lines of a ggdescribe library? Rather than returning graphical bits, each function would return appropriate text elements to the final overall description?!

I’m not sure if we could call the latter approach a “transliteration” of a chart from a graphical to a textual rendering of its structure and some salient parts of a basic (summary) interpretation of it, but it feels to me that this has some merit as a strategy for thinking about data2text conversions in general. Related to this, I note that Narrative Science have teamed up with Microsoft Power BI to produce a “Narratives for Power BI” plugin. I imagine this will start trickling down to Excel at some point, though I’ve already commented on how simple text summarisers can be added into to Excel using simple hand-crafted formulas (Using Spreadsheets That Generate Textual Summaries of Data – HSCIC).

PS does anyone know if Python/matplotib chart elements also be picked apart in a similar way to ggplot objects? [Seems as if they can: example via Ben Russert.]

PPS In passing, here’s something that does look like it could be handy for simple natural language processing tasks in Python: polyglot, “a natural language pipeline that supports massive multilingual applications”; includes language detection, tokenisation, named entity extraction, part of speech tagging, transliteration, polarity (i.e. crude sentiment).

Reading or typing on your phone while you’re behind the wheel is insanely dangerous; it’s exactly as safe as driving with your eyes closed.

What’s that you say? “Everyone knows that”?

Then why are 40 percent of car accidents in the U.S. these days caused by drivers with phones in their hands? And why do 27 percent of all American adults (and 35 percent of teenagers) admit to texting while driving in the past 30 days?

Here’s a slightly safer idea: When you hear a text coming in, use Siri (on an iPhone) to check it for you. Just hold down your iPhone’s Home button to summon Apple’s digital assistant, and say, “Read my messages.” Your phone will then read them aloud, so you can judge whether or not they’re important enough to justify pulling over and stopping your car to reply.

If you have an Android phone, you can download an app that gives you the same kind of feature: ReadItToMe, for example, is free.

It's the oft-unspoken thought on many lips: America's in decline. The glory days are over, the train's left the station. So: is this a great decline? Unfortunately--probably. And I'd suggest that when you take a hard, serious look into the economy--when you voyage past it's superficial, largely irrelevant position in terms of budgets, "gross product", or "unemployment"--that great decline is deeper and darker than pundits, beancounters, and politicians think, want to admit, or even suspect. The great crisis is a story of structural decline: a decline that's hardwired into the patterns amongst this great machine's many parts. They've settled, over the last three decades and more, into fundamentally bad, toxic equilibria--where speculation precedes investment, model precedes reality, management and financial jargon is a substitute for real insight, cheap talk substitutes for hard work, and indulgence has replaced inspiration.

Rob Ford's posturing on transit will have *even more* dire consequences than you think: http://bit.ly/hnD87m (h/t @c_9) – Jonathan Goldsbie (goldsbie) http://twitter.com/goldsbie/status/13288746099154944