Shared posts

04 Sep 04:58

Introducing ‘gepetto’ — a Splash-like REST API to Headless Chrome

by hrbrmstr

It’s been over a year since Headless Chrome was introduced and it has matured greatly over that time and has acquired a pretty large user base. The TLDR on it is that you can now use Chrome as you would any command-line interface (CLI) program and generate PDFs, images or render javascript-interpreted HTML by supplying some simple parameters. It has a REPL mode for interactive work and can be instrumented through a custom websockets protocol.

R folks have had the decapitated🔗 package available almost since the launch day of Headless Chrome. It provides a basic wrapper to the CLI. The package has been updated more recently to enable the downloading of a custom Chromium binary to use instead of the system Chrome installation (which is a highly recommended practice).

However, that nigh-mundane addition is not the only new feature in decapitated.

Introducing gepetto

While it would have been possible to create an R wrapper for the Headless Chrome websockets API, the reality is (and this is just my opinion) that it is better to integrate with a more robust and community supported interface to Headless Chrome instrumentation dubbed puppeteer🔗. Puppeteer is a javascript module that adds high level functions on top of the lower-level API and has a massive amount of functionality that can be easily tapped into.

Now, Selenium works really well with Headless Chrome and there’s little point in trying to reinvent that wheel. Rather, I wanted a way to interact with Headless Chrome the way one can with [ScrapingHub’s Splash](https://github.com/scrapinghub/splash service. That is, a simple REST API. To that end, I’ve started a project called gepetto🔗 which aims to do just that.

Gepetto is a Node.js application which uses puppeteer for all the hard work. After seeing that such a REST API interface was possible via the puppetron proof of concept I set out to build a framework which will (eventually) provide the same feature set that Splash has, substituting puppeteer-fueled javascript for the Lua interface.

A REST API has a number of advantages over repeated CLI calls. First, each CLI call means more more system() call to start up a new process. You also need to manage Chrome binaries in that mode and are fairly limited in what you can do. With a REST API, Chrome loads once and then pages can be created at-will with no process startup overhead. Plus (once the API is finished) you’ll have far more control over what you can do. Again, this is not going to cover the same ground as Selenium, but should be of sufficient utility to add to your web-scraping toolbox.

Installing gepetto

There are instructions over at the repo on installing gepetto but R users can try a shortcut by grabbing the latest version of decapitated from Git[La|Hu]b and running decapitated::install_gepetto() which should (hopefully) go as smoothly as this provided you have a fairly recent version of Node.js installed along with npm:

The installer provides some guidance should thing go awry. You’ll notice gepetto installs a current version of Chromium for your platform along with it, which helps to ensure smoother sailing than using the version of Chrome you may use for browsing.

Working with gepetto

Before showing off the R interface, it’s worth a look at the (still minimal) web interface. Bring up a terminal/command prompt and enter gepetto. You should see something like this:

$ gepetto
🚀 Launch browser!
👍 gepetto running on: http://localhost:3000

NOTE: You can use a different host/port by setting the HOST and PORT environment variables accordingly before startup.

You can then go to http://localhost:3000 in your browser and should see this:

Enter a URL into the input field and press the buttons! You can do quite a bit just from the web interface.

If you select “API Docs” (http://localhost:3000/documentation) you’ll get the Swagger-gen’d API documentation for all the API endpoints:

The Swagger definition JSON is also at http://localhost:3000/swagger.json.

The API documentation will be a bit more robust as the module’s corners are rounded out.

“But, this is supposed to be an R post…”

Yes. Yes it is.

If you followed along in the previous section and started gepetto from a command-line interface, kill the running service and fire up your favourite R environment and let’s scrape some content!

library(rvest)
library(decapitated)
library(tidyverse)

gpid % 
  gep_active()
## [1] TRUE

Anything other than a “running” response means there’s something wrong and you can use the various processx methods on that gpid object to inspect the error log. If you were able to run gepetto from the command line then it should be fine in R, too. The gep() function build a connection object and gep_active() tests an API endpoint to ensure you can communicate with the server.

Now, let’s try hitting a website that requires javascript. I’ll borrow an example from Brooke Watson. The data for http://therapboard.com/ loads via javascript and will not work with xml2::read_html().

gepetto() %>% 
  gep_render_html("http://therapboard.com/") -> doc

html_nodes(doc, xpath=".//source[contains(@src, 'mp3')]") %>%  
  html_attr("src") %>% 
  head(10)
## [1] "audio/2chainz_4.mp3"        "audio/2chainz_yeah2.mp3"   
## [3] "audio/2chainz_tellem.mp3"   "audio/2chainz_tru.mp3"     
## [5] "audio/2chainz_unh3.mp3"     "audio/2chainz_watchout.mp3"
## [7] "audio/2chainz_whistle.mp3"  "audio/2pac_4.mp3"          
## [9] "audio/2pac_5.mp3"           "audio/2pac_6.mp3"

Even with a Node.js and npm dependency, I think that’s a bit friendlier than interacting with phantomjs.

We can render a screenshot of a site as well. Since we’re not stealing content this way, I’m going to cheat a bit and grab the New York Times front page:

gepetto() %>% 
  gep_render_magick("https://nytimes.com/")
##   format width height colorspace matte filesize density
## 1    PNG  1440   6828       sRGB  TRUE        0   72x72

Astute readers will notice it returns a magick object so you can work with it immediately.

I’m still working out the interface for image capture and will also be supporting capturing the image of a CSS selector target. I mention that since the gep_render_magick() actually captured the entire page which you can see for yourself (the thumbnail doesn’t do it justice).

Testing gep_render_pdf() is an exercise left to the reader.

FIN

The gepetto REST API is at version 0.1.0 meaning it’s new, raw and likely to change (quickly, too). Jump on board in whatever repo you’re more comfortable with and kick the tyres + file issues or PRs (on either or both projects) as you are wont to do.

04 Sep 04:58

Arbutus Corridor Official Development Plan Amendments

by Stephen Rees

The image and text below is taken from the latest Arbutus Greenway Newsletter from the City. I would expect that if you are deeply concerned about the Greenway that you would already have signed up for, but I suspect there might be a wider audience who will find this information of interest.

The Greenway currently ends at Fir St. The former CP right of way ran beyond this point northwards to West 1st Avenue where it was joined by the line from Terminal Avenue (that became The Olympic Line) and then went over False Creek on a trestle. The Olympic Line terminated at Granville Island and the land between the station and West 2nd Avenue was used to build a Starbucks. If the proposed amendment goes forward non-transportation use of this land would be allowed, and the potential to re-establish a through route following the former Interurban line could be lost.

POSTSCRIPT

I have received that following information from the City

“The intention is to connect the former Olympic Line with the Arbutus line, and the Citywide streetcar network plan is considering routing options to do that. We are not using the Option Lands due to engineering and safety constraints.”

SECOND POSTSCRIPT August 31

Loonie deal: City of Vancouver may resell portions of Arbutus corridor for $1

In the Georgia Straight

“In the purchase agreement between the city and CPR, Article 11 stipulates that if these properties, which have a total combined area of 0.62 hectare, are removed from Arbutus Corridor Official Development Plan, CPR will have the option to buy back the lands for $1.”

Arbutus map

In line with the Arbutus Greenway Design Vision and the Arbutus Greenway purchase agreement we have determined that the area between West 1st Ave and West 5th Ave is not required for future greenway purposes. Therefore, the City is proposing to:

Amending these bylaws will not change the current zoning of the properties.

The proposed changes are supported by the Arbutus Greenway Design Vision which has highlighted a number of safe and accessible alternatives for the proposed greenway extension routes to False Creek and Granville Island.

To learn more about the amendments please visit our website here. You can also read the July 24, 2018 Policy Report.

 

Share Your Thoughts with Council on the Arbutus Corridor ODP

Vancouver City Council will hold a Public Hearing on the Arbutus Corridor ODP Amendment on:

  • Date: Wednesday, September 5, 2018
  • Time: 6:00 pm
  • Place: Council Chamber, Third Floor, City Hall

There are two ways you can participate:

 

04 Sep 04:57

The One Question You Need to Answer as a New Product Leader

by Cliff Gilley

Outline The One Question What Could Inhibit My Ability to Succeed? Is it people, process, policies, vision, etc? What do I have control over and what do I not? Do I have the influence i need to change things for the better? Prioritizing Potential Problem Areas Eisenhower Model – Urgency vs. Importance Avoid the time vampires and zombie efforts It’s not you, it’s me… Make a Plan to Tackle...

Source

04 Sep 04:57

They Promised Us Jetpacks

by ekai

Here’s your real life Tony Stark flying his jet suit around Hiller yesterday. Richard Browning of Gravity Industries, British inventor of the Daedalus Flight Pack, is as close to Iron Man as anyone has gotten. Just don’t ask how he scratches his nose.

04 Sep 04:57

Please let’s finally kill logins and passwords

by Doc Searls

How would you feel if you had been told in the early days of the Web that in the year 2018 you would still need logins and passwords for damned near everything.

Your faith in the tech world would be deeply shaken, no?

And what if you had been told that in 2018 logins and passwords would now be required for all kinds of other shit, from applications on mobile devices to subscription services on TV?

Or worse, that in 2018 you would be rob-logged-out of sites and services frequently, whether you were just there or not, for security purposes — and that logging back in would often require “two factor” authentication, meaning you have to do even more work to log in to something, and that (also for security purposes) every password you use would not only have be different, but impossible for any human to remember, especially when average connected human now has hundreds of login/password combinations, many of which change constantly?

Would you not imagine this to be a dystopian hell?

Welcome to now, folks. Our frog is so fully boiled that it looks like Brunswick stew.

Can we please fix this?

Please, please, please, tech world: move getting rid of logins and passwords to the top of your punch list, ahead of AI, ML, IoT, 5G, smart dust, driverless cars and going to Mars.

Your home planet thanks you.

[Addendum…] Early responses to this post suggest that I’m talking about fixing the problem at the superficial level of effects. So, to clarify, logins and passwords are an effect, and not a cause of anything other than inconvenience and annoyance. The causes are design and tech choices made long ago—choices that can be changed.

Not only that, but many people have been working on solving the identity side of this thing for many years. In fact we’re about to have our 27th Internet Identity Workshop in October at the Computer History Museum. If you want to work on this with other people who are doing the same, register here.

 

04 Sep 04:56

Don’t come between a social commenter and his Pecker

by Josh Bernoff

The Boston Globe published an AP story about David Pecker, CEO of the parent company of The National Enquirer, who kept the secrets of Donald Trump and other media figures in a safe as leverage. This kind of story is catnip to the mostly liberal Globe readers, but there’s a problem — the Globe’s content … Continued

The post Don’t come between a social commenter and his Pecker appeared first on without bullshit.

04 Sep 04:56

Work Your Career: Get What You Want from Your Social Sciences or Humanities PhD (my reading notes)

by Raul Pacheco-Vega

A few weeks ago, University of Toronto Press sent me a complimentary copy of “Work Your Career: Get What You Want from Your Social Sciences or Humanities PhD“, a book written by Dr. Loleen Berdahl and Dr. Jonathan Malloy. It’s the first time I have gotten a complimentary copy of a book of this type and while UTP did not ask me to review it, I figured that it would be good for those who follow me on Twitter to get a sense of what the book offered.

I stand by what I said on the above shown tweet: I don’t think anybody who wants to work in academia should do a doctoral degree right now. The market is atrocious and we don’t anticipate that it will improve any time soon. But since there are thousands of PhD students worldwide, and I DO want them to succeed, I will continue to write about how to do a doctoral degree as much as I can.

There are a number of things that make “Work Your Career” worth checking out. It’s an honestly written book, and it does offer lots of good suggestions. It also speaks to challenges doctoral supervisors face. And as always with books like these, there are important gaps. Berdahl and Malloy cover grant proposals (which many books don’t), but they skim over the thesis writing process (which is what many books DO cover, and one of the places where most students get stuck). As I said, I like this book as a backbone/workbook to work from. his book is an excellent and refreshing addition to my collection of books on how to help doctoral students.

I am glad they do offer suggestions to faculty who supervise doctoral dissertation, but if I may be so bold as to say, I think Berdahl and Malloy should have scolded supervisors more! At any rate, a good read, which I am sure will help my own doctoral students.

04 Sep 04:56

Review-Managing the Unexpected. Karl Weick and Kathleen Sutcliffe

by Jim

Cover - Managing the UnexpectedManaging the Unexpected: Sustained Performance in a Complex World. Third Edition. Karl Weick and Kathleen Sutcliffe

Conventional wisdom has it that the job of management is to “plan the work and work the plan.” Wall Street loves to see steady growth in reported earnings and managers learned to give Wall Street what they wanted. Sadly, the world is more complicated than Wall Street analysts would like to believe

Weick and Sutcliffe take an intriguing route in this book—now in its third edition. They ask what lessons might be found in the experiences and practices of high-reliability organizations. What’s an HRO? Flight-deck operations on an aircraft carrier. Nuclear power plant control room. Fire fighters. Cockpit operations on a 757. Common to all of these is a tension between routine operations and potential disaster. All face the problem of how to take ordinary, fallible, human beings and create organizations that work; organizations that operate reliably day-in and day-out, avoiding disasters for the most part, and coping effectively when they do.

While studying HROs is fascinating in its own right, Weick and Sutcliffe successfully connect lessons from HROs to the challenges of running more mundane organizations. The world is throwing more change and complexity at all of us. The problem is that most organizations, by design, are focused on the routine and the predictable. They effectively deny and eliminate the unexpected and the unpredictable, which works well in a stable environment. Less so in today’s world.

The core of the argument is that high-reliability organizations know how to operate mindfully as well as mindlessly (which is the default for most organizations). Mindfulness in this context breaks down into five characteristics focused toward two objectives.

The first objective is anticipating the unexpected. Three characteristics contribute to that:
1. preoccupation with failure,
2. reluctance to simplify interpretations, and
3. sensitivity to operations.

Each of these is a way to detect or amplify weak signals soon enough to do something useful. As Weick points out “unexpected” implies something that has already happened that wasn’t anticipated. You want to figure out that something relevant has happened as soon as possible. The problem is that stuff is always happening and we couldn’t get through an average day without ignoring most of it. The challenge is to differentiate between signal and noise.

One way of separating signal from noise is ignoring the routine. That’s why we call it routine. The trick is to avoid getting caught up with expanding the definition of routine so we can ignore more of it. Take a look back at the Challenger launch failure. Before the catastrophic failure, there had been a series of smaller failures of the O-rings. Each of these “failures” was explained away in the sense that the post-flight review processes concluded that the “minor” failures were actually evidence that the system was working as designed.

The issue is attitudinal. Most organizations, NASA included, treat earlier minor failures as “close calls” and ultimately interpret them as evidence of success. An HRO takes the same data but treats it as a “near miss.” Then the analysis focuses on how to avoid even a near miss the next time round. Small failures (weak signals) are sought out and treated as opportunities to learn instead of anomalies to be explained away.

If anticipating and recognizing the unexpected is the first objective, containing the unexpected in the second. Here the relevant characteristics are a commitment to resilience and a deference to expertise.

Resilience is the term of choice for Weick and Sutcliffe because it highlights key aspects of organizations that typically are denied or glossed over. It acknowledges the human fallibility is unavoidable, that error is pervasive, and reminds us that the unexpected has already happened. A strategy of resilience focuses on accepting that some small error has already occurred and on working to contain the consequences of that error while they are still small and manageable. To be resilient requires an organization to be serious about such practices as not shooting messengers.

Weick and Sutcliffe cite one example from carrier operations where operations were shutdown when a junior member of the crew reported a missing tool. Instead of punishing this person for losing the tool, the captain rewarded them even though operations were suspended while the missing tool was found. Dealing with the small problem was rewarded because everyone recognized the greater risk of ignoring it. The same issues exist in all organizations, although the responses are generally quite different. The result, of course, is that problems are ignored until they are too big both to ignore and, typically, to deal with.

The second dimension to containing problems while they are small and tractable is knowing how to defer to expertise. Expertise can correlate with experience (as long as the experience is relevant). It does not generally correlate with hierarchical rank. Successfully seeking out and benefitting from expertise takes two things. Those up the chain of command must be ready to set examples. Those on the lines need to be assertive about the expertise they have to offer, which includes developing a clearer sense for the expertise that they have.

While the world that Weick and Sutcliffe describe is quite different than the organizations we are accustomed to, it does not require wholesale organizational change programs to get there. The mindfulness that they describe–of anticipating and containing the unexpected–can be practiced at both the individual and small group level. If their analyses and recommendations are sound (they are), then those who practice this mindfulness will gradually take over on the basis of improved performance.

The post Review-Managing the Unexpected. Karl Weick and Kathleen Sutcliffe appeared first on McGee's Musings.

04 Sep 04:56

Constructing a Data Analysis

This week Hilary Parker and I have started our “Book Club” on Not So Standard Deviations where we will be discussing Nigel Cross’s book Design Thinking: Understanding How Designers Think and Work. We will be talking about how the work of designers parallels the work of data scientists and how many of the principles developed in design port over so well to data analysis. While data visualization has always taken cues from design, I think much broader aspects of data analysis could benefit from the work studying design. At any rate, I think this is a topic that should be discussed more amongst statisticians and data analysts.

One of the first revelations I’ve had recently is realizing that data analyses are not naturally occurring phenomena. You will not run into a data analysis while walking out in the woods. Data analyses must be created and constructed by people.

One way to think about a data analysis is to think of it as a product to be designed. Data analysis is not a theoretical exercise. The goal is not to reveal something new about the world or to discover truth (although knowledge and truth may be important by-products). The goal of data analysis is to produce something useful. Useful to the scientist, useful to the product manager, useful to the business executive, or useful to the policy-maker. In that sense, data analysis is a fairly down-to-Earth activity.

Producing a useful product requires careful consideration of who will be using it. Good data analysis can be useful to just about anyone. The fact that many different kinds of people make use of data analysis is not exactly news, but what is new is the tremendous availability of data in general.

If we consider a data analysis as something to be designed, this provides for us a rough road map for how to proceed.

Questioning the Question

A favorite quotation from John Tukey, legendary statistician and data analyst at Princeton, is

Far better an approximate answer to the right question, which is often vague, than the exact answer to the wrong question, which can always be made precise.

What does these words mean in the context of data analysis? In data analysis, we often start with a dataset or a question. But good data analysts do not solve the problem that is handed to them. The reason is not necessarily arrogance. often the problem as initially stated is only a first attempt. And that’s okay.

A good data analyst recognizes that the problem itself requires examination. For example, someone might ask “Is air pollution bad for your health?” That’s a great question, and a critical one for public policy, but it’s difficult to map to a specific data analysis. There are many different types of air pollution and there are many health outcomes we might be worried about. Prioritizing and refining the original problem is a key initial step for any data analysis. In my experience, this process usually leads to a question that is more meaningful and whose answer could lead to clear action.

The first job of the data analyst is to discover the real underlying problem. The fact that the problem we end up with may not be the problem we started with is not anyone’s fault in particular. It’s just the nature of things. Often, scientific collaborators will come to my office essentially just to talk. They come in with a clear question, but as I probe and ask questions–“What kinds of data are available?”; “What action might result in answering this question?”; “What resources are available to do this analysis?”; “Is it possible to collect new data?”–the question can shift and evolve. Good collaborators are not offended by this process, but rather appreciate it as it hones their thinking.

The bad collaborator comes in with the goal of handing off a question and waiting for the solution to appear. I have seen my fair share of these and it almost never works except perhaps with the most trivial problems. The process of designing a good data analysis cannot be modularized where we cleanly move from question, to data, to analysis, and to results, with each person involved doing their job and not talking to anyone else. One may wish it were so, because it would be much simpler this way, but wishing doesn’t make it so. This initial discussion of figuring out the right problem is an important part of designing a data analysis. But if one has a thorough discussion, we’re not done questioning the question yet.

Sometimes a problem does not become clear until we’ve attempted to solve it. A data analyst’s job is to propose solutions to a problem in order to explore the problem space. For example, in the above air pollution example, we might first express interest in looking at particulate matter air pollution. But when we look at the available data we see that there are too many missing values to do a useful analysis. So we might switch to looking at ozone pollution instead, which is equally important from a health perspective.

At this point it is important that you not get too far into the problem where a lot of time or resources have to be invested. Making that initial attempt to look at particulate matter probably didn’t involve more than downloading a file from a web site, but it allowed us to explore the boundary of what might be possible. Sometimes the proposed solution “just works”, but more often it raises new questions and forces the analyst to rethink the underlying problem. Initial solution attempts should be “sketches”, or rough models, to see if things will work. The preliminary data obtained by these sketches can be valuable for prioritizing the possible solutions and converging on the ultimate approach.

This initial process of questioning the question can feel frustrating to some, particularly for those who have come to the data analyst to get something done. Often to the collaborator, it feels like the analyst is questioning their own knowledge of the topic and is re-litigating old issues that have previously been resolved. The data analyst should be sensitive to these concerns and explain why such a discussion needs to be had. The analyst should also understand that the collaborator is an expert in their field and probably does know what they’re talking about. A better approach for the analyst might be to frame this process as a way for the analyst to learn more about the subject matter at hand, rather than simply challenging long-standing assumptions or beliefs. This has the co-benefit of actually being true, since the analyst is likely not an expert in the data before them. By asking simple questions in an attempt to learn the subject matter, often collaborators will be forced to re-visit some of their own ideas and to refine their thinking about the topic.

Engineering the Solution

Once we have refined the question we are asking and have carefully defined the scope of the problem we are solving, then we can proceed to engineer the solution. This will similarly be an iterative process, but we will be iterating over different things. At this point we will need a reasonably precise specification of the problem so that tools and methodologies can be mapped to various aspects of the problem. In addition, a workflow will need to be developed that will allow all of the interested parties to participate in the analysis and to play their proper role.

The analyst will need to setup the workflow for the analysis and adapt it to the various needs and capabilities of the collaborators. Every project will likely have a different workflow, especially if every project has a different set of collaborators involved. This is not just a comment about the tools involved for managing the workflow, but more generally about how information is exchanged and relayed to different people. Sometimes the analyst is a “central hub” through which all information is passed and sometimes it’s more of a “free for all” where everyone talks to everyone else. There’s no one right approach but it’s important that everyone understands what approach is being used.

The analyst is also responsible for selecting the methodologies for obtaining, wrangling, and summarizing the data. One might need to setup databases to store the data or retrieve it. Statistical methodologies might be a t-test for a two group comparison or a regression model to look at multivariate relationships. The selection of these tools and methodologies will be guided in part by the specification of the problem as well as the resources available and the audience who will receive the analysis. Identifying the optimal methodological approach, given the constraints put on the problem, is the unique job of the analyst. The analyst may need to delegate certain tasks based on the expertise available on the team.

Wrangling the Team

Any interesting or complex data analysis will likely involve people from different disciplines. In academia, you might be working with a biologist, an engineer, a computer scientist, and a physician at the same time. In business, you might need to interact with finance, marketing, manufacturing, and data engineering in a given analysis. A difficult part of an analyst’s job is managing all of these people’s interests simultaneously while integrating them into the final analysis.

The challenge facing the analyst is that each discipline is likely to think that their interests take priority over everyone else’s. Furthermore, collaborators are likely to think that the problems relevant to their discipline are the most important problems to address. However, the analyst cannot accept that every discipline is “tied for first place”; priorities must be set and a reasonable assessment must made regarding which problems should addressed first, second, and so on. This is a delicate operation on the part of the analyst and doing it successfully requires open communication and good relationships with the collaborators.

The fight for priority is also why it can be so difficult for a data analysis to be modularized. If an analysis is passed from person to person, then each person is inclined to take their specific disciplinary perspective on the problem and ignore the others. This is a naturally occurring phenomenon and it is the analyst’s job to prevent it from happening, lest the analysis be watered down to be incoherent or too tightly focused on one aspect. Ultimately, the analyst must take responsibility to see the “big picture” of the analysis, weigh each collaborator’s views, and choose a path that is acceptable to everyone.

It’s not an easy job.

04 Sep 04:54

Sneak peek at upcoming update and Pixelmator Pro beta program

by Pixelmator Team

Today is a very exciting day for us at the Pixelmator Team — we’re launching a Pixelmator Pro public beta program, giving you a way to test upcoming features and, in the process, help us make Pixelmator Pro even better!

To mark the occasion, we’ve also decided to show you a few things from the upcoming update, which you’ll be able to try in this beta. Point updates are usually a little smaller than major releases, but in this one, we’ve packed in tons of great new features. For example, new Light Leak and Bokeh effects, an Auto Selective Color adjustment, a way to copy Effects and Adjustments between layers, and (really!) a whole lot more.

New Light Leak and Bokeh effects

Two of the biggest new features are the new Light Leak and Bokeh effects. The gorgeous Light Leak effect was always a favorite in the original Pixelmator it’s now coming to Pixelmator Pro, giving you an easy way to add a sunny, old-school feel to any of your photos.

The beautiful new Bokeh effect was inspired by the Bokeh effect in Pixelmator for iOS and we think you’re really going to love it in Pixelmator Pro too! There are eight stunning Bokeh styles to choose from and, because Pixelmator Pro is a nondestructive editor, you can easily add multiple bokehs to the same image and edit each one individually.

Tons of other new features

The full release notes for this update are going to be pretty huge, but here are a few of the most awesome new things in Pixelmator Pro 1.1.4:

  • • Tons of new effects. We’re adding Spin Blur, Clouds, Noise, Threshold, Mask to Alpha, and several effects from the Tile and Distortion categories of the original Pixelmator.
  • • There will now be a way to switch histogram type when using the Color Adjustments tool. Simply Control-click the histogram and choose a different type (RGB, Luminance, or Colors).
  • • We’ll be adding an ML-powered Auto button to the Selective Color adjustment. Now you can selectively improve individual colors in an image using the awesome machine learning technologies we’ve been working on and talked about in the last blog post!
  • • You’ll now be able to copy all effects or adjustments from one layer and paste them onto another with a single click.

Pixelmator Pro beta program

So, this is going to be a fantastic update and, if you’re up for it, you can get your hands on it early by checking out the public beta. We’re opening it up to all owners of Pixelmator Pro as well as owners of the original Pixelmator.

Beta Program

Sign Up

P.S. We’re still big fans of big surprises, so we’re not planning to test absolutely every new Pixelmator Pro feature publicly but we’re really excited to see how this goes!

04 Sep 04:44

Support Localization – Top 20 Sprint and More

by Michał

Hello, current and future Mozillians :-)

It’s time to update you about the current status of the localization clean up initiative proposed a while ago. After an initial outreach to hundreds of previously registered contributors around Mozilla, small groups of still active localizers were asked to try and reach the goal of localizing the Top 20 articles into their language.

Arabic

With over 70 revisions submitted since the start of the Top 20 sprint, the Arabic locale is looking much better than in the last few months, mostly thanks to the amazing work of Ahmad who single-handedly is leading the Arabic content safely towards this month’s goal and beyond. I would also like to mention Tess and Ruba for their contributions so far. شكرا لكم!

Bengali

Bengali is slowly rising thanks to the efforts of Nazir (thank you!) and hopefully will reach the Top 20 goal by the end of August. I hope to get more contributors involved thanks to Nazir’s very positive example. ধন্যবাদ!

Bosnian

The summer break is definitely not making it easy for our small team of localizers working on improving the situation of Bosnian localization across our Support site. Thanks to kicin the way to the Top 20 is a bit shorter, so we’re all looking forward to a final push towards this month’s goal. Hvala ti!

Gujarati

Kartik is trying to give Gujarati a bit more attention, but without additional help from fellow localizers it may be hard to get things moving. The upcoming regional l10n hackathon may have a positive influence on that! આભાર, Kartik!

Hindi

With Hindi, I would like to call out Mahtab‘s great leadership and hard work on getting started with the whole localization campaign and his ownership of the review process. Not quite there yet, but definitely a great start. धन्यवाद, Mahtab!

Tamil

With Tamil, we are still waiting to see whether there will be enough time and energy available for Support pages to be localized.

Telugu

Two localizers – Dinesh and Veveen) have been continuously working on improving the situation in the Knowledge Base and it shows. I am pretty sure that with the final week ahead, they can get to the Top 20 and then start September setting their sights on the Top 50. ధన్యవాదాలు!

That’s right, once the localizers get to the Top 20, the next step is updating the Top 50 (including the Top 20 already localized, so actually… it could be the Top Additional 30 ;-)).

Of course, there is much more happening in the localization world around Support than just this one initiative. I would like to highlight the great work happening in different locales around support.mozilla.org and call out some of the most active people in the community at the moment.

Chinese (simplified)

Wengsheng has been one of the most prolific editors and reviewers in the recent years and continues to contribute with enthusiasm. 谢谢!

Greek

JimSpentzos is one of our quiet superheroes coming back after the summer break. I hope that with his leadership and example there can be more localizers joining his efforts soon and bringing Greek support for Mozilla’s products to everyone who needs it. Ευχαριστώ!

Hungarian

Kálmán is also still active for Hungarian and keeps making sure that users can find the most important information about Firefox (and more) in their language on our site. Köszönöm!

Polish

A Polish community site moderator by day, Krzysztof does not stop there and (among many other things) translates Support pages into Polish – and I am very grateful I can collaborate on this with him. Dziękuję!

Portuguese

Alberto, Claudio, and (especially nowadays) Manuela are a trio of localization greats that do not stop surprising me when it comes to making sure that Portuguese speakers find solutions to their potential problems with Firefox. Obrigado!

Slovenian

Lan has been around for a “mere” five years, but in that time has contributed to many projects around Mozilla with his localization skills, so I am extremely happy he still finds time to contribute to Support as well. Hvala vam!

Swedish

Andreas is yet another Mozillian that I need to mention and can only applaud for his dedication and involvement over the years. Swedish may not be the most used language in Europe, but with Andreas it’s in good hands. Tack!

Turkish

Ömer has been with us since last year and has already made his mark on the Turkish Knowledge Base, working together with Selim. With those two at the helm, I hope to see Mozilla reach even more users in Anatolia and beyond :) Teşekkür ederim!

So many languages, so many people, so many different stories and places they could tell… But I am very, very happy that a part of those stories would be about their involvement in making the open and helpful web a better place with Mozilla. I hope we will see more people inspired by them, joining the Mozillian ranks to keep the internet multilingual.

Speaking of which, I would like to invite everyone interested in localization at Mozilla to follow the news and updates posted on the official Mozilla L10n Blog – you may see some more Support-related content out there soon ;-)

I wish you all a great weekend! Keep rocking the helpful web!

04 Sep 04:44

Weighing the risk of moderate alcohol consumption

by Nathan Yau

A research study on mortality and alcohol consumption is making the rounds. Its main conclusion is that all alcohol consumption is bad for you, because of increased risk. David Spiegelhalter, the chair of the Winton Centre for Risk and Evidence Communication, offers a different interpretation of the data:

Let’s consider one drink a day (10g, 1.25 UK units) compared to none, for which the authors estimated an extra 4 (918–914) in 100,000 people would experience a (serious) alcohol-related condition.

That means, to experience one extra problem, 25,000 people need to drink 10g alcohol a day for a year, that’s 3,650g a year each.

To put this in perspective, a standard 70cl bottle of gin contains 224 g of alcohol, so 3,650g a year is equivalent to around 16 bottles of gin per person. That’s a total of 400,000 bottles of gin among 25,000 people, being associated with one extra health problem. Which indicates a rather low level of harm in these occasional drinkers.

Therefore:

The paper argues that their conclusions should lead public health bodies “to consider recommendations for abstention”.

But claiming there is no ‘safe’ level does not seem an argument for abstention. There is no safe level of driving, but government do not recommend that people avoid driving.

Come to think of it, there is no safe level of living, but nobody would recommend abstention.

Burn.

See also Spiegelhalter talk about weighing risk against benefits in a video from 2010.

Tags: alcohol, mortality, risk

04 Sep 04:44

Test Riding a Cargobike Electric Classic

by peter@rukavina.net (Peter Rukavina)

Malmö, like its Danish cousin across the sound Copenhagen, is a bicycling city, and the number and diversity of bicycles inspires awe, especially when compared to Charlottetown, where cycling remains the preserve, primarily, of the young, the car-less and the sporty.

Ever since my first visits to the area a dozen years ago, I’ve looked longingly at the cargo bikes that are used to haul around groceries, IKEA purchases and children. Nine years ago, when Oliver was 8 years old, I test rode a Nihola child-carrying bike with an eye, perhaps, to using it as a family transporter, but nothing came of it. But the candle burns as bright in me now as it did then, and finding myself in Malmö again, this time with Oliver, prompted a return to the notion.

I was fortunate in this regard to have stumbled across Cargo Bike of Sweden online; it’s based just east of the central train station and an easy bus ride from our apartment, and so we headed there this afternoon.

We were placed in the care of the personable Nils, who showed us first the Cargobike Classic Electric, a 6-speed cargo bike with the classic box-in-front design and a 250 watt electric motor for pedal assist. The box was roomy enough that Oliver could fit inside it comfortably, and we could likely squeeze Ethan in as well:

Cargo Bike Electric Classic

For such an ungainly-looking elephant, it was a joy to ride, and the electric assist allowed us to zip along at a comfortable pace. Oliver reports that the front bucket was comfortable and stable.

After a test drive of the electric assist, Nils asked if we wanted to test ride the non-electric version, and I took him up on it so that I could compare. As you might expect, it felt like pedaling a Sherman tank, and was the best advertisement for the wonders of electric assist you could imagine.

Here’s Oliver in the non-electric model, looking suitably unimpressed (although still comfortable):

Oliver in a Cargo Bike

The Cargobike Classic Electric sells for 21,995 SEK ($3142 Canadian); because of a generous rebate from the Swedish state, however, Swedes pay only 16,496 SEK ($2357 Canadian), a savings of 25%. That seems like an eminently reasonable price given what standard e-bikes are selling for. Cost to ship to Canada would need to be factored in, of course (the bike comes in two large boxes, one for the bucket and one for the bike; together they weigh 50 kg).

Our cycling season in Charlottetown is shorter than it is in Malmö where you can almost cycle all year round, but this is still an attractive option, and while we might not get it together for this season, perhaps for next.

04 Sep 04:40

Review of Chrome Cycling Shoes

by Average Joe Cyclist

Chrome cycling shoes are excellent quality, one of a kind cycling shoes. Best of all, they are waterproof. They are well worth the investment if you want comfortable cycling shoes you can wear both on and off the bike, while looking good!

The post Review of Chrome Cycling Shoes appeared first on Average Joe Cyclist.

04 Sep 04:40

Bias and Hiring: How We Hire UX Researchers

by Sharon Bautista

This year, the Firefox User Research team is planning to add two new researchers to our group. The job posting went live last month, and after just a few weeks of accepting applications, we had over 900 people apply.

Current members of the Firefox User Research Team fielded dozens of messages from prospective applicants during this time, most asking for informational meetings to discuss the open role. We decided as a team to decline these requests across the board because we did not have the bandwidth for the number of meetings requested, and more importantly we have spent a significant amount of time this year working on minimizing bias in our hiring process.

We felt that meeting with candidates outside of the formal hiring process would give unfair advantage to some candidates and undermine our de-biasing work. At the same time, in alignment with Mozilla’s values and to build on Mozilla’s diversity and inclusion disclosures from earlier this year, we realized there was an opportunity to be more transparent about our hiring process for the benefit of future job applicants and teams inside and outside Mozilla thinking about how they can minimize bias in their own hiring.

Our Hiring Process Before This Year

Before this year, our hiring process consisted of a number of steps. First, a Mozilla recruiter would screen resumes for basic work requirements such as legal status to work in the regions where we were hiring and high-level relevant work experience. Applicants with resumes that passed the initial screen would then be screened by the recruiter over the phone. The purpose of the phone screen was to verify the HR requirements, the applicant’s requirements, work history, and most relevant experience.

If the applicant passed the screen with the recruiter, two members of the research team would conduct individual phone screens with the applicant to understand the applicant’s experience with different research methods and any work with distributed teams. Applicants who passed the screen with the researchers would be invited to a Mozilla office for a day of 1:1 in-person interviews with researchers and non-researchers and asked to present a research exercise prepared in advance of the office visit.

Steps to hiring a UX researcher at Mozilla, from resume screen to hiring team debrief

This hiring process served us well in several ways. It involved both researchers and roles that interact with researchers regularly, such as UX designers and product managers. Also, the mix of remote and in-person components reflected the ways we actually work at Mozilla. The process overall yielded hires — our current research team members — who have worked well together and with cross-functional teams.

However, there were also a lot of limitations to our former hiring process. Each Mozilla staff person involved determined their own questions for the phone and in-person components. We had a living document of questions team members liked to ask, but staff were free to draw on this list as little or as much as they wanted. Moreover, while each staff person had to enter notes into our applicant tracking system after a phone screen or interview with an applicant, we had no explicit expectations about how these notes were to be structured. We were also inconsistent in how we referred to notes during the hiring team debrief meetings where final decisions about applicants were typically made.

Our New Hiring Process: What We’ve Done

Our new hiring process is a work in progress. We want to share the strides we have made and also what we would still like to do. Our first step in trying to reduce bias in our hiring process was to document our current hiring process, which was not documented comprehensively anywhere, and to try and identify areas for improvement. Simultaneously, we set out to learn as much as we could about bias in hiring in general. We consulted members of Mozilla’s Diversity and Inclusion team, dug into materials from Stanford’s Clayman Institute for Gender Research, and talked with several managers in other parts of Mozilla who had undertaken de-biasing efforts for their own hiring. This “discovery” period helped us identify a number of critical steps.

First, we needed to develop a list of essential and desired criteria for job candidates. The researcher job description we had been using reflected many of the criteria we ultimately kept, but the exercise of distilling essential and desired criteria allowed current research team members to make much that was implicit, explicit.

Team members were able to ask questions about the criteria, challenge assumptions, and in the end build a shared understanding of expectations for members of our team. For example, we previously sought out candidates with 1–3 years of work experience. With this criteria, we were receiving applications from some candidates who only had experience within academia. It was through discussing how each of our criteria relates to ways we actually work at Mozilla that we determined that what was even more essential than 1–3 years of any user research experience was that much experience specifically working in industry. The task of distilling our hiring criteria was not necessarily difficult, but it took several hours of synchronous and asynchronous discussion — time we all acknowledged was well-spent because our new hiring process would be built from these agreed-upon criteria.

Next, we wrote phone screen and interview questions that aligned with the essential and desired criteria. We completed this step mostly asynchronously, with each team member contributing and reviewing questions. We also asked UX designers, content strategists, and product managers that we work with to contribute questions, also aligned with our essential and desired criteria, that they would like to ask researcher candidates.

The next big piece was to develop a rubric for grading answers to the questions we had just written. For each question, again mostly asynchronously, research team members detailed what they thought were “excellent,” “acceptable,” and “poor answers,” with the goal of producing a rubric that was self-evident enough that it could be used by hiring team members other than ourselves. Like the exercise of crafting criteria, this step required as much research team discussion time as writing time. We then took our completed draft of a rubric and determined at which phase of the hiring process each question would be asked.

Additionally, we revisited the research exercise that we have candidates complete to make its purpose and the exercise constraints more explicit. Like we did for the phone screen and interview questions, we developed a detailed rubric for the research exercise based on our essential and desirable hiring criteria.

Most recently, we have turned our new questions and rubrics into worksheets, which Mozilla staff will use to document applicants’ answers. These worksheets will also allow staff to document any additional questions they pose to applicants and the corresponding answers, as well as questions applicants ask us. Completed worksheets will be linked to our applicant tracking system and be used to structure the hiring team debrief meetings where final decisions about leading applicants will be made.

From the work we have done to our hiring process, we anticipate a number of benefits, including:

  • Less bias on the part of hiring team members about what we think of as desirable qualities in a candidate
  • Less time spent screening resumes given the established criteria
  • Less time preparing for and processing interviews given the standardized questions and rubrics
  • Flexibility to add new questions to any of the hiring process steps but more attention to how these new questions are tracked and answers documented
  • Less time on final decision making given the criteria, rubrics, and explicit expectations for documenting candidates’ answers

Next Steps

Our Mozilla recruiter and members of the research team have started going through the 900+ resumes we have received to determine which candidates will be screened by phone. We fully expect to learn a lot and make changes to our hiring process after this first attempt at putting it into practice. There are also several other resource-intensive steps we would like to take in the near future to mitigate bias further, including:

  • Making our hiring process more transparent by publishing it where it would be discoverable (for instance, some Mozilla teams are publishing hiring materials to Github)
  • Establishing greater alignment between our new process and the mechanics of our applicant tracking system to make the hiring process easier for hiring team members
  • At the resume screening phase, blinding parts of resumes that can contribute to bias such as candidate names, names of academic institutions, and graduation dates
  • Sharing the work we have done on our hiring process via blog posts and other platforms to help foster critical discussion

Teams who are interested in trying out some of the exercises we carried out to improve our hiring process are welcome to use the template we developed for our purposes. We are also interested in learning how other teams have tackled bias in hiring and welcome suggestions, in particular, for blinding when hiring people from around the world.

We are looking forward to learning from this work and welcoming new research team members who can help us advance our efforts.

Thank you to Gemma Petrie and Mozilla’s Diversity & Inclusion Team for reviewing an early draft of this post.

Also published on the Firefox UX blog


Bias and Hiring: How We Hire UX Researchers was originally published in Firefox User Experience on Medium, where people are continuing the conversation by highlighting and responding to this story.

04 Sep 04:40

the missing user data platform

(update 20 Nov 2018: copy edit, add a link to Dr. Johnny Ryan's CNIL article)

Today's web advertising relies on 1990s browser behavior—most browsers fail to protect users from being tracked from site to site, and advertisers are used to taking advantage of that old defect. But because browsers do user research and respond to what users want, that's changing. Browsers are making it harder to track users from site to site without their permission. Along with privacy regulations, this change is creating an opportunity for new, "post-creepy" web advertising that:

The big opportunity is in enabling publishers to reclaim control over their own audience data, not in establishing a new choke point such as a cryptocurrency or paid whitelisting program. (If publishers wanted to give up control to a tech firm, they can do that already.) Most of the development that is needed here can be provided by third parties that publishers are already using, because third parties are coming into compliance with privacy regulations. For example, Google Tag Manager already has the required functionality in order to comply with the European GDPR.

The missing piece is a way for sites to collect and enough user data to show advertisers that the site is trusted by human users, in order to make the ads on that site saleable.

In the new environment, user data alone is insufficient—data must be accompanied by the consent required to use it. And that can't be just "click to make this dialog go away and consent to adtech as usual". Both regulators and browser developers are going to require real consent. So the web advertising system needs to evolve away from dependence on large quantities of un-permissioned data towards the ability to use less data accompanied by permission. (Post-creepy web ads won't be able to swim in abundant unpermissioned data with the nutria of the Lumascape. Consent is scarcer than raw data, and only data accompanied by consent is safe to use. Publishers will have to collect and conserve every drop of data, like muad'dib, the desert mouse of Arrakis.) Possible sources include:

  • Subscription and micropayment systems

  • Federated paywalls

  • Comments and surveys

  • Email newsletters

  • Events

  • Miscellaneous e-commerce (tote bags, mugs, clothing...)

  • Transparency and Consent Framework consent bits

  • Differences in browser behavior between trusted and untrusted sites

and more.

Consent management is a tricky problem. IAB Europe is doing some work toward addressing it, with the open-source Transparency and Consent Framework. Although existing implementations are designed to nudge the user into not-transparent data practices, and are not yet getting real consent, this framework does provide a starting point on which to build consent management that both implements the user's preferences accurately and provides a smooth user experience. (more info: Global Consent Manager. Global Consent Manager is a client-side component that you can try in Firefox now, that can interact with server-side data platforms.)

In principle, privacy regulation and browser privacy improvements have the potential to lower the return on investment on creepy tracking, and raise the return on investment on building reputation and getting consent. But publishers, who have the reputation to get users to agree that they have the right to use data, don't have the development budgets or time to build the tools for data gathering.

User data and opportunities to get to get consent are everywhere, in CMSs, other software, and in third-party services. The missing piece is a platform that will collect data, with permission, from all the above sources and

  • run either on the publisher's own infrastructure or as a third-party service so that small publishers don't need to touch the CMS or deploy and manage a new service

  • comply with current and future data protection regulations

  • work with and anticipate privacy improvements in browsers

  • provide reports and APIs in a usable format for advertisers and agencies

Many of today's ad agencies, even sympathetic ones, won't come to the new system by choice, because it won't allow for tracking desirable audiences to cheap sites. We can assume that advertisers and agencies will ignore the new system until they see that it’s a way to reach a significant audience that they can’t reach in other ways, today, and the mainstream tracking-protected web audience in the near future.

Bonus links

When it comes to user data, are we done catching Google red-handed?

Sick of Facebook’s creepy ad targeting? Try this new tool

That job offer you just got from an agency could be fake

Twitter Users Are Blocking Hundreds of Brands in the Hopes of Pressuring the Platform to Remove Alex Jones

Advertisers say Facebook is dumping data without a net

Under GDPR, publishers are adopting CMPs for fear of losing out on ad revenue

Human progress. Not a one-way process.

The Perfect Marketing Plan in a Tweet?

‘If you tell your agency to chase price, then that’s what they’ll do’: Confessions of a former agency exec

04 Sep 04:40

What's the worst thing you've ever done?

by george
Well, it's a long list, but I thought of one. It was when I worked at the Internet Archive, possibly 2009, when it was still in The Presidio, and not the church in Inner Richmond. We were in an old office building, which had two main floors and a third, with a little eyrie. It was the best smoke break spot a workplace could offer. So, I popped up there one day for a break. But, I stubbed my
04 Sep 04:40

2,500 Cyclists Set off in Ride to Conquer Cancer in BC

by Average Joe Cyclist

The 10th Annual Ride to Conquer Cancer Presented by Wheaton Precious Metals.BC’s largest cycling fundraiser draws over 2,500 riders, to advance cancer care for the people of BC. Check out videos of the opening ceremonies, and photographs of the cyclists!

The post 2,500 Cyclists Set off in Ride to Conquer Cancer in BC appeared first on Average Joe Cyclist.

04 Sep 04:40

The Climate Cannot Wait

by peter@rukavina.net (Peter Rukavina)

It is high election season here in Sweden, and election advertising is on every street corner (and consumed almost all of the pre-show at the movie we went to yesterday).

This ad for the Green Party is especially noticable: “Now,” it reads, “the climate cannot wait.”

04 Sep 04:39

Almost Everything You Need to Know About Kannada Transliteration

by Thejesh GN

Wikipedia says Transliteration is a type of conversion of a text from one script to another that involves swapping letters (thus trans- + liter-) in predictable ways.

For Kannada we usually use Latin letters to represent1 Kannada script. Most of the time this conversion is based on phonetic similarity. It sounds simple but not really. There is a lot of confusion as no one learns this formally. Most of us learn by experimenting with phonetic typing or go by intuition. There are others ways to do it too. These are the notes I made while learning about it.

Street Name Signs in India are usually dual language.  The English part is a transliterated version.

Street Name Signs in India is usually dual language. The English part is a transliterated version.

Standards

There are quite a few standards or conventions involved from naming the language, to script, to transliteration. Here are the few that matter

  • ISO 15924 is a coding system for the representation of names of scripts. For Kannada script it is Knda and number 345. Remember this is for the Kannada script2 and not for the language.
  • For Kannada Language representation we use ISO-639 series which is either kn or kan.
  • ISO 15919 – is a standard for “Transliteration of Devanagari and related Indic scripts into Latin characters”. IIt uses diacritics to map the much larger set of consonants and vowels in Indic scripts to the Latin script. At this point it is the most sophisticated system available only challenged by the popularity and usage of Hunterian transliteration.
  • The Hunterian transliteration system is supposed to be the “national system of romanization in India” and the one officially adopted by the Government of India. Initially created for Devanagari, It has improved over time to cover other Indian languages. Though I must say I couldn’t find the table for Kannada.
  • The International Alphabet of Sanskrit Transliteration (I.A.S.T.) is one of the oldest transliteration scheme for lossless romanization of Indic scripts. Today it is a subset of the ISO 15919. 3. It is not a standard but a convention.
  • Indian Script Code for Information Interchange (ISCII) similar to ASCII is a coding scheme for representing various writing systems of India. It is used for encoding Indic scripts and a Roman transliteration. As of now ISCII is replaced by Unicode. What is of interest is roman transliteration. Don’t know if anyone uses it.
  • National Library at Kolkata romanisation is transliteration scheme used by in dictionaries and grammars books etc. It’s also called Library of Congress (USA) version and is very similar to ISO 15919. Here is the Kannada table.
  • International Phonetic Alphabet(IPA) is an alphabetic system which uses phonetic notation. It uses Latin characters. Using IPA you can write down how a word is pronounced in any language of the world. It has one to one mapping between sound and letters. Here is the IPA chart for Kannada.
  • The Unicode is a standard that provides a unique number for every character on planet earth 4. On web 5 UTF-8 become the most popular way to encode the Unicode character. Unicode Consortium develops and manages these standards.
  • I am ignoring other transliteration systems like Harvard-Kyoto, ITRANS, Velthuis, SLP1 etc

At this point in my opinion ISO-15919 seems to be the winner for Kannada Transliteration.

TODO: As far as I know Kannada is very phonetic. So I think there should be a direct match between ISO-15924 and IPA.6

Unicode CLDR and ICU projects

Unicode CLDR aka Unicode Common Locale Data Repository Project by Unicode Consortium that provides data building blocks to support languages. This makes it easy for the software engineers to add internationalization and localization support or build sorting, transliteration features. All kinds of companies small and big use these libraries to add language feature to their products.

ICU (International Components for Unicode) is a set of open source C/C++ and Java libraries providing Unicode and internationalization/localization support for software applications. There are wrappers for other languages like pyICU for python.

Transliteration

The Unicode CLDR provides data and guidelines for transliteration. By default it uses ISO 15919 for transliteration. Internally all Indic scripts are first converted to an internal format called Inter-Indic and then transformed into target script. As per the guidelines for Kannada it supports

  • Romanization or into latin script. This is the most used format
  • Transliteration rules from Kannada (Indic) to Latin are reversible except ZWJ and ZWNJ used for rendering
  • Provides transliteration into other Indic script (excluding Urdu) like from Kannada Script to Malayalam Script transliteration. In most situations they are reversible.

The Latin – Indic transliteration chart is available if you prefer to use it and there is also just Kannada ISO-15915 mapping available for download in pdf format. I am embedding the same below as its useful.

Personally I use Python so pyICU is my preferred library to use for transliteration7. Below you can find code examples for transliteration that I use the most. From Kannada to Latin and from Latin to Kannada.

from icu import *
trans = Transliterator.createInstance("Kannada-Latin")
print trans.transliterate(u"ಅಕೌಂಟೆಂಟ್‌ ನೇಮಕಾತಿ: ಪರೀಕ್ಷೆ ಬರೆದ 8 ಸಾವಿರ ಅಭ್ಯರ್ಥಿಗಳೆಲ್ಲ ಫೇಲ್‌ !")

trans = Transliterator.createInstance("Latin-Kannada")
print trans.transliterate(u"nanage kannaḍa barutte")

Print 1# akauṇṭeṇṭ‌ nēmakāti: parīkṣe bareda 8 sāvira abhyarthigaḷella phēl‌ !
Print 2# ನನಗೆ ಕನ್ನಡ ಬರುತ್ತೆ

Transliteration in Kannada is mostly used for inputting (or typing). Given the popularity of Latin (English) keyboard, it’s probably the most easiest way to do as well. Since Kannada8 phonetic its easy even for the new users. Being phonetic has a benefit, we can read (pronounce) the transliterated text almost correctly9. You would see most boards using transliterated text for places, names etc. They use the most simplified format and hence sometimes confusions exist while pronouncing. For example

But given correct transliteration10, it can be pronounced correctly to a great degree. But the current romanized spellings are part of official documents11 for a long time now and it’s not worth changing. But we need to adopt to this for newer names and boards, like for example movies, songs etc. Specially if you are planning to go international.

So “ಉಳಿದವರು ಕಂಡಂತೆ” becomes “uḷidavaru kaṇḍante”12 and not “Ulidavaru Kandanthe”

How to type Accents and Symbols

It has become easier than before now to type accents and symbols, required for say maḍivāḷa. On Android and iOS – press and hold “a” you will get various choices for a including “ā” in a popup overlay then you can choose. It works on all inbuilt keyboards.

On Ubuntu it’s easy using a compose key. Setup your compose key using Preferences -> Keyboard.

To get ā you need to type Compose + - then a,
To get ḷ you need to type Compose + , then l etc

You can find the big list of compose key shortcuts here.

I have not used Windows in a long time. This Microsoft tutorial seems to say it’s easy.

Footnotes

  1. For inputting Kannada text, sign boards etc
  2. Kannada script is used by languages – Kannada, Tulu, Konkani, Kodava, Sanketi, Beary etc
  3. IAST came first, so ISO 15919 follows the IAST scheme with minor changes
  4. At least that is the idea
  5. Even in other places
  6. Of course in words ಮಂದ್ರ and ನಂದು, character ಂ sounds differently. There are a few like that. But mostly it should match.
  7. Also check transliteration, indic-trans projects by LibIndic
  8. Mostly phonetic
  9. Exceptions exist like in mandra and nandu n is pronounced differently. Try ಮಂದ್ರ and ನಂದು
  10. I know we should use IPA for pronunciation. I will write a separate blog post about Kannada TTS.
  11. Even on OSM where ISO 15919 is used, we stick to official documents.
  12. Difficult to type but that’s the price I think and added trouble of buying a ASCII URL. Nothing is easy!
04 Sep 04:39

Long Before The Matrix – Die Welt Am Draht

by Martin

Long before ‘The Matrix’ (the movie) was released in 1999, there were a number of other books and movies about people living virtual lives in virtual worlds. A couple of months ago I was recommended to watch ‘Die Welt Am Draht‘ (link to Wikipedia in German and English). A 1973 movie by Rainer Werner Fassbinder, available on Youtube (see here for a trailer with English subtitles), it is about worlds simulated in computers occupied by conscious beings that are unaware that they are just a simulation. Well, that is, all except a few and those that are, are quite unable to deal with their knowledge. Sounds very much like Matrix but 25 years earlier! The movie is based on the book Simulacron-3, written in 1964 by Daniel F. Galouye and seems to be quite faithful to the book’s original story.

Think about it: The book was published in 1964, a time when microprocessors on a chip did not yet exist and computes were large mainframes in companies and universities, seen and used by few and operated by even fewer. With a few kilobytes of memory and screens/keyboards not yet available to interact with a computer, it must have taken quite some imagination to make the jump from what computers were then to the idea that they could one day become powerful enough to simulate worlds and produce consciousness.

Highly recommended to watch!

04 Sep 04:39

5 Ways that Cycling Boosts Your Brain Function

by Average Joe Cyclist

This post is about 5 ways that cycling can boost your brain function. People who cycle often do so to boost their physical health. But it’s exciting to learn that cycling also helps to build a better brain, both functionally and structurally. Here’s what science says about the cognitive benefits of pedaling.

The post 5 Ways that Cycling Boosts Your Brain Function appeared first on Average Joe Cyclist.

04 Sep 04:39

Adobe

by Rui Carmo

I’ve been an Adobe user since when Photoshop came in 800K floppies, but I’m not a particular fan of many of their current products (or their subscription models), except perhaps Bridge CC, which is free and a passable digital asset manager.

However, even installing a free Adobe product brings with it an infestation of oodles of Creative Cloud daemons, which light up Little Snitch as a Christmas tree and need to be suitably exorcised–hence the existence of this page.

Removing all Adobe Creative Cloud daemons

This completely disables (but does not remove) all CC daemons (you may need to reboot to kill off Core Sync and the Adobe Desktop Service):

launchctl unload -w /Library/LaunchAgents/com.adobe.*

What especially irks me is that com.adobe.acc.installer.v2 pops up on Activity Monitor running as root, which means I will never wilfully install anything from Adobe again unless in dire need.

Another nuisance is AGSService, which keeps popping up. I will eventually have to resort to Adobe’s own tools to get rid of it, I suppose.

04 Sep 04:38

Your Members Really Want To Do Things That Matter

by Richard Millington

I think ProjectManagement.com is one of the best brand communities in the world.

Can you guess why? It might take you a second.

The discussions aren’t super active, the website doesn’t look great, and some of the most basic features are missing (@mentions).

Now click on the templates page.

Woah, do you see that? There are 1000+ member-created templates in this community covering almost every possible aspect of project management you can imagine.

Imagine how much time and money this saves members?

Imagine how much it raises the standard of the entire field? It’s incredible.

This is a great example of what happens when you raise the bar to contributing instead of lowering it to get the metrics up.

Once you realize engagement metrics are irrelevant, you can finally focus on getting your members to make their best possible contribution to a community.

 

Creating A Honeypot of Resources

Sometimes we do things which make no sense.

For example, I’ve seen community managers spend weeks (even months) crafting what they think will be a game-changing resource for their members.

Do you know what really is game-changing?

Spending weeks persuading members to share their best resources!

This is what turns your community into a honeypot of resources which attracts more and more members.

Soon having a featured resource becomes a badge of honour within the field.

 

But My Members Don’t Have Time To Do This

I know what some of you are thinking.

‘That’s great for Project Managers, but my members are too busy to do this.’

Do you honestly, seriously, think Project Managers are less busy than your audience?

Your audience doesn’t do this today because you haven’t made it happen yet.

You haven’t made it a priority for them (aside: if they already have the resource, it doesn’t take much time to upload it to your site).

Your members aren’t contributing their best material because they don’t see the community as indispensable to their career aspirations, their relationships, or a key source of joy in their lives.

Once your community provides members with the psychological value they can’t get anywhere else, they flock to it. They make their best contributions to it. They make the time to create contributions which will make them stand out above the crowd.

That’s the difference between ProjectManagement.com and your community today. They set a high bar and never wavered from their standards.

And where you set the bar for your members matters a great deal.

 

Set The Bar High To Drive The Best Contributions

If you want the community to be a honeypot of useful resources, you need to set the bar high.

You need to forget chasing engagement, engagement metrics are irrelevant when you chase quality.

Instead, the only objectives that matter are how many great resources you get.

That’s what you focus on, it’s what you report to your boss, and it’s what you drive your efforts to increasing.

This should completely change your mission and how you think about your community.

You should no longer send out mass emails to the entire community, but instead, segment your top members from the rest, run surveys and interviews to identify their indispensable value.

On Sept 17, we’re launching our Strategic Community Management course.

During this course, we’re going to teach you how to create powerful member segments, understand what value they find indispensable from the community, and then deliver on that value by using the most powerful tactics possible.

If you want to chase more engagement, don’t take this course.

If you want to build a community your organization and your members find indispensable, we’re here for you.

Prices are going up to $750 on August 31.

04 Sep 04:38

A search engine for color palettes

by Nathan Yau

Picular is a simple tool that lets you search for a topic, and with Google Images as source, outputs a set of colors related to your query. This is going to be a great timesaver.

Tags: color, search

04 Sep 04:29

Why do bikelanes (still) drive some people crazy

by Gordon Price

 

The issue that will not die.

Dan Furmano, in a dissection of the social-media manager for Coalition Vancouver, quotes Gary Bizzo:

Asked why he was getting involved with the Vancouver mayoral race when he lives in Burnaby, Bizzo said: “It affects everybody, doesn’t it? I have offices downtown and this bikelane stuff is driving me crazy.”

Well, of course they don’t drive most people crazy, as a couple of elections proved. And they don’t cause congestion – because if they did, boy, would we know it.  First, drivers would experience it.  Then there would be data. But above all, the media would relentlessly pound that story, with endlessly looped video for illustration.

(The story, arguably, is ‘why didn’t the bikelanes cause congestion?’  It was expected that by removing lanes from major roads and bridges, the result world be (doom music here) carmageddon.  Didn’t happen.  Not news.  No reports.)

And yet the craziness continues, notably among those like Bizzo who don’t live in Vancouver – along with the supposition that there is a war on the car.  (Vancouver Magazine asked “Is There an ‘Ideological War on Transportation’ in Vancouver?”  Answer: Nope.)

But of course that’s a matter of perspective:

 

03 Sep 17:28

Connected, Episode 206: A High Appreciation for Winning

by Federico Viticci

Federico is back from vacation with a list of demands and opinions. Stephen posted a whole bunch of screenshots, and Myke is trying some new things with Twitter.

I'm back on Connected this week and we had fun discussing the future of the Mac mini and whether or not AirPower is ever coming out. You can listen here.

Sponsored by:

  • TextExpander, from Smile: Communicate smarter. Use this link for 20% off your first year.
  • Linode: High performance SSD Linux servers for all of your infrastructure needs. Get a $20 credit with promo code ‘connected2018’
  • Molekule: The only air purifier that actually destroys pollutants. Use connected for $75 off.

→ Source: relay.fm

03 Sep 17:28

Day One 3.0: Three Headline Features Make the Journaling App Better Than Ever

by Ryan Christoffel

If you're unfamiliar with Day One, it's simply the best journaling app available on Apple platforms. This week version 3 of the iOS app was released, adding great new features while simultaneously fixing some long-standing issues I've had with the editor interface. Many of the improvements are exclusive to Day One Premium subscribers, but the editor improvements benefit everyone.

Editor

Day One's editor has been a source of frustration for me in the past due to its hard distinction between edit mode and reading mode. Upon first opening a journal, you would see your entries rendered in beautiful rich text, but to make changes you had to hit an Edit button and switch interfaces, which also changed the text appearance. Once edit mode was initiated, the rich text would fade away, and you'd see Markdown syntax in-line. My problems with this setup were simple: I didn't like having to enter a separate editing mode to make changes, and journal entries were unattractive while your actual writing took place due to the visible Markdown. I know some people like to always see their Markdown syntax, but I'm not one of those people, at least when it comes to my journaling app.

Day One 3 makes editing entries as simple as it should have always been: open an entry, and with a single tap on the interface, the cursor appears and you can type away to your heart's content. Also, Markdown will automatically convert to rich text as you're writing, giving you a beautiful, clutter-free environment in which to write.

One other editor change I love: when you add multiple photos to an entry, they're grouped together as a single entity, and even sorted into a grid when appropriate. I'd appreciate more customization options to make sorting match my exact preferences, but just with what's available today, it's a solid new feature that serves to beautify journal entries with no extra hassle.

Dark Mode

It was a long time coming, but Day One finally offers a dark mode – and it looks fantastic. No matter which device you're using, whether an OLED-equipped iPhone X or an iPad sporting an LCD, dark mode feels like it was made for just that display. The app achieves this thanks to its combination of grey and true black shades for different interface elements; the varying shades provide a helpful sense of division and context, blending together well.

One nice detail about dark mode is that it can be activated on a schedule, either from sunset to sunrise or at the specific times you choose. For some apps I stick with dark mode 24/7, but since Day One's light mode is so lovely, I think I'm going to put the scheduling feature to use.

Audio Recording

Another feature that's bound to satisfy a long list of user requests is audio recording support. Directly inside the app, you can record an audio clip up to 30 minutes long, or use transcription mode to have the app transcribe up to 1 minute of audio. If you'd rather bring in audio from other sources, you can do that by pasting a file in or using Day One's share extension; unfortunately, there's not support for importing files via drag and drop.

Audio recordings are a natural addition for a journaling tool. The feature is ultimately fairly simple, but there's really no need for additional complexity here anyways.


It's important to note that both dark mode and audio recordings are exclusive to Premium subscribers, so they're not for the casual user. They do, however, make for high profile additions to an already-valuable service.

Admittedly, I hadn't used Day One in a little while until this past week. I've always considered it a great app, but it wasn't quite at the level of an app I wanted to try finding space for in my life. With these latest improvements, that may start to change.

If you're already a Day One user, this latest update should inspire confidence in a bright future for the app; if you're more like me and haven't tried it in a while, or possibly ever, it may be time to give Day One a serious look.


Support MacStories Directly

Club MacStories offers exclusive access to extra MacStories content, delivered every week; it's also a way to support us directly.

Club MacStories will help you discover the best apps for your devices and get the most out of your iPhone, iPad, and Mac. Plus, it's made in Italy.

Join Now
03 Sep 16:54

The Commons Clause

Heather Meeker, GitHub, Aug 27, 2018


Icon

"Modern open source infrastructure software has created more value over the past decade than we could have ever imagined," write the authors at RedisLabs, makers of open source data management software. "However, today’s cloud providers have repeatedly violated this ethos by taking advantage of successful open source projects and repackaging them into competitive, proprietary service offerings." Cloud providers, in particular, don't support the software developers. So they've added what they call a 'Commons Clause' which "maintains all conditions of the underlying open source license, but limits commercial sale of the software."

It's the whole non-commercial license debate all over again, this time applied directly to open source software. And so not surprisingly there's a response. In The Commons Clause will destroy open source Drew DeVault writes "The Commons Clause promoted by Kevin Wang presents one of the greatest existential threats to open source I’ve ever seen." As for Redits, DeVault writes, "Everyone who has contributed to the now-proprietary Redis modules has had their hard work stolen and sold by RedisLabs under a proprietary license." Maybe he's right - but maybe Redis is right too. More from Hacker News, Reddit

Web: [Direct Link] [This Post]
03 Sep 16:54

Reusable content matters: a learning object authoring tool for smart learning environments

Arif Altun, Mehmet Ali Çinici, Smart Learning Objects, Sept 07, 2018


Icon

As the abstract says, this study "reports the design and evaluation process of an authoring tool that is developed for content developers and/or instructors to be able to reuse and repackage existing learning objects (LOs)". The tool is called ONYA (which stands for Learning object authoring tool in Turkish) and was built using Microsoft Visual Studio. Most of the article describes how the evaluation of the tool was conducted; at the end we read that, yes, the tool was satisfactory. I didn't see any notes on the wider availability of the tool, and so regard the evaluation process as the primary content here.

Web: [Direct Link] [This Post]