Shared posts

24 Jun 08:14

Let‘s tackle the same challenge again, and again.

by Markus Jaritz

Actually, let’s not!

The products we build get more design attention as our Firefox UX team has grown from about 15 to 45 people. Designers can now continue to focus on their product after the initial design is finished, instead of having to move to the next project. This is great as it helps us improve our products step by step. But this also leads to increasing efforts to keep this growing team in sync and able to timely answer all questions posed to us.

Scaling communication from small to big teams leads to massive effort for a few.

Especially for engineers and new designers it is often difficult to get timely answers to simple questions. Those answers are often in the original spec, which too often is hard to locate. Or worse, it may be in the mind of the designer, who may have left, or receives too many questions to respond timely.

In a survey we ran in early 2017, developers reported to feel they

  • spend too much time identifying the right specs to build from,
  • spend too much time waiting for feedback from designers, and
  • spend too much time mapping new designs to existing UI elements.

In the same survey designers reported to feel they

  • spend too much time identifying current UI to re-use in their designs, and
  • spend too much time re-building current UI to use in their designs.

All those repetitive tasks people feel they spend too much time on ultimately keep us from tackling newer and bigger challenges. ‒ So, actually, let‘s not spend our time on those.

Let’s help people spend time on what they love to do.

Shifting some communication to a central tool can reduce load on people and lower the barrier for entry.

Let’s build tools that help developers know what a given UI should look like, without them needing to wait for feedback from designers. And let’s use that system for designers to identify UI we already built, and to learn how they can re-use it.

We call this the Photon Design System,
and its first beta version is ready to be used:
design.firefox.com/photon

We are happy to receive feedback and contributions on the current content of the system, as well as on what content to add next.

Photon Design System

Based on what we learned from people, we are building our design system to help people:

  • find what they are looking for easily,
  • understand the context of that quickly, and
  • more deeply understand Firefox Design.

Currently the Photon Design System covers fundamental design elements like icons, colors, typography and copy-writing as well as our design principles and guidelines on how to design for scale. Defining those already helped designers better align across products and features, and developers have a definitive source to fall back to when a design does not specify a color, icon or other.

Growth

With all the design fundamentals in place we are starting to combine them into defined components that can easily be reused to create consistent Firefox UI across all platforms, from mobile to desktop, and from web-based to native. This will add value for people working on Firefox products, as well as help people working on extensions for Firefox.

If you are working on Firefox UI

We would love to learn from you what principles, patterns & components your team’s work touches, and what you feel is worth documenting for others to learn from, and use in their UI.

Share your principle/pattern/component with us!

And if you haven’t yet, ask yourself where you could use what’s already documented in the Photon Design System and help us find more and more synergies across our products to utilize.

If you are working on a Firefox extension

We would love to learn about where you would have wanted design support when building your extension, and when you had to spend more time on design then you intended to.

Share with us!


Let‘s tackle the same challenge again, and again. was originally published in Firefox User Experience on Medium, where people are continuing the conversation by highlighting and responding to this story.

24 Jun 08:14

Amazon Just Invented Borders Books

24 Jun 08:14

Cost Comparison: Cars vs. Bikes

by Thea Adler
We put together a cost comparison of yearly bicycle expenses versus yearly car expenses. While switching to a standard bike is not a feasible substitute for a car for many people, ebikes can make this switch easy and realistic! Our biggest shock is that the average car owner spends $3,164 a year of gas and maintenance yet most car trips are under 5 miles! 
24 Jun 08:14

Recommended on Medium: One Weird Trick to Lose Size

Popular social networking apps are over 400 megs. With weekly releases, over one year you’ll download twenty gigs of data.

Since we launched Halide, the most unexpected compliment we’ve heard is about its size. At 11 megs, we’ll push less data in one year than a social network pushes in a single update.

“So you aren’t using Swift,” asked a friend. After all, Swift bundles its standard libraries into your app, bloating its size. Halide is almost entirely Swift.

How did we do it? Let’s start with the technical bits. A lot of this is a review of QA1795, but it’s important.

Measure, Don’t Guess

Export a build from Xcode. Choose “Save for Ad Hoc deployment.” Assuming your app supports app thinning (which it really should at this point), choose “Export for Specific Devices.” Make sure “Rebuild from bitcode” is checked.

Not only do you get a final representation of your package size, but you also get an App Thinning Report. Inspect your app package to find the biggest offenders.

Use Asset Catalogs

Keep your assets in asset catalogs. When you upload your app, Apple will slice it up into device-specific versions, so devices with 2x screens don’t get 3x assets, and vice versa.

Run PNG-crush

Before putting your assets in the catalog, run pngcrush. According to QA1681, Xcode will automatically crush assets for PNGs outside asset catalogues.

Try JPEGs for Photos

Restrict UI assets and similar fine linework to PNGs. This probably makes up most of the assets in your app, but if you have photographs, experiment with JPEGs. A little compression goes a long way.

It’s Time for a Difficult Conversation

After all this hard work, you’ve only shaved a few megabytes off a hundred meg behemoth. I don’t know how to tell you this, but you need less code.

Pick the Right Battles

Halide has about 15,000 lines of Swift. This includes a realtime video processor, a slew of custom controls, and our platform controlling AVFoundation. What’s interesting is the the code I didn’t write.

I shaved thousands of lines of boilerplate by leveraging Auto Layout. Many developers still insist on manual layout. Maybe they don’t understand Auto-Layout, or maybe they heard a story from a friend of a friend about how Auto-Layout is slow. (It isn’t.)

I’ve seen too many developers— especially at large companies— invent in-house layout engine. That’s just nuts. Don’t bloat your app with a bespoke framework when Apple bundles a fine layout engine with the OS.

We could cut another 100k by dropping Interface Builder. The User Manual and Settings are almost entirely IB, laid out with constraints. The high-level arrangement of the Camera UI is managed similarly. But we decided the productivity is worth the size in the short term.

Avoid Library Bloat

Inspect the package of many large apps and you’ll find dozens of third-party frameworks weighing anywhere from 100k to several megabytes.

I use zero third-party libraries. It’s a bit extreme, but we’re in a bit of a unique situation.

There just aren’t many third-party libraries that do what we need. The iOS dev community has a plethora of JSON mappers, but nothing for low-level manipulation of DNG files.

But what about the video processing I mentioned earlier? I can hear you shout, “GPUImage is extendable! It’s crazy to roll your own!”

From my experience with Periscope’s stack, we saw significant gains moving from GPUImage to an in-house solution. GPUImage is just fine if realtime image processing isn’t part of your business. But given our long term vision for Halide, and the role realtime rendering plays, it’s important to own this component.

I never passed on GPUImage because of file size. But as a consequence of rolling our own, I avoided bundling 125 unused filters in our app.

PSPDFKit has similar success replacing a larger framework that did too much:

We are excited to tell you that with PSPDFKit 6.8 for iOS we rewrote the core of our Digital Signature implementation for improved detection, validation and better error reporting. As a result, we also managed to completely drop our dependency on OpenSSL, reducing the size of our binary in the process.

Don’t catch Not-Invented-Here syndrome, but there are good reasons to avoid libraries.

Don’t Waste Resources on Analytics and A/B Tests

We don’t use any third-party analytics or crash reporting services. First and foremost, we don’t feel comfortable sending user data to advertising companies. But let’s set aside ideals for a moment.

Data isn’t free. In big apps, every action records an analytics event. Big apps need logging infrastructure— stuff uniquely identifies users, deduplicates requests, buffers logs, retries on failure, etc. It all adds up.

A/B tests are even worse. Your typical social network app is full of dead A/B tests that nobody went back to clean up.

We didn’t set out to avoid analytics and A/B tests because of the code bloat. It’s just our philosophy around products. Knowing too much data warps your mind. You find yourself optimizing for a local maximum, instead of making big bets that really move the needle.

So we use Apple analytics. It just works, without any code changes. It’s free. It respects the user’s privacy, requiring opt-in. Our opt-in rate is 32%, which is just fine for our needs.

There’s a time and place for analytics. We aren’t sure about our optimal price point, so we may experiment there. However, we’re keeping a wall between business-driven analytics and product development.

You Need Aligned Goals

We’re a two man team. We make money selling a product. Our growth is entirely organic. When users are happy, they tell friends about us. Small apps make us happy, and we think it’ll make users happy.

Our advice doesn’t help the apps with the most bloat. Social networks make money from advertisers, and advertisers need detailed analytics for ad targeting.

Big apps have hundreds of developers, making up dozens of teams, each with independent quarterly goals. The faster you go, the more goals you land, the more likely your promotion.

It’s understandable to think, “This library saves us a week, but adds a megabyte to our app. Well, we’re already hundreds of megabytes, so what’s another?”

Large organizations are full of reasonable ideas with unintended consequences.

Say an engineer wants to move up the tech ladder. Shipping features won’t get you a promotion. Building a new layout engine does. The company even gets recruiting-bait for the engineering blog.

The only solution is senior leadership declaring, “We are going to slim down our app.” Unfortunately, tech CEOs don’t use iPhones with 8 gigs of storage, and they don’t live in areas with shoddy speeds.

This isn’t a thankless effort. Since shipping Halide, we’ve gotten tons of messages from people from all over the world grateful for our efforts in keeping it small.

There really is one weird trick to lose size: focus on your customers.


One Weird Trick to Lose Size was originally published in Halide on Medium, where people are continuing the conversation by highlighting and responding to this story.

24 Jun 08:14

The Elements of Product Design, and the discipline vs. the job

by Faruk Ateş

A Product Designer is a job title; doing product design is a discipline that almost anyone can apply regardless of their line of work. But can we be more vague and unhelpful?

Yes we can: Wikipedia helpfully informs that nobody really agrees on what Product Design is:

Due to the absence of a consensually accepted definition that reflects the breadth of the topic sufficiently, two discrete, yet interdependent, definitions are needed: one that explicitly defines product design in reference to the artifact, the other that defines the product design process in relation to this artifact.

Eric Eriksson wrote the popular “What is Product Design”, but the context of his definition is limited to the responsibilities of Product Designers as we know them today. I say limiting because the essence of “product design” goes back to the Stone Age and can elucidate matters for people everywhere, and this point of view is getting lost amidst the rising popularity of the function as detailed in Eriksson’s article.

My goal with Product Matters is to break free the definition of product design and help people learn from insights across fields and disciplines. After all, Product Design the job is fundamentally about cross-disciplined skills and know-how.

Cubes representing the elements of product design: material, action, business

The job

I see Product Design, the relatively recent job title, as a function at the intersection of the following: Material, Action, and Business.

Material regards the resources involved, from brick and mortar to silicone and CPU, from paint and plastics to PHP and Python. You’re creating something to newly exist in a tangible form, even if that form is all-digital.

Action describes the user’s way of interacting with the product, and what that user experience is like. Steve Jobs famously said “Design isn’t what it looks like, design is how it works.” Action covers all aspects of “how it works.” From UX to IxD to visual and graphic, from accessibility to how one accesses the product.

Business, finally, encompasses knowledge of the market you’re creating the product for: a business model to make it viable, how to sell it, your go-to-market strategy, user research, and so forth. Making the case for your product being the solution to someone else’s pain.

Without business, you are creating art: a passion project for which the first goal isn’t to sell but a means to express something. Without action, you are creating an exit strategy: a typical Silicon Valley dud of a product that doesn’t work or isn’t well-considered, but might get acquired for its technology and business value. And lastly, without material you’re that person with an MBA and an idea who’s looking for a technical co-founder.

Product Design the job brings knowledge and expertise of these three aspects (and some others) together. You take in a client or a market’s needs; you envision and articulate what solution satisfies those needs, and you can lead people to design and build that solution to completion.

Or as Eric Eriksson put it:

“A key aspect of Product Design is understanding the business value behind every decision. Data informs everything we do, user research checks our assumptions, and we measure our success through business and engagement metrics.” — Eric Eriksson

One important unmentioned skill is communication: Product Designers must articulate their ideas so clearly that people from quite different disciplines understand them. In a sense, they are like generalists acting as the glue between more distinct specialists and subject matter experts. They often have an eagerness to accumulate varying skills, so that they possess more than a cursory-level understanding of topics.

The discipline, a.k.a. “why should I care about this?”

In my article A Dao of Product Design, I describe product design as “the Ur-discipline” — the oldest discipline in the books. It’s not quite the job as described above, but rather it’s the functional principles that apply here. Think of the discipline as a more holistic evolution of planning.

An early-era toolmaker can put a sharp-edged stone on a stick and call it an axe, but unless they design the functional nature of the axe first, it’s not much of a product. Once the toolmaker strings the stone to the stick, sharp edge facing outward, she has created an axe. She has assessed a need—a weapon or tool we can wield for hunting or chopping—and designed a solution to satisfy it.

This is where the discipline diverges from the job: our crafty cavewoman leveraged principles that are universal. You can apply product design to your work, whether you make hardware tools, bake cakes or are in public office. We wouldn’t call you a product designer for making luxury chocolate or designing a house, but you can apply product design principles in your work:

First, articulate your goal: What do you want to create?

Then, conceive a rudimentary version of it: What does the end result look like?

Next, envision its functionality, its design: How would it work?

Assess your required materials: What resources are needed to make it?

Determine your path to success: Which optimal steps get you to the goal?

Now go forth and bake/craft/manufacture/paint/sculpt/write/design your product!

This process can be equally applied to artisanal cakes and luxury chocolate, that painting aching to be released from inside you, or someone’s dream iOS application. We could be discussing a new form of water pump for the desert or the weekly business analysis report you have to create each Friday—the process involves those same key steps: the elements of product design.

24 Jun 08:14

China’s bike rental consolidation begins with reported Mobike acquisition of Unibike

by Emma Lee

Tencent Tech (in Chinese) is reporting that China’s top bike rental startup Mobike has recently completed the acquisition of smaller player Unibike, citing people familiar with the matter. The source didn’t specify the sum for this acquisition but pointed out that Mobike is also an investor in their RMB 100 million (around US$ 14.6 million) Series A.

“The report is not accurate. Mobike focuses on continuously improving the user experience, enhancing the competitiveness of core technologies and accelerating the pace of expansion at home and abroad,” said a spokesperson for the company. They did not, however, go into detail about the inaccuracies.

In October 2016, Mobike invested RMB 5 million in UniBike. After that, Unibike started to operate as an affiliated brand for Mobike on China’s university campuses.

If true, this could be a major move for Mobike in their battle against ofo. UniBike resembles ofo in several ways with its original focus on campus market and offering deposit-free services. The source disclosed that Mobike is planning to move to the lower end market with the acquisition of UniBike, noting great opportunities for synergy.

In addition to the tightening battle between the two largest companies in the vertical, the deal is significant because it would mark the first major acquisition in China’s burgeoning bike rental industry.

China’s heated bike rental war leaves little chance for smaller players to survive when facing the companies like ofo and Mobike. The UniBike acquisition comes just a few days after the sector witnessed its first casualty at the beginning of this week: Chongqing-based bike rental startup Wukong Bike announced that it is shuttering just last week.

Despite the unclear answers from the company, we can bet on one thing: After the explosion in the market, China’s bike rental industry is moving fast towards the consolidation phase. Even if UniBike’s acquisition news isn’t true, it won’t take us long to record the first acquisition or dozens that are definitely going to follow.

24 Jun 08:14

Photon Engineering Newsletter #7

by dolske

Lucky you, here’s Photon update #7!

Let’s start off with a fresh new video that gives an overview of what we’re doing with the Quantum and Photon projects. If you’re not already running Nightly, but are willing to live on the cutting-edge, this would be a great time to give it a spin! Get involved to help us test out everything that’s new, and experience some of these great improvements first-hand!

 

Mozilla All-Hands

Next week, everyone at Mozilla will be gathering in San Francisco for our biannual All-Hands meeting. The Photon team will be using it as a repeat of our Toronto Work Week (as covered in Photon Update #2). So we’re going to be super-busy hacking on Photon. We’ve got even more great stuff coming up, and I can’t wait to talk about it in Photon Update #8. But… The intense focus means that I might not get that update out until the following week. I think the wait will be worth it. 🙂

 

Recent Changes

Menus/structure:

 

Animation:

  • Updated arrow-panel animations are going through review this week.
  • Users on macOS will notice that panel open/close animations are much smoother, as a result of a platform fix. (You’ll see more improvements soon, from the item above, as well as another platform fix to add a beautiful background blur to the panel).
  • Work continues on animations for the downloads toolbar button, stop/reload button, and page loading indicator.

 

Preferences:

 

Visual redesign:

  • Another community contribution: Oriol removed an small, unexpected line that was appearing at the top of some windows. Thanks for the patch!
  • Firefox will now automatically enable its touch mode (which increases the size of various UI to make it more touch-friendly) when used in Windows 10 Tablet mode.
  • The dark toolbar that previously landed for Windows 10 is now coming to macOS. (This just landed, and if it sticks will be in Friday’s Nightly build.)
    Screen Shot 2017-06-22 at 4.27.25 PM

 

Onboarding:

  • The onboarding tour content has landed and been polished to match the UI spec. You can click the Fox icon in about:home to give it a try! Currently it has 5 tours for existing (non-Photon) features — Private Browsing, Add-ons, Customization, Searching, and setting your Default Browser. These are planned to ship in Firefox 56 (for users installing Firefox for the first time). Additional tours will next be implemented for Firefox 57, to introduce new Photon features to existing Firefox users.
  • The onboarding tour now has UI to allow hiding it (so users who don’t want to go through each tour step can just make it go away).
  • The Mozilla logo and onboarding icon are now shown on the correct sides for RTL languages.
  • A Sync tour and tour notifications will be landing soon.

 

Performance:

  • Places (our bookmarks and history storage system) is now initialized after first paint on startup. This helps make Firefox feel faster to launch, because the window will be shown sooner.
  • More giant patches up for review for removal of Task.jsm calls, and fixed the last blocker to starting work on removing Promise.jsm usage.
  • More awesome work on improving Talos measurements and figuring out regressions. (Particularly some issues that have been holding up animations.)
  • Florian posted in firefox-dev about the browser_startup.js test, and asked everybody to have a look at the generated list to identify low hanging fruit. This test helps us find code that is loading too early, and prevents things from regressing once we fix it.

 

Thus concludes Photon update #7. As noted above, next week is going to be a little busy, so it may be a couple of weeks until the next update.


24 Jun 08:14

The Expert vs. Skills Problem

by Richard Millington

To continue from the subject matter expert post.

We turn down a lot of work because we won’t manage a community for a client.

We do plenty of other things, but hosts need to own those relationships.

The reason is simple.

When you build a community, you’re going to have a lot of people asking questions.

These are questions only people very familiar with the subject can answer. If you’re not an expert, you can spend the bulk of your time searching for answers to questions instead of proactively building communities.

This means pestering employees and hoping they give you a response.

This isn’t a good use of your time, so why handicap yourself?

In most sectors, it’s a lot easier to take a subject expert and teach them great community skills than teach someone with great community skills about the topic.

You can now spend the bulk of your time building the community instead of fishing for answers to questions.

There’s some nuance here, but the principle remains intact. Find someone smart and connected within your field. Check they have the attitude to help a community of people like themselves. Then teach them community skills.

p.s. your customer support team is a good place to look for talent.

24 Jun 08:13

Turn and Turn Again. The Wheels Go ‘Round

by Ken Ohrn

Build it; they come; business follows; and people turn around.

Vancouver’s separated bike lanes have attracted new people in new demographics to travel by bicycle.  But now, based on members’ opinions, Vancouver’s top business association has solidified their bike lane support and used terms like “evolution” and “competitive edge” to explain their reasoning.  Perhaps other business groups (Commercial Drive — are you following this?) will take note.

26.BtWW.29.May.2017

Click to enlarge (Ken Ohrn photo)

Tina Lovegreen on CBC News covers a story that regular PT readers know already.  But this tangible landmark move signals a big step.  The Downtown Vancouver Business Improvement Association (DVBIA), led by CEO Charles Gauthier, has partnered with Hub Cycling to the tune of $ 15,000 per year as a platinum member.

Gauthier said many employers, especially those in the tech sector, are interested in office spaces that accommodate different types of transportation, such as cycling or car sharing.

“They want those options available so it’s easy for their employees to get to work by bike or transit or to be able to walk to work,” Gauthier said.  “Parking of private vehicles is less of a top priority and building owners want to attract those employers,” he said.

“I think it provides us with a competitive edge.”

. . .   Gauthier said there might be a few retailers that won’t be pleased with the move to support cycling, but he said those businesses that rely on street parking will most likely move out and be replaced by other tenants.  That’s what happened on Hornby Street when the bike lanes were built there, said Gauthier.


24 Jun 08:13

Samsung Galaxy Note 8 Launching in Late September with Dual Camera Setup, 6GB RAM

by Rajesh Pandey
Prolific leaker @evleaks reports that the Samsung Galaxy Note 8 will be the company’s most expensive smartphone yet with a price tag of €999 While there were reports of Samsung launching the Note 8 in August, today’s report claims it will be launching in the second half of September. Continue reading →
24 Jun 08:13

BikeFACE: Caitlin Chee

by dandy

Rider statement by CAITLIN CHEE: "To me, having a bike is like having a key to the city. It allows me freedom of movement at any time of day or night, unhindered by the cost, hassle and stress of public transportation. Biking helps me feel independent and free, and refreshes my connection with the city I live in."

dandyhorse is pleased to present the photo essay BikeFACE! by Marc Bernhard

In 1890s Europe and North America, the bicycle was gaining popularity as a means of transportation, recreational activity and sport. However, medical opinion about the health benefits of cycling, especially for women, was mixed. One of the supposed risks for women was a condition known as “bicycle face.

At the turn of the century the bicycle afforded many women increased freedom and autonomy, and the bicycle itself became a symbol of women’s emancipation. With the ridiculous prognosis of "bicycle face" we saw how cycling, feminism, patriarchy and medical opinion collided in the1890s.

Photographer Marc Bernhard says, "The purpose of BikeFACE! is to promote and celebrate women’s commuter, recreational and sport cycling. It is a series of studio images in which the subjects were invited to ride their bikes on a stationary trainer at full exertion to see what their bicycle face might look like. The series is meant to be a light-hearted, tongue-in-cheek reference to the historical anti-feminist ideas against cycling, by showing the strength, intensity and determination of women cyclists today."

Featured rider Caitlin Chee says: "..having a bike is like having a key to the city. It allows me freedom of movement at any time of day or night, unhindered by the cost, hassle and stress of public transportation."

dandyhorse is pleased to present this essay celebrating women cyclists in our city -- just in time for Bike Month! You can read more about "bicycle face" here.

We'll be rolling out each profile one at a time throughout Bike Month and adding all the links here.

24 Jun 08:10

Quantum Flow Engineering Newsletter #14

by ehsan

We have about 13 more weeks before the train of Firefox 57 leaves the station.  Next week many of you will be at the upcoming work week, so I thought it may be a good time to have some retrospection over our progress so far, just so that you can get a good sense of how to extrapolate when you are planning things next week.

One difficulty with the Quantum Flow project is that since it touches many different areas of the browser, it doesn’t lend itself very easily to drawing nice charts for it.  🙂  It is hard to find one metric that all of this work fits inside, and that’s OK.  My goal this week is to highlight what we can achieve with focus in a limited amount of time, so I’ll bring a couple of examples.

This is a snapshot of our burndown chart1.  We currently have 182 closed bugs and 136 open bugs.  That’s great progress, and I’d like to thank everyone who helped with all aspects of this!

But to speak of a more direct measurement of performance, let’s look at our progress on Speedometer V2.  Today, I measured our progress so far on this benchmark by comparing Firefox 53, 54, 55.0b3 (latest beta as of this writing) and the latest Nightly, all x64 builds, on the reference hardware.  This is the result (numbers are the reported benchmark score, higher is better):

Speedometer improvements

There are also many other top level performance related projects that are ongoing and approaching final stages.  I’m really excited to see what the next few months are going to uncover for Firefox performance.

One bit of administrative note, as next week most people are able to get updates from each other face to face, I won’t send out the newsletter.  Now let’s finish with this week’s list of acknowledgements to those who helped make Firefox faster during the past week, hopefully I’m not forgetting any names!

[1] (The number of bug fixes is a weird metric to use for performance improvements, since we use bugs as a unit of work, and the performance impact of each bug can be vastly different.  But I have tried to describe the details of these bugs for the most part before so the detailed information is at least available.)

23 Jun 16:33

"As long as politics is the shadow cast on society by big business, the attenuation of the shadow..."

“As long as politics is the shadow cast on society by big business, the attenuation of the shadow will not change the substance.”

- | John Dewey, The Need for a New Party
23 Jun 16:32

Photo



23 Jun 16:32

"Don’t think of introversion as something that needs to be cured…Spend your free time the way you..."

“Don’t think of introversion as something that needs to be cured…Spend your free time the way you like, not the way you think you’re supposed to.”

- | Susan Cain
23 Jun 16:32

The sky above the port was the color of television, tuned to a...



The sky above the port was the color of television, tuned to a dead channel.

| William Gibson, Neuromancer

23 Jun 16:32

A Raspbian desktop update with some new programming tools

by Simon Long

Today we’ve released another update to the Raspbian desktop. In addition to the usual small tweaks and bug fixes, the big new changes are the inclusion of an offline version of Scratch 2.0, and of Thonny (a user-friendly IDE for Python which is excellent for beginners). We’ll look at all the changes in this post, but let’s start with the biggest…

Scratch 2.0 for Raspbian

Scratch is one of the most popular pieces of software on Raspberry Pi. This is largely due to the way it makes programming accessible – while it is simple to learn, it covers many of the concepts that are used in more advanced languages. Scratch really does provide a great introduction to programming for all ages.

Raspbian ships with the original version of Scratch, which is now at version 1.4. A few years ago, though, the Scratch team at the MIT Media Lab introduced the new and improved Scratch version 2.0, and ever since we’ve had numerous requests to offer it on the Pi.

There was, however, a problem with this. The original version of Scratch was written in a language called Squeak, which could run on the Pi in a Squeak interpreter. Scratch 2.0, however, was written in Flash, and was designed to run from a remote site in a web browser. While this made Scratch 2.0 a cross-platform application, which you could run without installing any Scratch software, it also meant that you had to be able to run Flash on your computer, and that you needed to be connected to the internet to program in Scratch.

We worked with Adobe to include the Pepper Flash plugin in Raspbian, which enables Flash sites to run in the Chromium browser. This addressed the first of these problems, so the Scratch 2.0 website has been available on Pi for a while. However, it still needed an internet connection to run, which wasn’t ideal in many circumstances. We’ve been working with the Scratch team to get an offline version of Scratch 2.0 running on Pi.

Screenshot of Scratch on Raspbian

The Scratch team had created a website to enable developers to create hardware and software extensions for Scratch 2.0; this provided a version of the Flash code for the Scratch editor which could be modified to run locally rather than over the internet. We combined this with a program called Electron, which effectively wraps up a local web page into a standalone application. We ended up with the Scratch 2.0 application that you can find in the Programming section of the main menu.

Physical computing with Scratch 2.0

We didn’t stop there though. We know that people want to use Scratch for physical computing, and it has always been a bit awkward to access GPIO pins from Scratch. In our Scratch 2.0 application, therefore, there is a custom extension which allows the user to control the Pi’s GPIO pins without difficulty. Simply click on ‘More Blocks’, choose ‘Add an Extension’, and select ‘Pi GPIO’. This loads two new blocks, one to read and one to write the state of a GPIO pin.

Screenshot of new Raspbian iteration of Scratch 2, featuring GPIO pin control blocks.

The Scratch team kindly allowed us to include all the sprites, backdrops, and sounds from the online version of Scratch 2.0. You can also use the Raspberry Pi Camera Module to create new sprites and backgrounds.

This first release works well, although it can be slow for some operations; this is largely unavoidable for Flash code running under Electron. Bear in mind that you will need to have the Pepper Flash plugin installed (which it is by default on standard Raspbian images). As Pepper Flash is only compatible with the processor in the Pi 2.0 and Pi 3, it is unfortunately not possible to run Scratch 2.0 on the Pi Zero or the original models of the Pi.

We hope that this makes Scratch 2.0 a more practical proposition for many users than it has been to date. Do let us know if you hit any problems, though!

Thonny: a more user-friendly IDE for Python

One of the paths from Scratch to ‘real’ programming is through Python. We know that the transition can be awkward, and this isn’t helped by the tools available for learning Python. It’s fair to say that IDLE, the Python IDE, isn’t the most popular piece of software ever written…

Earlier this year, we reviewed every Python IDE that we could find that would run on a Raspberry Pi, in an attempt to see if there was something better out there than IDLE. We wanted to find something that was easier for beginners to use but still useful for experienced Python programmers. We found one program, Thonny, which stood head and shoulders above all the rest. It’s a really user-friendly IDE, which still offers useful professional features like single-stepping of code and inspection of variables.

Screenshot of Thonny IDE in Raspbian

Thonny was created at the University of Tartu in Estonia; we’ve been working with Aivar Annamaa, the lead developer, on getting it into Raspbian. The original version of Thonny works well on the Pi, but because the GUI is written using Python’s default GUI toolkit, Tkinter, the appearance clashes with the rest of the Raspbian desktop, most of which is written using the GTK toolkit. We made some changes to bring things like fonts and graphics into line with the appearance of our other apps, and Aivar very kindly took that work and converted it into a theme package that could be applied to Thonny.

Due to the limitations of working within Tkinter, the result isn’t exactly like a native GTK application, but it’s pretty close. It’s probably good enough for anyone who isn’t a picky UI obsessive like me, anyway! Have a look at the Thonny webpage to see some more details of all the cool features it offers. We hope that having a more usable environment will help to ease the transition from graphical languages like Scratch into ‘proper’ languages like Python.

New icons

Other than these two new packages, this release is mostly bug fixes and small version bumps. One thing you might notice, though, is that we’ve made some tweaks to our custom icon set. We wondered if the icons might look better with slightly thinner outlines. We tried it, and they did: we hope you prefer them too.

Downloading the new image

You can either download a new image from the Downloads page, or you can use apt to update:

sudo apt-get update
sudo apt-get dist-upgrade

To install Scratch 2.0:

sudo apt-get install scratch2

To install Thonny:

sudo apt-get install python3-thonny

One more thing…

Before Christmas, we released an experimental version of the desktop running on Debian for x86-based computers. We were slightly taken aback by how popular it turned out to be! This made us realise that this was something we were going to need to support going forward. We’ve decided we’re going to try to make all new desktop releases for both Pi and x86 from now on.

The version of this we released last year was a live image that could run from a USB stick. Many people asked if we could make it permanently installable, so this version includes an installer. This uses the standard Debian install process, so it ought to work on most machines. I should stress, though, that we haven’t been able to test on every type of hardware, so there may be issues on some computers. Please be sure to back up your hard drive before installing it. Unlike the live image, this will erase and reformat your hard drive, and you will lose anything that is already on it!

You can still boot the image as a live image if you don’t want to install it, and it will create a persistence partition on the USB stick so you can save data. Just select ‘Run with persistence’ from the boot menu. To install, choose either ‘Install’ or ‘Graphical install’ from the same menu. The Debian installer will then walk you through the install process.

You can download the latest x86 image (which includes both Scratch 2.0 and Thonny) from here or here for a torrent file.

One final thing

This version of the desktop is based on Debian Jessie. Some of you will be aware that a new stable version of Debian (called Stretch) was released last week. Rest assured – we have been working on porting everything across to Stretch for some time now, and we will have a Stretch release ready some time over the summer.

The post A Raspbian desktop update with some new programming tools appeared first on Raspberry Pi.

23 Jun 16:30

Links for June 23rd

by delicious
23 Jun 16:29

Ghost Of Dominance Past

by Ken Ohrn

In a few places along the new and temporary Arbutus Greenway, it appears that heritage blackberries are attempting renewed dominance over the growing number of travelers there.  Really, it’s sort of spooky, given the decades of time that the railroad was abandoned and these blackberries ruled it all. And it seems they’re not giving up so easily.

Click to enlarge

01.Heritage.Blackberries02.Heritage.Blackberries


23 Jun 00:57

Make Servers Dumb Again

by mikecaulfield

After talking with Jon Udell and re-reading an old post of mine on storage-neutral web-infrastructure I realize I can make an old point much easier. So here goes:

Make Servers Dumb Again.

You’ve heard of the Dumb Terminal, right? The idea that a terminal wouldn’t do anything but display stuff composed on centralized servers?

Well, this is the opposite. I want dumb servers. I want smart front-ends hosted anywhere to make basic data queries to servers. I want those two things — data and display engines — to be run by separate folks, like in the original vision of the web. I store the HTML on my server under my rules. You display it in your browser under yours.

Why do this? Because the marriage of front-ends and data creates lock-in, lousy portability, surveillance models, and crappy incentives for a good user experience.

You can get around that by running your own server, sure. Now you’re still locked into something, but the thing you’re locked into is running your own server forever, which is frankly almost as horrifying as being tracked.

I am 100% sure this post will be misunderstood. So I’ll just end with Klint Finley’s list of the freedoms people actually want.

  • Freedom to run software that I’ve paid for on any device I want without hardware dongles or persistent online verification schemes.
  • Freedom from the prying eyes of government and corporations.
  • Freedom to move my data from one application to another.
  • Freedom to move an application from one hosting provider to another.
  • Freedom from contracts that lock me in to expensive monthly or annual plans.
  • Freedom from terms and conditions that offer a binary “my way or the highway” decision.

You’ll notice that the minute the data provider becomes unhitched from the display and interaction provider all this happens automatically. That makes for a more difficult time programming, but it ultimately gets the people what they want.

Make Servers Dumb Again. There, I said it.


23 Jun 00:56

Italian Airport Makes Exception To Liquid Limits For Pesto Sauce

by Mary Beth Quirk
mkalus shared this story from Consumerist.

It may come as no surprise that deep in the heart of pasta country, officials at the airport in Genoa, Italy, are offering a special culinary dispensation to anyone traveling with the city’s famed pesto sauce.

Genoa’s Cristforo Colombo Airport has been waiving the 100-milliter limit on liquids as long as it’s Genovese pesto as part of the “Il pesto è buono” campaign, which launched June 1. Thus far, more than 500 jars of the stuff have gone through the airport.

In order to fly with pesto jars of up to 500g, travelers simply have to make a donation to a charity called Flying Angels, which helps cover the cost to fly ill children abroad to receive treatment. They’ll then receive a special sticker to stick on the jar. Once the pesto has been scanned by an X-ray machine, passengers can carry it on board.

Officials say the plan was inspired by mountains of confiscated sauce jars piling up at security checkpoints.

Of course, if you’re flying into another country other than Italy, you’ll have to check the local customs rules on what kinds of foods you can bring with you. According to U.S. Customs and Border Protection rules, for example, canned goods like prepared sauces that don’t contain meat products are generally admissible. Pesto is made from basil, pine nuts, olive oil, and garlic, so it would likely pass muster.

[h/t BBC]





23 Jun 00:56

When babies are born, the cycle

by Nathan Yau

Movies would have you believe that birth is random and unpredictable. (And if you haven’t been part of the birth process, you’d be surprised by how slow it actually is.) While uncertainty is always in play, there’s a certain cycle to it all. Zan Armstrong and Nadieh Bremer for Scientific American, using 2014 data from the Centers for Disease Control and Prevention, examined the regularity and the reasons for the spikes.

I like the greater/lesser than average split for contrast. The circle time series layout doesn’t always fit the data, but in this case the metaphor fits the cyclical aspect.

Full graphics and breakdowns.

Tags: birth

23 Jun 00:56

OnePlus 5 features 1.6x optical zoom, not the claimed 2x

by Patrick O'Rourke
OnePlus 5 camera

In yet another bout of controversy, it looks like OnePlus’ recently revealed OnePlus 5 doesn’t actually include the advertised 2x optical zoom, a feature that puts the popular ‘never settle’ Android device’s camera in-line with the iPhone 7 Plus’ shooter, as well as Asus’ recently released ZenFone Zoom 3 (which actually features 2.3x optical zoom).

A Reddit user that goes by the handle ‘kumu_bandara‘ analyzed the EXIF data of an image shot with the OnePlus 5 that was shot at 1.33x zoom, showing data that indicates the zoom measures in at just 1.33x. The EXIF data also revealed that the OnePlus 5’s digital zoom is only capable of 1.5x optical zoom by default. These claims were later confirmed by a number of Reddit posters.

For those who are unaware, EXIF data is a standard file format used for images.

Optical and digital zoom are two very distinct things, with the camera’s internal glass creating optical, and software performing the leg-work for digital zoom. OnePlus claims that that OnePlus 5 is capable of 2x optical zoom, just like the iPhone 7 Plus, though that doesn’t seem to be the case.

On Twitter OnePlus co-founder, Carl Pei, explained that the OnePlus 5’s optical zoom actually measures in at 1.6x, with the remaining 0.4x reserved for the device’s SmartCapture software. While likely accurate, this still means that the OnePlus 5 still doesn’t feature the advertised true 2x zoom.

OnePlus has also been criticized for for cheating traditional benchmark systems with the OnePlus 5 in an effort to hit higher scores.

Given that the OnePlus 5 is one of the best Android devices around, especially when taking price into consideration, it’s strange that the company is using somewhat shady marketing tactics to promote the smartphone.

Source: Reddit Via: Phandroid 

The post OnePlus 5 features 1.6x optical zoom, not the claimed 2x appeared first on MobileSyrup.

23 Jun 00:56

Picnic

by Richard

The Icelandic Canadian Club of Toronto held their annual picnic last weekend, and before that, it occurred to me that I don't have the necessary equipment for having a picnic. That meant buying a blanket for sitting on and a picnic basket for taking along food and various whatnots. I shopped on Amazon.ca and found these beauties:

I went to the park next to my apartment on the Summer Solstice to try them out. I even brought my sharp water serving bottle and a tube of Pringles. I made a list of the things I forgot so that, during a real picnic, I'd be totally prepared.

This was my view as I lay down on the blanket:

Looking up at a tree.

On the way back I realized that the blanket folded up and fit under the handles of the basket. Bonus!

My arm, carrying a picnic basket and blanket.

The blanket folds up neatly (the tag has instructions in case I forget) and compactly. There are some limitations. The basket is too big to fit in the panniers of Toronto Bike Share bikes. The blanket is not machine washable.

I plan on picnicking every night in the summer and fall that I have leftovers from cooking to eat.

23 Jun 00:54

Solving SEO Problems with API Design

by mnally
Single-page applications are popular and easy to work with, but often make information hard to find

Last year I visited the software development arm of household-name retailer that was attempting to rebuild the user interface for their main website using a single-page application (SPA) implementation design. The project had failed.

It did so for for two main reasons: page load times were unacceptably long, and search engines were unable to effectively index the new site. 

At the time of my visit, the retailer had abandoned the SPA design approach in favor of "old-fashioned" HTML construction on the server. This is a pity, because a properly-designed and -implemented SPA could have provided a superior experience for the company’s users, and superior productivity for its developers.

There’s a lot of advice on the web on how to optimize load times for single-page applications, but less on how to deal with the problem of search-engine indexing. This post explains how the search engine indexing problem can be solved through thoughtful design of APIs—perhaps not the place many people might look for a solution to a search engine problem.

SPAs enable end users to navigate between different entities without performing an HTML document load for each entity. 

Note: HTML document loading is a concept fundamental to all web browsers—it’s precisely defined here and in other specifications. "Single-document application" would have been a better name than single-page application, if you value consistency with the terminology in the specifications.

Essentially, a JavaScript program is loaded into the browser as a result of an initial HTML document load, and that JavaScript program then uses a series of API calls to get the data and present a user interface for a succession of entities without having to load an HTML document (and corresponding JavaScript) for each one. 

The SPA experience

There are many reasons for the popularity of SPAs: they are easy and fun to write; they can provide a superior user experience; and they help provide a clean separation between user interface code and business logic.

Another important advantage of SPAs is that their overall design is similar to that of mobile apps that display the same data and access the same function—in fact many people use the same HTML5 SPA implementation for both web and mobile.

Early SPAs often failed to integrate well with the browser. One of the most visible mistakes was a failure to update the browser history appropriately. As the user navigated between resources, the SPA failed to update the address bar and the browser history, resulting in providing nothing for the user to bookmark and the back, forward, and history controls of the browser not working.

Better understanding of how a SPA should be written along with adoption of frameworks like Angular that help developers write good SPAs have resulted in more and more SPAs that integrate well with the browser. Yet there are still few SPAs that also work well with search engine optimization.

HTML5 improvements aren’t enough

If you look in the web browser address bar of a typical SPA, you will see addresses that look like this:

https://acme.com/#/entity/xxxx

 

Note: technically these addresses are called URI references, not URLs—the URL terminates at the # character.

The only HTML document that exists in a design like this is the document at the URL https://acme.com/. This is the document that will load a JavaScript application that will then make API calls to retrieve JSON representations of entities with identifiers of the form /entity/xxxx. The Javascript then creates a browser DOM whose rendering is what the user sees.

There is only one HTML document for search engines to retrieve—https://acme.com/—and it usually contains only code or references to code, which is not useful to a search bot. This completely defeats search engine indexing.

With the ubiquity of the HTML5 history API in browsers, SPAs can now be written to use URLs like this one instead: 

https://acme.com/entity/xxxx

 

This is an important improvement, because there is now a separate URL that is retrievable from the server for each entity that is exposed by the SPA. Although this is a step in the right direction, this does not by itself solve our SEO problem, because the resources at these URLs—like the single resource we had previously at https://acme.com/—typically only contain code or references to code, as illustrated in the following example. This is still useless to a search bot.

<!DOCTYPE html>

<html>

 <head>

   <script src="/mySPA.js"></script>

 </head>

 <body>

 </body>

</html>

 

Assuming the JavaScript is written appropriately, this HTML will display the correct user interface for each one of the entities of the app, whose URLs might look like https://acme.com/entity/xxxx. The JavaScript code will construct a browser document object model (DOM) for the user interface even though there is no content in the body of the HTML document to do this. The JavaScript code must also look at the URL of the document that was loaded to determine the initial entity to present to the user.

Add some meta

Consider the example of an application that displays information about Disney cartoon characters. In order to make the HTML useful for a search bot, we can simply add additional information, like this:

<!DOCTYPE html>

<html>

 <head>

   <meta name="description" content="Personal information about Mickey Mouse" />

   <meta name="keywords" content="Mickey Mouse Donald Duck Minnie Goofy Disney" />

   <script src="/mySPA.js"></script>

 </head>

 <body>

   <div style="display: none;">

     <p>name: <span>Mickey Mouse</span></p>

     <p>girlfriend: <a href="https://acme.com/entity/Minnie">Minnie Mouse</a></p>

     <p>friends:

       <ol>

         <li><a href="https://acme.com/entity/Donald">Donald Duck</a></li>

         <li><a href="https://acme.com/entity/Goofy">Goofy</a></li>

       </ol>

     </p>

   </div>

 </body>

</html>

 

 

All we did was add some <meta> elements to the head of the document (search engines take note of meta elements) and a simple <body> that will never be displayed to the user.

The essential idea—which might not be immediately obvious—is that the <body> element we included here has no influence on the user interface that a user will see—producing that user interface is the job of the JavaScript of the SPA. 

The goal of the <body> element is only to provide useful information for search bots, and other non-browser clients that understand HTML. Users will see only the user interface created by the JavaScript of the SPA, and search engines will see only the body shown above.

Obviously, this is not a sophisticated example of what you want to include in your HTML for SEO—this is not an SEO tutorial, which I'm anyway not qualified to write—but it shows how you can include HTML that is visible to search engines for all the entities that are displayed in your SPAs.

In summary, there are two steps required to solve the SEO problem for SPAs. The first is defining and using separate URLs—ones without fragment identifiers introduced by the "#" character—for each entity shown in the user interface.

The second step is providing meaningful HTML bodies for each entity, even though that HTML will not be seen by human users of the SPA.

This post has outlined a basic approach to make SPAs indexable by search engines, but we have not yet linked the story to API design. We'll do that in an upcoming post.

For more on API design, check out the free eBook, "Web API Design: The Missing Link."

Image: Flickr Creative Commons / ECP

23 Jun 00:54

Two years as a Data Scientist at Stack Overflow

by David Robinson

Last Friday marked my two year anniversary working as a data scientist at Stack Overflow. At the end of my first year I wrote a blog post about my experience, both to share some of what I’d learned and as a form of self-reflection.

After another year, I’d like to revisit the topic. While my first post focused mostly on the transition from my PhD to an industry position, here I’ll be sharing what has changed for me in my job in the last year, and what I hope the next year will bring.

Hiring a Second Data Scientist

In last year’s blog post, I noted how difficult it could be to be the only data scientist on a team:

Most of my current statistical education has to be self-driven, and I need to be very cautious about my work: if I use an inappropriate statistical assumption in a report, it’s unlikely anyone else will point it out.

This continued to be a challenge, and fortunately in December we hired our second data scientist, Julia Silge.

We started hiring for the position in September, and there were a lot of terrific candidates I got to meet and review during the application and review process. But I was particularly excited to welcome Julia to the team because we’d been working together during the course of the year, ever since we met and created the tidytext package at the 2016 rOpenSci unconference.

Julia, like me, works on analysis and visualization rather than building and productionizing features, and having a second person in that role has made our team much more productive. This is not just because Julia is an exceptional colleague, but because the two of us can now collaborate on statistical analyses or split them up to give each more focus. I did enjoy being the first data scientist at the company, but I’m glad I’m no longer the only one. Julia’s also a skilled writer and communicator, which was essential in achieving the next goal.

Company blog posts

In last year’s post, I shared some of the work that I’d done to explore the landscape of software developers, and set a goal for the following year (emphasis is new):

I’m also just intrinsically pretty interested in learning about and visualizing this kind of information; it’s one of the things that makes this a fun job. One plan for my second year here is to share more of these analyses publicly. In a previous post looked at which technologies were the most polarizing, and I’m looking forward to sharing more posts like that soon.

I’m happy to say that we’ve made this a priority in the last six months. Since December I’ve gotten the opportunity to write a number of posts for the Stack Overflow company blog:

Other members of the team have written data-driven blog posts as well, including:

I’ve really enjoyed sharing these snapshots of the software developer world, and I’m looking forward to sharing a lot more on the blog this next year.

Teaching R at Stack Overflow

Last year I mentioned that part of my work has been developing data science architecture, and trying to spread the use of R at the company.

This also has involved building R tutorials and writing “onboarding” materials… My hope is that as the data team grows and as more engineers learn R, this ecosystem of packages and guides can grow into a true internal data science platform.

At the time, R was used mostly by three of us on the data team (Jason Punyon, Nick Larsen, and me). I’m excited to say it’s grown since then, and not just because of my evangelism.

Every Friday since last September, I’ve met with a group of developers to run internal “R sessions”, in which we analyze some of our data to develop insights and models. Together we’ve made discoveries that have led to real projects and features, for both the Data Team and other parts of the engineering department.

There are about half a dozen developers who regularly take part, and they all do great work. But I especially appreciate Ian Allen and Jisoo Shin for coming up with the idea of these sessions back in September, and for following through in the months since. Ian and Jisoo joined the company last summer, and were interested in learning R to complement their development of product features. Their curiosity, and that of others in the team, has helped prove that data analysis can be a part of every engineer’s workflow.

Writing production code

My relationship to production code (the C# that runs the actual Stack Overflow website) has also changed. In my first year I wrote much more R code than C#, but in the second I’ve stopped writing C# entirely. (My last commit to production was more than a year ago, and I often go weeks without touching my Windows partition). This wasn’t really a conscious decision; it came from a gradual shift in my role on the engineering team. I’d usually rather be analyzing data than shipping features, and focusing entirely on R rather than splitting attention across languages has been helpful for my productivity.

Instead, I work with engineers to implement product changes based on analyses and push models into production. One skill I’ve had to work on is writing technical specifications, both for data sources that I need to query or models that I’m proposing for production. One developer I’d like to acknowledge specifically Nick Larsen, who works with me on the Data Team. Many of the blog posts I mention above answer questions like “What tags are visited in New York vs San Francisco”, or “What tags are visited at what hour of the day”, and these wouldn’t have been possible without Nick. Until recently, this kind of traffic data was very hard to extract and analyze, but he developed processes that extract and transform the data into more readily queryable tables. This has many important analyses possible besides the blog posts, and I can’t appreciate this work enough.

(Nick also recently wrote an awesome post, How to talk about yourself in a developer interview, that’s worth checking out).

Working with other teams

Last year I mentioned that one of my projects was developing targeting algorithms for Job Ads, which match Stack Overflow visitors with jobs they may be interested in (such as, for example, matching people who visit Python and Javascript questions with Python web developer jobs). These are an important part of our business and still make up part of my data science work. But I learned in the last year about a lot of components of the business that data could help more with.

One team that I’ve worked with that I hadn’t in the first year is Display Ads. Display Ads are separate from job ads, and are purchased by companies with developer-focused products and services.

For example, I’ve been excited to work closer with Steve Feldman on the Display Ad Operations team. If you’re wondering why I’m not ashamed to work on ads, please read Steve’s blog post on how we sell display ads at Stack Overflow- he explains it better than I could. We’ve worked on several new methods for display ad targeting and evaluation, and I think there’s a lot of potential for data to have a postive impact for the company.

Changes in the rest of my career

There’ve been other changes in my second year out of academia. In my first year, I attended only one conference (NYR 2016) but I’ve since had more of a chance to travel, including to useR and JSM 2017, PLOTCON, rstudio::conf 2017, and NYR 2017. I spoke at a few of these, about my broom package, about gganimate and about the history of R as seen by Stack Overflow.

Julia and I wrote and published an O’Reilly book, Text Mining with R (now available on Amazon and free online here). I also self-published an e-book, Introduction to Empirical Bayes: Examples from Baseball Statistics, based on a series of blog posts. I really enjoyed the experience of turning blog posts into a larger narrative, and I’d like to continue doing so this next year.

There are some goals I didn’t achieve. I’ve had a longstanding interest in getting R into production (and we’ve idly investigated some approaches like Microsoft R Server), but as of now we’re still productionizing models by rewriting them in C#. And there are many teams at Stack Overflow that I’d like to give better support to- prioritizing the Data Team’s time has been a challenge, though having a second data scientist has helped greatly. But I’m still happy with how my work has gone, and excited about the future.

In any case, this made the whole year worthwhile:

23 Jun 00:49

WWDC Clues Hint at Mac's Future

Apple used this year's WWDC keynote to let the world know it has every intention of supporting the Mac for the long haul. When reading between the lines, it becomes evident that the Mac's future is one of a niche product within Apple's portfolio. Meanwhile, Apple's messaging around the iPad during the same keynote couldn't be any different. As the Mac is going in the direction of niche, the iPad is being groomed to be the Mac/PC alternative for the masses.

Turning Point

The past few years have been an odd stretch for the Mac. Hardware updates have been unusually sporadic although the few updates that did ship were noteworthy. The MacBook Pro with Touch Bar, unveiled in October 2016, represented a prime example of Apple's strategy to use aspects of mobile to push the Mac forward. Apple silicon, Touch ID, and multi-touch displays bring a different kind of experience to the Mac. Throughout this awkward stretch for the Mac, the Apple Industrial Design team's influence on the Mac remained quite obvious, which only increased the level of uneasiness felt by many pro Mac users.

Management used this year's WWDC keynote to officially put an end to this odd Mac stretch. The keynote couldn't have been any different from the previous, somewhat awkward Mac event that took place this past October. This time around, Apple unveiled item after item, clearly targeting pro Mac users.

   

The new $4,999 iMac Pro was announced to much surprise and with certain configurations that were unimaginable for a Mac all-in-one just a few months earlier. Apple's decision to sell an external graphics development kit was also unthinkable just a few months ago.

It is likely that all of these Mac announcements had been in the pipeline for a while. Nevertheless, Apple seemed eager to spin a fresh Mac narrative around pro users. Much of this change in narrative and attitude was a carryover from management's recent Mac intervention with journalists at Apple HQ this past April. At that meeting, SVP Phil Schiller, SVP Craig Federighi, and VP John Ternus clearly telegraphed a revised Mac strategy focused on placing more attention on the tail end of the business. While one would think the new iMac Pro marks Apple's extent into niche Mac territory, the company still plans on releasing a completely redesigned Mac Pro. Given iMac Pro pricing, it is not out of question the for a new Mac Pro configuration to surpass $10,000. Let that sink in for a minute. 

The iPad Juxtaposition

If this year's WWDC keynote doubled as a Mac event (Apple dedicated 23% of stage time to the Mac), the event could have also moonlighted as an iPad event (Apple dedicated 21% of stage time to iPad). My full WWDC review is available for members hereHowever, when the two products were viewed back-to-back, there couldn't have been a more stark difference between them. While the pro in iMac Pro and Mac Pro stood for professional, the pro in iPad Pro stood for productivity.

During the Mac portion of the keynote, management's focus was on addressing the tail end of a 100M Mac user base. A $4,999 iMac Pro is not about adding productivity for the masses. Instead, it is targeting a particular kind of professional. Apple will likely sell fewer iMac Pros than cylinder Mac Pros sold to date. In terms of a redesigned Mac Pro, it is difficult to picture the machine even qualifying as niche. Instead, it will be a niche of a niche.  

Meanwhile, the iPad portion of the WWDC keynote was all about Apple bringing additional productivity to the masses. The new $649 10.5-inch iPad Pro continued Apple's multi-year bet on larger iPad screens (a very sound strategy). Apple also unveiled iOS 11 refinements for iPad that some had been hoping to see for years. One key aspect found in every major iPad software refinement was optionality. For those users craving additional capability, the new software features will prove to be quite valuable. However, for many iPad users, items like multitasking or the new Files app may never be used. In those cases, the same, familiar iOS experience will still be available. Apple was able to add productivity options to the iPad without changing what had gotten the product to where it is today - simplicity and ease of use. The not-so-subtle implication made on stage at WWDC was that the iPad is becoming more of a genuine laptop alternative for hundreds of millions of people.

Change in the Air

This year's WWDC keynote provided a glimpse of the Mac's future. A large portion of the Mac user base are going to find their computing needs met with iPad Pro. According to Apple, approximately 70% to 85% of the current Mac user base does not rely on professional Mac software. This amounts to approximately 80M people. These users are not app developers, nor do they have the need for the kind of power found in a MacBook Pro, iMac Pro, or Mac Pro. Instead, these users are likely attracted to the Mac for keyboard computing. 

As Apple pushes the iPad Pro forward with hardware and software advancements, including various keyboard improvements, these 80M Mac users are going to discover that the iPad is getting better at handling their computing needs. It's not that the Mac will lose value, but rather that a large multi-touch display running iOS will gain value. The shift won't occur overnight for the simple fact that consumers hold on to Macs for years. In addition, it is important to point out that Apple management won't have any issue with this development as long as these Mac users remain within the Apple ecosystem.

Over time, the exodus of non-pro Mac users to iPad Pro will transform the Mac into a niche product category. There will still be millions of users, but the machines will increasingly be geared toward narrow use cases such as VR and AR content creation. In addition, the Mac will become the preferred tool for those in various academic, science, and engineering fields.

  Screen Shot 2017-06-22 at 12.57.41 PM.png  

One may ask, what will happen to consumer-grade Macs, including the MacBook Air and 12-inch MacBook? They will be cannibalized by the iPad Pro line, much as the iPad mini has been cannibalized by larger iPhones. In fact, the entire Mac portable form factor is at risk of cannibalization at the hands of iOS screens. While this won't stop Apple management from pushing the MacBook form factor forward, consumer purchasing habits will speak volumes. 

The Achilles' Heel

Two months ago, I published "The Mac Is Turning into Apple's Achilles' Heel." My thesis was that Apple's inability to move beyond the Mac represents a vulnerability in an otherwise strong product strategy geared toward the mass market. Reaction to the piece came in swift and spanned the spectrum.

The issue facing the Mac has never been Apple's ability to give the product category attention. We saw evidence of this first-hand at this year's WWDC. Apple is able to update the Mac, along with every other product category. In fact, it is not a stretch to say the Mac's outlook within Apple has never been brighter and stronger than it stands today. If one were to place a bet on which current Apple product category will remain within Apple's product line the longest, the Mac would certainly be high on the list. This ends up supporting my thesis that the Mac is Apple's Achilles' heel.

It is very difficult envisioning Apple being able to move beyond the Mac. The product is on track to become a permanent niche within a continuously changing product line.

Apple is moving to a point where the product line will look something like: 

This may not seem like a problem for Apple. The Mac has been responsible for a lot of beneficial things at the company, including inspiring the current generation of content creators. However, the problem for Apple is that having a long tail end of the business comprised of niche Macs may pose a new kind of challenge. Apple Industrial Designers will need to look after the user experience found within a portfolio of mass market product. At the same time, they will need to handle the dramatically different user needs found with a niche product category, the Mac. It's not clear how they will do this. Will Apple designers cede control over the user experience to their engineering peers when it comes to niche Mac hardware? 

Unchartered Territory

While some people look at Apple's big risk as management's inability or unwillingness to move beyond the iPhone, that fear is misplaced. Apple is already moving beyond the iPhone as seen with more personal gadgets worn on our body.

Instead, the genuine risk facing management is that Apple will be unable to move beyond the Mac. This is unchartered territory for Apple. The theory that Apple has to move beyond legacy products in order to completely focus on the future is going to be put to test. It is also possible that the Mac will end up being the first product category that represents genuine growing pains for Apple. In the past, the company would have been able to bring its entire nimble user base from one product category to the next. A niche Mac line will put an end to that era.

Despite gaining niche status, the Mac will still play a major role in creating content consumed on future Apple products including wearables and transportation products. This will give the Mac a level of influence that should not be underestimated. While it is difficult for some to believe, now has never been a better time to be a pro Mac user. This year's WWDC made it clear that the Mac has a future at Apple. However, the amount of change headed towards the Mac should not be underestimated. 

Receive my analysis and perspective on Apple throughout the week via exclusive daily emails (2-3 stories a day, 10-12 stories a week). To sign up, visit the membership page.

22 Jun 18:16

San Jose California Partners with Google on new Transit Village

by Sandy James Planner

google-headquarters-tech-landmarks

The Mercury News reports that San Jose City Council has approved to negotiate only with Google to sell 16 city-owned parcels to the search engine company. Since September 2015 Trammell Crow, Google’s development partner has spent $58.5 million US dollars for an 8.3 acre “transit village” site to potentially build 1 million square feet of offices and 325 apartments.

“This is a once-in-a-century opportunity” for San Jose, Kim Walesh, the city’s economic development director, told the council. “This is a dramatic opportunity to expand the downtown core westward.”

“The transit village would generate millions of dollars in tax revenue and add thousands of tech jobs in an area where experts have estimated that up to 3,000 housing units could be built, city officials said Tuesday. “It will mean more local jobs closer to home,” Nanci Klein, the city’s assistant director of economic development, said in a presentation to the council.”

While thousands of housing units have now been built in the downtown core, it is estimated that a further 3,00 units can be built within this new area. This adds significant tax dollars for San Jose, as well as more high-tech jobs to boost the economy.

“I am supportive of Google’s interest in coming to San Jose and expect they will continue to be the great corporate citizen they have shown to be in other communities,” San Jose City Councilman Sergio Jimenez stated in a letter to the City Council. “It is my sense that Google recognizes and appreciates the impacts this project will have on our city.”

While some locals have decried Google’s choke hold on potential downtown properties in a city that is experiencing high housing demand, the Mayor of  San Jose is more upbeat: “Google is not in the business of solving the city’s problems,” the mayor said. “Google didn’t cause these problems. These are problems we have to solve.”

google-intern


22 Jun 18:16

Curve

by jwz
mkalus shared this story from jwz.

22 Jun 17:22

4 reasons to get the Moto E4

by Dean Daley
Moto E4 in grass

The Lenovo Moto E4, launching now in Canada through Freedom Mobile and various retailers, admittedly doesn’t have the most top-end specs.

The device features a 5-inch, 1280 x 720 pixel display, fingerprint sensor, a 8-megapixel rear facing camera and a 5-megapixel front facing camera. Additionally, it features a 1.4GHz quad-core Qualcomm Snapdragon 425 processor with an Adreno 308 GPU.

But while the internals may be less than impressive on paper, the phone features some unique qualities that give it an edge over other low-end devices on the market.

I’ve compiled a list of the four best things about the Moto E4.

Moto UI

Moto E4 front

Firstly, Moto E4 owners will be among the nearly 10 percent of smartphone users with Android 7.1.1 Nougat. The device gets access to Nougat’s Google Assistant AI, allowing users to have a flowing conversation with their device while the AI retains context.

Not only does the E4 come installed with Android N out of the box, it features many software enhancements that the Moto UI brings to the Android operating system.

One of my favourite things about the Moto E4 is that it runs nearly stock Android with only subtle Moto UI enhancements. Google Now is a swipe to left, while the device’s app tray is a swipe up. Moto also includes an all-in-one clock widget that shows battery life and a weather feature that can tell users how long it will be until it rains or how long the rain is expected to continue.

Another minor improvement over stock Android: the lock screen allows users to view notifications by just holding their thumb against the display is another functionality that slightly improves the user experience with the Moto E4.

Lastly the Moto E4 features a ‘One button nav’ that allows Moto users to navigate their devices using just the fingerprint sensor. Swipe left on the scanner to go the previous page, and swipe right to view recent apps. The one button nav replaces the need for the on-screen navigation buttons.

Price 

Moto E4 fingerprint scanner

The Moto E4 may not have the greatest specs, however, it’s definitely worth more than its cost. The device didn’t stutter or lag at all during my time with it, though I mostly used the smartphone for watching videos on YouTube and searching for memes on Facebook.

Anyone can purchase the Moto E4 for $200 CAD outright at Freedom Mobile, where it’s one of the least expensive phones to run on the budget carrier’s Band 66 LTE network.

When high end devices like the Samsung Galaxy S8 and the LG G6 cost more than $1,000 CAD it’s good to know there are still manageable smartphones out there at reasonable prices. The device will also be available for $249.99 unlocked at retailers including Wal-Mart and Staples.

Battery

Moto E4 side

The Moto E4 features a 2,800mAh battery that lasted throughout the entire day.

Though the device was my main driver while I was using it I’d still say my usage would qualify as minimal to average. This means throughout the day I would use it casually at work and home to search Facebook, stream video and to send texts.

With that being said, by the end of the day — around 9pm — I was left with 50 percent battery life on the device.  The small screen size and low-end processor doesn’t put too much stress on the device’s battery giving a 2,800mAh cell more usage than other smartphones with bigger batteries.

Front facing LED flash

Moto E3 front

Though the Moto E4 has a 5-megapixel camera, it gets a subtle boost in quality due to the front-facing LED flash. Front facing LED flashes are rare among flagship smartphone cameras, though those generally make up for it by featuring cameras with a higher megapixels sensor, a larger aperture to let in more light and a display that flashes white as the camera shoots.

The E4’s front-facing flash means, however, that even with a mere 5-megapixel shooter, the device takes decent selfies in low-light situations.

Bonus: The device feels very durable as it is coated in resin, and though we would never suggest dropping your smartphone — the E4 definitely feels like it can take a hit.

Things to note 

Moto E3 sideThough the E4 is definitely worth $200 it’s definitely not the complete package. The device is water repellent, protecting it from small spills, but it is not water and dust resistant and cannot be submerged completely underwater.

The Moto E4 also does not feature NFC, meaning you cannot use Android Pay or any other mobile wallet application. Lastly the Moto E4 doesn’t come with all of the Moto Actions seen on other Moto devices such as the flicking of the wrist action used to turn on the camera app, or the karate chop that’s used to turn on the torch app.

If you can bear not having any of these features on your device, the Moto E4 might be the budget smartphone for you.

Photography by Rose Behar.

The post 4 reasons to get the Moto E4 appeared first on MobileSyrup.