Here's the thing about living in 600 square feet with a toddler and a baby, you both have to be into it. By both, I mean, you and your partner. If one of you is secretly gunning for extra square footage or hasn't bought into the idea that less is more, then it's going to be a tough go. This was brought to my attention recently when I was complaining to my husband Trevor about some of the difficulties of living small. I was whining about something like no space for baby gear or nap schedules or comparing our space to others (comparison, the thief of all joy!). I was pushing buttons and implying that the space wasn't working for us. Trevor, (with a few more hours of sleep than me) didn't take the bait and quickly reminded me that we choose to live this way, small and with fewer things, and that we are gaining a lot from this choice. I asked him to remind me because in this baby and toddler haze, I sometimes can't remember my own name, let alone our mission to live consciously with less.
We are living within our means and not taking on a larger mortgage that would be stressful to us.
Less overhead expenses means more experiences are possible like travel.
Staying focused on less consumption of material goods for us and our kids. Not buying things for the sake of buying. Filling our house with things we don't need or want just for a moment of shopping bliss.
Less space means less space to maintain, we don't have a yard or large house in need of constant upkeep. Once our space is clean we can spend our time together doing things rather than maintaining our living space.
We have so many amenities and experiences at our fingertips living in the heart of Vancouver like Science World, the Aquarium, Pacific Spirit Park, World Class community centres, the Seawall, Ocean and Mountains (and great coffee and food ...goes without saying)
All the adventures we can have in our Van this summer! I forgot all about Ferdinand (something we couldn't afford if we lived in a much bigger place)
Most importantly we are all healthy and happy and together under one (small, shared) roof.
I am grateful that when I am faltering, he is there to raise me up. And while I was busy worrying about my own small space gripes, I noticed one morning that Trevor gets ready for work basically in the dark so as not to wake the rest of us. I saw him tiptoeing around one morning and realized that he never complains about this inconvenience. And sometimes he does his workouts in the (dark and depressing) parkade after we are all asleep because the gym is closed and we don't have any extra space for a workout. So while it may seem that because I write the blog, that I do this alone but I most certainly do not. Where I am the worrier, analyzer, doer and researcher in pursuit of beautiful things... Trevor is the dreamer, cheerleader, protector and relentless pursuer of functionality. I'm always trying to make our space beautiful and he's always trying to maximize functionality and after the differing opinions, we find our balance and home.
Supercreativity is the latest PopTech ‘Edition’, comprising videos, a reading list and an ongoing discussion forum. In their words:
Supercreativity provided an intimate setting to explore perspectives on creativity as technology tools help us shift from consumption to creation.
I particularly enjoyed the HBR article they link to: ‘Balancing “we” and “me”: the best collaborative spaces also support solitude‘ by Christine Congdon, Donna Flynn and Melanie Redman. While I didn’t agree with some of their conclusions, I did find it thought provoking.I’ve chosen Supercreativity as an inspiration for several reasons:
There are obvious thematic links with our own MEX exploration of ‘Intersection, the pivot between consumption and creativity’.
The format – combinining events, original content, external references and ongoing discussion – is similar in some ways to MEX, but it is always interesting to see how others are doing it.
There was an element of serendipity to this discovery. I found PopTech while visiting friends in the lovely little port of Camden, Maine a few years ago. Their conferences are held in the town each fall. I was there in late October sunshine, just after they’d held their annual event, and the PopTech posters were still up. In the harbour, boats were being put away for the winter, and I thought to myself what a relaxing, thoughtful place it would be to have a good old chat about life, technology and ideas.
The photo above is of the Camden waterfront, where PopTech is held. It is so cold in the winter they shrink wrap their boats to keep them cosy.
Part of Friday Inspirations, an ongoing MEX series exploring tangents and their relationship to better experience design. We explain the origins of the Inspirations series in this MEX podcast and article. Share your own inspirations on Twitter at #mexDTI.
This site features a curated selection of data visualization tools meant to bridge the gap between programmers/statisticians and the general public by only highlighting free/freemium, responsive and relatively simple-to-learn technologies for displaying both basic and complex, multivariate datasets. It leans heavily toward open-source software and plugins, rather than enterprise, expensive B.I. solutions.
I found some broken links, and the descriptions need a little editing, but it’s a good place to start.
Also, if you’re just starting out with visualization, you might find all the resources a bit overwhelming. If that’s the case, don’t fret. You don’t have to learn how to use all of them. Let your desired outcomes guide you. Here’s what I use.
Over the last few years, I’ve experimented on and off with various recipes for creating text reports from tabular data sets, (spreadsheet plugins are also starting to appear with a similar aim in mind). There are several issues associated with this, including:
identifying what data or insight you want to report from your dataset;
(automatically deriving the insights);
constructing appropriate sentences from the data;
organising the sentences into some sort of narrative structure;
making the sentences read well together.
Another approach to humanising the reporting of tabular data is to generate templated webpages that review and report on the contents of a dataset; this has certain similarities to dashboard style reporting, mixing tables and charts, although some simple templated text may also be generated to populate the page.
In a business context, reporting often happens via Powerpoint presentations. Slides within the presentation deck may include content pulled from a templated spreadsheet, which itself may automatically generate tables and charts for such reuse from a new dataset. In this case, the recipe may look something like:
So as hinted at previously, it’s possible to use the following sort of process to automatically generate reveal.js slideshows from a Jupyter notebook with appropriately configured slide cells (actually, normal cells with an appropriate metadata element set) used as an intermediate representation.
There’s an example slideshow based on October 2016 data here. Note that some slides have “subslides”, that is, slides underneath them, so watch the arrow indicators bottom left to keep track of when they’re available. Note also that the scrolling is a bit hit and miss – ideally, a new slide would always be scrolled to the top, and for fragments inserted into a slide one at a time the slide should scroll down to follow them).
The structure of the presentation is broadly as follows:
For example, here’s a summary slide of the spends by directorate – note that we can embed charts easily enough. (The charts are styled using seaborn, so a range of alternative themes are trivially available). The separate directorate items are brought in one at a time as fragments.
The next slide reviews the capital versus expenditure revenue spend for a particular directorate, broken down by expenses type (corresponding slides are generated for all other directorates). (I also did a breakdown for each directorate by service area.)
The items listed are ordered by value, and taken together account for at least 80% of the spend in the corresponding area. Any further items contributing more than 5%(?) of the corresponding spend are also listed.
Notice that subslides are available going down from this slide, rather than across the mains slides in the deck. This 1.5D structure means we can put an element of flexible narrative design into the presentation, giving the reader an opportunity to explore the data, but in a constrained way.
In this case, I generated subslides for each major contributing expenses type to the capital and revenue pots, and then added a breakdown of the major suppliers for that spending area.
This just represents a first pass at generating a 1.5D slide deck from a tabular dataset. A Pareto (80/20) heurstic is used to try to prioritise to the information displayed in order to account for 80% of spend in different areas, or other significant contributions.
Applying this principle repeatedly allows us to identify major spending areas, and then major suppliers within those spending areas.
The next step is to look at other ways of segmenting and structuring the data in order to produce reports that might actually be useful…
If you have any ideas, please let me know via the comments, or get in touch directly…
PS FWIW, it should be easy enough to run any other dataset that looks broadly like the example at the top through the same code with only a couple of minor tweaks…
I was first introduced to Paulo Freire by my wife, who asked me to create some illustrations for her PGCE presentation. It influenced what I now call conversational thinkery – articulating something with a person’s bias thrown in, giving you clues as to what the articulation might look like.
I’m now trying to permeate the text of Radical Pedagogy itself, with my slow-reading eyes and picture-oriented brain…
Amy Poehler explained a lot about how much work goes into being funny, but some people make you smile without years of training and practice. (Amy Poehler was probably like that before all those years of improv, but hardly anybody knew that.). But the strangest comedy highlight has got to be Kennedy Steve, an air traffic controller with a big fan base.
Seriously! This guy is a traffic cop, sorting out airplanes that aren’t even flying yet, just taxiing to and from their gates. Most of the time, he’s telling people to wait. Occasionally, he’ll ask someone to hurry up. Mostly, he gives planes cryptic instructions for taxiing through the big airport.
Jet Blue 359, after the RJ, you can continue on to ALPHA all the way to KILO-ECHO. ’Cause you’re the next to go.
But he’s infectiously funny. When he’s working a shift, he’ll be the most listened-to channel at LiveATC. At least two separate people condense his traffic alerts to take out the pauses between calls, type out transcripts and upload them to Youtube where they regularly get tens or hundreds of thousands of listeners. (There are also lesser stars – grumpy Kennedy Jack and enthusiastic Boston John – but Kennedy Steve is clearly the big draw.)
What makes Kennedy Steve funny? I’ve been trying to work it out. I have a few ideas:
Mock Epic: A subtext of air traffic control is always the risk of terrible accidents, but most interactions involve minor annoyances and small delays. The context of these annoyances, however, is a power struggle between airline pilots (who are Masters Of Their Craft) and controllers (who are telling those Masters where to go, which is something they cannot be expected to enjoy). Kennedy Steve systematically exaggerates the challenges and delays and vexations while making light of them. When bad weather shuts down a big airport somewhere and lots of departing flights have to wait, he doesn’t just tell them to wait – he tells them to “call Clearance for an incredibly creative re-routing!” When the gate supervisors are clogging up the taxiways, he tells pilots that “your ramp is simplystellar!”
Heroic Struggle: The Roadrunner always has the Coyote and Acme Industries. Steve has tugs — the vehicles push and tow airplanes at the gate. Like planes, tugs need to get permission from Ground Control to go places. Steve is always complaining about the tugs, because they dawdle on his taxiways or don’t listen to their radios. Especially super tugs. Steve loves to complain that super tugs aren’t very super. It’s an eternal struggle, and that’s funny.
Funny Names: Taxiways are designated by letters. Because it’s very hard to hear letters accurately, controllers use the phonetic alphabet, and avoid using initials for anything else. This is a problem when you want someone to follow the MD-80 ahead of them, so controllers sometimes call that airplane a “Mad Dog”. Steve tells pilots to “Follow the angry puppy and contact the tower.”
NetJet flights are “the 1%.” During the USAir/American merger, flights that used one airline’s flight number but the other airline’s paint were “in disguise.”
Schtick: Steve doesn’t have a ton of jokes, but he has some running gags. Like the funny names, these give people permission to laugh.
Because of the way the airport is set up, some British Airways (callsign “Speedbird”) flights have to ask special permission to push back from their gate. The typical interchange is:
BA17: Which direction do you want us to face? Speedbird 17.
Ground: Oh, face the front of the aircraft, sir! If you sit facing the rear, the people in back get frightened. But the plane can face SW.
It’s generally BA – I think I’ve heard it with Quantas, too. I don’t know if it’s limited to specific pilots. It’s not that funny, and it’s only a joke once, but the running gag quality sells the rest of it.
Discipline: Occasionally, people break rules, or things get fouled up. That’s always fraught, but Steve tends to finesse this by making fun of a common enemy. During rush hour when Steve is trying to get everyone parked or to the runway efficiently, there’s always some plane that has no gate and no place to go.
Plane: We’re gonna have to wait somewhere until the gate opens up.
Ground: Stellar!
Plane: Where’d you like us to wait?
Ground: Atlanta?
When pilots talk to air traffic controllers, the convention is that they identify who they’re calling (in case they’re on the wrong frequency) and who they are (because one controller is dealing with lots of planes). Inevitably, someone forgets.
Unknown Plane: Ground?
Ground: Plane?
But Steve makes an exception for his natural enemy, the tug: if a tug does something wrong, it’s a Big Deal and Steve will threaten to punish them by sending them the long way around.
Every Friday WordPress posts a single word prompt for a photo. Today’s is Graceful
The sculpture is called “Olas de Viento” and was installed in Garry Point Park in the City of Richmond BC by the Vancouver Biennale. The photo was taken in December 2009. I was very taken by the subject and made several images at that time. The City decided not to buy it and by March 8, 2012 it had gone.
The name translates as “Wind waves” and the sculptor is Yvonne Domenge from Mexico
It is now installed at Herman Park in Houston, who clearly have much better taste than the Mayor and Councillors of Richmond.
NewsBits for the week ending January 20th: Some PostgreSQL tips and SSL news, the RethinkDB post-mortem, a JetBrains Go IDE announced, and a brief comparison of some Functional Javascript systems.
NewsBits is database news, developer news, cloud news and some curiosities from the IT world. Set the controls for the heart of the NewsBits!
No PostgreSQL server restarts for SSL reconfigs in Postgres 10 - Currently new certs or any reconfig of SSL requires a full restart of PostgreSQL. Not great for production servers. We at Compose are excited to see the following feature coming in Postgres 10: Reload SSL certificates on SIGHUP.
Experimental Memory Defragmentation in Redis - While it may not apply to all, this experimental checkin might be important if you need it.
Always pay attention to MySQL/MariaDB warnings - A healthy reminder to heed warnings.
Other Bits
RethinkDB post-mortem from Slava Akmechet - A thoughtful and insightful review of why RethinkDB failed from the leader of the RethinkDB team.
Ransomware continues, now CouchDB and Hadoop - More openly accessible deployments are attacked and the data being wiped out.
Development Bits
Gogland a commercial IDE for Go from Jetbrains - Go's continued march of gaining mind share among developers will certainly be helped by this upcoming IDE from JetBrains, the makers of IntelliJ Idea.
Javascript vs Elm vs Purescript vs GHCjs vs Scalajs - A quick review of some of the tradeoffs between these purely functional programming platforms for the browser.
If you have any feedback about this or any other Compose article, drop the Compose Articles team a line at articles@compose.com. We're happy to hear from you.
Today wasn’t so bad. Sure, the enemies of democracy, the Constitution, rationality, compassion, national and international institutions, decency, competence, ethics, art, science, and truth itself — and of anyone who isn’t a white man — now darken the offices of power. But they haven’t done much yet. The bad days are still to come.
My loathing and contempt for President Trump feels complete — but it isn’t. It will continue to deepen.
I'm very happy with the discussion and chatroom approach in general!
Regarding cleaning up the planning code: I invented and started implementing a much nicer new architecture. It will probably take 1-2 more days to get everything working again, but oh, will it be worth it!
After doing 13 hours of research and considering 76 models, we performed dozens of hours of real-world testing and 13 additional hours of focused, in-depth testing on the top 14 Bluetooth-audio receivers for adding wireless connectivity to an existing audio system. We think the StarTech BT2A Bluetooth Audio Receiver is the best receiver for most people thanks to its combination of connectivity, range, audio quality, and usability.
On
the racket-users mailing list
yesterday, Matthias Felleisen issued "a research challenge that
is common in the Racket world":
If you are here and you see the blueprints for paradise over
there, don't just build paradise. Also build the bridge
from here to there.
This is one of the things I love about Racket. And I don't use
even 1% of the goodness that is Racket and its ecosystem.
Over the last couple of years, I have been migrating my
Programming Languages course from a Scheme subset of Racket to
Racket itself. Sometimes, this is simply a matter of talking
about Racket, not Scheme. Others, it means using some of the
data structures, functions, and tools Racket provides rather
than doing without or building our own. Occasionally, this
shift requires changing something I do in class, because Racket
is fussier than Scheme in some regards. That's usually a good
thing, because the change makes Racket a better language for
engineering big programs. In general, though, the shift goes
smoothly.
Occasionally, the only challenge is a personal one. For
example, I decided to use first and rest
this semester when working with lists, instead of car
and cdr. This should make some students' lives better.
Learning a new language and a new style and new IDE all at once
can be tough for students with only a couple of semesters'
programming experience, and using words that mean what they say
eliminates one unnecessary barrier. But, as I
tweeted,
I don't feel whole or entirely clean when I do so. As my
college humanities prof taught me through Greek tragedies, old
habits die hard, if at all.
One of my goals for the course this semester is to have the
course serve as a better introduction to Racket for students
who might be inclined to take advantage of its utility and
power in later courses, or who simply want to enjoy working in
a beautiful language. I always seem to have a few who do, but
it might be nice if even more left the course thinking of
Racket as a real alternative for their project work. We'll
see how it goes.
Ill. Mostly over last weekend, really, but I felt pretty rough on Monday. Just a cold (I can’t pretend it was ‘flu’ given I always get my shot) but I was glad to get rid of it!
Sending outThought Shrapnel, my weekly newsletter. Ostensibly it’s about education, technology, and productivity, but really it’s me trying to make sense of the many and varied things I’ve ready over the past week. Issue 242 included many things, including why ‘screen time’ is a useless concept.
Supporting a webinar, the second of a series from We Are Open Co-op and Educators Co-op. This one was on designing an effective badging website, led by Steve Regur, with help from Laura Hilliger and me. You can catch the recording and a short write-up via this post.
Travelling for the first time in 2017, this time to Jersey. I’ve got plenty coming up over the next few weeks…
Working with staff at Victoria College on their digital strategy. The staff are lovely there, they really are.
Reading more of Deep Work by Cal Newport and starting Luc P. Beaudoin’s Cognitive Productivity. I’m more impressed with the former than the latter.
Announcing a BADGE BOOTCAMP in London on February 15th. Spread the word!
Making changes to, and launching the WordPress version of, our new church website. Straightforward stuff, but just takes time.
Putting the final things in place around upcoming trips to Geneva and Rome with new clients.
Next week I’m working from home, setting up my new laptop (which should arrive on Monday) and then heading to London for BETT. I’ll be there on Friday afternoon and all day Saturday. If you’ll be there, send me a tweet! (@dajbelshaw)
I earn my living helping people and organisations become more productive in their use of technology. If you’ve got something that you think I, or the co-op I’m part of may be able to help with, please do get in touch! Email addresses below:
As of r209827, 64-bit ARM and x86 WebKit ports use a new garbage collector called Riptide. Riptide reduces worst-case pause times by allowing the app to run concurrently to the collector. This can make a big difference for responsiveness since garbage collection can easily take 10 ms or more, even on fast hardware. Riptide improves WebKit’s performance on the JetStream/splay-latency test by 5x, which leads to a 5% improvement on JetStream. Riptide also improves our Octane performance. We hope that Riptide will help to reduce the severity of GC pauses for many different kinds of applications.
This post begins with a brief background about concurrent GC (garbage collection). Then it describes the Riptide algorithm in detail, including the mature WebKit GC foundation, on which it is built. The field of incremental and concurrent GC goes back a long time and WebKit is not the first system to use it, so this post has a section about how Riptide fits into the related work. This post concludes with performance data.
Introduction
Garbage collection is expensive. In the worst case, for the collector to free a single object, it needs to scan the entire heap to ensure that no objects have any references to the one it wants to free. Traditional collectors scan the entire heap periodically, and this is roughly how WebKit’s collector has worked since the beginning.
The problem with this approach is that the GC pause can be long enough to cause rendering loops to miss frames, or in some cases it can even take so long as to manifest as a spin. This is a well-understood computer science problem. The originally proposed solution for janky GC pauses, by Guy Steele in 1975, was to have one CPU run the app and another CPU run the collector. This involves gnarly race conditions that Steele solved with a bunch of locks. Later algorithms like Baker’s were incremental: they assumed that there was one CPU, and sometimes the application would call into the collector but only for bounded increments of work. Since then, a huge variety of incremental and concurrent techniques have been explored. Incremental collectors avoid some synchronization overhead, but concurrent collectors scale better. Modern concurrent collectors like DLG (short for Doligez, Leroy, Gonthier, published in POPL ’93 and ’94) have very cheap synchronization and almost completely avoid pausing the application. Taking garbage collection off-core rather than merely shortening the pauses is the direction we want to take in WebKit, since almost all of the devices WebKit runs on have more than one core.
The goal of WebKit’s new Riptide concurrent GC is to achieve a big reduction in GC pauses by running most of the collector off the main thread. Because Riptide will be our always-on default GC, we also want it to be as efficient — in terms of speed and memory — as our previous collector.
The Riptide Algorithm
The Riptide collector combines:
Marking: The collector marks objects as it finds references to them. Objects not marked are deleted. Most of the collector’s time is spent visiting objects to find references to other objects.
Constraints: The collector allows the runtime to supply additional constraints on when objects should be marked, to support custom object lifetime rules.
Parallelism: Marking is parallelized on up to eight logical CPUs. (We limit to eight because we have not optimized it for more CPUs.)
Generations: The collector lets the mark state of objects stick if memory is plentiful, allowing the next collection to skip visiting those objects. Sticky mark bits are a common way of implementing generational collection without copying. Collection cycles that let mark bits stick are called eden collections in WebKit.
Concurrency: Most of the collector’s marking phase runs concurrently to the program. Because this is by far the longest part of collection, the remaining pauses tend to be 1 ms or less. Riptide’s concurrency features kick in for both eden and full collections.
Conservatism: The collector scans the stack and registers conservatively, that is, checking each word to see if it is in the bounds of some object and then marking it if it is. This means that all of the C++, assembly, and just-in-time (JIT) compiler-generated code in our system can store heap pointers in local variables without any hassles.
Efficiency: This is our always-on garbage collector. It has to be fast.
This section describes how the collector works. The first part of the algorithm description focuses on the WebKit mark-sweep algorithm on which Riptide is based. Then we dive into concurrency and how Riptide manages to walk the heap while the heap is in flux.
Efficient Mark-Sweep
Riptide retains most of the basic architecture of WebKit’s mature garbage collection code. This section gives an overview of how our mark-sweep collector works: WebKit uses a simple segregated storage heap structure. The DOM, the Objective-C API, the type inference runtime, and the compilers all introduce custom marking constraints, which the GC executes to fixpoint. Marking is done in parallel to maximize throughput. Generational collection is important, so WebKit implements it using sticky mark bits. The collector uses conservative stack scanning to ease integration with the rest of WebKit.
Simple Segregated Storage
WebKit has long used the simple segregated storage heap structure for small and medium-sized objects (up to about 8KB):
Small and medium-sized objects are allocated from segregated free lists. Given a desired object size, we perform a table lookup to find the appropriate free list and then pop the first object from this list. The lookup table is usually constant-folded by the compiler.
Memory is divided into 16KB blocks. Each block contains cells. All cells in a block have the same cell size, called the block’s size class. In WebKit jargon, an object is a cell whose JavaScript type is “object”. For example, a string is a cell but not an object. The GC literature would typically use object to refer to what our code would call a cell. Since this post is not really concerned with JavaScript types, we’ll use the term object to mean any cell in our heap.
At any time, the active free list for a size class contains only objects from a single block. When we run out of objects in a free list, we find the next block in that size class and sweep it to give it a free list.
Sweeping is incremental in the sense that we only sweep a block just before allocating in it. In WebKit, we optimize sweeping further with a hybrid bump-pointer/free-list allocator we call bump’n’pop (here it is in C++ and in the compilers). A per-block bit tells the sweeper if the block is completely empty. If it is, the sweeper will set up a bump-pointer arena over the whole block rather than constructing a free-list. Bump-pointer arenas can be set up in O(1) time while building a free-list is a O(n) operation. Bump’n’pop achieves a big speed-up on programs that allocate a lot because it avoids the sweep for totally-empty blocks. Bump’n’pop’s bump-allocator always bumps by the block’s cell size to make it look like the objects had been allocated from the free list. This preserves the block’s membership in its size class.
Large objects (larger than about 8KB) are allocated using malloc.
Constraint-Based Marking
Garbage collection is ordinarily a graph search problem and the heap is ordinarily just a graph: the roots are the local variables, their values are directional edges that point to objects, and those objects have fields that each create edges to some other objects. WebKit’s garbage collector also allows the DOM, compiler, and type inference system to install constraint callbacks. These constraints are allowed to query which objects are marked and they are allowed to mark objects. The WebKit GC algorithm executes these constraints to fixpoint. GC termination happens when all marked objects have been visited and none of the constraints want to mark anymore objects. In practice, the constraint-solving part of the fixpoint takes up a tiny fraction of the total time. Most of the time in GC is spent performing a depth-first search over marked objects that we call draining.
Parallel Draining
Draining takes up most of the collector’s time. One of our oldest collector optimizations is that draining is parallelized. The collector has a draining thread on each CPU. Each draining thread has its own worklist of objects to visit, and ordinarily it runs a graph search algorithm that only sees this worklist. Using a local worklist means avoiding worklist synchronization most of the time. Each draining thread will check in with a global worklist under these conditions:
It runs out of work. When a thread runs out of work, it will try to steal 1/Nth of the global worklist where N is the number of idle draining threads. This means acquiring the global worklist’s lock.
Every 100 objects visited, the draining thread will consider donating about half of its worklist to the global worklist. It will only do this if the global worklist is empty, the global worklist lock can be acquired without blocking, and the local worklist has at least two entries.
This algorithm appears to scale nicely to about eight cores, which is good enough for the kinds of systems that WebKit usually runs on.
Draining in parallel means having to synchronize marking. Our marking algorithm uses a lock-free CAS (atomic compare-and-swap instruction) loop to set mark bits.
Sticky Mark Bits
Generational garbage collection is a classic throughput optimization first introduced by Lieberman and Hewitt and Ungar. It assumes that objects that are allocated recently are unlikely to survive. Therefore, focusing the collector on objects that were allocated since the last GC is likely to free up lots of memory — almost as much as if we collected the whole heap. Generational collectors track the generation of objects: either young or old. Generational collectors have (at least) two modes: eden collection that only collects young objects and full collection that collects all objects. During an eden collection, old objects are only visited if they are suspected to contain pointers to new objects.
Generational collectors need to overcome two hurdles: how to track the generation of objects, and how to figure out which old objects have pointers to new objects.
The collector needs to know the generation of objects in order to determine which objects can be safely ignored during marking. In a traditional generational collector, eden collections move objects and then use the object’s address to determine its generation. Our collector does not move objects. Instead, it uses the mark bit to also track generation. Quite simply, we don’t clear any mark bits at the start of an eden collection. The marking algorithm will already ignore objects that have their mark bits set. This is called sticky mark bit generational garbage collection.
The collector will avoid visiting old objects during an eden collection. But it cannot avoid all of them: if an old object has pointers to new objects, then the collector needs to know to visit that old object. We use a write barrier — a small piece of instrumentation that executes after every write to an object — that tells the GC about writes to old objects. In order to cheaply know which objects are old, the object header also has a copy of the object’s state: either it is old or it is new. Objects are allocated new and labeled old when marked. When the write barrier detects a write to an old object, we tell the GC by setting the object’s state to old-but-remembered and putting it on the mark stack. We use separate mark stacks for objects marked by the write barrier, so when we visit the object, we know whether we are visiting it due to the barrier or because of normal marking (i.e. for the first time). Some accounting only needs to happen when visiting the object for the first time. The complete barrier is simply:
object->field = newValue;
if (object->cellState == Old)
remember(object);
Generational garbage collection is an enormous improvement in performance on programs that allocate a lot, which is common in JavaScript. Many new JavaScript features, like iterators, arrow functions, spread, and for-of allocate lots of objects and these objects die almost immediately. Generational GC means that our collector does not need to visit all of the old objects just to delete the short-lived garbage.
Conservative Roots
Garbage collection begins by looking at local variables and some global state to figure out the initial set of marked objects. Introspecting the values of local variables is tricky. WebKit uses C++ local variables for pointers to the garbage collector’s heap, but C-like languages provide no facility for precisely introspecting the values of specific variables of arbitrary stack frames. WebKit solves this problem by marking objects conservatively when scanning roots. We use the simple segregated storage heap structure in part because it makes it easy to ask whether an arbitrary bit pattern could possibly be a pointer to some object.
We view this as an important optimization. Without conservative root scanning, C++ code would have to use some API to notify the collector about what objects it points to. Conservative root scanning means not having to do any of that work.
Mark-Sweep Summary
Riptide implements complex notions of reachability via arbitrary constraint callbacks and allows C++ code to manipulate objects directly. For performance, it parallelizes marking and uses generations to reduce the average amount of marking work.
Handling Concurrency
Riptide makes the draining phase of garbage collection concurrent. This works because of a combination of concurrency features:
Riptide is able to stop the world for certain tricky operations like stack scanning and DOM constraint solving.
Riptide uses a retreating wavefront write barrier to manage races between marking and object mutation. Using retreating wavefront allows us to avoid any impedance mismatch between generational and concurrent collector optimizations.
Retreating wavefront collectors can suffer from the risk of GC death spirals, so Riptide uses a space-time scheduler to put that in check.
Visiting an object while it is being reshaped is particularly hard, and WebKit reshapes objects as part of type inference. We use an obstruction-free double collect snapshot to ensure that the collector never marks garbage memory due to a visit-reshape race.
Lots of objects have tricky races that aren’t on the critial path, so we put a fast, adaptive, and fair lock in every JavaScript object as a handy way to manage them. It fits in two otherwise unused bits.
While we wrote Riptide for WebKit, we suspect that the underlying intuitions could be useful for anyone wanting to write a concurrent, generational, parallel, conservative, and non-copying collector. This section describes Riptide in detail.
Stopping The World and Safepoints
Riptide does draining concurrently. It is a goal to eventually make other phases of the collector concurrent as well. But so long as some phases are not safe to run concurrently, we need to be able to bring the application to a stop before performing those phases. The place where the collector stops needs to be picked so as to avoid reentrancy issues: for example stopping to run the GC in the middle of the GC’s allocator would create subtle problems. The concurrent GC avoids these problems by only stopping the application at those points where the application would trigger a GC. We call these safepoints. When the collector brings the application to a safepoint, we say that it is stopping the world.
Riptide currently stops the world for most of the constraint fixpoint, and resumes the world for draining. After draining finishes, the world is again stopped. A typical collection cycle may have many stop-resume cycles.
Retreating Wavefront
Draining concurrently means that just as we finish visiting some object, the application may store to one of its fields. We could store a pointer to an unmarked object into an object that is already visited, in which case the collector might never find that unmarked object. If we don’t do something about this, the collector would be sure to prematurely delete objects due to races with the application. Concurrent garbage collectors avoid this problem using write barriers. This section describes Riptide’s write barrier.
Write barriers ensure that the state of the collector is still valid after any race, either by marking objects or by having objects revisited (GC Handbook, chapter 15). Marking objects helps the collector make forward progress; intuitively, it is like advancing the collector’s wavefront. Having objects revisited retreats the wavefront. The literature of full of concurrent GC algorithms, like the Metronome, C4, and DLG, that all use some kind of advancing wavefront write barrier. The simplest such barrier is Dijkstra’s, which marks objects anytime a reference to them is created. I used thesekinds of barriersin my pastwork because theymake it easyto make thecollector very deterministic. Adding one of those barriers to WebKit would be likely to create some performance overhead since this means adding new code to every write to the heap. But the retreating wavefront barrier, originally invented by Guy Steele in 1975, works on exactly the same principle as our existing generational barrier. This allows Riptide to achieve zero barrier overhead by reusing WebKit’s existing barrier.
It’s easiest to appreciate the similarity by looking at some barrier code. Our old generational barrier looked like this:
object->field = newValue;
if (object->cellState == Old)
remember(object);
Steele’s retreating wavefront barrier looks like this:
object->field = newValue;
if (object->cellState == Black)
revisit(object);
Retreating wavefront barriers operate on the same principle as generational barriers, so it’s possible to use the same barrier for both. The only difference is the terminology. The black state means that the collector has already visited the object. This barrier tells the collector to revisit the object if its cellState tells us that the collector had already visited it. This state is part of the classic tri-color abstraction: white means that the GC hasn’t marked the object, grey means that the object is marked and on the mark stack, and black means that the object is marked and has been visited (so is not on the mark stack anymore). In Riptide, the tri-color states that are relevant to concurrency (white, grey, black) perfectly overlap with the sticky mark-bit states that are relevant to generations (new, remembered, old). The Riptide cell states are as follows:
DefinitelyWhite: the object is new and white.
PossiblyGrey: the object is grey, or remembered, or new and white.
PossiblyBlack: the object is black and old, or grey, or remembered, or new and white.
A naive combination generational/concurrent barrier might look like this:
object->field = newValue;
if (object->cellState == PossiblyBlack)
slowPath(object);
This turns out to need tweaking to work. The PossiblyBlack state is too ambiguous, so the slowPath needs additional logic to work out what the object’s state really was. Also, the order of execution matters: the CPU must run the object->cellState load after it runs the object->field store. That’s hard, since CPUs don’t like to obey store-before-load orderings. Finally, we need to guarantee that the barrier cannot retreat the wavefront too much.
Disambiguating Object State
The GC uses the combination of the object’s mark bit in the block header and the cellState byte in the object’s header to determine the object’s state. The GC clears mark bits at the start of full collection, and it sets the cellState during marking and barriers. It doesn’t reset objects’ cellStates back to DefinitelyWhite at the start of a full collection, because it’s possible to infer that the cellState should have been reset by looking at the mark bit. It’s important that the collector never scans the heap to clear marking state, and even mark bits are logically cleared using versioning. If an object is PossiblyBlack or PossiblyGrey and its mark bit is logically clear, then this means that the object is really white. Riptide’s barrier slowPath is almost like our old generational slow path but it has a new check: it will not do anything if the mark bit of the target object is not set, since this means that we’re in the middle of a GC and the object is actually white. Additionally, the barrier will attempt to set the object back to DefinitelyWhite so that the slowPath path does not have to see the object again (at least not until it’s marked and visited).
Store-Before-Barrier Ordering
The GC must flag the object as PossiblyBlack just before it starts to visit it and the application must store to field before loading object->cellState. Such ordering is not guaranteed on any modern architecture: both x86 and ARM will sink the store below the load in some cases. Inserting an unconditional store-load fence, such as lock; orl $0, (%rsp) on x86 or dmb ish on ARM, would degrade performance way too much. So, we make the fence itself conditional by playing a trick with the barrier’s condition:
object->field = newValue;
if (object->cellState <= blackThreshold)
slowPath(object);
Where blackThreshold is a global variable. The PossiblyBlack state has the value 0, and when the collector is not running, blackThreshold is 0. But once the collector starts marking, it sets blackThreshold to 100 while the world is stopped. Then the barrier’s slowPath leads with a check like this:
storeLoadFence();
if (object->cellState != PossiblyBlack)
return;
This means that the application takes a slight performance hit while Riptide is running. In typical programs, this overhead is about 5% during GC and 0% when not GCing. The only additional cost when not GCing is that blackThreshold must be loaded from memory, but we could not detect a slow-down due to this change. The 5% hit during collection is worth fixing, but to put it in perspective, the application used to take a 100% performance hit during GC because the GC would stop the application from running.
The complete Riptide write barrier is emitted as if the following writeBarrier function had been inlined just after any store to target:
ALWAYS_INLINE void writeBarrier(JSCell* target)
{
if (LIKELY(target->cellState() > blackThreshold))
return;
storeLoadFence();
if (target->cellState() != PossiblyBlack)
return;
writeBarrierSlow(target);
}
NEVER_INLINE void writeBarrierSlow(JSCell* target)
{
if (!isMarked(target)) {
// Try to label this object white so that we don't take the barrier
// slow path again.
if (target->compareExchangeCellState(PossiblyBlack, DefinitelyWhite)) {
if (Heap::isMarked(target)) {
// A race! The GC marked the object in the meantime, so
// pessimistically label it black again.
target->setCellState(PossiblyBlack);
}
}
return;
}
target->setCellState(DefinitelyGrey);
m_mutatorMarkStack->append(target);
}
When the barrier does append the object to the m_mutatorMarkStack, the object will get revisited eventually. The revisit could happen concurrently to the application. That’s important since we have seen programs retreat the wavefront enough that the total revisit pause would be too big otherwise.
Unlike advancing wavefront, retreating wavefront means forcing the collector to redo work that it has already done. Without some facilities to ensure collector progress, the collector might never finish due to repeated revisit requests from the write barrier. Riptide tackles this problem in two ways. First, we defer all revisit requests. Draining threads do not service any revisit requests until they have no other work to do. When an object is flagged for revisiting, it stays in the grey state for a while and will only be revisited towards the end of GC. This ensures that if an old object often has its fields overwritten with pointers to new objects, then the GC will usually only scan two snapshots’ worth of those fields: one snapshot whenever the GC visited the object first, and another towards the end when the GC gets around to servicing deferred revisits. Revisit deferral reduces the likelihood of runaway GC, but fully eliminating such pathologies is left to our scheduler.
Space-Time Scheduler
The bitter end of a retreating wavefront GC cycle is not pretty: just as the collector goes to visit the last object on the mark stack, some object that had already been visited gets written to, and winds up back on the mark stack. This can go on for a while, and before we had any mitigations we saw Riptide using 5x more memory than with synchronous collection. This death spiral happens because programs allocate a lot all the time and the collector cannot free any memory until it finishes marking. Riptide prevents death spirals using a scheduler that controls the application’s pace. We call it the space-time scheduler because it links the amount of time that the application gets to run for in a timeslice to the amount of space that the application has used by allocating in the collector’s headroom.
The space-time scheduler ensures that the retreating wavefront barrier cannot wreak havoc by giving the collector an unfair advantage: it will periodically stop the world for short pauses even when the collector could be running concurrently. It does this just so the collector can always outpace the application in case of a race. If this was meant as a garbage collector for servers, you could imagine providing the user with a bunch of knobs to control the schedule of these synthetic pauses. Different applications will have different ideal pause lengths. Applications that often write to old memory will retreat the collector’s wavefront a lot, and so they will need a longer pause to ensure termination. Functional-style programs tend to only write to newly allocated objects, so those could get away with a shorter pause. We don’t want web users or web developers to have to configure our collector, so the space-time scheduler adaptively selects a pause schedule.
To be correct, the scheduler must eventually pause the world for long enough to let the collector terminate. The space-time scheduler is based on a simple idea: the length of pauses increases during collection in response to how much memory the application is using.
The space-time scheduler selects the duration and spacing of synthetic pauses based on the headroom ratio, which is a measure of the amount of extra memory that the application has allocated during the concurrent collection. A concurrent collection is triggered by memory usage crossing the trigger threshold. Since the collector allows the application to keep running, the application will keep allocating. The space that the collector makes available for allocation during collection is called the headroom. Riptide is tuned for a max headroom that is 50% larger than the trigger threshold: so if the app needed to allocate 100MB to trigger a collection, its max headroom is 50MB. We want the collector to complete synchronously if we ever deplete all of our headroom: at that point it’s better for the system to pause and free memory than to run and deplete even more memory. The headroom ratio is simply the available headroom divided by the max headroom. The space-time scheduler will divide time into fixed timeslices, and the headroom ratio determines how much time the application is resumed for during that period.
The default tuning of our collector is that the collector timeslice is 2 ms, and the first C ms of it is given to the collector and the remaining M ms is given to the mutator. We always let the collector pause for at least 0.6 ms. Let H be the headroom ratio: 1 at the start of collection, and 0 if we deplete all headroom. With a 0.6 ms minimum pause and a 2 ms timeslice, we define M and C as follows:
M = 1.4 H C = 2 – M
For example, at the start of usual collection we will give 0.6 ms to the collector and then 1.4 ms to the application, but as soon as the application starts allocating, this window shifts. Aggressive applications, which both allocate a lot and write to old objects a lot, will usually end collection with the split being closer to 1 ms for the collector followed by 1 ms for the application.
Thanks to the space-time scheduler, the worst that an adversarial program could do is cause the GC to keep revisiting some object. But it can’t cause the GC to run out of memory, since if the adversary uses up all of the headroom then M becomes 0 and the collector gets to stop the world until the end of the cycle.
Obstruction-Free Double Collect Snapshot
Concurrent garbage collection means finding exciting new ways of side-stepping expensive synchronization. In traditional concurrent mark-sweep GCs, which focused on nicely-typed languages, the worst race was the one covered by the write barrier. But since this is JavaScript, we get to have a lot more fun.
JavaScript objects may have properties added to them at any time. The WebKit JavaScript object model has three features that makes this efficient:
Each object has a structure ID: The first 32 bits of each object is its structure ID. Using a table lookup, this gives a pointer to the object’s structure: a kind of meta-object that describes how its object is supposed to look. The object’s layout is governed by its structure. Some objects have immutable structures, so for those we know that so long as their structure IDs stay the same, they will be laid out the same.
The structure may tell us that the object has inline storage. This is a slab of space in the object itself, left aside for JavaScript properties.
The structure may tell us about the object’s butterfly. Each object has room for a pointer that can be used to point to an overflow storage for additional properties that we call a butterfly. The butterfly is a bidirectional object that may store named properties to the left of the pointer and indexed properties to the right.
It’s imperative that the garbage collector visits the butterfly using exactly the structure that corresponds to it. If the object has a mutable structure, it’s imperative that the collector visits the butterfly using the data from the structure that corresponds to that butterfly. The collector would crash if it tried to decode the butterfly using wrong information.
To accomplish this, we use a very simple obstruction-free version of Afek et al’s double collect snapshot. To handle the immutable structure case, we just ensure that the application uses this protocol to set both the structure and butterfly:
Nuke the structure ID — this sets a bit in the structure ID to indicate to the GC that the structure and butterfly are changing.
Set the butterfly.
Set the new (decontaminated) structure ID — decontaminating means clearing the nuke bit.
Meanwhile the collector does this to read both the structure and the butterfly:
Read the structure ID.
Read the butterfly.
Read the structure ID again, and compare to (1).
If the collector ever reads a nuked structure ID, or if the structure ID in (1) and (3) are different, then we know that we will have a butterfly-structure mismatch. But if none of these conditions hold, then we are guaranteed that the collector will have a consistent structure and butterfly. See here for the proof.
Harder still is the case where the structure is mutable. In this case, we ensure that the protocol for setting the fields in the structure is to set them after the structure is nuked but before the new one is installed. The collector reads those fields before/after as well. This allows the collector to see a consistent snapshot of the structure, butterfly, and a bit inside the structure without using any locking. All that matters is that the stores in the application and the loads in the collector are ordered. We get this for free on x86, and on ARM we use store-store fences in the application (dmb ishst) and load-load fences in the collector (dmb ish).
This algorithm is said to be obstruction-free because it will complete in O(1) time no matter what kind of race it encounters, but if it does encounter a race then it’ll tell you to try again. Obstruction-free algorithms need some kind of contention manager to ensure that they do eventually complete. The contention manager must provably maximize the likelihood that the obstruction-free algorithm will eventually run without any race. For example, this would be a sound contention manager: exponential back-off in which the actual back-off amount is a random number between 0 and X where X increases exponentially on each try. It turns out that Riptide’s retreating wavefront revisit scheduler is already a natural contention manager. When the collector bails on visiting an object because it detected a race, it schedules revisiting of that object just as if a barrier had executed. So, the GC will visit any object that encountered such a race again anyway. The GC will visit the object much later and the timing will be somewhat pseudo-random due to OS scheduling. If an object did keep getting revisited, eventually the space-time scheduler will increase the collector’s synthetic pause to the point where the revisit will happen with the world stopped. Since there are no safepoints possible in any of the structure/butterfly atomic protocols, stopping the world ensures that the algorithm will not be obstructed.
Embedded WTF Locks
The obstruction-free object snapshot is great, but it’s not scalable — from a WebKit developer sanity standpoint — to use it everywhere. Because we have been adding more concurrency to WebKit for a while, we made this easier by already having a custom locking infrastructure in WTF (Web Template Framework). One of the goals of WTF locks was to fit locks in two bits so that we may one day stuff a lock into the header of each JavaScript object. Many of the loony corner-case race conditions in the concurrent garbage collector happen on paths where acquiring a lock is fine, particularly if that lock has a great inline fast path like WTF locks. So, all JavaScript objects in WebKit now have a fast, adaptive, and fair WTF lock embedded in two bits of what is otherwise the indexingType byte in the object header. This internal lock is used to protect mutations to all sorts of miscellaneous data structures. The collector will hold the internal lock while visiting those objects.
Locking should always be used with care since it can be a slow-down. In Riptide, we only use locking to protect uncommon operations. Additionally, we use an optimized lock implementation to reduce the cost of synchronization even further.
Algorithm Summary
Riptide is an improvement to WebKit’s collector and retains most of the things that made the old algorithm great. The changes that transformed WebKit’s collector were landed over the past six months, starting with the painful work of removing WebKit’s previous use of copying. Riptide combines Guy Steele’s classic retreating wavefront write barrier with a mature sticky-mark-sweep collector and lots of concurrency tricks to get a useful combination of high GC throughput and low GC latency.
Related Work
The paper that introduced retreating wavefront did not claim to implement the idea — it was just a thought experiment. We are aware of two other implementations of retreating wavefront. The oldest is the BDW (Boehm-Demers-Weiser) collector‘s incremental mode. That collector uses a page-granularity revisit because it relies entirely on page faults to trigger the barrier. The collector makes pages that have black objects read-only and then any write to that page triggers a fault. The fault handler makes the page read-write and logs the entire page for revisiting. Riptide uses a software barrier that precisely triggers revisiting only for the object that got stored to. The BDW collector uses page faults for a good reason: so that it can be used as a plug-in component to any kind of language environment. The compiler doesn’t have to be aware of retreating wavefronts or generations since the BDW collector will be sure to catch all of the writes that it cares about. But in WebKit we are happy to have everything tightly integrated and so Riptide relies on the rest of WebKit to use its barrier. This was not hard since the new barrier is almost identical to our old one.
Another user of retreating wavefront is ChakraCore. It appears to have both a page-fault-based barrier like BDW and a software card-marking barrier that can flag 128-byte regions of memory as needing revisit. (For a good explanation of card-marking, albeit in a different VM, see here.) Riptide uses an object-granularity barrier instead. We tried card-marking, but found that it was slower than our barrier unless we were willing to place our entire heap in a single large virtual memory reservation. We didn’t want our memory structure to be that deterministic. All retreating wavefront collectors require a stop-the-world snapshot-at-the-end increment that confirms that there is no more marking left to do. Both BDW and ChakraCore perform all revisiting during the snapshot-at-the-end. If there is a lot of revisiting work, that increment could take a while. That risk is particularly high with card-marking or fault-based barriers, in which a write to a single object usually causes the revisiting of multiple objects. Riptide can revisit objects with the application resumed. Riptide can also resume the application in between executions of custom constraints. Riptide is tuned so that the snapshot-at-the-end is only confirming that there is no more work, rather than spending an unbounded amount of time creating and chasing down new work.
Instead of retreating wavefront, most incremental, concurrent, and real-time collectors use some kind of advancing wavefront barrier. In those kinds of barriers, the application marks the objects it interacts with under certain conditions. Baker’s barrier marks every pointer you load from the heap. Dijkstra’s barrier marks every pointer you store into the heap. Yuasa’s barrier marks every pointer you overwrite. All of these barriers advance the collector’s wavefront in the sense that they reduce the amount of work that the collector will have to do — the thinking goes that the collector would have marked the object anyway so the barrier is helping. Since these collectors usually allocate objects black during collection, marking objects will not postpone when the collector can finish. This means that advancing wavefront collectors will mark all objects that were live at the very beginning of the cycle and all objects allocated during the cycle. Keeping objects allocated during the GC cycle (which may be long) is called floating garbage. Retreating wavefront collectors largely avoid floating garbage since in those collectors an object can only be marked if it is found to be referenced from another marked object.
Advancing wavefront barriers are not a great match for generational collection. The generational barrier isn’t going to overlap with an advancing wavefront barrier the way that Riptide’s, ChakraCore’s, and BDW’s do. This means double the barrier costs. Also, in an advancing wavefront generational collector, eden collections have to be careful to ensure that their floating garbage doesn’t get promoted. This requires distinguishing between an object being marked for survival versus being marked for promotion. For example, the Domani, Kolodner, Petrank collector has a “yellow” object state and special color-toggling machinery to manage this state, all so that it does not promote floating garbage. The Frampton, Bacon, Cheng, and Grove version of the Metronome collector maintains three nurseries to gracefully move objects between generations, and in their collector the eden collections and full collections can proceed concurrently to each other. While those collectors have incredible features, they are not in widespread use, probably because of increased baseline costs due to extra bookkeeping and extra barriers. To put in perspective how annoying the concurrent-generational integration is, many systems like V8 and HotSpot avoid the problem by using synchronous eden collections. We want eden collections to be concurrent because although they are usually fast, we have no bound on how long they could take in the worst case. Not having floating garbage is another reason why it’s so easy for retreating wavefront collectors to do concurrent eden collection: there’s no need to invent states for black-but-new objects.
Using retreating wavefront means we don’t get the advancing wavefront’s GC termination guarantee. We make up for it by having more aggressive scheduling. It’s common for advancing wavefront collectors to avoid all global pauses because all of collection is concurrent. In the most aggressive advancing wavefront concurrent collectors, the closest thing to a “pause” is that at some point each thread must produce a stack scan. Even if all of Riptide’s algorithms were concurrent, we would still have to artificially stop the application simply to ensure termination. That’s a trade-off that we’re happy with, since we get to control how long these synthetic pauses are.
In many ways, Riptide is a classic mark-sweep collector. Using simple segregated storage is very common, and variants of this technique can be found in Jikes RVM, the Metronome real-time garbage collector, the BDW collector, the Bartok concurrent mark-sweep collector, and probably many others. Combining mark-sweep with bump-pointer is not new; Immix is another way to do it. Our bump’n’pop allocator looks most like Hoard‘s, and the technique was also used in Vam and reaps. Our conservative scan is almost like what the BDW collector does. Sticky mark bits are also used in BDW, Jikes RVM, and ChakraCore.
Evaluation
We enabled Riptide once we were satisfied that it did not have any major remaining regressions (in stability, performance, and memory usage) and that it demonstrated an improvement on some test of GC pauses. Enabling it now enables us to expose it to a lot of testing as we continue to tune and validate this collector. This section summarizes what we know about Riptide’s performance so far.
The synchronization features that enable concurrent collection were landed in many revisions over a six month period starting in July 2016. This section focuses on the performance boost that we get once we enable Riptide. Enabling Riptide means that draining will resume the application and allow the application and collector to run alongside each other. The application will still experience pauses: both synthetic pauses from the space-time scheduler and mandatory pauses for things like DOM constraint evaluation. The goal of this evaluation is to give a glimpse of what Riptide can do for observed pauses.
The test that did the best job of demonstrating our garbage collector’s jankyness was the Octane SplayLatency test. This test is also included in JetStream. WebKit was previously not the best at either version of this test so we wanted a GC that would give us a big improvement. The Octane version of this test reports the reciprocal of the root-mean-squared, which rewards uniform performance. JetStream reports the reciprocal of the average of the worst 0.5% of samples, which rewards fast worst-case performance. We tuned Riptide on the JetStream version of this test, but we show results from both versions.
The performance data was gathered on a 15″ MacBook Pro with a 2.8 GHz Intel Core i7 and 16GB RAM. This machine has four cores, and eight logical CPUs thanks to hyperthreading. We took care to quiet down the machine before running benchmarks, by closing almost all apps, disconnecting from the network, disabling Spotlight, and disabling ReportCrash. Our GC is great at taking advantage of hyperthreaded CPUs, so it runs eight draining threads on this machine.
The figure above shows that Riptide improves the JetStream/splay-latency score by a factor of five.
The figure above shows that Riptide improves the Octane/SplayLatency score by a factor of 2.5.
The chart above shows what is happening over 10,000 iterations of the Splay benchmark: without Riptide, an occasional iteration will pause for >10 ms due to garbage collection. Enabling Riptide brings these hiccups below 3 ms.
You can run this benchmark interactively if you want to see how your browser’s GC performs. That version will plot the time per iteration in milliseconds over 2,000 iterations.
We continue to tune Riptide as we validate it on a larger variety of workloads. Our goal is to continue to reduce pause times. That means making more of the collector concurrent and improving the space-time scheduler. Continued tuning is tracked by bug 165909.
Conclusion
This post describes the new Riptide garbage collector in WebKit. Riptide does most of its work off the main thread, allowing for a significant reduction in worst-case pause times. Enabling Riptide leads to a five-fold improvement in latency as reported by the JetStream/splay-latency test. Riptide is now enabled by default in WebKit trunk and you can try it out in Safari Technology Preview 21. Please try it out and file bugs!
À noter si vous êtes utilisateur d’une liseuse Kindle : Amazon va proposer une grosse promotion sur mon livre surveillance:// (format électronique) le dimanche 22 janvier 2017.
Le blog d’Éric Lemoine : surveillance:// de Tristan Nitot. « un bon livre, offrant des explications simples, concises et claires, et illustrés d’exemples concrets et parlants. » ;
Reflets.info : Surveillance : Tristan Nitot à l’aide des internautes. « Sans tomber dans une paranoïa totale et sans décrire une infrastructure de psychopathe de la protection des communications, Tristan Nitot donne aux béotiens les moyens nécessaires pour assurer un minimum de privacy. Ce qui est une gageure. » ;
Des employés d’Apple (en magasin) récupèrent des photos de clients depuis des iPhones et attribuent des notes quant au physique sur les photos : Apple Store photo ring scandal in Brisbane: Privacy commissioner weighs in. On s’en rend compte parce que c’est dans un magasin. Mais qu’en est-il de nos données dans des data-centers de Google, Facebook ou Apple ?
Cellebrite piraté !. La célèbre boite israélienne qui vend des produits et services servant à pirater des smartphones (dont l’iPhone de San Bernardino) a vu 900 Go de données fuiter…
Pia Pandelakis explique pourquoi elle veut Quitter Facebook (et elle a raison). Pour ma part, je n’utilise quasiment pas Facebook. Je n’ai pas installé son application sur mon mobile et, comme je l’ai annoncé, j’ai désinstallé l’application Twitter de mon mobile (et je m’en réjouis) !
“The problem is not fake news but the speed and ease of its dissemination, and it exists primarily because today’s digital capitalism makes it extremely profitable – look at Google and Facebook – to produce and circulate false but click-worthy narratives.” explique Evgeny Morozov dans le Guardian (via Olivier, qui finalement n’est pas mort) ;
Programming is all about managing complexity. Without good tools to manage it, the complexity of programs becomes mentally intractable for our limited brains and we’d lose control and understanding of our programs (imagine writing a big software system entirely in assembly language).
And so the history of programming has been a series of advancements in both removing barriers to composability, and building new programming technologies that better facilitate composition. To the extent that software has compositional structure (as opposed to monolithic structure), it can be understood and managed by our limited brains, and we can build more complex software via composition of smaller pieces. Also very important is that composable artifacts can be assembled by thousands of people in loose communication, often working in parallel, whereas monolithic artifacts require small teams in close communication, often working sequentially.
So we’ve proceed in stages:
Stage 0: Composability is the most important thing; without it, complexity swallows us
Stage 1: Composability requires atomic units of composition with means of combination, therefore functions
Stage 2: Composability is limited by side effects, therefore pure functions and functional programming
Stage 3: Composability without mechanized reasoning becomes difficult for humans to track, therefore static types
Stage 4: Composability is destroyed at program boundaries, therefore extend these boundaries outward, until all the computational resources of civilization are joined in a single planetary-scale computer… this is the idea of Unison and unison.cloud.
…
Stage N: We’ll get back to this at the end of this post
Pause here. While you can definitely still compose programs from impure functions, doing so is less flexible and also more complicated for the programmer. I’m not saying any form of composition is impossible with impure functions; I’m saying it’s more difficult (for instance, it requires non-local reasoning). Likewise for the other stages. I’m not saying you have no composability without static types; I’m saying that static types more easily facilitate composition given our limited brains. (Also see: Turing tarpit arguments)
If Haskell (or XYZ) is so great, why hasn’t it taken over?
If Haskell is so great, why hasn’t it taken over the world?
But pick any non-mainstream tech that you think is better, it doesn’t have to be Haskell. You can invent all kinds of responses:
“It really IS taking over the world, even Java has lambdas (or <feature-related-to-my-pet-tech>) now!!” (okay, but let’s be real, pure FP at large scale is still not that common)
“It really IS taking over the world, just very slowly…” (I mean, maybe, but you could have said the same thing 10 years ago; how do you know you aren’t fooling yourself?)
“Everyone else outside my little tribe is a moron!” (You really shouldn’t think this… but even if you did think it, you should ask yourself: if Haskell were really that much better, wouldn’t eventually these supposed “morons” be convinced by the mountain of incontrovertible evidence that accumulated in favor of your preferred technology’s vast superiority??)
“It’s too hard to learn” (If your pet technology were 1000x more productive than, say, Java, would this learning curve really be a substantive barrier? Though this question is more complicated than you think—see the tech adoption post—if the multiplier is big enough, will these complexities matter? Imagine if DVORAK or some other keyboard layout were 1000x faster than QWERTY for typing.)
“It’s all about MARKETING. JavaScript has better marketing!! And its projects have cooler logos on their landing pages!” (Okay, now you’re really reaching…)
The simplest explanation is probably that Haskell is not that much better than, say, Java, for many of the software systems people write today. Why might this be?
The reason I’ll give is that Haskell’s otherwise excellent composability is destroyed at I/O boundaries, just like every other language. That is, we are at stage 4 above, where the bottleneck to further composition is these program boundaries. Since most software systems (especially those that span multiple nodes), have a large surface area in contact with the outside world, the code devoted to merely getting information at these boundaries into some more computable form is often the bulk of the work; once the data is in computable form, the actual computation needing to be done is easy.
“Look, I used a monad! And defined my own type class for custom folding of data! Isn’t that amazing?“. “What does it do?” “It’s a CRUD app”.
If you’re writing a CRUD app, or some other computationally boring system that has a large, complex surface area in contact with the outside world, writing code to deal with that program boundary often dominates the codebase.
Where we see Haskell (or more generally, typed FP) excel is for programs that have minimal surface area in contact with the outside world, but with a large amount of interesting computation happening internally. A good example: compilers. Compilers don’t have much interaction with the outside world—just reading some files—but have lots of interesting computation happening internally, for things like typechecking, code generation, and so on. Haskell excels here; I would not be surprised if Haskell were 100x better than Java for writing compilers. Writing CRUD apps? Haskell isn’t as much of a win.
I think this hypothesis also offers an explanation for why Go is popular, even though the language is “boring” and could have been designed in the 1970s. Go has found a niche as basically “a better C” or “a better Java” for writing high-performance servers that do lots of I/O. Unlike C or Java, it has a much more high-level I/O and concurrency story, but the language itself is otherwise very familiar to people with a background in these and other mainstream languages. Thus it serves a niche that wasn’t previously well-covered.
As soon as you need to be defining lots of complex or interesting computations, you start needing languages with good support for composability to manage that complexity. Here Go fails, for all the reasons that people have criticized it. But there’s still a good chunk of services where Go can do quite well!
Haskell programmers might object that, well, Haskell has its own very nice I/O and concurrency story, in many ways more sophisticated than Go (things like software-transactional memory, which make writing highly concurrent data structures and algorithms much simpler). But Haskell is “weird”. A C, Java, Python, or Ruby programmer can pick up Go easily. They can’t pick up Haskell so easily, as even in beginner Haskell, you are immediately confronted with lots of unfamiliar concepts. And since Haskell isn’t enough of a win for these “boring” services, Go can still make sense.
What’s next?
The Unison programming language, and the unison.cloud platform I’d like to build around it, is my effort to move programming beyond Stage 4. By removing any friction and non-uniformity when programming multi-node software systems, such systems can once be assembled in a compositional fashion. The better composability of typed, pure FP once again becomes a significant lever, because process boundaries no longer destroy composition.
I wonder what comes after that? When all the obvious barriers to composability have been removed, the ‘composability bottleneck’ must move somewhere else, somewhere that might not be obvious. Like where? Time for some vague speculation…
One other problem we have today is that composability is destroyed at “application” boundaries, at the interface between humans and our programs. We write a bunch of “backend code” in a compositional fashion, then build a bespoke, single purpose UI for interacting with some ad hoc subset of this, which is a dead end for further composition. (See this Conal Elliott talk on this) This is a problem, and it can be solved.
But even if we move beyond that, there will be other composability bottlenecks, and when those are removed, there will be others, and on and on, and in the end… well, I don’t know.
What do you think?
Appendix: Turing tarpit arguments
There’s a kind of argument that comes up a lot in discussion of programming languages. I call it a “Turing Tarpit” argument: programming tech A isn’t really better than tech B, it’s just that A is a bit more convenient than B for a few hand-picked little examples (implicitly: “Big deal, who cares?”).
Beware of the Turing tar-pit in which everything is possible but nothing of interest is easy..
The trouble is that very often, the sorts of examples that are easy to discuss aren’t of sufficient scale to reveal any major differences between A and B. It’s only when building much larger systems that the difference become more than “a little convenience”. That is, Turing tarpit arguments skip doing any analysis of how or whether the “little more convenience” might becomes bigger as N gets larger, and tacitly assumes that any language that’s Turing complete is just as good as any other. It’s a bit like saying: “Oh, geez, this heapsort algorithm you’ve got seems rather baroque and complicated. My insertion sort algorithm runs just as fast on this 10 element list.”
Imagine traveling back in time to the days before C, and trying to convince an assembly language programmer that C was a massive step forward for programming. In principle, you could build arbitrary programs by gluing together hand-written fragments of x86 assembly language. In practice, fragments of assembly language aren’t very composable given the limitations of our brains. But you might have a hard time convincing the assembly language programmer of this, because toy examples of the sort that are easy to discuss would not reveal any major differences.
What WAS likely convincing to assembly language programmers was the idea of not having to write the same program 5 times, for each different hardware architecture. This was a clear productivity boost that was immediately understandable to anyone who wrote assembly language and needed to target different architectures. And this huge advantage was enough to get “high-level” languages like C in the door. With time and experience using C, the more subtle, abstract benefits of increased composability of C over assembly language would become more apparent.
Editor’s note: This is the second chapter of a book I’m working on called Demystifying Artificial Intelligence. The goal of the book is to demystify what modern AI is and does for a general audience. So something to smooth the transition between AI fiction and highly mathematical descriptions of deep learning. I’m developing the book over time - so if you buy the book on Leanpub know that there are only two chapters in there so far, but I’ll be adding more over the next few weeks and you get free updates. The cover of the book was inspired by this amazing tweet by Twitter user @notajf. Feedback is welcome and encouraged!
“I am so clever that sometimes I don’t understand a single word of
what I am saying.” Oscar Wilde
As we have described it artificial intelligence applications consist of
three things:
A large collection of data examples
An algorithm for learning a model from that training set.
An interface with the world.
In the following chapters we will go into each of these components in
much more detail, but lets start with a a couple of very simple examples
to make sure that the components of an AI are clear. We will start with
a completely artificial example and then move to more complicated
examples.
Building an album
Lets start with a very simple hypothetical example that can be
understood even if you don’t have a technical background. We can also
use this example to define some of the terms we will be discussing later
in the book.
In our simple example the goal is to make an album of photos for a
friend. For example, suppose I want to take the photos in my photobook
and find all the ones that include pictures of myself and my son Dex for
his grandmother.
If you are anything like the author of this book, then you probably have
a very large number of pictures of your family on your phone. So the
first step in making the photo alubm would be to stort through all of my
pictures and pick out the ones that should be part of the album.
This is a typical example of the type of thing we might want to train a
computer to do in an artificial intelligence application. Each of the
components of an AI application is there:
The data: all of the pictures on the author’s phone (a big
training set!)
The algorithm: finding pictures of me and my son Dex
The interface: the album to give to Dex’s grandmother.
One way to solve this problem is for me to sort through the pictures one
by one and decide whether they should be in the album or not, then
assemble them together, and then put them into the album. If I did it
like this then I myself would be the AI! That wouldn’t be very
artificial though…imagine we instead wanted to teach a computer to
make this album..
But what does it mean to “teach” a computer to do something?
The terms “machine learning” and “artificial intelligence” invoke the
idea of teaching computers in the same way that we teach children. This
was a deliberate choice to make the analogy - both because in some ways
it is appropriate and because it is useful for explaining complicated
concepts to people with limited backgrounds. To teach a child to find
pictures of the author and his son, you would show her lots of examples
of that type of picture and maybe some examples of the author with other
kids who were not his son. You’d repeat to the child that the pictures
of the author and his son were the kinds you wanted and the others
weren’t. Eventually she would retain that information and if you gave
her a new picture she could tell you whether it was the right kind or
not.
To teach a machine to perform the same kind of recognition you go
through a similar process. You “show” the machine many pictures labeled
as either the ones you want or not. You repeat this process until the
machine “retains” the information and can correctly label a new photo.
Getting the machine to “retain” this information is a matter of getting
the machine to create a set of step by step instructions it can apply to
go from the image to the label that you want.
The data
The images are what people in the fields of artificial intelligence and
machine learning call “raw data” (Leek, n.d.). The categories of
pictures (a picture of the author and his son or a picture of something
else) are called the “labels” or “outcomes”. If the computer gets to
see the labels when it is learning then it is called “supervised
learning” (Wikipedia contributors 2016) and when the computer doesn’t
get to see the labels it is called “unsupervised learning” (Wikipedia
contributors 2017a).
Going back to our analogy with the child, supervised learning would be
teaching the child to recognize pictures of the author and his son
together. Unsupervised learning would be giving the child a pile of
pictures and asking them to sort them into groups. They might sort them
by color or subject or location - not necessarily into categories that
you care about. But probably one of the categories they would make would
be pictures of people - so she would have found some potentially useful
information even if it wasn’t exactly what you wanted. One whole field
of artificial intelligence is figuring out how to use the information
learned in this “unsupervised” setting and using it for supervised tasks
this is sometimes called “transfer learning” (Raina et al. 2007) by
people in the field since you are transferring information from one task
to another.
Returning to the task of “teaching” a computer to retain information
about what kind of pictures you want we run into a problem - computers
don’t know what pictures are! They also don’t know what audio clips,
text files, videos, or any other kind of information is. At least not
directly. They don’t have eyes, ears, and other senses along with a
brain designed to decode the information from these senses.
So what can a computer understand? A good rule of thumb is that a
computer works best with numbers. If you want a computer to sort
pictures into an album for you, the first thing you need to do is to
find a way to turn all of the information you want to “show” the
computer into numbers. In the case of sorting pictures into albums - a
supervised learning problem - we need to turn the labels and the images
into numbers the computer can use.
One way to do that would be for you to do it for the computer. You could
take every picture on your phone and label it with a 1 if it was a
picture of the author and his son and a 0 if not. Then you would have a
set of 1’s and 0’s corresponding to all of the pictures. This takes some
thing the computer can’t understand (the picture) and turns it into
something the computer can understand (the label).
This process would turn the labels into something a computer could
understand, it still isn’t something we could teach a computer to do.
The computer can’t actually “look” at the image and doesn’t know who the
author or his son are. So we need to figure out a way to turn the images
into numbers for the computer to use to generate those labels directly.
This is a little more complicated but you could still do it for the
computer. Let’s suppose that the author and his son always wear matching
blue shirts when they spend time together. Then you could go through and
look at each image and decide what fraction of the image is blue. So
each picture would get a number ranging from zero to one like 0.30 if
the picture was 30% blue and 0.53 if it was 53% blue.
The fraction of the picture that is blue is called a “feature” and the
process of creating that feature is called “feature engineering”
(Wikipedia contributors 2017b). Until very recently feature engineering
of text, audio, or video files was best performed by an expert human. In
later chapters we will discuss how one of the most exciting parts about
AI application is that it is now possible to have computers perform
feature engineering for you.
The algorithm
Now that we have converted the images to numbers and the labels to
numbers, we can talk about how to “teach” a computer to label the
pictures. A good rule of thumb when thinking about algorithms is that a
computer can’t “do” anything without being told very explicitly what to
do. It needs a step by step set of instructions. The instructions should
start with a calculation on the numbers for the image and should end
with a prediction of what label to apply to that image. The image
(converted to numbers) is the “input” and the label (also a number) is
the “output”. You may have heard the phrase:
“Garbage in, garbage out”
What this phrase means is if the inputs (the images) are bad - say they
are all very dark or hard to see. Then the output of the algorithm will
also be bad - the predictions won’t be very good.
A machine learning “algorithm” can be thought of as a set of
instructions with some of the parts left blank - sort of like mad-libs.
One example of a really simple algorithm for sorting pictures into the
album would be:
Calculate the fraction of blue in the image.
If the fraction of blue is above X label it 1
If the fraction of blue is less than X label it 0
Put all of the images labeled 1 in the album
The machine “learns” by using the examples to fill in the blanks in
the instructions. In the case of our really simple algorithm we need to
figure out what fraction of blue to use (X) for labeling the picture.
To figure out a guess for X we need to decide what we want the
algorithm to do. If we set X to be too low then all of the images will
be labeled with a 1 and put into the album. If we set X to be too high
then all of the images will be labeled 0 and none will appear in the
album. In between there is some grey area - do we care if we
accidentally get some pictures of the ocean or the sky with our
algorithm?
But the number of images in the album isn’t even the thing we really
care about. What we might care about is making sure that the album is
mostly pictures of the author and his son. In the field of AI they
usually turn this statement around - we want to make sure the album has
a very small fraction of pictures that are not of the author and his
son. This fraction - the fraction that are incorrectly placed in the
album is called the “loss”. You can think about it like a game where
the computer loses a point every time it puts the wrong kind of picture
into the album.
Using our loss (how many pictures we incorrectly placed in the album) we
can now use the data we have created (the numbers for the labels and the
images) to fill in the blanks in our mad-lib algorithm (picking the
cutoff on the amount of blue). We have a large number of pictures where
we know what fraction of each picture is blue and whether it is a
picture of the author and his son or not. We can try each possible X
and calculate the fraction of pictures in the album that are incorrectly
placed into the album (the loss) and find the X that produces the
smallest fraction.
Suppose that the value of X that gives the smallest faction of wrong
pictures in the album is 30. Then our “learned” model would be:
Calculate the fraction of blue in the image
If the fraction of blue is above 0.1 label it 1
If the fraction of blue is less than 0.1 label it 0
Put all of the images labeled 1 in the album
The interface
The last part of an AI application is the interface. In this case, the
interface would be the way that we share the pictures with Dex’s
grandmother. For example we could imagine uploading the pictures to
Shutterfly and having the album delivered
to Dex’s grandmother.
Putting this all together we could imagine an application using our
trained AI. The author uploads his unlabeled photos. The photos are then
passed to the computer program which calculates the fraction of the
image that is blue, then applies a label according to the algorithm we
learned, then takes all the images predicted to be of the author and his
son and sends them off to be a Shutterfly album mailed to the authors’
mother.
If the algorithm was good, then from the perspective of the author the
website would look “intelligent”. I just uploaded pictures and it
created an album for me with the pictures that I wanted. But the steps
in the process were very simple and understandable behind the scenes.
Raina, Rajat, Alexis Battle, Honglak Lee, Benjamin Packer, and Andrew Y
Ng. 2007. “Self-Taught Learning: Transfer Learning from Unlabeled Data.”
In Proceedings of the 24th International Conference on Machine
Learning, 759–66. ICML ’07. New York, NY, USA: ACM.
“A great business has to have a conscience. You have to know who you are and who you are not.”
— Howard Schultz, Starbucks
History has shown that conventional wisdom in corporate communications has been to keep company statements high-level, formal, and uncontroversial. In the past decade, however, we have seen a secular shift from leaders of large companies like Apple, Costco and Starbucks, who are now more inclined to take a risk and speak up on issues that can be polarizing to different audiences.
In the past six months, I’ve had the opportunity to take a public stand for Wealthfrontthree times, and we’ve been fortunate enough to see those efforts rewarded in our growth. But speaking out as a CEO is never easy and it is never comfortable, so many are now asking the question:
When is it OK for a CEO to take a public stand?
Three Things to Think About
Leaders reflect strongly on their organizations, and CEOs cannot escape explicit and implicit comparisons with a company’s brand. So when a CEO makes the decision to take a stand, it has to be evaluated in the context of what’s best for the company. There is no real way for a CEO to divorce their position from that of their business, and a public position can trigger a reaction from all stakeholders.
Because of this, there are three core questions CEOs needs to ask themselves before taking a public stand:
Do you have a mission-driven culture?
Who is your customer base?
Who are your suppliers, partners and investors?
Do You Have a Mission-Driven Culture?
“I think the currency of leadership is transparency. You’ve got to be truthful. I don’t think you should be vulnerable every day, but there are moments where you’ve got to share your soul and conscience with people and show them who you are, and not be afraid of it.” — Howard Schultz, Starbucks
One of the most difficult, and yet valuable aspects of building a successful company is building its culture. If your company is mission-driven and values transparency, you’ll find that taking a public stand is often rewarded with increased passion, engagement and pride from your employees. It can also help amplify the appeal of your organization to talent seeking purpose in their professional endeavors.
For example, the leadership at Tesla has made a conscious effort to ensure their mission to accelerate the transition to sustainable transportation drives (pun intended) the company culture, so when Elon Musk takes an aggressive stand, people, whether they agree or not, listen carefully. It is much harder for the leader of General Motors to take an aggressive public position.
There is no way to take a strong position on a controversial issue and not produce waves, both inside and outside the company. But mission-driven cultures are not only more tolerant of that debate, but also often deepen and strengthens because of it.
Who are your customers?
“Great companies that build an enduring brand have an emotional relationship with customers that has no barrier. And that emotional relationship is on the most important characteristic, which is trust.” — Howard Schultz, Starbucks
There is a saying in design that if you try to design for everyone, you end up designing for no one. Great consumer brands are like great designs – they resonate emotionally with a specific audience.
It is naive to think that taking a stand on ethical issues will result in universal support. That’s why it is incredibly important to not only know who your customers are, but also have a deep understanding of how they will react to a public position and specifically the one you are taking. While the specific position taken matters, too often leaders ignore the more subtle, but powerful issue, or whether or not their brand supports the idea of taking a strong, public position on the issue.
There is a reason why it’s easier for Costco to take a public position on some issues than Wal-Mart. It’s customers are primarily urban, mass affluent and well-educated. Their revenue per employee is much higher, and that allows them to pay their average worker more. As a result, it’s easier for Craig Jelinek to take strong public positions on issues like employee compensation and benefits that align with their brand and their customer base.
So if your position aligns with your brand and your customers, you’ll find a natural platform to amplify your message. But if it conflicts with what your customers expect from your company, it will not only detract from your message, it can also harm your company.
Who are your suppliers, partners and investors?
Companies have a wide variety of stakeholders, but one of the largest limiting factors in CEOs taking public stands on controversial issues are the often invisible dependencies they have on suppliers, partners and investors.
In the 1990s, Microsoft was infamous for exerting a strong level of silent influence over software and hardware partners who were dependent on their platform. Investors also can wield influence, sometimes directly through the Board of Directors, and sometimes less obviously through financing and other relationships. This is why it is incredibly important to be picky about your partners and chose those who align with your audience.
A CEO who takes a public stand at odds with critical suppliers, partners and investors can quickly find themselves and their companies in a difficult position. This is probably the most common reason that, historically, most CEOs have been forced to avoid controversial issues.
Leadership Beyond Metrics
By definition, opinionated positions will be polarizing. As a result, I’ve worked tirelessly at Wealthfront to build a company with purpose and mission, and build a brand supportive of taking on industry change directly. As a result, we’ve been incredibly vocal on issues that reflect the priorities and beliefs our our employees, our customers and our investors.
This past June, it was gratifying to see that our efforts around the fiduciary standard had an impact. In his four-page opening statement to Congress, Labor Secretary Thomas Perez cited Wealthfront as an example of a company serving the small investor and keeping their best interest front and center.
In July, it was heartening to see Acorns, another company in our space, respond positively to my call to fintech CEOs to drop monthly fees on small accounts. Their founder and CEO, Jeff Cruttenden decided to remove their monthly fee for students and investors under 24. Acorns is a mission-driven company, and it’s no surprise that they have quickly built the automated investment service with the most clients.
In general, taking a stand on an ethical issues is rarely good marketing, or positive for the metrics. Fortunately, July was a record month for Wealthfront. Over 3x as many people signed up for the service in July as did in January 2015, just six months ago. As it turns out, there is a huge population of young investors out there who are tired of business as usual, tired of the traditional financial services industry, and tired of rationalizations and empty promises.
Change does not come without risk, both personal and professional. Companies have to decide what they stand for, and leaders have to decide when it’s appropriate to take a stand.
Note: This post originally appeared on LinkedIn on August 13, 2015. It has been replicated here for archival purposes.
This article is from our archives. It was published in September 2012. “If you walk down a street with Kevin, it’s like parting the Red Sea,” says Stuart Coxe,…
For hardware companies, China is a no-brainer place to be, considering its market size and its role as a manufacturing hub. Hardware companies like Xiaomi and Huawei and success stories about crowdfunding campaigns have proven that the Middle Kingdom has gone from “Made in China” to “Designed in China.” This year’s CES 2017 was also splashed by Chinese founders’ innovative technologies and hardware, such as iGULU, NEOBEAR, and uSens.
When startups successfully launch their product, multinational companies sometimes approach for cooperation. Jason advises the startups to be protective of their product, saying that the benefit usually leans on the corporate’s side rather than the startup.
“Be cautious when taking your ideas or innovation to a larger company. Make sure your ideas is already well protected as corporations may take a financial interest approach to your ideas, and they may or may not license from you,” he said. “If you have strong IP protections you will have more confidence when working with corporate partners. Always read the fine print in any contracts, and pursue what makes sense to your business model.”
2. Focus both U.S. and China market
Hardware founders weigh between two markets. Strategically, some Chinese companies position them as an international company to get global attention.
“When you are competing on a global scale, you need to have a presence in your key markets. It is hard to be successful otherwise. For us that is the US and China. It is critical for us to work closely with our suppliers. Not everyone can be like Apple, with an established partnership with Foxconn. Manufacturing and supply chain is very critical in the hardware business,” Jason remarked.
3. Register your patent globally.
The patent issue is a controversial one. To avoid something similar to Apple losing it’s iPhone case in China, startups should register their patents globally as soon as possible.
“Always protect your ideas and apply for IP protection in your key markets using PCT (Patent Cooperation Treaty). It is great to launch a product on crowdfunding and get funded, but if you fail to register first, you will encounter other companies copying your product quickly,” Jason said.
4. Do not follow the trend.
Finally, Jason advises startups to follow their guts, rather than following the market trend.
“Rather than following major trends or hot sectors, focus your efforts on problems that you personally care about. It takes sometimes 5 to 10 years to scale your business, and you need to have passion in order to survive the journey. So ask yourself what you care about,” Jason added.
Interesting look at the process of winding up a strata corporation and selling its land. The 80% vote is a good beginning, but the courts have the last word. Final approval is contingent upon the court’s concern for minority owners.
There has been no shortage of Galaxy S8 leaks hailing from the farthest edges of the internet.
The latest of these leaks, originally published on SamMobile, details a camera that may or might be able to perform visual searches. The report cites unnamed sources and also outlines the potential for the much anticipated device to provide text recognition as well.
The leak claims that the camera on the S8 will include its own Bixby button, a rumour that’s been circulating as of late. Rumours indicate that Samsung will outfit its next flagship with a digital assistant named Bixby.
The digital assistant may make its debut on the upcoming Galaxy flagship. Furthermore, the camera, combined with Bixby, will use optical character recognition to identify any text contained in the image.
Bixby came to light a few weeks back when a subsection of the Samsung Pay website was said to include references to terms like “Shopping,” “Mini” and “Bixby.” A leak earlier this week showing off what is purported to be a glass S8 screen protector, indicates that the phone will have an extremely thin bezel.
Samsung has been said to be planning an April launch, though this date hasn’t been confirmed.
Apple is suing silicon manufacturer Qualcomm for roughly $1 billion USD, stating that the company has been”charging royalties for technologies they have nothing to do with.”
The lawsuit comes after the U.S. Federal Trade Commission (FTC) sued Qualcomm earlier this week over a dispute related to unfair patent licensing practises. Qualcomm says it plans to “vigorously contest” the FTC’s suit, which the company believes is based on inaccurate information.
“We are extremely disappointed in the way Qualcomm is conducting its business”
The Cupertino, California-based tech giant states that Qualcomm has taken “radical steps” that include “withholding nearly $1 billion in payments from Apple as retaliation for responding truthfully to law enforcement agencies investigating them.”
The tech giant also stated, “Despite being just one of over a dozen companies who contributed to basic cellular standards, Qualcomm insists on charging Apple at least five times more in payments than all the other cellular patent licensors we have agreements with combined.”
“We are extremely disappointed in the way Qualcomm is conducting its business with us and unfortunately after years of disagreement over what constitutes a fair and reasonable royalty we have no choice left but to turn to the courts,” continued Apple in a statement.
In the lawsuit filing, Apple accuses Qualcomm of overcharging for its silicon after refusing to pay approximately $1 billion in agreed upon rebates related to processor purchases. Apple says that Qualcomm withheld the rebates because of the company’s discussions with South Korea’s antitrust regulator, which has fined the silicon manufacturer in the past for patent licensing related issues.
Snapchat has signed a new partnership agreement with Oracle Data Cloud that will allow the company to deliver targeted ads using third-party data.
This agreement will help marketers use data from offline purchases to target users with more relevant ads on Snapchat. This is the first time Snap has agreed to ad targeting using third-party data, and the move suggests that the company is taking steps towards becoming more like its competitors.
Facebook, Google and Twitter also offer the same data targeting options through their own Datalogix partnerships, which was acquired by Oracle in 2014.
Furthermore, this partnership will allow marketers that advertise on Snapchat to measure whether ad campaigns on the app translate into real-time sales, reports The Wall Street Journal.
The past few years have seen Snap aggressively attempt to build out its digital marketing strategy, with its most recent update coming this past September with the rollout of Snap Audience Match.
This program allowed marketers and advertisers to use their own email lists for data marketing purposes. Furthermore, Snap has reportedly partnered with several ad management companies, and recently began automating the ad-buying process for marketers through an API.
Snap has previously been called slow in adopting advertising targeting options, compared to counterparts like Facebook. It’s also important to note that the company recently filed confidentially for IPO this past year.
After months of waiting, Xbox One owners can finally play one of the top party multiplayer games ever released.
Vancouver-developed TowerFall Ascension is finally headed to the Xbox One on January 25th. The game is an addictive mix of Super Smash Bros. style deathmatch and classic platforming action. Up to four players share a single screen and battle it out against one another in multiplayer action by shooting volleys of arrows at one another.
Visually the game also looks great, as long as you’re a fan of pixel art style graphics. TowerFall is also highly customizable, allowing players to tweak matches to play out exactly how they want.
The game, developed by Matt Thorson along with a team of occasional freelance developers, originally launched Kickstarter darling and then disaster, the Ouya. Shortly after release, the game was repackaged as TowerFall Accession and released on the PC and PS3.
Thorson originally revealed that TowerFall was coming to the Xbox One back in 2015.
In an industry that no longer places an emphasis on local multiplayer or co-op, TowerFall Accession is a breath of fresh air. In fact, I’d probably even go far as to say it’s one of the best modern multiplayer title next to Rocket League.
The Note 7’s self-immolation this past summer was caused by two separate battery issues, according to The Wall Street Journal, which was able to obtain the results of Samsung’s investigation into the incident before the company officially unveils them on Sunday.
The publication reports batteries from Samsung SDI, which Samsung initially blamed for the Note 7’s higher than average chance of catching on fire, were “irregularly” sized. In other words, the first batch of batteries from Samsung SDI did not properly fit into the Note 7.
This echoes the findings of an earlier independent teardown from manufacturing startup Instrumental, which concluded Samsung did not engineer the Note 7’s battery casing to provide a sufficient enough celling for the expansion of its power cell.
Meanwhile, batteries supplied by out-of-house manufacturer Amperex Technology Ltd, which Samsung identified as safe at the time of Note 7’s first recall, suffered from an “unidentified” manufacturing defect that only came up after ATL was asked to ramp up production quickly to meet demand for replacement devices.
We’ll have to wait until the company officially releases the results of its investigation, but, as Android Authority notes, the fact that simultaneous battery problems existed across two different suppliers calls Samsung’s quality and safety control into question.
Samsung reportedly told U.S. regulators it has implemented a new eight-step process that includes more testing, inspections and manufacturing quality assurances, according to The Wall Street Journal.
Canadian consumers that end up purchasing the Nintendo Switch at launch won’t be able to watch movies and TV shows on the next generation handheld console, with apps like Netflix, CraveTV and other on-demand streaming services.
In a Q&A session with Kotaku, a Nintendo spokesperson had this to say on the subject:
“All of our efforts have gone toward making the Nintendo Switch system an amazing dedicated video game platform, so it will not support any video-streaming services at launch. However, support for video-streaming services is being considered for a future update.”
This will likely come as sad news to those who had planned to use the Switch as an offline Netflix player while on the Toronto subway system or Vancouver Skytrain.
The other piece of bad news is that the Switch won’t be backward compatible with games designed for the Wii, Wii U and 3DS. Moreover, Nintendo is being tightlipped Virtual Console functionality. The company won’t say whether games bought through the service, which allows gamers to play older Nintendo games through emulation, will carry over from the Wii U and 3DS.
Check out the entire Q&A on Kotaku, just don’t expect a lot of answers; Nintendo says “We have nothing to announce at this time,” in response to many of the website’s questions.
