Adam Victor Brandizzi

Enlarge / Labeled 3D model. The little arrows are galaxies, and the lines coming from them depicts their velocities (with the influence of the Universe's expansion on their velocities removed). Our galaxy, the Milky Way, is located in the Local Group, near the center of the image. The motions of all the galaxies seen here are dominated by the Shapley Attractor and the Dipole Repeller. (credit: Hebrew University of Jerusalem)

You may not notice it, but our Milky Way galaxy is cruising along at 630 kilometers (~391 miles) per second. That speed is often attributed to the influence of a single gravitational source. But in a new study, a group of researchers has found that the motions of the Local Group—the cluster of galaxies that includes the Milky Way—are being driven by two primary sources: the previously known and incredibly massive Shapley Supercluster and a newly discovered repeller, which the researchers dub the Dipole Repeller.

Shapley’s contribution was already known, but the Dipole Repeller’s hadn't been recognized prior to this study.

The researchers plotted the motions of many galaxies in the nearby Universe in a 3D model, using data from the Cosmicflows-2 database. Since the Universe is expanding, most galaxies are moving away from ours, creating a red-shift in the light they emit. But since the researchers were more interested in the other influences on a galaxy’s motion, they simply subtracted the expansion’s contribution. The resulting plot shows what the motions of galaxies would look like if space wasn’t expanding.

Read 7 remaining paragraphs | Comments

booksofadam:

Happy super bowl!

In 2009, I helped sketch out a marketing campaign for an internet personality and blogger named Tucker Max. With a very limited advertising budget available for the independent movie he had written and produced, we had few options for getting the word out.

Maybe it was crazy but my thinking was that one of the best ways to get young men to go see a movie was to tell them they should not be allowed to see it. What ensued was several months of chaos and controversy that ultimately drove Tucker’s book to No. 1 on the New York Times Bestseller list, sold out a multi-college bus tour and ultimately sold millions of dollars worth of tickets, dvds and books.

It was a masterful bit of trolling that admittedly felt a lot more meaningful and exciting when I was younger than it does to me today: We encouraged protests at colleges by sending outraged emails to various activist groups and clubs on campuses where the movie was being screened. We sent fake tips to Gawker, which dutifully ate them up. We created a boycott group on Facebook that acquired thousands of members. We made deliberately offensive ads and ran them on websites where they would be written about by controversy-loving reporters. After I began vandalizing some of our own billboards in Los Angeles, the trend spread across the country, with parties of feminists roving the streets of New York to deface them (with the Village Voice in tow).

But my favorite was the campaign in Chicago—the only major city where we could afford transit advertising. After placing a series of offensive ads on buses and the metro, from my office I alternated between calling in angry complaints to the Chicago CTA and sending angry emails to city officials with reporters cc’d, until ‘under pressure,’ they announced that they would be banning our advertisements and returning our money. Then we put out a press release denouncing this cowardly decision.

I’ve never seen so much publicity. It was madness.

If any of this sounds familiar, it should. Because it’s basically the exact playbook that right wing blogger Milo Yiannopoulos is running on his own cross-country trolling tour. By almost any metric but political correctness, it’s been masterfully successful—his book has since been to #1 on Amazon twice, and the protests at UC Berkeley last week generated national headlines and were addressed directly by the President.

The déjà vu is not accidental. Numerous leaders of the alt-right movement read the book I published in 2012, which outlined exactly how this media strategy works. Several have told meTrust Me, I’m Lying is their bible.

It’s a sad irony for me, since I wrote the book as an explicit warning about how broken our media system was and why it needed to be fixed. As I would say in interviews, the strategies that I used were designed to market books and clothes for companies like American Apparel, but I was exposing how they worked because I worried how others might soon use them to sell something more nefarious.

I should be clear about a few things: I’m not advising Milo or the alt-right in any capacity. Though I have spoken (as well as interviewed several for this column) to several of them, I disagree with their message and their aims. I don’t think we should build a wall along the border, I have no problem with an all-female cast of Ghostbusters, I don’t think we should ban Muslims from this country and I think the first few weeks of Trump’s presidency are only a sign of the malevolence and incompetence to come.

This puts me in a somewhat unique position in which I am deeply disturbed by our current political situation but I am not so blinded by my outrage that I don’t see exactly what these media manipulators are doing. My conversations with these figures—including occasional friendly chats with Milo—brings me to this column: You guys are playing completely into their hands.

Most brands and personalities try to appeal to a wide swath of the population. Niche players and polarizing personalities are only ever going to be interesting to a small subgroup. While this might seem like a disadvantage, it’s actually a huge opportunity: Because it allows them to leverage the dismissals, anger, mockery, and contempt of the population at large as proof of their credibility. Someone like Milo or Mike Cernovich doesn’t care that you hate them—they like it. It’s proof to their followers that they are doing something subversive and meaningful. It gives their followers something to talk about. It imbues the whole movement with a sense of urgency and action—it creates purpose and meaning.

You’re worried about “normalizing” their behavior when in fact, that’s the one thing they don’t want to happen. The key tactic of alternative or provocative figures is to leverage the size and platform of their “not-audience” (i.e. their haters in the mainstream) to attract attention and build an actual audience. Let’s say 9 out of 10 people who hear something Milo says will find it repulsive and juvenile. Because of that response rate, it’s going to be hard for someone like Milo to market himself through traditional channels. His potential audience is too spread out, and doesn’t have that much in common. He can’t advertise, he can’t find them one by one. It’s just not going to scale.

But let’s say he can acquire massive amounts of negative publicity by pissing off people in the media? Well now all of a sudden someone is absorbing the cost of this inefficient form of marketing for him. If a CNN story reaches 100,000 people, that’s 90,000 people all patting themselves on the back for how smart and decent they are. They’re just missing the fact that the 10,000 new people that just heard about Milo for the first time. The same goes for when you angrily share on Facebook some godawful thing one of these people has said. The vast majority of your friends rush to agree, but your younger cousin has a dark switch in his brain go on for the first time.

This is what creates the incentives for trolls to be more and more provocative and to care less and less about what normal, middle of the group people think. With Tucker, we knew that feminists were never, ever going to like his stuff. So we wanted to leverage that anger and outrage as that incredible force that it can be. When we tried to pay to name a Planned Parenthood clinic after him, the point wasn’t to greenwash his name through charitable donations. That never would have worked. The point was that “HOW DARE YOU?!” coverage from sites like Jezebel would naturally reach a number of people who thought the whole thing was funny and absurd. It would reach the people who hate-read Jezebel. It’d also be fuel for Jezebel’s critics.

This approach requires a certain shamelessness but it is effective because it puts the dominant group into the horns of a dilemma: Ignore them and let them do something offensive or object and give them the attention they need to survive and thrive? It’s why for Milo, there is no such thing as bad publicity right now. He’s maneuvered his brand with ruthless, bulletproof perfection.

Milo Yiannopoulos in 2016. Drew Angerer/Getty Images

It’s here that I think people are really giving the alt-right exactly what they want. One of the best ways to sow and exploit division is to look for moral hypocrisy. With the Planned Parenthood stunt, we were able to say: “Look, you claim to care about reproductive rights, yet you are turning down $500,000.” (Your smarter fans appreciate the brilliant trolling and your dumber fans really buy the argument). When protesters try to revoke someone’s right to speak or when someone like Richard Spencer is physically assaulted on camera, you’re not intimidating anyone—you’re emboldening them. You’re giving them a wonderful recruiting tool. They’re laughing all the way to the bank.

That’s what’s so misguided about what happened at UC Berkeley. From what I understand, most of the violence was perpetrated by infiltrators who were looking to sow chaos and destruction. Yet many of the peaceful protesters and organizers have admitted that they too were attempting to shut down Milo’s talk. The last thing you ever want to do is give an opponent the moral high ground—and attempts to suppress, intimidate and revoke constitutional rights do exactly that.

There is absolutely nothing that Milo has said (and more importantly, done) that ought to revoke his First Amendment right to give a speech on a college campus. It’s profoundly hypocritical for the same activists who demanded safe spaces against microaggressions to march en masse and aggressively shut down a nerdy, gay conservative immigrant with a funny name (a minority if there ever was one) until he flees under armed guard. As much as you might dislike what he’s saying—and I personally dislike it a lot—I promise, you are not setting a good precedent by preventing him from saying it. Worse, you’re giving him more people to say it to when the ensuing media coverage explodes.

If you actually want to fight back against these trolls, here’s a strategy to consider: Organize all you want, get as many people as you can to show up at their events, but don’t try to shut them down. In fact, the only thing you should try to shut down are the instigators who try to incite violence. Regain the moral high ground by saying that you absolutely respect their right to free speech.

And then, actually listen and talk to them. To me, the most effective retorts against the alt-right were when Trevor Noah had Tomi Lahren on his show and when Elle Reeve profiled Richard Spencer for Vice. Both came off looking mostly like jokes. Tomi Lahren showed her age. Richard Spencer revealed his movement to be mostly a collection of a few thousand sad dorks. Wale’s Twitter exchange with Tomi was effective too—there was no outrage, no opposition, just teasing.

They say sunlight is the best disinfectant. But it is also what allows you to see whether the emperor has any clothes. And it’s this sad, and often pathetic reality, that the collective hysteria has beneficently covered up in those it’s trying to fight. What should be seen as farce somehow looks like real fascism.

I realize there is legitimate fear of normalizing repulsive behavior. I’m not suggesting anyone give credence to real Nazi doctrine. However, historically, it’s usually true that banning and blocking usually has the opposite of its intended effect. Effective counterinsurgency usually involves bargaining, partnering and the reestablishment of norms—not hardlines. And this is already happening, Politico Magazine’s profile reveals that the jockeying for power and mainstream acceptance is pitting various factions of the alt-right against each other.

Remember how we used to think that Perez Hilton and TMZ were going to be the end of Western Civilization? Perez has had his moral makeover and the once fearless TMZ cuts deals with celebrities for access. It’s the timeless cycle of corruption and the dissipation of destructive energy.

It’s easy to sound smart and provocative when you’re the underdog. It’s easier to be reckless when you have nothing to lose. It’s also easier to create a united front when you really are being persecuted or attacked—when you’re an outsider. At least all of this is harder than expressing a coherent, cogent message for an extended period of time.

That’s the true hurdle for the alt-right to get over: Put up or shut up. The sooner you give them that chance—and stop ceding the high ground—the sooner they will falter. Or, alternatively, normalize themselves, play by the rules, or, get bored and move on. (Milo compared himself to Cincinnatus and says he wants to go back to his “farm.”)

Look at Tucker—and he’s still a friend, so I don’t mean to conflate him with anyone mentioned here—but when the controversy and outrage about his books dissipated, largely so did the sales. When he published a book of positive advice for guys—which was loved by the mostly female publishing industry and got all sorts of friendly press—it didn’t translate into success. He wasn’t an outlaw anymore. There wasn’t anything to get excited about. And now he’s moved on to other projects.

The old playbook stopped working…until a new generation picked it up again.

Ryan Holiday is the best-selling author of Trust Me, I’m Lying: Confessions of a Media Manipulator. Ryan is an editor-at-large for the Observer, and you can subscribe to his posts via email. He lives in Austin, Texas.

Also by Ryan Holiday:

These Books Explain the Media Nightmare We Are Supposedly Living In
It’s Time for the Media to Do Something About Its Shameful HelpAReporterOut Problem
How the Online ‘Diversity Police’ Defeat Themselves, and Leave Us All Much Worse Off
We Are Living in a Post-Shame World—And That’s Not a Good Thing
We Don’t Have a Fake News Problem—We Are the Fake News Problem
Want to Really Make America Great Again? Stop Reading the News.
Exclusive Interview: How This Right-Wing ‘Troll’ Reaches 100M People a Month
The Real Reason We Need to Stop Trying to Protect Everyone’s Feelings

Let's block ads! (Why?)

A couple years ago, I took a road trip from Wisconsin to Washington and mostly stayed in rural hotels on the way. I expected the internet in rural areas too sparse to have cable internet to be slow, but I was still surprised that a large fraction of the web was inaccessible. Some blogs with lightweight styling were readable, as were pages by academics who hadn’t updated the styling on their website since 1995. But very few commercial websites were usable (other than Google). When I measured my connection, I found that the bandwidth was roughly comparable to what I got with a 56k modem in the 90s. The latency and packetloss were significantly worse than the average day on dialup: latency varied between 500ms and 1000ms and packetloss varied between 1% and 10%. Those numbers are comparable to what I’d see on dialup on a bad day.

Despite my connection being only a bit worse than it was in the 90s, the vast majority of the web wouldn’t load. Why shouldn’t the web work with dialup or a dialup-like connection? It would be one thing if I tried to watch youtube and read pinterest. It’s hard to serve videos and images without bandwidth. But my online interests are quite boring from a media standpoint. Pretty much everything I consume online is plain text, even if it happens to be styled with images and fancy javascript. In fact, I recently tried using w3m (a terminal-based web browser that, by default, doesn’t support css, javascript, or even images) for a week and it turns out there are only two websites I regularly visit that don’t really work in w3m (twitter and zulip, both fundamentally text based sites, at least as I use them)¹.

More recently, I was reminded of how poorly the web works for people on slow connections when I tried to read a joelonsoftware post while using a flaky mobile connection. The HTML loaded but either one of the five CSS requests or one of the thirteen javascript requests timed out, leaving me with a broken page. Instead of seeing the article, I saw three entire pages of sidebar, menu, and ads before getting to the title because the page required some kind of layout modification to display reasonably. Pages are often designed so that they're hard or impossible to read if some dependency fails to load. On a slow connection, it's quite common for at least one depedency to fail. After refreshing the page twice, the page loaded as it was supposed to and I was able to read the blog post, a fairly compelling post on eliminating dependencies.

Complaining that people don’t care about performance like they used to and that we’re letting bloat slow things down for no good reason is “old man yells at cloud” territory; I probably sound like that dude who complains that his word processor, which used to take 1MB of RAM, takes 1GB of RAM. Sure, that could be trimmed down, but there’s a real cost to spending time doing optimization and even a $300 laptop comes with 2GB of RAM, so why bother? But it’s not quite the same situation -- it’s not just nerds like me who care about web performance. When Microsoft looked at actual measured connection speeds, they found that half of Americans don't have broadband speed. Heck, AOL had 2 million dial-up subscribers in 2015, just AOL alone. Outside of the U.S., there are even more people with slow connections. I recently chatted with Ben Kuhn, who spends a fair amount of time in Africa, about his internet connection:

I've seen ping latencies as bad as ~45 sec and packet loss as bad as 50% on a mobile hotspot in the evenings from Jijiga, Ethiopia. (I'm here now and currently I have 150ms ping with no packet loss but it's 10am). There are some periods of the day where it ~never gets better than 10 sec and ~10% loss. The internet has gotten a lot better in the past ~year; it used to be that bad all the time except in the early mornings.

…

Speedtest.net reports 2.6 mbps download, 0.6 mbps upload. I realized I probably shouldn't run a speed test on my mobile data because bandwidth is really expensive.

Our server in Ethiopia is has a fiber uplink, but it frequently goes down and we fall back to a 16kbps satellite connection, though I think normal people would just stop using the Internet in that case.

If you think browsing on a 56k connection is bad, try a 16k connection from Ethiopia!

Everything we’ve seen so far is anecdotal. Let’s load some websites that programmers might frequent with a variety of simulated connections to get data on page load times. webpagetest lets us see how long it takes a web site to load (and why it takes that long) from locations all over the world. It even lets us simulate different kinds of connections as well as load sites on a variety of mobile devices. The times listed in the table below are the time until the page is “visually complete”; as measured by webpagetest, that’s the time until the above-the-fold content stops changing.

Each row is a website. For sites that support both plain HTTP as well as HTTPS, both were tested; URLs are HTTPS except where explicitly specified as HTTP. The first two columns show the amount of data transferred over the wire in MB (which includes headers, handshaking, compression, etc.) and the number of TCP connections made. The rest of the columns show the time in seconds to load the page on a variety of connections from fiber (FIOS) to less good connections. “Bad” has the bandwidth of dialup, but with 1000ms ping and 10% packetloss, which is roughly what I saw when using the internet in small rural hotels. “😱” simulates a 16kbps satellite connection from Jijiga, Ethiopia. Rows are sorted by the measured amount of data transferred.

The timeout for tests was 6 minutes; anything slower than that is listed as FAIL. Pages that failed to load are also listed as FAIL. A few things that jump out from the table are:

1. A large fraction of the web is unusable on a bad connection. Even on a good (0% packetloss, no ping spike) dialup connection, some sites won’t load.
2. Some sites will use a lot of data!

The web on bad connections

As commercial websites go, Google is basically as good as it gets for people on a slow connection. On dialup, the 50%-ile page load time is a minute and a half. But at least it loads -- when I was on a slow, shared, satellite connection in rural Montana, virtually no commercial websites would load at all. I could view websites that only had static content via Google cache, but the live site had no hope of loading.

Some sites will use a lot of data

Although only two really big sites were tested here, there are plenty of sites that will use 10MB or 20MB of data. If you’re reading this from the U.S., maybe you don’t care, but if you’re browsing from Mauritania, Madagascar, or Vanuatu, loading codinghorror once will cost you more than 10% of the daily per capita GNI.

Page weight matters

Despite the best efforts of Maciej, the meme that page weight doesn’t matter keeps getting spread around. AFAICT, the top HN link of all time on web page optimization is to an article titled “Ludicrously Fast Page Loads - A Guide for Full-Stack Devs”. At the bottom of the page, the author links to another one of his posts, titled “Page Weight Doesn’t Matter”.

Usually, the boogeyman that gets pointed at is bandwidth: users in low-bandwidth areas (3G, developing world) are getting shafted. But the math doesn’t quite work out. Akamai puts the global connection speed average at 3.9 megabits per second.

The “ludicrously fast” guide fails to display properly on dialup or slow mobile connections because the images time out. On reddit, it also fails under load: "Ironically, that page took so long to load that I closed the window.", "a lot of … gifs that do nothing but make your viewing experience worse", "I didn't even make it to the gifs; the header loaded then it just hung.", etc.

The flaw in the “page weight doesn’t matter because average speed is fast” is that if you average the connection of someone in my apartment building (which is wired for 1Gbps internet) and someone on 56k dialup, you get an average speed of 500 Mbps. That doesn’t mean the person on dialup is actually going to be able to load a 5MB website. The average speed of 3.9 Mbps comes from a 2014 Akamai report, but it’s just an average. If you look at Akamai’s 2016 report, you can find entire countries where more than 90% of IP addresses are slower than that!

Yes, there are a lot of factors besides page weight that matter, and yes it's possible to create a contrived page that's very small but loads slowly, as well as a huge page that loads ok because all of the weight isn't blocking, but total page weight is still pretty decently correlated with load time.

Since its publication, the "ludicrously fast" guide was updated with some javascript that only loads images if you scroll down far enough. That makes it look a lot better on webpagetest if you're looking at the page size number (if webpagetest isn't being scripted to scroll), but it's a worse user experience for people on slow connections who want to read the page. If you're going to read the entire page anyway, the weight increases, and you can no longer preload images by loading the site. Instead, if you're reading, you have to stop for a few minutes at every section to wait for the images from that section to load. And that's if you're lucky and the javascript for loading images didn't fail to load.

The average user fallacy

Just like many people develop with an average connection speed in mind, many people have a fixed view of who a user is. Maybe they think there are customers with a lot of money with fast connections and customers who won't spend money on slow connections. That is, very roughly speaking, perhaps true on average, but sites don't operate on average, they operate in particular domains. Jamie Brandon writes the following about his experience with Airbnb:

I spent three hours last night trying to book a room on airbnb through an overloaded wifi and presumably a satellite connection. OAuth seems to be particularly bad over poor connections. Facebook's OAuth wouldn't load at all and Google's sent me round a 'pick an account' -> 'please reenter you password' -> 'pick an account' loop several times. It took so many attempts to log in that I triggered some 2fa nonsense on airbnb that also didn't work (the confirmation link from the email led to a page that said 'please log in to view this page') and eventually I was just told to send an email to account.disabled@airbnb.com, who haven't replied.

It's particularly galling that airbnb doesn't test this stuff, because traveling is pretty much the whole point of the site so they can't even claim that there's no money in servicing people with poor connections.

What about tail latency?

My original plan for this was post was to show 50%-ile, 90%-ile, 99%-ile, etc., tail load times. But the 50%-ile results are so bad that I don’t know if there’s any point to showing the other results. If you were to look at the 90%-ile results, you’d see that most pages fail to load on dialup and the “Bad” and “😱” connections are hopeless for almost all sites.

HTTP vs HTTPs

You can see that for a very small site that doesn’t load many blocking resources, HTTPS is noticeably slower than HTTP, especially on slow connections. Practically speaking, this doesn’t matter today because virtually no sites are that small, but if you design a web site as if people with slow connections actually matter, this is noticeable.

How to make pages usable on slow connections

The long version is, to really understand what’s going on, considering reading high-performance browser networking, a great book on web performance that’s avaiable for free.

The short version is that most sites are so poorly optimized that someone who has no idea what they’re doing can get a 10x improvement in page load times for a site whose job is to serve up text with the occasional image. When I started this blog in 2013, I used Octopress because Jekyll/Octopress was the most widely recommended static site generator back then. A plain blog post with one or two images took 11s to load on a cable connection because the Octopress defaults included multiple useless javascript files in the header (for never-used-by-me things like embedding flash videos and delicious integration), which blocked page rendering. Just moving those javascript includes to the footer halved page load time, and making a few other tweaks decreased page load time by another order of magnitude. At the time I made those changes, I knew nothing about web page optimization, other than what I heard during a 2-minute blurb on optimization from a 40-minute talk on how the internet works and I was able to get a 20x speedup on my blog in a few hours. You might argue that I’ve now gone too far and removed too much CSS, but I got a 20x speedup for people on fast connections before making changes that affected the site’s appearance (and the speedup on slow connections was much larger).

That’s normal. Popular themes for many different kinds of blogging software and CMSs contain anti-optimizations so blatant that any programmer, even someone with no front-end experience, can find large gains by just pointing webpagetest at their site and looking at the output.

What about browsers?

While it's easy to blame page authors because there's a lot of low-hanging fruit on the page side, there's just as much low-hanging fruit on the browser side. Why does my browser open up 6 TCP connections to try to download six images at once when I'm on a slow satellite connection? That just guarantees that all six images will time out! Even if I tweak the timeout on the client side, servers that are configured to protect against DoS attacks won't allow long lived connections that aren't doing anything. I can sometimes get some images to load by refreshing the page a few times (and waiting ten minutes each time), but why shouldn't the browser handle retries for me? If you think about it for a few minutes, there are a lot of optimiztions that browsers could do for people on slow connections, but because they don't, the best current solution for users appears to be: use w3m when you can, and then switch to a browser with ad-blocking when that doesn't work. But why should users have to use two entirely different programs, one of which has a text-based interface only computer nerds will find palatable?

Conclusion

When I was at Google, someone told me a story about a time that “they” completed a big optimization push only to find that measured page load times increased. When they dug into the data, they found that the reason load times had increased was that they got a lot more traffic from Africa after doing the optimizations. The team’s product went from being unusable for people with slow connections to usable, which caused so many users with slow connections to start using the product that load times actually increased.

Last night, at a presentation on the websockets protocol, Gary Bernhardt made the observation that the people who designed the websockets protocol did things like using a variable length field for frame length to save a few bytes. By contrast, if you look at the Alexa top 100 sites, almost all of them have a huge amount of slop in them; it’s plausible that the total bandwidth used for those 100 sites is probably greater than the total bandwidth for all websockets connections combined. Despite that, if we just look at the three top 35 sites tested in this post, two send uncompressed javascript over the wire, two redirect the bare domain to the www subdomain, and two send a lot of extraneous information by not compressing images as much as they could be compressed without sacrificing quality. If you look at twitter, which isn’t in our table but was mentioned above, they actually do an anti-optimization where, if you upload a PNG which isn’t even particularly well optimized, they’ll re-encode it as a jpeg which is larger and has visible artifacts!

“Use bcrypt” has become the mantra for a reasonable default if you’re not sure what to do when storing passwords. The web would be a nicer place if “use webpagetest” caught on in the same way. It’s not always the best tool for the job, but it sure beats the current defaults.

Appendix: experimental caveats

The above tests were done by repeatedly loading pages via a private webpagetest image in AWS west 2, on a c4.xlarge VM, with simulated connections on a first page load in Chrome with no other tabs open and nothing running on the VM other than the webpagetest software and the browser. This is unrealistic in many ways.

In relative terms, this disadvantages sites that have a large edge presence. When I was in rural Montana, I ran some tests and found that I had noticeably better latency to Google than to basically any other site. This is not reflected in the test results. Furthermore, this setup means that pages are nearly certain to be served from a CDN cache. That shouldn't make any difference for sites like Google and Amazon, but it reduces the page load time of less-trafficked sites that aren't "always" served out of cache. For example, when I don't have a post trending on social media, between 55% and 75% of traffic is served out of a CDN cache, and when I do have something trending on social media, it's more like 90% to 99%. But the test setup means that the CDN cache hit rate during the test is likely to be > 99% for my site and other blogs which aren't so widely read that they'd normally always have a cached copy available.

All tests were run assuming a first page load, but it’s entirely reasonable for sites like Google and Amazon to assume that many or most of their assets are cached. Testing first page load times is perhaps reasonable for sites with a traffic profile like mine, where much of the traffic comes from social media referrals of people who’ve never visited the site before.

A c4.xlarge is a fairly powerful machine. Today, most page loads come from mobile and even the fastest mobile devices aren’t as fast as a c4.xlarge; most mobile devices are much slower than the fastest mobile devices. Most desktop page loads will also be from a machine that’s slower than a c4.xlarge. Although the results aren’t shown, I also ran a set of tests using a t2.micro instance: for simple sites, like mine, the difference was negligible, but for complex sites, like Amazon, page load times were as much as 2x worse. As you might expect, for any particular site, the difference got smaller as the connection got slower.

As Joey Hess pointed out, many dialup providers attempt to do compression or other tricks to reduce the effective weight of pages and none of these tests take that into account.

Firefox, IE, and Edge often have substantially different performance characteristics from Chrome. For that matter, different versions of Chrome can have different performance characteristics. I just used Chrome because it’s the most widely used desktop browser, and running this set of tests took over a full day of VM time with a single-browser.

The simulated bad connections add a constant latency and fixed (10%) packetloss. In reality, poor connections have highly variable latency with peaks that are much higher than the simulated latency and periods of much higher packetloss than can last for minutes, hours, or days. Putting 😱 at the rightmost side of the table may make it seem like the worst possible connection, but packetloss can get much worse.

Similarly, while codinghorror happens to be at the bottom of the page, it's nowhere to being the slowest loading page. Just for example, I originally considered including slashdot in the table but it was so slow that it caused a significant increase in total test run time because it timed out at six minutes so many times. Even on FIOS it takes 15s to load by making a whopping 223 requests over 100 TCP connections despite weighing in at "only" 1.9MB. Amazingly, slashdot also pegs the CPU at 100% for 17 entire seconds while loading on FIOS. In retrospect, this might have been a good site to include because it's pathologically mis-optimized sites like slashdot that allow the "page weight doesn't matter" meme to sound reasonable.

The websites compared don't do the same thing. Just looking at the blogs, some blogs put entire blog entries on the front page, which is more convenient in some ways, but also slower. Commercial sites are even more different -- they often can't reasonably be static sites and have to have relatively large javascrit payloads in order to work well.

Appendix: irony

The main table in this post is almost 50kB of HTML (without compression or minification); that’s larger than everything else in this post combined. That table is curiously large because I used a library (pandas) to generate the table instead of just writing a script to do it by hand, and as we know, the default settings for most libraries generate a massive amount of bloat. It didn’t even save time because every single built-in time-saving feature that I wanted to use was buggy, which forced me to write all of the heatmap/gradient/styling code myself anyway! Due to laziness, I left the pandas table generating scaffolding code, resulting in a table that looks like it’s roughly an order of magnitude larger than it needs to be.

This isn't a criticism of pandas. Pandas is probably quite good at what it's designed for; it's just not designed to produce slim websites. The CSS class names are huge, which is reasonable if you want to avoid accidental name collisions for generated CSS. Almost every td, th, and tr element is tagged with a redundant rowspan=1 or colspan=1, which is reasonable for generated code if you don't care about size. Each cell has its own CSS class, even though many cells share styling with other cells; again, this probably simplified things on the code generation. Every piece of bloat is totally reasonable. And unfortunately, there's no tool that I know of that will take a bloated table and turn it into a slim table. A pure HTML minifier can't change the class names because it doesn't know that some external CSS or JS doesn't depend on the class name. An HTML minifier could theoretically determine that different cells have the same styling and merge them, except for the aforementioned problem with potential but non-existent external depenencies, but that's beyond the capability of the tools I know of.

For another level of ironic, consider that while I think of a 50kB table as bloat, this page is 12kB when gzipped, even with all of the bloat. Google's AMP currently has > 100kB of blocking javascript that has to load before the page loads! There's no reason for me to use AMP pages because AMP is slower than my current setup of pure HTML with a few lines of embedded CSS and the occasional image, but, as a result, I'm penalized by Google (relative to AMP pages) for not "accelerating" (deccelerating) my page with AMP.

Thanks to Leah Hanson, Jason Owen, Ethan Willis, and Lindsey Kuper for comments/corrections

Esvaziou mesmo.

The LHCb detector. (credit: Fermilab)

Everyone, at some point in their lives, wonders why they are here. Existential questions don't stop at the personal level, though. Why is there a Universe, and why is it filled with matter? The last question is a puzzle that has gainfully occupied the minds of and employed physicists for many years. The time spent pondering such questions has not been wasted, as it turns out, as researchers from the LHCb detector report that one of the theoretical paths that allows matter to outnumber antimatter is open for business.

An overly simple reading of the Standard Model of physics predicts that matter will be produced at the same rate as antimatter. The antimatter and matter should, through simple statistics, collide and wipe each other out, leaving only energy. But that didn't happen. The substance we label matter was, somehow, produced in greater abundance than antimatter. In the beginnings of the Universe, antimatter was eliminated, leaving only matter.

A closer look at the Standard Model reveals that some imbalance is expected. But it also predicts a Universe with much less matter than we observe. And, experimentally, we've only observed the relevant matter/antimatter asymmetry for a particular class of particles, called mesons. That notably leaves out the particles that make up the Universe, called baryons. Luckily, baryon asymmetry is exactly what one of the LHC detectors, called LHCb, is designed to investigate.

Read 13 remaining paragraphs | Comments

Happy v day! Find more of my stuff here:

webtoon / website / facebook / twitter / patreon

This comic reinforces my opinion that I was, in fact, the villain of Basic Instructions. Look at this thing! That poor guy came to me to commiserate about his problems, and I accuse both him and his teenage daughter of having body odor.

 Yeah, Mullet Boss and the angry customer weren’t great either, but for the most part, Basic Instructions was a comic about me behaving selfishly and heaping smug verbal abuse on anyone who stumbled into range. That may be why it always amazed me when a reader would express an interest in actually meeting me.

Note from Missy: On the other hand, having been a teenage girl, I wish all the other teenage girls had been given the gift of a light blue container. :D

As always, thanks for using my Amazon Affiliate links (US, UK, Canada).

More quality parenting.

– Ei, por acaso conhece o clubinho da Anésia?

When Ric and I first met, we were both comedians. He was much farther along in his career than I was. Still, “mentor” is too strong a term for his place in my career. I’d describe his role as being somewhere between “occasional advisor” and “cautionary example.”

We knew a guy who taught a stand-up comedy class. He would occasionally ask comedians to speak at his classes. He never asked me or Ric to speak. We once spent a large portion of a drive to a gig thinking about what we would say if he did. Here are the few things we came up with that I remember:

“Fear can keep you alive.”

“A wish to be famous is a terrible reason to go into comedy. A wish to pay your bills is an even worse reason.”

“If there’s one thing I’ve learned traveling the country doing comedy, it’s this: ‘People hate comedians.’”

As always, thanks for using my Amazon Affiliate links (US, UK, Canada).

Let's block ads! (Why?)

Today I learned about a really cool machine learning art tool!!

It is called “paintschainer” and it lets you colour in black and white images!

The github repo is at https://github.com/pfnet/PaintsChainer and you can try it out yourself at http://paintschainer.preferred.tech/. It seems to be by this Japanese company.

Here is my favorite thing I used it for so far: (before & after). This is a picture of me from the networking zine cover that liz drew.

I also coloured in the whole cover of the networking zine as an experiment. It also made me happy but didn’t come out quite as awesome. Someone on twitter described this as “Confused Girl In Orange Hoodie Finds LSD, Hallucinates Purple Cat, Becomes Networking Genius” which I think is probably about right.

machine learning art tools & making something that feels like I made it

This made me think about what kinds of machine learning art tools I find compelling / exciting. I am not very good at a lot of technical art things yet – I can’t do shading, I don’t really know how to colour something in. I can’t draw a cartoon animal without following a tutorial.

deepart.io lets you take a photo and redraw it in the “style” of another painting. I thought this example was really cool. It takes a pretty ordinary photograph, combines it with a painting, and gives you something really cool looking!

I spent a bunch of time playing with deepart.io but couldn’t make anything I really was excited about. My current hypothesis is – I want to use magical art tools to make things that feel like I made them.

tablets & tracing

While we’re talking about art tools and technology that enables you to do something cool – I bought a 10 inch Android tablet in September, which is what I use to draw all my zines / programming drawings. Another fun thing I very occasionally do with it is to take photos and then trace them into sketches.

This one is a house a few blocks away from where I live.

This is dramatically better than my usual house drawing skills. Here is what it usually looks like when I draw a house freehand:

I wondered what this would look if I coloured it with the magical colouring tool paintschainer. It’s kind of cool!

I find this really cool because I can make an image with my hands (I drew all the lines in that image!) with a lot less skills than I would have needed without this magical technology. And it feel more like a thing that I made because I have a lot of creative control over what it looks like (I took the original photo, I decided what to trace, and I picked the colours to colour it in with).

a very short listing of magical art tools

• a tablet that you can use to trace photos with
• deepart.io (and this paper: a neural algorithm of artistic style. github repo)
• deep dream generator (“add dogs & creepy eyes to anything!“)
• paintschainer (what I used for the images in this post). github repo
• This cool “realistic image manipulation” paper has some nice videos (but you don’t appear to be able to use it). Thanks to Katherine Ye for showing it to me!!

if you know about more tools like this I would like to hear about them! This kind of thing is probably the application of neural networks that makes me the most excited.

Amazon today unveiled a new subscription program aimed at parents called STEM Club, which delivers educational toys to your home for $19.99 per month. The retailer says it will hand-pick which toys are shipped, and will ensure the items are age-appropriate. And by “STEM,” of course, Amazon means the toys will be focused on the areas of science, technology, engineering and math.

The subscription program won’t feature just any ol’ STEM toys, however, but will rather only include those that have recently launched or those that are exclusive to Amazon.

To sign up, parents visit the STEM Club homepage, then select the age range of their child (3-4, 5-7 or 8-13). The first toy will arrive in under a week’s time with free shipping. From that point forward, a new item will arrive on a monthly basis. The service is only available in the U.S., the website notes.

This isn’t Amazon’s first attempt at highlighting STEM toys on its site. In 2015, the retailer launched the STEM Toys & Games Store as a destination for browsing through this type of product in a dedicated area.

Of course, for Amazon, the launch of the new storefront wasn’t so much about trying to spark young minds and encourage learning, but to better capitalize on parents’ interest in the STEM toy trend in order to impact Amazon’s own bottom line. At the time, STEM toys were the second-most visited section and had seen the highest sales volume during the prior holidays.

Similarly, Amazon’s interest in launching a subscription service for these toys is also motivated by being able to capture a recurring revenue stream. Like its “Subscribe & Save” program, the hope is that the new subscription service will encourage a sort of “set it and forget it” mentality among shoppers.

But whether parents will sign up in the first place remains to be seen. After all – I don’t know about you – but we certainly have enough toys around here. I can’t imagine wanting to receive one more every month.

Let's block ads! (Why?)

Desculpa, gente. Não vai ter tira hoje. Parece que todos os problemas resolveram aparecer na mesma tarde. Zica aqui no blog, zica com clientes na lojinha. Espero que me perdoem um dia. A tira acima eu fiz ano passado pro pessoal do Kibe Loco. Eu gosto dela.