Shared posts

07 Aug 13:16

How Scalable Do Analytics Solutions Need to Be?

by Stephen Few

While being briefed on a product earlier this week, the company’s founder and I agreed on one point only: most of the people who are currently tasked with data analysis lack the skills that are required to do the work. He and I, however, imagine conflicting solutions to this problem. He believes that technology must come to the rescue by doing the work for these people who can’t do it for themselves. I believe that even the best technologies cannot do the work of skilled data analysts and that the problem can only be effectively addressed by helping people develop analytical skills. He agreed that equipping people with the necessary skills would work better, but dismissed it because it is not a “scalable solution.” The essence of his case went something like this: “Data is increasing at an exponential rate, so our need for analytics cannot be solved by investing in human resources because humans are not sufficiently scalable, but technologies are.” Consider this line of reasoning for a moment. It relies on the following premise: “Exponential increases in data can only be addressed by exponential increases in analytical horsepower.” This premise is fallacious. Nate Silver made this point in his book The Signal and the Noise when he wrote:

If the quantity of information is increasing by 2.5 quintillion bytes per day, the amount of useful information certainly isn’t. Most of it is just noise, and the noise is increasing faster than the signal. There are so many hypotheses to test, so many data sets to mine—but a relatively constant amount of objective truth.

The exponential growth in raw data that we’re experiencing is mostly producing noise. The amount of useful information is not increasing exponentially, therefore the need for analytical horsepower is also not increasing exponentially. Data sensemaking is a human activity that can at best be augmented and assisted by analytical tools. The only viable solution to the analytical challenges that we face is to develop the human resources that we need. This is where our attention and our investments should be focused. Don’t trust a technology vendor who claims that skilled data analysts can be replaced with his product. That analytical product does not exist.

This company’s founder claims that his product can analyze a data set and present all of the potentially useful findings in a series of simple graphs and plain English explanations without any human involvement. During the briefing, he made an off-the-cuff comment that caused the hairs on the back of my neck to bristle. He said that his product “empowers users.” He must understand empowerment quite differently than I do. As I understand it, empowerment involves an increase in ability. Software that does for you what you could do better yourself with proper training isn’t empowering.

This fellow’s notion of empowerment bothered me because I work hard to actually empower people by teaching them analytical skills. I know how much it means to people to become truly empowered with useful abilities that enable them to affect the world in beneficial ways. No one with an ounce of integrity wants to bear the title “data analyst” while doing nothing but delivering a computer’s findings to someone else without adding any value. If this is the future that analytics technologies promise, count me out. Fortunately, this isn’t a future that technologies are likely to achieve.

Take care,

Signature

19 Mar 18:42

Dennett tries to save free will, fails

by whyevolutionistrue

I’ve long been puzzled by the many writings of “compatibilists”: those philosophers and laypeople who accept physical determinism of our choices and behaviors, but still maintain that we have a kind of “free will.” Such people reject the classical form of free will that’s been so important to many people (especially religious ones)—the kind of “libertarian” free will that posits that we really can freely control our actions, and in many cases could have chosen to behave other than how we did. This is the kind of free will that most people accept, as they don’t see the world as deterministic; and most also feel that if the world were deterministic, people would lose moral responsibility for their actions (see my post on the work of Sarkissian et al.).

Based on statements of some compatibilists, I realized that one reason philosophers spend so much time trying to define forms of free will compatible with determinism is because they see bad consequences of rejecting all free will. Some compatibilists think that if people realized that they don’t have the kind of free will they thought they did, the world would disintegrate: people would either lie in bed out of sheer languor and despair, or behave “immorally” because, after all, we can’t choose how to behave.

I’ve been rebuked sharply for imputing these motivations to compatibilists. Their efforts, I’m told, have nothing to do with trying to stave off possible bad results of rejecting free will. Rather, they’re supposedly engaged in a purely philosophical exercise: trying to show that we still have a form of free will that really matters, even if the libertarian form has been killed off by science.  I have, however, responded by pointing out statements by compatibilists like Dan Dennett warning about the bad things that could happen if neuroscientists tell us that we don’t have free will.

If you ever doubted that compatibilism is motivated largely by philosophers’ fears about what would happen if people rejected classical free will, and weren’t presented with a shiny new compatibilist form, watch this “Big Think” video by Dan Dennett. It’s called “Stop telling people they have free will”:

Supposedly aimed at promulgating a better concept of free will, Dan’s video in fact doesn’t do that at all. Rather, Dennett tries to show that those neuroscientists who tell people they don’t have free will are being “mischievous” and “irresponsible.” He devises a thought experiment that shows only one thing: if people don’t think they have free will, they start behaving badly, and could even commit crimes! They become “morally incompetent people.” His short talk is an exercise in consequentialism, not a philosophical recasting of free will.

Dennett even cites the Vohs and Schooler experiment purporting to show that if people read passages showing that they have no free will, they tend to cheat more on subsequent puzzle-solving tests. (Note that those supposed effects are tested over a very short span—an hour or two—and say absolutely nothing about the long term effects of rejecting classical free will.)

Dennett, however, fails to cite the work of Rolf Zvaan at Rotterdam, who failed to replicate the results of Vohs and Schooler while pointing out defects in their experimental design. (See my post on that here.) Zvaan found absolutely no effect of reading pro- and anti-free will passages on the level of cheating in subsequent tests. His paper is being submitted for publication.

But even if people behaved worse if they were told that determinism reigns and libertarian free will doesn’t exist, so what? The truth is the truth, and if science shows us something like that, we simply have to deal with it. After all, science has found no evidence for God, either, and yet there are studies showing that belief in God similarly produces better short-term behavior on psychological tests. Do philosophers like Dennett then try to confect new definitions of God—”the kind of a God worth wanting”? Maybe we should redefine God to comport with science: “God is the Cosmos.” No, of course they don’t do that. They’re atheists!

It is curious that Dennett has spent a lot of time attacking the concept of “belief in belief”: the idea that we should tolerate religious belief because, even if not based on truth, it still makes people behave better. Yet when the “belief” is in free will rather than God, then “belief in belief” becomes not only okay, but essential.

And that, I think, is why some compatibilists try to invent forms of free will to replace the libertarian version. They do it, I believe, because they can then tell people that they really do have free will, and so we’ll all continue to behave well and society will thrive.

But I don’t believe that people will either run amok or become vegetables if they become incompatibilists and realize that all our behaviors are determined (or perhaps slightly affected by quantum indeterminacy, which still does not constitute anybody’s idea of “free will”). I’m an incompatibilist, and since I became one neither I nor anyone else has noticed a change in my behavior. I haven’t started robbing banks or assaulting people, and I sure don’t lie abed in the morning!

Society will learn to live with determinism, as it has learned to live with death and the absence of God. And, as I always maintain, abandoning the idea of free will is actually good for society in several ways: it undermines religion, and it is a highly useful attitude when thinking about how to reform the criminal justice system.

*******

BTW, while we’re on free will, reader Jim E. sent a short (2-minute) animation about the famous Libet experiment, and pointed out that Professor Ceiling Cat makes a cameo appearance as a critic of free will. And I do! Look for me at 1:05 in the video below. Dennett is in there, too—as an experimental subject!


07 Oct 16:14

Matryoshka Hardware

Introduction

Most activities that I do require a combination of activity-specific gear and a set of more general core hardware. A ski trip requires specific gear like skies and gloves, but also core hardware like a wallet and a good book. I’ve found that most activities of the same duration and complexity share the same set of core hardware. For example, a 3 day ski adventure, scientific conference, or holiday weekend requires much of the same core hardware.

A byproduct of this division between activity-specific gear and core hardware is that core hardware naturally organizes into layers. Core hardware that I use for shorter duration activities serves as the foundation for core hardware that I carry for longer duration activities. A smaller set of core hardware integrates into the layer above it. Each layer subsumes the previous layer like a series of Matryoshka dolls.

Nesting hardware like Matryoshka dolls has numerous advantages. It minimizes hardware redundancy and ipso facto reduces the number of items that need to be carried. Predetermined hardware configurations facilitates faster packing with greater fidelity. It’s harder to forget important items using logical hardware layers. This post explains my views about hardware abstraction and describes 3 of my core hardware layers that I use to get things done.

Layers

The atom of my hardware is an Every Day Carry (EDC). This layer is composed of the minimal set of gear that I carry each day. The items I carry in this layer go with me everywhere. I dislike carrying things, so this hardware layer aims to be high functional albeit with minimal size and weight.

In situations where my EDC is insufficient, I add additional items to the EDC to form layers with more functionality. The next layer up from my EDC is my Extended Carry Items (ECI). I use Extended Carry Items on longer day trips or in other contexts where I need more functionality than my EDC can provide. For activities where I need yet more hardware, I merge my ECI with more gear to form my Bug-out Bag(BOB).1 My BOB is typically used for longer, usually multi-day, excursions.

EDC

My base EDC hardware layer is composed of a knife, keychain, wallet and a phone:

At the foundation of my EDC is a good knife. I’ve tried many knives in my kit, but my favorite is the Benchmade Osbourne Mini-Barrage. Benchmade manufactures ideal EDC blades. They’re of high quality and are reasonably priced, so I’m not afraid to abuse this knife. The Mini-Barrage is an ideal EDC blade because it’s the perfect size—large enough to fit comfortably in my hand when open and compact enough to fit comfortability in my pocket when closed. I also like the AXIS-assist design which facilitates opening the knife with one hand:

For a key ring, I use the titanium short-coated wire. It has a unique design that only opens when it is twisted into a special shape. I find that it never accidentally comes apart like screw-type key rings that I’ve used. The ability to open and close the keychain also allows me to take tools or spare keys on and off for modularity and easier use.2 It also allows me to fasten the keychain to a backpack or other object so I don’t misplace my keys. This design minimizes bulk by eliminating the need to carry a bulky carabiner or other attachment device.

I carry three items on my short-coated wire—an LRI Photon light, USB stick, and an Atwood tool. The Photon is ideal for reading, unlocking a door at night, or finding something in the bottom of a backpack. It’s extremely bright, compact, and inexpensive. I use a USB for transferring photos or presentation slides for talks. I also keep several privacy and encryption tools on it like tails. My current USB drive is a Kensington DataTraveler—it’s small, fairly durable, and has a nice attachment eyelet. The last tool on my keychain is a Atwood Titanium Wrunt. I use the tool for all sorts of tasks such as prying, screwing, scraping, and researching beer. The Wrunt is an excellent complement to the Barrage. Together, these two tools can be used for a wide array of applications:

The iPhone is also part of my EDC. The apps I use everyday include Omnifocus and Downcast. Here’s a screenshot.

The last item in my EDC is a money clip. I dislike wallets because they’re bulky and prone to dropping their contents. For a money clip, I stole an idea from Instructables and use a binder clip. Binder clips take up almost no room in my pocket, costs pennies, and securely hold money and credit cards. They’re also multi-purpose and have many other uses. Another feature I like is that the binder clip can be flipped up and placed onto my short-coated wire. I find this extremely useful if I’m at the gym or any place where I don’t have pockets. I also wrap my money clip with a ranger band for added stability. The band can also be repurposed for other tasks as well.

Extended Carry

The next level up from my base EDC is my Extended Carry Items. This module adds three additional items to my EDC—an iPad, a notebook, and an organizer:

The majority of my ECI is housed in a Maxpedition Pocket Organizer. The Maxpedition excels at keeping numerous small pieces of hardware organized in a compact footprint. It has numerous attachment points, pockets, and lashing straps for organizing gear. The small profile makes the organizer easy to carry by itself or in a pack. Most of the gear I carry in the ECI is not directly visible in this picture. Here are a few of the items I carry in the Maxpedition:

Sunwayman V11R: This is a fantastic light that strikes a nice balance between price, size, and performance. I carry this model for its output-to-runtime efficiency, optional AA battery compatibility, and continuously variable beam control. The light is equally adept in situations where I need a low-intensity beam for an extended period of time or in specialized circumstances where I want to throw a 500 lumen blast. The light also makes a great lantern when attached with paracord to a Grimlock:

Distraction countermeasures: Sometimes I need to concentrate on the work that I’m doing or tune out disruptions to facilitate sleeping. Distraction countermeasures are absolutely indispensable in my ECI. I carry ear plugs and Etymotic Research HF3 headphones. The ear plugs are primarily used for sleeping on planes, trains, and automobiles. When I’m coding or doing almost any kind of work where I need to focus, I listen to music. The HF3 has reasonable audio quality, takes up little room, and has a nice mic and iPhone controls.

Adenosine triphosphate: For sustenance, I always keep a few snacks in my ECI. These items are handy when I don’t have time to eat Bieber, Gaga, and Eastwood for breakfast or if I don’t have access to food such as on a plane or in the backcountry. I like to carry Pemmican on shorter trips, but it has a short shelf-life and isn’t part of my ECI. Instead, I carry Active Green Bars, which have a reasonable nutritional profile for a pre-packaged bar. These bars are good for long-term storage in the ECI because they don’t have a coating that can melt and remain chewy even when cold.

Cable Management: Organizing cables is a real pain. When unmanaged, cables get wrapped around other gear and twisted together making them hard to work with. My ECI has to deal with headphones, cables for my iPhone/iPad, as well as any cables for other activity-specific gear I may be carrying. I’ve found three useful cable management gadgets that remove most of the cable organizing frustrations. Conventional cable ties, reusable Velcro cable ties, and Applecore cable oganizers.

Repair: Stuff breaks. Sometimes it’s gear, sometimes it’s me. I try to be prepared for these events and mitigate any issues before they become a problem. Some of the multi-functional items I carry for repair tasks include safety pins and a pocket screwdriver. For hands specific protection, I keep nail clippers, safety pins, hand repair creme, and hand sanitizer:3. A favorite repair item of mine is Gorilla tape. It’s the best kind of tape and has almost unlimited uses. I like to use it for things like protecting blisters on my feet or patching a holes in a water bottle. I keep Gorilla tape spooled around the sheath of an old pen as a dispenser. This design also allows me to store spare paracord in a Solomon Bar:

Writing/Drawing: I keep a few writing and drawing instruments in my ECI. Currently, my main workhorse is a Retro 51 Hex-o-matic. The Hex-o-matic ships as a ballpoint, but I use it with the excellent Schmidt 8900 Superbowl refills. This refill is unusual in that it transforms the Hex-o-matic into a liquid ink rollerball. I like rollerballs because they write well on most surfaces like a ballpoint and are smooth with good ink flow like a fountain pen. I also carry several Faber Castell Pitt pens for situations where I need to lay own a lot of ink. If the Pitt pens are good enough for Don Colley, they’re good enough for me.4

Weatherproofing: Several items in my ECI are prone to weather damage. I store my iPhone in my ECI inside a waterproof ziplock freezer bag. I also need to keep small items like extra money, my USB stick, and SD cards safe and dry. I find conical tubes to be an excellent storage containers. They’re lightweight, waterproof, crush-resistant, and can even hold liquids. Conical tubes are perfect containers for compartmentalizing small hardware items as well. My ECI can hold two 50 milliliter conical tubes for small hardware items and one 15 mL tube that I use for storing hand sanitizer. The contents of one 50mL tube include—Advil, spare money, an SD card, a MagSafe 2 converter, ear plugs, and Velcro cable ties:

BOB

The next level up in my hardware is the Bug-out Bag (BOB). I use this configuration in situations where my Extended Carry is insufficient. The BOB is designed to hold enough gear for a day to a week. I use one of two different bags as my BOBs depending on the situation. Both BOBs have a volume of roughly 25 liters, which I find is small enough to easily carry, yet large enough to hold adequate core hardware and activity specific-gear. The core hardware contents of either pack are the same.

Deciding what bag to carry largely hinges on weight and how long I need to carry a load. For most urban settings I select the Tom Bihn Empire Builder.6 The bag has a well thought out design that supposedly took 3 years to develop. The pockets are positioned well the carrying handles are extremely comfortable. I especially like the integration with the Braincell for securely carrying a laptop:

The other pack I use is the Arcteryx Quintic.7 I’ve just started using this pack after retiring an older BOB. The Quintic has a unique design that was developed primarily for backcountry skiing. The main access point to the central compartment is through the side of the pack. I like this feature because I can spin the pack around one shoulder to gain access to the Quintic’s contents while skiing, hiking, or maneuvering through a city. The best feature of this pack though is how well it carries a load. It was designed to hug a skier during descents and I find this quality translates to everyday situations as well:

Here are a few of the items I carry in my BOB:

Compression Sacks: An indispensable item in my BOB is the mighty compression sack. Compression sacks are designed to compact gear into smaller spaces. I usually carry several of these in my BOB. They provide a way to compartmentalize gear and to carry more things in less space. I like Granite Gear compression sacks in particular. I use several models ranging from the extremely light weight to waterproof models. My favorite models are semi-translucent making it easier to spot items inside the sack:

Liquids: A staple in my BOB is water. I carry water in the Platypus plusBottle and/or the Big Zip SL. I use several Platypus products primarily because they’re the only company I’ve found that doesn’t make water taste like plastic. When not in use, Platypus products weigh next to nothing and fold down to takes up little pack space. Another liquid I like to carry in my BOB is Spike. I have almost no response to coffee or other caffeine-based stimulants, but the combination of N-acetyl-Tyrosine and Yohimbine in Spike is a wonderful stimulant and a nice addition to my ECI distraction countermeasures.

Technical Wear: Two jackets that seem to always find their way into my BOB are the Patagonia Supercell and Retro-X Vest jackets. I usually pack the Supercell in my BOB to safeguard against precipitation and I wear the Retro-X everywhere. The combination of these jackets can handle a huge range of weather I encounter and both jackets are extremely lightweight and packable.

Books: A good book is an essential BOB item. I usually alternate between reading one technical and one recreational. A few of my favorite recent recreational books have been Seeking Wisdom, Ghost in the Wires and The Psychology of Persuasion.


  1. I’m borrowing the terms EDC and Bug-out Bag which generally has a more survivalist/tactical connotation. 

  2. I wish the short-coated wire was a touch longer. It’s usually not a problem for me, but the length of the wire makes it difficult to fasten if there are many items on the keychain. 

  3. Note that this is not a man-purse. Man-purses have shoulder straps. Man-purses are cool though, Indiana Jones carried one. 

  4. Hand sanitizer, hand repair creme, and lip balm are excellent fire starting reagents. 

  5. Don Colley is my favorite contemporary artist. John Thornton’s Have Pen Will Travel is a good introduction. 

  6. I really like Tom Bihn. I received the Empire Builder as a birthday gift a few years ago and it arrived directly from Tom Bihn hand decorated by the employees. 

  7. Quintic is a peculiar name for this pack; was it named after a fifth order polynomial

09 Jul 10:17

Osama bin Laden's life on the run revealed by Pakistani inquiry

Osama bin Laden lived in plain sight for almost a decade thanks to the incompetence of Pak's intelligence services, an official report into his killing said.