Nosimpler

Nature Neuroscience 16, 772 (2013). doi:10.1038/nn.3441

Author: Alfonso Renart

Chaotic networks produce rich temporal dynamics that could be useful for timing, but are extremely sensitive to perturbations. Work now shows that a learning rule for the weights of a chaotic recurrent network can stabilize time-varying activity patterns. This result can be used to train output units to produce generic timed responses.

Visual illusions and hallucinations are hallmarks of serotonergic hallucinogen-induced altered states of consciousness. Although the serotonergic hallucinogen psilocybin activates multiple serotonin (5-HT) receptors, recent evidence suggests that activation of 5-HT2A receptors may lead to the formation of visual hallucinations by increasing cortical excitability and altering visual-evoked cortical responses. To address this hypothesis, we assessed the effects of psilocybin (215 μg/kg vs placebo) on both α oscillations that regulate cortical excitability and early visual-evoked P1 and N170 potentials in healthy human subjects. To further disentangle the specific contributions of 5-HT2A receptors, subjects were additionally pretreated with the preferential 5-HT2A receptor antagonist ketanserin (50 mg vs placebo). We found that psilocybin strongly decreased prestimulus parieto-occipital α power values, thus precluding a subsequent stimulus-induced α power decrease. Furthermore, psilocybin strongly decreased N170 potentials associated with the appearance of visual perceptual alterations, including visual hallucinations. All of these effects were blocked by pretreatment with the 5-HT2A antagonist ketanserin, indicating that activation of 5-HT2A receptors by psilocybin profoundly modulates the neurophysiological and phenomenological indices of visual processing. Specifically, activation of 5-HT2A receptors may induce a processing mode in which stimulus-driven cortical excitation is overwhelmed by spontaneous neuronal excitation through the modulation of α oscillations. Furthermore, the observed reduction of N170 visual-evoked potentials may be a key mechanism underlying 5-HT2A receptor-mediated visual hallucinations. This change in N170 potentials may be important not only for psilocybin-induced states but also for understanding acute hallucinatory states seen in psychiatric disorders, such as schizophrenia and Parkinson's disease.

My friend Josh Vekhter sent me this blog post written by someone who calls herself celandine13 and tutors students with learning disabilities.

In the post, she reframes the concept of mistake or “being bad at something” as often stemming from some fundamental misunderstanding or poor procedure:

Once you move it to “you’re performing badly because you have the wrong fingerings,” or “you’re performing badly because you don’t understand what a limit is,” it’s no longer a vague personal failing but a causal necessity.  Anyone who never understood limits will flunk calculus.  It’s not you, it’s the bug.

This also applies to “lazy.”  Lazy just means “you’re not meeting your obligations and I don’t know why.”  If it turns out that you’ve been missing appointments because you don’t keep a calendar, then you’re not intrinsically “lazy,” you were just executing the wrong procedure.  And suddenly you stop wanting to call the person “lazy” when it makes more sense to say they need organizational tools.

And she wants us to stop with the labeling and get on with the understanding of why the mistake was made and addressing that, like she does when she tutors students. She even singles out certain approaches she considers to be flawed from the start:

This is part of why I think tools like Knewton, while they can be more effective than typical classroom instruction, aren’t the whole story.  The data they gather (at least so far) is statistical: how many questions did you get right, in which subjects, with what learning curve over time?  That’s important.  It allows them to do things that classroom teachers can’t always do, like estimate when it’s optimal to review old material to minimize forgetting.  But it’s still designed on the error model. It’s not approaching the most important job of teachers, which is to figure out why you’re getting things wrong — what conceptual misunderstanding, or what bad study habit, is behind your problems.  (Sometimes that can be a very hard and interesting problem.  For example: one teacher over many years figured out that the grammar of Black English was causing her students to make conceptual errors in math.)

On the one hand I like the reframing: it’s always good to see knee-jerk reactions become more contemplative, and it’s always good to see people trying to help rather than trying to blame. In fact, one of my tenets of real life is that mistakes will be made, and it’s not the mistake that we should be anxious about but how we act to fix the mistake that exposes who we are as people.

I would, however, like to take issue with her anti-example in the case of Knewton, which is an online adaptive learning company. Full disclosure: I interviewed with Knewton before I took my current job, and I like the guys who work there. But, I’d add, I like them partly because of the healthy degree of skepticism they take with them to their jobs.

What the blogwriter celandine13 is pointing out, correctly, is that understanding causality is pretty awesome when you can do it. If you can figure out why someone is having trouble learning something, and if you can address that underlying issue, then fixing the consequences of that issue get a ton easier. Agreed, but I have three points to make:

1. First, a non-causal data mining engine such as Knewton will also stumble upon a way to fix the underlying problem by dint of having a ton of data and noting that people who failed a calculus test, say, did much better after having limits explained to them in a certain way. This is much like the spellcheck engine of Google works by keeping track of previous spelling errors, and not by mind reading how people think about spelling wrong.
2. Second, it’s not always easy to find the underlying cause of bad testing performance, even if you’re looking for it directly. I’m not saying it’s fruitless – tutors I know are incredibly good at that – but there’s room for both “causality detectives” and tons of smart data mining in this field.
3. Third, it’s definitely not always easy to address the underlying cause of bad test performance. If you find out that the grammar of Black English affects students’ math test scores, what do you do about it?

Having said all that, I’d like to once more agree with the underlying message that a mistake is a first and foremost a signal rather than a reflection of someone’s internal thought processes. The more we think of mistakes as learning opportunities the faster we learn.

Boston Magazine has a fascinating article on the work of psychologist Lisa Feldman Barrett who has been leading the charge against the idea that we recognise the same facial expression of emotion across the world.

This was first suggested by Paul Ekman whose work suggested that humans can universally recognise six emotions: anger, disgust, fear, happiness, sadness, and surprise.

His research involved showing people from different cultures pictures of faces and asking them to label each expression from a choice of emotional words.

But Barrett has found a simple flaw in the procedure:

She returned to those famous cross-cultural studies that had launched Ekman’s career—and found that they were less than watertight. The problem was the options that Ekman had given his subjects when asking them to identify the emotions shown on the faces they were presented with. Those options, Barrett discovered, had limited the ways in which people allowed themselves to think.

Barrett explained the problem to me this way: “I can break that experiment really easily, just by removing the words. I can just show you a face and ask how this person feels. Or I can show you two faces, two scowling faces, and I can say, ‘Do these people feel the same thing?’ And agreement drops into the toilet.”

The article is on much more than this controversy in cognitive science and also tracks how research on emotion and facial expression is playing an increasing role in law enforcement – with not all of it well supported by evidence.

And if you want links to some of the scientific papers, the always interesting Neuroanthropology blog has more at the bottom of this post.
 

Link to Boston Magazine article ‘About Face’.

Weka is one of the most popular tools for data analysis.  But Weka takes 70 minutes to perform leave-one-out cross-validate using a simple naive bayes classifier on the census income data set, whereas Haskell’s HLearn library only takes 9 seconds.  Weka is 465x slower!

Code and instructions for reproducing these experiments are available on github.

Why is HLearn so much faster?

Well, it turns out that the bayesian classifier has the algebraic structure of a monoid, a group, and a vector space.  HLearn uses a new cross-validation algorithm that can exploit these algebraic structures.  The standard algorithm runs in time , where is the number of “folds” and is the number of data points.  The algebraic algorithms, however, run in time .  In other words, it doesn’t matter how many folds we do, the run time is constant!  And not only are we faster, but we get the exact same answer.  Algebraic cross-validation is not an approximation, it’s just fast.

Here’s some run times for k-fold cross-validation on the census income data set.  Notice that HLearn’s run time is constant as we add more folds.

And when we set k=n, we have leave-one-out cross-validation.  Notice that Weka’s cross-validation has quadratic run time, whereas HLearn has linear run time.

HLearn certainly isn’t going to replace Weka any time soon, but it’s got a number of cool tricks like this going on inside.  If you want to read more, you should check out these two recent papers:

• (ICML13) Algebraic Classifiers: a generic approach to fast cross-validation, online training, and parallel training

• (TFP13) HLearn: a machine learning library for Haskell

I’ll continue to write more about these tricks in future blog posts.

Subscribe to the RSS feed to stay tuned.

Occasionally I see mathematical questions that seem “grammatically incorrect” in some sense.

Example. “Is open or closed?”
Example. “Is a group?”
Example. “What’s the Fourier series of ?”

Here are some sillier examples.

Example. “Is a rectangle prime?”
Example. “Is ?”
Example. “What’s the Fourier series of the empty set?”

What all of these examples have in common is that they are type errors: they are attempts to apply some mathematical process to a kind of mathematical object it was never intended to take as input. If you tried to write a program in some highly mathematical programming language to answer these questions, it (hopefully!) wouldn’t compile.

Mathematical objects are usually not explicitly thought of as having types in the same way that objects in a programming language with a type system has types. Ordinary mathematics is supposed to be formalizable within Zermelo-Fraenkel (ZF) set theory, possibly with the axiom of choice, and in ZF every mathematical object is constructed as a set. In that sense they all have the same type. (In particular, the question “is ?” is perfectly meaningful in ZF! This is one reason not to like ZF as a foundation of mathematics.) However, I think that in practice mathematical objects are implicitly thought of as having types, and that this is a mental habit mathematicians pick up but don’t often talk about.

Instead of thinking in terms of set theory, thinking of mathematical objects as having types allows us to import various useful concepts into mathematics, such as the notions of type safety, typecasting, subtyping, and overloading, that help us make more precise what we mean by a mathematical sentence being “grammatically incorrect.” The rest of this post will be a leisurely discussion of these and other type-based concepts as applied to mathematics in general. There are various categorical ideas here, but for the sake of accessibility we will restrict them to parenthetical remarks.

An informal description of mathematical types

Informally, the type of a mathematical object describes what kind of object it is.

Example. The object has type (natural number).
Example. The object has type (integer).
Example. The object has type (rational number).
Example. The object has type (pair of natural numbers).
Example. The object has type (degree).
Example. The object has type (function from natural numbers to natural numbers).
Example. The object has type (Boolean).

Some less simple examples:

Example. The object has type (group).
Example. The object has type (topological space).
Example. The object has type (function from topological spaces to groups).

Types help you figure out what you can do with a collection of mathematical objects.

Example. You can take two objects of type and add or multiply them. In other words, there are functions .

Example. You can take two objects of type and, in addition to being able to add or multiply them, you can also subtract them. In other words, there are functions .

Example. You can take an object of type and apply a trigonometric function to it. In other words, there are functions .

Example. If are types, you can take an object of type and another object of type and apply the latter to the former. In other words, there is a function .

The type occupies a special place in this theory. There are only two objects of this type, namely and , and functions which output values can be used to check whether a property holds or doesn’t hold.

Example. You can take a natural number and ask whether it is prime. In other words, there is a function .

Example. If is a type, you can take two objects of type and ask whether they are equal. In other words, there is a function

,

which we might also write using the equals sign .

Example. You can take two integers and ask whether the first is greater than or equal to the second. In other words, there is a function

.

As the above examples suggest, there are various ways of combining types to form new types; these are called type constructors.

Example. An object of the product type is a pair of objects of type and respectively.

Example. An object of the sum type is either an object of type or an object of type .

Example. An object of the function type , also sometimes notated , is a function which takes as input objects of type and returns as output objects of type .

(These constructions should look familiar to fans of category theory: they are just the categorical product, categorical coproduct, and categorical exponential in a category of types. In other words, categories of types are bicartesian closed categories.)

Using the basic type constructors above we can build more complicated type constructors. For example, given a type we can form the list type , which is the type of (finite) lists of elements of type , as follows:

(where is a special type called the unit type; its defining feature is that there is exactly one object of unit type, which we will call a point). Heuristically the list type can be written as the “geometric series” .

Notation

Above we used the notation to denote that was an object of type . It makes sense to generalize this notation to arbitrary types, so that denotes that is an object of type . This is a type declaration.

Type safety and type errors

Besides helping you figure out what you can do with a collection of mathematical objects, types also help you figure out what you can’t do. Let’s go through the examples from the beginning of this post with this idea in mind.

Example. “Is open or closed?” The functions and don’t take as input a set or even a topological space; they take as input pairs where is a topological space and is a subspace of (and output whether or not is a closed or open subset of respectively). In other words, they don’t have type but type . The question of whether is open or closed depends on whether it’s being viewed as a subset of itself or, say, .

Example. “Is a group?” The function doesn’t take as input a set; it takes as input pairs where is a set and is a binary operation on (a magma). In other words, it doesn’t have type but type .

Example. “What’s the Fourier series of ?” The function doesn’t take as input a function on ; it takes as input a periodic function on , e.g. with period , or equivalently a function on , and is not periodic with any period. In other words, doesn’t have type but type .

The sillier examples can be analyzed similarly.

Looking for type errors, or type checking, can help you debug a piece of mathematics in the same way that a compiler looks for type errors to debug a piece of code. For example, whenever you see an expression of the form , you can check to see if it even makes sense before checking whether or not it’s true by checking whether and are of the same type. This is a very general version of dimensional analysis.

Type checking is also one way to check your understanding of a mathematical subject new to you. If you can’t say correctly typed sentences about the subject yet (as determined by other people who know the subject), then you haven’t yet understood the types of the main objects or functions in the subject. For example, if you say “fundamental group of a…” you’d better finish that sentence with either a pointed topological space or a path-connected topological space; otherwise, you haven’t understood an important aspect of the definition of the fundamental group, namely the role of basepoints.

Typecasting, subtyping, and overloading

One complication that makes type checking nontrivial is that most mathematical objects are naturally regarded as having more than one type. For example, we said above that has type , but it clearly also has type , and even . One way to describe this situation is that there are typecasting functions

that convert objects between types. If there is a typecasting operator , then we can use objects of type as inputs into functions that expect objects of type provided that we typecast first.

Another way to describe this situation is that some types are subtypes of other types; in other words, instead of being a collection of tyepcasting functions, the above describes a chain of subtype inclusions. Subtyping allows objects to have more than one type simultaneously, rather than to be a particular type at a particular point in some computation.

Subtypes have an interesting relationship to function types. If is a subtype of , then the function type is a subtype of , but if is a subtype of (a function which outputs objects of type also outputs objects of type ), then the function type is not a subtype but a supertype of (a function which accepts inputs of type also accepts inputs of type ). In other words, the construction of function types is covariant in output types but contravariant in input types with respect to subtypes.

(Again, this should be familiar to fans of category theory: this phenomenon reflects the fact that the construction of exponential objects is covariantly functorial in the target object but contravariantly functorial in the source object.)

A third way to describe the situation is that functions in mathematics are overloaded, and this is the description that is perhaps closest to mathematical practice. Overloading refers to the practice of defining functions which have the same name but which take different types of inputs. For example, the addition symbol refers not to one function but to many functions, all of which happen to have the same name, e.g.

and so forth. More generally we might use for the addition operation in any ring, and even more generally for the group operation in any abelian group. We even used it above to denote sum types!

Exponential notation is perhaps even worse. It might refer to functions of any of the following types:

,

or even more generally, might be an element of a group and might be an integer, or might be and might be an element of a topological ring, and we even used exponential notation above to denote function types! See also this math.SE question.

Overloading seems to confuse students, and I think part of the reason is that mathematicians rarely say explicitly that they are doing it. Overloading isn’t so bad when all the different instances of it are at least meaningfully related to each other, but sometimes concepts in mathematics are overloaded for no reason other than historical accident, e.g. words like “normal” and “regular.” This is even more confusing to students: sometimes the use of the same word in two different contexts is because of some meaningful relationship, e.g. normal extensions and normal subgroups, and sometimes it just isn’t. See also this MO question.

Recursive types

In type theory it is possible to define some types recursively, in terms of themselves, rather than directly in terms of other types. A surprising number of important kinds of mathematical objects can be defined this way.

Example. The type of natural numbers can be defined recursively as

;

that is, a natural number is either a point (namely ) or is another natural number (namely its predecessor).

Example. More generally, the list type can be defined recursively as

;

that is, a list of elements of type is either a point or it is an element of type (namely the beginning of the list) together with a list of elements of type (namely the rest of the list). In particular, is just the type of lists of points. Note that this is exactly the relationship one would expect to be satisfied by the “geometric series” .

Example. The type of (rooted, full binary) trees can be recursively defined as

;

that is, a tree is either a point or it is a pair of trees (namely the pair of trees obtained by removing its root).

Example. A version of the type of sets can be recursively defined as

;

that is, a set is a function on sets which returns either true (for the elements it contains) or false (for the elements it does not contain).

Example. The type of (not necessarily finite) combinatorial games can be recursively defined as

;

that is, a game is a pair of functions on games, which we might call and . Strictly speaking, a combinatorial game should really be called a game position, since it describes some particular moment in a game (e.g. the beginning of a game of chess) rather than what is colloquially referred to as a game (e.g. chess). We think of games as being played between two players, Left and Right, and at any game position Left has some possible options of game positions he can move the game to, namely the positions such that returns true, and Right has some possible options of game positions she can move to, namely the positions such that returns true.

Conway noticed that combinatorial games resemble a generalization of both ordinals (which can be described in terms of a set of ordinals) and Dedekind cuts (which are described by a pair of sets of rationals) and used this relationship to define a large class of numbers using games; see On Numbers and Games for details.

The type we defined above, , can also be thought of as the type of impartial games, which are games where the available moves don’t depend on which player is playing.

(For fans of category theory, recursive types are initial algebras relative to a suitable endofunctor of a category of types.)

Functions which take as input a recursive type can also be defined recursively. Examples involving the natural numbers should already be familiar; here are some other examples.

Example. The length of a list can be defined recursively as follows: the length of a point is . If the list is not a point, it consists of an element and a sublist, and then the length of the list is one greater than the length of the sublist. (I would write this down in pseudocode but I don’t know a clean way to display pseudocode on a WordPress site.)

Example. More generally, for any two types there is a function

called map which takes as input a function and returns as output a function defined recursively as follows: if the input list is empty, so is the output list. Otherwise, the input list consists of an object of type together with a sublist, and applies to the object and then to the sublist, and these together constitute the output list. The length of a list is the function obtained by applying to the unique function .

(Fans of category theory will recognize this as essentially the statement that the list constructor is a functor. There are even programming languages like Haskell which explicitly recognize this fact and put it to good use.)

Example. The height of a tree can be defined recursively as follows: the height of a point is . If the tree is not a point, it has two subtrees, and then the height of the tree is one greater than the maximum of the height of its subtrees.

Example. Games which are guaranteed to terminate can be organized into four disjoint classes: a game is

• positive if Left wins no matter who goes first,
• negative if Right wins no matter who goes first,
• zero if the second player to move wins,
• fuzzy if the first player to move wins.

Here “wins” means “wins according to the normal play condition,” where the first player who cannot make a move loses. The above classes correspond to four functions

which can be recursively defined in terms of each other as follows:

• A game is positive iff at least one of Left’s options is positive or zero and all of Right’s options are positive or fuzzy.
• A game is negative iff all of Left’s options are negative or fuzzy and at least one of Right’s options is negative or zero.
• A game is zero iff none of Left’s options are positive or zero and none of Right’s options are negative or zero.
• A game is fuzzy iff at least one of Left’s options is positive or zero and at least one of Right’s options is negative or zero.

(For fans of category theory, recursively defined functions are applications of the universal property of initial algebras.)

Recursive types are another reason to think of mathematics as having a type system: they allow for a much richer notion of recursion than is usually explicitly mentioned in mathematics, where we usually make do with induction (recursion on ) or sometimes transfinite induction (recursion on a modified version of ). Recursive types, by contrast, allow for a general notion of structural induction. Often we can reduce cases of structural induction we need to ordinary induction (e.g. if we want to perform structural induction over trees we can perform ordinary induction on their heights), but structural induction is conceptually cleaner (e.g. we can perform structural induction over trees without singling out a particular function called height).

Conclusion

Although mathematics is not often explicitly described as having a type system in theory, it seems both useful and accurate to describe mathematics as having a type system in practice. This gives us a language for understanding certain kinds of mathematical errors (type errors) as well as understanding certain implicit mathematical practices (overloading). Types also have an interesting mathematical structure and provide a richer language for defining and working with recursively defined mathematical objects than is commonly taught.

BMJ Case Reports has a paper that describes two patients with Parkinson’s disease who experienced hallucinations that transferred onto photos they took to try and prove they were real.

This is ‘Patient 1′ from the case report:

Patient 1 was first evaluated at age 66, having been diagnosed with PD [Parkinson's Disease] at age 58… She complained of daytime and night-time visual hallucinations for the past one year. Most of the time she did not have insight about them. She described seeing three children playing in her neighbour’s yard and a brunette woman sleeping under the covers in one of the beds in her house. She also saw images of different people sitting quietly in her living room. Most of her visual hallucinations subsided in open and brightly lit spaces but were, nevertheless, troublesome. In one instance, she saw a man covered in blood, holding a child and called 911.

Her husband, in an attempt to prove to her that these were hallucinations, took pictures of the neighbour’s yard and the bed in their house. Surprisingly, when shown these photos, the patient continued to identify the same children playing in the yard and the same brunette woman sleeping under the covers. This perception was present every time the patient looked at these photos. Within 6 months of stopping ropinirole and titrating quetiapine to 75 mg every night at bedtime the hallucinations were less severe and shorter in duration, but the patient continued to see them in the photos.

 

Link to locked article in BMJ Case Reports.

Since Einstein's fundamental paper of 1915, gravitation has been synonymous with General Relativitiy, while theories based on Lorentz invariance have been dismissed as formal analogies. Consequently modern cosmology focusses exclusively on Einstein's geometric theory, which is unnecessarily cumbersome in regions of weak fields, while Maxwell's equations are far more tractable.

It is shown here that Maxwell's equations are an {\it artifact} of Hodge theory, geometric in nature, independent of any specific physical mechanisms, and valid for any force field, attractive or repulsive, generated by a material density and current. In particular, Maxwell's equations apply to weak gravitational fields -- those in Minkowski space-time.

Classical analysis and linear partial differential equations therefore have a role to play in modern problems in cosmology. The gravitational field generated by the rotation of a spherical body is computed explicitly using a multipole expansion; and the method is compared with the Lense-Thirring theory of General Relativity. Maxwell's equations for gravitation are a suitable model for the dynamics of galaxies and galaxy clusters, and the problem of dark matter is discussed from the perspective of dynamic anomalies, which are not accounted for by Newtonian mechanics.

We show that a classical fluid mechanical system can violate Bell's inequality because the fluid motion is correlated over large distances.

It shows one half-second of trading in just one stock, boring old Johnson & Johnson, on May 2. The video slows down the trades so that the milliseconds -- thousandths of a second -- tick slowly by, and so that human eyes can comprehend what's happening.

What you see is trading gone haywire, hopelessly beyond the control of any regulators that might want to make sure all of these trades are legitimate. This flood of trading confuses even other machines, creating mismatches in orders that high-speed traders can exploit, millisecond by millisecond.

"These guys are not stealing dollars, they're stealing pennies," says Nanex founder Eric Hunsader, who presented the video at a recent Wired conference. "It's like paper cuts instead of first-degree murder."..

Inside of the one half-second of trading represented by the video, more than 1,200 orders and 215 actual trades occur, Hunsader says. (The colored boxes in the video represent exchanges, and the dots that go flying represent individual orders.)  

I fully understand that all of this is legal and I will readily concede that the process might "enhance liquidity in the market" (I know that when I place an order online at Schwab, it is completed as fast as I can click the "check order status" button).  But... this still scares the shit out of me.

Regulators are desperately trying to keep up. The European Union last year approved a new rule mandating that all trades must exist for at least a half-second, in order to try to minimize the kind of quote-stuffing that frightens and confuses markets. It turns out that half-second is an eternity. 

It is a financial Armageddon just waiting to happen.  Why doesn't anyone have the cojones to limit this activity?

Addendum:  A well-informed reader wrote to me privately re the content of this post, asking to remain anonymous, and offering the following counterpoints, which I thought were valid enough to bring above the fold to the content of the post -

That HuffPo article is kind of silly, what it shows and what it concludes don't follow at all. What the video seems to show is the pretty straightforward implementation of a regulation called RegNMS. If you send an order to one exchange but there's a better price at another one it's required that the exchanges get you that better price. Obviously that means the exchanges are going to have to talk to each other whenever an order comes in that might have a better price somewhere else. All you're seeing is those "hey can you beat this price" messages going out to all the other exchanges. Skip to 2:50 and you can see one or two happen all alone, the crazy visual is just a lot of that happening all at once. 

Notably there's no indication that anything in the video has anything at all to do with HFT. This is the exchanges and only the exchanges trying to clear trades as fast as they can while complying with the rules they operate under. The cacophony seems to be because there are a lot of orders and computers can execute the straightforward (and, again, legally mandated) arithmetic astonishingly quickly. All the underlying orders could very well be boring mutual funds trading with each other without it looking very different.

Which is not to say that there's no nefarious stuff going on in the markets just that this video doesn't show anything like that. I'm happy to be corrected if I've misinterpreted it of course; I'm not an expert in market microstructure.

(Nanex has an older but more detailed page about some of this at http://www.nanex.net/Research/IsNBBOIgnored.html that seems to be addressing a slightly different issue about which I know nothing)

As an aside, one reason your Schwab trades execute immediately may be that they never even make it to the public market. They're either matched inside Schwab by another customer doing the reverse trade or sent to a 'dark' venue that pays Schwab for doing so:

http://www.schwab.com/public/schwab/nn/legal_compliance/important_notices/order_routing.html

This is called 'payment for order flow'. Note that this still falls under RegNMS so Schwab (or the venue where your trade eventually ends up) must do the 'notify all exchanges' dance from the video. You can see a nice breakdown of where actual trading volume happens in the US at Bats' website: http://www.batstrading.com/market_summary/. The NASD and TRF volumes are things like Schwab filling the trade internally without ever reaching an exchange but also trades that happen on 'dark pools', like the ones that Schwab sends to. Note I'm not calling out Schwab in particular here, this is common practice. 

I will be the first to agree that the Huffington Post tends to be sensationalist in hyping mundane material to increase clicks, and while the comments above temper my angst a bit, I remain uncomfortable with the overall concept of HFT.  Re the latter, the reader who wrote to me also provided a link to A High Frequency Trader's Apology, which discusses the concept in some detail.

In ordinal symbolic dynamics, transcripts describe the algebraic relationship between ordinal patterns. Using the concept of transcript, we exploit the mathematical structure of the group of permutations to derive properties and relations among information measures of the symbolic representations of time series. These theoretical results are then applied for the assessment of coupling directionality in dynamical systems, where suitable coupling directionality measures are introduced depending only on transcripts. These novel measures estimate information flow in lower space dimension and reduce to well-established coupling directionality quantifiers when some general conditions are satisfied. Furthermore, by generalizing the definition of transcript to ordinal patterns of different lengths, several of the commonly used information directionality measures can be encompassed within the same framework.

This is a guest post by Josh Snodgrass.

As the Mathbabe noted recently, a lot of companies are collecting a lot of information about you. Thanks to two Firefox add-ons – Collusion (hat tip to Cathy) and NoScript — you can watch the process and even interfere with it to a degree.

Collusion is a beautiful app that creates a network graph of the various companies that have information about your web activity. Here is an example.

 

On this graph, I can see that nytimes.com has sent info on me to 2mdn.net, linkstorm.net, serving-sys.com, nyt.com and doubleclick.net. Who are these guys? All I know is that they know more about me than I know about them.

Doubleclick is particularly well-informed. They have gotten information on me from nytimes.com, yahoo.com and ft.com. You may not be able to see it on the picture but there are faint links between the nodes. Some (few) of the nodes are sites I have visited. Most of the nodes, especially some of the central ones are data collectors such as doubleclick and googleanalytics. They have gotten info from sites I’ve visited.

This graph is pretty sparse because I cleared all of my cookies recently. If I let it go for a week and the graph will be so crowded it won’t all fit on a screen.

Pretty much everyone is sharing info about me (and presumably you, too). And, I do mean everyone. Mathbabe is a dot near the top. Collusion tells me that mathbabe.org has shared info with google.com, wordpress.com, wp.com, 52shadesofgreed.com, youtube.com and quantserve.com. Google has passed the info on to googleusercontent.com and gstatic.com

I can understand why. WordPress and presumably wp.com are hosting her blog. Google is providing search capabilities. 52shadesofgreed has an ad posted (You can still buy the decks but even better, come to Alt-Banking meetings and get one free). Youtube is providing some content. It is all innocent enough in a way but it means my surfing is being tracked even on non-commercial sites.

These are the conveniences of modern life. Try blocking all cookies and you will find it pretty inconvenient to use the internet. It would be nice to be selective about cookies but that seems very hard. All of this is happening even though I’ve told my browser not to allow third-party cookies. If you look at cookie policies, it seems you have two alternatives:

• Block all cookies and the site won’t work very well
• Allow cookies and we will send your info to whomever we choose (within the law, of course).

So, it would be nice if there were a law that constrained what they do. My impression is that we Americans have virtually no protection. Europe is better from what I understand.

I’ve found another add-on called NoScript that is very helpful but also very disturbing. It tells you about JavaScripts that want to run when you visit a site.

I’m trying to access a site and there are scripts waiting to run from:

• Brightcove.com
• Quantserve.com
• Facebook.com
• Po.st Scorecard.com
• Wxug.com
• Admeld.com
• Googleadservices.com
• Legolas-media.com
• Criteo.com
• Crwdcntrl.com

Clearly a lot of those are about tracking me or showing me ads. As with cookies, if you block all the scripts, the site probably won’t function properly. But the great thing about NoScript is that is makes it easy to allow scripts one by one. So, you can allow the ones that look more legitimate until the site works well enough. Also, you can allow them temporarily.

NoScript and Collusion are great. But mostly they are making me more aware of all the tracking that is going on. And they are also making it clear how hard it is to keep your privacy.

This isn’t just on the internet. Years ago, an economist had an idea about having people put boxes on their cars that would track where they went and charge them for driving, particularly in high congestion times and places. The motivation was to reduce travel that causes a lot of pollution while no one is going anywhere. But people ridiculed the idea. Who would let themselves be tracked everywhere they went.

Well, 40 years later, nearly everyone who has a car has an EZ-pass. And, even if you don’t, they will take a picture of your license plate and keep it on file. All in the name of improving traffic flow.

And, if you use credit cards, there are some big companies that have records of your spending.

What to do about this?

I don’t know.

I like conveniences. Keeping your privacy is hard. DuckDuckGo is a search engine that doesn’t track you (another hat tip to Cathy). But their search results are not as good as Google’s.

Google has all these nice tools that are free. Even if you don’t use them, the web sites you visit surely do. And if they do, google is getting information from them, about you.

This experience has made me even more of a fan of Firefox and add-ons available in it. But what else should I use. And, none of these tools is going to be perfect.

What information gets tracked? A lot of privacy policies say they don’t give out identifying information. But how can we tell?

Just keeping on top of what is going on is hard. For example: what are LSOs? They seem to be a kind of “supercookies”. And Better Privacy seems to be an add-on to help with them.

FT.com’s cookie policy tells me that:

“Our emails may contain a single, campaign-unique “web beacon pixel” to tell us whether our emails are opened and verify any clicks through to links or advertisements within the email”

Who knew that a pixel could do so much?

The truth is, I want to see these sites. So I am enabling scripts (some of them, as few as I can). The question is how to make the tradeoff. Figuring that out is time consuming. I’ve got better things to do with my life.

I’m going to go read a book.

Front Comput Neurosci. 2013; 7: 34
Modolo J, Thomas AW, Legros A

Neural mass models are an appropriate framework to study brain activity, combining a high degree of biological realism while being mathematically tractable. These models have been used, with a certain success, to simulate brain electric (electroencephalography, EEG) and metabolic (functional magnetic resonance imaging, fMRI) activity. However, concrete applications of neural mass models have remained limited to date. Motivated by experimental results obtained in humans, we propose in this paper a neural mass model designed to study the interaction between power-line magnetic fields (MFs) (60 Hz in North America) and brain activity. The model includes pyramidal cells; dendrite-projecting, slow GABAergic neurons; soma-projecting, fast GABAergic neurons; and glutamatergic interneurons. A simple phenomenological model of interaction between the induced electric field and neuron membranes is also considered, along with a model of post-synaptic calcium concentration and associated changes in synaptic weights Simulated EEG signals are produced in a simple protocol, both in the absence and presence of a 60 Hz MF. These results are discussed based on results obtained previously in humans. Notably, results highlight that (1) EEG alpha (8-12 Hz) power can be modulated by weak membrane depolarizations induced by the exposure; (2) the level of input noise has a significant impact on EEG power modulation; and (3) the threshold value in MF flux density resulting in a significant effect on the EEG depends on the type of neuronal populations modulated by the MF exposure. Results obtained from the model shed new light on the effects of power-line MFs on brain activity, and will provide guidance in future human experiments. This may represent a valuable contribution to international regulation agencies setting guidelines on MF values to which the general public and workers can be exposed.

In the autumn of 1813, I left my house at Henderson, on the banks of the Ohio River, on my way to Louisville... The air was literally filled with pigeons; the light of noonday was obscured as by an eclipse; the dung fell in spots, not unlike melting flakes of snow, and the continued buzz of wings had a tendency to lull my senses to repose.

Before sunset I reached Louisville, distant from Hardinsburgh fifty-five miles. The pigeons were still passing in undiminished numbers and continued to do so for three days in succession. The people were all in arms. The banks of the Ohio were crowded with men and boys incessantly shooting at the pilgrims, which there flew lower as they passed the river. Multitudes were thus destroyed...

It may not, perhaps, be out of place to attempt an estimate of the number of pigeons contained in one of those mighty flocks, and of the quantity of food daily consumed by its members. The inquiry will tend to show the astonishing bounty of the great Author of Nature in providing for the wants of His creatures. Let us take a column of one mile in breadth, which is far below the average size, and suppose it passing over us without interruption for three hours, at the rate mentioned above of one mile in the minute. This will give us a parallelogram of 180 miles by one, covering 180 square miles. Allowing two pigeons to the square yard, we have 1,115,136,000 pigeons in one flock. As every pigeon daily consumes fully half a pint of food, the quantity necessary for supplying this vast multitude must be 8,712,000 bushels per day...

Let us now, kind reader, inspect their place of nightly rendezvous... The dung lay several inches deep, covering the whole extent of the roosting place, like a bed of snow... As the period of their arrival approached, their foes anxiously prepared to receive them. Some were furnished with iron pots containing sulphur, others with torches of pine knots, many with poles, and the rest with guns.. Suddenly there burst forth a general cry of “Here they come!” The noise which they made, though yet distant, reminded me of a hard gale at sea passing through the rigging of a close-reefed vessel. As the birds arrived and passed over me, I felt a current of air that surprised me. Thousands were soon knocked down by the pole men. The birds continued to pour in. The fires were lighted, and a magnificent as well as wonderful and almost terrifying sight presented itself. The pigeons, arriving by thousands, alighted everywhere, one above another, until solid masses as large as hogsheads were formed on the branches all round. Here and there the perches gave way under the weight with a crash and, falling to the ground, destroyed hundreds of the birds beneath, forcing down the dense groups with which every stick was loaded. It was a scene of uproar and confusion. I found it quite useless to speak, or even to shout to those persons who were nearest to me. Even the reports of the guns were seldom heard, and I was made aware of the firing only by seeing the shooters reloading.

No one dared venture within the line of devastation. The hogs had been penned up in due time, the picking up of the dead and wounded being left for the next morning’s employment. The pigeons were constantly coming, and it was past midnight before I perceived a decrease in the number of those that arrived... Toward the approach of day, the noise in some measure subsided, long before objects were distinguishable, the pigeons began to move off in a direction quite different from that in which they had arrived the evening before, and at sunrise all that were able to fly had disappeared. The howlings of the wolves now reached our ears, and the foxes, lynxes, cougars, bears, raccoons, opossums, and polecats were seen sneaking off, while eagles and hawks of different species, accompanied by a crowd of vultures, came to supplant them and enjoy their share of the spoil.

It was then that the authors of all this devastation began their entry among the dead, the dying, and the mangled. The pigeons were picked up and piled in heaps, until each had as many as he could possibly dispose of, when the hogs were let loose to feed on the remainder.

Persons unacquainted with these birds might naturally conclude that such dreadful havoc would soon put an end to the species. But I have satisfied myself by long observation that nothing but the gradual diminution of our forests can accomplish their decrease.

The latter proved to be a prescient comment.  The last known passenger pigeon died in 1914, as a result of combined predation and habitat loss.

Text excerpted from Audobon's Ornithological Biography, via Lapham's Quarterly.  Image source uncertain, via The Scientist.

Addendum: Reposted from 2013 to add some salient paragraphs from an essay in Harper's Magazine (November 2015) entitled "Rethinking Extinction: Toward a less gloomy environmentalism" -

The birds that most of us eat today are chickens — lots of them — and turkeys, with the occasional duck, quail, or pheasant thrown in. So it is something of a shock to remember that, not so long ago, Americans were happy to eat just about anything with wings. An 1867 inventory of fowl available in the game markets of New York City and Boston featured not only wild turkeys, partridges, and grouse but also robins, great blue herons, sandpipers, meadowlarks, blue jays, and snow buntings.

In season, passenger pigeons were especially plentiful. Alexander Wilson reported they were sometimes eaten for breakfast, lunch, and dinner. The pigeon potpie — sometimes garnished with pigeon feet stuck in the middle — was common fare in colonial America. Passenger pigeons were preserved for out-of-season consumption by being salted, pickled in apple cider, smoked to make jerky, or sealed in casks with molten fat.

According to Schorger, the birds were “a boon to the poor”: in 1754, a half dozen sold in New York for a penny, a sum equivalent to thirty cents today. In times of surplus, they were fed to hogs.

By the middle of the nineteenth century, railroads had connected the cities of the eastern seaboard to the great nesting colonies of the Midwest. Word of the flocks’ locations spread rapidly thanks to another new technology, the telegraph, which allowed professional market hunters, as well as local amateurs, to converge on a site.

The most common way to kill passenger pigeons was to shoot them. Because the birds clustered so densely, no great skill was required to blast them from trees or out of the sky with a shotgun. Nets were widely used as well. Trappers broadcast grain and deployed captive “stool pigeons” to attract the birds, enabling them to snare hundreds at once. Captured pigeons could be killed by crushing their skulls between the thumb and forefinger, though, as Schorger notes, “It was difficult to continue this method without fatigue when many birds were handled.” Some hunters used specially designed pliers to break the birds’ necks. Others used their teeth...

Continued at the link.

Since I already cracked open this door and was not punished by a flood of angry comments, I guess I’ll throw the door wide open and see what happens. This is one of my favorite non-math topics to read about, but I’ve always had a policy of keeping my posts as non-controversial as possible. I guess violating this rule once won’t be that bad. In honor of Easter, I’ll do a literary analysis of the passion narrative as presented in two of the Gospels.

The type of thing I’m going to do gets me in trouble with both the literalists and with fellow atheists. Most atheists believe that everyone should have a fairly good acquaintance with the Bible, and many think you should actually read the thing once. When I express that I think a lot can be gained by studying it as a work of literature, many scoff at that as nonsense. I aim to point out that by doing proper textual analysis you can learn a lot of interesting things.

Everything I write here should be attributed to either Bart Ehrman or Richard Carrier (and their many sources), unless you find some error. I’ve read several books by both of them and seen many debates and lectures they’ve given. I’m going to go almost entirely off of memory (and the text in the Bible itself) and hence will not be able to cite where I learned which interpretations. Both people mentioned are absolutely great Biblical scholars and if you find this post interesting you should look up their books which are excellent. I think essentially everything here should be in Jesus, Interrupted by Ehrman.

If you grew up in some Christian denomination, then you probably have this idea of what “the” passion narrative is. What I mean by this is that even if you’ve read all four accounts as given in the four Gospels, your brain has probably meshed them into one coherent piece. If asked to name any key difference between the four accounts you may have trouble coming up with any or might think they are all basically the same. This is not your fault, because it is probably what you were taught.

The reality is if you closely read any of the four accounts of any given particular aspect of Jesus’ life, you’ll find that there are not only discrepancies, but almost every single detail is in direct contradiction with some other detail of a different account. Ah. But the Christian retorts that even though the details are different the overall main point and content of the story is the same. The purpose of this post is to show you that not only is this false, but it is false for very good reasons.

The reason there are discrepancies even in the main overall points of the stories is that the authors want you to get different things out of it. It may be possible to shove interpretations together in an attempt to make some coherent overall story, but to do so is just wrong-headed. You’ll miss what the authors are trying to tell you by doing this.

Of course, Christians don’t like this idea because implicit in what I’m saying is that these are not meant to be read as historical documents describing what actually happened, but as long-form parables (or as I’ve heard Carrier say, “meta-parables”) with Jesus as the main character whose details are literary devices meant to have theological interpretation. Although this isn’t a well-known idea, it is in fact what the majority of historians (including Christians) believe (see The Homeric Epics and the Gospel of Mark by Dennis MacDonald for this idea fleshed out to a whole book).

Alright, on to the details (I can already tell this is going to be like 2000 words or something, but what else am I going to do while I sit in an airport for two hours). Let’s compare the death of Jesus as portrayed by Mark and Luke (I use the term “by” colloquially to mean “as written in the book of Mark and Luke respectively” because the authors were anonymous). The facts: In Mark the scene is solemn and tragic. Jesus says nothing and is mocked by everyone as he travels to his execution. The only words spoken by Jesus appear right before his death when he says “My God, my God, why have you forsaken me?” He dies. Then the veil at the temple rips. Then the centurion says that he must have been the son of God.

The two literary details to pay attention to are the following. First, Jesus seems to feel totally betrayed through the whole scene. He even asks why God has forsaken him. He doesn’t seem to realize what is happening. Right after his death the veil rips and the centurion tells you how to interpret this detail. The ripping of the veil (or curtain according to some translations) symbolizes that it was a Jewish ritual atonement. It seems clear that the whole scene is a literary construction to make a theological point. Jesus was an innocent lamb being lead unknowingly to a sacrificial atonement ritual.

Mark emphasizes the suffering and the feeling that God has left you. This was written to put the early persecution of Christians into context. It is those who suffer and are meek who will inherit the Earth. This theme is consistent throughout Mark. It is re-emphasized by the fact that he has women find the empty tomb later on as the last thing to happen in the story. It seems deliberate to show that women (who had less status) are the ones to learn of Jesus’ resurrection, and that this is the last event of the book (assuming that Mark actually ends at 16:8 which we won’t get into since books have been written on this debate, e.g. Perspectives on the Ending of Mark: Four Views).

In Luke we get a much different picture. Jesus is not silent while traveling to his execution. He comforts women on the way (tells them not to cry). He asks God to forgive the soldiers nailing him to the cross. He is not excessively mocked in the same way. Jesus tells one of the people being crucified with him that he will see him in heaven shortly. In other words, Jesus seems perfectly aware of his innocence and what is going on and why. He is not concerned that God has left him and seems to be in direct communion with God the whole time.

He even says “Into your hands I commend my spirit.” I repeat, there is no cry of being forsaken in this account at all. I was probably 20 before I realized this. I grew up believing (like most other Christians) that both are said. In reality one is said in one account for a very specific reason and the other is said in another account for a different reason. To combine these into one “historical” account is to completely miss the point of both authors.

In Luke, the temple curtain rips while Jesus is still alive. The centurion does not proclaim him the son of God in this version. Thus our earlier interpretation of this symbol is absolutely impossible here. A common interpretation of this is that the ripped curtain symbolizes God’s rejection of the Jewish system of worship. Sometimes said that the Old Covenant was repealed and Jesus brought the New Covenant.

Corroboration for this evidence is that Luke seems to be referencing Hebrews “By this the Holy Spirit indicates that the way into the holy places is not yet opened as long as the first section is still standing.” Or Jesus’ prophecy earlier in Luke that as long as the temple stands it signifies the continuation of the Old Covenant. Or even later in Acts (written by the same author as Luke) that God left the temple.

If again we take Luke’s account as a long-form parable, the moral does not seem to be that God is there even during great suffering and persecution and even when you feel forsaken by God. The moral in Luke seems to be that during times of suffering you should stay calm and confident of your future in heaven.

Of course the three hours of darkness occur in both, and the symbolism of that is pretty obvious. But remember, we can say with great certainty that this event never happened in history, so we are forced to think of it as a purely symbolic detail. A much more realistic interpretation of what the Gospels are, given this information, is that they are merely literary constructions and not historical accounts to teach the particular author’s theological point-of-view of this new emerging religion.

If you are a Christian and are persuaded of some level of historicity of the Gospels, the takeaway of this post should be the following. At very least, when you hear something in church you should go see what each of the four Gospels say about it separately by carefully comparing the details. Try to figure out the author’s intent rather than blindly taking some mixed interpretation that was presented to you. Question what they tell you and look it up for yourself. You’ll probably be surprised at what you find, and at very least you’ll find much richer theological interpretations of the authors.

Now to tie this back to the beginning. When I point out that almost every detail provided in the New Testament is contradicted somewhere else, the most common pushback I get is that “these are minor details, but the Bible is consistent on its main points and overall story it tells.” I guess you can decide that for yourself, but I’ll leave you with the this question. You could probably extract three major theological questions from the above analysis that get answered differently, but it seems to me that the most important theological question in all of Christianity is “Why did Jesus die?” If you read Mark you get one answer. If you read Luke you get a different answer. Should this be consider a minor detail?

The [British]government is considering whether to propose legal changes that would allow radical new treatments for families at risk of incurable genetic diseases that involve the creation of so-called "three-person embryos".

A national consultation released on Wednesday by the UK's fertility watchdog found public support for techniques that involve introducing DNA from a third person to embryos which could prevent mothers from passing on devastating diseases, such as muscular dystrophy, to their children.

If ministers and MPs give the procedures the green light, Britain would become the first country to offer treatments that lead to children being born with DNA from three people: their parents and a woman donor. The amount of DNA from the donor is tiny compared with the parents...

Scientists have developed two techniques to prevent faulty mitochondria being passed on to children. Known as maternal spindle transfer and pronuclear transfer, they both involve transferring the genetic material from the parents into an egg donated by a healthy woman.

The treatment is controversial on several grounds, not least that the genetic modifications in the embryo pass down to all future generations. The techniques have never been tried in humans, but have worked in animal studies.

Further details at The Guardian.

Headache. Image source: Mupso.

In an opinion piece from December, Brian Cox and Robin Ince argued that opinion must be separated from science when it comes to policy decisions:

“[T]here must be a place where science stops and politics begins, and this border is an extremely complex and uncomfortable one. Science can’t tell us what to do… The choice of policy response itself is not a purely scientific question, however, because it necessarily has moral, geopolitical and economic components.”

I used to say the same, that politics unfortunately mixes up scientific questions with unscientific ones, and that informed decision making requires us to first distinguish these. But then I went down a windy road trying to understand where science ends and where decision making begins. This eventually lead to my paper on the measurement of happiness. It also lead me to the conviction that the “extremely complex and uncomfortable border” doesn’t exist. Cox and Ince come to the right conclusion, but for the wrong reasons.

What is and what isn’t in the realm of science, and what is the role of science is in our political system are questions I care about deeply. And so I could not avoid noticing Sean Carroll and Lubos Motl recently discussed whether morals can be reduced to science. They come down, in rare agreement, on the side of “no”. It’s a variant of the “boundary” Cox and Ince touched on, so let us see what they had to say.

Sean and Lubos start by elaborating on what is and what isn’t a scientific statement. A scientific statement, they say, is one that could be false and whose truth value could at least in principle be empirically evaluated. The problem is then that the statement that morals can’t be reduced to science itself isn’t scientific. It isn’t because a definition for “moral” is lacking. Then, all answers to this question are just opinion so why bother with it? Lubos alludes to this by saying that whenever one could answer the question one way or the other somebody might just change the definition of moral

“Imagine that you find some quantity M encoded in the equations of M(orality)-theory in the future and you will claim that it measures morality… The problem is that even with this nice and well-defined formula, one may always legitimately refuse such a measure of morality and choose a completely different one.”

This lack of proper definition is an example for what I complained about in my recent post, that many philosophical questions are a waste of time if one doesn’t know what one is talking about to begin with. So let’s not debate the meaning of words and instead identify the real issue behind it.

What people really want to know is where science leaves them the freedom to make decisions. That’s why they are looking for a border between scientific and unscientific questions, the former can be answered by science, the latter presumably can only be answered by humans. In other words, they’re asking for their space to exercise free will.

Free will is an illusion that people hold on to quite stubbornly and that they protect vehemently, so the debate about the unscientificness of morals shouldn’t come as a surprise. The thought that science might tell people what they should or shouldn’t do is a great threat to free will, one that gets addressed in a forward defense. But that’s a misunderstanding. Science has never and will never tell anybody what should or shouldn’t be done because “should” is another one of these ill-defined words. “Should” implicitly necessitates a goal or a purpose.

“Science can’t tell us what to do”, as Cox and Ince write - correctly. But science can in principle tell us what we do. To understand how let’s have a look at what people mean when they refer to “morals” or “values”.

Humans are self-aware complex systems that have to process a lot of information to make informed decisions. Human self-awareness however is limited. We are not normally aware how the detailed processes of our thoughts proceed. In fact recent research in neuroscience seems to show that what we think of as “I” is primarily an aggregating mechanism of various deeper level systems whose detailed procedures the “I” does not normally take note of.

Thus, “we” don’t consciously know the details of how we make decisions. Moreover, a central element of human decision making is ignorance and oversimplification. The one thing that the human brain is really good at is energy efficiency. Which is why the default is to avoid thinking if unnecessary.

What we do instead of monitoring all that information from the input that we receive is learning to construct models of behavior that make use of simplified patterns and categories. Then we explain our decisions and those of others in terms of these simplified patterns. You chose this job because independence is important to you. You think polygamy is immoral and should be punished. These are rough summaries of longwinded thought processes which made use of experience, evolutionary traits, and random noise. They classify decisions in values like “independence” or morals like “faithfulness.”

Morals and values are thus just categories that people use to classify and explain the way they make decisions. Over time, using these simplified models, the higher level “I” system becomes good at predicting what will happen, and interprets this as an exercise of “free will.”

That having been said, if you believe in reductionism, morals and values are just emergent patterns in highly complex systems. It is clearly impractical and anyway presently impossible, but in principle one could define morals in this way. Imagine you’d do this. Now you have a definition for moral. An individual one, one that depends on cultural history as well as genetic ancestry. Here you have it. These are your morals.

You might then go and say that’s not what you mean with moral. And that would be fine with me because I don’t want to argue about words, so just call these emerging patterns something else. The point is that they’re what people make use of when they make decisions, and recall that this is the question we really want to address: What decisions are humans free to make because they’re allegedly unscientific?

If you have such a definition for morals then would science then tell you what you should do? No. It would in the best case simply tell you what you do. The best case being one in which scientists would be able to construct a complete model for human behavior. Depending on your attitude you might call that the worst case.

But while in principle possible, it is questionable that such a model is feasible to construct at all. It seems plausible to me that the process of thought is irreducible in the sense that if you tried to predict it you’d have to create an almost perfect copy of the original system and watch it in real time, in which case you’d just duplicate rather than predict decisions.

In other words, while the “border” between scientific and unscientific questions does not exist in principle, it does exist in practice. And it’s located where our ability to model complex systems ends, an end that might shift somewhat in the future but quite possibly will never entirely recede. The best way we presently know to find out what decisions humans make is to ask them. The best way we know to find out what the global climate does is not to ask humans but a computer model.

What does this have to do with happiness? Well, striving to achieve happiness is a human universal, so much so that you might want to raise the maximization of well-being of conscious beings to a universal goal. Having defined such a goal it would fill in the blank of the “purpose” and the “should” that was previously missing, or at least it seems so.

The problem is however that happiness is a byeffect of natural selection, it’s a simplified response to behavior that has in the past been beneficial for reproduction. Elevating happiness to an end unto itself is a circular definition of purpose, it’s fundamentally meaningless. Which is why, in my paper I argued we should forget about trying to define happiness and its maximization as a proxy to understand human behavior. Instead we should look for a properly defined quantity that has predictive power to describe the evolution of our economic, politic, and social systems, and the suggestion I made was maximizing the number of possible decisions that we (think we can) make. Which might or might not be correct. A scientific question that’s waiting to be answered.

Summary: Ill-defined questions are unscientific, but uninterestingly so. Once a question is well-defined science is in principle able to answer it, but not necessarily in practice. A scientific definition for morals might exist, but quite plausibly we will never be able to construct it. And even if we could, it wouldn’t tell us what should be done, but simply what is done. Opinion begins where our ability to model complex systems ends. This border will inevitably shift over time and it’s this “shift” that makes it uncomfortable. And no, I don’t believe in free will.

UPDATE: A law-enforcement source and a fan say the victim in the attack was One Armed Push-Up Guy, well known outside Boylston Street bars and in the Fenway for doing one-armed pushups for a fee, like this:

Boston Police report that around 2 a.m., a guy standing in the middle of Boylston Street outside Whiskey's, 885 Boylston St., yelled "Fuck Boston and fuck your businesses!" and then turned around and punched a nearby man, who fell unconscious into the street:

Witnesses reported a cab traveling inbound on Boylston Street ran over the victim while he was unconscious in the road and dragged him under the vehicle.

The screaming puncher then ran inbound on Boylston and disappeared down Fairfield Street, police say. He's described as 30 to 32, 5'10" and clad in a dark jacket with a fur-lined hood.

The cab driver told police he did not realize he'd hit somebody and stopped only when bystanders started motioning to him. The victim was taken to Mass. General and is expected to survive, police say.

If you know the guy, contact D-4 detectives at 617-343-5619 or the anonymous tip line by calling 800-494-TIPS or texting TIP to CRIME (27463).

It is widely believed that, in cortical pyramidal cells, action potentials (APs) initiate in the distal portion of axon initial segment (AIS) because that is where Na+ channel density is highest. To investigate the relationship between the density of Na+ channels and the spatiotemporal pattern of AP initiation, we simultaneously...

The image comes from a Reddit post, with this explanation:

Was walking on the local trail today, going through the forest part. When suddenly I feel a very sharp pain in my arm, like a shot. Look down and there is a long dart, from maybe a blow gun of sorts? I don't know but scared the shit out of me, I looked and could see no one.

A simple Google Image search yields a wide array of photos of this weaponry.  The Reddit thread focuses on potential health consequences (probably minimal other than at the site of entry) and the range (up to about 100 feet, if you want to locate and try to wreak retribution on the malefactor).

Guest post by Bruce Bartlett

We know about Category Theory for Mathematicians, we’ve all read Category Theory for Physicists, and we also know about Category Theory for Computer Scientists, and we’ve even seen the videos.

But how about Category Theory for Scientists? I spotted this on the arXiv listings.

David Spivak, Category Theory for Scientists.

Abstract: There are many books designed to introduce category theory to either a mathematical audience or a computer science audience. In this book, our audience is the broader scientific community. We attempt to show that category theory can be applied throughout the sciences as a framework for modeling phenomena and communicating results. In order to target the scientific audience, this book is example-based rather than proof-based. For example, monoids are framed in terms of agents acting on objects, sheaves are introduced with primary examples coming from geography, and colored operads are discussed in terms of their ability to model self-similarity.

I’m afraid this little post is just a shout-out as I’ve only hurriedly browsed through the pages.

Towards the end of the book he gets to sheaves; he is certainly an expert on these as his PhD thesis was on derived smooth manifolds). His motivating example is stitching together pictures of the night sky, which I thought was really cool:

Paging through, I see the Yoneda lemma only gets a small paragraph, with a reference to Mac Lane. I’m kind of sad about that, since I do regard it as the fundamental theorem of category theory. Too bad.

If the world is just, only the guilty are tortured. So believers in a just world are more likely to think that the people who are tortured are guilty. Perhaps especially so if they experience the torture closely and so feel a greater need to overcome cognitive dissonance. On the other hand, those farther away from the experience of torture may feel less need to justify it and they may be more likely to identify the tortured as victims. The theory of moral typecasting suggests that victims are also more likely to be seen as innocents (a la Jesus).

The theory is tested in a lab setting by Gray and Wegner. Experimental subjects are told that “Carol”, really a confederate, may have lied about a dice roll and that stress often encourages people to admit guilt. Subjects then listen to a torture session as Carol’s hand is plunged into a bucket of ice water for 80s. Subjects are then asked how likely is it that the torture victim was lying (1 to 5 with 5 being extremely likely). There are two intervention variables: 1) some of the subjects meet the torture victim before she is tortured, this is the close condition and some do not (distance condition) and 2) in some torture sessions the victim evinces pain (pain) and in others not (no pain). The key figure is shown below:

The most striking result is that in the close condition, the evincing of pain was associated with an increased judgment of guilt, consistent with torture causing cognitive dissonance which is relieved by a judgment of guilt (restoring the just world). But in the distance condition, the evincing of pain was associated with a decreased judgement of guilt, consistent with pain increasing the identification of the tortured as a victim and therefore innocent (a la moral typecasting).

Closeness in the experiment was reasonably literal but may also be interpreted in terms of identification with the torturer. If the church is doing the torturing then the especially religious may be more likely to think the tortured are guilty. If the state is doing the torturing then the especially patriotic (close to their country) may be more likely to think that the tortured/killed/jailed/abused are guilty. That part is fairly obvious but note the second less obvious implication–the worse the victim is treated the more the religious/patriotic will believe the victim is guilty.

The theory has interesting lessons for entrepreneurs of social change. Suppose you want to change a policy such as prisoner abuse (e.g. Abu Ghraib) or no-knock police raids or the war on drugs or even tax policy. Convincing people that the abuse is grave may increase their belief that the victim is guilty. Instead, you want to do one of two things. Among the patriotic you may want to sell the problem as a minor problem that We Can Fix – making them feel good about both the we and the fixing. Or, you may want to create distance – The problem is bad and THEY are the cause. People in the North, for example, became more concerned about slavery once the US became us and them.

I think research in moral reasoning is important because understanding why good people do evil things is more important than understanding why evil people do evil things.

Protest with #OWS Alternative Banking Group

I’m writing to invite you to a protest against mega-bank HSBC at noon on Valentine’s Day (Thursday) starting on the steps of the New York Public Library at 42nd and 5th. Details are here but it’s the big green box on the map on the Fifth Avenue side:

Why are we protesting?

Like you, I’m sure, I’d like nothing more than to stop worrying about shit that goes on in our country’s banks.

We have better things to do with out time than to get annoyed over enormous bonuses being given to idiots for their repeated failures. We’re frankly exhausted from the outrage.

I mean, the average person doesn’t have a job where they get an $11 million bonus instead of a $22 million dollar bonus when they royally screw up. Outside the surreal realm of international banking, the normal response to screw-ups on that level is to get fired.

You might expect a company that has been caught criminally screwing minorities out of fair contracts might be at risk of being closed down, but in this day and age you’d know that big banks, or TIBACO (too interconnected, big, and complex to oversee) institutions, as we in Alt Banking like to call them, are immune to such action.

There’s a clear evolving standard of treatment in the banking sector when it comes to criminal activity:

• the powers that be (SEC, DOJ, etc.) make a huge production over the severity of the fine,
• which is large in dollar amounts but
• usually represents about 10% of the overall profit the given banks made during their exploit.
• Nobody ever goes to jail, and
• the shareholders pay the fine, not the perpetrators.
• The perps get somewhat diminished bonuses. At worst.

The bottomline: we have an entire class of citizens that are immune to the laws because they are considered too important to our financial stability.

But why HSBC?

HSBC is a perfect example of this. An outrageous example.

HSBC didn’t get a bailout in 2008 like many other banks, even though they were ranked #2 in subprime mortgage lending. But that’s not because they didn’t lose money – in fact they lost $6 billion but somehow kept afloat.

And now we know why.

Namely, they were money-laundering, earning asstons by  facilitating drugs and terrorism. This was blood money, make no mistake, and it went directly into the pockets of HSBC bankers in the form of bonuses.

When this years-long criminal mafia activity was discovered, nothing much happened beyond a fine, as per usual. Well, to be honest, they were fined $1.9 billion dollars, which is a lot of money, but is only 5 weeks of earnings for the mammoth institution – depending on the way you look at it, HSBC is the 2nd largest bank in the world.

Too big to jail

And that’s when “Too big to fail” became “Too big to jail.” Even the New York Times was outraged. From their editorial page:

Federal and state authorities have chosen not to indict HSBC, the London-based bank, on charges of vast and prolonged money laundering, for fear that criminal prosecution would topple the bank and, in the process, endanger the financial system. They also have not charged any top HSBC banker in the case, though it boggles the mind that a bank could launder money as HSBC did without anyone in a position of authority making culpable decisions.

Clearly, the government has bought into the notion that too big to fail is too big to jail. When prosecutors choose not to prosecute to the full extent of the law in a case as egregious as this, the law itself is diminished. The deterrence that comes from the threat of criminal prosecution is weakened, if not lost.

National Threat

You may recall that there was an extensive FBI investigation of OWS before Zuccotti Park was even occupied.

Ironic? As the Village Voice said, “apparently non-violent demonstration against corrupt banking is subject to more criminal scrutiny than actual corrupt banking.”

Question for you: which is the bigger national security threat, OWS or HSBC?

We demand

HSBC needs its license revoked, and there need to be prosecutions. Those who are guilty need to be punished or else we have an official invitation to criminal acts by bankers. We simply can’t live in a country which rewards this kind of behavior.

Mind you, this isn’t just about HSBC. This is about all the megabanks. Citi or BoA are exempt from prosecution, too. Our message needs to be “break up the megabanks”.

I’ll end with what Matt Taibbi had to say about the HSBC settlement:

On the other hand, if you are an important person, and you work for a big international bank, you won’t be prosecuted even if you launder nine billion dollars. Even if you actively collude with the people at the very top of the international narcotics trade, your punishment will be far smaller than that of the person at the very bottom of the world drug pyramid. You will be treated with more deference and sympathy than a junkie passing out on a subway car in Manhattan (using two seats of a subway car is a common prosecutable offense in this city). An international drug trafficker is a criminal and usually a murderer; the drug addict walking the street is one of his victims. But thanks to Breuer, we’re now in the business, officially, of jailing the victims and enabling the criminals.

Join us on Valentine’s Day at noon on the steps of the New York Public Library and help us Occupy HSBC. Please redistribute widely!

Merriam-Webster's Word of the Day for February 05, 2013 is:

sangfroid • \SAHNG-FRWAH\  • noun
: self-possession or imperturbability especially under strain

Examples:
The lecturer's sangfroid never faltered, even in the face of some tough questions from the audience.

"Daniel Craig portrays a vulnerability far removed from the glib sangfroid of his celluloid predecessors and has retired to an exotic bolt-hole after he is assumed to have died during a botched operation." — From a movie review by Des O' Neill in the Irish Times, January 2, 2013

Did you know?
If you're a lizard, "cold-blooded" means your body temperature is strongly influenced by your environment. If you're an English-speaking human, it means you are callous and unfeeling. If you're a French speaker, it means that you're calm, cool, and collected in stressful situations. By the mid-1700s, English speakers had already been using "cold-blooded" for more than a century, but they must have liked the more positive spin the French put on having "cold blood" because they borrowed the French "sang-froid" (literally, "cold blood") for someone who is imperturbable under strain. The French term, by the way, developed from the Latin words "sanguis" ("blood") and "frigidus" ("cold").

The new documentary, "Fire in the Blood," examines how millions have died from AIDS because big pharmaceutical companies and the United States have refused to allow developing nations to import life-saving generic drugs. The problem continues today as the World Trade Organization continues to block the importation of generic drugs in many countries because of a trade deal known as the TRIPS Agreement. We’re joined by the film’s director, Dylan Mohan Gray, and Ugandan AIDS doctor Peter Mugyenyi, who was arrested for trying to import generic drugs, and is recognized as one of the world’s foremost specialists and researchers in the field of HIV/AIDS. [includes rush transcript]

 The categorical distribution is the main distribution for handling discrete data. I like to think of it as a histogram.  For example, let’s say Simon has a bag full of marbles.  There are four “categories” of marbles—red, green, blue, and white.  Now, if Simon reaches into the bag and randomly selects a marble, what’s the probability it will be green?  We would use the categorical distribution to find out.

In this article, we’ll go over the math behind the categorical distribution, the algebraic structure of the distribution, and how to manipulate it within Haskell’s HLearn library.  We’ll also see some examples of how this focus on algebra makes HLearn’s interface more powerful than other common statistical packages.  Everything that we’re going to see is in a certain sense very “obvious” to a statistician, but this algebraic framework also makes it convenient.  And since programmers are inherently lazy, this is a Very Good Thing.

Before delving into the “cool stuff,” we have to look at some of the mechanics of the HLearn library.

Preliminaries

The HLearn-distributions package contains all the functions we need to manipulate categorical distributions. Let’s install it:

$ cabal install HLearn-distributions

We import our libraries:

>import Control.DeepSeq
>import HLearn.Algebra
>import HLearn.Gnuplot.Distributions
>import HLearn.Models.Distributions

We create a data type for Simon’s marbles:

>data Marble = Red | Green | Blue | White
>    deriving (Read,Show,Eq,Ord)

The easiest way to represent Simon’s bag of marbles is with a list:

>simonBag :: [Marble]
>simonBag = [Red, Red, Red, Green, Blue, Green, Red, Blue, Green, Green, Red, Red, Blue, Red, Red, Red, White]

And now we’re ready to train a categorical distribution of the marbles in Simon’s bag:

>simonDist = train simonBag :: Categorical Marble Double

We can load up ghci and plot the distribution with the conveniently named function plotDistribution:

ghci> plotDistribution (plotFile "simonDist") simonDist

This gives us a histogram of probabilities:

In the HLearn library, every statistical model is generated from data using either train or train’.  Because these functions are overloaded, we must specify the type of simonDist so that the compiler knows which model to generate. Categorical takes two parameters. The first is the type of the discrete data (Marble). The second is the type of the probability (Double). We could easily create Categorical distributions with different types depending on the requirements for our application. For example:

>stringDist = train (map show simonBag) :: Categorical String Float

This is the first “cool thing” about Categorical:  We can make distributions over any user-defined type.  This makes programming with probabilities easier, more intuitive, and more convenient.  Most other statistical libraries would require you to assign numbers corresponding to each color of marble, and then create a distribution over those numbers.

Now that we have a distribution, we can find some probabilities. If Simon pulls a marble from the bag, what’s the probability that it would Red?

We can use the pdf function to do this calculation for us:

ghci> pdf simonDist Red
0.5626
ghci> pdf simonDist Blue
0.1876
ghci> pdf simonDist Green
0.1876
ghci> pdf simonDist White
6.26e-2

If we sum all the probabilities, as expected we would get 1:

ghci> sum $ map (pdf simonDist) [Red,Green,Blue,White]
1.0

Due to rounding errors, you may not always get 1. If you absolutely, positively, have to avoid rounding errors, you should use Rational probabilities:

>simonDistRational = train simonBag :: Categorical Marble Rational

Rationals are slower, but won’t be subject to floating point errors.

This is just about all the functionality you would get in a “normal” stats package like R or NumPy. But using Haskell’s nice support for algebra, we can get some extra cool features.

Semigroup

First, let’s talk about semigroups. A semigroup is any data structure that has a binary operation (<>) that joins two of those data structures together. The categorical distribution is a semigroup.

Don wants to play marbles with Simon, and he has his own bag. Don’s bag contains only red and blue marbles:

>donBag = [Red,Blue,Red,Blue,Red,Blue,Blue,Red,Blue,Blue]

We can train a categorical distribution on Don’s bag in the same way we did earlier:

>donDist = train donBag :: Categorical Marble Double

In order to play marbles together, Don and Simon will have to add their bags together.

>bothBag = simonBag ++ donBag

Now, we have two options for training our distribution. First is the naive way, we can train the distribution directly on the combined bag:

>bothDist = train bothBag :: Categorical Marble Double

This is the way we would have to approach this problem in most statistical libraries. But with HLearn, we have a more efficient alternative. We can combine the trained distributions using the semigroup operation:

>bothDist' = simonDist <> donDist

Under the hood, the categorical distribution stores the number of times each possibility occurred in the training data.  The <> operator just adds the corresponding counts from each distribution together:

This method is more efficient because it avoids repeating work we’ve already done. Categorical’s semigroup operation runs in time O(1), so no matter how big the bags are, we can calculate the distribution very quickly. The naive method, in contrast, requires time O(n). If our bags had millions or billions of marbles inside them, this would be a considerable savings!

We get another cool performance trick “for free” based on the fact that Categorical is a semigroup: The function train can be automatically parallelized using the higher order function parallel. I won’t go into the details about how this works, but here’s how you do it in practice.

First, we must show the compiler how to resolve the Marble data type down to “normal form.” This basically means we must show the compiler how to fully compute the data type. (We only have to do this because Marble is a type we created.  If we were using a built in type, like a String, we could skip this step.) This is fairly easy for a type as simple as Marble:

>instance NFData Marble where
>    rnf Red   = ()
>    rnf Blue  = ()
>    rnf Green = ()
>    rnf White = ()

Then, we can perform the parallel computation by:

>simonDist_par = parallel train simonBag :: Categorical Marble Double

Other languages require a programmer to manually create parallel versions of their functions. But in Haskell with the HLearn library, we get these parallel versions for free! All we have to do is ask for it!

Monoid

A monoid is a semigroup with an empty element, which is called mempty in Haskell. It obeys the law that:

M <> mempty == mempty <> M == M

And it is easy to show that Categorical is also a monoid. We get this empty element by training on an empty data set:

mempty = train ([] :: [Marble]) :: Categorical Marble Double

The HomTrainer type class requires that all its instances also be instances of Monoid. This lets the compiler automatically derive “online trainers” for us. An online trainer can add new data points to our statistical model without retraining it from scratch.

For example, we could use the function add1dp (stands for: add one data point) to add another white marble into Simon’s bag:

>simonDistWhite = add1dp simonDist White

This also gives us another approach for our earlier problem of combining Simon and Don’s bags. We could use the function addBatch:

>bothDist'' = addBatch simonDist donBag

Because Categorical is a monoid, we maintain the property that:

bothDist == bothDist' == bothDist''

Again, statisticians have always known that you could add new points into a categorical distribution without training from scratch.  The cool thing here is that the compiler is deriving all of these functions for us, and it’s giving us a consistent interface for use with different data structures.  All we had to do to get these benefits was tell the compiler that Categorical is a monoid.  This makes designing and programming libraries much easier, quicker, and less error prone.

Group

A group is a monoid with the additional property that all elements have an inverse. This lets us perform subtraction on groups.  And Categorical is a group.

Ed wants to play marbles too, but he doesn’t have any of his own. So Simon offers to give Ed some of from his own bag. He gives Ed one of each color:

>edBag = [Red,Green,Blue,White]

Now, if Simon draws a marble from his bag, what’s the probability it will be blue?

To answer this question without algebra, we’d have to go back to the original data set, remove the marbles Simon gave Ed, then retrain the distribution. This is awkward and computationally expensive. But if we take advantage of Categorical’s group structure, we can just subtract directly from the distribution itself. This makes more sense intuitively and is easier computationally.

>simonDist2 = subBatch simonDist edBag

This is a shorthand notation for using the group operations directly:

>edDist = train edBag :: Categorical Marble Double
>simonDist2' = simonDist <> (inverse edDist)

The way the inverse operation works is it multiplies the counts for each category by -1. In picture form, this flips the distribution upside down:

Then, adding an upside down distribution to a normal one is just subtracting the histogram columns and renormalizing:

Notice that the green bar in edDist looks really big—much bigger than the green bar in simonDist.  But when we subtract it away from simonDist, we still have some green marbles left over in simonDist2.  This is because the histogram is only showing the probability of a green marble, and not the actual number of marbles.

Finally, there’s one more crazy trick we can perform with the Categorical group.  It’s perfectly okay to have both positive and negative marbles in the same distribution.  For example:

ghci> plotDistribution (plotFile "mixedDist") (edDist <> (inverse donDist))

results in:

Most statisticians would probably say that these upside down Categoricals are not “real distributions.” But at the very least, they are a convenient mathematical trick that makes working with distributions much more pleasant.

Module

Finally, an R-Module is a group with two additional properties. First, it is abelian. That means <> is commutative. So, for all a, b:

a <> b == b <> a

Second, the data type supports multiplication by any element in the ring R. In Haskell, you can think of a ring as any member of the Num type class.

How is this useful?  It let’s “retrain” our distribution on the data points it has already seen.  Back to the example…

Well, Ed—being the clever guy that he is—recently developed a marble copying machine. That’s right! You just stick some marbles in on one end, and on the other end out pop 10 exact duplicates. Ed’s not just clever, but pretty nice too. He duplicates his new marbles and gives all of them back to Simon. What’s Simon’s new distribution look like?

Again, the naive way to answer this question would be to retrain from scratch:

>duplicateBag = simonBag ++ (concat $ replicate 10 edBag)
>duplicateDist = train duplicateBag :: Categorical Marble Double

Slightly better is to take advantage of the Semigroup property, and just apply that over and over again:

>duplicateDist' = simonDist2 <> (foldl1 (<>) $ replicate 10 edDist)

But even better is to take advantage of the fact that Categorical is a module and the (.*) operator:

>duplicateDist'' = simonDist2 <> 10 .* edDist

In picture form:

Also notice that without the scalar multiplication, we would get back our original distribution:

Another way to think about the module’s scalar multiplication is that it allows us to weight our distributions.

Ed just realized that he still needs a marble, and has decided to take one.  Someone has left their Marble bag sitting nearby, but he’s not sure whose it is.  He thinks that Simon is more forgetful than Don is, so he assigns a 60% probability that the bag is Simon’s and a 40% probability that it is Don’s.  When he takes a marble, what’s the probability that it is red?

We create a weighted distribution using module multiplication:

>weightedDist = 0.6 .* simonDist <> 0.4 .* donDist

Then in ghci:

ghci> pdf weightedDist Red
0.4929577464788732

We can also train directly on weighted data:

>weightedDataDist = train [(0.4,Red),(0.5,Green),(0.2,Green),(3.7,White)] :: Categorical Marble Double

which gives us:

The Takeaway and next posts

Talking about the categorical distribution in algebraic terms let’s us do some cool new stuff with our distributions that we can’t easily do in other libraries.  None of this is statistically ground breaking. The cool thing is that algebra just makes everything so convenient to work with.

I think I’ll do another post on some cool tricks with the kernel density estimator that are not possible at all in other libraries, then do a post about the category (formal category-theoretic sense) of statistical training methods.  At that point, we’ll be ready to jump into machine learning tasks.  Depending on my mood we might take a pit stop to discuss the computational aspects of free groups and modules and how these relate to machine learning applications.

Sign up for the RSS feed to stay tuned!

A group is an abstraction for things that can be «added» and «subtracted» (in some sense).

Most of the groups that programmers deal with are numeric, although some interesting groups arise in graphics and cryptography.

If we only require that things can be added, but not subtracted, we get semigroups and monoids (the difference is that a monoid also has a «zero» element). These are plentiful, with applications ranging from diagrams to regular expressions.

The reason is, of course, that addition is very natural — we can consider any way to combine objects together an addition as soon as it is associative — while subtraction, the inverse of addition, often doesn’t make sense.

Dates

But there are also things that can be subtracted, but not added. The simplest example is dates. You can subtract two dates to get the time interval between them, but you cannot add two dates. It doesn’t make sense to ask what today plus tomorrow is. You could try to add the numeric values corresponding to the dates and convert the answer back to a date, but the result would depend on whether you count from the start of the Common Era, the Unix epoch, or something else.

While adding two dates is not possible, it is possible to add a time interval to a date («five days from today»). This suggests that we should not confound dates and time intervals — they are different types of values.

For example, in the Haskell time package, there are types UTCTime and NominalDiffTime, and the following functions:

addUTCTime :: NominalDiffTime -> UTCTime -> UTCTime
diffUTCTime :: UTCTime -> UTCTime -> NominalDiffTime

Time intervals can also be added and subtracted, as witnessed by the Num NominalDiffTime instance.

Ask mathematics

What is the mathematical notion that characterizes dates? It should involve two sets: the set V of values and the set D of value differences.

The set D must form an additive group, so that we can add and subtract the differences. (The word «additive» means that we assume that the group operation, identity element and inverse operation are called +, 0 and − respectively. In group theory it is more common to call them *, 1 and ⁻¹.)

Thus we have the “zero” difference 0∈D and operations +:D×D→D, −:D→D which satisfy the usual group laws:

+/0 (identity element)

0+x=x+0=x

+/+ (associativity)

(x+y)+z=x+(y+z)

+/− (inverse element)

x+(−x)=(−x)+x=0

(We can write x−y as a shorthand for x+(−y).)

We do not require + to be commutative to preserve generality, even though it is commutative for our current example.

We need to be able to add a difference to a value. Thus, we need a function add:D×V→V. We would expect it to play well with the group structure of D:

add/0

add(0,v)=v

add/+

add(x+y,v)=add(x,add(y,v))

An alternative view on these laws is that the curried version of add, curry(add):D→(V→V), is a group homomorphism from D to the group of bijections on V.

The mathematical abstraction corresponding to our add function is an action of group D on the set V. But it doesn’t yet capture the fact that we can subtract two values to get a difference. For that we need a function diff:V×V→D inverse to add:

diff/add

diff(add(x,v),v)=x

add/diff

add(diff(u,v),v)=u

If such a function exists, the set V is called a torsor over the group D.

Thus, it is the mathematical notion of torsor that characterizes subtractable values.

Conal Elliott defines torsors in Haskell under the name of affine spaces.

Other examples

Points

Another simple example of a torsor is points. In linear algebra we get used to conflating points and vectors, because we consider vectors originating at zero.

But the geometrical intuition suggests that vectors do not have to be tied to zero. A vector is a directed segment which can be freely translated (shifted) on the plane (or in the space).

We can subtract two points to get a vector from one point to another. We can add a vector to a point, by translating it so that its origin becomes the given point. The end point of the vector then becomes the result of addition.

Vectors form an additive group, and points form a torsor over the vectors.

Like with dates, results of torsor operations on points and vectors do not depend on (and do not require the existence of) a coordinate system, while the «sum» of two points will change if you change the origin.

Files

There are also some examples that look very much like torsors, but do not satisfy all the laws.

One of such examples is files. The difference of two files is what we call a patch, or a diff. We can diff two files and apply the resulting patch to a third file («cherry-picking»).

Although patches do not form a group, they can be modelled as inverse semigroups. This motivates us to consider torsors based on inverse semigroups rather than on groups.

Haskell dialects

Haskell dialects can be considered as a torsor-like structure over Haskell extensions. We can add extensions to languages, e.g. add(RankNTypes+GADTs−MonomorphismRestriction, Haskell2010).

We can also subtract two languages to find out the difference between them in terms of extensions: diff(Haskell2010, Haskell98)=DoAndIfThenElse+PatternGuards+ForeignFunctionInterface+EmptyDataDecls−NPlusKPatterns.

Extensions do not form a group. The easiest way to see that is to observe that extensions are idempotent, i.e. X+X=X for any extension X. However, in a group there is only one idempotent element — zero.

Extensions do form an inverse semigroup, though. Thus we again arrive at inverse semigroup torsors.

Acknowledgements

A recent discussion with Niklas Broberg about Haskell extensions motivated me to explore these concepts. His HIW’12 lightning talk on Haskell Modular Mindset is also relevant.

Oleksandr Manzyuk kindly pointed me to the concept of a torsor.