Glennzilla gives good interview.

But if you want a highlight:

So, for the top national security official in the United States to go to the Senate and lie to their faces and deny that the NSA is doing exactly that which our reporting proved that the NSA was in fact doing is plainly a crime, and of course he should be prosecuted, and would be prosecuted if we lived under anything resembling the rule of law, where everybody is held and treated equally under the law, regardless of position or prestige. Of course, we don’t have that kind of system, which is why no Wall Street executives have been prosecuted, no top-level Bush officials were prosecuted for torture or warrantless eavesdropping, and why James Clapper hasn’t been prosecuted despite telling an overt lie to Congress. And what’s even more amazing, though, Amy, is that not only has James Clapper not been prosecuted, he hasn’t even lost his job. He’s still the director of national intelligence many months after his lie was revealed, because there is no accountability for the top-level people in Washington.

And the final thing to say about that is, there’s all kinds of American journalists who love to go on television and accuse Edward Snowden of committing all these grave and horrible crimes. They’re so brave when it comes to declaring Edward Snowden to be a criminal and calling for [inaudible]. Not one of them has ever gone on television and said, "James Clapper committed crimes, and he ought to be prosecuted." The question that you just asked journalistically is such an important and obvious one, yet not—none of the David Gregorys or Jeffrey Toobins or all these American journalists who fancy themselves as aggressive, tough reporters, would ever dare utter the idea that James Clapper ought to be arrested or prosecuted for the crimes that he committed, because they’re there to serve those interests and not to challenge or be adversarial to them.

Update: Commenters said if the Amy Goodman interview wasn't too longer enough for  you, see the email exchange with Keller.

In 20 years no one will admit to have ever opposed legal gay marriage. The pace of change amazes me. I do wonder how many people still admit to ever opposing interracial marriage. A majority did, until 1997.

Google's quantum lab has released a version of the computer game designed to teach players about the weird principles of quantum mechanics

One of the hallmarks of the technocratic centrists is the belief that industry regulation is passé--that policy makers must rely on private enterprises because that is where you find the real experts. Which leads to stuff like this:

Quality Software Services Inc., or Q.S.S.I., a unit of the UnitedHealth Group, developed the identity management system, another major component that allowed consumers to register and establish accounts.

I suppose it's too way too late to bring this up again, but the bill was called the Patient Protection and Affordable Care Act. "Protection" largely referring to the insurance companies.

Many urological studies rely upon animal models such as rats and pigs whose urination physics and correlation to humans are poorly understood. Here we elucidate the hydrodynamics of urination across five orders of magnitude in animal mass. Using high-speed videography and flow rate measurement at Zoo Atlanta, we discover the "Law of Urination," which states animals empty their bladders over nearly constant duration of average 21 seconds (standard deviation 13 seconds). This feat is made possible by larger animals having longer urethras, thus higher gravitational force and flow speed. Smaller mammals are challenged during urination due to high viscous and surface tension forces that limit their urine to single drops. Our findings reveal the urethra constitutes as a flow enhancing device, enabling the urinary system to be scaled up without compromising its function. This study may help in the diagnosis of urinary problems in animals and in inspiring the design of scalable hydrodynamic systems based on those in nature.

Here is a quick thought about the mathematics of the US shutdown, not to be taken too seriously (the thought I mean — the shutdown obviously is to be taken seriously). It’s for the benefit of anyone who is puzzled that the Tea Party can have such a large influence, and more generally how a political system can be stable when almost nobody likes it. I’m going to prove that in a country of people, it is possible to devise a democratic system in which of those people control the decisions, where . For example, in a population of 100,000,000, all you need is a band of fanatics with about 112,000 people — or approximately 0.1% of the population. Although we do not have such a system and the distribution is unlikely, the systems and distributions we do have still allow a minority to have undue influence, and for similar reasons. What I’m about to describe is the extreme case.

The system I have in mind works as follows. It’s a multilevel representative democracy. Suppose for convenience that for some positive integer . (It is easy, but slightly tedious, to modify what I am about to write to take care of more general .) Suppose that the country is divided into three “super-constituencies”, each of which gets a vote in the top-level decision-making body (known as the triumvirate). Suppose that decisions in that body are passed by a majority vote. A group of people that wants to control the country can do so as long as it can control at least two votes in the triumvirate.

How are the members of the triumvirate chosen? They are elected by another triumvirate one level down. The representative in the top-level triumvirate is representing the views of the three people in the triumvirate one level down, and is worried about stepping out of line, since then he/she risks being deselected by the three people in the level-2 body.

So if a merry band of fanatics wants to control a representative in the top-level triumvirate, it is enough to control at least two of the representatives in the second-level triumvirate that selects the top-level representative.

Of course, we can iterate this argument. So how many people do we need to control the country? We need two at the top level, and therefore four at the second level, and so on. Therefore, we need at the bottom level. (Note that the representatives do not have to be fanatics themselves — if they don’t vote in the way that the fanatics want, then they get deselected by the people one level down, losing all those lovely perks that go with a high-level job in politics.) If , then , so we’re done.

One might want to make small adjustments to the bound to allow all the different levels of influence to be disjoint. So then . But this is within a constant of . Similarly, if we start with some that is not of that precise form, that again affects the estimate by just a constant factor.

So the conclusion is that in principle people can mess up a country with population . If you have more people than that, then the main thing you want is a system with a few levels of groups within groups — not necessarily formal at every level — and a distribution that is not too concentrated and not too diffuse. (If it is too concentrated, then you’ll end up wasting a lot of votes on controlling representatives who are already controlled, but if it is too diffuse, then you won’t control anybody except at very low levels. In the extreme case, what you want is to be arranged in what can be viewed as a discrete approximation to the Cantor set: in less extreme cases you still want to be somewhat “fractal” and “Cantor-like”.)

Preface

The intended audience of this article is someone who knows something about Machine Learning and Artifical Neural Networks (ANNs) in particular and who recalls that fitting an ANN required a technique called backpropagation. The goal of this post is to refresh the reader’s knowledge of ANNs and backpropagation and to show that the latter is merely a specialised version of automatic differentiation, a tool that all Machine Learning practitioners should know about and have in their toolkit.

Introduction

The problem is simple to state: we have a (highly) non-linear function, the cost function of an Artificial Neural Network (ANN), and we wish to minimize this so as to estimate the parameters / weights of the function.

In order to minimise the function, one obvious approach is to use steepest descent: start with random values for the parameters to be estimated, find the direction in which the the function decreases most quickly, step a small amount in that direction and repeat until close enough.

But we have two problems:

• We have an algorithm or a computer program that calculates the non-linear function rather than the function itself.

• The function has a very large number of parameters, hundreds if not thousands.

One thing we could try is bumping each parameter by a small amount to get partial derivatives numerically

But this would mean evaluating our function many times and moreover we could easily get numerical errors as a result of the vagaries of floating point arithmetic.

As an alternative we could turn our algorithm or computer program into a function more recognisable as a mathematical function and then compute the differential itself as a function either by hand or by using a symbolic differentiation package. For the complicated expression which is our mathematical function, the former would be error prone and the latter could easily generate something which would be even more complex and costly to evaluate than the original expression.

The standard approach is to use a technique called backpropagation and the understanding and application of this technique forms a large part of many machine learning lecture courses.

Since at least the 1960s techniques for automatically differentiating computer programs have been discovered and re-discovered. Anyone who knows about these techniques and reads about backpropagation quickly realises that backpropagation is just automatic differentiation and steepest descent.

This article is divided into

• Refresher on neural networks and backpropagation;

• Methods for differentiation;

• Backward and forward automatic differentiation and

• Concluding thoughts.

The only thing important to remember throughout is the chain rule

in alternative notation

where . More suggestively we can write

where it is understood that and are evaluated at and is evaluated at .

For example,

Acknowledgements

Sadly I cannot recall all the sources I looked at in order to produce this article but I have made heavy use of the following.

• Justin Domke’s blog

• Conal Elliot’s website

• Jerzy Karczmarczuk’s papers

• The Community Portal for Automatic Differentiation

• Alexey Radul’s blog

Neural Network Refresher

Here is our model, with the input, the predicted output and the actual output and the weights in the -th layer. We have concretised the transfer function as but it is quite popular to use the function.

with the loss or cost function

The diagram below depicts a neural network with a single hidden layer.

In order to apply the steepest descent algorithm we need to calculate the differentials of this latter function with respect to the weights, that is, we need to calculate

Applying the chain rule

Since

we have

Defining

we obtain

Finding the for each layer is straightforward: we start with the inputs and propagate forward. In order to find the we need to start with the outputs a propagate backwards:

For the output layer we have (since )

For a hidden layer using the chain rule

Now

so that

and thus

Summarising

1. We calculate all and for each layer starting with the input layer and propagating forward.

2. We evaluate in the output layer using .

3. We evaluate in each layer using starting with the output layer and propagating backwards.

4. Use to obtain the required derivatives in each layer.

For the particular activation function we have . And finally we can use the partial derivatives to step in the right direction using steepest descent

where is the step size aka the learning rate.

Differentiation

So now we have an efficient algorithm for differentiating the cost function for an ANN and thus estimating its parameters but it seems quite complex. In the introduction we alluded to other methods of differentiation. Let us examine those in a bit more detail before moving on to a general technique for differentiating programs of which backpropagation turns out to be a specialisation.

Numerical Differentiation

Consider the function then its differential and we can easily compare a numerical approximation of this with the exact result. The numeric approximation is given by

In theory we should get a closer and closer approximation as epsilon decreases but as the chart below shows at some point (with ) the approximation worsens as a result of the fact that we are using floating point arithmetic. For a complex function such as one which calculates the cost function of an ANN, there is a risk that we may end up getting a poor approximation for the derivative and thus a poor estimate for the parameters of the model.

Symbolic Differentiation

Suppose we have the following program (written in Python)

import numpy as np

def many_sines(x):
    y = x
    for i in range(1,7):
        y = np.sin(x+y)
    return y

When we unroll the loop we are actually evaluating

Now suppose we want to get the differential of this function. Symbolically this would be

Typically the non-linear function that an ANN gives is much more complex than the simple function given above. Thus its derivative will correspondingly more complex and therefore expensive to compute. Moreover calculating this derivative by hand could easily introduce errors. And in order to have a computer perform the symbolic calculation we would have to encode our cost function somehow so that it is amenable to this form of manipulation.

Automatic Differentiation

Reverse Mode

Traditionally, forward mode is introduced first as this is considered easier to understand. We introduce reverse mode first as it can be seen to be a generalization of backpropagation.

Consider the function

Let us write this a data flow graph.

We can thus re-write our function as a sequence of simpler functions in which each function only depends on variables earlier in the sequence.

In our particular example, since do not depend on

Further does not depend on so we also have

Now things become more interesting as and both depend on and so the chain rule makes an explicit appearance

Carrying on

Note that having worked from top to bottom (the forward sweep) in the graph to calculate the function itself, we have to work backwards from bottom to top (the backward sweep) to calculate the derivative.

So provided we can translate our program into a call graph, we can apply this procedure to calculate the differential with the same complexity as the original program.

The pictorial representation of an ANN is effectively the data flow graph of the cost function (without the final cost calculation itself) and its differential can be calculated as just being identical to backpropagation.

Forward Mode

An alternative method for automatic differentiation is called forward mode and has a simple implementation. Let us illustrate this using Haskell 98. The actual implementation is about 20 lines of code.

First some boilerplate declarations that need not concern us further.

> {-# LANGUAGE NoMonomorphismRestriction #-}
> 
> module AD (
>     Dual(..)
>   , f
>   , idD
>   , cost
>   , zs
>   ) where
> 
> default ()

Let us define dual numbers

> data Dual = Dual Double Double
>   deriving (Eq, Show)

We can think of these pairs as first order polynomials in the indeterminate , such that

Thus, for example, we have

Notice that these equations implicitly encode the chain rule. For example, we know, using the chain rule, that

And using the example equations above we have

Notice that dual numbers carry around the calculation and the derivative of the calculation. To actually evaluate at a particular value, say 2, we plug in 2 for and 1 for

Thus the derivative of at 2 is .

With a couple of helper functions we can implement this rule () by making Dual an instance of Num, Fractional and Floating.

> constD :: Double -> Dual
> constD x = Dual x 0
> 
> idD :: Double -> Dual
> idD x = Dual x 1.0

Let us implement the rules above by declaring Dual to be an instance of Num. A Haskell class such as Num simply states that it is possible to perform a (usually) related collection of operations on any type which is declared as an instance of that class. For example, Integer and Double are both types which are instances on Num and thus one can add, multiply, etc. values of these types (but note one cannot add an Integer to a Double without first converting a value of the former to a value of the latter).

As an aside, we will never need the functions signum and abs and declare them as undefined; in a robust implementation we would specify an error if they were ever accidentally used.

> instance Num Dual where
>   fromInteger n             = constD $ fromInteger n
>   (Dual x x') + (Dual y y') = Dual (x + y) (x' + y')
>   (Dual x x') * (Dual y y') = Dual (x * y) (x * y' + y * x')
>   negate (Dual x x')        = Dual (negate x) (negate x')
>   signum _                  = undefined
>   abs _                     = undefined

We need to be able to perform division on Dual so we further declare it to be an instance of Fractional.

> instance Fractional Dual where
>   fromRational p = constD $ fromRational p
>   recip (Dual x x') = Dual (1.0 / x) (-x' / (x * x))

We want to be able to perform the same operations on Dual as we can on Float and Double. Thus we make Dual an instance of Floating which means we can now operate on values of this type as though, in some sense, they are the same as values of Float or Double (in Haskell 98 only instances for Float and Double are defined for the class Floating).

> instance Floating Dual where
>   pi = constD pi
>   exp   (Dual x x') = Dual (exp x)   (x' * exp x)
>   log   (Dual x x') = Dual (log x)   (x' / x)
>   sqrt  (Dual x x') = Dual (sqrt x)  (x' / (2 * sqrt x))
>   sin   (Dual x x') = Dual (sin x)   (x' * cos x)
>   cos   (Dual x x') = Dual (cos x)   (x' * (- sin x))
>   sinh  (Dual x x') = Dual (sinh x)  (x' * cosh x)
>   cosh  (Dual x x') = Dual (cosh x)  (x' * sinh x)
>   asin  (Dual x x') = Dual (asin x)  (x' / sqrt (1 - x*x))
>   acos  (Dual x x') = Dual (acos x)  (x' / (-sqrt (1 - x*x)))
>   atan  (Dual x x') = Dual (atan x)  (x' / (1 + x*x))
>   asinh (Dual x x') = Dual (asinh x) (x' / sqrt (1 + x*x))
>   acosh (Dual x x') = Dual (acosh x) (x' / (sqrt (x*x - 1)))
>   atanh (Dual x x') = Dual (atanh x) (x' / (1 - x*x))

That’s all we need to do. Let us implement the function we considered earlier.

> f =  sqrt . (* 3) . sin

The compiler can infer its type

ghci> :t f
  f :: Floating c => c -> c

We know the derivative of the function and can also implement it directly in Haskell.

> f' x = 3 * cos x / (2 * sqrt (3 * sin x))

Now we can evaluate the function along with its automatically calculated derivative and compare that with the derivative we calculated symbolically by hand.

ghci> f $ idD 2
  Dual 1.6516332160855343 (-0.3779412091869595)

ghci> f' 2
  -0.3779412091869595

To see that we are not doing symbolic differentiation (it’s easy to see we are not doing numerical differentiation) let us step through the actual evaluation.

A Simple Application

In order not to make this blog post too long let us apply AD to finding parameters for a simple regression. The application to ANNs is described in a previous blog post. Note that in a real application we would use the the Haskell AD and furthermore use reverse AD as in this case it would be more efficient.

First our cost function

> cost m c xs ys = (/ (2 * (fromIntegral $ length xs))) $
>                  sum $
>                  zipWith errSq xs ys
>   where
>     errSq x y = z * z
>       where
>         z = y - (m * x + c)

ghci> :t cost
  cost :: Fractional a => a -> a -> [a] -> [a] -> a

Some test data

> xs = [1,2,3,4,5,6,7,8,9,10]
> ys = [3,5,7,9,11,13,15,17,19,21]

and a learning rate

> gamma = 0.04

Now we create a function of the two parameters in our model by applying the cost function to the data. We need the (partial) derivatives of both the slope and the offset.

> g m c = cost m c xs ys

Now we can take use our Dual numbers to calculate the required partial derivatives and update our estimates of the parameter. We create a stream of estimates.

> zs = (0.1, 0.1) : map f zs
>   where
> 
>     deriv (Dual _ x') = x'
> 
>     f (c, m) = (c - gamma * cDeriv, m - gamma * mDeriv)
>       where
>         cDeriv = deriv $ g (constD m) $ idD c
>         mDeriv = deriv $ flip g (constD c) $ idD m

And we can calculate the cost of each estimate to check our algorithm converges and then take the the estimated parameters when the change in cost per iteration has reached an acceptable level.

ghci> take 2 $ drop 1000 $ map (\(c, m) -> cost m c xs ys) zs
  [1.9088215184565296e-9,1.876891490619424e-9]

ghci> take 2 $ drop 1000 zs
  [(0.9998665320141327,2.0000191714150106),(0.999867653022265,2.0000190103927853)]

Concluding Thoughts

Efficiency

Perhaps AD is underused because of efficiency?

It seems that the Financial Services industry is aware that AD is more efficient than current practice albeit the technique is only slowly permeating. Order of magnitude improvements have been reported.

• Smoking Adjoints: fast evaluation of Greeks in Monte Carlo Calculations

• Adjoints and automatic (algorithmic) differentiation in computational finance

Perhaps AD is slowly permeating into Machine Learning as well but there seem to be no easy to find benchmarks.

• Good Morning, Economics Blog

• Andrew Gelman’s Blog

Automatic Differentiation Tools

If it were only possible to implement automatic differentiation in Haskell then its applicability would be somewhat limited. Fortunately this is not the case and it can be used in many languages.

In general, there are three different approaches:

• Operator overloading: available for Haskell and C++. See the Haskell ad package and the C++ FADBAD approach using templates.

• Source to source translators: available for Fortran, C and other languages e.g., ADIFOR, TAPENADE and see the wikipedia entry for a more comprehensive list.

• New languages with built-in AD primitives. I have not listed any as it seems unlikely that anyone practicing Machine Learning would want to transfer their existing code to a research language. Maybe AD researchers could invest time in understanding what language feature improvements are needed to support AD natively in existing languages.

Author(s): Ray Ng, Erik S. Sørensen, and Piotr Deuar

We reformulate the full quantum dynamics of spin systems using a phase-space representation based on SU(2) coherent states which generates an exact mapping of the dynamics of any spin system onto a set of stochastic differential equations. This representation is superior in practice to an earlier ph...

[Phys. Rev. B 88, 144304] Published Fri Oct 04, 2013

Topological motifs in synaptic connectivity—such as the cortical column—are fundamental to processing of information in cortical structures. However, the mesoscale topology of cortical networks beyond columns remains largely unknown. In the olfactory cortex, which lacks an obvious columnar structure, sensory-evoked patterns of activity have failed to reveal organizational principles of...

Nature Physics 9, 673 (2013). doi:10.1038/nphys2741

Authors: Baruch Barzel & Albert-László Barabási

Computing is a high-level process of a physical system. Recent interest in non-standard computing systems, including quantum and biological computers, has brought this physical basis of computing to the forefront. There has been, however, no consensus on how to tell if a given physical system is acting as a computer or not; leading to confusion over novel computational devices, and even claims that every physical event is a computation. In this paper we introduce a formal framework that can be used to determine whether or not a physical system is performing a computation. We demonstrate how the abstract computational level interacts with the physical device level, drawing the comparison with the use of mathematical models to represent physical objects in experimental science. This powerful formulation allows a precise description of the similarities between experiments, computation, simulation, and technology, leading to our central conclusion: physical computing is the use of a physical system to predict the outcome of an abstract evolution. We give conditions that must be satisfied in order for computation to be occurring, and illustrate these with a range of non-standard computing scenarios. The framework also covers broader computing contexts, where there is no obvious human computer user. We define the critical notion of a 'computational entity', and show the role this plays in defining when computing is taking place in physical systems.

I'm about to fly to Vienna where I'll be attending a conference on Emergent Quantum Mechanics. I'm not entirely sure why I was invited to this event, but I suspect it's got something to do with me being one of the three people on the planet who like superdeterministic hidden variables theories, more commonly known as "conspiracy theories".

Leaving aside some loopholes that are about to be closed, tests of Bell's theorem rule out local hidden variables theories. But any theorem is only as good as the assumptions that go into it, and one of these assumptions is that the experimenter can freely chose the detector settings. As you know, I don't believe in free will, so I have an issue with this. You can see though why theories in which this assumption does not hold are known as "conspiracy theories". While they are not strictly speaking ruled out, it seems that the universe must be deliberately mean to prevent the experimentalists from doing what they want, and this option is thus often not taken seriously.

But really, this is a very misleading interpretation of superdeterminism. All that superdeterminism means is that a state cannot be prepared independently of the detector settings. That's non-local of course, but it's non-local in a soft way, in the sense that it's a correlation but doesn't necessarily imply a 'spooky' action at a distance because the backwards lightcones of the detector and state (in a reasonable universe) intersect anyway.

That having been said, you might like or not like superdeterministic hidden variables theories, the real question is if there is some way to test if that's how nature works, because one can't use Bell's theorem here. After some failed attempts, I finally came up with a possible test that is almost model-independent, and it was published in my paper "Testing super-deterministic hidden variables theories".

I actually wrote this paper in the hospital when I was pregnant. The nurses kept asking me if I'm writing a book. They were quite disappointed to be drowned in elaborations on the foundations of quantum mechanics rather than hearing a vampire story. In any case, in the expectation that the readers on this blog are somewhat more sympathetic to the question whether the universe is fundamentally deterministic or not, here a brief summary of the idea.

The central difference between standard quantum mechanics and superdeterministic hidden variables theories is that in the former case two identically prepared states can give two different measurement outcomes, while in the latter case that's not possible. Unfortunately, "identically prepared" includes the hidden variables and it's difficult to identically prepare something that you can't measure. That is after all the reason why it looks indeterministic.

However, rather than trying to prepare identical states we can try to make repeated measurements on the same state. For that, take two non-commuting variables (for example the spin or polarization in two different directions) and measure them alternately. In standard quantum mechanics the measurement outcomes will be non-correlated. In a superdeterministric hidden variables theory, they'll be correlated - provided you can make a case that the hidden variables don't change in between the measurements. The figure below shows an example for an experimental setup.

A particle (electron/photon) is bounced back and forth between two mirrors (grey bars). The blue and red bars indicate measurements of two non-commuting variables, only one eigenvalue passes, the other leaves the system. The quantity to measure is the average time it takes until the particle leaves. In a superdeterministic theory, it can be significantly longer than in standard quantum mechanics.

The provision that the hidden variables don't change is the reason why the test is only 'almost' model independent, because I made the assumptions that the hidden variables are due to the environment (the experimental setup) down to the relevant scales of the interactions taking place. This means basically if you make the system small and cool and measure quickly enough you have a chance to see the correlation between subsequent measurements. I made some estimates (see paper) and it seems possible with today's technology to make this test.

Interestingly, after I had finished a draft of the paper, Chris Fuchs sent me a reference to a 1970 article by Eugene Wigner where, in a footnote, Wigner mentions Von Neumann discussing exactly this type of experiment:

“Von Neumann often discussed the measurement of the spin component of a spin-1/2 particle in various directions. Clearly, the possibilities for the two possible outcomes of a single such measurement can be easily accounted for by hidden variables [...] However, Von Neumann felt that this is not the case for many consecutive measurements of the spin component in various different directions. The outcome of the first such measurement restricts the range of values which the hidden parameters must have had before that first measurement was undertaken. The restriction will be present also after the measurement so that the probability distribution of the hidden variables characterizing the spin will be different for particles for which the measurement gave a positive result from that of the particles for which the measurement gave a negative result. The range of the hidden variables will be further restricted in the particles for which a second measurement of the spin component, in a different direction, also gave a positive result...”

Apparently there was a longer discussion with Schrödinger following this proposal, which could be summarized with saying that the experiment cannot test generic superdeterminism, but only certain types as I already said above. If you think about it for a moment, you can never rule out generic superdeterminism anyway, so why even bother.

I'm quite looking forward to this conference, to begin with because Vienna is a beautiful city and I haven't been there for a while, but also because I'm hoping to meet some experimentalists who can tell me if I'm nuts :p

Update: Slides of my talk are here.

In the 1960s literacy tests were used in some states in the United States to suppress voting.  The Civil Rights Movement Veterans website has collected a number of these.

In addition to completing the application and swearing the oaths, you had to pass the actual "Literacy Test" itself. Because the Freedom Movement was running "Citizenship Schools" to help people learn how to fill out the forms and pass the test, Alabama changed the test 4 times in less than two years (1964-1965). At the time of the Selma Voting Rights campaign there were actually 100 different tests in use across the state...

Most of the tests collected here are a battery of trivia questions related to civic procedure and citizenship. (Two from the Alabama test: “Name the attorney general of the United States” and “Can you be imprisoned, under Alabama law, for a debt?”).

But this Louisiana “literacy” test, singular among its fellows, has nothing to do with citizenship. Designed to put the applicant through mental contortions, the test's questions are often confusingly worded. If some of them seem unanswerable, that effect was intentional. The (white) registrar would be the ultimate judge of whether an answer was correct.

Try this one: “Write every other word in this first line and print every third word in same line (original type smaller and first line ended at comma) but capitalize the fifth word that you write.” 

Done with page one (above)?  Here are pages 2 and 3:

The test was to be taken in 10 minutes flat, and a single wrong answer meant a failing grade.

The neurotransmitter dopamine is one we all seem to have heard about, claimed to be central to love, gambling, reward, addiction, etc. A recent article by Howe et al. shows a bit more nuance than previously assumed in what dopamine levels are signaling. They ramp up during navigation towards an expected reward. I thought I would pass on a clip from Niv's summary of their findings, followed by the Howe et al. abstract

a, Dopaminergic neurons in the midbrain project to all brain areas, most prominently to the striatum (black arrows). These cells fire at a constant rate of 3–5 spikes per second, with occasional phasic bursts or pauses on the occurrence of positive reward prediction errors (discovering that the local supermarket now supplies your favourite coffee beans, b) or negative reward prediction errors (sipping your coffee and finding that the milk has gone sour, c), respectively. The background (tonic) level of dopamine fluctuates slowly, possibly tracking the average rate of rewards (not shown). d, By measuring dopamine concentrations in the striatum of rats navigating mazes, Howe et al.1 reveal a third mode of dopaminergic signalling: when a prolonged series of actions must be completed to obtain a reward (for instance, all the steps it takes to make a cup of coffee), dopamine concentration ramps up gradually, at each point in time signalling the predicted distance from the goal.

The abstract:

Predictions about future rewarding events have a powerful influence on behaviour. The phasic spike activity of dopamine-containing neurons, and corresponding dopamine transients in the striatum, are thought to underlie these predictions, encoding positive and negative reward prediction errors. However, many behaviours are directed towards distant goals, for which transient signals may fail to provide sustained drive. Here we report an extended mode of reward-predictive dopamine signalling in the striatum that emerged as rats moved towards distant goals. These dopamine signals, which were detected with fast-scan cyclic voltammetry (FSCV), gradually increased or—in rare instances—decreased as the animals navigated mazes to reach remote rewards, rather than having phasic or steady tonic profiles. These dopamine increases (ramps) scaled flexibly with both the distance and size of the rewards. During learning, these dopamine signals showed spatial preferences for goals in different locations and readily changed in magnitude to reflect changing values of the distant rewards. Such prolonged dopamine signalling could provide sustained motivational drive, a control mechanism that may be important for normal behaviour and that can be impaired in a range of neurologic and neuropsychiatric disorders.

"What would you do, hippie?" is the other shorter Kristof. Violence escalation is always the only answer, despite having had a couple of lessons in the past decade about why that really isn't such a good idea.

I don't have the knowledge to plan the logistics for a massive refugee resettlement program (for example) and fit it neatly into a 700 word column. But someone should be thinking of ways to help the people we claim to care about.

Research previously conducted on rats did not adequately control for the environment in which the rats were studied:

Alexander's hypothesis was that drugs do not cause addiction, and that the apparent addiction to opiate drugs commonly observed in laboratory rats exposed to it is attributable to their living conditions, and not to any addictive property of the drug itself. He told the Canadian Senate in 2001 that prior experiments in which laboratory rats were kept isolated in cramped metal cages, tethered to a self-injection apparatus, show only that "severely distressed animals, like severely distressed people, will relieve their distress pharmacologically if they can."

To test his hypothesis, Alexander built Rat Park, an 8.8 m2 (95 sq ft) housing colony, 200 times the square footage of a standard laboratory cage. There were 16–20 rats of both sexes in residence, an abundance of food, balls and wheels for play, and enough space for mating and raising litters. The results of the experiment appeared to support his hypothesis. Rats who had been forced to consume morphine hydrochloride for 57 consecutive days were brought to Rat Park and given a choice between plain tap water and water laced with morphine. For the most part, they chose the plain water. "Nothing that we tried," Alexander wrote, "... produced anything that looked like addiction in rats that were housed in a reasonably normal environment." Control groups of rats isolated in small cages consumed much more morphine in this and several subsequent experiments. 

Via garry's subposthaven.  There is a relevant discussion at Reddit.

Addendum:   A hat tip to Fletcher in Portugal for noting that the research described above has been summarized in a well-illustrated 40-page cartoon.

I watched Network last night on the advice of my friends in Occupy. More like insistence than advice, actually: they claimed I absolutely needed to see it, that it would blow me away with its prescience and wisdom.

Turns out they were absolutely right.

Here’s the thing, though. Given that Network was released in 1977, I’m hesitant to even suggest to young people today (defined as: younger than me) that they watch it because they it’s so true, its predictions are so spot-on accurate, that anyone who wasn’t alive in 1977 might not – probably cannot – appreciate how incredible it must have seemed back then. It might even seem boring to someone who is used to a world of Fox News and the internet’s filter bubble.

Then again, that’s not entirely true. It’s not just an amazing prediction about what TV and society would turn into. The other strength of the movie is that it keeps changing, in a mostly painful but sometimes hilarious way, from scene to scene, subplot to subplot, and that keeps it from being about just one idea or just one person.

A particularly powerful scene of a jilted wife really got to me, and even though the movie isn’t particularly about that relationship, the movie manages to make it work.

And the most ridiculous scene, which involves two revolutionary groups reading over a contract with a crowd of network lawyers, might also be the most convincingly depressing: we might have our own particular emotional and political issues and rebellions, but we are all cowed by the power of money.

If you wanted to force Network to be about one thing in particular, it would have to be an argument concerning the role of the individual in the modern world. Here’s the protagonist, Howard Beale, preaching to his television audience from this YouTube clip of Network:

… when the twelfth largest company in the world controls the most awesome goddamned propaganda force in the whole godless world, who knows what shit will be peddled for truth on this tube?

So, listen to me! Television is not the truth!  Television is a goddamned amusement park, that’s what television is! Television is a circus, a carnival, a travelling troupe of acrobats and story-tellers, singers and dancers, jugglers, side-show freaks, lion-tamers and football players.  We’re in the boredom-killing business!

If you want truth, go to God, go to your guru, go to yourself because that’s the only place you’ll ever find any real truth!  But, man, you’re never going to get any truth from us.

We’ll tell you anything you want to hear.  We lie like hell! We’ll tell you Kojack always gets the killer, and nobody ever gets cancer in Archie Bunker’s house. And no matter how much trouble the hero is in, don’t worry:  just look at your watch — at the end of the hour, he’s going to win.  We’ll tell you any shit you want to hear!

We deal in illusion, man! None of it’s true! But you people sit there — all of you — day after day, night after night, all ages, colors, creeds — we’re all you know. You’re beginning to believe this illusion we’re spinning here. You’re beginning to think the tube is reality and your own lives are unreal.

You do whatever the tube tells you.  You dress like the tube, you eat like the tube, you raise your children like the tube, you think like the tube. This is mass madness, you maniacs!

In God’s name, you people are the real thing!  We’re the illusions!  So turn off this goddam set! Turn it off right now!  Turn it off and leave it off.  Turn it off right now, right in the middle of this very sentence I’m speaking now.”

After a while, the head of the news corporation decides he’s had enough of Beale’s message and decides to give him the corporation’s perspective on the discussion. From this YouTube clip:

You get up on your little twenty-one inch screen, and howl about America and democracy.

There is no America. There is no democracy. There is only IBM and ITT and AT&T and Dupont, Dow, Union Carbide and Exxon. Those are the nations of the world today.

What do you think the Russians talk about in their councils of state — Karl Marx? They pull out their linear programming charts, statistical decision theories and minimax solutions and compute the price-cost probabilities of their transactions and investments just like we do.

We no longer live in a world of nations and ideologies, Mr. Beale.  The world is a college of corporations, inexorably determined by the immutable by-laws of business.

The world is a business, Mr. Beale!

It has been since man crawled out of the slime, and our children, Mr. Beale, will live to see that perfect world in which there is no war and famine, oppression and brutality — one vast and ecumenical holding company, for whom all men will work to serve a common profit, in which all men will hold a share of stock, all necessities provided, all anxieties tranquilized, all boredom amused.  And I have chosen you to preach this evangel, Mr. Beale.

What’s incredible about Network is that, until possibly the last 2 minutes, none of it seems particularly unrealistic. It’s satire that rings so true that it manages to avoid the standard skeptical or baffled response. And although it is not uplifting, Network is incredibly thought-provoking and current.

Finally, the movie also has some show-biz advice for anyone trying to communicate a message. Namely, being consistently depressing and apocalyptic gets old, even if there’s an element of truth to it. It’s critical to balance that with hope about the power of individual action, sprinkled with outrage and impulsive energy.

With this paper we bring about a discussion on the computing potential of complex optical networks and provide experimental demonstration that an optical fiber network can be used as an analog processor to calculate matrix inversion. A 3x3 matrix is inverted as a proof-of-concept demonstration using a fiber network containing three nodes and operating at telecomm wavelength. For an NxN matrix, the overall solving time (including setting time of the matrix elements and calculation time of inversion) scales as O(N^2), whereas matrix inversion by most advanced computer algorithms requires ~O(N^2.37) computational time. For well-conditioned matrices, the error of the inversion performed optically is found to be less than 3%, limited by the accuracy of measurement equipment.

History: Science luminaries are often religious

Nature 501, 7465 (2013). doi:10.1038/501033c

Authors: Robert White, George Ellis & Denis Alexander

Young Earth creationists are easy to lampoon (see G.BranchNature500, 149; 2013). However, using reasoned arguments might hold more sway with the US creationist movement.PZ Myers, author of The Happy Atheist (which Branch reviewed), should remember that the

Found at imgur.

In form, a review of Stung! On Jellyfish Blooms and the Future of the Ocean, by Lisa-ann Gershwin. In substance, a compendium of amazing and mostly horrifying facts about jellyfish, which are, apparently, taking over the world, or at least the marine portion of it. They can halt battleships, overturn trawlers, shut down power stations, wipe out fisheries, blockade continents. The Black Sea has become “effectively jellified”

Source.

Addendum:  In response to a comment that was written and later deleted, I should clarify that I posted this item not to minimize or make fun of the suffering of the people of Syria, but as a reflection of a prevailing (and increasing) attitude of Americans toward their elected "leadership."  The people of the country have been lied to - repeatedly and almost systematically - by a long series of Presidents and Congresses on countless matters relating to foreign policy.  Invariably the lies and deceptions have led to increased military aggressiveness abroad and an insane funding of the military-industrial complex to a degree that has depleted the resources of this country for domestic purposes and resulted in a deterioration of our infrastructure and degradation of our environment.  This has to stop.  I hope a similar sentiment is what motivated the staff at The Onion.

I presume the backyard rink must be intended only for peewee hockey, because I wouldn't want to be checked against those low sideboards.

Found at Izismile - unfortunately without credits re sources.

Yet another aspect of Gary Shteyngart’s dystopian fiction novel Super Sad True Love Story is coming true for reals this week.

Besides anticipating Occupy Wall Street, as well as Bloomberg’s sweep of Zuccotti Park (although getting it wrong on how utterly successful such sweeping would be), Shteyngart proposed the idea of instant, real-time and broadcast credit ratings.

Anyone walking around the streets of New York, as they’d pass a certain type of telephone pole – the kind that identifies you via your cell phone and communicates with data warehousing services and databases – would have their credit rating flashed onto a screen. If you went to a party, depending on how you impressed the other party go-ers, your score could plummet or rise in real time, and everyone would be able to keep track and treat you accordingly.

I mean, there were other things about the novel too, but as a data person these details certainly stuck with me since they are both extremely gross and utterly plausible.

And why do I say they are coming true now? I base my claim on two news stories I’ve been sent by my various blog readers recently.

[Aside: if you read my blog and find an awesome article that you want to send me, by all means do! My email address is available on my "About" page.]

First, coming via Suresh and Marcos, we learn that data broker Acxiom is letting people see their warehoused data. A few caveats, bien sûr:

1. You get to see your own profile, here, starting in 2 days, but only your own.
2. And actually, you only get to see some of your data. So they won’t tell you if you’re a suspected gambling addict, for example. It’s a curated view, and they want your help curating it more. You know, for your own good.
3. And they’re doing it so that people have clarity on their business.
4. Haha! Just kidding. They’re doing it because they’re trying to avoid regulations and they feel like this gesture of transparency might make people less suspicious of them.
5. And they’re counting on people’s laziness. They’re allowing people to opt out, but of course the people who should opt out would likely never even know about that possibility.
6. Just keep in mind that, as an individual, you won’t know what they really think they know about you, but as a corporation you can buy complete information about anyone who hasn’t opted out.

In any case those credit scores that Shteyngart talks about are already happening. The only issue is who gets flashed those numbers and when. Instead of the answers being “anyone walking down the street” and “when you walk by a pole” it’s “any corporation on the interweb” and “whenever you browse”.

After all, why would they give something away for free? Where’s the profit in showing the credit scores of anyone to everyone? Hmmmm….

That brings me to my second news story of the morning coming to me via Constantine, namely this TechCrunch story which explains how a startup called Fantex is planning to allow individuals to invest in celebrity athletes’ stocks. Yes, you too can own a tiny little piece of someone famous, for a price. From the article:

People can then buy shares of that player’s brand, like a stock, in the Fantex-consumer market. Presumably, if San Francisco 49ers tight end Vernon Davis has a monster year and looks like he’s going to get a bigger endorsement deal or a larger contract in a few years, his stock would rise and a fan could sell their Davis stock and cash out with a real, monetary profit. People would own tracking or targeted stocks in Fantex that would depend on the specific brand that they choose; these stocks would then rise and fall based on their own performance, not on the overall performance of Fantex.

Let’s put these two things together. I think it’s not too much of a stretch to acknowledge a reason for everyone to know everyone else’s credit score! Namely, we can can bet on each other’s futures!

I can’t think of any set-up more exhilarating to the community of hedge fund assholes than a huge, new open market – containing profit potentials for every single citizen of earth – where you get to make money when someone goes to the wrong college, or when someone enters into an unfortunate marriage and needs a divorce, or when someone gets predictably sick. An orgy in the exact center of tech and finance.

Are you with me peoples?!

I don’t know what your Labor Day plans are, but I’m getting ready my list of people to short in this spanking new market.