Aicm.v2

© www.CartoonStock.com

When I was starting out on a doctorate, I’d look at the senior people in my field and wonder if I’d ever be like them. It must be great, I thought, to reach the advanced age of 40. By then you’d have learned everything you needed to know to do great science, and you could just focus on doing it. I suspect today’s crop of grad students are a bit more savvy than I was, but all the same, I wonder if they realise just how wrong that picture is – for two reasons.

First, you never stop learning. The field moves on. Instead of getting easier, it gets harder. I remember when techniques such as functional brain imaging first came along. The most competent people in that area were either those who had developed the methods, or young people who learned them as grad students. If you were of the generation above, you had three choices: ignore the methods, spend time learning them, or hire junior people who knew what they were doing. As the methods evolve, they get ever more complex, and meanwhile, your own brain starts to shrink. So if you are anticipating making it to a tenured post and then settling down in your armchair, think again.

Second, the more senior you get, the more of your time is spent, not on doing your own research, but on evaluation. You learn that an email entitled ‘invitation’ should not make your spirits rise: it’s just a desperate attempt to put a positive spin on a request for you to do more work for no reward. You get regular ‘invitations’ to review papers and grants, write job references, appraise promotion bids, sit on interview panels and examine theses. If you are involved in teaching, you’ll also be engaged in numerous other forms of appraisal.

I was prompted to think about this when someone asked on an electronic forum what was a reasonable number of doctoral theses to examine each year. The general consensus was two: though it will obviously depend on what other commitments someone has. It also varies from country to country. There are some jolly places in Europe where a PhD viva is just an excuse for a boozy party with a lot of dressing up in funny gowns and hats. In UK psychology, the whole thing is no fun at all: you have to read a document of 50,000-70,000 words reporting a body of work based on a series of experimental studies. You then write a report on it and see the candidate for a face-to-face viva, which is typically 2 to 3 hours long. Although failure is uncommon, it is not assumed that the candidate will pass (unlike in the viva-as-party countries), and weeping or catatonic candidates are not unheard of. Taking into account travel, etc., if you are going to do a proper job, you are probably talking about three days’ work. For this you get paid around the minimum wage – the fee for examining is typically somewhere between £120 and £200.

So why do we do it? The major reason is because the entire academic enterprise depends on reciprocity: we want people to examine our students and review our papers and grants. In addition, it’s important to maintain standards, and to ensure that degrees, promotions, publications and grants go to those who merit them. But the demands keep growing. In the 37 weeks of this year I’ve been asked to review 76 papers and six grants. I agreed to review 16 papers and three of the grants. This, of course, is nothing compared with being a journal editor or serving on a grants board, something that most of us will do at some point.

Clearly, if I agreed to do everything I was asked, I’d have no time for anything else. Of course, one learns to say no. But awareness of these pressures has made me look with rather a critical eye at how we use evaluation. There is, for instance, research suggesting that job interviews aren’t very useful at identifying good candidates:  we tend to be seduced by immediate impressions, which may not be a good indicator of a person’s suitability. Like most people, I’d be reluctant to take on an employee I hadn’t interviewed, but if Daniel Kahneman is to be believed, this is just because I am a victim of the Illusion of Validity.

I’m a supporter of the peer review system used by journals, and here I feel  I’m on more solid ground, because I can point to instances where my papers have been improved by input from reviewers. Nevertheless, where reviewing is used simply to reject/accept papers or grant proposals,  and where fine-grained decisions have to be made between many high-quality submissions, agreement between experts may be little better than chance (e.g. Fogelholm et al, 2012). Nevertheless, we stick with it, because it’s hard to know what to put in its place.

I’ve written a fair bit about that expensive and time-consuming evaluation process that UK academics engage in, the REF. It requires experts to make judgements of whether, for instance, papers are of 3* or 4* quality, a distinction based on whether the research is “world leading” or “internationally excellent…. but falls short of the highest standards of excellence.” The reliability of such judgements has not, to my knowledge, been evaluated, yet large amounts of funding depend on them. Those on REF committees are in the same situation as Pavlov’s poor dogs, having to make distinctions that are on the one hand impossible (discriminating circles and ellipses that become increasingly similar) and on the other hand very important (get it wrong and you get a shock).

There is one good thing about doing so much evaluation. You have the opportunity to see what others are doing – you may be the first person to read an important new paper, or examine a ground-breaking thesis. You may be forced to engage with different ways of thinking, and confronted with new topics and ideas. You may be able to provide useful input to authors. And since you yourself will be evaluated, it can be useful to see life from the other side of the table, as the person doing the evaluating. But all too often, even these advantages fail to compensate for the fact that as a senior academic you will spend more and more time on evaluation of others and less and less doing your own research.

Reference
Fogelholm, Mikael, Leppinen, Saara, Auvinen, Anssi, Raitanen, Jani, Nuutinen, Anu, & Väänänen, Kalervo (2012). Panel discussion does not improve reliability of peer review for medical research grant proposals Journal of Clinical Epidemiology, 65 (1), 47-52 DOI: 10.1016/j.jclinepi.2011.05.001

A new game, inspired by the protein-folding puzzle FoldIt, lets gamers design nanoparticles to target tumours and could result in novel treatments

geardrops:

This is why I prefer email/text to phone conversations :-/

Researchers at Stanford University in the US have developed a technique to make the brain see-through, allowing them to trace individual neurons, and their sometimes far-flung connections, from end to end. They’ve succeeded in doing so with an entire mouse brain and portions of human brains. To get light through the brain to reveal neural connections, scientists replaced the light-blocking fat cells.

Watch a fly-through video of the clarified rodent brain…

Meanwhile, the US government in 2014 is investing R1 billion in Brain Research through Advancing Innovative Neurotechnologies (BRAIN). The BRAIN scientists want to chart the simultaneous functions and activity of the brain’s 100 billion neurons and its 100 trillion synapses at an unprecedented resolution.

Understanding the brain’s circuitry could lead to better treatments for dementia and other neurological ailments, as well as developing mind-controlled prostheses and computers that “think” like humans. The Americans’ National Science Foundation, National Institutes of Health, and DARPA will contribute to the project, expected to cost billions more over the next few years.

Meanwhile, in your brain…

What scientists already know:

Brain highways”

When brain signals need to travel long distances, they take the highway. Researchers at the USA’s Indiana University and the University Medical Centre Utrecht in the Netherlands have uncovered high-capacity pathways that form the backbone of the neural network, facilitating communication between far-off brain regions.

Gene mapping:

The variance in how different areas of the brain express the same genes may play a role in shaping the brain’s architecture. Icelandic Heart Association researchers discovered a gene that’s overactive in the brains of people with Alzheimer’s, disrupting the activity of many other genes.

Deep brain stimulation:

Some diseases interrupt the electrical signals neurons use to communicate. In deep brain stimulation, implanted electrodes deliver weak impulses to the brain, changing how neurons fire. The technique, a Parkinson’s disease treatment since 2002, may also be effective against depression and bipolar disorder.

Mind-controlled prostheses:

Electrode implants allow patients paralysed from the neck down to operate thought-controlled robotic arms to independently drink coffee, eat chocolate, and reach out to a loved one. Better understanding of the motor cortex’s circuitry may unlock new prosthesis abilities.