It seems statistics in MR don't matter.....

I was reading Ray Poynters excellent blog article here: 

http://thefutureplace.typepad.com/the_future_place/ 

about the Likert scale. Ray stated:

“If a researcher makes the wild and unjustified decision (IMHO) to treat Likert numbers as an interval scale...”

I took this as him saying he agreed it was wrong to treat Likert as an interval scale, he did only say that it is “wild and unjustified” to treat them as interval. This is a decision that is made constantly in MR.

I will go further and say it is plain wrong and bad statistical practice to treat Likert scales as interval data. It is an ordinal scale. It should be treated as such. You can't take the mean and standard deviation of ordinal scales. They don't have them. They have a mode (sometimes two modes), they have a median, they don't have a mean.

I got these radical ideas some years ago. After my degree in Psychology and post graduate work I worked in a medical research facility called the “Institute of Neurology” (ION) in London UK. I worked in the “Computing and Medical Statistics Unit” in a basement in Guildford Street complete with IBM card punch machines, cockroaches and CDC UT2000 remote RJE terminal for the University of London Computer Center. I think the drum printer on the UT2000 gave me hearing damage, it sounded like a machine gun.

I worked for a medical statistician called Liz. I was what I called a “data monkey”, I ran programs, wrote them, cleaned data, killed cockroaches and generally helped out. Liz had worked with Sir Richard Peto, who is currently Professor of Medical Statistics and Epidemiology at the University of Oxford. Liz was passionate about her profession. I recall several studies we worked on together. One was a drug trial for a drug to cure Multiple Sclerosis, another was a long term epidemiological study of Multiple Sclerosis, another was a drug trial of a chemotherapy drug to be used against a particularly evil form of brain cancer called Gliomas. Then there were studies about stroke models in rats, muscular dystrophy and Tourettes syndrome. 

We had a strict procedure for all data. First all interval data variables were plotted and looked at. Then these variables were tested for normality of distribution, if they were not normal appropriate transformations were applied to correct any anomalies and then they were re-tested for normality. If they still didn't pass the normality test they were only analysed with non-parametric techniques. Other data variables were plotted too, and the histograms looked at carefully for anomalies. I can recall Liz spending quite some time researching if you could use a T-test on percentages. She concluded you could not. She decided percentages belonged to the Cauchy distribution, which has no mean or higher moments. Thus a T-test would be statistically invalid.

I asked Liz about this procedure of treating data and the rigour she applied. I came from psychology, we were a little more lax in our approach. She said we had to remember that the results we obtained mattered. They could be life and death decisions. A type I error on a drug trial could lead to more people dying because they were given a drug that didn't really work. A type II error on the epidemiological work may miss an important antecedent to Multiple Sclerosis, a crippling disease. 

We were working on data for the drug trial for treating Gliomas, a form of brain cancer. Gliomas remain a deadly form of cancer, with only a 50% survival rate within one year of diagnosis. We were using something called Survival Analysis to test the effect of a chemotherapy drug. Liz said we had to wait for more events to be sure of the results. An event was someone dying. I happened to look at the columns in the data which contained the ages of the subjects. 18, 19, 21 – these were people only a little younger than I was at the time. We had to wait for them to die to be sure the conclusions that were made about the drug (Vincristine) were correct. The results really mattered.

It seems to me the question about Likert scales is not so much about if you can treat them as an interval scale rather it is this: do the results matter ? Do you care that the results are correct ? Do the results matter enough to do the work properly ? If the results do matter, do it properly.

From what I can see very often it seems market researchers think the results don't matter......

Surveys, Genetics.....and Descartes..

A study from the University of North Carlolina says that survey taking is a genetic trait. They studied twins and determined that there was a 45% influence of genetics on survey taking. The study has been reported in numerous places, here for instance http://bit.ly/cG9E2d .

Is this finding so surprising ? It means that the remaining 55% of “influence” affecting survey taking is unaccounted for or attributed to environmental effects. Biologists know that genetic predispositions account for some behavior, so it is not so surprising that survey taking is influenced by genetics. Of course the study doesn't identify exactly how genetics influence survey taking. We never seem to hear that trait X predisposes people to take or not take surveys. It is a step in the right direction though. It is a step to what I call “Respondent Theory”, that is a theory of why respondents take surveys.

For psychologists interested in the antecedent of behavior there has long been the “nature vs nurture” debate. Genetics vs environment. It is a bit of a false debate though, as it is clear there is complex interactions between the two influences. It isn't just one or the other.

This is where Descartes comes in, sort of. He is probably remembered best for his dictum “I think therefore I am”. He also believed, for reasons which are not wholly clear, that the Pineal gland, a small rice grain sized gland at the base of the brain, was the seat of the human soul. There is still no evidence that the Pineal gland has any metaphysical significance, however there is plenty of evidence for its role in the hormone system of the body.

The Pineal gland is the source of Melatonin, a powerful hormone which regulates the sleep/wake cycle and which is also chemically related to steroids and a neurotransmitter called Serotonin (http://en.wikipedia.org/wiki/Pineal_gland). It is also called the “third eye” because it appears to be sensitive to light, remarkable given it is buried away just above the roof of your mouth. The Pineal glands' production of Melatonin is increased by darkness and inhibited by light. It also receives nervous input from the retinas of the eyes.

Consultant psychiatrist Emad Salib of Peasley Cross Hospital in St Helens, Merseyside, UK believes that the level of Melatonin is somehow related to the incidence of suicide. He performed a study which linked birth in late spring to suicide. Suicide has some seasonality, peaking in months of maximum daylight. Salib thinks that people born April, May and June who were conceived therefore in July, August and September had their brain development influenced by high level of melatonin during important development periods of their brain.

It all sounds a bit far fetched, but remember that up until relatively recently we had no idea that the cause of stomach ulcers were bacteria. 30 years ago saying ulcers were caused by bacteria would have been regarded by the medical profession as fringe lunacy as it was thought that stomach acid was too corrosive to allow for bacterial growth (http://en.wikipedia.org/wiki/Heliobacter_pylori) .

There are other behavior traits which seem to have seasonal influences related to birth month, as show below (this diagram was taken from a New Scientist article on the subject):

So what about survey taking ? If there is a 45% genetic influence maybe there are some other influences that can be detected ? I have always wanted to analyse panel members dates of birth to see if there is any seasonality effect on panel membership or responding. If anyone has a panel and wants to test this I will gladly do the analysis.

What ever the case, it points to the importance of having some form of Respondent Theory. If we know why people do or don't join panels, or do or don't take surveys it should help sort out the biases due to panel membership.

We now know about the 45% genetic influence, presumably mediated by some sort of personality/biological trait, what about the other 55% ? It would be nice to know.....