Scraping By....

The Wall Street Journal has a long article today about the web and privacy. It choses to dissect a company called TapLeaf and how it collates data on individuals which is then resold.

I feel slightly sorry for RapLeaf, they are getting more attention than I am sure they think they deserve. The truth is the whole “privacy” issue and the web is exploding, and I expect to see a lot more about this in the coming months. Pretty soon the government(s) will get involved, whether we like it or not. We have to hark back to the era when telephone research was king, it was great for a while, then all the calls got out of control. The government took notice. We can point fingers at the direct marketing industry, but we had a hand in it too. Eventually we got the “do not call” list and exemptions for MR, but it was a close thing.

Over the last couple of weeks there has been a long debate in a LinkedIn discussion group (“Text Analytics Professionals”) about the ethics of “screen scraping”, which is downloading or scraping information from websites for use in market research. A company called Buzzback, owned by Nielsen, was caught scraping a website called “Patients Like Me”. It was against the terms of service (TOS) for the Patients Like Me (PLM) website. The debate has gone back and forth, there are those who feel anything on a website is “fair game”, there are those who disagree. Do you obey the TOS or ignore it ?

I think this debate is going to be irrelevant soon. Like it or not, as sure as night follows day, there will be more legislation about web privacy, and I would expect it to cover screen scraping. What the MR community have to do is to make sure it does not get side swiped by this. Social media is now hugely important to MR and we can expect the new rules, when ever they come, to cover information posted on social media. Complaining about the government(s) is all well and good, but we have to be part of the process. And there will be a process.

It seems statistics in MR don't matter.....

I was reading Ray Poynters excellent blog article here: 

http://thefutureplace.typepad.com/the_future_place/ 

about the Likert scale. Ray stated:

“If a researcher makes the wild and unjustified decision (IMHO) to treat Likert numbers as an interval scale...”

I took this as him saying he agreed it was wrong to treat Likert as an interval scale, he did only say that it is “wild and unjustified” to treat them as interval. This is a decision that is made constantly in MR.

I will go further and say it is plain wrong and bad statistical practice to treat Likert scales as interval data. It is an ordinal scale. It should be treated as such. You can't take the mean and standard deviation of ordinal scales. They don't have them. They have a mode (sometimes two modes), they have a median, they don't have a mean.

I got these radical ideas some years ago. After my degree in Psychology and post graduate work I worked in a medical research facility called the “Institute of Neurology” (ION) in London UK. I worked in the “Computing and Medical Statistics Unit” in a basement in Guildford Street complete with IBM card punch machines, cockroaches and CDC UT2000 remote RJE terminal for the University of London Computer Center. I think the drum printer on the UT2000 gave me hearing damage, it sounded like a machine gun.

I worked for a medical statistician called Liz. I was what I called a “data monkey”, I ran programs, wrote them, cleaned data, killed cockroaches and generally helped out. Liz had worked with Sir Richard Peto, who is currently Professor of Medical Statistics and Epidemiology at the University of Oxford. Liz was passionate about her profession. I recall several studies we worked on together. One was a drug trial for a drug to cure Multiple Sclerosis, another was a long term epidemiological study of Multiple Sclerosis, another was a drug trial of a chemotherapy drug to be used against a particularly evil form of brain cancer called Gliomas. Then there were studies about stroke models in rats, muscular dystrophy and Tourettes syndrome. 

We had a strict procedure for all data. First all interval data variables were plotted and looked at. Then these variables were tested for normality of distribution, if they were not normal appropriate transformations were applied to correct any anomalies and then they were re-tested for normality. If they still didn't pass the normality test they were only analysed with non-parametric techniques. Other data variables were plotted too, and the histograms looked at carefully for anomalies. I can recall Liz spending quite some time researching if you could use a T-test on percentages. She concluded you could not. She decided percentages belonged to the Cauchy distribution, which has no mean or higher moments. Thus a T-test would be statistically invalid.

I asked Liz about this procedure of treating data and the rigour she applied. I came from psychology, we were a little more lax in our approach. She said we had to remember that the results we obtained mattered. They could be life and death decisions. A type I error on a drug trial could lead to more people dying because they were given a drug that didn't really work. A type II error on the epidemiological work may miss an important antecedent to Multiple Sclerosis, a crippling disease. 

We were working on data for the drug trial for treating Gliomas, a form of brain cancer. Gliomas remain a deadly form of cancer, with only a 50% survival rate within one year of diagnosis. We were using something called Survival Analysis to test the effect of a chemotherapy drug. Liz said we had to wait for more events to be sure of the results. An event was someone dying. I happened to look at the columns in the data which contained the ages of the subjects. 18, 19, 21 – these were people only a little younger than I was at the time. We had to wait for them to die to be sure the conclusions that were made about the drug (Vincristine) were correct. The results really mattered.

It seems to me the question about Likert scales is not so much about if you can treat them as an interval scale rather it is this: do the results matter ? Do you care that the results are correct ? Do the results matter enough to do the work properly ? If the results do matter, do it properly.

From what I can see very often it seems market researchers think the results don't matter......

Surveys, Genetics.....and Descartes..

A study from the University of North Carlolina says that survey taking is a genetic trait. They studied twins and determined that there was a 45% influence of genetics on survey taking. The study has been reported in numerous places, here for instance http://bit.ly/cG9E2d .

Is this finding so surprising ? It means that the remaining 55% of “influence” affecting survey taking is unaccounted for or attributed to environmental effects. Biologists know that genetic predispositions account for some behavior, so it is not so surprising that survey taking is influenced by genetics. Of course the study doesn't identify exactly how genetics influence survey taking. We never seem to hear that trait X predisposes people to take or not take surveys. It is a step in the right direction though. It is a step to what I call “Respondent Theory”, that is a theory of why respondents take surveys.

For psychologists interested in the antecedent of behavior there has long been the “nature vs nurture” debate. Genetics vs environment. It is a bit of a false debate though, as it is clear there is complex interactions between the two influences. It isn't just one or the other.

This is where Descartes comes in, sort of. He is probably remembered best for his dictum “I think therefore I am”. He also believed, for reasons which are not wholly clear, that the Pineal gland, a small rice grain sized gland at the base of the brain, was the seat of the human soul. There is still no evidence that the Pineal gland has any metaphysical significance, however there is plenty of evidence for its role in the hormone system of the body.

The Pineal gland is the source of Melatonin, a powerful hormone which regulates the sleep/wake cycle and which is also chemically related to steroids and a neurotransmitter called Serotonin (http://en.wikipedia.org/wiki/Pineal_gland). It is also called the “third eye” because it appears to be sensitive to light, remarkable given it is buried away just above the roof of your mouth. The Pineal glands' production of Melatonin is increased by darkness and inhibited by light. It also receives nervous input from the retinas of the eyes.

Consultant psychiatrist Emad Salib of Peasley Cross Hospital in St Helens, Merseyside, UK believes that the level of Melatonin is somehow related to the incidence of suicide. He performed a study which linked birth in late spring to suicide. Suicide has some seasonality, peaking in months of maximum daylight. Salib thinks that people born April, May and June who were conceived therefore in July, August and September had their brain development influenced by high level of melatonin during important development periods of their brain.

It all sounds a bit far fetched, but remember that up until relatively recently we had no idea that the cause of stomach ulcers were bacteria. 30 years ago saying ulcers were caused by bacteria would have been regarded by the medical profession as fringe lunacy as it was thought that stomach acid was too corrosive to allow for bacterial growth (http://en.wikipedia.org/wiki/Heliobacter_pylori) .

There are other behavior traits which seem to have seasonal influences related to birth month, as show below (this diagram was taken from a New Scientist article on the subject):

So what about survey taking ? If there is a 45% genetic influence maybe there are some other influences that can be detected ? I have always wanted to analyse panel members dates of birth to see if there is any seasonality effect on panel membership or responding. If anyone has a panel and wants to test this I will gladly do the analysis.

What ever the case, it points to the importance of having some form of Respondent Theory. If we know why people do or don't join panels, or do or don't take surveys it should help sort out the biases due to panel membership.

We now know about the 45% genetic influence, presumably mediated by some sort of personality/biological trait, what about the other 55% ? It would be nice to know.....

Location, Location, Location

The old realtors adage of “Location, Location, Location” seems to be applying to social media. Facebook have announced their new places service (http://www.facebook.com/places/). It's available on the iPhone Facebook app or via touch.facebook.com . Of course it's nothing new , sites such as Foursquare.com have been around for a while, offering the same geo-locational social networking. It is important not to dismiss geographic social media (geosocial media). Foursquare is everywhere. I live in Cincinnati, hardly a teeming metropolis, but pretty much everywhere I seem to go has a Foursquare “major”. A major in Foursquare terms is a status given to someone who visits a place the most. Even my neighbors house had a major, which turned out to be someone no one knew. A friend of mine traveled across the USA and he said Foursquare was everywhere he went too. This is not some passing trend. Now that Facebook has go into the scene it will only grow.

There is the privacy paradox of course. On the one side there is a lot of noise about not wanting to tracked by your friends, about how criminals can use the services to work out which houses to burgle. The flip side is that it is everywhere and there seem to be a lot of people using it. So I think the privacy arguments are simply not strong enough, maybe it just means you have to be careful about who your friends are, not bad advice in general.

The potential for businesses of this type of geosocial media are huge. Many types of businesses don't really know their clientele very well, but this is a way to know who comes to your business and more importantly to know when they arrive. It offers all sorts of potential for surveys and rewards. I was struck at a market research conference a couple of months ago how few people there knew about geosocial media. If the geosocial media takes off we have a new way to find out what people are doing, and when they are doing it. It seems it will pay to know where your friends are....

The Problem with Neuromarketing.....

There are several problems really. The first is the name. It should really be called applied cognitive neuroscience (ACN), because that is what it is. Hopefully this would counter all the specious arguments about it being scientific. The New Scientist (http://www.prnewswire.com/news-releases/neurofocus-and-new-scientist-magazine-apply-neuromarketing-to-select-cover-design-100053049.html) test raised some comments about the science of ACN. I have to disclose in what seems like a previous life I studied cognitive neuroscience (we called it neuropsychology back then). Trust me, it is a science, it has been around a *long time*, many decades, and it also uses statistics correctly. This latter fact is a novelty for a lot of MR, I know. At least ACN tries.

There is also the privacy "discussion". ACN is about as invasive as looking at someone who is blushing and deducing they may be embarrassed. ACN measures physiological correlates of mental states or processes. It happens to do them via electrical signals measures from the brain. We do this all the while with body language, speech tones and so on. Just because there is a lot of equipment in ACN and latin words doesn't make it any different.

The biggest problem ACN has is sample size. N = 19, as in the New Scientist test, isn't much. It is barely enough for a single quota cell. Making big decisions based on tiny samples mostly ends in tears. The sample size issue relates to the technology of ACN. The electrodes on the scalp can take time to set up and this limits sample sizes. However several companies have ways round this with either limited electrode placement (not so good - one electrode gets you nothing except muscle noise) and less "invasive" caps that hold the electrodes on the scalp without glue. The latter holds the most promise so far as I can see. Sample size is a solvable problem, scalability may take time, but compared to the rest of the technology used in ACN it is not the most complex problem. Several companies are building normative databases which will be hugely useful.

The problems with ACN are solvable, the potential is huge...

Come back Whorf and Sapir: all is forgiven ?

"Language is not a cloak covering the contours of thought,
Languages are molds into which infant minds are poured..."

The above is about the most prosaic statement of the "strong" form of the Whorf-Sapir hypothesis (WSH), also known as linguistic relativism (http://en.wikipedia.org/wiki/Linguistic_relativity) . This quote is from Brown 1954 (I think - I know there was snow in it anyway). This is all I remember of the reference except it had to do with Eskimo's (non PC back then) and how they "saw" snow differently because they had 12 words for snow. I only have one or two words for snow and I use them preceded by a word that begins with 'f' generally.

The Wall Street Journal rediscovered WSH this yesterday with an interesting article about Spanish, Japanese and English cultures and how the eye witness reports varies of an event which the authors attribute to language effects (http://ht.ly/2hJy7) . They seem to be stating what is called the strong version of the Whorf Sapir hypothesis - again. The problem is that a lot of work has been done on the WSH and the general conclusion has been that language is "necessary but not sufficient" for complex cognition. The Wikipedia article covers the area quite well, the conclusion is that the weak version of the WSH is probably true, but we have to accept that many aspects of cognition are universal and independent of the language used. One area that had a lot of research was color naming, different cultures have different ways of naming colors, from two (for instance "dark" and "light") to the vast range we have. Yet you will find that core colors, so called focal colors, are the same across cultures. For instance focal red (the "reddest red" ) is the same across many cultures. The invention of aniline dyes had a significant effect on color names, they became abstract rather than object referenced. It's an old debate, and I thought the authors in the WSJ should at least *mention* the WSH. But probably not - too inconvenient for their hypothesis.

It just shows that everything comes around again, and while it makes good copy for the WSJ a little bit of research may have provided a more accurate article. The WSH debate is interesting though.

I always thought that David Bohm's book "Wholeness and the Implicate Order" (http://en.wikipedia.org/wiki/Wholeness_and_the_Implicate_Order) had a very beautiful account of the Hopi indian language relates to his ideas about Quantum mechanics and how we see the world, and the merging of time. Now that is another way of looking at reality....

Social Evolution...there are friends and friends....

My late father once told me that anyone who counts his friends on more than one hand is either very lucky or a fool. I recalled this when reading some research posted on Slideshare (http://www.slideshare.net/padday/the-real-life-social-network-v2) by Google on the evolution of social networks and media. The deck is 216 slides, but very well worth the time to read it. It points out that we can't all be friends, much as we would like to be.

About 18 months ago I "unfriended" on Facebook a number of people who weren't, well, friends. They were all people I knew through business, people I like, people I like to see and talk to. But just not personal friends. The Google research points out that we have very few close friends, we may have a lot of people who we keep in contact with though. So I adopted a policy of Facebook for personal contacts and friends with LinkedIn for business contacts. Twitter is sort of mixed at the moment, Foursquare too. It's not that I am being anti-social it is just that this mirrors the way we lead our lives.

The problem with most social media is that it assumes our personal relationships are homogenous. They aren't. And for social media to advance it has to take that into account. We partition relationships, we have to. If we told everything we know to everyone we know the world would simply not work. However that is what Facebook assumes. Pretty much there is one channel to communicate with social media sites, you can't partition friends up to different group as we do in real life. We mix our groups of friends in the real world carefully if at all, most social media now assumes they mix easily and freely. In order for sites like Facebook to grow more they have to reflect the real structure of society, which means the way social networks (which have existed for thousands of years) are structured. In a sense location based social media (like Foursquare) is an evolution of this, where we are in the world also partitions our social networks, and adding that locational dimension to online social media is a real advance. Another adage from my past is that we are the company we keep. Well hopefully....

Crowd Sourcing: The real wisdom of crowds ?

I'm not very convinced of the “wisdom of crowds”. There are numerous examples of how “the wisdom of crowds” is in fact the “idiocy of the mob”. Look at some political movements or some of the more extreme religions for instance, a good few of these make no sense, but they have a lot of people who believe them. In Vanatu, an island in the Pacific, there is a cargo cult called the “John Frum” cult who think that building replicas of USA air force bases from World War II will bring the USA and all their goods back to the island. A lot of people believe this.

There is a lot of research from social psychology showing that groups polarize decisions in contrast to individuals. A group will make a more extreme decision (cautious or risky) than an individual. There is also the fact that estimations of physical sizes and weights will tend to show a normal distribution, with the most common estimate, the mode, being the correct once. Here there is wisdom in crowds, or more like the wisdom of the normal distribution, the central limit theorem and statistics in general. Distributions are wonderful things.

One of the aspects of a lot of people being asked a question is that you are able to take advantage of a lot of peoples experience and knowledge. Recently a company called “Netflix” in the USA used the web and the crowds that access it to solve an interesting problem. While it is not the usual meaning of the term the “wisdom of crowds”, it is an example of the crowd can solve the problem. Netflix (www.netflix.com) rent DVD's of movies and TV programs to people. They send them via the mail and you maintain a list of which DVD's you want. They also try to predict which DVD's people who like to watch based on the DVD's they have already watched. Amazon does a similar thing with recommendations of products for people to buy. Netflix wanted to improve their predictive algorithm by 10% , which is quite a large improvement. They could have tried to hire all sorts of geniuses, but they chose a very unique way to solve the problem. They set up a web site (www.netflixprize.com) , posted a huge data set of movie DVD's, data about the movies and subscriber choices. They then offered $1,000,000 to anyone who could improve their algorithm by 10%. There were a few conditions, there was a deadline (September of 2009) and anyone who submitted a solution had to agree to document what the solution was publicly. Many companies allowed their employees to set up teams and compete, some individuals competed, and teams merged and re-formed over time. In the end there was a winning team, “Bellkors Pragmatic Chaos” .

In this case the wisdom was not “crowd think”, whatever that is. It was that Netflix leveraged the web and all the people who constantly look at it to get hold of people who wanted to solve this problem. For them the $1,000,000 was cheap. They could never have afforded to hire all the people who took place in the contest. They got access to world class computing facilities and minds and got some great publicity as well.

The winning algorithm was a technique called a “Restricted Boltzmann Machine”. It proved that numbers and math matter. It wasn't the crowd that solved the problem, but the crowd was the mechanism that made the solution possible. I'm inclined to think that this is the real wisdom of the crowd. People can come up with all sorts of strange beliefs, the ability to get people to address your problem is the wise part of the crowd. It's another of those examples of how the web changed the world in a radical way. Twenty years ago it simply would not have been possible. I gather there is going to be another Netflix prize. It's nice that it was the math that was wise in the end....

Reward the chain: Incentives for Surveys using Social Media

So here's the thing. All those nice shiny, familiar email addresses we use to send survey notifications are decaying, they are losing value by the moment. I've had an email address since 1984, and very little in my life dates back from then now (well except me that is). A couple of weeks ago my primary email address stopped working (I didn't pay the hosting bill on time). I had a moment of panic and then I realized: it really didn't matter that much. Personal friends are pretty much all on Facebook, business contacts via LinkedIn, anyone who really needs to get to me fast has my mobile number.

Email is so over.

Obviously the next way to get to people for surveys are via mobiles or social networks. It struck me that the usual model of incentives for respondents doesn't fit the social media world very well. We want people to do the survey via social media sites AND pass on the link. There has to be an incentive for the latter to happen, let's be reasonable, people want to get paid for helping.

We need to come up with something that will reward respondents for sending the survey link on to their friends as well as them completing it. How about trying to reward people for reposting based on the number of people they repost to (hard to track I would guess) ? Or if they or someone they repost to is the Nth complete of the survey ? So if you repost the link and you or one of your friends is the 10th, 100th, 1000th et al complete you get money/something ? Maybe report how long your "chain" of completes are ? It would be a sort of survey incentive pyramid scheme. Is anyone doing this ? Anyone have any other ideas ? So far the whole social media and surveys melange has not totally taken off. Viral dissemination is nice, but we need to be able to push to a lot of people....