Wednesday, July 28, 2010

Come back Whorf and Sapir: all is forgiven ?

"Language is not a cloak covering the contours of thought,
Languages are molds into which infant minds are poured..."

The above is about the most prosaic statement of the "strong" form of the Whorf-Sapir hypothesis (WSH), also known as linguistic relativism (http://en.wikipedia.org/wiki/Linguistic_relativity) . This quote is from Brown 1954 (I think - I know there was snow in it anyway). This is all I remember of the reference except it had to do with Eskimo's (non PC back then) and how they "saw" snow differently because they had 12 words for snow. I only have one or two words for snow and I use them preceded by a word that begins with 'f' generally.

The Wall Street Journal rediscovered WSH this yesterday with an interesting article about Spanish, Japanese and English cultures and how the eye witness reports varies of an event which the authors attribute to language effects (http://ht.ly/2hJy7) . They seem to be stating what is called the strong version of the Whorf Sapir hypothesis - again. The problem is that a lot of work has been done on the WSH and the general conclusion has been that language is "necessary but not sufficient" for complex cognition. The Wikipedia article covers the area quite well, the conclusion is that the weak version of the WSH is probably true, but we have to accept that many aspects of cognition are universal and independent of the language used. One area that had a lot of research was color naming, different cultures have different ways of naming colors, from two (for instance "dark" and "light") to the vast range we have. Yet you will find that core colors, so called focal colors, are the same across cultures. For instance focal red (the "reddest red" ) is the same across many cultures. The invention of aniline dyes had a significant effect on color names, they became abstract rather than object referenced. It's an old debate, and I thought the authors in the WSJ should at least *mention* the WSH. But probably not - too inconvenient for their hypothesis.

It just shows that everything comes around again, and while it makes good copy for the WSJ a little bit of research may have provided a more accurate article. The WSH debate is interesting though.

I always thought that David Bohm's book "Wholeness and the Implicate Order" (http://en.wikipedia.org/wiki/Wholeness_and_the_Implicate_Order) had a very beautiful account of the Hopi indian language relates to his ideas about Quantum mechanics and how we see the world, and the merging of time. Now that is another way of looking at reality....

Wednesday, July 14, 2010

Social Evolution...there are friends and friends....

My late father once told me that anyone who counts his friends on more than one hand is either very lucky or a fool. I recalled this when reading some research posted on Slideshare (http://www.slideshare.net/padday/the-real-life-social-network-v2) by Google on the evolution of social networks and media. The deck is 216 slides, but very well worth the time to read it. It points out that we can't all be friends, much as we would like to be.

About 18 months ago I "unfriended" on Facebook a number of people who weren't, well, friends. They were all people I knew through business, people I like, people I like to see and talk to. But just not personal friends. The Google research points out that we have very few close friends, we may have a lot of people who we keep in contact with though. So I adopted a policy of Facebook for personal contacts and friends with LinkedIn for business contacts. Twitter is sort of mixed at the moment, Foursquare too. It's not that I am being anti-social it is just that this mirrors the way we lead our lives.

The problem with most social media is that it assumes our personal relationships are homogenous. They aren't. And for social media to advance it has to take that into account. We partition relationships, we have to. If we told everything we know to everyone we know the world would simply not work. However that is what Facebook assumes. Pretty much there is one channel to communicate with social media sites, you can't partition friends up to different group as we do in real life. We mix our groups of friends in the real world carefully if at all, most social media now assumes they mix easily and freely. In order for sites like Facebook to grow more they have to reflect the real structure of society, which means the way social networks (which have existed for thousands of years) are structured. In a sense location based social media (like Foursquare) is an evolution of this, where we are in the world also partitions our social networks, and adding that locational dimension to online social media is a real advance. Another adage from my past is that we are the company we keep. Well hopefully....

Monday, July 5, 2010

Crowd Sourcing: The real wisdom of crowds ?

I'm not very convinced of the “wisdom of crowds”. There are numerous examples of how “the wisdom of crowds” is in fact the “idiocy of the mob”. Look at some political movements or some of the more extreme religions for instance, a good few of these make no sense, but they have a lot of people who believe them. In Vanatu, an island in the Pacific, there is a cargo cult called the “John Frum” cult who think that building replicas of USA air force bases from World War II will bring the USA and all their goods back to the island. A lot of people believe this.

There is a lot of research from social psychology showing that groups polarize decisions in contrast to individuals. A group will make a more extreme decision (cautious or risky) than an individual. There is also the fact that estimations of physical sizes and weights will tend to show a normal distribution, with the most common estimate, the mode, being the correct once. Here there is wisdom in crowds, or more like the wisdom of the normal distribution, the central limit theorem and statistics in general. Distributions are wonderful things.

One of the aspects of a lot of people being asked a question is that you are able to take advantage of a lot of peoples experience and knowledge. Recently a company called “Netflix” in the USA used the web and the crowds that access it to solve an interesting problem. While it is not the usual meaning of the term the “wisdom of crowds”, it is an example of the crowd can solve the problem. Netflix (www.netflix.com) rent DVD's of movies and TV programs to people. They send them via the mail and you maintain a list of which DVD's you want. They also try to predict which DVD's people who like to watch based on the DVD's they have already watched. Amazon does a similar thing with recommendations of products for people to buy. Netflix wanted to improve their predictive algorithm by 10% , which is quite a large improvement. They could have tried to hire all sorts of geniuses, but they chose a very unique way to solve the problem. They set up a web site (www.netflixprize.com) , posted a huge data set of movie DVD's, data about the movies and subscriber choices. They then offered $1,000,000 to anyone who could improve their algorithm by 10%. There were a few conditions, there was a deadline (September of 2009) and anyone who submitted a solution had to agree to document what the solution was publicly. Many companies allowed their employees to set up teams and compete, some individuals competed, and teams merged and re-formed over time. In the end there was a winning team, “Bellkors Pragmatic Chaos” .

In this case the wisdom was not “crowd think”, whatever that is. It was that Netflix leveraged the web and all the people who constantly look at it to get hold of people who wanted to solve this problem. For them the $1,000,000 was cheap. They could never have afforded to hire all the people who took place in the contest. They got access to world class computing facilities and minds and got some great publicity as well.

The winning algorithm was a technique called a “Restricted Boltzmann Machine”. It proved that numbers and math matter. It wasn't the crowd that solved the problem, but the crowd was the mechanism that made the solution possible. I'm inclined to think that this is the real wisdom of the crowd. People can come up with all sorts of strange beliefs, the ability to get people to address your problem is the wise part of the crowd. It's another of those examples of how the web changed the world in a radical way. Twenty years ago it simply would not have been possible. I gather there is going to be another Netflix prize. It's nice that it was the math that was wise in the end....