I can assure you that measuring what people actually do when they use Trade Me is a very important part of the development of the site. I will devote an entire post to this later in this series.

However, what I’m really talking about here is trying to understand users in more than a mathematical sense.

It’s more than just trying to maximise their behaviour, it’s trying to get inside their head and understand what they like and want and expect.

It’s understanding what causes pain and frustration and then discussing possible changes that could alleviate those.

Often times these types of things are not recorded in the hard evidence of page views and clicks. But when you spend some time with people and talk to them they usually quickly become apparent.

Another even more effective way for us to understand people is to track what they actually do.

Rowan, I am not sure if the way Trade Me is trying to get measurement from the users is the right way. Amazon doesn’t do it that way, they use data-mining and experimentation to find out how users are interacting with their site and then adapt to it. It would be impossible for Amazon to interview a large number of users to find out how their site is to be organized. Here is a reason, even if you manage to interview and get feedback from a few hundreds of users (say, 500), that number is statistically insignificant to represent the huge volume of users that are surfing Trade Me site, so to say that useful information had been gathered from 500 users is enough to represent the 300000 or so total users. The number 500 is so small where any information gathered from observing them is like anecdotal evidence and not evidence based on statistical hypothesis testing. This is exactly how Amazon is adapting their site is to experiment and use statistical hypothesis testing to validate if it is indeed true that one design is more statistically significant over another one. The decision based on small sample analysis (such as 500 users), is dangerous because it could be biased (skew or non-normal distribution), and usually lead to wrong conclusions.

I wrote a technology recommendation report for Ferrit last year 2006, but I am not sure if they have implemented any of those. I covered the major technologies and algorithms that Amazon is currently implementing based on a number of published research papers about the way how Amazon works, mainly from the Journal of Machine Learning and Knowledge Discovery & Data-mining and SIAM Data-mining Journal.

Perhaps Ferrit thinks that the way Amazon works doesn’t suit of how they want Ferrit to operate.