Data envy

There’s an interesting article on I, Cringley about Apple’s latest video ipod and more importantly a study on Google’s AdWords. I think it’s fascinating that Google has risen to the level that people are spending this much time analyzing the way that they work. I also just finished reading Battelle’s book Search that discusses some similar topics. Battelle talks about Google having a pulse of the entire world since it is able to see what the world is searching for at any given time. He argues that search queries are the best view we have inside people’s mind to their true desires. I agree that of all the ways that we have to watch people from a distance, search terms are probaly the best way to take the world’s pulse.
I believe that Google is in a truly envious position. Having access to this data, the computing power to process it, the ability to hire the best people, and then the abilty to give those people the time to sift through it is amazing. Being able to study and test economic theories in a very live and large lab is unprecedented.
Cringley talks about a study of AdWords, where some customers try and optimize their performance using economic theories. He hints at an underlying false assumption, that the Google Adwords system is optimized for customer performance, where it is much more likely optimized for Google’s performance. I hope that Google is sticking to their “Don’t be evil” theme, but it’s so tempting for corporations when there’s so much money involved and since this is Google’s cash cow funding everything else they do. However, as a software engineer I think another assumption that probably is incorrect is that the AdWords system is static at all. There are a very large number of smart people over there all trying to put their stamp on creating the best product possible. This calls for constant modification and optimization. I have no doubt that the algorithms are being adjusted daily and that they are trying out different theories.
My only hope is that with all the possiblility for true science to come out of the experiment that is Google, that some of it gets published for those of us that don’t work within the campus.