What Google knows about you? And why it will rule the world?

I didn't want to put the following statement at end of post, wanted to get the message out first.

"I'd complete control over the personal data storage settings in Google's products and still trust Google more than any other company to store my personal information."

With that statement out of the way, now to the main topic.

Google has released so many products that it's really tough to keep rack of all of them, even if you are an ardent Google fanboy. These products are feature rich and knowing all the features of all the products of Google is next to impossible for one individual.

This post is about one such feature: "Search History" (not sure if it's a product).

Like any Google fanboy I tried out "Search history" immediately when it was launched, played with it and after sometime completely forgot about it. The "Search history" has been renamed as "Web History".

To access /activate "Web History" you need to visit this page. Google account owners can access through 'My Account" after "Sign in". "Web History" is listed under "My Products".

Recently I was checking some other stuff in "My Accounts" and accidentally checked "Web History". I was accessing "Web history" after a period of 3 1/2 years!. Though "Web history" was released only on 4/19/2007, I'd data since 2005 as I'd previously signed up for "Search History".

Here are some of my personal data as recorded by "Web Hisory"

• I'd done 3,040 Google searches in 3.5 years. I expected the numbers to be high as I'm hooked to internet most of the time and Google is my default search engine.

So, I started to dig into the data. I picked a month (September) and checked number of searches done over last 4 years. The following graphs shows the trend line.


What do you notice?

• Sudden jump in 2008. In fact ~1500 out of total 3,040 searches were done in 2008 and still 3 more months go!

What is the Inference?
Does it mean I've become more clueless in 2008 and need Google to figure things out?

NOPE! My cluelessness is constant :-)

The unarguable reasons are : -

• "Web history" can record data only if I'm logged in. Its only in 2008 I've started to use "Gmail" more regularly (logged into it most of the time).
• Nobody will exclusively log in (Sign in) to do searches.
  
• However if you are already logged into one of the Google services, cookies are set to automatically sign in users to search service.
• Being signed in is very important. (I'll come back to this point)

The coolest and scariest feature within "Web History" is the "trend" feature. This feature gives information like : -

• Shows My top 10 queries, top clicks, sites in last 1 year, 30 days and 7 days.
• Tells me that I've searched more during week days, however this is changing, even during weekend my search usage is increasing.
• I've done search during all hours of a day, from 0 to 24hrs. Semi insomniac!
  

Other information about me that surprised me.

• I've just clicked only 4 advertisements (Google Ads). Certainly not a profitable customer!
  
• Shows all videos I've seen
• Lists News that I've followed in last 3 1/2 years
• Traces Routes and Map searches I've done
• Books I checked out.
  
• Blogs that I've followed

I know about the popular Google's disclaimer "No human at Google, ever reads user data". So, I can't blame that a Google engineer might know about me. However Google's code (Algorithm) does surely know more about me than I've cared for.

With this data and advanced algorithm, Google's code will surely know : -

• What is the best time to connect with me?
• What is the best time to sell?
• And what to sell to me?

User relevant Data is important, without this data, no matter how advanced the algorithm is, intelligent and useful services can't built.

Relevant user data can be collected only if the user is logged in. The personal data collected is the insurance against any new algorithm or massive data center.

The relevant user data is the 'positive network effect" similar to platform network effect that made Microsoft so powerful.

That is why Gmail is very important and central to Google's service. It enables automatic "sign in" for all its service. It is also equally important to have users always logged in so that user specific information can be collected all the time. That means user should not close Gmail, but let it run in the background.

Lets look at some of recent Google's initiative.

• Gtalk in Gmail. It makes sense to have Gtalk built into Gmail. Always logged in users.
  
• Google Chrome makes running rich application as long as needed without the browser hanging. This enables running "logged in" applications like Gmail as long as needed. If Gmail is logged in most of the time, more user specific information can be collected. (Can you guess why Chrome was built? ofcourse there can be many more reasons)

I'm not sure if there is any method to Google's product or feature release madness. All I can say is that Google power is growing greater than normal users or business can imagine.

Only one company can compete with Google - Microsoft. However the "Window" of opportunity is closing faster ;-)

A great American financial crisis - Emperor's new cloth!

A friend of mine asked me to explain the reason behind the current American financial crisis. This post is an attempt at it.

Non controversial and simple explanation: Banks and Financial institutions that are part of this crisis (most of American Banks/Financial institutions) had lots of debts to pay and they did not have enough money to pay :-) That's it!

Reason is simple, but the real difficult question is, how did the Banks and Financial institution get into this messy situation?! Especially if one considers the following facts.

1) These institutions attracted the brightest people
2) Offered highest remuneration
3) Had all the support of government and policy makers (remember we're living in a relative world)
4) No major world disasters, no world war, no calamities, no terrorism (9/11)
5) These institution did not do illegal practice, no frauds, no drug trafficking, arms trade etc. As per books they all had fair business practice
6) World economies were booming until the point of failure.

Then how come this sudden heart attack?!

Well, the last word has the key - "heart attack" affects the 'over' healthy- obese people rather than people who are starved.

Now moving towards analyzing the cause of this sudden financial heart attack. I believe its because of the following 4 reasons. (Disclaimer: these are not researched opinion just another Sunday afternoon thoughts)

1) Emperor's new cloth
2) 'Messy' complexity : Domino effect
3) Social trap : Nazi concentration camp
4) Too much power in one place: Alavuku minjinaal amudhamum nanju - A Tamil proverb which translates to "Over the limit even nectar becomes a poison"

Diving into more details. I'll be using word 'Bank' to denote = Banks and all financial institutions that includes even the glamorous Investment banks.

1) Emperor's new cloth

During early days of banking, Banks offered simple services e.g safe guarding one's money and then moved on to offer lending (loans) etc.

Even buying was simple, early days if you had money you bought stuff or saved money to buy, however this changed to credit cards and EMIs.

Trends shows new products being invented by Banks to increase their service offering and thus fueling their growth. Note, current corporate culture is focused mainly at growth. This puts huge pressure year on year to grow and the going only gets tough as the size of the company increases. Its easy to grow from 1 to 5 than from 5000 to 25000, both has same percentage (500%) growth.

This natural pressure to grow fuels creation of 'innovative' financial instruments, two such examples are Derivatives and Hedge Funds.

These and many new products were churned out by banks/financial institution. These product were not understood (real risks/benefit) by most people but who wants to be seen as stupid - Just like the Emperor's new cloth.

There are exception - Warren buffet calls derivatives a 'time bomb'

2) 'Messy' complexity: Domino effect

Level 1:

In good old days it would have been easier for the bank branch manager to look into books and sort things out if there were any transaction flaws. Now a top manager in any bank can't help without an army of database administrators, system administrators, business analyst, domain manager and support staff working on to address certain flaws.Technology that drives the systems in these Banks has added to the complexity. It is now beyond a human capability to understand the entire system of even a single bank.

Level 2:

The financial product have became interwoven, repackaged and re-sold to others. No body understands the fundamentals of these products. I'm not denying that there weren't any shady practice like wrong rating by credit firms etc., fact is that I don't know about them so I'm leaving it out of the analysis.

However I'm convinced that even if you put the smartest mind, now, to dig out the risks of these products he/she will do a very poor job. There is just too much inter linking and assumptions.

Level 3:

So much interwoven and complex were these products and services that one event like 'sub prime debacle' had a domino effect that is threating to pull the entire financial system down.

Why the word 'messy'?

Truly complex system are simple to interact e.g the Google search system, you don't need to spend hours and hours to understand how to use Google. However building a Google like search engine is complex. (I tend to give Google as example, only because its an easy example)

3) Social trap: Nazi concentration camp

I've read about the greed of the Wall street people, I would strongly argue against it. Even if you or me were in the place we would have done the same thing because it was considered the smartest and typically a great business sense.

The power of social trap is so strong that even high IQ and highly paid people can be made to do stupid things without their knowledge.

The social trap is also one of the reason for bizarre human behavior like concentration camp, motivating suicide terrorist, Hindu's attack on churches etc

The competition to grow and harness opportunity was a Zero Sum Game that all the banks had to play otherwise they would have supposedly lost business. It was considered normal as everyone were into these kind of products and service.

4) Too much power in one place: Alavuku minjinaal amudhamum nanju - A Tamil proverb which translates to "Over the limit even nectar becomes a poison"

It is easy to make more money with money than anything else and Wall street was the shining example of this. People at Wall street made more money than any hard working researcher, entrepreneur, manufacturer or worker.

The power of Wall street was so huge that CEOs and company's board made short term decision that would please Wall Street.

Folklore has it that Google had avoided going to IPO for long time because of the fear of power of Wall Street on its free decision taking.

The Wall Street was beyond the regulators and could decide fate of many companies. The bankers and financial honchos ruled the engineers, workers, entrepreneurs, creators and became the heart of the economy.

Concluding remarks:

Real growth will only come by scientific discoveries, technology breakthrough, creation, enriching the culture, Art, building a healthy community and producing clean and green goods and not by manipulating numbers, power point, conferences or networking events.

More quotes from Prof B.M Hegde's Oration

This post lists some of the interesting quotes from Prof B.M Hegde's oration. (I'm not sure if these are his own quotes. I heard it from him first)

"Bad teacher preaches,
good teacher teaches,
great teacher reaches"

"Competition with others is mediocrity,
competition with self is excellence"

"Tree with fruits bends low"

Which loosely translates to: people with true skills are humble.

"There is no pill for every ill,
but definitely there is ill for every pill"

"Pill thrills but kills"

About pain killers

"Eat 6 times a day - eat anything, but in limited quantity"

"Good health = Capacity to love and work"

"If you need justice, don't go to court
If you need honor, don't go to police station
If you need good health, don't go to hospital"

"Leading cause of death

1. Modern Medical treatment
2. Heart attack
3. Cancer

These are results of a scientific study"

" For ~1400 years human believed blood was pumped by liver

This concept was first published by Greek physician Claudius Galen (A.D 131 -200 "

The new paradigm that heart pumps the blood was discovered by English physician William Harvey. Human civilization has been through many such paradigm shift, earth is flat, sun/moon revolves around earth etc. Dare to question the popular knowledge/belief.

" Be wise as a serpent
and harmless as dove"

He quoted from the Bible.

Human body and non-linear mathematics.

Didn't I promise more articles that justifies the title of this blog!

Recently I attended Prof B.M Hegde's oration, I was forced to attend this, but eventually glad that I was forced.

B.M Hegde was former vice-chancellor of Manipal University. The oration was an eye-opener to the challenges faced by modern medicine. This post is not about the oration, the medical challenges or the brilliant speaker that Prof B.M Hedge is.

It is about one particular theory (actually hypothesis) that the oration touched upon.

"...this talk will point out the fallacy in the very basis of modern medical science (the linear mathematical laws). Human body, in modern medicine, is treated as simple electro-biochemical machine like a motor car engine. Unfortunately, the human body is only a non-linear chaotic dynamic system continuously run by food and oxygen."

Prof B.M Hegde shared that David Eddy (Stanford professor of Cardiac surgery) resigned his job to study mathematics to give modern medicine a better mathematical base for non-linearity.

Prof also said that 30 years research on non-linear working of the heart will be presented soon and went on to say that if the heart were to be simple muscular pump pushing blood into the vast swimming pool of millions of miles of capillaries which are so narrow that even a single red blood cell cannot pass through without changing its shape, it would have stopped functioning almost when we were born! There is a lot more to it than what we have been thinking so far! The present concept of heart as muscular pump makes a mockery of all known physical laws of the universe.

I've been making related statement to my friends circle for a while: "All human scientific advancement so far has been made based on linear mathematical model, however all future advancement will be made by understanding of non-linear mathematical models"

When I mean all...I mean all - electrical, computing, civil, mechanical and medicine (you could add all streams of education). I added medicine after listening to prof's oration, it re-enforced by own understanding that indeed its all fields of study. All these field's goal has been to build a predictable system (linear system).

Look around, gadgets, white goods and most stuff used in every day life is based on linear mathematical model except love, happiness, sorrow, fun and sensex !(list is not complete)

Just in case you need to know what is linear and nonlinear system?

What is linear system (mathematics)?

- Linear
- Linear system

What is non-linear system?

- Nonlinear system

Ain't there any popular example of non-linear system. Yes!

Google is the first glimpse of true non-linear man-made technical advancement and of course Sensex. However the credit will still go to Google as Sensex is not build on non-linear system, in fact it completely relies on linear system. Number of share in one's DMAT account doesn't change randomly, only the value of shares change to reflect market value, but number of share is same unless one buys or sells shares, it is just the market dynamics which is unpredictable.

But the truth is we're surrounded more by non linear systems than linear - e.g Traffic pattern, Disease spread, Progression of Rumors, City's growth, audience clapping for a performance, Star hysteria, Climate, fashion trends....countless.

Its just that we don't see them yet as non-linear mathematical models.

Why the word 'Non Linear' in the blog title?

Starting a blog is easy, blogger and wordpress, makes it a no brainer. Sustaining it, posting regularly, is very difficult. Yeah, every blogger knows it.

The main challenge is getting the theme, style and purpose of the blog right. One needs to answer: Why? for whom? Is there enough to be written?

Lets come to my blog, with a weird blog title like 'non linear media', I certainly can't post about the jolly good time I had with my school friends during annual get together or why I almost had got into road rage when crazy two wheeler bumped into my car?

Yes, I know there is not even one follower for my blog! So I can write whatever I can/want and it will not matter, but ya 'now wat, it doesn't feel right nor its fun, so the motivation gets killed.

So, when a friend suggested that I can overcome blogger's posting block (BPB) by changing the title and starting afresh, I gave it a hard thought. I'd only 2 posts in '08 and 1 post in '07.

After much thinking I couldn't let go of my love for "Non linear" and "Media", however I changed everything else the template, look and feel, additional information, tagging etc.

I added the tag line "Sharing my love for non linear mathematics, media and technology" and suddenly I was able to feel the purpose of this blog and answer the questions why? for whom? will I've enough to write?

Don't expect the answers in the post, the answers will be in form of various posts that I'll be posting hence forth. All the gadgets (that you see on the right) will make this blog dynamic and constantly changing, thus taking pressure of me.

Googlable Flex/Flash Application

Until now Flash and Flex content could not be searched. In fact in even couple of years back I was advised not to use flash, as it is not search engine friendly.

Not being search engine friendly can make or break many companies. I personally know of a company that makes 10cr annually purely relying on search engine traffic. I can't give link as its a closely guarded secret :-)


So, when Google and Adobe announced plans to make flash/flex searchable, it created waves in RIA community. This is important if RIA application (mostly flash/flex) have to become pervasive rather than just being used for advertisement banners.

Adobe platform evangelist Ryan Stewart, announced a SEO (Search Engine Optimization) contest for Flex application. Here is the link.

As per the rules it looks like this application might win. Try googling "fleximagically searchable" and this is the first application.

E learning growth

Has the envisioned growth started to happen in e-learning space?

This article Technology reshapes America's classrooms seems to indicate the trend.

Movie Guru's impact on Dhirubhai's wiki entry

With, Mani Ratnam and Ambanis refusing that Guru is based on Dhirubhai's life, lets hunt for the truth by finding what Netizen are thinking.

Post movie Guru's release the activity in Wikipedia has increased significantly. Check this small analysis on the number of entries happening in Dhirubhai's wikipedia entry. You will find small dips after 20th Jan, this is because most entries happen on Saturday/Sunday than weekdays. Weekend free time! Is this is a general trend, that most entries in Wikipedia happens on weekends? Will leave that analysis to others.

J!NX Forums - The math behind subnetting

J!NX Forums - The math behind subnetting:

Very useful. Don't underestimate the knowledge harnessed by understanding TCP/IP.

Introducing sIFR: The Healthy Alternative to Browser Text

Mike Davidson: Introducing sIFR : So much to know about fonts. Good fonts can improve readability and differentiate the presentation.

Springframework J2EE Framework

Springframework:
leading full-stack Java/J2EE application framework

Hibernate - Persistence Architecture

hibernate
is a powerful, high performance object/relational persistence and query service.

Hibernate does not hide the power of SQL from you and guarantees that your investment in relational technology and knowledge is as valid as always.

Choice of Architecture

Java Pro - J2EE Persistence from JDBC to CMP

There are basically two architecture - Data driven and Object driven. The above article gives a good explanation for choosing the right architecture and especially persistence architecture.

"Data driven architeture: Are window-on data with little or no business logic maintained in the middle tier. ften, these applications are data-centric based on their origins—for instance, they were previously accessed through a Rapid Application Development tool or similar technology—and developer skill sets. Applications that are data-centric should focus on implementations based on JDBC and frameworks that maintain the relational model all the way through to the presentation layer.

Object driven: In this development style, application logic is usually built and maintained within the J2EE middle tier and involves introducing a persistence layer into the application that will convert between the object model of the application and the relational schema of the database or database"

Top 20 Web development links of 2005

Web Graphic's (WG) Top 20 of 2005

These kind of list is good and should be shared. Essential resource for WEB 2.0 enthusiasts.

Why Haven't Online Newspapers Gotten it Right?

Kirkville - Why Haven't Online Newspapers Gotten it Right?

Kirk, I liked your point "serendipity of discovering".

Another main concern is "Newsworthiness". Sometime "News" is news only because it is printed in the newspaper. Newspapers have build that perception and trustworthiness over a period of time. However even new online media (Blogs) are able to create ripples in the society with their news.

Asking newspaper to get online right is like asking the good thearterical team to make a movie. Just videotaping a play and projecting is on the screen doesn't make it a movie. The movie has its own rules/tricks/grammar.

Exploratory testing

Good article by Jonathan Kohl on Exploratory testing for Agile Teams.

Take aways from the article:
Keep the test data central and up to date.
Testers involvement and analysis is critical.
Automation combined with testers manual testing based on analysis will lead to efficient testing.
If your looking for Software testing courses, check the following.

Rapid software testing

Michael Bolton's course

List of technical books that I want to read

This is the list of technical books culled out from slashdot over the years.

1. Spring into technical writing.
   
2. Mathematic and sex
3. Fact and fallacies of software engineering
4. HTTP developer's handbook
5. Executives guide to information technology.

About Me

I'm, what I do.

• Head of Technology (a.k.a CTO)for an e-learning company.
• Hanging around with awesome technology team, which is pure fun.
  
• Young Indian (YI - CII).
• YI Mentor for Bhavan and Jeppiaar College
• Active in Adobe Flex user community. One of the community leaders in India.
  
• Advisor for 4 empirical projects at Great Lakes Institute of Management (GLIM)
• Active in Rich Internet Application and Rich User Interface
• Project guide for CEG com-sci students
• Couple of secret/underground projects done in stealth mode.

I'm, partly also what I was

• Worked at Cognizant as business analyst, project management but mostly known for decent work done under Srivatsan/Subbu as part of Test Consulting projects and creating the product design for first ever workflow/pega based system. Unexpected opportunity to give a talk as Linux Asia star at Linux Asia '05 along with open source biggies.
  
• Times of India, honed many roles from EA (to CEO), handling online properties, Business analyst and managing the Ad serving technology.
• NIT BE e&p. Mixed bag
• IIM -Bangalore. Made me a Man.

I'm, what I want to be?

• Contribute to make India a healthy, happy and proud country.

Video on Internet

Roger McNamee blogs a 4 part series on Video on Internet.

Learning System

Behavior predicting can be surprisingly easy. Its eerie, are we leading a pre-defined life!? Ok, I can reason out the science of probability and all, but still , to be able to predict correctly 90% of the time what you will eat by what car you drive in to the restaurant, is completely surprising!!

Read about this amazing predicting system at : Hyperactive Bob

Amazon with its wealth of customer buying information can go to greater length in predicting your complete online behavior without touching any privacy concerns. Read the Amazon story at New Scientist