The Importance of Perspective and Intelligent Analysis
During World War II, numerous fighter planes were getting hit by anti-aircraft guns. Air Force officers wanted to add some protective armour/shield to the planes. The question was "where"? The planes could only support few more kilos of weight. A group of mathematicians and engineers were called for a short consulting project.
Fighter planes returning from missions were analysed for bullet holes per square foot. They found 1.93 bullet holes/sq. foot near the tail of the planes whereas only 1.11 bullet holes/sq. foot close to the engine.
The Air Force officers thought that since the tail portion had the greatest density of bullets, that would be the logical location for putting an anti-bullet shield.
A mathematician named Abraham Wald said exactly the opposite; more protection is needed where the bullet holes aren't - that is -around the engines.
Debrief
If you go to the recovery room at the hospital, you'll see a lot more people with bullet holes in their legs than people with bullet holes in their chests. That's not because people don't get shot in the chest; it's because the people who get shot in the chest don't recover.
What you don't see in the data is often more important than what you do.
How analysis in detail actually helps?
We have been working on a technology to automatically identify road conditions (still in development phase) and while testing there was one particular stretch of road that completely baffled us for some time.
Our tech is built in a manner that it should should be able to detect potholes and speed breakers on the road automatically using sensors. However, on one stretch of road (which is complete ridden with potholes - to an extent that you can say that the road does not exist), our technology would never gives us an alert about the condition.
We had tested over 1000 kms and we were getting data with more than 90% accuracy, but this 100 meters stretch kept eluding us.
Why lack of data is also an important part of data analysis?
Being in the service industry and consulting clients for more than a decade now, I have been used to hearing the phrase 'We need data analysis to take decisions'. However, a lot of times, we get into the 'data' itself so much that 'analysis' takes a back seat. First-hand understanding is often what no amount of data can replicate.
Therefore, it is imperative that while analysing situations, you do not just look at the information itself, but the way information was collected, the sources used and the conditions in which information was collected. One a wholesome understanding, can give a result that is is valid.