Get Prepared
for
the Uncertain Future

I would like to introduce the book written by Nate Silver and entitled The Signal and the Noise: Why So Many Predictions Fail — but Some Don’t, which was published in 2012 by the Penguin Press.

Through many real-world case studies, the author introduces where signals and noises can be found, how well or how badly we have done in prediction, and how to make more accurate forecasts by using Bayes’s Theorem.

In the first half of the book, Silver spends seven chapters discussing many unsuccessful predictions in financial crises, political elections, baseball games, earthquakes, economics and infectious diseases, but with one successful prediction in weather forecasting. Silver has found that failures are mainly due to personal bias and overconfidence, and ignorance or distrust of data and forecasts. On the other hand, a successful prediction such as in weather may be made if we try our best to avoid “putting politics, personal glory or economic benefit before the truth of the forecast”.

In the second half of the book, Silver utilizes six chapters to showcase solutions to prediction problems in sports, gambling, chess, poker, the stock market, greenhouse effect and even terrorism. In each case, Bayes’s Theorem is applied.

Silver suggests we should admit the existing limitations in prediction. To obtain more accurate predictions, a huge quantity of data, along with many trials and errors are needed. A prediction will be a probability distribution, meaning that certainty cannot be guaranteed, but this is the real Bayesland. In the end, the aim is to learn from the previous incidents, and avoid repeating mistakes.

Among all chapters, chapter 5 — “Desperately Seeking Signal”, which recounts the story of earthquake prediction, impresses me the most. Silver points out the two main factors that make earthquake prediction hard and fallible. Firstly, the earthquake distribution follows the occurrence-magnitude power-law. There are very few data of high-magnitude earthquakes. The lack of such data may result in the inadequacies of the prediction model, which in turn leads to radically distinct results. However, it is these high magnitude earthquakes that incur the most severe disasters. Secondly, we are always too greedy in manipulating the data points and mistake noises for signals by overfitting. It seems that we are utilizing the data to its best, but in fact the results we have got are farthest from the truth. There are similar situations in our daily life. We are receiving too much information every day, but a very tiny amount of them is important. We must learn how to extract signals from noises, deal with the crucial info carefully, so that we can learn useful knowledge, improve ourselves, and predict better to get more prepared for the uncertain future.

Overall, I recommend this book The Signal and the noise: Why So Many Predictions Fail — but Some Don’t to you. You will gain from this book, which is not only about theories but also values practicality.