-
Nate’s Numbers
Sunday Times, 23 October 2012
The Signal and the Noise:
The Art and Science of Prediction
by Nate Silver
Allen Lane, £20.00. Pp 534.
American congressman have a remarkable investment record. Their personal portfolios beat the markets by between five and ten per cent a year, a return with which Bernie Madoff, the Ponzi scheme fraudster, would have been more than happy.
The truth is that nobody can consistently make those kind of returns unless they are doing something illegal and it is pretty clear what the distinguished House members are up to – insider trading. Using confidential information acquired in their work, they play the market and break the law.
That peculiarly satisfying fact is one of many to be found in this fat and fascinating book. ‘Fascinating’ is perhaps not a word you associate with statistics. Well, get used to it. Statisticians are to our age what engineers were to the Victorians, they are the makers of the particular forms of truth we value and crave, or, in the case of the congressmen, fear.
Nate Silver, to pursue the analogy, is being tipped to be our age’s Brunel. He is a 34-year-old who gave up an accountancy job to make a living playing Poker. He moved on to betting on sports, notably the statistics-infested game of baseball and then made a big splash when, in the 2008 presidential election, he predicted the winner in 49 out of 50 states and in all 35 senate contests. This was serious pay dirt and, in 2009, he was named one of the world’s most influential people by Time magazine.His FiveThirtyEight blog now appears on the New York Times web site. His latest forecast for this year’s election is that Romney has a one third chance of winning, not as good as some Republicans might have expected.
Statisticians are to our age what engineers were to the Victorians, they are the makers of the particular forms of truth we value and crave, or, in the case of the congressmen, fear
This book is Nateism in a nutshell as well as being a compendious guide to the world as seen through a statistician’s eyes. You should read it but first, I warn you, it has been appallingly edited. Unnecessarily mangled sentences have not been unmangled, there are misprints, some charts have been mislabelled and, at one bizarre moment, while making a point about perception, Silver says the cover of this book is yellow. In fact, the hard boards are orange and the slip cover is white. Also, like many highly technical writers, when Silver goes off-piste he tends to hit a tree. He describes Isaiah Berlin’s sinuous, thunderous, subtle prose as “flowery” and he attributes to Conan Doyle that famous line that begins “Once you eliminate the impossible…..” It was Sherlock Holmes, which is, as any novelist will tell you, a quite different matter.
But these are zits on an otherwise handsome face. Silver’s book is a fine and very useful attempt to explain a very complicated and dynamic field. The reason statisticians have become so important is that now familiar phenomenon, the information explosion. This is not simply about the computer age, information first became a political and economic force with Gutenberg’s printing press in the fifteenth century. The Industrial Revolution, with its creation of high economic growth, urbanisation and mass production increased the demand and produced the first statistics explosion in the nineteenth century. In the last twenty years, computerisation has seen the second.
Information is not inherently useful until it is organised, primarily by statistics. Masses of information requires masses of statisticians, but, even then, things can go horribly wrong because, as Silver repeatedly makes clear, statisticians are as fallible as anybody else. The great statistician and heavy smoker R.A.Fisher, for example, all but ruined his reputation by persisting in his belief that there was no correlation between tobacco and lung cancer. He had been a grandmaster of his craft, now he is being written out of the curriculum.
Meanwhile, statistics are more or less on a level with quantum chromodynamics when it comes to popular understanding, a problem compounded by the fact that people don’t know what they don’t know. So, for example, you probably think your chances of being killed by a bolt of lightning are pretty much the same now as they were, say, seventy years ago. Wrong! Americans – and, probably, Brits – are now thirty times less likely to die in such a dramatic way. Why? Who knows?
The heart of this book are the long and detailed analyses of big, complex systems. The baseball and poker analyses are interesting but those of climate change, banking, earthquakes and politics are vital. Silver’s view on the first in sceptical and sympathetic belief, but I suspect global warmng deniers will find more comfort than they expect. On the second he zooms in, with an uncharacteristically but entirely justified grin on his face, the credit rating agencies, who basically lied and lied and lied and who, from the first, were hopelessly biased by their sources of income. Earthquakes still pretty much defy statistics, though his charting of the territory is gripping. Finally, in politics he remains the world expert on disentangling polling data, but, in reality, there is almost always too much noise in this game to find the signal that matters.
you probably think your chances of being killed by a bolt of lightning are pretty much the same now as they were, say, seventy years ago
The big message here is all about “the difference between what we know and what we think we know”. As Nassim Nicholas Taleb never tires of pointing out, being grown up means learning to live with dignity in an uncertain world. Silver does not have Taleb’s panoramic ethical and cultural view, but he does share enough of his bracing scepticism to justify more than one footnote mention.
He is, crucially, much more optimistic than Taleb about our ability to get an ever tighter grip on the future by the clever and careful deployment of statistics. The chapter on earthquakes shows the problems involved, but also the astonishing possibilities of solutions.
He concludes, however, on a pessimistic note: “There is no reason to conclude that the affairs of men are becoming more predictable. The opposite may well be true.” Actually I think this is wisdom, not pessimism, but I can certainly see that it represents an unscratchable itch to a statistician.
One Response to “Nate’s Numbers”
Leave a Reply
- The Ecstasy of Bedford Park
- The Mormons and Liberty Valance
- The Bad-Good and the Good-Bad
- Compassion, Chris Huhne and Twenty Years of Respect
- Christmas Continues
- American Guns
- Cruel Jerks
- Thou Shalt Not Cover Thine Ass
- The Resistible Rise of the Corporate Con Artist
- The Prig Imperiale
- Melvyn Bragg: The Mark of the Mother
- The Future According to Google
- Osborne’s Tears for Thatcher
- Tom Waits: Growling Through the Grain
- Bedford Park: The Enchanted Suburb
- The New Ritual of Dying in Public
- Museums: The Price of Free
- Rage Against the Machines
- Mohsin Hamid: Help for the Dislocated Self
- Tony Versus the Philistines
14 January 2013 at 5:47 am
poker videos, poker shows,…
[...]Bryan Appleyard » Blog Archive » Nate’s Numbers[...]…