Howzat - Limitations of batsmen rankings revealed

Current systems for ranking the best batsmen in test cricket have been bowled out by a new study.

In a paper which could give sleepless night to cricket statisticians all over the world, researchers from Northumbria and Newcastle Universities delivered their ‘out’ verdict to current methods after analysing the two most popular test cricket rankings. They found that rating batsmen by average score alone is not enough to determine who is best.

In fact, to build an accurate picture and truly determine the best batsman of all time, our cricketers would have to play a million innings each – compared to their current average of 12.

Publishing their results in the Journal of the Royal Statistical Society, Professor Richard Boys from Newcastle University, and Dr Peter Philipson from Northumbria University, built a statistical model to rank the best test cricket batsmen of all time.

The study revealed significant uncertainty in the current rankings of players. For example, England Captain Joe Root could be ranked anywhere from ninth to 303rd, while New Zealand’s star batsman Kane Williamson would feature in the list anywhere between positions four and 343.

Alastair Cook is ranked 115th in the study, out of 2,856 batsmen. The researchers are 95% confident the recently retired captain of the England Test and One-Day International is ranked between 82nd and 402nd, highlighting the wide rank intervals that the study uncovered.

The one constant for all ranking systems is Sir Donald Bradman, recognised as the best batsman of all time and confirmed by the study.

Studying all players that have ever played test cricket – which totals 2,855 test batsmen from ten countries playing between 1877 and 2017 – the researchers found that the ranking of a test batsman can easily be too high if, for example, they only played against some of the weaker test sides or only played in home test matches.

A major component of the model, therefore, was to account for the many factors that influence performance, such as era, number of innings played, player’s age, home advantage and the opposition team. Even when standardising performances after correcting for these factors, there is considerable variation in the innings-by-innings performance of a batsman.

This high level of variation means that we cannot determine the true ranking for a batsman by simply looking at their performance over a couple of hundred innings. Instead this innings-by-innings variation was modelled, essentially to determine what might have happened if the batsmen had played many more test matches. Their results show that rankings are not as clear-cut as suggested by test averages alone.

Richard Boys, Professor of Applied Statistics and Head of Pure Mathematics and Statistics at Newcastle University, co-led the study. He explains: “What we have shown is that it doesn't make sense to rank batsmen solely on the basis of what they have achieved in test matches because this simply doesn’t give an accurate picture. There are too many factors influencing their success: who they're playing, is it a home or away, and how experienced they are.

“More importantly, batsmen play in relatively few test matches to give an accurate picture of their ability and so there can be considerable uncertainty on a player’s rank. People have their own views as to who’s ranked second, third, fourth, etc. but what we have shown is that while the top spot remains fairly static, when you get nearer the middle of the table, the uncertainty is huge.

“Current rankings based on averages don’t take account of all the different influencing factors or the uncertainty every time you step out to bat. And because the number of innings we are looking at are relatively small you can’t say what their true worth is – a batsman would have to take millions of innings for us to be able to rank them accurately.”

Dr Peter Philipson, Senior Lecturer and Researcher in Statistics at Northumbria University, said: “The current method for calculating the batting average assumes that players are equally likely to get out at any time, something which is demonstrably not true for the majority of batsmen based on our findings. Many players have an increased chance of getting out for a duck. We also see that there is very little practical difference between an average of, say, 55 and one of 50 bearing in mind the large uncertainty.

“The maxim that batting last is difficult was borne out from our work. Interestingly, playing away from home is equivalent to playing on a pitch in its third innings.”

The researchers analysed 2,855 test match cricketers, from the first test played in 1877 up to 2259th test in August 2017. The overlapping nature of playing careers was applied to form a bridge from past to present so that all players can be compared simultaneously, rather than just relative to their contemporaries.

When the study rankings were compared with those of the career test batting average and the ‘ICC best-ever test championship rating’ list, a discrepancy between all three lists emerged. The authors argue that this illustrates a central problem in ranking batsmen by a single-number summary when there is a high level of innings-to-innings variation in runs scored by each batsman.

For further research, the authors suggest applying the model to the performance of both batsmen and bowlers. The approach could be further extended to analyse data from 1-day internationals, which, despite being an international sport since only 1971, has already seen around 3,900 fixtures take place.