Statistical dances: Why no statistical analysis is reliable and what to do about it

We need to improve the way we do statistics in HCI, but more training in statistical theory is not enough. We also need good “intuition pumps” to develop our statistical thinking skills. In this talk I explore the basic concept of statistical dance. The dance analogy has been used by Geoff Cumming to describe the variability of p-values across hypothetical replications. Through visual examples, I show why any statistical analysis and any statistical chart actually dances across replications. I discuss why most attempts at stabilizing statistical dances are either insufficient or misguided. The solution is to embrace the uncertainty and messiness in our data. We need to develop a good intuition of this uncertainty and communicate it faithfully to our peers. I give a few tips for conveying and interpreting interval estimates in our papers in a honest and truthful way.

Bio:

Pierre Dragicevic is working in the Aviz team at Inria as a permanent research scientist (CR). He studies information visualization and human-computer interaction. He co-signed many research articles with p-values until he grew dissatisfied and banished p-values from all his publications. Since then, he has been promoting an approach to statistics based on planned analyses, interval estimation, graphical communication, and nuanced interpretations.