7  Conclusions

“You have been given questions to which you cannot be given answers. You will have to live them out—perhaps a little at a time.”
“And how long is that going to take?”
“I don’t know. As long as you live, perhaps.”
“That could be a long time.”
“I will tell you a further mystery,” he said. “It may take longer.”

—Dr. Admire and Jayber Crow, Jayber Crow: A Novel, by Wendell Berry

The chapters of this book tell a story:

Philosophy and statistics are deeply connected. Concepts from probability and inference to evidence and causality are unavoidably philosophical. Statistical inference methods take a stand on these concepts in an attempt to formalize and strengthen (or in some cases circumvent) inductive logic. There are deep disagreements about the efficacy of various methods. Many objections to particular methods—for example, the charge of the base-rate fallacy against frequentist hypothesis testing—are misguided or answerable. As such, foundational issues in statistics are not settled.

But after all of that disagreement, it is reasonable to ask, what should we believe about statistics and its ability to infer what is true? The short answer is, I don’t know. It may be disappointing to reach the end of a book on the foundations of statistics to find out that its author does not confidently claim to be a frequentist or Bayesian or causal counterfactualist or whatever. For me, the goal is not to arrive at confident partisan claims (e.g., “I am a committed Bayesian”). Instead, to paraphrase T.S. Eliot, we should explore ceaselessly, arrive back where we started, and see it anew. I hope this book helps readers see statistics and data science with a fresh perspective.

Seeing anew does not necessarily mean having conviction in a particular view (though if that’s what it means for you, great!). It means seeing things more clearly, understanding and appreciating complexity, and perhaps as a result, standing on firmer ground. With that said, after years of engaging with these ideas, writing this book, and teaching and using statistics, there are some things I can confidently say.

Get the full book: Buy Patterns from Static