Predicting the Future and Overfitting

When we see a bunch of data points, we usually draw a straight line through them in the Ordinary Least Squares method and come to a prediction. We don’t overfit by drawing a polynomial spline through each of the points and then predict. That is because the Ordinary Least Squares approach is statistically motivated.

Similarly, when we tell stories to inform each other of information that may predict the future, we may overfit our stories if we don’t bring in the concept of statistics into our stories and that is most importantly that the future is unknowable: it contains new and exogenous shocks, and there is therefore no such thing as betting on a sure thing. Such a mindset is a gaming fallacy as only in confined games are there sure things.

The key takeaway here is all stories overfit the actors, as they aren’t statistically motivated and as such can’t be used to predict the future, which is in fact, a hopeless task, to predict the future accurately and consistently. You may say: don’t meteorologists predict the future and doesn’t bringing an umbrella change the future? Of course, and so we can say sometimes our stories exist to capture our feelings, but if someone doesn’t wish to share those feelings, there is no story that will help you win her over. Ah, but there is a story that may help there, and that is the Bible, it’s difficult to love without knowing God, and sometimes after we know God, we have to trust Him on these matters.