Predicting the Future and Overfitting

When we see a bunch of data points, we usually draw a straight line through them in the Ordinary Least Squares method and come to a prediction. We don’t overfit by drawing a polynomial spline through each of the points and then predict. That is because the Ordinary Least Squares approach is statistically motivated.

Similarly, when we tell stories to inform each other of information that may predict the future, we may overfit our stories if we don’t bring in the concept of statistics into our stories and that is most importantly that the future is unknowable: it contains new and exogenous shocks, and there is therefore no such thing as betting on a sure thing. Such a mindset is a gaming fallacy as only in confined games are there sure things.

The key takeaway here is all stories overfit the actors, as they aren’t statistically motivated and as such can’t be used to predict the future, which is in fact, a hopeless task, to predict the future accurately and consistently. You may say: don’t meteorologists predict the future and doesn’t bringing an umbrella change the future? Of course, and so we can say sometimes our stories exist to capture our feelings, but if someone doesn’t wish to share those feelings, there is no story that will help you win her over. Ah, but there is a story that may help there, and that is the Bible, it’s difficult to love without knowing God, and sometimes after we know God, we have to trust Him on these matters.


No comments

Other posts

Writing Standards
Hi Lucas Orchard readers, I will be improving the writing standards of existing posts. This process may take some time and there may be a few new posts in the meantime, but the goal will be to make this website into a stocks analysis site as we’ve done the groundwork. Thanks, Alex
Read more
Psychology: Role Playing and Pitfalls
The type of role playing game that comes to mind is way back in elementary school when people use fantastical imaginations to construct worlds and ask you to play as if you were in that world. Not everyone participates in these games but some people do. The downside of thinking this way is sooner or later you live in your own world playing a role playing game for yourself and...
Read more