Backtesting – A Useful Tool When Handled Properly

Everyone has experienced something similar. We are particularly convinced by an idea or an argument. Suddenly everything makes sense and we are full of adrenaline and want to act immediately. But beware, instead of proactively falling into the trap of our own thinking, backtesting our old ideas could save us from more serious damage. Backtesting quickly brings us back down to earth. The same applies to the flowery promises or apocalyptic scenarios of the prophets.

The scenario: anglers who want to be geologists

I see a company or read an interesting analysis of a particular stock and am immediately hooked. Everything sounds logical, the potential is huge and I still have some cash lying around, just waiting for another investment.

I read how other colleagues also invest and a hype arises. All of a sudden, FOMO, the fear of missing out on returns, is coming up. Although I have developed a certain immunity now, I speak from my own experience here.

The scenario is always the same: a hype is driven in a bubble and carried by certain actors. These actors can be bloggers, YouTubers, or any other stakeholders who are perceived by others as experts.

So mostly it’s about anglers who want to be geologists.

Our toolbox

Decisions are tools to achieve a result. Some want to retire early. For them, frugalism and investing is a tool to achieve this goal.

Others like me want to live an antifragile life. They invest, live hedonistically, and continually increase their human capital. These are also tools.

It is not a matter of what is better or worse. Everyone is the architect of his or her own fortune and has different ideas, and therefore uses different tools from the toolbox.

Using the wrong tools is good

Mistakes are good. They happen. Mistakes are part of our development. They result because we try to master a challenge or a risk with method A, although this method is unsuitable.

Simply put: When I made a mistake, I took the wrong tool from my toolbox.

Having tried method A is great because now we know that we should rather not resort to method A anymore.

Our toolbox is lighter by one superfluous tool. This also makes us more antifragile ourselves. We get rid of the things that don’t work and give more weight to the methods that do.

It is quite simple. The good is the absence of the bad (“Nimium boni estcui nihil est mali”).

This not only makes us more resilient in times of danger or risk, it actually allows us to profit. With the right tools, we can approach problems, while other colleagues keep falling back on failed models.

Using the wrong tools over and over again is bad

It is only troublesome when we stubbornly and steadily fall back on method A again and again and become a fly that seeks its way through the window but constantly bangs its head against the glass. I don’t want to be that kind of fly.

The problem, however, is that many faulty behaviors or habits tend to creep in unconsciously.

Studies with mice have shown how unconscious habits are. For an experiment, mice first had to press a button to get a reward. After they repeated this many times (i.e., get used to it), the researchers simply placed the reward in the cage. The mice still pressed the lever over and over again.

So we don’t necessarily have control over them. And so it can be that I always reach for an unusable tool in my toolbox, that as if by magic lies right at the top and especially within reach.

Backtesting own decisions

I am therefore a fan of always looking first at myself for the cause of failure before pointing the finger at others. Here I backtest my past decisions to see which of my tools might have been useless. It goes like this:

Until proven innocent, I don’t blame others. Many people make a mistake here: they find no proof of their guilt and are convinced that they are not to blame. Yet we know that the lack of proof of guilt does not automatically mean that someone is innocent.

The consequences are different: in criminal law, we cannot punish someone if there is no evidence of guilt.

But for own risk management, it is different. Here, I am equally suspicious of methods/tools that have gone wrong in the past, even if there is no evidence that it was their fault.

The heuristic (for risk management, not for criminal law) is thus: The phenotype of failure is sufficient for me without having to understand the mechanism that led to the failure.

In other words, risk management is different from criminal law, where the motto is: innocent until proven guilty. For my own risk management, it should be: Until proven otherwise, there is a presumption that the primacy of failure lies with myself. In short: guilty until proven innocent.

Ticking the checklist before I take action

I mentioned the problem of unconsciously reaching for the wrong tool above. That is why it is not only helpful to backtest one’s own decision but to keep the findings in mind once again before making new decisions.

So, at the moment when the angler promises us gold in the ground, it is important to check exactly which tool we are reaching for.

I’m talking about a checklist that I tick off. I ask myself whether there are parallels to past mistakes and whether I should therefore rather keep my hands off a decision.

Backtesting companies and their management

We can apply backtesting in many ways. For example, I become suspicious of a company that has made promises several times without fulfilling them.

As recently as March 2020, Hugo Boss CEO promised a higher dividend (5 percent more distributions). I was suspicious but in an analysis, I wrote:

Management will probably know best how far it wants to go. And if management has a positive view of the cash flow development for the next few years, then it is a reasonable approach to push the payout ratio a little bit for one or two years.

A month later, Hugo Boss canceled the dividend and paid out only a few symbolic cents in 2021. CEO Mark Langer had to resign.

Economically, this was the right decision; the only mistake was to promise an increase beforehand in an uncertain situation, thereby giving investors a completely false impression of the situation and business situation.

Therefore, I do not attach any importance to the company’s forecasts at the moment and consider the Hugo Boss dividend as a nice gimmick to the currently good share price performance.

Another example is AT&T, which has turned its own business strategy 180 degrees several times within a few years. My AT&T articles about this on Seeking Alpha bear witness to this.

Backtesting prophepts

Just as we can test companies or management, we can also test prophets and experts. This is exactly what I do.

I invite any reader to read my old articles. I’ve been wrong so many times that I think I’m a good example of how stock picking is unlikely to beat investing in a broad ETF over the long term.

But I don’t claim to be right either, and I don’t depend on being right at all.  However, when geologists offer their services and knowledge of rocks, stones, and soil to others, I look very carefully to see if it is not just an angler who once inherited a lot of money.

I don’t backtest for positive results, only for negative ones

While backtesting is a mighty tool it is noteworthy that it works only in one direction. It is only useful when we want to see if a method failed or not. It doesn’t work the other way around. Let’s remember. The lack of evidence that a tool does not work/fail does not evidence that it works.

One single negative case shows that method A is not as good as the previous 99 positive cases suggested. I therefore only look for evidence that a tool failed to avoid having to experience it myself. So when I say “looking for negative results”, “negative results” means finding results with negative effects.

These negative cases/failures must then be analyzed. A negative result can have a bad influence or only a not-so-positive one. Here it is crucial to how I deal with mistakes.

Real mistakes for me are only those that have led to irrevocable destruction. A decision that did not bring the best possible return is not a mistake for me. Falling short of perfection is not a mistake. Suffering irrevocable losses or irreversible wealth-destroying events is.

Backtesting, therefore, functions less as a filter for superior strategies and more as a detector for bullshit and false promises. For that, it is an excellent tool.

Notify of
Inline Feedbacks
View all comments
Share your thoughts by writing a comment! :)x