Scrab's backtesting tools let you test investment strategies using past data, showing how they might have performed historically.

Backtests can be thought of as two separate stages: Generating data per your criteria and then simulating transactions within the time you set.

In this article we'll cover:

This type is based on the S&P 500 index composition, accounting for changes over time. Scores are calculated only for companies that were constituents of the S&P 500 through the backtest period.

If a company is no longer in the S&P 500, it is automatically removed from the portfolio during simulated rebalancing, regardless of its scores.

These backtests are based on a static list of companies added to the scoring model. This list remains unchanged over time, meaning the same companies are used for each backtest period.

Whatever companies are in the scoring model are considered for each backtest without accounting for changes over time.

This method uses saved Screener results and considers how these results and scores looked in the past. For instance, if the Screener criteria are Market Cap > $10B and debt level < 30%, a company may match the criteria in one month but not in another.

This approach helps avoid survivorship bias and includes delisted companies like Twitter. When many companies match the Screener criteria, scores are calculated for up to 1,000 companies with the largest market capitalization.

It is recommended to narrow down the Screener criteria to return fewer than 1,000 companies at any given period.

We believe that Historical Screener Backtests are most useful because they avoid survivorship bias, include delisted companies, and closely mirror real-life trading scenarios.
​

Survivorship bias in backtesting means you're only looking at stocks that have survived, which can lead to overly optimistic results. Our tool counters this by analyzing the stock pool as it was at each point in the past, ensuring a more accurate simulation.

The S&P 500's constant updating further complicates backtesting, as it swaps out weaker stocks for stronger ones each quarter. Our system's approach accounts for these changes, conducting rolling backtests that reflect the actual composition of the S&P 500 over time, providing you with reliable insights into your strategy's effectiveness.

Backtesting your stock selections by looking at their historical performance may lead to misleadingly positive results. Doing that often involves choosing stocks that are successful now, not necessarily those that would have been chosen in the past.

Choosing to backtest Netflix wouldn't make sense since it's based on hindsight (you already know the stock is successful), and this process needs to reflect an investment strategy's effectiveness accurately.

Although Scrab offers a Model Portfolio for straightforward portfolio comparisons, the more impactful tool is our Backtests, designed to offer a deeper, more accurate analysis of investment strategies over the years.

The effective method for backtesting involves assessing the criteria for stock selection (or strategy) rather than the stocks themselves, following three steps:

This approach simulates long-term investing behavior and evaluates the strategy's potential success without risking real money.

In backtesting, adjusting the number of stocks, their weights, and rebalancing frequency can significantly impact investment outcomes. There is no universal solution, so exploring different strategies and stock selections is crucial to finding what works best.

Practical wisdom suggests starting with about 25 stocks and monthly rebalancing. Yet, integrating risk-adjusted weighting schemes like Hierarchical Risk Parity might offer additional gains. Ultimately, only through backtesting with Scrab can you accurately evaluate and refine your strategy with concrete data, moving beyond theory to informed, data-driven decisions.

While past backtest results offer insights, they don't assure future gains. Historical performance serves as a guide, not a guarantee. For accurate investment decisions, it’s crucial to understand the reasons behind the data patterns and distinguish between correlation and causation. Investments should be based on solid, fundamental explanations, ensuring strategies are sensible and data-driven.

Understanding the underlying causes behind correlations ensures that investment decisions are not just data-driven but fundamentally sound, maximizing the chances of future success. If you want to learn more, you can see our article, Interpreting Backtest Results.