RE:CZ

Three-Layer Structure and Experimental Design Reconstruction of Backtesting Systems

Quantitative Finance

👤 Quantitative investment researchers, backtesting system developers, investment strategy analysts
This paper proposes reconstructing the backtesting system into a three-layer structure: market sequences as investment objects, signal strategies as responses, and betting strategies as investment subjects. The author suggests splitting each experiment into combinations of these three components and emphasizes the need for a thorough revision of experimental design and evaluation systems. The new evaluation focus shifts from peak expectations to the frequency and distribution characteristics of profit-taking events, particularly the average time interval of profit-taking events given M_T, to provide more meaningful investment guidance.
  • ✨ The backtesting system should be reconstructed into a three-layer structure of market sequences, signal strategies, and betting strategies
  • ✨ Market sequences are investment objects, which can be generated from synthetic or historical data
  • ✨ Signal strategies are responses to market sequences, producing buy/sell signals
  • ✨ Betting strategies determine capital allocation and risk management, reflecting investment subject preferences
  • ✨ Experimental design requires evaluating each signal strategy under all market sequences and betting strategies
📅 2026-02-01 · 571 words · ~3 min read
  • Backtesting System
  • Experimental Design
  • Market Sequences
  • Signal Strategies
  • Betting Strategies
  • Investment Evaluation
  • Risk Management

It is now Sunday, February 1, 2026, morning.

Still working hard on the experimental framework for the protracted war with capital.

After some research, I believe the research paradigm of this project should be restructured.

Each experiment should be broken down into a combination of three components: Market Sequence + Signal Strategy + Betting Strategy.

Philosophically speaking, the market sequence represents the investment object, the signal sequence represents the reaction, and the betting strategy represents the investment subject.

Three-Layer Structure of the Backtesting System

Market Sequence (Object)

The market sequence is generated by a market generator based on characteristic parameters, such as volatility, drift, etc. The market sequence is the object, the external environment.

Market sequences can be generated using models like GBM / GARCH / Heston. Given parameters and a random seed, a deterministic market sequence can be generated.

Market sequences can use synthetic data or real historical data. We can pre-generate a batch of market sequences for subsequent evaluation by signal strategies and betting strategies. Whether synthetic or historical data, their format is consistent.

Signal Strategy (Reaction)

The signal strategy is generated by an underlying strategy and is a reaction to the market sequence.

Signal strategies can be mean reversion, trend following, breakout strategies, etc. Given a market sequence, a signal strategy produces a series of buy/sell signals.

Use random signals as a control group.

Future research and development of signal strategies essentially involves continuously creating new signal strategy modules to respond to market sequences.

Betting Strategy (Subject)

The betting strategy determines how to allocate capital and manage risk after receiving a signal. Betting strategies can be fixed-fraction betting, Kelly criterion, Martingale, Anti-Martingale, etc. The key lies in the investment subject's goals and risk preferences.

We cannot discuss all investors' tendencies beforehand; even an investor's own tendency is vague and variable: no one would refuse to earn more money. However, investors can only make decisions based on the existing frontier of choices.

Therefore, we cannot design experiments for specific, concrete investor preferences. However, we can pre-calculate for the power sequence M = 2, 4, 8, 16, ..., 1024. Afterwards, investors can simply take their own requirements and refer to the nearest value in this sequence. This can provide investors with a more comprehensive perspective.

However, in actual investment decisions, an investor still must choose a specific M_T value as an instance of the betting strategy to execute the investment operation.

Restructuring the Experimental Design

For each signal strategy, we must evaluate it under all market sequences and all betting strategies.

The evaluation system has changed: we should no longer focus on the expected value of the peak E(M). This value is now meaningless; it cannot guide an investor in setting their own M_T value, and it is easily influenced by extreme values.

We should focus on the frequency and distribution characteristics of the profit-taking event that occurs when M >= M_T. I am particularly concerned with the average time interval for the profit-taking event to occur under a given M_T, which means profit-taking events should occur from time to time. This is meaningful; I can expect the next one to take about the same amount of time. Therefore, P(M >= M_T) is also no longer important, because success is inevitable given enough time.

Thus, it is necessary to thoroughly revise all previous experimental steps, evaluation modules, and the conclusions highlighted in reports. This is a significant shift in perspective.

See Also

Referenced By