Backtesting

DocumentationRunning a test

Backtesting

The engine is event driven. For each timestamp it releases the bar onto a time ordered queue and drains it fully before time advances. Orders created on a bar's close become pending and fill on the next bar, so the whole path is free of lookahead by construction.

Before the first bar

The run resolves the instrument before it reads a price. Six letter symbols made of two listed currency codes are recognised by their shape, and so are the listed coin pairs; the two metals, the eleven cash indices and the three energy CFDs are listed by name. Anything none of those can place is refused with a message instead of run, unless the run declares an asset class or a custom symbol defines it. The class that was used is recorded on the run, because it decides the pip size and the contract size, and through them every money figure in the result. Asset Classes has the table and the per class numbers.

Data sources

Every run the app starts reads real bars from the local store. The store is filled by the built in Dukascopy download, by a data source a plugin registers, or by a CSV import. There is no synthetic or cached choice on the Test page. When part of the range is not on this machine, Start fetches it from the archive first, with the progress in the time bar, if your plan allows it. When none of the range arrives, the run does not start and the page points to Data. A range that is only partly there runs on the bars that are there. The engine's own synthetic generator is left for its tests and the guest preview; no run you start from the app uses it.

The source that actually drove the run is reported with the result.

Accuracy tiers

  • OHLC bar level fills.
  • M1 proxy the default. Each bar is walked with one minute intrabar samples: cached ticks aggregated to a minute when there are any, and the cached one minute bars otherwise.
  • TICK tick by tick fills from cached ticks, on Max and Enterprise.
  • L2 the ticks with the order book beside them, on the desktop, on the plans with TICK. It is not a choice you make: a tick run whose strategy reads the book in code (ctx.depth, or GetMarketDepth in a C# robot) runs as L2 on its own. A market order walks the book, and a resting limit waits for its place in the queue at its price. The order book files come from Databento with a key of your own, as Strategies API explains under ctx.depth.

At the TICK tier an entry also meets the size offered at the touch: the ask side for a buy, the bid side for a sell. An entry larger than that size is marked on its trade (the ! beside the entry price) and counted in the report. It still fills at the touch, because without the order book nobody knows what the next level would have cost; Slip beyond the touch in the run settings adds that many pips against such an entry, as a stated assumption. The comparison is made where the vendor's size unit is established, which today is forex. For the other asset classes the size is recorded on each trade and the report says it was not compared.

A tier is a request, not a promise. TICK with no cached ticks runs on plain bars, and so does M1 proxy with neither cached ticks nor cached minute bars. An M1 run has nothing smaller to sample, so it never reads the minute bars at all. The tier recorded on the result is the one the data allowed, read from the data the run actually had rather than from what was asked for.

What the result says about the run itself

What it recordsWhat it tells you
The data sourceThe stored bars the run read
The accuracy tier reachedOHLC, M1 proxy or ticks, whichever the cached data allowed
The asset classForex, metal, index, crypto, equity or CFD
The wipe outWhether the account was closed out, and on which bar
Days the vendor could not serveOn a run that fetched while it went
Bars processedHow many bars the run actually saw

Where you can read them: the exported HTML report and PDF report print the requested and the used data source, the requested and the used accuracy tier, the asset class, and a row for the closed out account when it happened. The Performance report export, Pro and above, is the whole result as JSON and carries all of them. No view inside the app shows the accuracy tier or the asset class, so exporting is the only way to see them.

Two limits on those rows. Only the ordinary single position run fills them in: Grid Basket (cTrader MDB), Lot Doubling + Hedge (benchmark) and a C# strategy that places its own orders run differently and leave the class and the wipe out at their defaults, forex and no, which are not measurements of those runs. The two basket engines leave the tier at OHLC too. A C# strategy that places its own orders records TICK or L2 when the data allowed it, and OHLC otherwise. And a server side run keeps only the data source and the accuracy tier, so its report names no asset class and no wipe out, and prints the class as forex whatever the instrument was.

Running out of money

Equity is marked once per bar. When it reaches zero or below, the run ends there:

  1. The open position is closed at the price where its loss equals exactly the money that was left, so no trade reports a loss larger than the account. The closing trade is in the trade log with the exit reason STOP.
  2. The run is marked as wiped out, with the time of that bar.
  3. The loop stops. The equity curve, the bar list and the bar count all end on that bar.

This is not a margin model. A real broker watches the maintenance requirement and closes well before equity reaches zero, so an account that gets this far would have been stopped out earlier and at a better price. What the flag rules out is an account wiped out in week three that goes on trading with money that does not exist and comes back with a drawdown above 100%.

When a run is shorter than its date range

Two reasons, and the result says which:

  • The account was wiped out. The result says so, and names the bar it happened on.
  • The bars ran out. A CSV run only sees what the local store holds for that range, and closed market days, weekends and an incomplete download all leave holes.

Compare the bar count against the range you asked for before reading anything else.

Portfolio runs

A run is one strategy on one instrument unless it carries legs. Each leg adds another instrument on the same account, with its own timeframe, lot size, strategy and cost schedule; leaving any of those blank means the run's own.

This is not the same as running each instrument separately and adding the results, and the difference is the point:

  • Margin is shared. One account funds every leg, so a leg can be refused an entry because another leg is already holding what the account had. Measured on two currency pairs that each fill all fourteen of their entries alone: together on a 10,000 account at eighteen to one they fill fourteen between them. Run separately and added up, that basket reports twenty eight trades on money that never existed.
  • Drawdown is shared. An account lives through one curve, and its deepest fall is the sum of the legs' deepest falls only when the legs fell together. The report shows both numbers and how much the account avoided.

Each leg starts with an equal share of the balance, so the legs sum to the account and a leg's own return percent means something. A leg's margin room is not its share: the check reads the whole account.

A strategy sees only its own instrument. No leg can read another's bars, positions or signals, so this is a portfolio of strategies rather than one strategy over a basket.

If the account is wiped out, every leg is closed, not just the one that was pricing a bar at the time.

Fetching while the run goes

Fetch while the run goes, on the Execution tab of the settings window, downloads the range day by day as the run needs it and throws the ticks away once they have been turned into bars.

It does not make the run faster. The loop replays millions of bars a second and the network does not, so overlapping them means the run takes as long as the download and no longer, which is the same wall clock as downloading first and then running. What it buys is that nothing large is written to disk and that the run starts producing output on the first day instead of after the last. A day of EURGBP ticks is 588 KB on disk against about 5 KB of M15 bars, so the bars are roughly one percent of what a full download stores.

Accuracy is bars only on a streaming run. There are no ticks left to replay, so asking for tick accuracy and streaming at the same time is refused rather than quietly downgraded: both are things you turned on, and an OHLC result labelled OHLC somewhere you might not look is worse than being told the two do not go together. Turn streaming off to download ticks, or pick a bar based tier.

A streaming run reads minute bars from OrionQuant's data archive, the same place every download comes from. The whole range is ordered when the run starts, each year arrives as one package per side the first time the run reaches it, and anything coarser than a minute is those minutes added up. A year the archive does not hold yet is fetched there first, so the first run over years nobody has asked for before waits while that happens.

Days are fetched a few ahead so the engine is never waiting on a request that could have started earlier, and progress counts days rather than bars, because the download is what is taking the time. A day the archive cannot supply is skipped and counted rather than ending the run. Keeping the aggregated bars is optional and on, which warms the cache as a side effect of a run you wanted anyway.

Pitfalls

  • Lookahead is impossible by construction, but optimistic fills are not. The intrabar rule decides stop versus target when both sit inside one bar, and the pessimistic default assumes the worse one fills first. A bar that opens beyond one of the levels has already answered the question: the fill is the open, not the level, and the priority does not apply.
  • Costs matter. Set the commission and the swap to your broker's real values, or the result flatters itself. The spread comes from the data.
  • One good period is not a strategy. Read the drawdown and the trade distribution, not just the net profit.
OrionQuant