Optimization

DocumentationRunning a test

Optimization

An optimization is many backtests over a grid or an evolving population of parameters. Flag the parameters to optimize, set their ranges, choose criteria, and run. Every pass uses the base configuration and the same seed, so a pass is determined by its parameters alone.

On this machine or on the server

The run location beside Run says where a sweep goes. Local runs it in the desktop app and streams each pass into the table as it finishes. Server queues it on our server, which runs the same optimiser on the data the server has, and the passes arrive together when it is done. Closing the app does not stop it; the app picks it up again when you sign back in. In the browser build, Server is the only place a sweep can go.

  • It uses server minutes for as long as it runs, as a server backtest does, and two workers, whatever the worker count says.
  • It waits in the same queue as server backtests and starts once nothing else is running, because it takes both of the server's slots. Runs queued after it wait for it.
  • It runs for at most half an hour, and never past the minutes you have left. A sweep that reaches that limit stops between passes and returns the ones it finished, marked stopped.
  • At most 5,000 passes. The genetic search needs Max or Enterprise there as well.
  • A walk forward runs on this machine, wherever the run location points.

Grid versus genetic

  • Grid every combination of the ranges, in enumeration order. Needs Pro or above. When the combinations exceed Max passes the sweep keeps the first ones in that order and drops the rest. It does not sample across the space.
  • Genetic an evolving population, on Max and Enterprise. Each generation keeps the top fifth as elites, fills the rest from tournaments of two, crosses each gene from one parent or the other at even odds, and mutates a gene with probability 0.1 by drawing from its range again, which can return the value it already had. Parameter sets already evaluated are not run again, so the real pass count is usually below population times generations.

The Resources tab holds the rest. Max passes caps both algorithms and defaults to 200. Workers default to 4, bounded by the plan (Pro 4, Max 16, Enterprise 32) and by the cores the machine reports. Population defaults to 20 and generations to 10, and both fields appear only when the algorithm is genetic.

Parameter ranges

In the sweep setup on the left each flagged parameter contributes start, step and stop, and takes the stop minus the start, divided by the step, floored, plus one values. A step of zero or less, or a stop below the start, collapses to the single start value. The estimate on the run bar multiplies those counts for a grid sweep, so the pass count is visible before running. For genetic it reads as an upper bound instead, population times generations capped at Max passes.

Fitness

A criterion, set in the Fitness criteria tab, names a metric and a direction. The metrics are net P/L, profit factor, max drawdown, win rate, winning trades, Sharpe, recovery factor and expectancy. With no criterion set the engine maximizes net P/L.

Each criterion's metric is min and max normalized to the range 0 to 1 across the evaluated set, the direction is applied, and the criteria are averaged. The engine supports a weight per criterion, but the app sends the same weight for all of them, so in the app the average is plain. A metric that is identical across the set normalizes to 0.5.

Passes stream with a provisional Fitness computed over the passes seen so far. The finished sweep recomputes Fitness over the whole set, and that is the ranking the table and the best pass use.

A pass that throws is recorded as failed: its status reads failed to run, its metrics are zero, and the engine prints the first exception of each sweep once per process, so a sweep that is failing does not look like a sweep of losing strategies. A failed pass is left out of the normalization and ranks below every pass that ran, so it is never the best pass; a sweep in which every pass failed has none. A pass that finishes with equity at or below zero is marked stopped out, keeps the metrics it produced, and ranks with the rest, because a stop out is a result.

Reading results

The table streams each finished pass. The Columns menu chooses which parameters and metrics are shown, and every column sorts except Status. Selecting a pass fills the Selected pass tab with its metrics and runs one further backtest to draw its equity curve. That preview covers the whole period on the run bar, including anything a split held back.

Compare all N runs opens every pass on one surface, stopped out and failed ones included with a checkbox to hide them, and the metric and Fitness columns shaded across the sweep so a lone spike is visible next to a plateau. Apply to Backtest copies the selected parameters into the Backtest view.

Stop drops the connection to the engine and keeps the passes already streamed. Their Fitness stays the provisional one, because the recompute only happens when a sweep finishes.

In sample and out of sample

The split control sits on the run bar and is shared with Backtest. It decides what the sweep is fitted on.

  • Off the whole period is in sample.
  • By ratio the in-sample share, 50% to 95% in steps of 5, default 70%.
  • By date the day the out-of-sample segment starts.

The boundary is rounded to whole days, and the out-of-sample segment starts the day after the in-sample segment ends, so no bar is in both and none is skipped between them. A split date outside the period, or a period too short to divide, is reported on the control and the sweep runs on the whole period instead.

With a split set, the sweep optimizes on the in-sample segment, the run button reads Optimize in sample, and the engine runs every pass twice, once per segment. The conditions on the same control (minimum return, minimum profit factor, maximum drawdown, minimum trades, minimum win rate, defaulting to profit factor 1.1 and 30 trades) and the automatic held back run belong to the Backtest view. The sweep does not read them.

The overfit gap

With a split set, each pass carries an out-of-sample Fitness and an overfit gap, in-sample Fitness minus out-of-sample Fitness. Both appear in Compare all runs.

Read the gap knowing how it is computed. The out-of-sample Fitness of a pass is normalized against that pass alone, so its minimum and maximum are the same number, every criterion normalizes to 0.5, and the out-of-sample Fitness is 0.5 for every pass in every sweep. The gap is therefore the pass's in-sample Fitness minus 0.5, and it ranks the passes in the order their streamed Fitness ranked them. It is not an independent measure of robustness. The gap is taken from the provisional Fitness as the pass streams and is not recomputed with the final ranking, so it does not line up with the Fitness column. The out-of-sample metrics behind it are carried on each pass, but no view displays them.

To judge a winner out of sample, apply it to Backtest and run it there with the same split. Backtest runs the held back segment, compares the two segments metric by metric, and states the give back as a percentage of the in-sample return. Above 50% it says plainly that this is what an overfitted strategy looks like.

The rest is judgement the app cannot make for you. The best pass on the fitted window is the one most shaped by that window's noise. Prefer a broad plateau of similar parameter values over a single sharp peak, and be sceptical of a Fitness that towers over its neighbours.

Walk forward

On Max and Enterprise the Algorithm dropdown also offers Walk forward. It fits the parameters on a training window, tests them on the period after it, steps forward and repeats, and stitches the tested periods into one curve. The training, test and step lengths, the number of windows and whether the training window is anchored are set below the dropdown. On Pro the option is listed, and choosing it opens the upgrade dialog.

Not implemented

  • The sweep wide overfitting and out-of-sample summaries on the result message. The fields exist and are never filled in.
  • A walk forward on the server.
OrionQuant