External Data

DocumentationWriting strategies

External Data

A backtest can pull in data beyond price bars: an economic news calendar and any data a plugin fetches. Both are fetched when you tick Fetch external data (news and plugin feeds), in the Backtest view's settings window, on the Execution tab. Plugin feeds are also fetched without it when the strategy reads one with ctx.data(. Both cache into an external folder in the data root, so a repeat run stays fast and offline friendly.

The flag has no effect on a server side run. See Where external data is not available at the end of this page.

Economic news

For a filter that keeps a strategy out of the market around releases, use ctx.news_window (see Strategies API, News windows): it never runs on without the calendar. The list below is the route for a strategy that wants to look at the events itself.

When the flag is on, the engine loads the economic calendar the news windows read before the first bar. A strategy reads it through ctx.news_data, a list of events:

pythondef on_market(self, ctx):
    bars = ctx.bars(1)
    if not bars:
        return
    today = str(bars[-1].ts)[:10]
    events = [e for e in ctx.news_data if e["date"] == today]
    if any(e["impact"] == "HIGH" for e in events):
        return  # skip trading on high impact news days

bar.ts is a Python datetime, not a string, so it is converted before it is compared with the date field.

Each event has date (YYYY-MM-DD) and time (HH:MM), both in UTC, currency, impact (LOW, MEDIUM or HIGH), event (ForexFactory's name for it), actual, forecast and previous. The figures are numbers, or None where the source gives none. The list holds the run's days, in time order.

The events are those of the news windows' calendar: ForexFactory's weeks where this computer has them or can load them (the current week always), and the official release calendar that comes with the app everywhere else. That one holds only the rate decisions, jobs reports and consumer prices of EUR, GBP, USD, AUD and NZD from 2016 to the end of 2026. Where the list is not complete for the run, ctx.news_data_problem says so in a sentence, and it is None when the list is complete. A strategy that needs every release can stop there instead of trading on a short list.

Currency filtering only happens when the symbol has at least six letters once non letters are stripped: EURUSD keeps EUR and USD events, XAUUSD keeps XAU and USD. A symbol that does not, an index or a stock, gives the engine no currencies to match on, so the list is the whole calendar for the run, unfiltered.

The news cache

The calendar keeps each ForexFactory week it loaded under the data root (news/forexfactory/v1/), so a run over weeks seen before needs no network. The official calendar comes with the app.

Plugin data feeds (the VIX plugin)

A data plugin registers a hook with api.register_data_fetch(fn). When the flag is on, or when the strategy's code reads ctx.data( anywhere, the engine calls the hook of every switched on plugin before the loop starts, passing a context that exposes data_dir, start_date, end_date and instrument. The Logs say so when it happens for the second reason. The hook downloads and caches a date to value JSON map, which a strategy reads with ctx.data(name, date). One fetch runs at a time: a run started while another fetches waits for it, up to ten minutes, and the same instrument and dates fetched in the last two minutes are not fetched again.

ctx.data takes the first ten characters of whatever you pass as the date, so a bar timestamp works directly. Called with no date it returns the whole map. Keys that start with an underscore, such as _meta, are the feed's own notes and are left out.

A day the feed has no value for reads as None. When the strategy cannot trade without the value, ask for it with required=True, and the run stops with a reason instead of going on:

pythonvix = ctx.data("vix", bars[-1].ts, required=True)

It stops with plugin_feed_missing when there is no feed of that name (the plugin is not installed, is switched off, or its fetch failed: its log in Plugins says why), and with plugin_data_missing when the feed exists and has no value for that day. The message says which dates the feed covers. Without a date, required=True asks only for the feed.

The bundled VIX plugin caches vix.json from FRED, series VIXCLS by default, and refetches only when the cache is older than its cache_days setting, 30 days by default. Read it with ctx.data("vix", date).

pythondef on_market(self, ctx):
    bars = ctx.bars(1)
    if not bars:
        return
    vix = ctx.data("vix", bars[-1].ts)
    if vix and vix > 30:
        ctx.signal("EXIT")

A hook that raises is contained per plugin and recorded in that plugin's log. The run continues with whatever was cached before, or with nothing.

Adding a custom data source

Write a plugin that calls api.register_data_fetch, fetch your series in the hook, and write a {name}.json date to value map into context.data_dir.

The plugin sandbox shapes how you do it. os and pathlib cannot be imported, so join paths as plain strings. open() is available but confined: it writes only inside the data cache, and reads only from the data cache or the plugin's own directory. urllib and http can be imported, because fetching is the point of a data plugin. Track freshness with a timestamp written inside the cache file, as the VIX plugin does, rather than a file modification time you have no way to read.

Honour your own cache interval so you do not hit the source every run, and respect the upstream limits: most free tiers cap requests per day.

Where external data is not available

  • Server side runs. The server forces the flag off whatever the configuration asked for, because it has no network reach and a fetch would hang until the run timed out rather than fail cleanly. Nothing in the result reports the override, so ctx.news_data is an empty list and ctx.data(...) finds no feed. A read with required=True stops the run with plugin_data_server instead. Run locally if the strategy depends on either.
  • The browser build. It has no local engine and no plugin folder, so every run is a server side run and external data is never fetched.
  • cTrader export. The exporter refuses ctx.news_data, because cTrader has no economic calendar a cBot can read, and refuses ctx.data(...), because it reads something that exists only inside this application. A strategy that uses either cannot be exported.
OrionQuant