Skip to content

Equity market orders placed intraday under daily data fill at a stale previous-day close #9518

@DerekMelchin

Description

@DerekMelchin

Expected Behavior

When an algorithm subscribes to daily data (AddEquity("SPY", Resolution.Daily)) and places a market order intraday — e.g. from a Scheduled Event that fires while the exchange is open — the fill should reflect a price the algorithm could realistically have achieved given the data it actually had:

  • It should not fill at a daily-bar close that predates the order (the previous trading day's close), a price that is no longer obtainable.
  • It should not enable look-ahead: deciding on a closing price and then filling at that same close.

The realistic execution for a daily-data decision made during the session is that day's close — either via the closing auction (a MarketOnClose fill) or via continuous trading near the close — not yesterday's close.

Actual Behavior

An intraday market order placed while the exchange is open fills immediately at the last cached daily bar's close — the previous trading day's close, with only a non-blocking warning attached.

Walking the flow for a buy market order submitted at noon on day D with a daily SPY subscription:

  1. QCAlgorithm.MarketOrder already converts market orders to MarketOnOpen when the exchange is closed (!security.Exchange.ExchangeOpen), and even warns: "all market orders sent using daily data, or market orders sent after hours are automatically converted into MarketOnOpen orders." But at noon the exchange is open, so this conversion is skipped and a plain Market order is submitted.
  2. EquityFillModel.MarketFill passes the IsExchangeOpen gate (regular hours at noon) and calls GetBestEffortAskPrice(asset, order.Time).
  3. The only cached bar is day D-1's daily bar (delivered at D-1's close). Its EndTime (D-1 16:00) is older than cutOffTime = orderTime − StalePriceTimeSpan (default StalePriceTimeSpan = 1 hour), so the "fresh price" fast-return is skipped and execution falls through to return bestEffortAskPrice, i.e. the previous day's close.

So the order fills at a ~20-hour-old price. This is unrealistic on its own, and it directly enables look-ahead bias: if the decision was driven by an indicator that updated on that same daily close, the algorithm gets to trade at a price it has already observed. In reality, by the time the close exists you can no longer trade at it; the price has moved overnight and into the next session.

The detection gap is precise: LEAN's existing auto-conversion only fires when the exchange is closed (!ExchangeOpen). The exchange-open intraday case — most commonly a Scheduled Event — slips through and fills against the stale daily close.

Potential Solution

Treat an intraday equity market order under daily data according to three time regions, anchored on the MarketOnClose submission buffer (MarketOnCloseOrder.SubmissionTimeBuffer, default 15.5 min). Using US equity hours as the example:

Region Clock (US equity) Realistic outcome Mechanism
1. Open → MOC cutoff 09:30 – 15:44:30 fills today's close (closing auction) convert to MarketOnClose (new)
2. Past cutoff, still open 15:44:30 – 16:00 fills today's close (continuous trade near close) keep Market, wait for the fresh close bar (new)
3. Exchange closed 16:00 → next 09:30 fills next open MarketOnOpen (already implemented)

Both new regions end up filling at the close, for two different but defensible reasons: Region 1 can participate in the auction (so the submission buffer applies and MarketOnClose is the right type); Region 2 is past the auction cutoff but can still trade continuously, where a market fill near 15:50 ≈ the close, and the 16:00 close is the only fresh price that postdates the order. The submission buffer is an auction constraint, not a continuous-trading one, so it is consistent to forbid an MOC in Region 2 yet allow a continuous market fill at the close.

Region 1 — convert to MarketOnClose. Extend the existing conversion branch in QCAlgorithm.MarketOrder (the same place that already converts to MarketOnOpen when closed). When the order is intraday, before the cutoff, and the gating conditions below hold, build a MarketOnCloseOrder instead of a Market order. This reuses the battle-tested EquityFillModel.MarketOnCloseFill, which is time-gated (if (asset.LocalTime < nextMarketClose) return fill;) — it fills at the close clock-time using the last cached close even if the security prints no further bar that day.

Region 2 — keep Market, wait for a fresh close bar. Past the cutoff an MOC submission is rejected (MarketOnCloseOrderTooLate), and a MarketOnOpen cannot be submitted while the market is open (MarketOnOpenNotAllowedDuringRegularHours). So leave the order as Market and add a stale-data guard to MarketFill so it does not fill until a bar that ends after the order time is available. This is exactly the guard StopMarketFill already uses via GetBestEffortTradeBar(asset, order.Time), which returns null for any bar with EndTime <= orderTime. When day D's close bar arrives at 16:00 the pending order fills at that close.

Two requirements make Region 2 safe:

  • The guard must be narrowly gated, or it regresses minute/tick orders. A minute market order placed in the 14:30 OnData has order.Time == 14:30 and the latest bar's EndTime == 14:30 — equal — so an unconditional EndTime <= order.Time guard would delay every minute order by one bar (the same one-bar lag stop orders have). Gate the wait to session-scale bars only. The clean discriminator already exists in IsExchangeOpen, which uses barSpan > Time.OneHour to distinguish daily/hourly bars from intraday bars; reuse it so only daily-resolution intraday orders wait.
  • Never fall back to filling at the last cached price. Region 2 must only ever fill on a genuinely fresh bar (EndTime > order.Time). Waiting naturally handles illiquidity: the order fills whenever the security next actually trades (typically the next daily bar), at a real post-decision price — not a stale one. Do not add a "fill at the last cached price if no fresh bar arrives by the close" backstop — that simply reintroduces the stale-close fill this change exists to prevent (and is worst exactly for the illiquid names that most need protecting). If the security never trades again (delisting, halt), the order should stay unfilled, faithfully representing that no realistic fill was possible; if a dangling open order is undesirable, cancel it with a clear message once the session ends, but never fill it at stale data. (Region 1's MarketOnClose sidesteps this entirely because it is time-gated, filling at the close clock-time regardless of whether a bar prints.)

Region 3 is unchanged. The existing !ExchangeOpen → MarketOnOpen conversion already handles the exchange-closed case correctly (fills next open). Scope "otherwise" in this proposal to Region 2 only; do not let it swallow Region 3, or "wait for the close price" would mean tomorrow's close and silently override established MarketOnOpen behavior.

Gating / fallbacks (applies to both new regions). Convert/wait only when:

  • Security is an equity (matches the delisting-fix scoping; MarketOnClose lives in EquityFillModel).
  • Market is not always-openMarketOnClose/MarketOnOpen both throw for IsMarketAlwaysOpen, and for a continuous market the stale-close concern is largely moot (no session boundary, no "you couldn't trade then"). Leave crypto/forex as a regular market order.
  • Not warming up — orders are not placed during warm-up; leave existing behavior.
  • Highest subscription resolution is Daily (seed from the existing anyNonDailySubscriptions check in MarketOrder).
  • Backtest only. Live trading has a real-time feed and the broker executes intraday orders normally; this is a backtest-fidelity change.

Everything excluded (futures/FOP — already passed through; options; intraday resolutions; live) lands on an existing code path, so each fallback is "do nothing new" rather than a bespoke branch.

Trade-offs to weigh:

  1. Results change → regression-stat regeneration. Every daily-resolution algorithm placing intraday market orders gets different fills (today's close, possibly a different timestamp, instead of yesterday's stale close). The full expected-statistics suite would need regenerating — the bulk of the work.
  2. Synchronous-fill expectation. A non-async Market order currently returns and is reflected promptly. Both regions defer the fill to the close (a later time step), so Portfolio.Invested, the returned ticket, and the Algorithm Framework's Execution/Risk models see a not-yet-filled order — the same consequence the existing MarketOnOpen conversion already has.
  3. Backtest/live divergence. Auto-converting only in backtest makes backtest behave unlike live, in tension with LEAN's "backtest == live" goal. The existing MarketOnOpen conversion has the same property, so there is precedent, but it is worth surfacing.
  4. DailyPreciseEndTime = false. With precise end times off, the daily bar ends at next-day midnight, so LocalTime never reaches 16:00 intraday and fills still land at midnight. This is inherent to that mode; the modern default (true) gives the 16:00 close fill.

Reproducing the Problem

Minimal algorithm — daily SPY, a Scheduled Event at noon that places a market order, logging the fill price against the daily close the algorithm has already observed:

from AlgorithmImports import *


class DailyDataIntradayFillAlgorithm(QCAlgorithm):

    def initialize(self):
        self.set_start_date(2023, 5, 25)
        self.set_end_date(2023, 6, 25)
        self.set_cash(100000)
        self._spy = self.add_equity('SPY', Resolution.DAILY)
        self._prev_close = None
        self._traded = False
        self.schedule.on(
            self.date_rules.every_day(self._spy.symbol),
            self.time_rules.at(12, 0),
            self._trade
        )

    def on_data(self, slice):
        bar = slice.bars.get(self._spy.symbol)
        if bar is not None:
            self._prev_close = bar.close   # last completed daily close

    def _trade(self):
        # Act only once a PRIOR daily bar is cached, so the order is placed at noon on a
        # day whose most recent data is the *previous* day's close. Skipping the first day
        # is essential: on day 1 no prior bar exists at noon, so the order instead waits for
        # that day's close bar and fills (correctly) at the same-day close — not the defect.
        if self._traded or self._prev_close is None:
            return
        self._traded = True
        ticket = self.market_order(self._spy.symbol, 100)
        # fill price == the previous day's close, filled ~20 h later at noon, in regular hours.
        self.debug(f'placed={self.time} fill={ticket.average_fill_price} prev_close={self._prev_close}')

Observed: fill == prev_close while placed is noon of the next day — the market order fills at the previous trading day's close (the price that last updated any daily indicator), ~20 hours stale, not at a price the algorithm could realistically obtain at noon.

Note — skipping the first day is essential. On the very first day there is no prior daily bar cached at noon, so the security "does not have an accurate price" yet and the order does not produce a stale prior-close fill (it waits for the day's close bar, or is left unfilled). The stale fill only appears once a prior session's bar is the most recent cached data — i.e. from the second trading day onward, where a noon order fills at noon, at the previous day's close (fill_price == prev_close, fill_utc == 12:00 ET).

Deterministic fill-model unit test (makes the defect independent of why the order was placed):

// A daily SPY bar for day D-1 (ends at D-1 16:00) is the only cached data.
// A Market order is submitted intraday on day D (noon), after that bar's EndTime.
var fill = new EquityFillModel().Fill(new FillModelParameters(
    spyDaily, marketOrderAtNoonDayD, configProvider, stalePriceTimeSpan,
    securities, onOrderUpdated)).Single();

// Current (buggy) behavior:
//   fill.Status    == OrderStatus.Filled
//   fill.FillPrice == dayMinus1Close   (a stale close that predates the order)
//
// Expected after fix:
//   Region 2 (past MOC cutoff): fill.Status != Filled until day D's close bar arrives,
//                               then fill.FillPrice == dayClose.
//   Region 1 (before cutoff):   the order is a MarketOnClose order and fills at day D's close.

System Information

  • OS: Windows 11 (10.0.26200)
  • Runtime: .NET 10
  • LEAN: master

Checklist

  • I have completely filled out this template
  • I have confirmed that this issue exists on the current master branch
  • I have confirmed that this is not a duplicate issue by searching issues
  • I have provided detailed steps to reproduce the issue

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions