The ETA is a decision: regret-weighted dispatch and promise setting for US meal delivery#

HitBase Research · Last-Mile Delivery

Abstract#

US meal delivery platforms optimize two proxies: the accuracy of a delivery time prediction, and the speed with which the nearest courier is assigned. This whitepaper argues, and then measures, that both proxies are wrong in predictable directions. The promise a platform quotes at checkout is not a forecast; it is a newsvendor decision against a target the platform sets itself. The cost of missing that promise is not symmetric; it is convex, with a step at the refund threshold. The cost of a slow delivery is not a property of the clock; it is a property of the order, because fries and milkshakes are destroyed on timescales where brisket is untouched. And the minutes a courier spends waiting at a restaurant counter are not a courier problem; they are fleet capacity destroyed by dispatching against a mis-estimated food-ready time. We build a system that makes each claim measurable: an interval-censored ready-time model, an Experience-at-Risk (EAR) objective in USD, and a coupled promise/dispatch optimizer with an explicit hold action. We score it against the best conventional approach in a discrete-event simulation of a US metro zone over 200 seeds, with bootstrap confidence intervals on every published number. Every figure in this document traces to a committed file under results/.

1. The problem#

A US meal delivery platform makes three decisions in the seconds after checkout: what delivery time to promise, which courier to send, and when to send them. The industry-standard pipeline treats the first as a regression problem (predict delivery time, add a buffer) and the last two as a speed problem (assign the nearest free courier immediately).

Lateness, however, is defined against a promise the platform itself chooses. Quote longer and you hit the target more often but convert fewer checkouts; quote shorter and you win the order and inherit the right tail of the delivery distribution. That is a newsvendor problem with an endogenous target, and a model that minimizes mean absolute error against realized delivery time is optimizing a quantity nobody is paid on.

The loss function on the other side of the promise is not symmetric either. Two minutes late is free. Ten minutes costs goodwill. Twenty-five minutes costs a refund credit, a support contact, and a measurable churn hazard for a repeat customer. Symmetric loss on a convex cost curve is a systematic error, and it errs in a predictable direction: it underweights exactly the tail breaches that cost the most.

Meanwhile the product itself decays at rates the clock cannot see. A carton of fries is ruined in about twelve minutes in the bag; a brisket sandwich is fine at forty. A $19 wings-and-fries order at 10:15 PM in the snow can carry more regret than an $84 BBQ platter, yet a dispatcher ranking by order value or lateness risk alone cannot see it. Systems built on those rankings deprioritize precisely the orders that produce the worst experiences on the platform.

2. Why this is hard#

Four structural difficulties separate this from a standard prediction exercise.

Observability depends on the decision being made. The platform never observes food-ready time; it observes a pickup scan. If the courier arrived before the food was ready, the scan approximates ready time. If the courier arrived after, the scan reveals only that ready time was at or before that moment, a left-censored observation. Dispatch earlier and your training data gets cleaner while your couriers idle; dispatch later and your ready-time signal degrades into censoring. The instrument and the decision are coupled.

Logged data is policy-confounded. Outcomes in production logs exist only under the dispatch and quoting policy that produced them. No public dataset carries counterfactuals, so no policy-comparison claim can be validated on logged data alone (see §10).

The conversion response is not identifiable here. How many checkouts a longer quote loses is the load-bearing curve that stops "quote 90 minutes" from trivially winning. It cannot be identified from synthetic data we generated ourselves. We therefore state it as an assumption: a logistic curve with a citation-shaped default slope. We sweep it as a first-class axis in the sensitivity analysis (§9.4).

The objective is not separable across orders. Assigning a courier to order A removes them from the pool for order B, and batching couples delivery times directly. Ranking orders independently and assigning down the list is structurally unable to price this; the coupling must live in the assignment search itself.

3. System architecture#

The system is a Python package (mealsync) with a deliberately thin waist:

data/ (synthetic zone + licence-gated US loaders)
prep/ (censored ready-time quantiles)     travel/ (two-segment ETA model)
decay/ (thermal classes, fragility)       regret/ (breach cost, EAR)
promise/ (newsvendor quoting)             dispatch/ (planner + exact reference)
simulate/ (discrete-event engine, 5 policies, bootstrap reporting)
metrics/ (results pipeline)  api/ (FastAPI + SPA)  llm/ (grounded narration)

The core decision entry point is a pure function:

def plan_dispatch(state: ZoneState, config: DispatchConfig) -> DispatchPlan: ...

No I/O, no global state, no wall-clock reads, no hidden mutation: the function takes the observable world at time t and returns decisions. The simulator calls this function; it does not contain the logic. The same purity boundary separates world truth from platform belief everywhere: policies see only an observable projection (model quantiles, courier positions, ticket counts), and latent truth is used solely for scoring. Property-based tests enforce the planner's invariants: no courier double-assigned, batches never exceed B, shift windows never violated, and no order starved indefinitely by the hold logic. A purity test asserts identical output on identical input with the input state unmutated.

Every business-cost parameter, including grace band, goodwill slope, refund threshold and credit, churn hazard, and contribution margin, is a config input in USD, surfaced in the UI. A tool whose conclusions depend on hidden constants will not be trusted, so there are none.

4. Ready-time methodology under interval censoring#

The target is order acceptance to food ready. The dominant feature is the kitchen's concurrent open-ticket count at creation (as a ratio to capacity); the model exists to price kitchen load, not to average over it.

The censoring structure is handled with an exact likelihood rather than an approximation. We fit a lognormal accelerated-failure-time model, log T = βx + σε, by maximum likelihood: exactly observed pickups contribute the density, courier-late pickups contribute the distribution function at the courier's arrival (all we learned is T ≤ arrival). The naive comparison is the same model family trained on raw scan times, so the ablation isolates censoring handling and nothing else. One further observable-only correction: on courier-late orders, scan minus arrival is the counter handoff lag exactly, so its median de-biases the waited-order labels.

The naive fit learns max(ready, arrival), which reflects courier arrival behavior rather than kitchen behavior, and is biased upward by construction. On the held-out synthetic corpus the numbers are stark: the naive fit's median prediction carries a +6.4-minute bias and an 8.8-minute MAE against latent ground truth, and its q90 "coverage" of 0.986 reveals a model that has quietly learned to predict courier arrival rather than kitchen completion. The censored fit cuts the MAE to 3.6 minutes with residual bias of −0.8 minutes and q90 coverage of 0.909 against a nominal 0.90 (results/prep/censoring_ablation.csv). Replicated across 200 independently generated corpora with both models refit per corpus, the error reduction is 56% with a 95% bootstrap CI over seeds of [55.5%, 56.8%] (results/prep/censoring_ablation_by_seed.csv; one of the three published headline metrics, §9.5). Calibration is reported per kitchen-load decile, not in aggregate, because miscalibration at high load is where dispatch breaks (results/prep/coverage_calibration.csv).

The travel layer is deliberately simple and honest: two segments modeled separately (courier-to-restaurant is dominated by positioning, restaurant-to-customer by distance and congestion), ordinary least squares on log minutes with hour-bin, weather and courier-speed effects, lognormal quantile output, miles and mph throughout. It beats constant-speed and distance-linear baselines (results/travel/travel_error.csv) and is not dressed up as a contribution.

5. Decay and order fragility#

Each menu item carries one of seven thermal/texture classes. Each class has a decay curve Q_c(τ), the retained experience quality after τ minutes in the bag, parameterized by a half-life and a hard cliff (the point where fries become a different, worse product). Crisp-fried and frozen-dessert are genuinely punishing (half-lives of 9 and 7 minutes, cliffs at 12 and 10); grilled protein and hot-saucy are genuinely forgiving.

An order's fragility is driven by its worst meaningful component:

fragility(order, τ) = 1 − min over meaningful items i of Q_class(i)(τ)

because one carton of soggy fries ruins the order regardless of what else is in the bag. "Meaningful" is a salience rule: an item counts if its line total is at least $3.00 or 10% of basket value. That rule is a modelling choice with arbitrary constants, not a measurement; its only job is to stop a condiment from defining an order's fragility, and every surface that presents fragility says so.

The definition of τ follows T_deliver − max(T_ready, T_arrive), so quality decay while finished food waits on the counter for a late courier is not counted. That exclusion biases the objective mildly in favour of later dispatch; we state the direction here rather than in a footnote.

6. Experience at Risk#

For order o under candidate decision d (courier, dispatch time, batch, promise):

EAR(o,d) = E[C_breach(L)]                    breach cost, convex
         + E[V(o) · fragility(o, τ)]         quality destruction
         + κ · E[idle(o,d)]                  capacity tax

C_breach is piecewise: a free grace band (2 min), a rising goodwill cost ($0.35 per late minute), a step at the 25-minute refund threshold (a credit of $7.00 or 25% of order value, whichever is larger), plus a churn-hazard term for repeat customers (4% × $120 future value). Two couplings are configured business inputs rather than fitted quantities: refund credits scale with order value, and the goodwill slope steepens with fragility (a late order that is also ruined compounds anger). Every one of these parameters is editable in the UI in USD; change them and the rankings move.

κ prices a courier-minute and is computed from two configured business inputs rather than fitted: contribution margin × marginal PEAK orders per courier-minute, $0.165 at the defaults. The peak-versus-average distinction matters because the simulation's realized shift-average throughput is ~0.8-0.9 orders per courier-hour, while κ prices the contended peak minute. This distinction is spelled out in the decision log. The expectation over ready time uses a five-point scenario approximation on the model's published quantiles (q50–q95), weighted to respect the right tail where breaches live.

Direction-of-bias notes for the assumptions: excluding counter-sit decay favors later dispatch (§5); the scenario approximation slightly smooths the extreme tail, which understates rare-event breach cost for both compared policies equally; the fragility-goodwill coupling raises quoted promises on fragile orders and is swept implicitly through decay severity.

7. Promise setting as a newsvendor decision#

Given the full delivery-time distribution, the quote minimizes

(1 − conversion(q)) · margin  +  conversion(q) · E[C_breach(T_deliver − q)]

This represents expected margin lost to non-conversion plus expected breach cost if the order is won. This is a newsvendor decision, not a point forecast plus a buffer, and the optimal quantile is not fixed: it shifts up with repeat status (churn hazard), with order value (refund scaling), and with fragility (goodwill coupling), behavior a fixed buffer cannot reproduce, as demonstrated interactively in the promise explorer and asserted in tests.

The conversion curve doing the balancing is an assumption (§2). The default logistic slope loses roughly 1 percentage point of conversion per added promised minute near the reference quote, inside the range platform publications report for ETA sensitivity. Section 9.4 shows how the policy gap moves as this assumption is varied by a factor of six in both directions.

8. Dispatch: coupling, holds, and an honest bound#

Dispatch is a matching over (orders × couriers × dispatch-time offsets × batch groupings) minimizing total EAR subject to fleet availability, batch size B = 3, batch geographic coherence, and shift windows. Candidates are ranked by EAR plus κ × pre-dispatch wait. Committing a courier to a future dispatch time locks them from the moment of the decision, so that wait is capacity consumed and is priced like any other courier-minute (an adversarial review of an earlier build caught this wait being free in the objective while the simulator charged it; pricing it improved every headline outcome). The heuristic: regret-based global matching (the order with the most to lose between its best and second-best courier chooses first), a whole-fleet rescue pass, batch insertion and merge for same-restaurant trips with coherent drop-offs, local courier swaps, and an explicit, first-class hold: the deliberate decision not to dispatch because the food will not be ready and the courier is worth more in the pool. Holds carry a written reason, a review time, and a hard starvation deadline (20 minutes) after which the order must dispatch on its best available option; the no-starvation property is tested explicitly. The spec expected most of the capacity gain to come from holds; the clean ablation (deliberate holds on vs. off, fleet-exhaustion queueing recorded separately; results/dispatch/hold_decision_impact.csv) attributes about 1.0 of the 3.1 minutes-per-order counter-idle reduction and $0.23 of EAR per order to the hold itself, with throughput unchanged; the rest comes from pool discipline (commit horizon, regret matching, priced pre-dispatch waits) and load-aware quoting. Reported as measured.

The exact reference: with batching disabled and per-pair timing optimized independently, the problem is a rectangular assignment problem, which scipy's min-cost matching solves to proven optimality on instances up to 50 orders × 30 couriers. Measured this way over 24 seeded instances from 10×12 to 30×30, the heuristic's median optimality gap is 2.7%; the worst single instance measured 10.5% (results/dispatch/optimality_gap.csv). The bound must also be stated honestly: with batching enabled, or with more orders than couriers (where leaving an order unserved becomes a decision), the problem is no longer a clean assignment problem and the exact reference no longer bounds the heuristic. We do not extend the guarantee past where it applies.

9. Results#

Methodology first: the policy comparison aggregates a discrete-event simulation of one Friday service day (10:00–23:00) in a 5×5-mile US metro zone over 200 seeds; the sensitivity sweep uses 50 seeds per axis value, the hold ablation 40 paired seeds, the optimality-gap harness 24 seeded instances, the regret attribution 20 seeds of per-order records, and the ready-time ablation 200 independently generated corpora; each source's count is stated where its numbers appear, and results/env.json records every seed set. All intervals are 95% bootstrap CIs; policy deltas use paired bootstrap over per-seed differences, because all policies face the same world per seed. The baseline is mae_optimal, the best conventional approach (best point-ETA model, point + fixed buffer promise, greedy assignment), not the weakest policy on the board. Breach metrics are reported against a fixed neutral 40-minute benchmark as well as each policy's own promise, because a policy quoting 90 minutes trivially wins on its own promise. Single-run numbers appear nowhere.

9.1 Policy board#

Per-seed rows are in results/simulation/policy_comparison.csv; means, medians and CIs in results/simulation/policy_summary.json. The headline comparison, treatment (ear_coupled) vs. baseline (mae_optimal), paired over 200 shared seeds; every interval below is a 95% paired bootstrap CI on the per-seed difference:

The capacity metric deserves its footnote in the main text: it counts counter idle AND the pre-dispatch lock created when a courier is committed to a future dispatch time. An adversarial review of an earlier build found the pre-dispatch component unpriced in the objective and uncounted in the metric, which had inflated the apparent idle reduction to 85%. Pricing and counting it shrank the published number to 58% and, by removing the distortion, improved every other outcome above. Residual pre-dispatch wait is 0.17 minutes per order.

The board behaves as §5.6 predicted, now with five genuinely distinct policies: nearest_courier (model-free constant-prep quote, greedy dispatch) posts $17.62 EAR per order; mae_optimal's calibrated point promise closes most of that gap to $12.71; the remaining $0.72 per order, plus the capacity, refund, throughput and conversion columns above, is what distribution-aware, regret-coupled decisions buy on top of a good point model. fastest_promise wins conversion and drowns in own-promise breaches; utilisation_max batches hard and pays in quality.

9.2 The dual breach metrics#

Against its own promise, the regret-coupled policy breaches more often than the baseline (39.6% vs. 17.1%) and shallowly: the optimizer discovered that lateness inside the grace and goodwill bands is cheap, and spent promise length on conversion instead (its mean quote is 37.3 minutes vs. the baseline's 45.5). Against the fixed neutral 40-minute benchmark it is better on both measures: overall breaches fall 37.4% → 33.9% and refund-band breaches, the expensive tail, fall by 43%. A policy scored only on its own promise would look worse while delivering faster at the same benchmark; this asymmetry is the convex cost curve doing exactly what §1 said it would, and it is why the neutral benchmark is non-negotiable in every published breach number.

9.3 Where the regret lives: an honest correction to the framing#

We expected "the highest-regret orders are not the highest-value ones" in its strong form. That is not what we found: in results/decay/value_vs_fragility.csv only 2.6% of the top realized-regret decile has below-median basket value, because quality destruction is V(o) × fragility. Value multiplies regret by construction, and the top of the regret ranking is value-dominated.

What the data does show is the operationally important form of the claim: fragility massively reorders regret within and across value bands. Holding value fixed, the most fragile band carries 2.3–4.9× the realized regret of the least fragile band in every value decile. Crossing bands: a fragile order from value decile 3 (mean realized EAR $11.80) outranks a resilient order from value decile 8 ($6.27). That is the $19 wings order outranking the $84 platter. The result is real, measured, and invisible to a value-ranked queue, and is stated with the correlation caveat a careful reader would demand. results/simulation/regret_attribution.csv shows the treatment's gains concentrating in the fragile deciles, as predicted.

9.4 Sensitivity#

results/simulation/sensitivity_sweep.csv sweeps decay severity (0.5–2×), prep-time variance (0.6–1.5×), fleet tightness (0.30–0.60 couriers per peak order), conversion elasticity (0.015–0.09 logit/min) and the refund threshold (20–30 min), 50 seeds per axis value, treatment vs. baseline. The EAR advantage holds at every swept value and grows where the thesis says it should: with decay severity (−$0.52 to −$0.69 per order across the range) and dramatically with fleet tightness (−$0.62 per order at 0.60 couriers per peak order, −$22.2 at 0.30, where the baseline collapses under scarcity).

Two regime boundaries deserve precision rather than a blanket robustness claim. First, at the tightest fleet setting (0.30) the treatment serves significantly fewer orders per courier-hour (−0.077, CI [−0.093, −0.061]). Under extreme scarcity it holds and quotes conservatively to protect experience rather than maximizing raw throughput, and its EAR advantage there is the largest on the board. Second, at the steepest conversion elasticity (0.09 logit/min) the refund-band advantage compresses to statistical indistinguishability (+0.08pp, CI crosses zero): when every added promised minute is punishing, both policies quote short and the refund tail converges. Both boundaries are visible in the committed sweep artifact.

9.5 The published headline metrics#

results/headline.json publishes exactly three metrics: the largest CI-supported effects, computed from committed artifacts, baseline mae_optimal, never a policy's own promise:

• 58%: Courier capacity recovered. Courier minutes waiting per order (counter idle + pre-dispatch lock): 5.0 → 2.1 (CI on relative change [56.5%, 60.2%], 200 comparison seeds).
• 56%: Honest ready-time estimates. Ready-time error vs. ground truth, censored vs. naive fit: 8.2 → 3.6 minutes (CI [55.5%, 56.8%], 200 independently generated corpora).
• 43%: Fewer refund-threshold breaches. Refund-band breach rate at the fixed 40-minute benchmark: 4.0% → 2.2% (CI [35.6%, 52.0%], 200 comparison seeds).

No candidate had to be excluded for a zero-crossing CI in the committed run; the exclusion rule and the candidates it would apply to are recorded in results/REPORT.md. Nothing is stated anywhere in this repository that does not trace to a file under results/.

10. Limitations#

• Synthetic ground truth is the only counterfactual source. The policy comparison is internally valid (same worlds, latent truth held out from policies) but the world itself is generated. The censoring mechanism, load-dependent prep, and thermal decay are modelled, not measured.
• Public US datasets cannot validate the core claims. The Grubhub MDRP instances, the DoorDash duration dataset and NYC TLC records carry no item-level composition (no decay claim) and no counterfactuals (no policy claim). They can check travel/prep model structure and stress the allocator; docs/DATA.md states each licence finding and limit plainly.
• Conversion response is assumed, stated in §2 and §7, and swept rather than estimated.
• Single zone, v0 baselines. No multi-zone balancing, no courier supply behavior, including tips (a real US economic factor deliberately out of scope on the supply side), no real road network (a travel-time model, not a router), and same-restaurant-only batching.
• The exact reference bounds a restricted problem (§8); heuristic quality under batching is not certified.

11. Roadmap#

• Multi-zone balancing and courier repositioning.
• Courier supply response: acceptance, tips, incentive elasticity.
• Off-policy evaluation on logged platform data (importance weighting against the logging policy), replacing simulator counterfactuals where possible.
• Live KDS integration for observed ready times, which collapses the censoring problem at its source instead of modelling around it.
• Cross-restaurant batching with cascade-aware timing.

Reproduce everything: uv run python -m mealsync.metrics.headline --full regenerates results/ from a clean checkout; results/env.json pins seeds, package versions and the git SHA of the committed run.