Does on-time delivery make customers come back?

The problem

Does an order arriving on time cause the customer to order again within 21 days?

This study uses 101,684 real JD.com orders from March 2018 (the public 2020 MSOM Data-Driven Research Challenge dataset), 18 confounders chosen from a causal graph, and three estimators: g-computation, inverse probability weighting and augmented IPW.

Headline numbers

• Raw comparison: on-time customers repurchased less, 3.70% versus 4.46%.
• Doubly robust (AIPW) primary estimate: -0.38 points, 95% CI [-0.73, +0.11].
• Under a strict definition of on time the effect is +0.51 points [+0.09, +0.86]; under a timestamp definition it is -0.06.
• E-value of 1.68: a hidden confounder about 1.7 times as associated with both delivery and repurchase would be enough to erase the estimate.

What it means

The sign of the answer depended on a metric definition that is usually chosen by whoever built the dashboard. What the data supports is narrower: the effect on 21-day repurchase is small under every definition, within about half a point either way. Written as the capstone for the M.S. in Business Analytics at Mercy University.