Reinforcement Learning Based Computationally Efficient Conditional Choice Simulation Estimation of Dynamic Discrete Choice Models
(A2026-133656)

AUTHORS

Sonal Srivastava, University of Cambridge; Ahmed Khwaja, University of Cambridge Judge Business School

ABSTRACT

Dynamic discrete choice (DDC) models are widely used in marketing, but their estimation becomes challenging in big data settings with high-dimensional state–action spaces. We propose a reinforcement learning (RL)–based, two-step Conditional Choice Simulation (CCS) estimation approach that improves scalability while preserving the interpretability of structural models, which is particularly valuable for counterfactual policy analysis. Our approach learns value function along simulated paths, allowing accurate estimation from much shorter trajectories. Monte Carlo studies show that the proposed methods match or exceed the accuracy of CCS while reducing computation time by up to an order of magnitude, and by nearly two orders of magnitude when the discount factor is close to one. Framing CCS estimation of DDC models as an RL problem increases their applicability and scalability to high-dimensional marketing problems while retaining both interpretability and tractability.