Differentiable Agent-Based World Models for Counterfactual Policy Analysis
Discrete agent choices block gradient flow in classical agent-based models. We relax via Gumbel-softmax, calibrate by simulated minimum distance, define interventions as program transformations, and treat counterfactuals as fixed-noise replay. A synthetic P&C claims environment demonstrates fraud detection and policy evaluation.
Figure 1. The agent-based roll-out as a single stochastic computation graph. Parameters θ flow forward through policy logits, Gumbel-softmax sampling, state recurrence, and aggregation into a scalar loss. Gradients propagate backward through reverse-mode automatic differentiation.
The differentiability barrier
Agent-based models capture micro-founded behaviour: individual agents make discrete choices that aggregate into macro phenomena. Classical ABMs suffer two practical problems: calibration is expensive, and counterfactual analysis is difficult, because the discrete choices block gradient flow.
This report shows how to relax discrete choices via the Gumbel-softmax (Concrete) distribution, calibrate by simulated minimum distance, define interventions as program transformations, and treat counterfactuals as fixed-noise replay.
Gradients through stochastic simulators
Two estimators are commonly used to backpropagate through stochastic operations. The score-function (REINFORCE) estimator is unbiased but high-variance. The pathwise estimator has lower variance but requires a differentiable sampling path. Gumbel-softmax provides such a path for categorical variables.
Calibration by simulated minimum distance
Given observed moments m_obs of the real system and simulated moments m_sim from the ABM at parameter θ, calibration minimises the weighted squared distance. With Gumbel-softmax relaxation, ∂L/∂θ flows through the entire roll-out and a standard optimiser converges in minutes where genetic algorithms took days.
Interventions and counterfactuals
Interventions are formalised as program transformations: a function that substitutes one sub-expression of the simulator with another. Counterfactual analysis becomes 'replay with the same noise vector but a different program', mirroring the abduction step of structural causal models.
This structure permits the substrate to answer 'what would have happened if?' with the same level of rigor as it answers 'what is happening now?', under the same noise realisation, eliminating Monte Carlo confounding.
Case study: P&C claims
We instantiate a synthetic property-and-casualty environment with heterogeneous claimant agents (including a planted fraud ring) routed to adjuster agents through assignment and settlement dynamics. The differentiable ABM recovers the planted fraud structure under simulated minimum distance and supports counterfactual policy evaluation against a held-out adjuster value flip.
Papadopoulos, L. (2026). Differentiable Agent-Based World Models for Counterfactual Policy Analysis. Helios Brain, HB-TR-2026-03.