Malliavin Calculus with Weak Derivatives for Counterfactual Stochastic Optimization

Krishnamurthy, Vikram; Snow, Luke

Permalink

https://hdl.handle.net/2142/130297 Copy

Description

Title

Malliavin Calculus with Weak Derivatives for Counterfactual Stochastic Optimization

Author(s)

• Krishnamurthy, Vikram
• Snow, Luke

Issue Date

2025-09-17

Keyword(s)

• Malliavin derivative
• Weak derivative estimator
• Neural stochastic differential equation
• Stochastic gradient algorithm

Abstract

We study counterfactual stochastic optimization of conditional loss functionals under misspecified and noisy gradient information. The difficulty is that when the conditioning event has vanishing or zero probability, naïve Monte Carlo estimators are prohibitively inefficient; kernel smoothing, though common, suffers from slow convergence. We propose a two-stage kernel-free methodology. First, we show using Malliavin calculus that the conditional loss functional of a diffusion process admits an exact representation as a Skorohod integral, yielding variance comparable to classical Monte-Carlo variance. Second, we establish that a weak derivative estimate of the conditional loss functional with respect to model parameters can be evaluated with constant variance, in contrast to the widely used score function method whose variance grows linearly in the sample path length. Together, these results yield an efficient framework for counterfactual conditional stochastic gradient algorithms in rare-event regimes.

Publisher

Allerton Conference on Communication, Control, and Computing

Series/Report Name or Number

2025 61st Allerton Conference on Communication, Control, and Computing Proceedings

ISSN

2836-4503

Type of Resource

Text

Genre of Resource

Conference Paper/Presentation

Language

eng

Handle URL

https://hdl.handle.net/2142/130297&&

Copyright and License Information

Copyright 2025 is held by Vikram Krishnamurthy and Luke Snow.

Owning Collections