Likelihood Ratio Gradient Estimation: An Overview

Abstract
The likelihood ratio method for gradient estimation is briefly surveyed. Two applications settings are described, namely Monte Carlo optimization and statistical analysis of complex stochastic systems. Steady- state gradient estimation is emphasized, and both regenerative and non- regenerative approaches are given. The paper also indicates how these methods apply to general discrete-event simulations; the idea is to view such systems as general state space Markov chains. This document considers a single-server queue in which the service rate theta is a decision variable. Given that alpha(theta) is the steady-state cost of running the queue at parameter level theta, one is frequently interested in minimizing alpha (theta) over a suitable constraint set. Since alpha is often difficult to evaluate analytically, Monte Carlo optimization is an attractive methodology. By analogy with deterministic mathematical programming, efficient Monte Carlo gradient estimation is typically an important ingredient of simulation based optimization algorithms. As a consequence, gradient estimation has recently attracted considerable attention in th simulation community. It is the author's goal, in this paper, to describe one such method for estimating gradients in the Monte Carlo setting, namely the likelihood ratio method.

This publication has 0 references indexed in Scilit: