Performance gradient estimation for the very large finite Markov chains

1 January 1991

journal article
Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control

Vol. 36 (11) , 1218-1227
https://doi.org/10.1109/9.100931

Abstract

Using an embedded Markov chain, the steady-state performance gradient estimation for very large Markov chains is decomposed into two smaller problems, one solved by the stochastic recursive (SR) method and the other by the likelihood ratio (LR) method. The combination, named SR-LR, can have several magnitudes of lower space requirement than the SR and several magnitudes of lower time consumptions than the LR. Both the LR and the SR are the extreme cases of the combined algorithm. The authors demonstrate the SR-LR methods on Markov chains with hundreds of millions of states, which are created using queuing networks.

Keywords

This publication has 13 references indexed in Scilit:

Variance reduction for likelihood-ratio method
Published by Institute of Electrical and Electronics Engineers (IEEE) ,2003
Improvements in the likelihood ratio method for steady-state sensitivity analysis and simulation
Performance Evaluation, 1992
Sensitivity Analysis for Simulations via Likelihood Ratios
Operations Research, 1989
Light traffic derivatives via likelihood ratios
IEEE Transactions on Information Theory, 1989
Sensitivity Analysis and Performance Extrapolation for Computer Simulation Models
Operations Research, 1989
Likelilood ratio gradient estimation
Published by Association for Computing Machinery (ACM) ,1987
Regeneration and Networks of Queues
Published by Springer Nature ,1987
The score function approach for sensitivity analysis of computer simulation models
Mathematics and Computers in Simulation, 1986
Stochastic Approximation Methods for Constrained and Unconstrained Systems
Published by Springer Nature ,1978
Simulating Stable Stochastic Systems: III. Regenerative Processes and Discrete-Event Simulations
Operations Research, 1975