Some implications of the cutoff-rate criterion for coded direct-detection optical communication systems

Abstract
The cutoff rate is derived for a digital communication system employing an optical carrier and direct detection. The coordinated design of the encoder, optical modulator, and demodulator is then studied using the cutoff rate as a performance measure rather than the more commonly employed error probability. Modulator design is studied when transmitted optical signals are subject simultaneously to average-energy and peak-value constraints. Pulse-position modulation is shown to maximize the cutoff rate when the average-energy constraint predominates, and the best signals when the peak-value constraint predominates are identified in terms of Hadamard matrices. A time-sharing of these signals maximizes the cutoff rate when neither constraint dominates the other. Problems of efficient energy utilization, choice of input and output alphabet dimension, and the effect of random detector gain are addressed.