Abstract
We analytically derive probability density functions of the sampled electrically filtered photo current for optical amplitude-shift keying and differential binary phase-shift keying using the Karhunen-Loe/spl grave/ve series expansion method. This allows us to determine exact branch metrics for a Viterbi equalizer. We then compare the performance of histogram-based Viterbi equalizers with those using analytically calculated branch metrics.