Experimenters report an upper limit if the signal they are trying to detect is non-existent or below their experiment's sensitivity. Such experiments may be contaminated with a background too poorly understood to subtract. If the background is distributed differently in some parameter from the expected signal, it is possible to take advantage of this difference to get a stronger limit than would be possible if the difference in distribution were ignored. We discuss the ``Maximum Gap'' method, which finds the best gap between events for setting an upper limit, and generalize to ``Optimum Interval'' methods, which use intervals with especially few events. These methods, which apply to the case of relatively small backgrounds, do not use binning, are relatively insensitive to cuts on the range of the parameter, are parameter independent (i.e., do not change when a one-one change of variables is made), and provide true, though possibly conservative, classical one-sided confidence intervals.