Robust limits on Lorentz violation from gamma-ray bursts