Are rogue waves robust against perturbations?