Instability of a coupled rotor-phonon system