Shear instabilities of a vortex lattice in layered superconductors