The magnetic field dependence of the anomalous attenuation peak for ultrasonic shear waves propagating along the c axis in Ho single crystals near 90 K has been studied. We find that a magnetic field has a pronounced effect on the attenuation peak and that for certain field orientations the attenuation anomaly disappears altogether. For some field orientations the attenuation showed hysteresis effects when the field was cycled. We have also found evidence for considerable rotation of the plane of polarization of the sound waves when the field was near the c axis or the basal plane. It is hoped that the results will contribute to the understanding of the nature of the attenuation anomaly.