Impurity dephasing by optical modes in polymers and glasses