A non-linear viscoelastic model with structure-dependent relaxation times