Nuclear mass formula with Bogolyubov-enhanced shell-quenching: application to r-process