On the cross-correlation of sub-mm sources and optically selected galaxies