Abstract
We studied the structural, magnetic, and transport properties of the quasi-two-dimensional Mott transition system Ca2xSrxRuO4. In the vicinity of the metal-nonmetal (M-NM) transition at x0.2, we found a structural transition accompanied by a structural symmetry change with the instability point at xc0.5. The critical change across the structural transition in the temperature dependence of the susceptibility indicates a crossover of the metallic state, most likely from the nearly antiferromagnetic state next to the M-NM transition to the nearly ferromagnetic state around xc. The latter evolves into the spin-triplet superconductor Sr2RuO4 (x=2) with enhanced paramagnetism. We argue that the competition between the structural instability and the ferromagnetism results in such a structural symmetry change with orbital degeneracy lifting, which induces the switching of magnetic coupling. In addition, a changeover from metallic to nonmetallic behavior was observed across the structural transition in the out-of-plane resistivity, which reveals highly anisotropic transport due to the quasi-two-dimensional electronic structure.