Coupling of metal-insulator and antiferromagnetic transitions in the highly correlated organic conductor incorporating magnetic anions, $\lambda -{\mathrm{BETS}}_2{\mathrm{FeBr}}_x{\mathrm{Cl}}_{4-x}$ $[\mathrm{B}\mathrm{E}\mathrm{T}\mathrm{S}=\mathrm{B}\mathrm{i}\mathrm{s}(\mathrm{ethylenedithio}\left)\mathrm{tetraselenafulvalene}\right]$

The electric and magnetic properties of a series of highly correlated two-dimensional organic π conductors incorporating magnetic ions (${\mathrm{Fe}}^{3+}$ with $S=\frac{5}{2}$), $\lambda -{\mathrm{BETS}}_2{\mathrm{FeBr}}_x{\mathrm{Cl}}_{4-x}$ [$\mathrm{B}\mathrm{E}\mathrm{T}\mathrm{S}=\mathrm{b}\mathrm{i}\mathrm{s}\left(\mathrm{ethylenedithio}\right)\mathrm{tetraselenafulvalene},$ $x=0\mathrm{}–0.8$] were examined by controlling the Br content (x). A broad resistivity maximum at 100 $\left(x\approx 0\right)–50\mathrm{K}$ $(x\approx 0.7)$ indicating the strong correlation of π conduction electrons becomes prominent with increasing x. At the same time the metal-insulator (MI) transition temperature ${(T}_{\mathrm{MI}})$ increases from 8.5 $\mathrm{}(x=0\mathrm{})\mathrm{}$ to 18 K $\mathrm{}(x=0.7\mathrm{}).$ The crystal with $x=0.8\mathrm{}$ shows a semiconductor-insulator transition around 20 K. At $0<x<\mathrm{}0.2,$ the MI transition and the antiferromagnetic (AF) transitions take place cooperatively around 8.5 K ${(T}_N\approx T_{\mathrm{MI}}).\mathrm{}$ A large magnetization drop was observed at $T_{\mathrm{MI}}$ for the magnetic field parallel to the c axis $\left(H\parallel c\right),$ which indicates the appearance of localized π spins $(S=\frac{1}{2})$ and a strong AF coupling of π and d spin systems. For $0.3<x<\mathrm{}0.5,$ two anomalies were observed in the magnetization-temperature $\mathrm{}(M-T)\mathrm{}$ curve. The high-temperature anomaly corresponds to a MI transition ${(T}_{\mathrm{MI}})$ and the low-temperature one corresponds to an AF ordering of the ${\mathrm{Fe}}^{3+}$ spins ${(T}_N).\mathrm{}$ In this case, the relatively small magnetization drop observed at $T_{\mathrm{MI}}$ suggests a small coupling of π and d spin systems. For $x>\mathrm{}0.6,$ the magnetization drop at $T_{\mathrm{MI}}$ disappeared, which shows that the π and d electron systems are decoupled and the π electron system undergoes MI transition independently of the d spin systems. The disappearance of the susceptibility anomaly at $T_{\mathrm{MI}}$ indicates that the π electron system transforms to a nonmagnetic insulating state below $T_{\mathrm{MI}}.$ On the other hand, the anisotropy of M showing the development of a AF spin structure of the Fe spins was observed independently of $x(0<x<\mathrm{}0.8\mathrm{}).$ Around $x=0.3\mathrm{}-0.5,$ the direction of the easy axis of the AF spin structure changes from parallel to the c axis $\mathrm{}(x<\mathrm{}0.2\mathrm{})\mathrm{}$ to perpendicular to it $\mathrm{}(x>\mathrm{}0.6\mathrm{}).$ In other words, the direction of easy axis is varied according to the magnitude of $\pi -d$ coupling. Magnetoresistance measurements showed that $T_{\mathrm{MI}}$ of $\lambda -{\mathrm{BETS}}_2{\mathrm{FeBr}}_{0.7}{\mathrm{Cl}}_{3.3}$ is almost independent of the magnetic field below 1 kbar. But a field-restored highly conducting state similar to that of $\lambda -{\mathrm{BETS}}_2{\mathrm{FeCl}}_4$ was observed at high pressure. The Weiss temperature (θ) estimated from the $M-T$ curve at $T_{\mathrm{MI}}<T<\mathrm{}30\mathrm{}\mathrm{K}$ decreases with increasing x.