Time-integrated and time-dependent angular analyses ofB→J/ψKπ: A measurement ofcos2βwith no sign ambiguity from strong phases

We present results on $B\to J/\psi K\pi$ decays using $e^{+}{e}^{-}$annihilation data collected with the BABAR detector at the ${\rm Y}(4S)$ resonance. The detector is located at the PEP-II asymmetric-energy storage ring facility at the Stanford Linear Accelerator Center. Using approximately $88\times {10}^6$ $B\overline{B}$ pairs, we measure the decay amplitudes for the flavor eigenmodes and observe strong-phase differences indicative of final-state interactions with a significance of 7.6 standard deviations. We use the interference between the $K\pi$ $S$-wave and $P$-wave amplitudes in the region of the $K^{*}(892)$ to resolve the ambiguity in the determination of these strong phases. We then perform an ambiguity-free measurement of $\cos 2\beta$ using the angular and time-dependent asymmetry in $B\to J/\psi K^{*0}(K_S^0{\pi }^0)$ decays. With $\sin 2\beta$ fixed at its measured value and $\cos 2\beta$ treated as an independent parameter, we find $\cos 2\beta ={2.72}_{-0.79}^{+0.50}(\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{t})\pm 0.27(\mathrm{s}\mathrm{y}\mathrm{s}\mathrm{t})$, determining the sign of $\cos 2\beta$ to be positive at 86% C.L.