Matrices of random variables that are symmetric but not necessarily positive definite can arise in various ways. Assuming normality and a certain ‘invariant’ covariance structure, or alternatively assuming a Wishart structure, we derive exact likelihood ratio tests of certain hypotheses on the latent vectors. In the former case, these lead to exact confidence regions for the unknown vectors; approximations to these regions are given. The approximate validity of the regions in non-normal and non-invariant cases (for example, in the case of rounding-off errors) is investigated. An application to the analysis of a quadratic response surface is described.