This paper explores in some detail a semi-popular text written by Einstein to present his theory of relativity. Semiotic tools are used to compare what Einstein says about the activity of building spaces and times with what sociologists of science can tell us. Einstein's text is read as a contribution to the sociology of delegation. Once the drama of Einstein's argument has been reconstructed, it is possible to learn from his theory of relativity something about the classical problem of `relativity' in the STS field. A comparison is established between the notion of social context and that of the aether, and an argument is developed to lead us beyond `social' explanations. The goal of such a semiotic study is twofold: to allow the adaptation of the strong programme to the peculiar conditions of the theoretical sciences; and to find a vocabulary for an activity best defined as infra-physics.