Matrix-valued special functions and representation theory of the conformal group. I. The generalized gamma function

This article examines in detail the matrix-valued gamma function \[ Γ λ 0 ( α ) = ∫ P e − tr ( r ) λ 0 ( r , r ¯ ) ( det r ) α − 2 d r {\Gamma ^{{\lambda ^0}}}\,(\alpha )\, = \,\int _P {{e^{ - {\text {tr}}(r)}}{\lambda ^0}(r,\,\bar r){{(\det \,r)}^{\alpha - 2}}\,} dr \] associated to the conformal group G = U ( 2 , 2 ) G\, = \,U(2,\,2) . Here, α \alpha is a continuous complex parameter, λ 0 {\lambda ^0} runs through a family of “weights” of K = U ( 2 ) × U ( 2 ) K\, = \,U(2)\, \times \,U(2) , P is the cone of 2 × 2 2\, \times \,2 positive-definite Hermitian matrices, and the integral is well known to converge absolutely for Re ( α ) > 1 {\text {Re}}(\alpha )\, > \,1 . However, until now very little has been known about the analytic continuation for the general weight λ 0 {\lambda ^0} . The results of this paper include the following: The complete analytic continuation of Γ λ 0 {\Gamma ^{{\lambda ^0}}} is determined for all weights λ 0 {\lambda ^0} . In analogy to the case of the classical gamma function it is proved that for any weight λ 0 {\lambda ^0} the mapping α → Γ λ 0 ( α ) − 1 \alpha \, \to \,{\Gamma ^{{\lambda ^0}}}\,{(\alpha )^{ - 1}} is entire. A new integral formula is given for the inverse of the gamma function. An explicit calculation is given for the normalized variant of the gamma matrix that arises in the reproducing kernel for the spaces in which the holomorphic discrete series of G is realized, and one observes that the behavior of the analytic continuation for weights “in general position” is markedly different from the special cases in which the gamma function “is scalar". The full analytic continuation of the holomorphic discrete series for G is determined. The gamma function for the forward light cone (the boundary orbit) is found, and the associated Hardy space of vector-valued holomorphic functions is described. Analogs are given for some of the well-known formulas for the classical gamma function. As an epilogue, applications of the matrix-valued gamma function, such as generalizations to 2 × 2 2\, \times \,2 matrix space of the classical binomial theorem, are announced. These applications require a detailed understanding of the (generalized) Bessel functions associated to the conformal group that will be treated in the sequel to this paper.