A RANKIN-SELBERG INTEGRAL USING

THE AUTOMORPHIC MINIMAL REPRESENTATION OF SO(7)

by Daniel Bump, Solomon Friedberg, and David Ginzburg

Stanford University Boston College Tel-Aviv University

Abstract. We show that a certain residue of a metaplectic Eisenstein series on SO_{7} affords the automorphic minimal representation, whose local theory was described by Torasso, Roskies and Sabourin. As an application, we give a Rankin-Selberg construction of the Langlands L-function for the third fundamental representation of the L-group Sp(6, ℂ), which is an Euler product of degree 14 attached to a generic cusp form on SO_{7}.

Let SO_{7} denote the special split orthogonal group stabilizing the quadratic form with matrix

( \begin{matrix}& & & & & & 1 \\ & & & & & 1 \\ & & & & 1 \\ & & & 1 \\ & & 1 \\ & 1 \\ 1 \end{matrix} ) .

If F is a local field of residue characteristic not equal to 2, there exists a projective representation of SO_{2n+1}(F) attached to the minimal coadjoint orbit only if 2n+1 ≤ 7. By contrast, minimal representations for simply laced groups—those whose root systems have only one length of root—were constructed in every case by Kazhdan and Savin [K-S].

If 2n + 1 = 7 the construction of the minimal representation of SO_{7} over a local field may be carried out along the lines of Kazhdan and Savin [K-S], though the construction breaks down for SO_{9} and beyond. The SO_{7} construction was carried out by Roskies [Ro] in the nonarchimedean case and by Sabourin [Sab] in the real archimedean case. Torasso [To] also constructs the minimal representation of SO_{7}

Typeset by AMS-TEX

1991 Mathematics Subject Classification. Primary 11F70, Secondary 11F46, 22E46.

Key words and phrases. Rankin-Selberg integral, minimal representation.

Research supported by NSF grants DMS-9622819 (Bump) and DMS-9896186 (Friedberg), by the Institute for Advanced Study and the Banker’s Trust Company Foundation.

as one case of a general theory encompassing many groups. These constructions give a true representation not of SO_{7}, but of a metaplectic double cover S̃O_{7}. We will describe this group carefully later under the assumption that F contains the group μ_{4} of fourth roots of unity in its algebraic closure.

The minimal representation of SO_{7} may be constructed as follows. Begin with the parabolic subgroup Q which is the semidirect product of its Levi factor GL_{2} × SO_{3} with a Heisenberg group H_{7}; let Q^{0} be the subgroup of Q which is the semidirect product of SL_{2} × SO_{3} and H_{7}. Let Q̃(F) and Q̃^{0}(F) be the preimages of Q(F) and Q^{0}(F) in S̃O_{7}(F), respectively. The usual Weil representation ([We]) is a projective representation of precisely the semidirect product Q̃^{0}(F) = (S̃L_{2}(F) × SO_{3}(F)).H_{7}(F). It acts on the Schwartz space of functions on F^{3}, so its Gelfand-Kirillov dimension is 3. The representation of Q̃(F) compactly induced from this representation of the codimension one subgroup Q̃^{0}(F) therefore has Gelfand-Kirillov dimension 4. The theorem proved by Roskies, Torasso and Sabourin is that this representation may be extended to a representation Θ of S̃O_{7}(F).

There is a gap (presumably not serious) in that we do not have a satisfactory reference for the Langlands parameters of the minimal representation in the nonarchimedean case. Roskies constructs a functional which seems to embed the representation in the correct quotient of a principal series representation. Unfortunately, due to the difficulty of characterizing the smooth vectors in her geometric realization of the representation, it is not clear that the integral defining this functional is convergent for all smooth vectors. Therefore we will assume a technical Hypothesis 1.6.

Now let F be a global field, 𝔸 its adele ring. One may ask if the minimal representation corresponds to a global automorphic representation. We will establish this automorphicity by showing that a certain Eisenstein series has a pole at a location whose Langlands parameters correspond to the expected Langlands parameters of the minimal representation. Thus if (Θ, V_{Θ}) denotes the restricted tensor product of the local representations described above, the existence of this pole implies (under Hypothesis 1.6) that Θ is automorphic. A similar result is obtained for simply laced groups by Ginzburg, Rallis and Soudry [G-R-S].

Let π be a generic cuspidal automorphic representation of SO_{7}(𝔸). The L-group Sp(6, ℂ) of SO_{7} has two fourteen dimensional irreducible representations. The one which we are concerned with is the representation ρ corresponding to the third fundamental weight. We will give a Rankin-Selberg integral for the Langlands Lfunction L(s, π, ρ). We integrate the product of a cusp form ϕ in the space of π against an Eisenstein series and a “theta function” in the space V_{Θ}.

Assuming the functoriality conjecture of Langlands, the L-function L(s, π, ρ) can have a pole if π is a lift from SU(2, 1). The Rankin-Selberg integral representation gives a criterion for this L-function to have a pole, namely π must occur in Θ ⊗ Θ′, where Θ′ is the representation corresponding to the residue of the Eisenstein series in the Rankin-Selberg integral. Θ′ is not Θ, but another small representation.

THE MINIMAL REPRESENTATION OF SO_{7} 3

1. Basic Definitions and Properties. If F is a field let SO′_{7}(F) denote the subgroup of SO_{7}(F) generated by the images of SL_{2} corresponding to the three simple roots of SO_{7}. It is the kernel of the spinor norm homomorphism SO_{7}(F) → F^{×}/(F^{×})^{2}. The standard maximal torus of SO′_{7} is

{diag(a, b, c, 1, c^{−1}, b^{−1}, a^{−1}) : abc is a square} .

Let P = GL_{3}U denote the standard Siegel maximal parabolic of SO_{7}. Here g ∈ GL_{3} is embedded in SO_{7} via

g ⟶ g^{∧} := ( \begin{matrix}g \\ & 1 \\ & & g^{∗} \end{matrix} ) ,

(g^{∗} is determined by g so that the matrix is in SO_{7}) and U consists of upper triangular matrices. Let Q = (GL_{2} × SO_{3})V be the maximal parabolic of SO_{7} where (g, h) ∈ GL_{2} × SO_{3} is embedded in SO_{7} as

(g, h) ⟶ ( \begin{matrix}g \\ & h \\ & & g^{∗} \end{matrix} ) .

We shall denote Q^{0} = (SL_{2}×SO_{3})V. Let N denote the maximal unipotent subgroup of SO_{7} consisting of upper unipotent matrices. Thus U ⊂ N and V ⊂ N. Clearly U, V and N are subgroups of SO′_{7}.

First let F be a nonarchimedean local field of characteristic not equal to 2, and let 𝔬 be its ring of integers. Fix an algebraic closure \overline{F} of F. Let μ_{n} denote the group of n-th roots of unity in \overline{F}. We assume that F contains μ_{4}. If the residue characteristic of F is two, we will also assume that F contains the group μ_{8}. We will denote by ( , ) the fourth order Hilbert symbol F^{×} × F^{×} → μ_{4}.

Lemma 1.1. Suppose that either the residue characteristic of F is not two, or that μ_{8} ⊂ F. Then there exists a subgroup H of F^{×} containing (F^{×})^{2} which is maximal with respect to the property that (H, H) ⊂ μ_{2}, and a function γ : H → ℂ^{×} such that

(a, b) = \frac{γ(a)γ(b)}{γ(ab)}

for a, b ∈ H.

Proof. First assume that μ_{8} ⊂ F. In this case let H be any subgroup containing (F^{×})^{2} such that (H, H) ⊂ μ_{2}. We will construct γ taking values ±1. (Of course we can take H to be maximal, a property needed for the application, but that will play no role in the construction.)

The group H/(F^{×})^{2} is an elementary 2-group, hence a vector space over ℤ/2ℤ. Let α_{1}, ⋯, α_{r} be elements of H whose images in H/(F^{×})^{2} form a basis. We will show that there exist t_{i} ∈ F^{×} such that if β_{i} = t_{i}^{2}α_{i}, then (β_{i}, β_{j}) = 1. Indeed, for each i there is character χ_{i} of H/(F^{×})^{2} such that

χ_{i}(α_{j}) = \{ \begin{matrix}(α_{i}, α_{j}) & if i ≤ j, \\ 1 & otherwise. \end{matrix}

We may extend χ_{i} to a character of F^{×}/(F^{×})^{2}, and every such character is of the form a → (t_{i}^{2}, a) for some t_{i}. Consequently we can find t_{i} such that

(t_{i}^{2}, α_{j}) = \{ \begin{matrix}(α_{i}, α_{j}) & if i ≤ j, \\ 1 & otherwise. \end{matrix}

Now let β_{i} = t_{i}^{2}α_{i}. Then we claim that (β_{i}, β_{j}) = 1 for all i, j. Because μ_{8} ⊂ F, −1 is a fourth power so (x, x) = (−x, x) = 1 for all x ∈ F^{×}. Thus we may assume that i ≠ j, or without loss of generality that i < j. Then

(β_{i}, β_{j}) = (t_{i}^{2}α_{i}, α_{j})(t_{i}^{2}α_{i}, t_{j})^{2} = 1.

Now let H_{0} = {h ∈ H|(h, H) ⊂ {1}}. This is an open subgroup containing both (F^{×})^{4} and H^{2}. Let H_{1} be the group generated by β_{1}, ⋯, β_{r} and H_{0}. Then (H_{1}, H_{1}) ⊂ {1} and H_{1}(F^{×})^{2} = H. We now define γ : H → {±1} by γ(ht^{2}) = (h, t^{2}) for h ∈ H_{1}, t ∈ F^{×}. This is well-defined and easily checked to have the required property.

If the characteristic of F is not two, we can give an alternative construction. We will take H to be the group of elements of even valuation. Let ϖ be a prime element, and if u ∈ 𝔬^{×} is a unit, define

γ(ϖ^{2k}u) = \{ \begin{matrix}(ϖ^{2}, u) i^{ϵ} & if k is odd; \\ 1 & otherwise. \end{matrix}

where ϵ = 0 if (−1, ϖ) = 1 (i.e. if μ_{8} ⊂ F) and ϵ = 1 otherwise. Here i = \sqrt{−1}. Then making use of the fact that (u, v) = 1 when u and v are units, it is easily checked that this function has the required property.

Let G̃L_{3}(F) denote the metaplectic double cover with the parameter c = 0 in the notation of Kazhdan and Patterson [K-P]. Let S̃L(7) denote the metaplectic four-fold cover, again with the parameter c = 0. We pull this cover back to the embedded SO_{7}. Let σ_{7} ∈ H^{2}(SO_{7}(F), ℂ^{×}) denote the corresponding cocycle. This cover is morally a double cover in the following sense: if we square σ_{7}, we get a cocycle that is almost trivial. That is, its restriction to SO′_{7} is trivial, and in fact we have

σ_{7}(g, h)^{2} = (N(g), N(h))_{2},

where N : SO_{7}(F) → F^{×}/(F^{×})^{2} is the spinor norm having kernel SO′_{7}, and ( , )_{2} is the quadratic Hilbert symbol. Equality here is in the sense of cohomology classes: the two sides of this equation are cocycles differing by a coboundary, and the right side is well defined even though N(g) and N(h) are only square classes.

The restriction of σ_{7} to GL_{3}(F), which is the Levi factor of the parabolic subgroup P, has the form

σ′_{3}(g, h) = σ_{3}(g, h) ( det(g), det(h))_{4}

(times a coboundary), where σ_{3} : GL_{3}(F) × GL_{3}(F) → {±1} is the Kazhdan-Patterson cocycle (with their c = 0). Let G̃L_{3}(F) and G̃L′_{3}(F) be the covers of GL_{3}(F) corresponding to the cocycles σ_{3} and σ′_{3}.

Let θ_{3} denote the maximal irreducible quotient of the representation parabolically induced from the character δ_{B}^{1/4} of the Borel subgroup of G̃L_{3}(F), where δ_{B} is the modular character. This is a generalized theta representation in the sense of Kazhdan and Patterson [K-P]. For later use we note that θ_{3} has a vector fixed under the maximal compact subgroup GL_{3}(𝔬) (which embeds canonically in G̃L_{3}(F)) if

the residual characteristic is not 2, and a vector with support in B̃ ( \begin{matrix}& & 1 \\ & 1 \\ 1 \end{matrix} ) Γ(𝔞)

which is Γ(𝔞)-invariant, where Γ(𝔞) is a suitable principal congruence subgroup, when the residual characteristic is 2; see [K-P], pg. 129. We will now explain how to induce this to a representation of S̃O_{7}(F). The first step is to compensate for the fact that the restriction of the cover of SO_{7}(F) is not G̃L_{3}(F) but G̃L′_{3}(F).

Let H be as in Lemma 1.1, and let GL_{3,H} be the subgroup of g ∈ GL_{3} with determinant in H. Let G̃L_{3,H}(F) and G̃L′_{3,H}(F) be the preimages in G̃L_{3}(F) and G̃L′_{3}(F) of this group. The restriction of θ_{3} to G̃L_{3,H}(F) remains irreducible. Now

g → γ( det(g)) θ_{3}(g)

is a representation of G̃L′_{3,H}(F), where γ is as in the Lemma. Thus extended, we

may induce θ_{3} to G̃L′_{3}(F).

Proposition 1.2. The representation of G̃L_{3}(F)′ induced from the restriction of the theta representation of G̃L_{3}(F) to G̃L′_{3,H}(F) is irreducible.

Proof. This is a consequence of Clifford’s theorem. What must be checked is that if we conjugate θ_{3} by a matrix with determinant outside of H, then its isomorphism class changes, and indeed, its central character is not the same. This is a consequence of the maximality property in Lemma 1.1.

Now the representation θ_{3} (induced to an irreducible representation of G̃L_{3}(F)′ as in the Proposition) may be parabolically induced to S̃O_{7}. Let δ_{P} be the modular character of the parabolic subgroup P. In usual notation, let Ind(θ_{3} ⊗ δ_{P}^{s}) denote this induced representation. If s is in general position, this representation is irreducible. However if s = \frac{5}{6} this representation has a unique irreducible quotient, which (by virtue of a standard intertwining integral) may also be realized as the unique irreducible subrepresentation when s = \frac{1}{6}. This quotient may be identified as the minimal representation, modulo the calculation of the Langlands parameters of the representation, which, as explained in the introduction, has been done at a formal level but not rigorously.

Now let F be a global field of characteristic not equal to 2 and let 𝔸 be its ring of adeles. We will assume that F contains the group μ_{4} of fourth roots of unity in its algebraic closure, and that every dyadic completion contains the eighth roots of unity (because of our reliance on the Lemma 1.1 for irreducibility of the theta representation). The global theory of Kazhdan and Patterson [K-P] shows that there is an automorphic representation of G̃L_{3}(𝔸) whose local constituent at every finite place coincides with the representations denoted θ_{3} above. We will denote this global representation again by θ_{3}.

We can adapt the representation θ_{3} to an irreducible representation of G̃L_{3}′(𝔸) in the same way as in the local case. Let H be any subgroup of 𝔸^{×} which is maximal with respect to the property that (H, H) ⊂ {±1}, where ( , ) : 𝔸^{×} × 𝔸^{×} → μ_{4} is the global 4-th order Hilbert symbol. We need to know that there exists a map γ : H → ℂ such that

(a, b) = \frac{γ(a)γ(b)}{γ(ab)}.

In practice there are two natural choices for H. Firstly, we can take H = ∏_{v} H_{v}, where for each place v of F, H_{v} is as described in the Lemma 1.1. This choice allows us to assert that the global induced representation is the restricted tensor product of the local ones. The other natural choice for H is F^{×}(𝔸^{×})^{2}. For this H, by Hilbert’s reciprocity law, we may take γ = 1.

Proposition 1.3. The representation induced from θ_{3} in this way does not depend on the choice of H or γ.

Proof. The fact that either choice of H gives the same representation may be seen as follows. Either representation may be characterized as the maximal irreducible quotient of a representation parabolically induced from a representation of T̃′_{3}(𝔸), which is the full preimage of the diagonal maximal torus in GL_{3}. Thus it is sufficient to know that these representations of T̃′_{3}(𝔸) are isomorphic. This is another consequence of Clifford theory, an analog of the Stone von Neumann theorem for the 2-step nilpotent group T′_{3}(𝔸). See Bump and Ginzburg [B-G] Proposition 1.1 and Kazhdan and Patterson [K-P], section 0.3.

We form the following induced representations and Eisenstein series. Let s ∈ ℂ

and δ_{P} be the modular function of P. Denote I(s) = Ind_{_{P}(𝔸)}^{_{SO}_{7}(𝔸)}θ_{3} ⊗ δ_{P}^{s}.

For f_{s} ∈ I(s), define the Eisenstein series

Ẽ_{θ_{3}}(g, f_{s}) = ∑_{γ∈P (F)\SO_{7}(F)} f_{s}(γg),  g ∈ S̃O_{7}(𝔸) .

This series converges for Re(s) sufficiently large and admits meromorphic continuation to the whole complex plane. It may be normalized by multiplication by the zeta factor ζ(6s − 1) ζ(12s − 4), and the product

Ẽ_{θ_{3}}(g, f_{s})^{∗} = ζ(6s − 1) ζ(12s − 4)Ẽ_{θ_{3}}(g, f_{s})

has only a finite number of poles. A calculation with intertwining operators shows that there are no poles outside of the set {1/6, 1/3, 5/12, 7/12, 2/3, 5/6}. The point of interest to us here is the rightmost location: s = 5/6.

We restrict our attention to standard sections in I(s), that is, sections which are finite under the maximal compact subgroup K of S̃O_{7}(𝔸) and whose restriction to K is independent of s. We start with Theorem 1.4. The Eisenstein series Ẽ_{θ_{3}}(g, f_{s})^{∗} has at most a simple pole at s = 5/6 and it is obtained for some choice of standard section f_{s} ∈ I(s).

Proof. We will study the constant term along the unipotent radical U, as in [K-R]. Let

w_{1} = ( \begin{matrix}0 & 1 \\ −1 & 0 \\ & & 1 \end{matrix} )^{∧}  w_{2} = ( \begin{matrix}1 \\ & 0 & 1 \\ & −1 & 0 \end{matrix} )^{∧}

w_{3} = ( \begin{matrix}1 \\ & 1 \\ & & & & 1 \\ & & & 1 \\ & & −1 \\ & & & & & 1 \\ & & & & & & 1 \end{matrix} ) .

We have

∫_{U(F)\U(𝔸)} Ẽ_{θ_{3}}(ug, f_{s}) du = ∑_{δ∈P (F)\SO_{7}(F)/U(F)} ∫_{(U(F)∩δ^{−1}P (F)δ)\U(𝔸)} f_{s}(δug) du.

We factor δ = wγ with w ∈ P (F)\SO_{7}(F)/P (F) and

γ ∈ (w^{−1}P (F)w ∩ P (F))\P (F)/U(F) = (GL_{3}(F) ∩ w^{−1}P (F)w)\GL_{3}(F)

to obtain

∑_{w∈P (F)\SO_{7}(F)/P (F)} ∑_{γ∈(GL_{3}(F)∩w^{−1}P (F)w)\GL_{3}(F)} ∫(U(F)∩γ^{−1}w^{−1}P (F)wγ)\U(𝔸)
f_{s}(wγug) du.

Conjugating u by γ, this may be rewritten

∑_{w∈P (F)\SO_{7}(F)/P (F)} ∑_{γ∈(GL_{3}(F)∩w^{−1}P (F)w)\GL_{3}(F)} ∫(U(F)∩w^{−1}P (F)w)\U(𝔸)
f_{s}(wuγg) du.

Let U_{w} = U ∩ w^{−1}P w and R_{w} = GL_{3} ∩ w^{−1}P w. Then we may write this as (1.1)

∑_{w∈P (F)\SO_{7}(F)/P (F)} ∑_{γ∈R_{w}(F)\GL_{3}(F)} ∫_{U_{w}(F)\U_{w}(𝔸)} ∫_{U_{w}(𝔸)\U(𝔸)}f_{s}(wuvγg) dv du.

Representatives for the double cosets in this sum are:

e, w_{3}, w_{3}w_{2}w_{3}, and w_{0} = w_{3}w_{2}w_{3}w_{1}w_{2}w_{3}.

We shall compute the contribution to (1.1) from each representative. Let g ∈ G̃L_{3}(𝔸) as embedded in S̃O_{7}(𝔸). Then we shall show that

(1.2) ∫_{U(F)\U(𝔸)} Ẽ_{θ_{3}}(ug, f_{s}) du = f_{s}(g) + Ẽ_{θ_{2}}^{_{GL}_{3}}(g, M_{w_{3}}f_{s}, s)_{1}

+ Ẽ_{θ_{2}}^{_{GL}_{3}}(g, M_{w_{3}w_{2}w_{3}}f_{s}, s)_{2} + (M_{w_{0}}f_{s})(g).

Here the M_{w} denote intertwining operators given below, and the Ẽ_{θ_{2}}^{_{GL}_{3}}(·)_{i}, i = 1, 2, denote the two maximal parabolic Eisenstein series on G̃L_{3} induced from the theta function θ_{2} on G̃L_{2}.

We turn to the details. The double coset represented by the identity clearly gives the first term. Also, the last term, corresponding to the long element in the Weyl group, gives

(1.3)  ∫_{U(𝔸)} f(w_{0}ug) du := (M_{w_{0}}f_{s})(g).

Next consider the case w = w_{3}. Then we may replace the integration in (1.1) over U_{w}(𝔸)\U(𝔸) by an integration over V_{w}(𝔸), where

V_{w} = \{( \begin{matrix}1 \\ & 1 \\ & & 1 & x \\ & & & 1 & −x \\ & & & & 1 \\ & & & & & 1 \\ & & & & & & 1 \end{matrix} )\} .

We find that U_{w} = U_{w}^{1}U_{w}^{2} where

U_{w}^{1} = \{( \begin{matrix}1 & & & & ∗ \\ & 1 & & & ∗ \\ & & 1 & & & ∗ & ∗ \\ & & & 1 \\ & & & & 1 \\ & & & & & 1 \\ & & & & & & 1 \end{matrix} )\} and U_{w}^{2} = \{( \begin{matrix}1 & & & ∗ & & ∗ & ∗ \\ & 1 & & ∗ & & ∗ & ∗ \\ & & 1 \\ & & & 1 & & ∗ & ∗ \\ & & & & 1 \\ & & & & & 1 \\ & & & & & & 1 \end{matrix} )\} .

An element of U_{w}^{2} may be moved to the left and discarded, so the contribution for w = w_{3} equals

∑_{γ∈R_{w}(F)\GL_{3}(F)} ∫_{U_{w}^{1}(F)\U_{w}^{1}(𝔸)} ∫_{V_{w}(𝔸)}f_{s}(wuvγg) dv du.

Let

f_{2}(g) = ∫_{U_{w}^{1}(F)\U_{w}^{1}(𝔸)} f_{s}(wuw^{−1}g) du.

Then f_{2} is a constant term along a parabolic subgroup and lies in Ind_{P̃_{2,1}(𝔸)}^{_{SO}_{7}(𝔸)}(θ_{2} ⊗

χ_{2}(s)), where P̃_{2,1} is the metaplectic cover of the standard parabolic subgroup of SO_{7} of type (2, 1, 1, 1, 2), and

(θ_{2} ⊗ χ_{2}(s)) (( \begin{matrix}h \\ & t \end{matrix} )^{∧}) = θ_{2}(h) | det h|^{3s+1/4} |t|^{3s−1/2}.

Here h ∈ GL_{2}(𝔸) and t ∈ 𝔸^{×}. Let

(θ ⊗ χ_{2}^{♯}(s)) (( \begin{matrix}h \\ & t \end{matrix} )^{∧}) = |t| (θ ⊗ χ_{2}(s)) (( \begin{matrix}h \\ & t^{−1} \end{matrix} )^{∧})
= θ_{2}(h) | det h|^{3s+1/4} |t|^{3/2−3s}.

We have an intertwining operator M_{w_{3}} : Ind_{P̃_{2,1}(𝔸)}^{_{SO}_{7}(𝔸)}(θ_{2} ⊗ χ_{2}(s)) → Ind_{P̃_{2,1}(𝔸)}^{_{SO}_{7}(𝔸)}(θ_{2} ⊗

χ_{2}^{♯}(s)), defined by

(M_{w_{3}}f_{2})(g) = ∫_{V_{w}(𝔸)} f_{2}(wvg) dv.

The normalizing factor for this intertwining operator is ζ(6s − 1), which is part of the normalizing factor for the original Eisenstein series. The contribution to (1.1) thus is equal to

(1.4)  ∑_{γ∈R_{w_{3}} (F)\GL_{3}(F)} (M_{w_{3}}f_{2})(γg),

with R_{w_{3}} the standard parabolic subgroup of type (2, 1) in GL_{3}. This is the Eisen-

stein series denoted Ẽ_{θ_{2}}^{_{GL}_{3}}(g, M_{w_{3}}f_{s}, s)_{1} in (1.2) above.

The computation of the contribution to (1.1) from w = w_{3}w_{2}w_{3} is similar. We may replace the integration over U_{w}(𝔸)\U(𝔸) by an integration over V_{w}(𝔸), where now

V_{w} = \{( \begin{matrix}1 \\ & 1 & & x & y & ∗ \\ & & 1 & z & ∗ & −y \\ & & & 1 & −z & −x \\ & & & & 1 \\ & & & & & 1 \\ & & & & & & 1 \end{matrix} )\} .

We find that U_{w} = U_{w}^{1}U_{w}^{2} where

U_{w}^{1} = \{( \begin{matrix}1 & & & & ∗ & ∗ \\ & 1 & & & & & ∗ \\ & & 1 & & & & ∗ \\ & & & 1 \\ & & & & 1 \\ & & & & & 1 \\ & & & & & & 1 \end{matrix} )\} and U_{w}^{2} = \{( \begin{matrix}1 & & & ∗ & & & ∗ \\ & 1 \\ & & 1 \\ & & & 1 & & & ∗ \\ & & & & 1 \\ & & & & & 1 \\ & & & & & & 1 \end{matrix} )\} .

An element of U_{w}^{2} may once again be moved to the left and discarded, so the contribution for w = w_{3}w_{2}w_{3} equals

∑_{γ∈R_{w}(F)\GL_{3}(F)} ∫_{U_{w}^{1}(F)\U_{w}^{1}(𝔸)} ∫_{V_{w}(𝔸)}f(wuvγg) dv du.

For this Weyl group element R_{w} is the standard (1, 2) parabolic subgroup. Let

f_{3}(g) = ∫_{U_{w}^{1}(F)\U_{w}^{1}(𝔸)} f(wuw^{−1}g) du.

This is a constant term along a parabolic subgroup and lies in Ind_{P̃_{1,2}(𝔸)}^{_{SO}_{7}(𝔸)}(χ_{3}(s)⊗θ_{2}), where

(χ_{3}(s) ⊗ θ_{2}) (( \begin{matrix}t \\ & h \end{matrix} )^{∧}) = θ_{2}(h) | det h|^{3s−1/4} |t|^{3s+1/2}.

Let

(χ_{3}(s) ⊗ θ_{2})^{♯} (( \begin{matrix}t \\ & h \\ & & 1 \\ & & & h^{∗} \\ & & & & t^{−1} \end{matrix} )) =

| det h|^{2} (χ_{3}(s) ⊗ θ_{2}) (( \begin{matrix}t \\ & h^{∗} \\ & & 1 \\ & & & h \\ & & & & t^{−1} \end{matrix} )) = θ_{2}(h^{∗}) | det h|^{9/4−3s} |t|^{1/2+3s}.

We have an intertwining operator M_{w} : Ind_{P̃_{1,2}(𝔸)}^{_{SO}_{7}(𝔸)}(χ_{3}(s)⊗θ_{2}) → Ind_{P̃_{1,2}(𝔸)}^{_{SO}_{7}(𝔸)}((χ_{3}(s)⊗

θ_{2})^{♯}), defined by

(M_{w}f_{3})(g) = ∫_{V_{w}(𝔸)} f_{3}(wvg) dv.

The normalizing factor for this intertwining operator is ζ(6s − 1)ζ(12s − 4), which is the normalizing factor for the original Eisenstein series. The contribution works out to

(1.5)  ∑_{γ∈R_{w}(F)\GL_{3}(F)} (M_{w}f_{3})(γg),

with R_{w} the standard parabolic subgroup of type (1, 2) in GL_{3}. This is the Eisen-

stein series denoted Ẽ_{θ_{2}}^{_{GL}_{3}}(g, M_{w_{3}w_{2}w_{3}}f_{s}, s)_{2} in (1.2) above.

To complete the proof of the Theorem we show that the first three summands in (1.2) are holomorphic at s = 5/6 while the final summand has at most a simple pole there, and it is obtained for some choice of standard section f_{s} ∈ I(s).

The first summand is visibly holomorphic at s = 5/6. To study the next two summands, note that the normalized local integrals corresponding to M_{w_{3}}f_{s} and M_{w_{3}w_{2}w_{3}}f_{s} are holomorphic. However, we must study the Eisenstein series. Let R_{2,1} denote the standard parabolic subgroup of GL_{3} of type (2, 1), and consider

f ∈ Ind_{R̃_{2,1}(𝔸)}^{_{GL}_{3}(𝔸)}(θ ⊗ χ) for some character χ of R̃_{2,1}(𝔸). Define the Eisenstein series

Ẽ_{2,1}(g) = ∑_{R_{2,1}(F)\GL(3,F)} f(γg).

Let

Φ(g) = ∫_{𝔸/F} f(( \begin{matrix}1 & x \\ & 1 \\ & & 1 \end{matrix} ) g) dx.

Let N_{3} be the unipotent radical of the Borel subgroup B_{3} of GL_{3}. Then an easy calculation shows that

(1.6) ∫_{N_{3}(F)\N_{3}(𝔸)} Ẽ_{2,1}(ng) dn = Φ(g) + ∫_{𝔸} Φ(( \begin{matrix}1 \\ & & 1 \\ & 1 \end{matrix} ) ( \begin{matrix}1 \\ & 1 & x \\ & & 1 \end{matrix} ) g) dx+

∫_{𝔸^{2}} Φ(( \begin{matrix}& 1 \\ & & 1 \\ 1 \end{matrix} ) ( \begin{matrix}1 & y & z \\ & 1 \\ & & 1 \end{matrix} ) g) dy dz.

We can apply this to (1.4). For (1.4) (with g ∈ GL_{3}(𝔸)), we find that taking f = M_{w_{3}}f_{2} in this calculation gives

Φ ∈ Ind_{B̃_{3}(𝔸)}^{_{GL}_{3}(𝔸)}(χ_{Φ}), where χ_{Φ}( \begin{matrix}t_{1} \\ & t_{2} \\ & & t_{3} \end{matrix} ) = |t_{1}|^{3s+1/2}|t_{2}|^{3s}|t_{3}|^{3/2−3s}.

The three intertwining operators in (1.6) multiply the spherical vector by (respectively) 1, ζ(12s − 3)/ζ(12s − 2) and ζ(12s − 3)/ζ(12s − 1). None of these has a pole at s = 5/6. Therefore the contribution to the constant term from w = w_{3} does not have a pole at s = 5/6.

The analysis of the third summand is similar. Let R_{1,2} denote the standard

parabolic subgroup of GL_{3} of type (1, 2), and consider f ∈ Ind_{R̃_{1,2}(𝔸)}^{_{GL}_{3}(𝔸)}(χ ⊗ θ) for

some character χ of R̃_{1,2}(𝔸). Define the Eisenstein series

Ẽ_{1,2}(g) = ∑_{R_{1,2}(F)\GL(3,F)} f(γg).

Let

Ψ(g) = ∫_{𝔸/F} f(( \begin{matrix}1 \\ & 1 & x \\ & & 1 \end{matrix} ) g) dx.

Then

(1.7) ∫_{N_{3}(F)\N_{3}(𝔸)} Ẽ_{1,2}(ng) dn = Ψ(g) + ∫_{𝔸} Ψ(( \begin{matrix}& 1 \\ 1 \\ & & 1 \end{matrix} ) ( \begin{matrix}1 & x \\ & 1 \\ & & 1 \end{matrix} ) g) dx+

∫_{𝔸^{2}} Ψ(( \begin{matrix}& & 1 \\ 1 \\ & 1 \end{matrix} ) ( \begin{matrix}1 & & y \\ & 1 & z \\ & & 1 \end{matrix} ) g) dy dz.

We apply this to (1.5). Taking f = M_{w_{3}w_{2}w_{3}}f_{3} in (1.5) gives

Ψ ∈ Ind_{B̃_{3}(𝔸)}^{_{GL}_{3}(𝔸)}(χ_{Ψ}), where χ_{Ψ}( \begin{matrix}t_{1} \\ & t_{2} \\ & & t_{3} \end{matrix} ) = |t_{1}|^{3s+1/2}|t_{2}|^{5/2−3s}|t_{3}|^{2−3s}.

The three intertwining operators in (1.7) multiply the spherical vector by (respectively) 1, ζ(12s − 4)/ζ(12s − 3) and ζ(12s − 4)/ζ(12s − 2). None of these has a pole at s = 5/6. Therefore the contribution to the constant term from w_{3}w_{2}w_{3} also does not have a pole at s = 5/6.

We move on to the fourth and final summand in the constant term (1.2), given by (1.3). To prove that this has a pole, and simultaneously to calculate the exponent for the Jacquet criterion for the residue to be square-integrable [Ja], let g ∈ S̃O_{7}(𝔸), and consider the constant term of (M_{w_{0}}f_{s})(g) down to the Borel subgroup B̃ ⊂ S̃O_{7}:

∫_{N_{3}(F)\N_{3}(𝔸)}(M_{w_{0}}f_{s})(ng) dn = ∫_{U(𝔸)} ∫_{N_{3}(F)\N_{3}(𝔸)} f_{s}(w_{0}ung) dn du = (M_{w_{0}}Υ_{s})(g),

where

Υ_{s}(g) = ∫_{N_{3}(F)\N_{3}(𝔸)} f_{s}(ng) dn.

We find that Υ_{s} ∈ Ind_{B̃(𝔸)}^{_{SO}_{7}(𝔸)}(χ_{s}) where

χ_{s}(t) = |t_{1}|^{1/2}|t_{3}|^{−1/2}|t_{1}t_{2}t_{3}|^{3s}, t = ( \begin{matrix}t_{1} \\ & t_{2} \\ & & t_{3} \end{matrix} )^{∧} ,

and hence that (M_{w_{0}}Υ_{s}) ∈ Ind_{B̃(𝔸)}^{_{SO}_{7}(𝔸)}(χ′_{s}) where χ′_{s}(t) = |t_{1}|^{3s−5/2}|t_{2}|^{3s−1}|t_{3}|^{3s+1/2}.

The factor c_{w}(χ_{s})

∏_{α>0,w(α)<0} \frac{(1 − χ(a_{α})^{n(α)})^{−1}}{(1 − q^{−1}χ(a_{α})^{n(α)})^{−1}} ,

where the n(α) equals 1 on the short roots and 2 on the long roots. This works out to

\frac{ζ(6s − 4) ζ(12s − 7)}{ζ(6s − 1) ζ(12s − 4)}.

Note that the denominator is the normalizing factor which multiplies everything. The numerator has a factor of ζ(6s − 4) and this is polar at s = 5/6. The Theorem follows. Also, note that the exponent δ_{B}^{−1/2}χ′_{5/6} equals

|t_{1}|^{−15/2}|t_{2}|^{−10/2}|t_{3}|^{−5/2},

which is in the negative Weyl chamber for Jacquet’s criterion for the squareintegrability of the residue.

This completes the proof of Theorem 1.4.

Given a standard section f_{s}, define

θ̃(g) = res_{s=5/6}Ẽ_{θ_{3}}(g, f_{s})

for g ∈ S̃O_{7}(𝔸). Here we suppress the dependence on the choice of section. We have

Theorem 1.5. The functions θ̃(g) are in L^{2}(SO_{7}(F)\S̃O_{7}(𝔸)).

Proof. This follows from the criterion of Jacquet [Ja] together with the proof of Theorem 1.4.

To address the gap in the literature described in the introduction we will now impose the following:

Hypothesis 1.6. For every local completion F_{v} of F the minimal representation

of S̃O_{7}(F_{v}) is the unique irreducible quotient of Ind_{P̃(F_{v})}^{_{SO}_{7}(F_{v})}(θ_{3} ⊗ δ_{P}^{5/6}).

This hypothesis amounts to the calculation of the Langlands parameters of the minimal representation. It is surely correct though apparently no proof is in the literature. (In the real case, the Langlands parameters are computed rigorously by Sabourin, and in the p-adic case there is a formal calculation by Roskies. Since the representation is spherical, it has a nonzero Jacquet module with respect to the Borel subgroup. The Langlands parameters are determined by the expectation that the Jacquet modules down to GL(3) and SO(5) produce the minimal representations for those Levi subgroups.)

We will assume this hypothesis throughout the remainder of the paper.

Consider now the subspace A_{θ} of automorphic forms on S̃O_{7}(𝔸) generated by the residues θ̃(g) obtained from sections f_{s} = ⊗f_{v,s} with the following property: for each non-dyadic place v, the local section f_{v,s} is in the cyclic S̃O_{7}(F_{v})-module generated by the K_{v}-fixed vector, and at each dyadic place v, the local section is in the cyclic S̃O_{7}(F_{v})-module generated by a vector with support in the subgroup B̃w_{0}Γ(𝔞) which is Γ(𝔞)-invariant, where Γ(𝔞) is the principal congruence subgroup of SO_{7}(𝔬_{v}), 𝔞 as above. We will denote the corresponding automorphic representation θ̃.

Theorem 1.7. Assuming Hypothesis 1.6, the representation θ̃ is irreducible, and is the minimal representation of S̃O_{7}(𝔸).

The proof is similar to that of Theorem 4.3 in Ginzburg, Rallis and Soudry [G-R-S], and is omitted here.

2. Fourier Coefficients of the Theta Function. In this section we shall study the Fourier expansion of θ̃(g) ∈ A_{θ} and establish the vanishing of many coefficients. Similar results are obtained for automorphic minimal representations of simply laced groups by Ginzburg, Rallis and Soudry [G-R-S]. Let ψ be a nontrivial character of 𝔸/F.

The basic idea is to relate θ̃(g) to the theta function on S̃p_{6}, the double cover of the rank 3 symplectic group Sp_{6}. More precisely, let θ_{ψ}^{ϕ}(lh) denote the theta function on S̃p_{6}(𝔸). Here ϕ ∈ S(𝔸^{3}), h ∈ S̃p_{6}(𝔸) and ℓ ∈ H_{7}(𝔸), the Heisenberg group in 7 variables. We can embed Q^{0} = (SL_{2} ×SO_{3})V in H_{7}Sp_{6} as follows. First, if g ∈ SO_{3} we embed

g ⟶ ( \begin{matrix}g \\ & g^{∗} \end{matrix} ) ∈ Sp_{6} .

For h = ( \begin{matrix}a & b \\ c & d \end{matrix} ) ∈ SL_{2} we embed

h ⟶ ( \begin{matrix}aI_{3} & bI_{3} \\ cI_{3} & dI_{3} \end{matrix} ) ∈ Sp_{6}.

The group V is isomorphic to H_{7}. Write (x_{1}, x_{2}, x_{3} | y_{1}, y_{2}, y_{3} | z) ∈ H_{7}. Then we identify

(x_{1}, x_{2}, x_{3}|0|0) ⟷ ( \begin{matrix}1 \\ & 1 & x_{1} & x_{2} & x_{3} & ∗ \\ & & 1 & 0 & 0 & −x_{3} \\ & & & 1 & 0 & −x_{2} \\ & & & & 1 & −x_{1} \\ & & & & & 1 \\ & & & & & & 1 \end{matrix} )

(0|y_{1}, y_{2}, y_{3}|z) ⟷ ( \begin{matrix}1 & 0 & y_{1} & y_{2} & y_{3} & z & ∗ \\ & 1 & & & & & −z \\ & & 1 & & & & −y_{3} \\ & & & 1 & & & −y_{2} \\ & & & & 1 & & −y_{1} \\ & & & & & 1 & 0 \\ & & & & & & 1 \end{matrix} ) .

Denote by

x(z) = ( \begin{matrix}1 & 0 & 0 & 0 & 0 & z & 0 \\ & 1 & & & & & −z \\ & & 1 & & & & 0 \\ & & & 1 & & & 0 \\ & & & & 1 & & 0 \\ & & & & & 1 & 0 \\ & & & & & & 1 \end{matrix} ) .

Theorem 2.1. There exists ϕ in the Bruhat-Schwartz space S(𝔸^{3}) so that

∫_{F \𝔸} θ̃(x(r)q^{0}) ψ^{−1}(r) dr = θ_{ψ}^{ϕ}(q^{0})

for q^{0} ∈ Q^{0}(𝔸).

Proof. As explained in the introduction, the minimal representation Θ may be constructed locally by compactly inducing the classical Weil representation from Q^{0} to Q, then extending to the S̃O_{7}. This is true locally and (thus) globally.

The 1-dimensional unipotent subgroup x(𝔸) is normalized by Q(𝔸), which acts transitively on the nontrivial characters of x(𝔸). The group x(𝔸) acts by the character x(r) → ψ(r) in the Weil representation. The stabilizer of this chararacter in Q(𝔸) is the subgroup Q^{0}(𝔸). We will show that the twisted Jacquet module relative to this character of x(𝔸) is just the Weil representation of Q^{0}(𝔸) which was induced. The theorem then now a consequence of the fact that the functional

θ̃ ↦ ∫_{F \𝔸} θ̃(x(r)q^{0}) ψ^{−1}(r) dr

factors through this Jacquet module.

The fact to be proved is essentially local. We will prove that if (τ, V) is any irreducible admissible representation of Q^{0}(F_{v}) such that τ(x(r)) ξ = ψ(r) ξ for all r ∈ F_{v} and ξ ∈ V, and if (T, V_{T}) is the representation of Q(F_{v}) compactly induced from τ, then we will show that the twisted Jacquet module of T is isomorphic to τ.

In the usual model, the space V_{T,∞} of smooth vectors in V_{T} consists of smooth, compactly supported functions f : Q(F_{v}) → V_{T} such that f(q^{0}q) = τ(q^{0}) f(q) for q^{0} ∈ Q^{0}(F_{v}), q ∈ Q(F_{v}). If a ∈ F_{v} let

q_{a} = ( \begin{matrix}a \\ & I_{5} \\ & & a^{−1} \end{matrix} ) .

We associate with f the function a → f(q_{a}). This is a smooth function F_{v} → V, which by abuse of notation we will denote by the same letter f. Thus we obtain a realization of T in the space of smooth functions f : F_{v} → V. In this model

T (x(r))f(a) = ψ(ar) f(a).

Using this model, define a linear transformation Λ : V_{T ∞} → V by Λ(f) = f(1). What we will show is that any (Fréchet) continuous linear functional L : V_{T,∞} → V satisfying L(T (x(r)) ξ −ψ(r) ξ) = 0 for all r ∈ F_{v} and ξ ∈ V_{T,∞} must be a constant multiple of Λ. We may express this by saying that Λ induces an isomorphism (over Q^{0}(F_{v})) of the twisted Jacquet module of T with V.

Let ϕ ∈ C_{c}^{∞}(F_{v}), and consider

∫_{F_{v}} ϕ(r)(T (x(r)) f − ψ(r) f) dr.

If v is nonarchimedean, since the integrand is locally constant this is actually a finite sum of generators of V_{T,ψ}. In any case this is the function a → (ϕ̂(a) − ϕ̂(1)) f(a), where ϕ̂ is the Fourier transform. Now if f is in the kernel of Λ, then f(1) = 0. The archimedean case is similarly clear.

Let R_{1} denote the unipotent radical of SO_{7} of the standard parabolic subgroup with Levi factor GL(1) × SO(5). Thus R_{1} consists of all matrices of the form

x(r_{1}, r_{2}, r_{3}, r_{4}, r_{5}) = ( \begin{matrix}1 & r_{1} & r_{2} & r_{3} & r_{4} & r_{5} & ∗ \\ & 1 & & & & & −r_{5} \\ & & 1 & & & & −r_{4} \\ & & & 1 & & & −r_{3} \\ & & & & 1 & & −r_{2} \\ & & & & & 1 & −r_{1} \\ & & & & & & 1 \end{matrix} )

The group SO_{5}(F) acts on the character group of R_{1}(F)\R_{1}(𝔸). A set of representatives for the nontrivial orbits are given by

χ_{α}(x(r_{1}, …, r_{5})) = ψ(αr_{1} + r_{5})

where α ∈ F.

Proposition 2.2. If α ≠ 0 then

∫_{(F \𝔸)^{5}} θ̃(x(r_{1}, …, r_{5})g) ψ(αr_{1} + r_{5}) dr_{i}

is zero for all choice of data.

Proof. We may take g = 1 and from Theorem 2.1 it is enough to show that

∫_{(F \𝔸)^{4}} θ_{ψ}^{ϕ}((0|r_{2}, r_{3}, r_{4}|0) ( \begin{matrix}1 & & & r_{1} \\ & 1 & & & r_{1} \\ & & 1 & & & r_{1} \\ & & & 1 \\ & & & & 1 \\ & & & & & 1 \end{matrix} )) ψ(αr_{1}) dr_{i}

is zero for all choice of data. Unfolding θ_{ψ}^{ϕ} the above integral equals

∫_{(F \𝔸)^{4}} ∑_{ξ_{i}∈F} ω_{ψ}((0|r_{2}, r_{3}, r_{4}|0) ( \begin{matrix}1 & & & r_{1} \\ & 1 & & & r_{1} \\ & & 1 & & & r_{1} \\ & & & 1 \\ & & & & 1 \\ & & & & & 1 \end{matrix} ))ϕ(ξ_{2}, ξ_{3}, ξ_{4}) ψ(αr_{1}) dr_{i}

= ∫_{(F \𝔸)^{4}} ∑_{ξ_{i}∈F} ω_{ψ}( ( \begin{matrix}1 & & & r_{1} \\ & 1 & & & r_{1} \\ & & 1 & & & r_{1} \\ & & & 1 \\ & & & & 1 \\ & & & & & 1 \end{matrix} ) )ϕ(ξ_{2}, ξ_{3}, ξ_{4})×

ψ(r_{2}ξ_{2} + r_{3}ξ_{3} + r_{4}ξ_{4}) ψ(αr_{1}) dr_{i} , where ω_{ψ} is the Weil representation. Since ∫_{F \𝔸} ψ(ξ_{i}r_{i}) dr_{i} = 0 unless ξ_{i} = 0 the

above equals

∫_{F \𝔸} ω_{ψ}( ( \begin{matrix}1 & & & r_{1} \\ & 1 & & & r_{1} \\ & & 1 & & & r_{1} \\ & & & 1 \\ & & & & 1 \\ & & & & & 1 \end{matrix} ) )ϕ(0, 0, 0) ψ(αr_{1}) dr_{1} =

= ϕ(0, 0, 0) ∫_{F \𝔸} ψ(αr_{1}) dr_{1} = 0 .

Let θ̃^{R_{1}}(g) denote the constant term of θ̃ along R_{1}; we will use a similar notation for other constant terms below. Then from Proposition 2.2 we deduce

Corollary 2.3. Let SO_{5} be embedded in SO_{7} via its embedding in the maximal parabolic R_{1}, and let M^{0} be the subgroup of SO_{5} which stabilizes the character χ_{0} of R_{1}(F)\R_{1}(𝔸). Then

θ̃(g) = θ̃^{R_{1}}(g) + ∑_{γ∈M^{0}(F)\SO_{5}(F)} ∫_{(F \𝔸)^{5}} θ̃(x(r_{1}, …, r_{5})γg) ψ(r_{1}) dr_{i}.

Corollary 2.4. If u = (u_{ij}) ∈ U, the integral

∫_{U(F)\U(𝔸)} θ̃(ug) ψ(u_{3,4}) du

is zero for all choice of data.

Proof. After a suitable conjugation by a Weyl element of GL_{3} we obtain as an inner integration

∫_{(F \𝔸)^{3}} θ̃(x(0, 0, r_{3}, r_{4}, r_{5})g) ψ(r_{3}) dr_{i} .

Expanding along r_{1}, r_{2} ∈ F \𝔸 it is enough to show that

∫_{(F \𝔸)^{5}} θ̃(x(r_{1}, r_{2}, r_{3}, r_{4}, r_{5})g) ψ(αr_{1} + βr_{2} + r_{3}) dr_{i}

is zero for all α, β ∈ F. By a suitable conjugation by γ ∈ SO_{5}(F) the above integral equals

∫_{(F \𝔸)^{5}} θ̃(x(r_{1}, r_{2}, r_{3}, r_{4}, r_{5})γg) ψ(δr_{1} + r_{5}) dr_{i}

where δ ∈ F^{×}. The result follows from Proposition 2.2.

Let ψ_{N}(n_{ij}) = ψ(n_{12} + n_{23} + n_{34}) and ψ_{N}^{0}(n_{ij}) = ψ(n_{12} + n_{23}) be characters of N. Next we prove

Proposition 2.5.

(a) Ẽ_{θ_{3}} has no Whittaker model.

(b) The integral

(2.1)  ∫_{N(F)\N(𝔸)} θ̃(ng)ψ_{N}^{0}(n) dn

is zero for all choice of data.

Proof. For (a) we must show that the integral

∫_{N(F)\N(𝔸)} Ẽ_{θ_{3}}(ng, f_{s}) ψ_{N}(n) dn

is zero. Unfolding the Eisenstein series we need to consider P \SO_{7}/N. This set consists of 8 elements which we can choose to be

e, w_{3}, w_{3}w_{2}, w_{3}w_{2}w_{1}, w_{3}w_{2}w_{3}, w_{3}w_{2}w_{1}w_{3}, w_{3}w_{2}w_{1}w_{3}w_{2}, w_{0}

The first and last representatives contribute zero since the representation θ_{3} has no Whittaker model. In all other cases we can produce a one-parameter subgroup x_{α}(ℓ) such that wx_{α}(ℓ)w^{−1} ∈ U and such that ψ_{N}(x_{α}(ℓ)) is not identically 1. Thus we obtain ∫_{F \𝔸} ψ(ℓ) dℓ which is zero.

To prove (b), we can integrate over U(F)\U(𝔸) first. Hence to compute (2.1) we need to compute the integral of the residue of the right-hand-side of (1.2). The first and last terms are zero since θ_{3} has no Whittaker model. The middle two, if they were nonzero, would also be in Whittaker models of θ_{3} because the residue of Ẽ_{θ_{2}}^{GL′_{3}} is θ_{3}. So these also contribute zero.

We describe the group M^{0} introduced in Corollary 2.3. For convenience let y(ℓ_{1}, ℓ_{2}, ℓ_{3}) = (ℓ_{1}, ℓ_{2}, ℓ_{3}|0|0), and let R_{2} ⊂ N denote the unipotent subgroup of N consisting of all matrices y(ℓ_{1}, ℓ_{2}, ℓ_{3}) with ℓ_{1}, ℓ_{2}, ℓ_{3} ∈ F. It is not hard to check that M^{0} = R_{2} · SO_{3}. We prove:

Proposition 2.6. For all m ∈ M^{0}(𝔸),

∫_{(F \𝔸)^{5}} θ̃(x(r_{1}, …, r_{5})mg) ψ(r_{1}) dr_{i} = ∫_{(F \𝔸)^{5}} θ̃(x(r_{1}, …, r_{5})g) ψ(r_{1}) dr_{i}.

Proof. Suppose first that m ∈ SO_{3}(𝔸). If

w = ( \begin{matrix}1 \\ & & & & & 1 \\ & & 1 \\ & & & 1 \\ & & & & 1 \\ & 1 \\ & & & & & & 1 \end{matrix} )

then we have

∫_{(F \𝔸)^{5}} θ̃(x(r_{1}, …, r_{5})mg) ψ(r_{1}) dr_{i} = ∫_{R_{1}(F)\R_{1}(𝔸)} θ̃(rmwg) χ_{0}(r) dr.

Thus it is enough to show that for m ∈ SO_{3}(𝔸),

∫_{R_{1}(F)\R_{1}(𝔸)} θ̃(rm) χ_{0}(r) dr = ∫_{R_{1}(F)\R_{1}(𝔸)} θ̃(r) χ_{0}(r) dr.

Repeating the same arguments as in Proposition 2.2 we get

∫_{R_{1}(F)\R_{1}(𝔸)} θ̃(rm) χ_{0}(r) dr = (ω_{ψ}(m)ϕ)(0, 0, 0) = ϕ(0, 0, 0)

and

∫_{R_{1}(F)\R_{1}(𝔸)} θ̃(r) χ_{0}(r) dr = ϕ(0, 0, 0).

To prove the Proposition for m ∈ R_{2}(𝔸), we consider the Fourier expansion along R_{2}(F)\R_{2}(𝔸). That is

∫_{(F \𝔸)^{5}} θ̃(x(r_{1}, …, r_{5})g) ψ(r_{1}) dr_{i} =

∑_{β_{i}∈F} ∫_{(F \𝔸)^{8}} θ̃(x(r_{1}, …, r_{5})y(ℓ_{1}, ℓ_{2}, ℓ_{3})g) ψ(r_{1} + β_{1}ℓ_{1} + β_{2}ℓ_{2} + β_{3}ℓ_{3}) dr_{i} dℓ_{j}.

We will show that each summand on the right is zero unless β_{i} = 0 for 1 ≤ i ≤ 3. This will prove the statement.

Under the action of SO_{3}(F), we can choose as representatives for the nontrivial orbits the characters

μ_{β}(y(ℓ_{1}, ℓ_{2}, ℓ_{3})) = ψ(ℓ_{1} + βℓ_{3}) β ∈ F.

Thus it is enough to prove that the integral

(2.2)  ∫_{(F \𝔸)^{8}} θ̃(x(r_{1}, …, r_{5})y(ℓ_{1}, ℓ_{2}, ℓ_{3})) ψ(r_{1} + ℓ_{1} + βℓ_{3}) dr_{i} dℓ_{j}

is zero for all choice of data. Assume first that β ≠ 0. We will show that the inner integration of (2.2)

∫_{(F \𝔸)^{4}} θ̃(x(0, 0, 0, 0, r_{5})y(ℓ_{1}, ℓ_{2}, ℓ_{3})) ψ(ℓ_{1} + βℓ_{3}) dr_{5} dℓ_{j}

is zero for all choice of data. Indeed, conjugating by w_{1} the integral equals

∫_{(F \𝔸)^{4}} θ̃(x(0, ℓ_{1}, ℓ_{2}, ℓ_{3}, r_{5})w_{1}) ψ(ℓ_{1} + βℓ_{3}) dr_{5} dℓ_{j}.

Expanding along r_{1} ∈ F \𝔸, this equals

∑_{γ∈F} ∫_{(F \𝔸)^{5}} θ̃(x(r_{1}, ℓ_{1}, ℓ_{2}, ℓ_{3}, r_{5})w_{1}) ψ(γr_{1} + ℓ_{1} + βℓ_{3}) dr_{i} dℓ_{j}.

It is enough to show that each summand is zero. For each summand there is matrix h ∈ SO_{5}(F) such that

∫_{(F \𝔸)^{5}} θ̃(x(r_{1}, ℓ_{1}, ℓ_{2}, ℓ_{3}, r_{5})w_{1}) ψ(γr_{1} + ℓ_{1} + βℓ_{3}) dr_{i} dℓ_{j}

= ∫_{(F \𝔸)^{5}} θ̃(x(r_{1}, r_{2}, r_{3}, r_{4}, r_{5})hw_{1}) ψ(βr_{1} + r_{5}) dr_{i}.

Since β ≠ 0 it follows from Proposition 2.1 that this last integral is zero. Finally we have to show that (2.2) is zero when β = 0. Define

z(r) = ( \begin{matrix}1 \\ & 1 \\ & & 1 & r & ∗ \\ & & & 1 & −r \\ & & & & 1 \\ & & & & & 1 \\ & & & & & & 1 \end{matrix} ) .

Considering the Fourier expansion of (2.2) relative to z(r) with r ∈ F \𝔸, it is enough to show that

∫_{N(F)\N(𝔸)} θ̃(ng) ψ_{N}(n) dn  and ∫_{N(F)\N(𝔸)} θ̃(ng) ψ_{N}^{0}(n) dn

are zero for all choice of data. (Notice that N is generated by R_{1}, R_{2} and ⟨z(r)⟩.) The vanishing of these two integrals is Proposition 2.5.

3. A Global Integral. Let π be an irreducible generic cuspidal representation of SO_{7}(𝔸). Let θ̃(g) ∈ A_{θ}. For φ ∈ π we define

I(φ, f_{s}) = ∫_{SO_{7}(F)\SO_{7}(𝔸)} φ(g) θ̃(g) \overline{Ẽ_{θ_{3}}(g, fs̄)} dg.

We will show that I(φ, f_{s}) is Eulerian. To do this, for n = (x_{ij}) ∈ N and g ∈ S̃O_{7}(𝔸) let

θ^{N,ψ}(g) = ∫_{N(F)\N(𝔸)} θ̃(ng) ψ(x_{23}) dn

and

f_{W_{θ}}(g, s) = ∫_{(F \𝔸)^{3}} f̄s̄[ ( \begin{matrix}1 & r_{1} & r_{3} \\ & 1 & r_{2} \\ & & 1 \end{matrix} )^{∧} g] ψ(r_{1}) dr_{i} .

Also let W_{φ} denote the Whittaker function of φ:

W_{φ}(g) = ∫_{N(F)\N(𝔸)} φ(ng) ψ_{N}(n) dn.

We prove:

Theorem 3.1. For Re(s) large

I(φ, f_{s}) = ∫_{N(𝔸)\SO_{7}(𝔸)} ∫_{𝔸} W_{φ}(g) θ^{N,ψ}(g) f_{W_{θ}}(w_{3}z(r)g, s) ψ(r) dr dg .

Proof. Unfolding the Eisenstein series we get

(3.1)  I(φ, f_{s}) =  ∫_{GL_{3}(F)U(F)\SO_{7}(𝔸)} φ(g) θ̃(g) f̄s̄(g) dg .

For convenience we change notation, replacing f̄s̄ by f_{s}. Define the subgroup C ⊂ U of all matrices of the form

\{c(r_{1}, r_{2}, r_{3}) = ( \begin{matrix}1 & & & & r_{2} & r_{3} & 0 \\ & 1 & & & r_{1} & 0 & −r_{3} \\ & & 1 & & 0 & −r_{1} & −r_{2} \\ & & & 1 \\ & & & & 1 \\ & & & & & 1 \\ & & & & & & 1 \end{matrix} ) \}.

In the above integral, we consider the Fourier expansion in θ̃(g) along C(F)\C(𝔸). The group GL_{3}(F) acts on the group characters of C(F)\C(𝔸) with two orbits. Define a character

ψ_{C}(c) = ψ_{C}(c(r_{1}, r_{2}, r_{3})) = ψ(r_{1}).

Let L^{0} be the subgroup of R_{1,2}(F) ⊆ GL_{3}(F) consisting of matrices of the form

( \begin{matrix}a & ∗ \\ 0 & h \end{matrix} ) where a ∈ F^{×} and h ∈ SL_{2}(F). Then

θ̃(g) = ∫_{C(F)\C(𝔸)} θ̃(cg) dc + ∑_{γ∈L^{0}(F)\GL_{3}(F)} ∫_{C(F)\C(𝔸)} θ̃(cγg) ψ_{C}(c) dc.

We consider the contribution of each summand to (3.1). The first summand will contribute

I_{1} =  ∫_{GL_{3}(F)U(F)\SO_{7}(𝔸)} ∫_{C(F)\C(𝔸)} φ(g) θ̃(cg) f_{s}(g) dc dg.

We will show that I_{1} = 0. Indeed, factoring

∫_{GL_{3}(F)U(F)\SO_{7}(𝔸)} =  ∫_{GL_{3}(F)U(F)C(𝔸)\SO_{7}(𝔸)} ∫_{C(F)\C(𝔸)} ,

I_{1} equals

∫_{GL_{3}(F)U(F)C(𝔸)\SO_{7}(𝔸)} ∫_{C(F)\C(𝔸)} ∫_{C(F)\C(𝔸)} φ(c_{1}g) θ̃(c_{2}g) f_{s}(g) dc_{i} dg .

Here we have used the C(𝔸)-invariance property of f_{s}(g) and a change of variables in C(F)\C(𝔸). ¿From [PS] we have

∫_{C(F)\C(𝔸)} φ(c_{1}g) dc_{1} = ∑_{γ∈N_{3}(F)\GL_{3}(F)} W_{φ}(γg)

where we recall that N_{3} denotes the maximal upper unipotent subgroup of GL_{3} and γ ∈ GL_{3} is embedded in SO_{7} in the standard way. Thus

I_{1} =  ∫_{N(F)C(𝔸)\SO_{7}(𝔸)} ∫_{C(F)\C(𝔸)} W_{φ}(g) θ̃(c_{2}g) f_{s}(g) dc_{2} dg.

Factoring the measure, this equals

∫_{N(𝔸)\SO_{7}(𝔸)} W_{φ}(g) ∫_{N(F)\N(𝔸)} θ̃(ng) f_{s}(ng) ψ_{N}(n) dn dg .

Since f_{s}(g) is invariant under U(𝔸) we get as an inner integration

∫_{U(F)\U(𝔸)} θ̃(ug) ψ(x_{3,4}) du

which equals zero by Corollary 2.4. Thus I(φ, f_{s}) equals

∫_{L^{0}(F)U(F)\SO_{7}(𝔸)} ∫_{C(F)\C(𝔸)} φ(g) θ̃(cg) f_{s}(g) ψ_{C}(c) dc dg.

We shall now apply Corollary 2.3 to the above integral. Recall that Corollary 2.3 states that

θ̃(g) = θ̃^{R_{1}}(g) + ∑_{γ∈M^{0}(F)\SO_{5}(F)} ∫_{(F \𝔸)^{5}} θ̃(x(r_{1}, …, r_{5})γg) ψ(r_{1}) dr_{i}.

Let us first show that the second summand contributes zero to I(φ, f_{s}). Indeed, in this case we get as an inner integration

∫_{C(F)\C(𝔸)} ∑_{γ∈M^{0}(F)\SO_{5}(F)} ∫_{(F \𝔸)^{5}} θ̃(x(r_{1}, …, r_{5})γcg) ψ(r_{1}) ψ_{C}(c) dr_{i} dc .

Let S^{0} ⊂ SO_{5} denote the set of all matrices in SO_{5} of the form ( \begin{matrix}g & & ∗ \\ & 1 \\ & & g^{∗} \end{matrix} ) where

g ∈ SL_{2}. Thus the above equals

∫_{C(F)\C(𝔸)} ∑_{γ∈M^{0}(F)\SO_{5}(F)/S^{0}(F)} ∑_{δ∈S_{γ}^{0}(F)\S^{0}(F)} ∫_{(F \𝔸)^{5}} θ̃(x(r_{1}, …, r_{5}γδcg) ×
ψ(r_{1}) ψ_{C}(c) dr_{i} dz .

Here S_{γ}^{0} = S^{0} ∩ γ^{−1}M^{0}γ. It is not hard to check that M^{0}\SO_{5}/S^{0} consists of 2 elements and as representatives we may choose e and w_{2}w_{3}. Notice also that we may conjugate δ across c and ψ_{C} is not changed when we change variables. Thus it is enough to prove that

∫_{C(F)\C(𝔸)} ∫_{(F \𝔸)^{5}} θ̃(x(r_{1}, …, r_{5})γcg) ψ(r_{1}) ψ_{C}(c) dr_{i} dc

is zero when γ = e or γ = w_{2}w_{3}. If γ = e we use Proposition 2.6 to obtain ∫_{C(F)\C(𝔸)} ψ_{C}(c) dc as an inner integration. Thus we get zero. When γ = w_{2}w_{3} one

can check that

x(r_{1}, …, r_{5})w_{2}w_{3}c(0, ℓ_{2}, 0) = x(r_{1} + ℓ_{2}, r_{2}, …, r_{5})w_{2}w_{3} .

Changing variables in r_{1} we get ∫_{F \𝔸} ψ(ℓ_{2})dℓ_{2} which equals zero. Thus we conclude

that I(φ, f_{s}) equals

∫_{Q^{0}(F)U(F)\SO_{7}(𝔸)} ∫_{C(F)\C(𝔸)} φ(g) θ̃^{R_{1}}(cg) f_{s}(g) ψ_{C}(c) dc dg .

Recall that f_{s} is induced from the complex conjugate of θ_{3}, the theta function on

G̃L_{3}. Let B^{0} denote the subgroup of L^{0} of matrices of the form \{ ( \begin{matrix}a & ∗ & ∗ \\ & a & ∗ \\ & & a^{−1} \end{matrix} ) \}.

The usual Fourier expansion in GL_{3} gives us

f_{s}(g) = ∫_{(F \𝔸)^{2}} f_{s}[ ( \begin{matrix}1 & m_{1} & m_{2} \\ & 1 \\ & & 1 \end{matrix} )^{∧} g] dm_{i}

+ ∑_{γ∈B^{0}(F)\L^{0}(F)} ∫_{(F \𝔸)^{2}} f_{s}[ ( \begin{matrix}1 & m_{1} & m_{2} \\ & 1 \\ & & 1 \end{matrix} )^{∧} γg] ψ(m_{1}) dm_{i}.

We plug this in to I(φ, f_{s}) and compute the contribution of each term. The first summand contributes zero. Indeed, the function

∫_{C(F)\C(𝔸)} ∫_{(F \𝔸)^{2}} θ̃^{R_{1}}(cg) f_{s}[ ( \begin{matrix}1 & m_{1} & m_{2} \\ & 1 \\ & & 1 \end{matrix} )^{∧} g] dc dm_{i}

of g is left-invariant by R_{1}(𝔸). Factoring the measure L^{0}(F)U(F)\SO_{7}(𝔸) we obtain

∫_{R_{1}(F)\R_{1}(𝔸)} φ(rg) dr

as inner integration. By the cuspidality of φ we obtain zero.

Thus, I(φ, f_{s}) equals

∫_{B^{0}(F)U(F)\SO_{7}(𝔸)} ∫_{C(F)\C(𝔸)} φ(g) θ̃^{R_{1}}(cg) ∫_{(F \𝔸)^{2}} f_{s}[ ( \begin{matrix}1 & m_{1} & m_{2} \\ & 1 \\ & & 1 \end{matrix} )^{∧} g]×

ψ(m_{1}) dm_{i} ψ_{C}(c) dc dg.

In coordinates we can write

∫_{C(F)\C(𝔸)} θ̃^{R_{1}}(cg)ψ_{C}(c) dc = ∫_{(F \𝔸)^{8}} θ̃(x(r_{1}, …, r_{5})c(ℓ_{1}, ℓ_{2}, ℓ_{3})g) ψ(ℓ_{1}) dr_{i} dℓ_{j} .

Changing variables r_{4} → r_{4} + ℓ_{2} and r_{5} → r_{5} + ℓ_{3} this equals

∫_{(F \𝔸)^{6}} θ̃(x(r_{1}, …, r_{5})c(ℓ_{1}, 0, 0)g) ψ(ℓ_{1}) dr_{i} dℓ_{1} .

Since c(ℓ_{1}, 0, 0) = y(0, 0, ℓ_{1}), this equals

∫_{(F \𝔸)^{6}} θ̃(x(r_{1}, …, r_{5})y(0, 0, ℓ_{3})g) ψ(ℓ_{3}) dr_{i} dℓ_{3} .

We consider the expansion of the above integral along R_{2}(F)\R_{2}(𝔸). As in the proof of Proposition 2.6 above, the group SO_{3}(F) acts on the group character of R_{2}(F)\R_{2}(𝔸). As representatives of the various orbits of nontrivial characters of this group we again choose

μ_{β}(y(t_{1}, t_{2}, t_{3})) = ψ(t_{1} + βt_{3}) β ∈ F.

As in the proof of Proposition 2.6 we can show that

∫_{(F \𝔸)^{8}} θ̃(x(r_{1}, …, r_{5})y(t_{1}, t_{2}, t_{3})g) ψ(t_{1} + βt_{3}) dr_{i} dt_{j}

is zero if β ≠ 0. Thus we get

∫_{(F \𝔸)^{6}} θ̃(x(r_{1}, …, r_{5})y(0, 0, ℓ_{3})g) ψ(ℓ_{3}) dr_{i} dℓ_{3} =

∫_{(F \𝔸)^{9}} θ̃(x(r_{1}, …, r_{5})y(t_{1}, t_{2}, t_{3} + ℓ_{3})g) ψ(ℓ_{3}) dr_{i} dt_{j} dℓ_{3}+

∑_{γ∈N^{0}(F)\SO_{3}(F)} ∫_{(F \𝔸)^{9}} θ̃(x(r_{1}, …, r_{5})y(t_{1}, t_{2}, t_{3})γy(0, 0, ℓ_{3})g) ψ(t_{1} + ℓ_{3}) dr_{i} dt_{j} dℓ_{3}

where N^{0} is the maximal unipotent of SO_{3}. Changing variables ℓ_{3} → ℓ_{3} − t_{3} the first summand vanishes. We can choose as a set of representatives for N^{0}\SL_{3}, the elements h_{1}(α) and h_{1}(α)w_{3}^{−1}z(β) for α ∈ F^{×} and β ∈ F where h_{1}(α) = diag(1, 1, α, 1, α^{−1}, 1, 1). If γ = h_{1}(α), then γy(0, 0, ℓ_{3}) = y(0, 0, αℓ_{3})γ and changing variables in t_{3} we get a zero contribution. If γ = h_{1}(α)w_{3}^{−1}z(β), then γy(0, 0, ℓ_{3}) = y(α^{−1}ℓ_{3}, 0, 0)γ, and changing variables in t_{1}, we get ∫_{F \𝔸} ψ(ℓ_{3}(1 − α^{−1})) dℓ_{3} which

equals zero unless α = 1. Hence we conclude that

∫_{C(F)\C(𝔸)} θ̃^{R_{1}}(cg) ψ_{C}(c) dc =

∑_{β∈F} ∫_{(F \𝔸)^{8}} θ̃(x(r_{1}, …, r_{5})y(t_{1}, t_{2}, t_{3})w_{3}^{−1}z(β)g) ψ(t_{1}) dr_{i} dt_{j}.

Thus I(φ, f_{s}) equals

∫_{B^{0}(F)U(F)\SO_{7}(𝔸)} φ(g) ∑_{β∈F} ∫_{(F \𝔸)^{8}} θ̃(x(r_{1}, …, r_{5})y(t_{1}, t_{2}, t_{3})w_{3}^{−1}z(β)g) ψ(t_{1})

× ∫_{(F \𝔸)^{2}} f_{s}[ ( \begin{matrix}1 & m_{1} & m_{2} \\ & 1 \\ & & 1 \end{matrix} )^{∧} g] ψ(m_{1}) dm_{i} dr_{j} dt_{k} dg .

By definition B^{0}U = GL_{1}N = GL_{1}R_{1}R_{2}R_{3} where the GL_{1} consists of all matrices of the form

{diag(α, α, α^{−1}, 1, α, α^{−1}, α^{−1}), α ∈ F^{×}},

and R_{3} consists of all matrices of the form {z(β) : β ∈ F }. Thus we may collapse the summation over β with the suitable integration to obtain

∫_{GL_{1}(F)R_{1}(F)R_{2}(F)\SO_{7}(𝔸)} φ(g) ∫_{(F \𝔸)^{8}} θ̃(x(r_{1}, …, r_{5})y(t_{1}, t_{2}, t_{3})w_{3}^{−1}g) ψ(t_{1})

× ∫_{(F \𝔸)^{2}} f_{s}[ ( \begin{matrix}1 & m_{1} & m_{2} \\ & 1 \\ & & 1 \end{matrix} )^{∧} g] ψ(m_{1}) dm_{i} dr_{j} dt_{k} dg .

Since θ_{3} has no Whittaker model we have

∫_{(F \𝔸)^{2}} f_{s}[ ( \begin{matrix}1 & m_{1} & m_{2} \\ & 1 \\ & & 1 \end{matrix} )^{∧} g] ψ(m_{1}) dm_{i}

= ∫_{(F \𝔸)^{3}} f_{s}[ ( \begin{matrix}1 & m_{1} & m_{2} \\ & 1 & m_{3} \\ & & 1 \end{matrix} )^{∧} g] ψ(m_{1}) dm_{i}.

Also changing variables g → w_{3}g, I(φ, f_{s}) equals

∫_{GL_{1}(F)R_{1}(F)R_{2}(F)\SO_{7}(𝔸)} φ(g)×

∫_{(F \𝔸)^{8}} θ̃(x(r_{1}, …, r_{5})y(t_{1}, t_{2}, t_{3})g) ψ(t_{1}) f_{W_{θ}}(w_{3}g, s) dr_{i} dt_{j} dg

where now GL_{1} consists of all matrices of the form

{h(α) = diag(α, α, α, 1, α^{−1}, α^{−1}, α^{−1})}.

We have

∫_{R_{1}(F)R_{2}(F)\R_{1}(𝔸)R_{2}(𝔸)} φ(x(r_{1}, …, r_{5})y(t_{1}, t_{2}, t_{3})g) ψ(r_{1} + t_{1}) dr_{i} dt_{j} =

∑_{α∈F^{×}} W_{φ}(h(α)g).

Returning to I(φ, f_{s}) and factoring the measure

∫_{GL_{1}(F)R_{1}(F)R_{2}(F)\SO_{7}(𝔸)} = ∫_{GL_{1}(F)R_{1}(𝔸)R_{2}(𝔸)\SO_{7}(𝔸)} ∫_{R_{1}(F)R_{2}(F)\R_{1}(𝔸)R_{2}(𝔸)}

THE MINIMAL REPRESENTATION OF SO_{7} 27

we may use the last identity to obtain

∫_{R_{1}(𝔸)R_{2}(𝔸)\SO_{7}(𝔸)} W_{φ}(g) ∫_{(F \𝔸)^{8}} θ̃(x(r_{1}, …, r_{5})y(t_{1}, t_{2}, t_{3})g)×

ψ(t_{1}) f_{W_{θ}}(w_{3}g, s) dr_{i} dt_{j} dg.

Write ∫_{R_{1}(𝔸)R_{2}(𝔸)\SO_{7}(𝔸)} = ∫_{N(𝔸)\SO_{7}(𝔸)} ∫_{𝔸} where the integration over 𝔸 is on the group {z(r) : r ∈ 𝔸}. From Corollary 2.4 the integral of θ̃ is invariant under z(r) with r ∈ 𝔸. Hence I(φ, f_{s}) equals

∫_{N(𝔸)\SO_{7}(𝔸)} W_{φ}(g) θ^{N,ψ}(g) ∫_{𝔸} f_{W_{θ}}(w_{3}z(r)g, s) ψ(r) dr dg.

This completes the proof of the theorem.

Let us show that I(φ, f_{s}) is factorizable. First let W_{θ_{3}} denote the space of all complex valued functions of h ∈ G̃L_{3}(𝔸) of the form

W_{θ_{3}}(h) = ∫_{(F \𝔸)^{3}} η_{3}[ ( \begin{matrix}1 & x & z \\ & 1 & y \\ & & 1 \end{matrix} ) h] ψ(x) dx dy dz

where η_{3} ∈ θ_{3}. Next, we denote I(W_{θ_{3}}, s) = Ind_{_{P}(𝔸)}^{_{SO_{7}}(𝔸)}(W_{θ_{3}} ⊗ δ_{P}^{s}). Using Theorem 1.4 in [B-G] we may deduce that for factorizable data

I(W_{θ_{3}}, s) = ⊗_{v}I_{v}(W_{θ_{3}}^{(v)}, s)

where the product is over all places of F and I_{v}(W_{θ_{3}}^{(v)}, s) = Ind_{SO}_{7}^{(v)} ; _{P_{v}}(W_{θ_{3}}^{(v)} ⊗ δ_{P}^{s}). For the precise definition of W_{θ_{3}}^{(v)} see [B-G] p. 164-165.

A similar statement holds for θ̃^{N,ψ}(g). More precisely, let θ̃ = ⊗_{v}θ̃_{v}. For w ∈ θ̃_{v} consider the space of functionals LN,ψ_{v} ; _{θ_{v}} defined on θ̃_{v} which satisfy

LN,ψ_{v} ; _{θ_{v}}(ry(t_{1}, t_{2}, t_{3})z(ℓ)w) = ψ_{v}(t_{1}) LN,ψ ; _{θ_{v}}(w)

where r ∈ R_{1}(F_{v}). As in [B-G] Theorem 1.4 we have Lemma 3.2. The space of functionals LN,ψ ; _{θ_{v}} is one-dimensional.

Let WN,ψ_{v} ; _{θ} denote the space of all functions on S̃O_{7}(F_{v}) defined by Q_{θ_{v}}(g) = LN,ψ ; _{θ_{v}}(gw). From the usual factorization of W_{φ}, we may deduce that for Re(s) large

I(φ, f_{s}) = ∏_{v} I_{v}(W_{v}, Q̃_{θ_{v}}, f_{W_{θ}}^{(v)}, s)

where

I_{v}(W_{v}, Q̃_{θ_{v}}, f_{W_{θ}}^{(v)}, s) = ∫_{N(F_{v})\SO_{7}(F_{v})} W_{v}(g) Q̃_{θ_{v}}(g) ∫_{F} f_{W_{θ}}^{(v)}(w_{3}z(r)g, s) ψ_{v}(r) dr dg.

In the next section, we shall compute the unramified local integrals.

4. The Unramified Integral. In this section, we assume F is a nonarchimedean local field. We shall assume that all functions in question are unramified, that is they are fixed by K the maximal compact subgroup of SO_{7} (or S̃O_{7} with the canonical splitting) and their values at the identity are one.

Given π an unramified irreducible admissible representation of SO_{7}(F) there is associated a semisimple conjugacy class t_{π} in Sp_{6}(ℂ), the L group of SO_{7}. Let ρ_{3} denote the third fundamental representation of Sp_{6}(ℂ). This is a 14-dimensional representation. Let

L(s, π, ρ_{3}) = det[I − ρ_{3}(t_{π})q^{−s}]^{−1}

where q = |p| and p is a generator of the maximal ideal in the ring of integers in F. Finally, let ζ(s) = (1 − q^{−s})^{−1}. We prove

Theorem 4.1. For all unramified data

I(W, Q_{θ}, f_{W_{θ}}, s) = \frac{L(3s − 1, π, ρ_{3})}{ζ(6s − 1)ζ(12s − 4)}.

Proof. Let T denote the maximal torus of SO_{7}(F). Since all functions in question are spherical, I(W, Q_{θ}, f_{W_{θ}}, s) equals

∫_{T} W (t) Q̃_{θ}(t) ∫_{F} f_{W_{θ}}(w_{3}z(r)t, s) ψ(r) dr δ_{B}^{−1}(t) d^{∗}t

where d^{∗}t denotes Haar measure on T and we recall that B denotes the Borel subgroup of SO_{7}. Parametrize T as

t = diag(abc, ab, a, 1, a^{−1}, a^{−1}b^{−1}, a^{−1}b^{−1}c^{−1}) .

We have

w_{3}z(r)t = ( \begin{matrix}abc \\ & ab \\ & & a^{−1} \end{matrix} )^{∧} w_{3}z(a^{−1}r).

One can check that the above conjugation does not contribute a cocycle. Write K(t) = W (t)δ_{B}^{1/2}(t). Changing variables in r we obtain

∫_{(F^{×})^{3}} K(t) Q̃_{θ}(t) |a| δ_{B}^{−1/2}(t) ∫_{F} f_{W_{θ}}( ( \begin{matrix}abc \\ & ab \\ & & a^{−1} \end{matrix} )^{∧} w_{3}z(r), s) ψ(ar) dr d^{∗}t.

Consider the inner integral. It equals

f_{W_{θ}}( ( \begin{matrix}abc \\ & ab \\ & & a^{−1} \end{matrix} )^{∧} , s)+

∫_{|r|>1} f_{W_{θ}}( ( \begin{matrix}abc \\ & ab \\ & & a^{−1} \end{matrix} )^{∧} w_{3}z(r), s) ψ(ar) dr.

Performing the Iwasawa decomposition for w_{3}z(r) and using the left and right invariant properties of f_{W_{θ}} we get (see [B-G] Theorem 4.1, p. 170)

f_{W_{θ}}( ( \begin{matrix}abc \\ & ab \\ & & a^{−1} \end{matrix} )^{∧} , s)(1 + ∫_{|r|>1} |r|^{−6s+1} ψ(ar) dr).

In obtaining this we have used the evaluation

δ_{P} (diag(t_{1}, t_{2}, t_{3}, 1, t_{3}^{−1}, t_{2}^{−1}, t_{1}^{−1})) = |t_{1}t_{2}t_{3}|^{3}

and from [B-G] Theorem 4.1

f_{W_{θ}}[ ( \begin{matrix}1 \\ & 1 \\ & & r^{−2} \end{matrix} )^{∧} , s] = |r|^{−6s+1} .

¿From the support condition of K(t) we may assume that |a|, |b|, |c| ≤ 1. Then a standard calculation shows that, for |a| ≤ 1,

1 + ∫_{|r|>1} |r|^{−6s+1}ψ(ar) dr = \frac{ζ(6s − 2)}{ζ(6s − 1)}(1 − |a|^{6s−2}q^{−6s+2}) .

Hence I(W, Q_{θ}, f_{W_{θ}}, s) equals

\frac{ζ(6s − 2)}{ζ(6s − 1)} ∫_{|a|,|b|,|c|≤1} K(t) Q̃_{θ}(t) f_{W_{θ}}( ( \begin{matrix}abc \\ & ab \\ & & a^{−1} \end{matrix} )^{∧} , s)×

δ_{B}^{−1/2}(t) |a| (1 − |a|^{6s−2}q^{−6s+2}) d^{∗}t.

Using [B-G] p. 170 we get

f_{W_{θ}}( ( \begin{matrix}abc \\ & ab \\ & & a^{−1} \end{matrix} )^{∧} , s) =

δ_{P}^{s} ( ( \begin{matrix}abc \\ & ab \\ & & a^{−1} \end{matrix} )^{∧} )δ_{B_{3}}^{1/4} ( \begin{matrix}abc \\ & ab \\ & & a^{−1} \end{matrix} )

if c is a square and zero otherwise. (Recall that B_{3} is the Borel of GL_{3}.) We have a similar formula for Q̃_{θ}(t).

Lemma 4.2. For unramified data

Q̃_{θ}( ( \begin{matrix}abc \\ & ab \\ & & a \end{matrix} )^{∧} ) = δ_{P}^{1/6}( ( \begin{matrix}abc \\ & ab \\ & & a \end{matrix} )^{∧} )δ_{B_{3}}^{1/4}( ( \begin{matrix}abc \\ & ab \\ & & a \end{matrix} ) )

if b is a square and zero otherwise.

Proof. This is similar to [B-G] p. 170. Indeed, the theta function is a subrepresen-

tation of Ind_{SO}_{7} ; _{P}θ_{3} ⊗δ_{P}^{1/6}. This follows from the fact that the pole for the Eisenstein series for which the theta function is defined occurs at s = 5/6. The exponents of the Jacquet model along U can be read off by taking 1 − 5/6 = 1/6.

Returning to the proof of the theorem the local integral then equals

\frac{ζ(6s − 2)}{ζ(6s − 1)} ∫_{|a|,|b|,|c|≤1} K( ( \begin{matrix}ab^{2}c^{2} \\ & ab^{2} \\ & & a \end{matrix} )^{∧} )×

|a|^{3s−1}|b|^{12s−4}|c|^{6s−2}(1 − |a|^{6s−2}q^{−6s+2}) d^{×}a d^{×}b d^{×}c.

Denote x = q^{−3s+1}. If a = p^{n_{1}}ϵ_{1}, c = p^{n_{2}}ϵ_{2} and b = p^{n_{3}}ϵ_{3}, we get

\frac{ζ(6s − 2)}{ζ(6s − 1)} ∑_{n_{i}=0}^{∞} K( ( \begin{matrix}p^{n_{1}+2n_{2}+2n_{3}} \\ & p^{n_{1}+2n_{3}} \\ & & p^{n_{1}} \end{matrix} )^{∧} )x^{n_{1}+2n_{2}+4n_{3}}(1 − x^{2n_{1}+2}) .

Let ϖ_{i} for 1 ≤ i ≤ 3 denote the three fundamental representations of Sp_{6}(ℂ). Thus ρ_{3} = ϖ_{3}. Given an irreducible representation m_{1}ϖ_{1} + m_{2}ϖ_{2} + m_{3}ϖ_{3} of Sp_{6}(ℂ), we shall denote by (m_{1}, m_{2}, m_{3}) the trace of the character of this representation evaluated at t_{π}. Using the Casselman-Shalika formula [C-S], I(W, Q_{θ}, f_{W_{θ}}, s) equals

\frac{ζ(6s − 2)}{ζ(6s − 1)} ∑_{n_{i}=0}^{∞}(2n_{2}, 2n_{3}, n_{1})x^{n_{1}+2n_{2}+4n_{3}}(1 − x^{2n_{1}+2}) .

Next consider the L-function. Using the Poincaré identity we have

L(3s − 1, π, ρ_{3}) = ∑_{m=0}^{∞} trS^{m}(t_{π})x^{m}.

Here S^{m} denotes the symmetric m-th power operation. Hence to prove the theorem we need to prove (4.1)

∑_{n_{i}=0}^{∞}(2n_{2}, 2n_{3}, n_{1})x^{n_{1}+2n_{2}+4n_{3}}(1 − x^{2n_{1}+2}) = (1 − x^{2})(1 − x^{4}) ∑_{m=0}^{∞} trS^{m}(t_{π})x^{m}.

To prove this identity we need to use the result of Brion [Br] to decompose the symmetric algebra. Let V_{ρ_{3}} denote the vector space over ℂ on which ρ_{3} acts. Let U^{0} denote the maximal unipotent of Sp_{6}(ℂ). The result of Brion states that ℂ[V_{ρ_{3}}]^{U^{0}} is freely generated by (1, ϖ_{3}); (2, 2ϖ_{1}); (3, ϖ_{3}); (4, 2ϖ_{2}) and (4, 0). (See [Br] for notations.) From this we may deduce that

trS^{m}(t_{π}) = ∑(2r_{2}, 2r_{3}, r_{1} + r_{4})

where the sum is over all r_{i}, 1 ≤ i ≤ 5, such that r_{1} + 2r_{2} + 4r_{3} + 3r_{4} + 4r_{5} = m. Summing over all m we get

∑_{m=0}^{∞} trS^{m}(t_{π})x^{m} = ∑_{r_{i}=0}^{∞}(2r_{2}, 2r_{3}, r_{1} + r_{4})x^{r_{1}+2r_{2}+4r_{3}+3r_{4}+4r_{5}} .

The right-hand-side equals

\frac{1}{1 − x^{4}} ∑_{r_{i}=0}^{∞}(2r_{2}, 2r_{3}, r_{1} + r_{4})x^{r_{1}+2r_{2}+4r_{3}+3r_{4}} .

Changing variables r_{1} → r_{1} − r_{4} we get

\frac{1}{1 − x^{4}} ∑_{r_{2},r_{3},r_{4}=0}^{∞} ∑_{r_{1}=r_{4}}^{∞} (2r_{2}, 2r_{3}, r_{1})x^{r_{1}+2r_{2}+4r_{3}+2r_{4}} .

Since

∑_{r_{4}=0}^{∞} ∑_{r_{1}=r_{4}}^{∞} = ∑_{r_{1}=0}^{∞} ∑_{r_{4}=0}^{r_{1}} ,

we get

\frac{1}{1 − x^{4}} ∑_{r_{1},r_{2},r_{3}=0}^{∞}(2r_{2}, 2r_{3}, r_{1})x^{r_{1}+2r_{2}+4r_{3}} ∑_{r_{4}=0}^{r_{1}} x^{2r_{4}}

= \frac{1}{1 − x^{4}} ∑_{r_{i}=0}^{∞}(2r_{2}, 2r_{3}, r_{1})x^{r_{1}+2r_{2}+4r_{3}} \frac{1 − x^{2r_{1}+2}}{1 − x^{2}} .

¿From this (4.1) follows and the Theorem is proved.

References

[Br] M. Brion, Invariants d’un sous-groupe unipotent maximal d’un groupe semi-simple, Ann. Inst. Fourier (Grenoble) 33 (1983), 1–27.

[B-G] D. Bump and D. Ginzburg, Symmetric square L-functions on GL(r), Ann. of Math. 136

(1992), 137–205.

[C-S] W. Casselman and J. Shalika, The unramified principal series of p-adic groups. I: The

spherical function, Compositio Math. 40 (1980), 387–406.

[G-R-S] D. Ginzburg, S. Rallis and D. Soudry, On the automorphic theta representation for simply

laced groups, Israel J. Math. 100 (1997), 61–116.

[Ja] H. Jacquet, On the residual spectrum of GL(n), in Lie Group Representations II, Proceedings, University of Maryland 1982-1983, Lecture Notes in Mathematics 1041, Spinger-Verlag, Berlin, 1984, pp. 185–208.

[K-P] D. Kazhdan and S. Patterson, Metaplectic Forms, Publ. Math. IHES 59 (1984). [K-S] D. Kazhdan and G. Savin, The smallest representation of simply laced groups, in Fest-

schrift in honor of I.I. Piatetski-Shapiro on the occasion of his sixtieth birthday, Part I, (S. Gelbart, R. Howe, P. Sarnak, eds.), Isr. Math. Conf. Proc. 2, The Weizmann Science Press of Israel, Jerusalem, 1990, pp. 209–223.

[K-R] S. Kudla and S. Rallis, Poles of Eisenstein series and L-functions, in Festschrift in honor of I.I. Piatetski-Shapiro on the occasion of his sixtieth birthday, Part II, (S. Gelbart, R. Howe, P. Sarnak, eds.), Isr. Math. Conf. Proc. 3, The Weizmann Science Press of Israel, Jerusalem, 1990, pp. 81–110.

[PS] I. Piatetski-Shapiro, Euler subgroups, in Lie groups and their representations (Proc. Summer School, Bolyai Janos Math. Soc., Budapest, 1971), Halsted, New York, 1975, pp. 597– 620.

[Ro] J. Roskies, The minimal representation of SO(4, 3) over a P -adic field, Dissertation, Stanford University (1996).

[Sab] H. Sabourin, Une représentation unipotente associé à l’orbite minimale: le cas de SO(4, 3), J. Funct. Anal. 137 (1996), 394–465.

[To] P. Torasso, Méthode des orbites de Kirillov-Duflo et représentations minimales des groupes simples sur un corps local de caractéristique nulle, Duke Math. J. 90 (1997), 261–377.

[We] A. Weil, Sur certains groupes d’opérateurs unitaires, Acta Math. 111 (1964), 143–211.