Multi-dimensional sequential testing and detection

We study extensions to higher dimensions of the classical Bayesian sequential testing and detection problems for Brownian motion. In the main result we show that, for a large class of problem formulations, the cost function is unilaterally concave. This concavity result is then used to deduce structural properties for the continuation and stopping regions in specific examples.


Introduction
In the seminal paper [17], two sequential problems of determining an unknown drift of a one-dimensional Wiener process were solved. To describe these problems, let Y be a continuous-time Markov chain with state space {0, 1} and transition rate matrix where λ ≥ 0 is a known constant, and with random starting point such that P(Y 0 = 1) = π and P(Y 0 = 0) = 1 − π for π ∈ [0, 1]. Moreover, let X be a stochastic process given by where µ = 0 and W is a one-dimensional standard Brownian motion independent of Y . In [17], the problem of sequential testing between two hypotheses and the problem of quickest detection of a drift change are solved. Sequential testing: In this problem, λ = 0 so that Y t = Y 0 for all t ≥ 0, and one seeks to determine the drift µY 0 as accurately as possible but also as quickly as possible. More precisely, for constants a, b, c ∈ (0, ∞), [17] considers the problem (1) inf where the infimum is taken over F X -stopping times τ and decisions d ∈ {0, 1} such that d is F X τ -measurable. Quickest detection: In this problem, λ > 0, and one seeks to detect the jump-time of Y as quickly as possible. More precisely, for b, c ∈ (0, ∞), [17] considers the problem where the infimum is taken over F X -stopping times τ .
In [17], problems (1) and (2) are solved separately using a 'guess and verify' approach involving an associated free-boundary problem for the cost function.
The two examples above are the generic formulations in the one-dimensional case, and there is a rich literature on various extensions. For example, testing and detection problems for a Poisson process with unknown intensity have been studied in [15] and [16], and a multi-source variant has been studied in [3]. The references [6] and [7] treat some aspects of testing and detection problems with general distributions of the random drift; since the penalties for a wrong decision in [6] and [7] are binary, the sufficient statistic is a one-dimensional, but time-inhomogeneous, Markov process. Formulations allowing for non-binary penalties appear in [1], [13] and [18], in which the natural sufficient statistic is two-dimensional and the analysis thus becomes more involved.
In the literature cited above, the observation process is one-dimensional; the existing literature on multi-dimensional versions is sparser. In [9], a three-dimensional Brownian motion is observed for which exactly one coordinate has non-zero drift, and the problem of determining this coordinate as quickly as possible is studied. In the set-up of [9], the three random drifts are heavily dependent; in fact, if one drift is non-zero then the remaining two drifts have to be zero. In [2] a less constrained set-up is used, in which two Poisson processes change intensity at two independent exponential times, and the problem of detecting the minimum of these two times is considered.
In the current article, we use a similar unconstrained set-up as in [2] to study sequential testing and detection problems for a multi-dimensional Wiener process. The variety of possible versions of such testing and detection problems is very rich; indeed, in some applications it would be natural to seek to determine all drifts as accurately as possible, whereas it would be more natural in other applications to determine only one of all possible drifts. Similarly, in the quickest detection problem some applications would suggest to look for the smallest change-point (as in [2]), whereas one in other applications would try to detect the last change-point; further variants are listed below. Rather than studying all different formulations on a case by case basis, the multitude of multi-dimensional formulations motivates a unified treatment of the corresponding stopping problems. It turns out that a large class of such problems can be written in the form (3) (or rather, a multi-dimensional version of (3)), with g and h both unilaterally concave (concave in each variable separately). In our main result we show that unilateral concavity of the penalty functions is preserved in the sense that also the corresponding cost function is unilaterally concave. Since many multi-dimensional penalty functions are unilaterally piecewise affine, the concavity property provides valuable information about the structure of the corresponding continuation and stopping regions.
There is related literature on preservation of spatial concavity/convexity (and consequences for volatility mis-specification) for martingale diffusions within the mathematical finance literature, see for example [8], [10] and [11] for one-dimensional results. In higher dimensions, preservation of concavity is a rather rare property, compare [12] and [5]. With this in mind, we point out that preservation of unilateral concavity is a weaker property; however, it is of less financial importance, and has therefore been less studied in the financial literature. Also note that for the multi-dimensional version of (3), the natural choices of g and h are typically not concave, but only unilaterally concave. We also remark that the authors of [2] use a three-dimensional embedding of a detection problem in order to obtain concavity of the value function; for unilateral concavity, however, one may remain in the two-dimensional set-up of the problem.
The paper is organised as follows. In Section 2 we specify the multi-dimensional versions of the sequential testing and quickest detection problems, and we provide a list of natural examples. In Section 3 we provide our unilateral concavity result for the multidimensional problem, and in Sections 4-5 we use the unilateral concavity to derive structural properties of continuation regions for the specific examples.

The multi-dimensional set-up
We consider a problem where one continuously observes an n-dimensional process X, and where the drift of each component X i is modeled using a continuous time Markov chain Y i with state space {0, 1} and transition rate matrix with λ i ≥ 0. Moreover, the initial condition satisfies P(Y 0 = 1) = π i ∈ [0, 1]. The observation process (X t ) t≥0 = (X 1 t , X 2 t , . . . , X n t ) t≥0 is assumed to be given by Here µ i > 0, i = 1, ..., n are known constants and W i , . . . , W n are one-dimensional standard Brownian motions such that Y 1 , ..., Y n , W 1 , ..., W n are independent. In parallel to the one-dimensional case, we introduce the multi-dimensional posterior probability process Π = (Π 1 , . . . Π n ) by , and, in particular, that the coordinates of Π are independent.
Below we list a few natural formulations of multi-dimensional sequential testing problems and multi-dimensional detection problems. These examples can be written as stopping problems of the form as in the one-dimensional case, but with g, h : [0, 1] n → [0, ∞) now being functions of the multi-dimensional process Π. Sequential testing. Assume that λ i = 0, i = 1, ..., n and that the penalisation in time is linear, i.e. of the type cE[τ ] for some constant c > 0. All formulations below can then be written on the form (4) with h = c but with different penalty functions g.
For simplicity we consider symmetric penalization (corresponding to a = b = 1 in (1)); generalizations to set-ups with non-symmetric weights are straightforward.
where the infimum is taken over F X -stopping times τ and decisions d ∈ {0, 1} n such that d is F X τ -measurable, i.e. the tester is penalised equally for every faulty decision. This problem can be written on the form (4) with where the infimum is taken over F X -stopping times τ and decisions d ∈ {0, 1} andd ∈ {1, . . . , n} that are F X τ -measurable. Thus the tester seeks to determine as quickly as possible a drift for only one of the processes. For this problem, the penalty function is given by (ST3) Let for simplicity n = 2 (generalizations are straightforward), let µ 1 = µ 2 =: µ and let γ ∈ [0, 1] be a given constant. Consider the problem where the infimum is taken over stopping times τ 1 , τ 2 and decisions d 1 , Here γ is a cost reduction parameter which describes how the cost (per unit of time) of observing two processes relates to the cost of observing only one process. Using the strong Markov property, this multiple stopping problem reduces to a problem of type (4) with penalty where u µ,c(1−γ) is the value function of the one-dimensional sequential testing problem with cost c(1 − γ) per unit of time, see (7) below.
Quickest detection. Now assume that λ i > 0 for all i, and let c > 0 be a constant. In all of the formulations below, the infimum is taken over F X -stopping times. (QD1) Consider the problem Here one seeks to determine the first change-point (this problem formulation was treated in [2] for a detection problem involving two Poisson processes); the problem can be written on the form (4) with Again, this problem can be written on the form (4); the corresponding functions g and h are given by (QD3) Assume that a tester wants to detect one coordinate for which the change-point has happened. One possible formulation of this is where the infimum is taken also over F X τ -measurable decisionsd ∈ {1, ..., n}. The problem can be written on the form (4) where the corresponding functions g and h are given by Now consider the stopping problem (4) for given functions g and h. Throughout the remainder of this article we make the following assumption.
It is well-known that the Π process satisfies . . , n, where the innovation processW = (W 1 , ...,W n ) defined bȳ is an n-dimensional Brownian motion with independent coordinates. Consequently, Π is an n-dimensional time-homogeneous Markov process with independent coordinates; allowing for an arbitrary starting point π ∈ [0, 1] n , we define a cost function V : We also introduce the continuation region and its complement, the stopping region D = [0, 1] n \ C, and we recall from optimal stopping theory that the stopping time is optimal in (6). We end this section with a short review of the one-dimensional problems.
2.1.1. Sequential testing. With the notation of the introduction, let The notation u µ,c is used when we want to emphasize the dependence on the drift µ and the cost of observation parameter c, and we refer to this one-dimensional testing problem as ST (µ, c). We then know that u : [0, 1] → [0, 1] is concave with u(π) ≤ π ∧ (1 − π). Moreover, for some A * ∈ (0, 1/2); further details on u and A * can be found in [14].
2.1.2. Quickest detection. Again with the notation of the introduction, let where the notation u µ,λ,c is used when we want to emphasize the dependence on the parameters µ, λ and c, and we refer to this one-dimensional detection problem as QD(µ, λ, c). The function u : [0, 1] → [0, 1] is then concave and non-increasing. Moreover, for some B * ∈ (0, 1); again, further details on u and B * can be found in [14].

Properties of the cost function
In this section we derive Lipschitz continuity and unilateral concavity for the multidimensional stopping problem (4). Proof. It suffices to check that V is Lipschitz in each variable π i separately. To do that, let i = 1 and denote by Π t the solution of (5) with initial condition Π t = π ∈ [0, 1] n , and denote byΠ t the solution with initial conditionπ = (π 1 , ...,π n ), where π j =π j , j = 2, . . . , n and π 1 <π 1 . By a comparison result for one-dimensional stochastic differential equations, where C is a Lipschitz constant of g. This shows that if h is constant, then V is Lipschitz in its first argument, and thus it is Lipschitz also in π.
Furthermore, if λ 1 > 0, then where D is a Lipschitz constant of h. It follows that for any stopping time τ . Therefore V (π 1 , . . . , π n ) is Lipschitz in its first argument. Consequently, if λ i > 0 for all i = 1, ..., n, then V is Lipschitz also in π.
Remark 3.2. It follows from the proof above that if λ i = 0 for i = 1, . . . , n, g is Lipschitz 1 in each variable separately and h is constant, then also V is Lipschitz 1 in each variable separately.

3.2.
Unilateral concavity. Next we study unilateral concavity of the value function. LetP be a new measure defined so that dP dP and denote byẼ the corresponding expectation operator. By the Girsanov theorem, X t is an n-dimensionalP-Brownian motion. Define the probability likelihood process Φ = (Φ 1 , ..., Φ n ) by and observe that Φ i 0 = π i 1−π i =: φ i . Also note that an application of Ito's formula yields and observe that Y 0 = 1. Using Ito's formula and (5) we find that so we can rewrite the value function as which completes the proof.
Proof. It suffices to check that π 1 → V (π) is concave. To do that, first note that since (8) is a linear equation, it can be solved explicitly as Thus Φ i t is affine in φ i and independent of φ j , j = i. Moreover, denoting we have that Fix an F X -stopping time τ ; we next claim that To see this, assume that g is C 2 (the general case follows by approximation). Then (11) is concave in π 1 . Taking expectation we have that is concave in π 1 . By similar arguments, is concave in π 1 for each stopping time τ . Taking infimum over stopping times τ , it follows that π 1 → V (π) concave, which completes the proof.

Sequential testing problems
In this section we use the general results of Section 3 to provide structural results for the multi-dimensional sequential testing problems (ST1)-(ST3). For the sake of graphical illustrations, we present the results for the case n = 2; the higher-dimensional version works similarly, and our results easily carry over to that case.
Remark 4.1. In the structural studies of (ST1)-(ST3) and (QD1)-(QD3) below, we focus on what conclusions can be drawn from our main result on unilateral concavity. Refined studies would aim at further properties of the stopping boundaries that we find. For example, a lower bound on the continuation region is provided by the set {Lg + h < 0}, where L is the generator of Π, methods to prove that wedges of g are automatically contained in the continuation region can be obtained, and studies of continuity of stopping boundaries can be performed along the lines of [4].
The monotonicity property is a consequence of symmetry and concavity: if (π 1 , π 2 ) ∈ T ∩ D then also (1 − π 1 , π 2 ) ∈ T ∩ D, so unilateral concavity yields that the whole line segment {(p, π 2 ); π 1 ≤ p ≤ 1 − π 1 } belongs to the stopping region. Finally, the asserted upper semi-continuity of b follows from the continuity of V . Proof. Take π ∈ R, and let i ∈ {1, 2} be such that g(π) = π i ∧ (1 − π i ). Define } to be the optimal stopping time in the one-dimensional problem of determining Y i . Then , which shows that π ∈ C.
For a graphical illustration of the continuation region in (ST2), see Figure 1(b).
Proof. We first claim that [0, 1] × {0} ⊆ D. To see this, note that g(π 1 , 0) = u(π 1 ) and that u(Π 1 t ) + ct is a submartingale. It follows that V (π 1 , 0) = u(π 1 ). Next, the fact that g is affine in π 2 on T together with unilateral concavity of V give the existence of b. The upper semi-continuity of b follows from continuity of V , and the symmetry of b follows from the symmetric set-up.

Quickest detection problems
In this section we provide structural results for the multi-dimensional quickest detection problems (QD1)-(QD3). For the sake of graphical illustrations, we present the results for n = 2.
For a graphical illustration, see Figure 3(a).
The continuation region in (QD2) is illustrated in Figure 3(b).
Moreover, b is lower semi-continuous and first non-increasing and then non-decreasing.
Proof. We first note that g(π 1 , 1) = 0, so [0, 1] × {1} ⊆ D, and that g is affine in π 2 on T ; concavity thus implies the existence of b such that (13) holds. Moreover, g is affine also in π 1 on T , so horizontal sections of the stopping region inside T are intervals. Consequently, the function b is first non-increasing and then non-decreasing. Lower semi-continuity follows from the continuity of V .