Application of optimal control to the dynamic advertising decisions for supply chain with multiple delays

This paper is concerned with application problem of optimal control to a class of dynamic advertising models with multiple delays. Here, a dynamic model with state and control delays is introduced to describe the impacts of advertising delayed and memory effects on the evolution of brand goodwill. In the decentralized and centralized systems, the optimal advertising strategies of supply chain members are presented by utilizing the nonzero-sum differential game governed by differential equation with multiple delays in state and control variables. Special effort is made to analyse the geometrical shapes of advertising strategy and provide the optimal decision structure of supply chain system. Furthermore, nonzero-sum stochastic differential game is applied to explore the dynamic advertising decision problems of supply chain system with the multiple delays and stochastic cases. Finally, numerical examples are exploited to illustrate the effectiveness of the proposed results.


Introduction
The rapid development of economic globalization made the cut-throat competition between enterprises. As a result, advertising campaign has played an essential role in expanding product demand and enhancing brand awareness. Reasonable advertising strategy, as a core part of operation management, is one critical factor to maintain the sustainable development of enterprise. The past decade has witnessed fruitful research results on the advertising decision problems primarily owing to their application in the marketing science domain. In terms of theoretical research, a rich body of literature has been reported on the static and dynamic models. To be specific, the cooperative advertising problem has been studied in Yan, Cao, and Pei (2016) for a two-tier supply chain in demand uncertainty or information sharing situations. It's pointed out that the whole profit of supply chain can benefit from the cooperative advertising programme. In Hong, Xu, Du, and Wang (2015), Zhao, Zhang, and Xie (2016), Chernonog and Avinadav (2019), the advertising, pricing and inventory decision problems have been simultaneously explored for supply chain in the demand uncertainty. Among them, static models cannot capture the impact of current period decision on the CONTACT Hui Yu hyu_hust@163.com future performance. Hence, we aim to investigate the advertising decision problems of supply chain system in the dynamic circumstance. Dynamic advertising models have a long history in the marketing science, which can date back at least to the seminal paper of Nerlove and Arrow (1962), and Vidale and Wolfe (1957). Since then, a great deal of effort has been made concerning with the extension of these models and their applications to advertising decision problems, both in monopolistic, oligopoly and competitive settings (Chutani & Sethi, 2018;De Giovanni, Karray, & Martin-Herron, 2019;He, Gou, Wu, & Yue, 2013;Thorstenson & Ramani, 2018;Wu, Chen, & Feng, 2018;. To mention just a few, the impact of retailer's competition intensity has been studied in He et al. (2013) on the choice of decision mechanism in the competitive supply chains. It is pointed out that the profit of supply chain is higher in the centralized system than the one in the decentralized case. Motivated by He et al. (2013), the impacts of retailer's competition and cooperation behaviour have been explored in Yu et al. (2018) on the optimal advertising strategies and the choice of decision mechanism. In addition, consignment contract and bilateral subsidy over advertising contract have been designed to coordinate the competitive supply chains, see e.g. Wu et al. (2018), Thorstenson andRamani (2018), De Giovanni et al. (2019). When the competition between supply chains becomes a concern, the frameworks have been presented in Huang et al. (2018) and Chutani and Sethi (2018) to discuss the 'multiple manufacturers vs multiple retailers'. For example, the optimal decision problems have been explored in Huang et al. (2018) for a class of dynamic promotion models in the multiple competitive supply chains, where there exist vertical and horizon joint promotion programmes in this complex system. In Chutani and Sethi (2018), consider the situation that every retailer simultaneously sells the products of the whole manufacturers, dynamic cooperative advertising problems have been studied in a market with a finite number of independent manufacturers and retailers. Moreover, along with the previous results in , Berkowitz, Allaway, and D'souza (2001), Baack, Wilson, and Till (2008), it is reasonable to assume that more realistic dynamic advertising models should allow for delay time in the advertising effectiveness. Namely, this phenomenon is called delayed effect.
As is well known, the delayed effect is a time gap between advertising exposure and the corresponding effect on product or brand goodwill, which frequently occurs in the process of advertising marketing. The causes of delayed effect lie in objective and subjective aspects, such as transportation or communication lags, feedback delays and so on. In the past years, continuous effort has been devoted to the equilibrium analysis, prediction and decision problems. It is notable that a discrete delay advertising model has been proposed in Bultez and Naert (1979) for a monopolistic firm and optimal advertising decision has been determined. In Liang (2007), the sale prediction models have been constructed by applying Ito equation to deal with random parameters. Stochastic optimal control approach has been utilized to deal with the advertising decision problem in Gozzi, Marinelli, and Prato (2005) for one firm. Moreover, by applying a cooperative differential game with immediate and delayed effects of control variables, the optimal advertising strategies have been obtained in the competitive market and the length of delayed response in generating goodwill has been examined in Machowska (2019). Very recently, except [21], little research attention has been paid on the advertising decision problems for supply chain system with delayed effect, not to mention the case when the delayed effects of manufacturer's and retailer's advertising and memory effect are also taken into account. This study contributes to the literature in two main aspects: (1) It simultaneously introduces delayed and memory effects into the dynamic advertising model and explores the impacts of delayed and memory effects on the advertising strategies of supply chain members. (2) It constructs a comprehensive model including advertising effects and stochastic disturbance. The stochastic disturbance are divided into two parts: internal disturbance and external disturbance. In general, the internal disturbance in the processing of brand evolution are inevitable as industrial background and human factors. Besides, the external disturbance originates in one or more uncontrollable variables of enterprises. The introduction of these factors will enrich dynamic model series and bring the model close to the reality.
Motivated by the above discussion, we aim to investigate the optimal control problems for a class of dynamic advertising models with multiple delays. Here, a dynamic model with multiple delays in state and control variables is introduced to depict the impact of manufacturer's and retailer's delayed effects and memory effect on the evolution of brand goodwill. Suppose that product demand increases linearly with manufacturer's advertising effort, retailer's advertising effort and brand goodwill in an additive way, optimal advertising strategy of each member is obtained in the decentralized and centralized systems, respectively. Also, the geometrical shapes of advertising strategy have been examined. Besides, the extensive concern has been focused on dynamic advertising model with multiple delays and stochastic disturbance. The nonzero-sum stochastic differential game is applied to obtain optimal advertising strategies of supply chain members under the advertising delayed effect, memory effect and stochastic circumstance. Finally, illustrative examples are exploited to show the effectiveness of the addressed results. An outline of this paper is as follows. Taking the decentralized and centralized decision systems into account, Section II develops a differential game model involving a single manufacturer and a single retailer, and discusses advertising decision problems for supply chain system subject to advertising delayed and memory effects. Section 3 investigates the advertising decision problems for supply chain system subject to advertising delayed effect, memory effect and stochastic disturbance. The main results of these two decision systems are studied and compared, followed by the illustrative examples in Section V. The illustrative examples are given to show the feasibility and effectiveness of the proposed results. Finally, Section IV summarizes the results.
Notations: The notations used here are summarized as follows. d i ≥ 0 (i = M, R) denotes the delay time of advertising effectiveness, and τ reflects the memory span of advertising. * and * * stand for, respectively, the equilibrium results in the decentralized and centralized systems. E is the mathematical expectation operator andĠ stands for the derivative of G.

Advertising model with multiple delays
Let a supply chain consist of a manufacturer and a retailer, in which the downstream retailer sells products made by the upstream manufacturer to the end consumers. To build up the brand goodwill and derive the incremental product sales, the manufacturer and the retailer invest in national advertising and local advertising, respectively. We denote the manufacturer's national advertising effort over time t as U M (t) and retailer's local advertising effort over time t as U R (t). In general, the advertising efforts of the manufacturer and the retailer can simultaneously contribute to the development of brand goodwill, and delayed effects are occurred in the process of their advertising. Then, the evolution of brand goodwill with delayed and memory effects is described by the following differential delay equation: where G(t) is brand goodwill with the initial value of brand goodwill G 0 . γ i (i = M, R) and δ are positive constants representing the effectiveness of national or local advertising and decay rate. From the differential delay equation (1), the growth of brand goodwill depends on the manufacturer's national advertising U M (t) that prevailed d M periods ago, the retailer's local advertising U R (t) that prevailed d R periods ago and the brand goodwill G(t) that prevailed τ periods ago. Consequently, the brand goodwill does not decline instantly and stays the initial goodwill value in the interval [0, τ ]. Similarly, during the interval [0, d i ] (i = M, R), the advertising campaigns also do not generate immediate effect.
One common way to model the product demand is to consider the sum of the advertising effort and the brand goodwill in a separable form. Therefore, the product demand function S(t) can be expressed as follows: ( 2 ) Here, α, β M and β R are all positive constants. α is the positive impact of brand goodwill on the product demand. β i (i = M, R) measures the sensitivity of product demand to the advertising effort. In accordance with the previous literatures (Ezimadu, 2019;Shi & Petrosyan, 2018;Wang, Gou, Choi, & Liang, 2016), the quadratic cost functions about advertising efforts are proposed below: In (3), k i > 0(i = M, R) is the cost coefficient of advertising. These convex and increasing functions mean increasing marginal costs of advertising efforts. Let exogenous parameters ρ M and ρ R be the marginal profit of manufacturer and retailer, respectively. Supposing a same discount rate ρ > 0 and an infinite horizon, the profit functions of the manufacturer and the retailer can be denoted, respectively, as follows: For the supply chain, the profit function is To recapitulate, we have defined an infinite-horizon nonzero-sum differential game with two control variables U M (t), U R (t) and one state variable G(t). In the following subsection, by applying the Maximum principle, we will calculate the optimal advertising efforts of supply chain members in the decentralized and centralized systems and analyse their strategic results. In order to facilitate the subsequent developments, the following Lemma in Bokov (2011) is introduced without proof which will be useful to know the Nash equilibrium point of the above game problem.
Lemma 2.1: The controlled system is modelled by the following differential delay equation: and the performance functional is where the state x 0 (·) : [t 0 − τ , t 0 ] → R n is piecewise smooth function and the control u 0 (·) : [t 0 − d, t 0 ] → U is a piecewise continuous function. Assume that functions g(·, ·, ·, ·, ·), L(·, ·, ·, ·, ·) and their partial derivatives with respect to the variables x are continuous. If (u * (t), x * (t)) is an optimal control process of the above optimal problem, then there exists Lagrange multiplier p(t) such that the following conditions hold: is the Hamilton function.

Decentralized system
In the decentralized system, the manufacturer and the retailer make various strategies to maximize the present value of their own profits. By considering the evolution of brand goodwill, the profit maximization problems in the decentralized system can be given by: and max U R (t) Based on Nash equilibrium game and Lemma 2.1, the following results depict the optimum strategies on national and local advertising efforts in presence of delayed and memory effects.
Theorem 2.1: In the decentralized system, the manufacturer's and retailer's optimal advertising efforts satisfy

two branches of the Lambert's W function in the real-valued domain.
Proof: In the following proof, we utilize the Lemma 2.1 to obtain the optimal advertising efforts of manufacturer and retailer. For the manufacturer, combining the Equations (1) with (7), we introduce the Lagrange multiplier p M (t) to construct the Hamilton function of manufacturer as We obtain the equilibrium strategy of manufacturer by solving the first-order condition, (9) Next, by utilizing the second equation in the Lemma 2.1, we haveṗ The overall solution of Equation (10) is denoted as is the particular solution of delay differential Equation (10) and p MH (t) is the general solution of the following homogeneous differential equatioṅ To obtain the general solution, letting p MH (t) = e st and substituting it into the homogeneous differential Equation (11), one has (−sτ + ρτ )e −sτ +ρτ = −δτ .
By applying the Lambert's W function in Aravindakshan and Naik (2015), we have Specifically, it possesses two real-valued branches W 0 (x) and W −1 (x), then To obtain the particular solution, supposing p MP (t) = A + Bt with A and B being unknown parameters and inserting it into the differential equation (10), we get Therefore, it follows from (12) and (13) that Substituting p M (t) into Equation (10), the optimal advertising effort of manufacturer is given in Theorem 2.1. Similar to the solution process of manufacturer's advertising decision, it's easy to obtain the optimal advertising effort of retailer.
Proof: According to the properties of Lambert's W function, when δτ > 1/e, both W 0 (−δτ ) and W −1 (−δτ ) are complex values. Denoting the conjugally of W 0 (−δτ ) = a + bi and Obviously, p iH (t) (i = M, R) is the oscillation solution of homogeneous differential equation, then supply chain members adopt pulsing advertising schedule or flighting advertising schedule. On the other hand, when δτ < 1/e, both W 0 (−δτ ) and W −1 (−δτ ) are real values. Hence, p iH (t) (i = M, R) increasing with t is the non-oscillation solution of homogeneous differential equation, then supply chain members adopt monotonic advertising schedule.
Remark 2.1: From Theorem 2.1, it can be easily observed that all the optimal advertising strategies of manufacturer and retailer rely on memory span τ , decay rate δ, discount rate ρ and delay time d i . To be specific, the memory effect decides the advertising schedule. The impact of delayed effect on advertising strategy mainly focus on the input time and the effort level.
Although these findings can be proven analytically, we illustrate them via numerical examples for clarity. Figures 1-4 show the geometrical shapes of advertising strategy.
In Figure 1, since memory span τ = 0, brand goodwill decay begins instantaneously. To maintain the level of brand goodwill, a brand or a product must advertise continuously at a constant situations. Hence, the optimal advertising effort remains constant level over time. When τ = 0, brand goodwill decay is delayed. Figures 2-4 display advertising schedule for various (δ, τ ). Non-constant advertising strategies merge because, as brand goodwill builded, the brand goodwill can afford to reduce or stop spending for a certain time. Due to the memory effect, the brand goodwill is better off by not advertising and depending on consumer's ability to remember the advertising information.
Up to now, the analysed research of even advertising schedule has been investigated extensively, see, e.g. He    optimal brand goodwill of product is where a = Re(W 0 (−δτ )), b = Im(W 0 (−δτ )), and X(t) satisfies as follows: Proof: Based on the Corollary 2.1, when both supply chain members adopt the pulsing advertising schedule, the optimal advertising efforts of manufacturer and retailer are Inserting Equations (14) and (15) into delay differential Equation (1), the optimal brand goodwill of product can be obtained by applying the solving approach in Zheng (1992).

Centralized system
In the centralized system, the manufacturer and the retailer maximize the profit of supply chain by making the optimal advertising decisions and the corresponding optimization problem is given by:

s.t. (1)
To derive the optimal advertising efforts in the centralized system, we firstly introduce the Lagrange multiplier p S (t) to construct Hamilton function as The following results characterize the optimal advertising decisions of supply chain members in the centralized system. The proofs for these results are similar to the Theorems 2.1 and 2.2.

two branches of the Lambert's W function in the real-valued domain.
When manufacturer's advertising and retailer's advertising affect the brand goodwill of product in common, no matter what kind of decision-making systems, the optimal advertising efforts of supply chain members both consist of short-term effect and long-term effect. The first item in the equilibrium results implies the short-term effect of advertising investment. Having failed to take the evolution of brand goodwill into account, myopic manager merely makes the optimal decisions to maximize the short-term profit of their own or supply chain, and then advertising effort depends on their own marginal profit. Moreover, the second item in these equilibrium results implies the long-term effect of advertising investment. In this case, the advertising effort relies on delayed effect, memory effect as well as decay rate of brand goodwill.
Theorem 2.4: In the centralized system, when both supply chain members adopt pulsing advertising schedule, the optimal brand goodwill of product is where a = Re(W 0 (−δτ )), b = Im(W 0 (−δτ )), and X(t) satisfies as follows: Based on the aforementioned models, we compare the optimal advertising decisions in the centralized with those in the decentralized one. The following corollary recapitulates the relative findings.

Corollary 2.2: Comparing the decentralized with the centralized systems, we obtain:
(a) Whatever advertising schedule is adopted, the optimal advertising efforts of supply chain members decrease in the decentralized system, that is, When supply chain members both adopt the pulsing adverting schedule, the brand goodwill level of product and product demand decrease in the decentralized system, that is, G * * (t) > G * (t), S * * (t) > S * (t). (c) When supply chain members both adopt the pulsing adverting schedule and memory span is less than −ln ρ/ρ, the optimal profit is higher in the centralized system, that is, Corollary 2.2 indicates that, the centralized system is an effective incentive policy for stimulating advertising efforts, strengthening the brand goodwill and further pushing up the product demand. These results are similar to the previous research in He et al. (2013), Wu et al. (2018, Zhao et al. (2016). However, different from those, it should be noted in this paper that, only when the memory span locates in the interval [0, −ln ρ/ρ], the profit of supply chain is better off in the centralized system.

Extensive advertising model with multiple delays and stochastic cases
It is now well recognized that the stochastic disturbances are frequently encountered in practice. Also, the emergence of randomness in the dynamics of goodwill is quite natural for several reasons: one may think, for example, that random fluctuations in the goodwill level are the effect of external factors beyond the control of the firm, or that noise enters through the control, since the effect of advertising may be partly uncertain. In this section, the dynamic advertising decision problems will be discussed for supply chain system with multiple delays and stochastic disturbance cases. In general, brand goodwill of product evolves according to the stochastic differential delay equation (SDDE): (18) where ω(t) is the standard Brownian motion, σ (G(t − τ )) is effect coefficient of stochastic disturbance. Then, the manufacturer's and the retailer's objectives are eager to find the optimal advertising decisions to maximize the expected profit of their own or supply chain system: and (1) (21) here, ε M G(T) and ε R G(T) are the salvage values of terminal goodwill.
Remark 3.1: A short planning horizon is inconsistent with dynamic optimization, but a long planning horizon is undesirable in a rapidly changing industry or in an uncertain environment. Therefore, a finite-horizon nonzerosum differential game with two control variables U M (t), U R (t) and one state variable G(t) is defined.
The following Lemma is introduced without proof to give the Nash equilibrium point of finite-horizon nonzerosum differential game problem.
Lemma 3.1: The controlled system is modelled by the following stochastic differential delay equation: and the performance functional is is an optimal control process of the above optimal problem, then there exist Lagrange multipliers p i (t) and z i (t) such that the following conditions hold: Here, H i (t, (t), ) + l i (t, (t)) is the Hamilton function, in which H iu i represents the partial derivative of Hamilton function H i with respect to u i (t), and Lagrange multipliers (p i (t), z i (t)) satisfy: Similar to the proofs of Theorems 2.1 and 2.2, we can get the following results by utilizing the Lemma 3.1.
Theorem 3.1: In the decentralized system, the manufacturer's and retailer's optimal advertising efforts satisfy Theorem 3.2: In the centralized system, the manufacturer's and retailer's optimal advertising efforts satisfy where, ε MR = ε M + ε R and ( MR (t), MR (t)) is the unique solution of Remark 3.2: According to the algorithm of literature (Chen & Yu, 2015), we will solve the stochastic differential delay equation (24) step by step, at each step, the stochastic differential delay equation is simplified to a Lebesgue's interval without randomness. Then, one has i (t) ≡ 0, t ∈ [0, T] holds and Next, we will compare the optimal advertising efforts of supply chain members in the different decision systems via numerical example.

Numerical examples
Based on the results proposed in the previous sections, we use illustrative examples here to illustrate the impacts of discount rate on advertising schedule and comparative analysis of manufacturer's optimal advertising efforts in the decentralized and centralized systems with stochastic disturbance.
Example 4.1: The following parameters are used in our analysis: decay rate of brand goodwill δ ∈ (0, 1), memory span of advertising τ ∈ (0, 30) and discount rate ρ ∈ [0.08, 0.1]. The distribution of advertising schedule under the different discount rates is shown in Figure 5.
It is easy to know from Figure 5 that when the discount rate of society is ρ = 10%, in the Regions I and IV, the managers adopt the monotonic and pulsing advertising schedule, respectively. Otherwise, the flighting advertising schedule is adopted in the Regions II and III. On the other hand, when the discount rate of society is ρ = 8%, in the Regions I and II, the managers adopt the monotonic and flighting advertising schedule, respectively. Otherwise, the pulsing advertising schedule is adopted in the Regions III and IV. These findings show that with the increase of discount rate, a part of enterprises (Region III) will change their advertising schedule and then adopt flighting advertising schedule. Due to its flexible and elastic feature, flighting advertising schedule can help enterprises to timely adjust the best advertising exposure time and speedily improve the reach rate of target audience. However, everything has two sides. The disadvantages of flighting advertising schedule are two aspects: (1) If gap period of advertising has an excessive length, it may seriously weaken the memory effect of advertising and then difficulty rebuild the brand goodwill; (2) If gap period of advertising does not have an excessive length, continuous advertising investment will bring the higher operation cost of enterprise, so as to reduce their gross profit.

Example 4.2:
To compare the optimal advertising efforts of decentralized and centralized systems in the stochastic circumstance, we use the following parameter values to establish ranges for model parameters: Specifically, choose that memory span and decay rate satisfy: (1) δ = 0.01, τ = 1; (2) δ = 0.1, τ = 4; (3) δ = 0.7, τ = 4. Since both the optimal advertising efforts of retailer are similar to the manufacturer's ones, the manufacturer is regarded as a research subject. Moreover, by using the three different groups of numerical parameters, the manufacturer adopts the monotonic, pulsing and flighting advertising schedule, respectively. Later, the comparative analysis of manufacturer's decision results is illustrated in the decentralized and centralized systems and shown in  the centralized system than those in the decentralized system. Integrating with the results in Corollary 2.2, we can see that the centralized system is an effective incentive policy for stimulating advertising efforts of supply chain members.

Conclusions
This paper has concerned with application problem of optimal control to a class of dynamic advertising models with multiple delays and stochastic disturbance. First, a dynamic model with state and control delays has been introduced to describe the impact of delayed and memory effects on the evolution of brand goodwill. By utilizing the nonzero-sum differential game approach, the optimal advertising strategies of supply chain members have been obtained in the decentralized and centralized systems. Second, nonzero-sum stochastic differential game has been applied to explore the dynamic advertising decision problem of supply chain system with the multiple delays and stochastic disturbance. Finally, illustrative examples have been exploited to show the effectiveness of the proposed results. This research can be extended in several directions. The competition between enterprises or regions has changed into the competition between supply chain. Hence, The competition between supply chains in the real world is of great importance, which has two forms, namely, internal competition (Karray, 2015) and external competition (Karray, Martin-Herron, & Zaccour, 2017). In general, the occurrence of the internal competition attributes to the lack of cooperation, the exclusion of operation management and the conflict of strategic objective. The external competition originates in one or more external environment parameters of enterprises. Besides, the competition between supply chains influence not only the decision-marking of supply chain, but also the choice of decision mechanism. Therefore, it is necessary to investigate the optimal advertising decision problems for a class of competitive supply chain systems with taking the multiple delays and stochastic disturbance.into account.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Funding
This work was supported by the Natural Science Foundation of Heilongjiang Province of China No. A2018007; the Science Funds for the Young Innovative Talents of HUST of China No. 201508.