An overview of the consensus problem in the control of multi-agent systems

ABSTRACT As a solution to the distributed multi-agent coordination, the problems of consensus or agreement have been widely explored and studied in the literature. This document provides an overview of consensus problems in multi-agent cooperative control with the goal of exposing the related literature and promote the research in this area. The document presents the theoretical results concerning the search for consensus in the involved topologies with information exchange that is invariant in time and change dynamically. Applications related to consensus protocols are studied for the cooperation of multi-agent systems. The presentation includes as well open problems and offers future research direction.


Introduction
In recent years, the consensus problem in multi-agent systems (MAS) has become an attractive research area of interest, due to the potential applications in such broad areas as in control of unmanned aerial vehicles (UAVs), sensor networks, distributed computing in computer science, medicine, environmental controlling system and military observation [1][2][3][4][5][6][7][8]. The aim is to control a group of agents connected by a communication network so as to reach an agreement on common static value [9][10][11][12][13].
A typical field coordination problem for distributed MAS is the consensus problem, which entails the design of a distributed control policy based on local information gained by each agent, so that all agents can reach an agreement on certain quantities of interest in negotiating with their respective neighbours [14][15][16][17]. In general, all these problems have agents in a group communicating with one another to exchange information [11]. In most of the current research works on consensus associated problems, the agreement between the agents is obtained through their cooperation [12]. The complexity is usually the result of the topology and the nature of the interactions between agents that are often stochastic [18]. A multi-agent in the most general sense is defined as a network system of several local interactions of autonomous agents. Each agent is assumed to hold a state regarding a certain quantity of interest according to the contextual agents' states which could be related to opinions, values, beliefs, positions, speed, among others [19].
The application of consensus can be found in many research areas. In biology, the dynamic of consensus is studied for instance in the flocking aggregation behaviour of fish and bird schools [20]. Consensus models can be used to analyze, predict and elucidate the behaviour of flocking. In robotics and control, consensus problems come to light in coordination and cooperation of agents in robots and sensors, where this is an important matter in the network environmental applications [21,22]. In economics, the consensus is used to determine agreement on a common confidence in the price decision process. In management science, the consensus problem has been studied for community management decision [23]. In sociology, it is employed for a common language in primal societies and for the dynamics of opinion formation in social networks [24]. In computer science, it has been a widely researched and explored subject of interest [25].
Generally, interest in distributed systems is motivated by coordinate and control of multi-agents in the large-scale networks by accessing shared information to reach upon an agreed decision (value), or consensus convergence on a common point of interest. Many results have been achieved in this area. Another work on distributed computing over networks was also presented in the research of Borkar and Varaiya [26]. In parallel computing, the problem of asynchronous computation was considered and studied by Tsitsiklis [27]. Fax and Murray [28] on the other hand, offered the steadiness study of multivehicle formation control in dynamic system for their realization of desired trajectories. In another work, a theoretical framework for solving the problem of consensus was introduced by Olfati-Saber and Murray [29]. Later on, Cao et al. [30] addressed a graphical approach of a linear model for consensus in a dynamically changing environment. The consensus issue was then explored further in [31], where the authors studied the problem of stress consensus in MAS to dynamically change asymmetric networks with communication delays. Additionally, Hendrickx et al. [32] discussed a linear consensus in finite time by stochastic matrix with positive diagonals. Most recently, Hu et al. [33] investigated a general linear dynamics consensus of MAS controlled by event-triggered scheme with some required features of a distributed environment.
However, the studies mentioned above are all constructed on the conjecture that the dynamics relating to the consensus of agents are linear protocols. This conjecture cannot always be satisfied because of the fact that physical systems of engineering are of a special type of consensus problem [34]. For these physical systems, it is not appropriate to accept that their behaviour can be changed via an unbounded value. This in effect, recommends developing consensus protocols which guarantees that the general initial statuses are bounded [34]. Further, the yield protocol runs can be used to improve the performance of dynamic consensus algorithm or to satisfy other restrictions [35]. From these reasons, the work at hand is motivated to present an overview of linear, nonlinear protocols of some researches, and propose a nonlinear protocol for consensus problem in MAS.
Yu-Mei and Xin-Ping [36] proposed nonlinear consensus protocols and presented the related simulations of their new protocols. In their work, the authors' results demonstrated that the new proposed method is more effective and serves as an improvement compared to the linear protocols for the formation control of the agents. Wang et al. [37] mentioned that in cases where the dynamic agents are physical models, then the proposed nonlinear protocol to control the MAS under a fixed network topology is appropriate to avoid a possible loss of power during consensus controls if there are uncertain measurements. The nonlinear protocol to the problem of consensus has a better performance and robustness than that of linear characteristics [38].
This background in turn motivated and triggered the design of nonlinear consensus protocols for consensus problems in MAS. However, there still exists considerable difficulty in constructing a concrete Lyapunov function for a nonlinear system, which also motivates this work to seek for a more suitable tool for judging the stability of nonlinear systems. Hence, the difficulties in building a nonlinear system require research effort, which serves as an incentive for exploring and investigating further the stability of nonlinear systems.
Many research works have also attempted to consider nonlinear protocols in developing the consensus convergence for consensus problem in MAS. The nonlinear system poses difficult challenges in studying the consensus problem of a static graph for the nodes [39]. Indeed, the early-related research is nonlinear control theory of the stability which proved to be constructive [40]. Olfati-Saber and Murray [41] proposed linear and nonlinear protocols for consensus agreement in distributed and cooperative systems. It is argued that, the analysis of nonlinear consensus protocols has to be considered in a case where the applications of dynamic agents are physical models taking the input constraints into account. Additionally, Bauso et al. [42] has considered nonlinear consensus protocols for networks of dynamic agents under an undirected network of fixed topologies. The nonlinear framework of discrete time was then proposed by Moreau [43]. A new technique via nonlinear dynamics was then established in the works of Yu et al. [44] for the consensus problem of cooperative agents in a network environment. Another work based on nonlinear operator has been presented in [45] for convergence of the individual agents in network connectivity to reach a common value. In yet another consideration, Hui and Haddad [46] designed nonlinear protocols for consensus problems that guarantee convergence for multi-agent dynamical systems and at the same time addressed the Lyapunov's stable equilibrium. Zhu and Martínez [47] on the other hand, have proposed a class of discrete-time dynamic approach where the output stability properties are reckoned for convergence. Moreover, Meng et al. [48] obtained an iterative learning control by an agent's interaction with respective neighbours and through the use of stochastic matrices to reach a convergence formation control of nonlinear MAS. The approach uses optimistic optimization to control the behaviour of agents and simplify a black box with unknown nonlinearities in mathematical form [49]. A different work by Ajorlou et al. [50] studied nonlinear consensus class of continuous time for convergence. In this case, each agent is controlled by its state of combination with its neighbour's information in the graph. The consensus algorithm is also provided where some sufficient conditions are guaranteed for the convergence of the agents at a common point.
The traditional approach to controlling consensus problem is often based on linear models, which take its origin from DeGroot's model [23]. Nonetheless, various researchers in recent times have proposed nonlinear models owing to the fact that linear models converge slowly, with higher number of iterations, and being incapable of converging to optimal consensus. Nonlinear models on the other hand, converge faster, with lesser number of iterations and to approximate optimal consensus [51][52][53][54][55][56].
Despite their advantage, the downside of nonlinear models is that, they are often characterized by higher complexity and are setup with restricted conditions. The present concern is to investigate possible nonlinear models with faster convergence to optimal consensus, and yet with relatively low complexity and more flexible system conditions.

Consensus problem
In many literatures, the consensus or agreement problem is defined as the convergence to a common value [10]. The most famous and more action challenges for MAS are: operate, negotiate and reach an agreement [9]. The main problem often focused in distributed systems is a consensus problem. The consensus problem depicts how several autonomous agents (MAS) converge to reach agreements through their local interactions. Further, the expression of common agreement means that all the cases of the autonomous agents are equal [39]. This work's interest in distributed systems is motivated by coordination and control of multi-agents in the large-scale networks and through access of information to reach an agreed upon decision (value) or convergence to a common consensus.
The consensus problem has a long history in the work of DeGroot [57] and the necessary and sufficient conditions of DeGroot's model were addressed by Berger [58]. A consideration in terms of distributed computing over networks has also been presented in the research of [26]. The collocation issue is yet another study related to the consensus problem which was then explored by Jadbabaie et al. [22].
The interactions among agents can either be selfish as in the (for instance in a free market economy) or cooperative as in the (for example in an ant colony) [59].

Overview
Over the past few years, a lot of research has been carried out in the field of cooperative control and with respect to distributed systems. These studies have examined and well explored the related problem of consensus among agents within such a distributed system. In the process, such works have attempted to guarantee that a consensus is reached by the said agents through proposed linear consensus systems. Consensus problems have a long history involving groups, management science and statistics, starting from a "Parimutuel" linear method for agreement to a consensus of individual distributions for subjective probability distribution [60]. Similarly, DeGroot [23] proposed a solution to address the linear consensus problems using stochastic matrix along with a feasible model called the "DeGroot Model". This model covers all possibilities by individuals to reach a certain point of agreement despite differences of opinion. It utilizes transition matrix (stochastic matrix) to prove how consensus is reached. The adaptive linear control of the Markov chain was then proposed to converge by the interaction among the parameterization under the feedback rule [61]. Meanwhile, Berger [58] improved on DeGroot's [23] linear model and proved that reaching a consensus also depends on vector columns (initial values) that represent opinions on the DeGroot model. Tsitsiklis [21] and [27] on the other hand, presented an agreementbased protocol in form of a distributed linear algorithm for asynchronous problems in parallel computing. Vicsek et al. [20] in turn, conducted a linear model study for a specific situation on the movement of independent agents at the same constant speed and maintaining their positions in the closest neighbour. Furthermore, generalization concepts of distributed linear algorithms for fixed-topology networks were presented by Lynch [62]. Later on, Olfati-Saber and Murray [41] introduced linear and nonlinear consensus protocols using undirected graph theory for distributed systems. The author's protocols in this case are capable of solving an average consensus problem by analysing nonlinear protocols for constructed disagreement and minimizing the costs of the distributed system. Meantime, Jadbabaie et al. [22] provided a theoretical explanation of linear consensus for Vicsek's work in flocking behaviour cohesion and speed where it is determined in both cases, and observed in a convergence test. Moreover, Olfati-Saber and Murray [29] presented a consensus protocol and established the convergence analysis for fixed/switching topology of directed and undirected networks with time-delays communication. The authors proposed a consensus algorithm that operates in such a way that, each agent combines its value linearly with the neighbouring agent's values received in a varying time manner. In another study conducted for vehicles in place of agents as an application area, a linear system was proposed for the cooperation problem among a collection of vehicles. The vehicles in this setup shared tasks and communicated amongst each other to coordinate their actions. In the process, an approach of decentralized information exchange between the vehicles was proposed to achieve a dynamic system. The dynamic system in this scenario provides each vehicle with a common reference to use for their related cooperative movements [63]. In their study, Lin et al. [64] studied a linear consensus control to coordinate groups of several autonomous agents to converge at the same point, where each agent relied on the information available at the local level. Blondel et al. [65] in turn, introduced an agreement algorithm of synchronism without the communication delays, where each agent starts with a scalar value updated by the linear equation with the stochastic matrix in discrete time. What is special about this model is that it is symmetric, whenever an agent communicates with a neighbour, then the neighbour gets as well similar communication in a reciprocating manner. Furthermore, Olfati-Saber et al. [66] have provided an analysis of a theoretical framework for linear consensus algorithm in MAS with a directed dynamic topology network. It has been explained that there are applications of flocking, formation control, fast consensus in small-world networks, Markov processes, load balancing in networks and distributed sensor in sensor networks. Another study by Lin and Jia [67] investigated a linear consensus for multi-agent networks based on the Lyapunov-Krasovskii theory with the sufficient conditions derived to guarantee the average consensus. An alternative view was given based on a stochastic matrix graph in linear control for consensus problems considered in dynamically changing environments. This view was presented by Cao et al. [30] who also considered another modified control of the Vicsek problem wherein each agent updates regardless of the position in a time controlled by its own clock. Nedić and Ozdaglar [68] provided a sub-gradient method for solving optimization of a distributed linear computation model. In this method, each agent minimizes its own objective function while its information is updated locally with other agents in a timevarying manner. In particular, it is used in two models: an information exchange model to describe the evolution of the information of agents and an optimization model to minimize own local objectives. The objective of such models is to achieve the optimal solution. However, the doubly stochastic matrix is one way to reach a consensus on the optimal solution of the problem. Same authors [69] have studied the resolution of a consensus problem for a distributed system of agents in time-varying scenario with delay and by way of an algorithm that operates asynchronously. The proposed analysis reduced the consensus problem with delay to a problem without delay by including new agents for each delay element. Moreover, the algorithm for convergence of the consensus problem was used to show the evaluation of agents via linear dynamic model of stochastic matrix. Olshevsky [70] has shown existing results closely related to the consensus problem by introducing an averaging linear algorithm. This was further advanced in [71] where it was mentioned that the averaging problem could be solved with the appropriate assumptions by iterative algorithms of the form of a linear model. Moreover, Dal Col et al. [72] designed a dynamic linear feedback control to guarantee a local convergence rate for consensus problem in MAS through linear matrix inequality.
The applications and motivations of the consensus problem and averaging algorithms have been amply explained in the literature. In particular, Olfati-Saber and Murray [29] and Xiao and Boyd [73] used doubly stochastic matrices to briefly compare the two methods. The outcome was that doubly stochastic matrices gave rise directly to an averaging algorithm. Moreover, what is important is the view that the doubly stochastic matrix is in agreement with the averaging algorithm. However, it was established that convergence of agents to a common opinion in a convex combination of the initial opinions is needed for the matrix to be stochastic and reaching a consensus problem for a distributed system of agents in time-varying with delay. The robust linear consensus protocols mentioned in [74] for the synchronization problem of multi-agent networks under the transfer matrices in additive uncertain perturbations is bounded. In [75], a linear model reduction scheme was derived for synchronization and stabilization of MAS in which the dynamic range of the agent is reduced while the graphics are unchanged and communication is established. Hendrickx et al. [32] discussed the possibility of reaching a consensus in finite time via linear iterations where the transition matrix should be a stochastic one with a positive diagonal. The positive diagonal means that when agents update their states they send positive values to them-selves. In [31], a linear distributed protocol of first-order and secondorder integrals for MAS with communication noises was studied. Hendrickx et al. [32] discussed a consensus reached in finite time by linear iterations, with additional restrictions to be updated by stochastic matrices with positive diagonal and consistent under a certain graphic structure. Hu et al. [33] studied general linear dynamics control for consensus problem in MAS by a novel scheme of the event-triggered scenario with some required distributed features.
Many different models have been proposed to achieve the agreement of autonomous agents. A new control algorithm was tested for controlling the sensors and processes in UAV by Jaimes et al. [76]. Chan and Ning [77] achieved the consensus convergence in dynamic networks where each time step is updated by the node with the neighbour's old values. The results of the authors' experimental investigation on dynamic network observed fast convergence behaviour. The consensus problem for MAS based on impulsive differential equations is studied in [78]. It has been concluded that the convergence speed of the proposed algorithm by impulsive systems is faster than the standard consensus algorithm. The improvement of communication delay and the convergence speed for consensus protocol are achieved by the weighted average prediction in [79]. Moreover, LeBlanc et al. [80] have provided an approach for reaching consensus of normal nodes, although the influence of malicious nodes under different assumptions is a threat. These conditions are applied for robust network of a novel graph-theoretic approach. The exact rate of convergence is produced in [81] for the average agreement by using stochastic matrices in probability. The analysis of this convergence is based on the connection of random matrices. A basic theory is reviewed for random consensus dynamics to be applied in many applications [18]. A distributed dual averaging algorithm is proposed by Wang et al. [82] to solve the cooperative optimization problem encountered in a computational multi-agent network with delay. The convergence of stochastic optimization in a numerical result of this algorithm is investigated. In [83], the equilibrium of initial opinions of the agents is examined based on local interactions, network structure, initial opinions and the extent of agents stubbornness. The fast time convergence of the function to achieve the equilibrium has also been studied. In the work of Priolo et al. [84], a novel distributed algorithm is addressed to solve the average consensus problem of discrete-time over any strongly connected weighted digraph. The scenario here is more general where the communication between agents is depicted on a directed graph. Lin and Ren [31] studied the problem of stress consensus in MAS to dynamically change asymmetric networks with communication delays. On this basis, it was also demonstrated that the original system, finally reached a consensus asymptotically even if communication delays were defined arbitrarily. Ren et al. [10] conducted earlier, a very significant survey of consensus problems in multi-agent cooperative control and indicated the dynamic exchange information between agents for consensus under time-invariant environment. Furthermore, it was a classification of applications of consensus seeking multi-agents' systems. One of the most interesting open problems proposed for further research was on the complications of nonlinear dynamics for a team of agents.
Linear consensus has some limitations in physical systems, where it is not that their behaviour can be changed via an unbounded value [34,36], and where the nonlinear protocol consensus has improved the performance of dynamic consensus algorithm as well as linear consensus protocols [35]. The fast reaching to a consensus by nonlinear control has been proved in [85]. There has been much motivation to design a nonlinear protocol for consensus problem in MAS. The early research in nonlinear control theory of the system's stability was constructive [40]. Olfati-Saber and Murray [41] provided linear and nonlinear protocols to agree in distributed and cooperative systems. The analysis of nonlinear consensus protocols has to be considered in a case where the applications of dynamic agents are physical models to take the input constraints into account. The nonlinear framework of discrete time in this case was approached by Moreau [43]. Meanwhile, Lin et al. [39] proved that the consensus of nonlinear subsystems can be achieved only when agents have a connection with sufficient dynamic interactions. The idea of this approach is that, the consensus is achieved when each agent interacts with the set of its neighbours of the convex hull for all iterations. The existing literature shows that the number of research works that have studied the nonlinear update rules for the consensus problem is only a few. The nonlinear system is one of the most difficult challenges in the study of the consensus problem of a static graph for the nodes [39]. An algorithm based on nonlinear update protocol has been presented in the research of Georgopoulos and Hasler [86]. It has been proven that the fast convergence rate is exploited by using a nonlinear function. Furthermore, the overall result of this nonlinear algorithm has better performance against the linear model. The new technique via nonlinear dynamics was established by Yu et al. [44] for the consensus problem of cooperative agents in a given network. The nonlinear class of discrete-time dynamic algorithms [47] was studied to reach an agreement for a group of agents from their initial values. It is necessary to investigate the stability of an average consensus of algorithms in this case, which has a strong connection with agents over a bounded period.
In [2], a novel nonlinear approach was then developed to solve the problem of average consensus. The numerical simulations of this novel algorithm achieved the best convergence behaviour in the dynamic scenarios. The necessary and sufficient conditions play a big role to achieve a convergence of problem consensus. All researches that have studied the convergence of consensus problem have relied upon suggested sufficient conditions. The stochastic system also improves the speed of convergence. The properties of linear convergence of consensus problem are used for a communication network modelled by a random graph based on a Markov process [87]. The related work has provided mathematical proof on necessary and sufficient conditions for achieving an average consensus. The mathematical techniques use the theory of the stability of the Markov systems, and together with the results of matrix theory and the graph data structure can be used to test the results of convergence for consensus problem in a stochastic framework. Meng et al. [48] obtained an iterative learning control algorithm by an agent's interaction with the neighbours and using stochastic matrices to reach a convergence for nonlinear MAS. Further consideration include the nonlinear operator that is represented in [45] for convergence of the individual agents in network connectivity. The novel algorithm and convergence results for distributed consensus are introduced in this work to reach a common value. A generic consensus condition with nonlinear agent dynamic was designed for consensus problem in MAS. The approach uses optimistic optimization to control the behaviour of agents and simplify a black box with unknown nonlinearities in mathematical form [49].
Additionally, nonlinear rules for consensus based on the traditional Lyapunov function have been evaluated in many researches. Olfati-Saber and Murray [41] introduced linear and nonlinear consensus protocols for networks of dynamic agents that allow the agents to agree in a distributed and cooperative fashion. The consideration is done for the cases of networks with communication time-delays and channels that have filtering effects. Hayashi et al. [88] described a problem of consensus in discrete time nonlinear functions. This is done based on interpretation of the dynamic evolution of communication topologies where; each agent has a value of performance based on their status inside information exchange and the value of performance with other agents. This in turn makes it possible to reach a consensus with sufficient conditions for a global consensus while utilizing the theory of algebraic graph. Hui and Haddad [46] designed nonlinear protocols that guarantee convergence for consensus problems for multi-agent dynamic systems and the stable equilibrium of Lyapunov is addressed as well in the process. In [36], a protocol on nonlinear dynamic network consensus control MAS with fixed and switching topologies was discussed. The authors' discussion was based on the technical base reduction centres collector where, it is shown that a group of agents can reach a consensus value which is the value of the group decisions ranging from the maximum values of the initial states of the agents to minimal values. Li et al. [35] introduced a nonlinear feedback control protocol to coordinate the value related to consensus of MAS for the internal information state so as to reach consensus based on Lyapunov theory. In [89], a mathematical analysis by nonlinearities of multiple random controllers for distributed synchronization principles of multi-agent networks was given and had a significant impact on the convergence speed and force control terminal using the Lyapunov function technique. The nonlinear dynamic here is derived by algebraic theory, matrix theory and Lyapunov control approach. In the case of nonlinear models, the averaging algorithms work with coefficients update dynamically based on the evolution process for the network [19]. Zhang et al. [90] on the other hand, studied the synchronization problem in a class of nonlinear dynamic networks with heterogeneous pulse delay for MAS by means of Lyapunov function.
Otherwise, nonlinear consensus under first and second orders has been investigated in the context of agents' reaching a consensus based on complex protocol computation. Bauso et al. [42] proposed a stationary distributed non-linear protocol that reaches consensus using some simple first-order dynamics in making the problem of the agents' states to converge on a decision value of interest. Yu et al. [44] later on, considered a nonlinear dynamic for second-order consensus problem of MAS where the problem refers to how the group of agents reaches an agreement. This protocol method is controlled by second-order dynamics such as the direction and speed. A connectivity-preserving of nonlinear second-order consensus algorithm in multiple dynamic mobile agents was investigated in [91]. The investigation was done by assuming that the initial network is connected and provides local adaptation strategies for both the weights of the force feedback speed, browsing speed, and coupling, allowing all agents to be synchronized with the virtual leader, even if only officially reported, without requiring knowledge of the dynamics of the agent. Andreasson et al. [92] considered a kind of consensus protocol on nonlinear dynamics of first-order and second-order with sufficient conditions for sufficient consensus, which were calculated for stationary communication topologies in the dynamics of single and double integrators. Zheng and Wang [93] on the other hand, introduced a nonlinear consensus of first order and second order with fixed topology for consensus problem in MAS utilizing nonnegative matrix theory of the graph theory. In [94], the problem of regulation of production cooperation for a class of heterogeneous multi-agent second-order uncertain nonlinear systems was studied through greater state law with distributed feedback control, which led to the solution of the original problem of consensus for MAS. Nonlinear consensus algorithms for multi-agent networks with a higherorder dynamic agent variable topology in time were studied in [95] where the conditions of convergence of these algorithms were obtained by the Kalman-Yakubovich Popov lemma and technical absolute stability. Li et al. [96] meanwhile, described the second-order dynamic random consensus on directed switched networks where the theoretical results are limited only to local consensus, although several sufficient conditions were established to reach consensus for second local order and are derived for the case without delay time. The inherent nonlinear dynamic systems were examined by Liu et al. [97] for the problem of multi-agent consensus under first-order and second-order systems with conditions for sufficient feedback gains been given on the basis of a Lyapunov function method. Yu and Long [98] in turn, examined the distributed consensus problem of finite-time of MAS in second-order consensus with the presence of bounded disturbances where the protocol of integral sliding mode was developed to achieve exact finite-time consensus despite interruptions. In another study by Feng et al. [99], the consensus values of all agents was established via a proposed nonlinear consensus protocol of first-order and secondorder dynamic systems with necessary and sufficient conditions based on the matrix theory for consensus problem in the context of heterogeneous MAS. Zhao and Jia [100] on the other hand, studied the problem related to consensus probability of multi-agent secondorder stochastic finite time with non-identical nonlinear dynamics setup. Their study was based on the theorems of finite time stochastic stability and adding an integrator technical power where the control algorithm is distributed for multi-agent stochastic that can ensure all agents converge to a consensus in finite time systems. In most recent work of [101], the feedback control of nonlinear dynamics consensus protocol of secondorder was designed for MAS under the cases of strongly connected networks using local and available sampling data for the update managers. On other hand, Macellari et al. [102] have addressed second-order average consensus for a group of double-integrator agents with prescribed transient behaviour. Nevertheless, a fractional calculus order was derived also from the consensus problem in MAS from an earlier work. In this case, a nonlinear model of a general fractional-order coordination was derived and sufficient conditions of the interaction among agents were derived to guarantee consensus coordination [103].

The nonlinear stochastic control for consensus problem in MAS
The consensus protocol for MAS in this study is traced back to the theories of dimensional simplex, doubly stochastic matrix, majorization concept, quadratic stochastic operators (QSO), doubly stochastic quadratic operators (DSQO) and extreme doubly stochastic quadratic operators (EDSQO). The concept of these theories will be presented as follows.

Dimensional simplex
A simplex is defined as the set of points, comprising the convex hull of a set of linear independent points. The (m − 1)-dimensional simplex is defined (see Figure 1) as follows [104,105].

Majorization
When considering two vectors say x and y, then the term majorization is closely depicted by the fuzzy notion relating the components of the two vectors. That is, y majorizes x entails the components of the x-vector with "less diffusion" or "closer equal" to the components of the y-vector. For that, the appropriate accurate statement can be addressed which is "x is majorized by y" and written as x ≺ y. The history of majorization is appropriately referred back to the works of [106][107][108][109][110]. Such works introduced the type of theory which is very important from the economic standpoint, leading to shortcomings in the distribution of income. There are also many important contributions which were established by other researches. In particular, Ando [111] provided a survey of the generalization on majorization and doubly stochastic matrix from various point of views. Olkin and Marshall [112] presented a wide discussion of the majorization theory and its related applications. In [113,114], the authors explored and examined the theory of majorization for QSO to produce simpler and desirable nonlinear operators of QSO called DSQO.
The concept of majorization is the extension of inequalities. It is obvious that the elementary inequalities can be formed as follows: (y 1 , . . . , y n ), where x 1 , . . . , x n is required not for all components being equal, and that is for it only be "less spread out" (y 1 , . . . , y n ).
For examples: Furthermore, one of the best methods for solving the optimization problem is the greedy method. The greedy method has provided an algorithm to rate the optimal solution [115]. Unfortunately, the greedy method was not generalized to deal with all the optimization problems. The motivation of using the concept of majorization in the optimization solution stems from the work of Parker and Ram [116] who have shown that the majorization theory can be related to the greedy method in immediate direction of the related problem solution. Conceptually, in the greedy method, solution for the given problem is solved by repeating the selection of the best possible exchange. In the majorization concept however, the approach solves the optimization problem with respect to ≺ operator, where the starting point is with the preordering and followed up with the related sorting of elements through the appropriate exchanges in the vectors [116]. All the formulations given above have used a linear operator. In the case of DSQO and EDSQO on the other hand, the approach has considered the majorization concept as a nonlinear operator.

Stochastic matrix and doubly stochastic matrix
A stochastic matrix is defined as a square matrix P = (a ij )where all entries (a ij ) are nonnegative and representing some probabilities. However, the matrix is called doubly stochastic if all its entry elements are nonnegative and each column or each row sums up to one [112]. It is also termed as probability matrix, transition matrix, stationary matrix, substitution matrix, or Markov matrix.
The conditions of stochastic matrix and doubly stochastic are: The following is an example for stochastic matrix P 1 : Examples of doubly stochastic matrices can be given by P 2 and P 3 as follows:

Quadratic stochastic operators
The history of QSO can be attributed to [117]. The main result of QSO on 2DS was investigated by [118].
However, the study of QSO on 1DS was completed by [119]. Further exploration on the subject was then carried out by [120] who studied the dynamics of QSO on infinite-dimensional simplex. There are various classes of QSO which have already been addressed in the literature such as volterra, dissipative and doubly QSO [121]. Moreover, a new subclass of QSO has been investigated in the work of [122]. The QSO can be interpreted as an operator of population evolution, which is considered for m species and the x 0 i = (x 0 i , . . . , x 0 m ) as the probability of individual's distribution for the initial generations, where the 0'z is the initial generation and the 'i' represents the number of distribution. There is a probability of interbreed P ij,k between the probability of individual distribution of x 0 i from ith to jth to produce an individual k.
To produce the state of the first generation i → x i can be defined by the following evolution operator V [121,123]: The description of this equation for MAS in this sense means that the x k represent agents (x 1 , where kis the number of agents. Nevertheless, P ij,k is the distribution matrix which portrays the interactions among the agents from i to j the k identify the respective agent's matrix. The dimension or simply the size of the matrix is given by m. Finally, the x 0 i and x 0 j are the states of the agent starting from the initial state 0, where i and j are the states of the agents in row and column cases, respectively, as shown in the following equation. The symbol of ( ) in x stands for the next state for x, where it has the same meaning of x t+1 = x t .
The state of agents starts with a random initial state x 0 i , which is then updated to the generation x i = V(x 0 i ), followed by the next generation x i = V(x i ), then to x i = x (Vx i ), and the sequence of updates continues as appropriately. The V operator is as depicted in Equation (1) and called stochastic quadratic operator (QSO). This exchanging of states is referred to as a dynamic system which and described as follows: where the process of the dynamic system of V n ( indicates the n times iteration of the evolution operator . . . In [121], it was observed that the main problem for the dynamic system of Equation (3) is to study the limit behaviour of the trajectories of QSO, specifically the limit defined by lim 0→∞ V n (x 0 i ). The distribution of the interactions among the agents is described in matrix case, where the matrix of P ij,k is distributed to k matrices (p ij,1 , p ij,2 , . . . , p ij,k ), and under the following conditions of the matrices: (1) The matrices are square . . . Then using the QSO model, the limit behaviour of the trajectories of the three agents converges to the centre 1 3 (as seen in Figure 2). The number of iterations to reach a consensus in this case, has been achieved through four iterations.
The convergence of the QSO model goes to the centre (average) due to the fact that the sum of all the distributed matrices p ij,1 , p ij,2 , p ij,3 is a matrix having all of its elements equal to one.

Doubly stochastic quadratic operators
The class of DSQO can be traced back to [120]. It was called the class of bistochastic quadratic operators, where the theorem with necessary and sufficient conditions were proved for this class. This theorem was also independently obtained in [124,125]. The concept of DSQO is associated to the notion of majorization [112]. The DSQO method of approach has been applied for the problem in population genetics [120], and is a sub-class of QSO, where the difference between them is that the DSQOs are defined by the matrices under the majorization concept. The matrices by the notion of majorization are referred to as the welfare operators, and were applied for specific problems in economics [112]. The completed works have been studied on the DSQO on finit diminsion (FD) in [113,[126][127][128].
The distinction between QSO and DSQO can also be made in that the DSQO refers to the theory of QSO and including together the concept of majorization theory, which is written as Vx ≺ x. The given condition is defined and interpreted in such a way that the QSOs are called DSQOs if one necessary condition is satisfied as follows: If an operator is DSQO, then its matrices must belong to the set U. So in FD (where F = (m − 1)), the model will have m matrices each of size m × m and satisfying the condition (on set U) given by Equation (4) [120]: where a ij are the elements of the matrix, i the number of row, j the number of column, and ∝ ⊂ I = {1, 2, . . . , m}.
In addition to Equation (4), the matrices should also satisfy the following conditions: (1) All elements of each matrix are from 0 to 1.
(2) Each matrix is symmetric. Here, using DSQO then the limit behaviour of the trajectories of the three agents converges to the centre 1 3 (see Figure 3). The number of iterations to reach a consensus is achieved through 4 iterations also. The convergence of DSQO goes to the centre (average) because the sum of all distributed the matrices p ij,1 , p ij,2 , p ij,3 is a matrix having all the elements equal to one as well. The difference between DSQO and QSO is that the matrices of the DSQO are under the majorization condition which is i,j∈α a ij ≤ |α| for α ⊂ I = {1, 2, . . . , m}.

Extreme doubly stochastic quadratic operators
Ganikhodzhaev [120] has investigated a sub-class of DSQO called EXTREME DSQO and abbreviated as EDSQO. The EDSQO is a sub nonlinear mathematical model that belongs to a theory of DSQO and QSO. In other words, EDSQO is a subclass of DSQO, and DSQO is a subclass of QSO. EDSQO is a new class which is based on majorization concept with additional conditions to the concept of DSQO. The concept of EDSQO is referred to stochastic analysis, matrix theory and graph theory. It is based on the fact that the set of DSQO form a polyhedron, in such a way that each DSQO may be viewed or interpreted as a point in some dimensional space. All these points will give a polyhedron geometrically, that consists of vertices which are considered exactly as extreme DSQO. That is a geometric meaning of extreme, and the reason for their reference as extreme is simply due to the fact that they are vertices. The concept of EDSQO is also presented in the works of [108,117]. More specifically, necessary and sufficient conditions for EDSQO were introduced by  [114], and at the same time leaving a number of related open problems. One of the open problems in EDSQO is regarding the limit behaviour of the underlying trajectories which is also considered as a central problem. EDSQOs on 2D have been completely defined in [129]. Both EDSQO and DSQO must satisfy Vx ≺ x.
The DSQO is called EDSQO if its matrices belong to the set of ExtrU defined as follows: So in FD (where F = (m − 1)), there will be m matrices each of size m × m and satisfying the following conditions: (1) The elements of each diagonal matrix are either 0 or 1 (0 ∨ 1). (2) The elements of each non-diagonal matrix are 0 or Here, using EDSQO then the limit behaviour of the trajectories of the three agents converges to the centre 1 3 (see Figure 4). The number of iterations to reach a consensus is achieved through 12 iterations. Likewise, the convergence of EDSQO goes to the centre (average) since the sum of all the distributed matrices p ij,1 , p ij,2 , p ij,3 is a matrix with all the elements equal to one. The difference between EDSQO compared to DSQO and QSO is that all the interior elements of the distributed matrices are 1or 1 2 or 0, which make the coefficients of the elements x i equal to one leading to easier calculations.

Conclusion and future work
In this paper, the resent literature of consensus problem has been reviewed. Since most of the researches on consensus problem are ongoing, this survey focused on linear and nonlinear consensus for MAS. In such recent works, most consensus problems in MAS have been considered in the framework of linear dynamics model. Some results of linear dynamics model imply that they can be extended by nonlinear dynamics model. As the outcome, the framework of nonlinear stochastic dynamics consensus can be applied to centralized robot, spacecraft and UAV formation flying scenarios. The study of the consensus problem for the team of agents through complex nonlinear dynamics and in the case of heterogeneous agents is a motivating issue for future research. Majority of researches on the problems of consensus expected that the status of final achieved consensus is intrinsically constant, which may not be the case in the sense that the status information of each agent can be dynamically changing over time as an inherent dynamic. It will be interesting to investigate a new nonlinear model so as to study the consensus problem as the final consensus status changes with time. In the present reality, most researches have considered the linear model and complex nonlinear model. This serves as a motivating factor) for evolving the solution into a low complexity nonlinear model for the consensus problem in MAS. Furthermore, most works are concentrated on the theoretical study of the consensus problems and most results are verified by simulations with the exception of some experimental results on the coordination of MAS communicated in invariant time. The implementation application of consensus plan for MAS is a main section of research in the future. In addition, topics such as disorders, time delay and noise of sensor models should also be considered.

Disclosure statement
No potential conflict of interest was reported by the authors.

Funding
We would like to thank Malaysia-Japan International Institute of Technology, University Technology Malaysia KL Campus, for the support. This work was supported by MJIIT UTM Research Grant Project R.K430000.7743.4J009.