Reproduction operators in solving LABS problem using EMAS meta-heuristic with various local optimization techniques

ABSTRACT Agent-based evolutionary, computational systems have been proven to be an efficient concept for solving complex computational problems. This paper is an extension of [Biełaszek, S., Piętak, K., & Kisiel-Dorohinicki, M. (2021). New extensions of reproduction operators in solving LABS problem using EMAS meta-heuristic. Springer, cop. 2021. – Lecture Notes in Artificial Intelligence, Computational collective intelligence 12876 304-316. 13th International Conference, ICCCI 2021: Rhodes, Greece, September 29ŰOctober 1, 2021.] where we proposed new variants of reproduction operators together with new heuristics for the generation of initial population, dedicated to LABS – a hard discrete optimization problem. In this research, we verify if the proposed recombination operators improve EMAS efficiency also with different local optimization techniques such as Tabu Search and Self-avoiding walk, and therefore can be seen as better recombination operators dedicated to LABS problem in general. This paper recalls the definition of new recombination variants dedicated to LABS and verify if they can be successfully used in many different evolutionary configurations.


Introduction
This paper concentrates on improving agent-based evolutionary computational systems for solving the low autocorrelation binary sequence problem (LABS). In Biełaszek et al. (2021), we introduced new reproduction operators as well as improvements in the process of initial population generation. The proposed improvements increased the effectiveness of the computational system with chosen local optimization algorithm. The goal of this paper is to verify if the proposed operators increase also the effectiveness of evolutionary computations with other local optimizations algorithms such as TabuSearch or Self-avoiding Walk and then can be seen as promising operators dedicated to the LABS problem.
First, let us introduce shortly the low autocorrelation binary sequence problem, which is one of the hard discrete problems despite wide research, remains an open optimization problem for long sequences. It belongs to CSPLIB library, which consists problems that 'pose a significant challenge to local search methods'. It has also a wide range of applications including communication engineering (Zeng et al., 2020;Zhao et al., 2017), statistical mechanics (Bernasconi, 1987;Leukhin & Potekhin, 2015, may) and mathematics (Günther & Schmidt, 2016;Jedwab et al., 2013).
LABS (Golay, 1977) consists of finding a binary sequence S = {s 0 , s 1 , . . . , s L−1 } with length L, where s i [ { − 1, 1}, which minimizes energy function E(S): MJ Golay also defined a so-called merit factor (Golay, 1982), which binds the LABS energy level to the length of a given sequence: The search space for the problem with length L has the size 2 L (Militzer et al., 1998) and the energy and merit factor of a sequence can be computed in time O(L 2 ). The LABS problem has no constraints, so S can be represented naturally as an array of binary values. One of the reasons for high complexity of the problem is, that in LABS, all sequence elements are correlated. One change that improves some C i (S) has also an impact on many other C j (S) and can lead to significant changes of solution's energy.
There is a lot of various techniques that try to solve the problem. The simplest method of solving LABS is exhaustive enumeration that provides the best results, but can be applied only to small values of L. There are also a lot of various heuristic algorithms that use some plausible rules to locate good sequences more quickly. A well-known method for such techniques is steepest descend local search (SDLS) (Bartholomew-Biggs, 2008) or tabu search (Gallardo et al., 2009). In recent years, a few modern solvers based on the self-avoiding walk concept have been proposed. The most promising solvers are lssOrel (Bošković et al., 2014) and xLostavka (Brest & Bošković, 2018), which are successfully used for finding skew-symmetric sequences of lengths between 301 and 401 (Brest & Bošković, 2020). These techniques can be also parallelized utilizing GPGPU architectures what was shown in Pietak et al. (2019); Żurek et al. (2017).
Local optimization techniques are usually simple and effective methods for solving LABS problem, however, they can easily get stuck in local optima. Therefore, some of them introduce various caching mechanisms that allow to block parts of or whole sequences explored in the previous steps what gives very good results. As an extension of this approach, combining them with other heuristics has been proposed to balance the exploitation and exploration of the solution space. Random generated sequences as a starting point for each run of local optimization method have been replaced by a global knowledge that allows to manage the optimization process.
A good example of such an approach is memetic algorithms that combine various evolutionary algorithms with local optimization techniques. Here, a promising direction of research is using agent-based biologically inspired computational systems. One of such meta-heuristics is the concept of an evolutionary multi-agent system (EMAS) proposed by K. Cetnarowicz (Cetnarowicz et al., 1996) and successfully applied for solving complex problems. EMAS has already been used to optimize some complex continues and discrete problems such as LABS (Kowol et al., 2017;Pietak et al., 2019;Żurek et al., 2017), TSP Dreżewski, Woźniak, et al. (2009) or Investment Strategies Generation (Drezewski, Sepielak, et al., 2009).
In case of LABS, EMAS system generates an initial population of binary sequences. During the main phase, individuals are paired and depending on their energy level, they either fight (if energy level is low) or reproduce (if energy level is above a certain threshold). During the fight, energy is transferred from one individual (usually the better one) to another. In contrast, during the reproduction the sequences are crossed, using Uniform or Two-point recombination, mutated and the descendants receive some of their parents' energy. In memetic version, new individuals are further optimized with local search techniques. At last, individuals with very low energy are dying which means they are removed from the population.
In such context, the key issue is adjusting evolutionary operators in EMAS to LABS problem to find a proper balance between diversity of population and exploitation when working various local optimization techniques. In Biełaszek et al. (2021), we proposed a few recombination operators that gave a promising result when combined with SDLS, a now, in this paper we verify if they also improve LABS optimization when combining with Tabu Search and Self-avoiding walk.
This paper is organized as follows. In the following section, the evolutionary multiagent system together with local optimization techniques for LABS are described. In the third section, new evolutionary operators are reminded, and then the experimental results together with conclusions are drawn in the following section. This paper is summarized in the last section where also future work is indicated.

Evolutionary multi-agent systems for discrete optimization problems
Let us first introduce evolutionary multi-agent systems that are composed of communicating and cooperating agents pursuing common goals. An agent is an autonomous unit that perceives the environment and somehow interacts with it, having a certain amount of energy that it exchanges with the environment and other agents during fight, reproduction or death. The level of agents energy is related to its quality, is evaluated by the fitness function and determines the agent's ability to reproduce or fight. In optimization problems, the individuals contain a solution decoded in the shape of genotype.
In the memetic variant of EMAS, the evaluation can be further improved by utilizing various local optimization techniques such as the steepest descent local search, the tabu search, the self-avoiding walk and others.
An example of a hybrid environment in which we combine evolutionary computing with local optimization performed on GPGPU is described in Pietak et al. (2019). In this context, the development of evolutionary operators is important because it gives further opportunities to use them in hybrid environments. Figure 1 illustrates a single step of the memetic EMASsquares represent individuals gathered in a population and arrows depict the flow of the algorithm. The number of individuals in a population can vary from step to step as shown in Figure 1 there are two new individuals and only one is death (removed from the population). The global size of population is managed by energy resource that is constant for the whole population.

Structure of EMAS algorithm
Algorithm 1 A simplified algorithm of population processing in memetic EMAS 1: function PROCESS POPULATION (population) 2: pairs := selectPairs(population) 3: newBorns for pair in pairs do 5: if canReproduce(pair) then 6: newBorns+ = reproduce(pair) 7: else 8: fight(pair) 9: newBorns := evaluate(newBorn) 10: newBorns := improve(newBorn) 11: population + =newBorns 12: for ind in population do 13: if shouldDie(ind) then 14: remove(population, ind) return population A formal description of the memetic EMAS is presented in Algorithm 1. The processPopulation function contains a single 'step' that is called sequentially until a stop condition is reached. Within each step, individual agents try to meet other agents in a population and in this way a new set of pairs is formulated. During the 'meeting', agents in a pair can reproduce or fight, depending on the level of agents' energy. If a selected pair is 'good enough', then a new individual is 'born' from the selected parents. Otherwise, individuals from the pair fight with each other, and in consequence a portion of life energy is transferred between them. Next, all new born individuals are then evaluated and improved using local optimization techniques. Finally, at the end of the step, all weak individuals are removed from the population (death operation).
To use the EMAS for a problem optimization, a few elements dedicated for the problem are required to be delivered. First of all, a representation of a problem's solutions has to be declared. Based on that, adequate evolutionary operators such as mutation and crossover, as well as memetic and evaluation operators are needed. All of them have to be compliant (Piętak et al., 2009) and should be further adjusted by specifying proper values of their parameters.
The representation of the LABS problem is an array of binary values. At the beginning of a computation individuals are usually generated randomly. The fitness function for LABS is the energy (Equation 1) or merit factor (e.g. (2)) for a given sequence used to determine the quality of an individual in comparisons with other individuals in the population.

Local optimization heuristics
There is also a lot of local search heuristics that explore a neighbourhood of an individual to find better solution. In Biełaszek et al. (2021), we utilized steepest descent local search (SDLS), which is an algorithm belonging to the class of gradient methods. Its formal definition is presented in Algorithm 2.
Algorithm 2 Steepest Descent Local Search for LABS problem. Based on Gallardo et al. (2007). Symbols: San input sequence; Llength of the sequence; Fsequence evaluation(merit factor); S i /F ithe best sequence and its evaluation in a single iteration 1: function SDLS(S) 2: S best = S 3: F best = EVALUATE(S best ) 4: improvement = true 5: while improvement do 6: F i = −1 7: for j = 0 to L−1 do 8: S tmp = S best 9: S In this paper, we focus on two other local optimization algorithms, TabuSearch and Self-avoiding walk, based on a slightly different concept.
One of assumptions of first of them, proposed in Dotú and Van Hentenryck (2006), was complete reinitialization, if for a certain number of iterations in a row it was not possible to improve the evaluation. The method was modified in 2007 by Gallardo et al. (2007), using the idea of forbidden states during the execution of the algorithm. The detailed concept of Tabu Search is presented in Algorithm 3.
Algorithm 3 Local TABU Search optimization algorithm. Accepted symbols: Sinput sequence; Lsequence length; maxItersnumber of iterations of the algorithm; Fsequence evaluation, determined as the value of its merit factor; EVALUATE (S)a method that evaluates the sequence S; tabua vector that determines how long certain sequence elements are locked from modification; minTabu, extraTabuauxiliary integers, used to determine the value of the vector tabu; changedBitan index of the sequence element whose negation resulted in the greatest improvement evaluation in a given iteration 1: function TABUSEARCH (S, maxIters) 2: int[]tabu 3: minTabu = maxIters/10 4: extraTabu = maxIters/50 5: S start = S 6: S best = S 7: F best = EVALUATE(S best ) 8: for iteration = 0 to maxIters − 1 do 9: Fi = −1 10: for j = 0 to L − 1 do 11: S tmp = S start 12: S if F i . S best then 24: S best = S i 25: F best = F i 26: return S best The basic idea of the Tabu Search algorithm is to search space, made up of all possible solutions, using a sequence of movements. There are illegal movements, taboo movements, in the sequence of movements. Tabu Search, compared to its SDLS introduces a concept of 'locks' that block modification of the sequence elements at specific positions for a predefined number of iterations. For this purpose, an auxiliary table was used in which the iteration numbers to which the lock is active are stored (line 2). As a result, if we negate an element of the sequence, we will not be able to modify it again for several iterations. The algorithm avoids oscillations around the local optimum by storing information about already proven solutions in the form of a tabu list.
The Self-avoiding walk (SAW), developed by Bošković et al. (2014), is based on the concept of an indirect graph that passes through a given vertex no more than once. In the context of the LABS problem, the path is identified with the set of sequences analysed in the subsequent steps of the search algorithm as it is defined in Algorithm 4.
In the SAW algorithm, we see some analogies to local Tabu Search optimization. Both algorithms use a neighbourhood search and do not require improvement of the evaluation between successive iterations. They also prevent the situation in which the same sequence is analysed as in the previous step, using various types of blocking mechanisms for this purpose. However, there is a fundamental difference between these algorithmsin SAW, 'locks' are placed on entire sequences and expire only when the algorithm is reinitialized, while in Tabu Search only selected bits are prohibited, and the prohibition itself is valid only for a few iterations.
Algorithm 4 Local Self-Avoiding Walk optimization algorithm. Accepted symbols: Sinput sequence; Lsequence length; maxItersnumber of iterations of the algorithm; Fsequence evaluation, determined as the value of its merit factor; EVALUATE (S)a method that performs evaluation the S sequence; walkLista list of sequences that were chosen as the best in the next steps of the algorithm 1: functionSAWSEARCH (S, maxIters) 2: walkList = {S} 3: S i = S 4: S best = S 5: F best = EVALUATE(S best ) 6: for iteration = 0 to maxIters − 1 do 7: Fi = -∞ 8: for j = 0 to L−1 do 9: S tmp = S i 10: S

Extended recombination operators for LABS problem
In this section, we are reminding extensions of reproduction operators dedicated to LABS problem, presented in article for Biełaszek et al. (2021), that introduce additional modifications to the newly created solutions. Moreover, there are introduced improvements in initial population generation.
In the previous paper, we proposed four new recombination operators adapted to the LABS problem, which introduced additional modifications to the newly created sequence based on uniform ( Figure 2a) and two-point (Figure 2b) crossover results. We recall them briefly below and present visual examples in Figure 2.
The first, INV extension, assumes that the bit order of the entire sequence is reversed in the case of uniform recombination (Figure 2 c), and the bit order of the middle segment defined by the parental split points is reversed in the case of two-point recombination (Figure 2 d).
The second extension called SHIFT involves a cyclic shift of a specific bit length. We divide the result of the uniform recombination into two equal segments (if the result has an even length) or segments differing by 1 bit (if the result has an odd length), and then we swap them with each other (Figure 2 e). In the case of two-point recombination, the division and replacement of sections are performed in the middle section defined by the dividing points of the parent individuals (Figure 2(f)).
The negHB extension in the case of a two-point recombination is the negation of a randomly selected half of the bits of the longest segment from among the segments determined by the dividing points of the parent individuals (Figure 2 h). In the case of uniform recombination, we negate a randomly selected half of the bits of the entire sequence (Figure 2 g).
The last extension, negEv2, is defined only for two-point recombination. It negates every second bit of the middle segment defined by the dividing points of the parent individuals (Figure 2 i).

Experimental study
The goal of the carried experiments was to verify if the proposed recombination operators dedicated to LABS problem increase the efficiency of EMAS with other local optimization techniques such as Tabu Search and SAW.
The basis for EMAS configurations was set of parameters described in Kowol et al. (2017), however, we introduce modifications to energy settings and of course genetic operators. The starting energy of an individual in our research was set to 5, the reproductive energy was set to 7 and the energy transfer between fighting individuals was set to 1.
In EMAS, similarly to other population algorithms, the initial population is usually generated randomly. However, in case of LABS problem, starting computation from locally improved individuals seems to speed-up the whole process. Therefore, instead of randomly generated individuals, we put into a new population random solutions, each improved by SDLS algorithm. This algorithm, among other local search techniques for LABS, is a compromise between effectiveness and efficiency.
In EMAS, as well as other evolutionary algorithms, the mutation introduces with some low probability random changes to the genotype. The aim of this process is to introduce diversity to the population, i.e. to prevent premature convergence. However, as our crossover operators introduce additional randomness to the newly created solution, there is no need to apply standard mutation methods. Table 1 presents configurations of performed computations, including various recombination operators as well as local optimization techniques.
In our research sequences of length 201 were tested, we used the construction properties of skew-symmetric sequences in the calculations. An initial population size was set to 50. Each configuration has been run for 600 seconds and repeated 10 times. Values showed in the presented charts are the arithmetic mean of these repetitions.
All computations performed in this paper, have been implemented using AgE platform 1a Java-based solution developed as an open-source project by the Intelligent Information Systems Group of AGH-UST. AgE provides a platform for the development and execution of agent-based applications in mainly simulation and computational tasks Piętak et al., 2009). Computations were performed at a PC workstation with Windows 7 Intel Core i5-2520M 2.50 GHz, 8 GB RAM memory and using Intel HD Graphics 3000 graphic card. 2p Self-avoiding walk U-rndS-mutS(saws) U 2p-negEv2(saws) 2p+negEv2 2p-INV(saws) 2p+INV 2p-SHIFT(saws) 2p+SHIFT 2p-negHB(saws) 2p+negHB U-negHB(saws) U+negHB U-INV(saws) U+INV U-SHIFT(saws) U+SHIFT

Results
The efficiency of various configurations of EMAS for LABS is presented in Figures 3 and 4. The charts show how new recombination algorithms change the efficiency comparing to variants with standard 2p and uniform algorithms (the series with 2p-rndS-mutS and U-rndS-mutS labels).
The experimental results show that in the 2p recombination variants (Figures 3a and  4a) all proposed recombination operators allow to achieve better merit factor than the reference configuration. It is also worth mentioning that both SHIFT and negEv2 algorithm introduce significant improvement for Tabu Search as well as Self-avoiding walk.
In case of operators derived from uniform recombination (Figures 3b and 4b) the introduced modifications improve slightly the results, but the difference is not significant. In our previous research (Biełaszek et al., 2021) when SDLS algorithm was used as the local optimization operator, the best results were achieved by the 2p-negEv2, 2p-INV ant 2p-negEv2 variants. In case of Tabu Search and Self-avoiding walk, configurations 2p-negEv2 and 2p-SHIFT give the best results, significantly better than analogous configurations with standard two point recombination. This fact proves that this two proposed variants of recombination algorithms are good choice for LABS problem.

Summary of the experimental results
The main concern of this paper was to describe the impact of introduced changes on the effectiveness of solving LABS problems using the EMAS system. From the research that has been conducted, it is possible to conclude that the introduced operations improved the results for all local optimization techniques comparing to standard recombination operators.
Best result in whole tested group was achieved for two point recombination with negEv2 and SHIFT extensions. The number of experiments with different variants of EMAS, allows us to conclude that the proposed recombination operators are good choice for optimizing LABS problem with evolutionary techniques.

Conclusion and future work
In this article, we examined how the extensions to recombination operators dedicated to LABS problem in EMAS systems with SDLS local optimization, described in Biełaszek et al. (2021), behave for other operators of local optimization, such as SAW and TABU.
This extensions leads to better results comparing to the previous results and proves that the proposed operators are suitable for solving LABS problem. The results of presented experiments prove that the proposed concept speeds up computations and allows to get better solutions than the same version realized using standard EMAS operators.
The further work will be conducted to adjust and verify new extensions to GPGPU architecture to process LABS optimization in very efficient way. Note 1. Project homepage: https://gitlab.com/age-agh/age3 Notes on contributors Sylwia Biełaszek, PhD student at the AGH University of Science and Technology in Krakow. Her research focuses on evolutionary algorithms.
Kamil Piętak obtained Ph.D. in 2017 at AGH University of Science and Technology in Cracow and currently works as an assistant professor in the Institute of Computer Science of AGH-UST. His research interests include hybrid architecture for computational intelligence, multi-agent systems, biologically-inspired computing and other soft computing methods.
Marek Kisiel-Dorohinicki obtained Ph.D. in 2001 and D.Sc. in 2013, works at AGH University of Science and Technology in Krakow. His research focuses on intelligent software systems, particularly using agent technology and evolutionary algorithms. He works as a Full Professor in the Institute of Computer Science of AGH-UST.