Exploration strategies for balancing efficiency and comprehensibility in model checking with ant colony optimization

ABSTRACT Model checking is a formal and automated verification technique to show that a software system behaves in accordance with the given specification. Traditional model checking uses exhaustive search techniques for finding violative behaviours of the specification. The techniques, however, often do not work for huge systems because it demands a huge amount of computational resources. Search-Based Software Engineering is known to effectively solve many software engineering problems including model checking. It pursues the good balance between efficiency and qualities of solutions by using swarm intelligence and metaheuristic search methodologies. This article focuses on the state-of-the-art model checking with Ant Colony Optimization. Ant Colony Optimization is a metaheuristic, population-based and stochastic optimization algorithm. We propose two exploration strategies to further improve the balance in model checking based on Ant Colony Optimization. The proposed strategies introduce different kinds of randomized selection mechanisms to diversify solutions found by many agents. The strategies help the search algorithm extend the reachable regions effectively. Through numerical experiments, we confirmed that the proposed strategies require less computation time and memory as compared to the existing model checking with Ant Colony Optimization at the cost of finding slightly less qualified solutions.


Introduction
Search-Based Software Engineering (SBSE) (Harman et al., 2012) is a research field that aims to solve software engineering problems using search methodologies. The search methodologies contain Swarm Intelligence and metaheuristics, which are based on distributed, multi-agent and collectively intelligent approaches for finding better solutions efficiently. While SBSE is applied to many kinds of research topics, such as software design and refactoring, the most promising domain is formal verification.
Formal verification is a mathematical approach to prove or refute correctness of software systems. This article focuses on model checking, which verifies software systems' behavioural aspects automatically (Clarke & Emerson, 1982;Clarke et al., 2018). In model checking, the description of important behaviour of the target system is called a model. The desired property of the system is written as a specification. Model checking assumes that the model and the specification are given by the user. Then it decides whether the model satisfies the specification or not. The problem can be reduced to the exhaustive search problem over the state space constructed from the model and the specification. A model checking tool outputs a violative behaviour, i.e. a counterexample, if it concludes that the model does not satisfy the specification. There are two major research challenges of model checking: State Explosion Problem and the problem of Generating Comprehensible and Short Counterexamples. The state explosion is a problem about verification efficiency. State spaces tend to be huge. For example, the number of states in a concurrent and multiprocessing system often grows exponentially as the number of component processes increases linearly. The state explosion problem claims that huge state spaces make model checking infeasible in practice because the classical exhaustive algorithms deplete computational resources. On the other hand, generating comprehensible counterexamples is a usability problem. A counterexample provides diagnostic information that helps human users understand a specification's violation easily. Therefore, it is more favourable to obtain short counterexamples than to obtain lengthy ones. These problems are in a trade-off relationship. When we want a short counterexample, the exhaustive search is necessary but it suffers from the state explosion. Therefore, we need to pursue a search algorithm such that it achieves the best balance between efficiency and shortness of counterexamples.
In order to mitigate those problems and to attain an appropriate balance, researchers have actively developed non-exhaustive verification techniques using swarm intelligence and metaheuristics (Kumazawa et al., 2020). One of such algorithms is Ant Colony Optimization (ACO) (Dorigo & Stützle, 2004). ACO solves the shortest path search problem effectively. ACOhg (Alba & Chicano, 2007a) is a variant of ACO and is suitable for model checking with huge state spaces (Alba & Chicano, 2007b;Chicano & Alba, 2008a). Chicano and Alba (2008b) proposed ACOhg-live, which is a state-of-the-art model checking algorithm by running ACOhg at two phases.
In this article, we propose two novel exploration strategies with the aim of enhancing the performance of ACOhg-live. We call the two strategies the skip strategy and the replacement strategy, respectively. In general, many states should be explored to arrive at goal states for large state spaces. Based on the idea, the proposed strategies adopt simple randomized approaches to realize the effective and efficient exploration of many states. By conducting comparative experiments, we confirmed that the proposed strategies improve the balance between efficiency and shortness of counterexamples as compared to ACOhg-live.
The rest of this article is organized as follows. Section 2 explains the outlines of model checking, namely ACOhg, and ACOhg-live. Section 3 presents the review on previous researches. Section 4 proposes our exploration strategies. Section 5 reports the results of our comparative experiments between ACOhg-live and the proposed strategies. We conclude our discussion in Section 6. This article is an extension of the work originally presented in the Thirteenth International Conference on Computational Collective Intelligence (Kumazawa et al., 2021). The main extensions in this article are as follows.
. We add the state-of-the-art search techniques to the literature review (Section 3). . In Section 4.1, we newly introduce two variants of the skip strategy. The proposed variants employ the different stochastic path selection methods that are different from the selection method in the original skip strategy proposed in our previous work (Kumazawa et al., 2021). The performance evaluation of the proposed variants is added in Section 5. . We show the whole pseudocodes of the skip strategy (Algorithm 3) and the replacement strategy (Algorithm 4). . We add another experimental result using a benchmark of random graphs (Section 5.2.2). . We discuss our experiments in detail (Section 5.3). The newly added topics contain the analysis of the drawback found in our proposed strategies and the internal validity of our experiments.

Background
This section briefly introduces model checking and its automata-theoretic approach. We then present a model checking method based on Ant Colony Optimization.

Automata-theoretic model checking
Model checking is an automated approach to verify software systems in a formal fashion. It checks the behaviour of the target software system and determines whether it conforms to the predefined specifications. This article focuses on an automata-theoretic approach for model checking (Vardi & Wolper, 1986). This approach assumes that the state spaces are represented as finite automata. The model checking problem is reduced to the emptiness checking of the automata. Thus, the automata that are used in model checking are called Büchi automata. Just like a classical Nondeterministic Finite Automaton (NFA), a Büchi automaton consists of a set of states and a set of transition edges, a set of accepting states, and at least one initial state. A Büchi automaton can be represented as a directed graph. While an NFA accepts the finite words that reach some accepting state, a Büchi automaton recognizes the words of infinite length that visit some accepting state infinitely many times. In other words, the Büchi automaton accepts a word whose path, i.e. a sequence of states, is comprised of a cycle having some accepting states and a path from an initial state to the accepting states ( Figure 1). We indicate an initial state by an incoming edge without source state. A doubly concentric circle represents an accepting state. For example, consider a Büchi automaton in Figure 2. It has two states 0 and 1, where 0 is the initial state and 1 is the accepting state, respectively. It accepts words passing state 1 infinitely many times, e.g. 0, 1, 1, 1, . . .. Model checking is conducted in the following manner (Clarke et al., 2018). First, we construct a Büchi automaton that accepts the words conforming to the system model but contrary to the specification. When a specification is formulated with Linear Temporal Logic (Manna & Pnueli, 1992), its negation is transformed to the corresponding Büchi automaton (Clarke et al., 2018). The automaton constructed in this step is the intersection between the model and the automaton of the negated specification. Second, the emptiness of the Büchi automaton is examined with graph search algorithms. Classical model checking adopts exhaustive algorithms to find strongly connected components on directed graphs (Tarjan, 1972). If the Büchi automaton has an accepted word, the word is reported to the user as a counterexample. Otherwise, it concludes that the model satisfies the specification.

Model checking based on ant colony optimization
We introduce an automata-theoretic model checking technique using a variant of Ant Colony Optimization (ACO) (Dorigo & Stützle, 2004), called ACOhg (Alba & Chicano, 2007a, 2007b. ACO is motivated by foraging behaviour of worker ants. In ACO, many mobile agents (ants) search for paths over the given graph from an initial state (nest) to some goal state (food). They communicate with each other by means of pheromone deposited by the preceding agents. When each agent transits from a state to an adjacency, it prefers a state with denser pheromone trail to the ones with other pheromone trails. After the exploration, the agent puts the pheromone trail on its path. ACO converges to the shortest path since the denser pheromone trails a path has, the more likely it is selected by the succeeding agents. Pheromone trails evaporate as time elapses with the aim of avoiding local optima. ACO often suffers from state explosion for huge graphs because agents have to visit too many states. ACOhg is an extension of ACO specific to exploring huge graphs. In order to avoid depleting computational resources, ACOhg introduces the upper bound l ant to the length of the path that an agent traverses. Agents gradually go to distant states by updating their initial states to the ones that the preceding agents have reached on a regular basis.  Algorithm 1 is the pseudocode of ACOhg. In lines 1-3, an initial state and pheromone trails are initialized to random densities. Here, the term initial node indicates the initial state of the given Büchi automaton, i.e. the search space. After the initialization, the search process is repeated msteps times (line 6). The number of agents, i.e. the colony size, is denoted by colsize (line 7). In lines 9-13, each agent in the colony starts an initial state and keeps on moving over the state space until either the length of its path reaches l ant or it encounters a goal state. The terms p k , |p k | and p k −1 represent a path traversed by agent k, its length and its last state, respectively. T(i) is the set of successors of state i and F denotes a set of the goal states. At line 10, a successor of the state where agent k currently stays is selected. The probability p k ij where agent k in state i selects the adjacent state j is computed as follows: where t j is the density of pheromone deposited on state j, h ij is an appropriate heuristics, and α and β are the scaling parameters. The density of pheromone trails on the visited states are weakened at a constant rate ξ, i.e. t j = (1 − j)t j (line 12). After the traversal, the highly qualified paths are selected with the objective function f so that the shorter a path is, the better the value of f is (line 14). The function f for a path p k of agent k is defined as follows: Algorithm 1 ACOhg (cited from (Chicano & Alba, 2008b while step ≤ msteps do 7: for k 1 to colsize do 8: t local_pheromone_update (t, j, node); 13: end while 14: next_init where p k full is a path from the initial state of ACOhg to p k −1 , and h is a heuristic function. The parameters p p and p c are the scales of the penalties for paths that do not reach any goal states and paths having cycles, respectively. In lines 15-17, the best path having found thus far is selected. Pheromone trails evaporate at a constant rate ρ, i.e. t i = (1 − r)t i for each state i (line 19). The lower bound of the trails is controlled by the parameter a. The pheromone trails deposited on the best-so-far path are then intensified using Equation (2) (line 20). From line 21 to 25, the initial states are updated to the last states of the best paths selected at line 14. The number of the updated initial states is up to the value of the predefined parameter ι (Chicano & Alba, 2008a). ACOhg conducts this update procedure once every s s times, where s s is a parameter.
ACOhg-live (Chicano & Alba, 2008b) is a model checking algorithm that finds short counterexamples. It uses ACOhg at two phases: one for detecting a path from an initial state to some accepting states, and the other for detecting a cycle containing the accepting states. Algorithm 2 summarizes ACOhg-live. If ACOhg-live finds some accepting states at the first phase (line 2), it starts the second phase to search for the path from the states to themselves (lines 3-5). Once the second phase finds such a path, ACOhg-live stops and outputs the counterexample. Otherwise, the first phase is conducted again until it fails to find any accepting states (line 6).

Related work
Classical automata-theoretic model checking adopts exhaustive search algorithms based on Depth-First search (DFS), e.g. Nested DFS (Clarke et al., 2018) and Tarjan's algorithm (Tarjan, 1972). As discussed before in this article, it is difficult for such classical methods to generate short counterexamples due to the state explosion problem. Recent work attempts to fully utilize the power of multi-core processors (Zakharov, 2018) in order to enhance efficiency. Schuppan and Biere (2005) investigated short counterexamples generation from the viewpoint of automata construction at the cost of enlarging state spaces.
Another research direction of model checking is the use of heuristic searches with the aim of enhancing check efficiency. Directed Model Checking (Edelkamp et al., 2008) has investigated heuristics for model checking. Randomized algorithms are also proposed such as DFS with randomized backtracking (Parízek & Lhoták, 2019) and Vagabond algorithm (Daszczuk, 2021). Swarm verification parallelizes multiple search strategies including randomization (DeFrancisco et al., 2020;Holzmann et al., 2011).
Metaheuristic approaches for formal verification have been studied since 1990s. Genetic Algorithm (GA) is a pioneering method that has been studied intensively (Alba & Troya, 1996;Godefroid & Khurshid, 2004;Yousefian et al., 2014). Besides ACOhg, several techniques based on ACO also have been proposed (Francesca et al., 2011; Algorithm 2 ACOhg-live (Chicano & Alba, 2008b) Rebiha & Ciampaglia, 2007). Other researches employed Particle Swarm Optimization (PSO) (Ferreira et al., 2008), Estimation of Distribution Algorithm (Staunton & Clark, 2010, Bayesian Optimization Algorithm (Pira et al., 2017) and Monte Carlo Tree Search (Milewicz & Poulding, 2018;Poulding & Feldt, 2015). Chicano et al. (2011) employed Simulated Annealing (SA) and reported that metaheuristic methods outperform classical deterministic methods through their comparative experiments. In order to make model checking more efficient, several researches employed hybrid approaches, which combine more than one search method. Researchers have hybridized Gravitational Search and PSO (Rafe et al., 2015), Artificial Bee Colony algorithm and SA (Rezaee & Momeni, 2020), and ACOhg and a randomized backward search (Kumazawa et al., 2019(Kumazawa et al., , 2016. The researches discussed above present ample evidence that heuristic and metaheuristic approaches are effective in model checking. This article focuses on ACOhg with the aim of improving the balance between efficiency and comprehensibility in model checking.

Proposed exploration strategies
This section proposes two novel strategies to enhance the exploration capability of ACOhg-live. We call them the skip strategy and the replacement strategy, respectively. Note that the proposed strategies have no additional user-defined parameters.

Skip strategy
The first phase of ACOhg-live finds an accepting state in a Büchi automaton. However, when the automaton has several accepting states and some of them are on the same path, the first phase ACOhg can retard agents' exploring due to the intensive search within the small region. Since a Büchi automaton has more than one accepting state in general, this problem deteriorates the performance of ACOhg-live strikingly. Consider applying ACOhg-live to a Büchi automaton shown in Figure 3. Although the first phase finds a path from the initial state 0 to an accepting state 1, the consecutive second phase fails because state 1 is not a part of a strongly connected component. Thus, the first phase has to start from state 0 again to find the other accepting state 2. If ACOhg arrived at state 2 by further exploration, ACOhg-live would succeed in finding a cycle without running the first phase repetitively. The aforementioned problem becomes serious when ACOhg-live is applied to practical and industrial software verification problems. Models of such software systems tend to be very large and also have many accepting states. As a consequence, the first phase that has to explore the large state space may be repeated many times. This phenomenon has a negative impact on both search efficiency and generation of short counterexamples. We propose the skip strategy to address this problem. The idea of the strategy is that each agent keeps on moving until the length of its path reaches the upper bound l ant even when it encounters some goal state halfway. After it stops exploring, the strategy extracts one of the subpaths that ends at goal states randomly. Back to the example in Figure 3, when l ant . 2, the skip strategy enables the first phase ACOhg to reach both state 1 and 2.
Algorithm 3 shows the modified ACOhg with the skip strategy. At line 6, we add another termination condition such that the algorithm ends if there are one or more agents that arrive at one of the goal states. We call this termination condition Stopon-Arrival condition in the following. Stop-on-Arrival condition is equivalent to the one adopted in (Chicano & Alba, 2008a) and is necessary for the efficiency of the skip strategy. We then change the manner of exploration by each agent, as shown in lines 10-20. From line 10 to 17, an agent repeatedly transits states up to visiting l ant states. Whenever it comes across accepting states on its way, it ignores the states and keep on moving. We store a set of its subpaths whose last states are goal states. After the agent builds a path, a random selection is conducted to extract one of the stored subpaths (lines 18-20). The selection is performed in accordance with Uniform Distribution. We enumerate all of the nearby accepting states as candidates if the size of the colony is sufficiently large. The procedure below line 21 of Algorithm 3 is the same as that below line 14 of Algorithm 1. We can consider a variety of the skip strategy by changing the probability distribution used in random selection at line 19. In addition to the random path selection according to Uniform Distribution, we introduce the following weighted probability for the different path selection methods. In random selection, we first index the subpaths that are considered. Then, we select the i-th path from the subpaths according to the following probability.
where N is the size of the subpaths and i = 1, . . ., N. Using Equation (3), we propose two variants of the skip strategy whose manners of indexing the subpaths are as follows.
. Forward Indexing: We give the subpaths indices by the ascending order from the first detected subpath to the last. . Backward Indexing: We give the subpaths indices by the descending order. (3), the forward indexing prioritizes selecting the subpath that is discovered later. Thus, this indexing encourages the agents to explore deeply. In contrast, the backward indexing selects the subpath found earlier with high probability. This indexing makes the agents behave in the manner similar to ACOhg.

Replacement strategy
Agents in ACOhg need to extend their exploration and their paths to goal states when their preceding agents do not reach goal states. However, if the paths found by the preceding agents are evaluated well as compared to the ones found by the followers, it is likely that the already-known paths are selected many times at line 14 of Algorithm 1. This stagnation is unfavourable because the agents tend to explore some small region of the huge state space intensively and cannot extend their exploration. This situation can be in danger of falling into local optima.
In order to address this problem, we introduce the other exploration strategy called the replacement strategy. The idea of this strategy is to jump into relatively inferior solutions stochastically in case of stagnation. When agents fail to find superior paths, the proposed strategy selects some of the superior paths randomly and replaces them with inferior ones (Figure 4). After the exploration step at lines 7-18 of Algorithm 1, we identify the stagnation by checking whether or not the selected paths at line 14 are the same as the one before the selection. The replacement strategy then removes each of the paths at probability 0.5. The highly ranked paths found as the result of the procedure from line 9 to 13 are added to the remaining best-so-far paths so that the number of the selected paths can be up to the parameter ι to conduct line 14 and its subsequent lines.
Based on the above idea, Algorithm 4 presents the whole algorithm of ACOhg with the replacement strategy. Its description from line 1 to 20 is the same as that of Algorithm 1 from line 1 to 18, except that the paths found by the agents are stored into temporary  for k 1 to colsize do 9: p k ∅; 10: variables at lines 15 and 16. Also, we conduct the procedure from line 19 to 26 of Algorithm 1 without modification as described at line 41 in Algorithm 4. The main body of the replacement strategy is the procedure from line 21 to 40. Line 21 checks whether the stagnation arises or not. If we detect the stagnation, we first remove some best-so-far paths at random from line 23 to 28. Then, the inferior paths found by the agents are added to the remaining best-so-far paths from line 30 to 37. If we do not find the stagnation, the bestso-far paths are passed to the next iteration without replacement (line 39).

Experiments
We conducted performance comparison experiments between the original ACOhg-live and the modified ACOhg-live with the proposed strategies using two benchmarks.

Experimental settings
We implemented the prototype tool that runs ACOhg-live and the proposed strategies from scratch with Python 3.6. The models and the specifications are inputted as Labeled Transition Systems (LTSs) (Magee & Kramer, 2006). LTSs are suitable for describing concurrent state transition systems firing event sequences. In order to make the prototype efficient, we implemented on-the-fly checking (Clarke et al., 2018), which creates the state space as needed. The tool outputs either a counterexample as an event sequence if a violation is found, or None, i.e. a special value built in Python, otherwise. We prepared two benchmarks. The first benchmark is a database ring system, whose model is based on (Magee & Kramer, 2006). This benchmark describes a distributed database that consists of several communication nodes organizing a ring-shaped network. Each node receives updated information from one of its adjacent nodes, and changes its internally stored data. The node notifies the other adjacent node of the information to keep the consistency of the network. We prepared three specifications, called Spec. A, B and C, which are related to the requirements that the all nodes update their internal states successfully.
The other benchmark is a model that consists of random graphs. We generated five Erdős-Rényi graphs (Erdős & Rényi, 1959), where the number of nodes is 20 and the probability of creating an edge is 0.1. The graphs are then transformed to LTSs. We regard the LTSs as component processes. We followed this procedure to generate another Büchi automaton of a specification. This time we randomly selected some states as accepting states.
We set the following search conditions and ran the prototype thirty times for each specification and each experimental condition: . OR: running the original ACOhg-live without our proposed strategies, . ORT: running the original ACOhg-live by corresponding the terminal condition of its first phase with Stop-on-Arrival condition, with the aim of evaluating our strategies under the fair terminal condition, . SK: applying the skip strategy with Uniform Distribution in the first phase, . SF: applying the skip strategy with the weighted probability defined by Equation (3) and the forward indexing in the first phase, . SB: applying the skip strategy with the weighted probability defined by Equation (3) and the backward indexing in the first phase, . RE: applying the replacement strategy in the second phase, . RET: running the first stage of the original ACOhg-live by corresponding the terminal condition with Stop-on-Arrival condition and applying the replacement strategy in the second phase, . HY: applying the skip strategy with Uniform Distribution and the replacement strategy in the first and the second phase, respectively, . HF: applying the skip strategy with the weighted probability defined by Equation (3) and the forward indexing, and the replacement strategy in the first and the second phase, respectively, and . HB: applying the skip strategy with the weighted probability defined by Equation (3) and the backward indexing, and the replacement strategy in the first and the second phase, respectively.
For each experimental condition, we measured the length of the detected counterexample, the running time, and the memory consumption. The length of a counterexample evaluates the comprehensibility problem. Running time and memory consumption evaluate how efficiently the state explosion problem is handled. We used the settings of the parameters for ACOhg-live shown in Table 1. We selected this configuration tuned the previously established study (Chicano & Alba, 2008b). ACOhg is assumed to combine heuristics (Section 2.2).
In our experiments, we did not use any heuristics for the first phase since accepting states are not known in advance. For the second phase, we employed the hamming distance of binary representations of states (Chicano & Alba, 2008b). This heuristics estimates the distances between the current state and the accepting state to be searched. We used HP ProDesk 600 G4 SFF with Intel Core i7-8700, CPU 3.20GHz, RAM 16.0 GB, and Windows 10 Pro for the experiments. We turned off the garbage collection built in Python when running the prototype in order to avoid its influence on the performance.

Database ring system
We show the results of our experiments in Figure 5 and Table 2. In Table 2, the lengths of counterexamples, running time and memory consumption are denoted by 'Length',  'Time' and 'Memory', respectively. The terms 'Sp.', 'Avg.', 'SD' and 'MD' are abbreviations of the target specification, average, standard deviation, and median, respectively. For running time and the memory consumption, the skip strategies (SK, SF and SB) improved the original ACOhg-live (OR) and ACOhg-live with Stop-on-Arrival condition (ORT and RET) for each specification. The results suggest that the skip strategy promotes effective exploration of the state space. SK, SF and SB tend to generate a bit longer counterexamples as compared to OR, ORT and RE on average for Spec. B and C. SK, SF and SB make many agents extend on their exploration preferentially and weakens the feature of ACOhg-live to generate short counterexamples. The replacement strategy (RE) does not present significant differences about the performance as compared to OR. The observation indicates that it is difficult to improve the performance of ACOhg-live with the replacement strategy alone.
We discuss the effectiveness of combining both the skip and replacement strategies (HY, HF and HB) next. For the running time in Spec. A, the average and the median of HY, HF and HB are almost comparable to those of the corresponding skip strategies, i.e. SK, SF and SB, respectively, and are small as compared to those of OR, ORT, RE and RET. According to the result in Spec. B, the average, standard deviation and median of the memory consumption of the hybrid methods HY, HF and HB are the smallest. The hybrid methods HY, HF and HB are the most efficient with respect to both the running time and memory consumption for Spec. C. Except the running time in Spec. A, HY, HF and HB tend to decrease the standard deviations of the running time and memory consumption as compared to the corresponding skip strategies SK, SF and SB respectively. Although the lengths of counterexamples found by the hybrid methods worsen especially for Spec. B and C as compared to the other conditions, we believe that the performance improvement with the hybridization of our strategies compensates for this disadvantage of detecting long counterexamples. Our strategies are expected to provide ACOhg-live with the good balance between performance and comprehensibility of counterexamples.
Among three stochastic path selection methods employed in the skip strategy, the weighted probability given by Equation (3) with the forward indexing (SF and HF) tends to present good running time and memory consumption, in particular in Spec. B and C. All of the path selection methods yield counterexamples of almost comparable length.

Randomly generated model
The experimental results are shown in Figure 6 and Table 3. OR generates the shortest counterexamples on average of all experimental conditions. For running time and memory consumption, Stop-on-Arrival condition (ORT and RET) and our strategies (SK, SF, SB, HY, HF and HB) show the good results as compared to OR. Whereas the memory consumption of our strategies is comparable to that of Stop-on-Arrival condition, the contribution of our strategies to running time is more considerable than that of Stop-on-Arrival condition. Considering these observations, the proposed strategies balance well in comparison with the original ACOhg-live in terms of both efficiency and comprehensibility of counterexamples as the case of the database ring system. We do not observe considerable differences between the results of three variants of the skip strategy.

Discussion
We discuss why the proposed strategies make ACOhg-live generate rather long counterexamples. Recalling that Stop-on-Arrival condition (ORT and RET) tends to generate long counterexamples as compared to the original ACOhg-live (OR) according to the experimental results, we conjecture that this condition can weaken ACOhg's effect of shortening paths. In fact, the employment of Stop-on-Arrival condition implies that the process of shortening paths works well until the arrival at some accepting state. Unlike ordinal ACO, which searches and shortens paths from the initial state to the goal state, ACOhg with Stop-on-Arrival condition shortens only subpaths and sometimes get into local optima. Whereas Stop-on-Arrival condition has the drawback in terms of solution quality, it presents an advantage in terms of both time and space performance. A potential  improvement for the drawback is to use effective heuristics because they have great influences on performance and quality of solutions.
In our experiments, we investigated three variants of the skip strategy. According to our results, although the weighted probability defined by Equation (3) with the forward indexing tends to perform the most favourably for the database ring system, we observe little difference among the proposed variants for the randomly generated model. These results suggest that we have to select the most appropriate variants depending on verification problems. We have to conduct further investigation to understand the relationships of the proposed variants with characteristics of verification problems.
We next discuss the internal issues of our experiments. First, our prototype implementation has much room for improvement especially in terms of performance enhancement. We are planning to incorporate various optimization techniques of model checking, e.g. partial order reduction (Chicano & Alba, 2008a). Since our implementation runs in a sequential manner, we can improve the performance of the proposed strategies and our search procedure by parallelizing them. Second, our parameter settings of ACOhg can affect its convergence efficiency. Although our settings are based on the one established in the previous study (Chicano & Alba, 2008b), there may exist more efficient settings than ours for each benchmark. However, such settings can be too specific to the problem domain and can arise difficulty with respect to generalization. Third, because ACO is a stochastic search algorithm, we have to draw many samples and observe the averaged performance. Unfortunately, it is very difficult to determine the appropriate sample size in general. In our experiments, we sampled thirty runs independently for each condition. We believe that our sample size is reasonable.
We also discuss the external conditions of our experimental results. The database ring benchmark is a typical distributed and concurrent system. Since one of the most prominent features of model checking is verifying concurrent systems (Clarke et al., 2018), we believe that our experiments are appropriate for evaluating the proposed strategies under a representative use case of model checking. The other benchmark is comprised of random graphs. We expect that the benchmark does not depend on characteristics that are specific to the database ring benchmark. Hence, the model provides us with evidence of the general effectiveness of our proposed strategies.

Conclusion
This article tackles with two main challenges in model checking: the state explosion problem and the problem of finding short counterexamples. Our approach is involved with Search-Based Software Engineering, which tries to overcome the problems with non-exhaustive search algorithms based on swarm intelligence. We propose two novel exploration strategies for making an existing model checking technique based on Ant Colony Optimization, ACOhg-live, more efficient. Our experimental results lead us to the conclusion that ACOhg-live with the proposed strategies outperforms the original one with respect to runtime performance. Although our strategies detect longer counterexamples than the original ACOhg-live does, we believe that the drawback is subtle.
As future work, we plan to combine appropriate heuristics with our strategies. We will select the heuristics that are suitable for the formalism of automata-theoretic model checking in our implementation. We also have to compare the proposed strategies with the search methods other than ACOhg-live, e.g. the techniques discussed in Section 3.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Notes on contributors
Tsutomu Kumazawa is a researcher at Key Technology Laboratory in Software Research Associates, Inc., and a visiting fellow in Tokyo University of Science. His main research interests are formal verification and its industrial applications. He received his M.S. and Ph.D. from the University of Tokyo and B.A. in Engineering from Tokyo Metropolitan University.
Munehiro Takimoto is a professor in the Department of Information Sciences from Tokyo University of Science. His research interests include theory and practice of programming languages, and the various things derived from them, which include mobile agent systems and their applications. He received his Ph.D., M.S., and B.A. in Engineering from Keio University. He is a member of ACM, IEEE Computer Society, IPSJ, JSSST, and IEICE.