Pipeline leak diagnosis based on leak-augmented scalograms and deep learning

This paper proposes a new framework for leak diagnosis in pipelines using leak-augmented scalograms and deep learning. Acoustic emission (AE) scalogram images obtained from the continuous wavelet transform have been useful for pipeline health diagnosis, particularly when combined with deep learning. However, background noise has a significant impact on AE signals, which can reduce the accuracy of pipeline health identification using classification models. To address this issue, a new type of scalograms called leak-augmented scalogram is introduced, which enhances the variation in colour intensities of AE scalogram images. The leak-augmented scalograms are obtained by pre-processing them using image-enhancing Gaussian and Laplacian filters. The proposed method utilizes convolutional neural networks (CNNs) and convolutional autoencoders (CAEs) for feature extraction. The CNN extracts patterns specific to local changes, while the CAE extracts holistic patterns from the leak-augmented scalograms. The resulting leak susceptible and leak holistic indicators are merged into a single feature pool and provided as input to a shallow artificial neural network (ANN) to evaluate pipeline health conditions. The proposed method achieves high classification as well as accuracy, precision, F-1 Score and recall, compared to existing state of the art methods.


Introduction
Pipelines provide the cheapest medium for the transportation of gas, oil, and water.However, prolonged exposure to harsh conditions can lead to corrosion, leaks, and cracks, causing significant financial losses and environmental damage.To mitigate these issues, advanced techniques that can quickly detect and locate leaks in pipelines are of primary interest (Ahmad et al., 2022).In recent years, advanced techniques based on artificial intelligence (AI) and machine learning (ML) have been developed to improve the accuracy and efficiency of pipeline leak detection (Banjara et al., 2020).Different monitoring and protection strategies have been devised to maintain the safe and effective functioning of pipelines.These consist of time-domain reflectometry, techniques based on vibration, methods involving pressure waves, and the utilization of AE technology (Che et al., 2021).Among these strategies, AE technology has been particularly attractive due to its sensitivity to leaks and ability to detect leaks in real-time (Bergmann et al., 2018).Therefore, in this study, AE technology is used for the identification of pipeline health conditions.In order to prevent serious consequences, it is essential to detect pipeline leaks early.Leak detection involves evaluating the working status of the pipeline to determine whether a leak is present or not.Nowadays, the pipeline industry prioritizes cost-effective remediation methods, such as using repair clamps and encapsulation collars if the size of the leak allows it, rather than automatically replacing the damaged pipeline (Ahmad et al., 2023;Pipeline Repair Guide, 2020).
Pipeline leak detection has been the subject of extensive investigation.Vision systems, sensors, data, transient response models, and other models, have served as the foundations for pipeline health management (Caesarendra et al., 2016;Kim, 2022).In the past decade, research presented for pipeline health diagnosis primarily focused on feature extraction and AI-based recognition models.These models use advanced techniques, based on AI and ML, to analyze the AE data and identify patterns that indicate the presence of leaks in pipelines (Chan et al., 2019;Cheng et al., 2021).Elforjani et al. (Elforjani & Mba, 2009) utilized AE technology for the detection of fracture initiation in pipelines.Banjara et al. (2020) extracted AE waveform indicators and provided them to support vector machines (SVMs) and relevant vector machines, to identify leakage in a pipeline.Rai et al. (2021) created a pipeline health index using multiscale analysis and the Kolmogorov-Smirnov (KS) test, Furthermore, a Gaussian mixture model was applied to assess the severity of the leak.Kim et al. (n.d.) created a pipeline leak indicator by using AE waveform characteristics and a two-sample KS test.The recommended leak indicator performs better than traditional feature-based leak indicators based on the mean, variance, and mean root square measures.Li et al. (2018) produced a hybrid feature vector by merging AE time-domain characteristics and AE frequency-domain features.In this research, cross-entropy was applied to hybrid feature vectors to extract discriminant features that were then employed in conjunction with artificial neural networks (ANNs) to improve their performance in the area of leak detection.Xu et al. (Zhou et al., 2021) used empirical mode decomposition and continuous wavelet transforms as time-frequency empirical mode decomposition (EMD) approaches to pinpoint the leak's site.Xu et al. (2021) used variational mode decomposition (VMD) to denoise the AE signal.A characteristic produced using the Melfrequency cepstral coefficients (MFCCs) was also taken from the highly correlated VMD coefficients.An SVM was used to classify the MFCCs to determine the state of the pipeline.These studies performed better for pipeline leak diagnosis, however, there exist several shortcomings.First, the noise in the AE signal could cause false alarms if the threshold for extracting the AE features was too low.In addition to the problem with the predetermined threshold, extracting features from the AE signal required human expertise and knowledge of the subject.When EMD is used to pull out the natural modes of an AE signal, extreme interpolation could happen (Xu et al., 2020).The presence of mode mixing is another complication associated with EMD.
Information about leaks can be extracted and leakrelated patterns can be identified without requiring human participation using techniques based on Deep learning (DL).DL algorithms allow feature extraction from the time-frequency picture (Goodfellow et al., 2016).Despite advances in leak detection, detecting transients caused by unforeseen leaks remains a challenge.DL algorithms can pull out characteristics that help find leaks, but their accuracy may not always be perfect.CNNs are the most common type of DL technique, and they have proven to be very useful for finding faults and leaks (Alzubaidi et al., 2021;Jiao et al., 2020a).A CNN can extract discriminant in-formation related to leaks from acoustic images, which can subsequently be used to classify pipeline states (Jiao et al., 2020b).Jiao et al. (2019) used a deep coupled dense convolutional network for intelligent fault diagnosis.A residual joint adaptation adversarial network is also developed for intelligent defect detection (Jiao et al., 2020c).CAEs can compress images and find abnormalities by learning useful characteristics from the input data (Jiao et al., 2020b).The AE sensors detect a rise in the AE signal's energy in the form of AE hits as the AE phenomena evolve.Yet, background noise may be able to obscure these AE hits (Cody et al., 2020).Analyzing leak-related valuable hits may be done using the continuous wavelet transform (CWT), in which the time-domain signal is transformed into time-frequency scales (Cheng et al., 2021;Piñal-Moctezuma et al., 2020).The time-frequency scales are used to make a scalogram, which can show the hits in the AE signal as a three-dimensional surface across different time-frequency scales, where the third dimension is the magnitude.This is possible because the scalogram can display the time-frequency scales visually.
A leak in a pipeline can cause a change in the material's structural integrity, leading to various forms of damage, including fatigue rupture, stress cracks, corrosion cracks, and structural discontinuities.Despite the cause, the leak or structural discontinuity can disrupt the fluid or gas flow inside the pipeline.However, the fluid's intramolecular interactions or chemical bonding can help it maintain a consistent flow despite the disruption (Rai et al., 2021).An Acoustic Emission (AE) event refers to a transient sound wave produced by the change in the material's structural integrity, such as that of a pipeline, due to a short, rapid release of energy.When a pipeline leaks, stress waves are generated, and AE sensors on the pipeline can detect these waves as they travel through the walls of the pipeline.Stress waves coupled with leaks are the source of transients in the AE signal, referred to as 'hits' or 'AE events' (Ahmad et al., 2023).A threshold must be set above the constant level of background noise in order to extract important AE parameters from the AE signal, such as the rise time, decay time, and counts (Masci et al., 2011).Nevertheless, the use of a predetermined threshold for AE characteristic extraction may lead to false alarms because of noise in the AE signal.The process of isolating AE features from the AE signal and determining a threshold that is higher than the level of continuous background noise calls for the involvement of humans who have experience and expertise in the relevant fields.This paper presents a comprehensive framework for detecting leaks and identifying their sizes in pipelines using leak-augmented scalograms and deep learning.The traditional approach of using AE scalogram images obtained from the continuous wavelet transform for pipeline health diagnosis based on deep learning is hindered by the presence of background noise that can mask the colour intensity variations in the scalogram images caused by the acoustic emission hits.To address this limitation, the proposed method introduces a new leak-augmented scalogram.The acoustic emission signals are first converted into time-frequency scalogram images, with colour intensities that vary based on frequency changes across different time scales due to pipeline operating conditions.The images are then preprocessed using Gaussian and Laplace filters to enhance the variations, resulting in leak-augmented scalograms (Gupta, 2022;Ullah et al., 2022).The leak-augmented scalograms are provided as input to convolutional neural networks (CNNs) and convolutional auto-encoders (CAEs) for feature extraction.The CNN extracts patterns specific to local changes, while the CAE extracts holistic patterns from the leak-augmented scalograms.These patterns result in new leak susceptible and leak holistic indicators, which are combined into a single feature pool and provided as input to a shallow artificial neural network (ANN) for evaluating pipeline health conditions.The proposed method demonstrated high classification accuracy compared with existing state-of-the-art methods.The new feature pool resulting from the combination of leak-susceptible indicators and leak holistic indicators can be further explained as follows.The leak-susceptible indicators are obtained by applying several convolutional filters from a CNN to extract patterns in specific local sections of the input leak-augmented scalograms.The leak holistic indicator is the representation of latent space that is acquired in the hidden layer of the trained convolutional autoencoder.These leak holistic indicators form the basis for re-constructing the input scalogram.The study uses an ANN to assess the state of the pipeline using a feature pool comprised of leak-susceptible indicators and leak holistic indicators.A metallic steel pipe was utilized to test the proposed leak-detecting technique (Ahmad et al., 2022).The novelty of this study can be summarized as follows: • New leak-augmented scalograms are introduced.The leak-augmented scalograms are obtained by preprocessing them using image-enhancing Gaussian and Laplacian filters.The main contributions of this study are: • To visualize and distinguish leak features from noisy data, time-domain AE signals were converted to images using the CWT.A Gaussian filter was applied to the noisy image to smooth it, followed using a Laplacian filter for edge detection which results in leak-augmented scalograms.These leak-augmented scalograms revealed leak-related characteristics.A new approach is developed which utilizes leak-augmented scalograms and a CNN-CAE deep neural framework for pipeline health diagnosis.Additionally, the proposed method is designed to be effective in detecting leaks in pipelines transporting various types of fluids such as water and gases.To the best of the authors' knowledge, this approach is not reported for pipeline leak diagnosis in the literature.• The t-distributed stochastic neighbour embedding (t-SNE) is a dimensionality reduction technique commonly used in machine learning for visualizing highdimensional data (Erfani & Goharian, 2023;Van der Maaten & Hinton, 2008).This study, its goal is to preserve the local and global data structures while mapping the complicated, nonlinear connections between the data points into a two-or three-dimensional space.• Data from real industrial pipelines testbed developed on lab scale were utilized to validate the suggested methodology of the proposed method.
The organization of this research work is as follows: Part 2 covers problem definition, Section 3 the proposed technique, Section 4 describes the data collection and experimental setup for detecting pipeline leakage, Section 5 examines the proposed method's findings, and Section 6 concludes this study with future directions.

Problem definition
A leak in the pipeline can be caused by damage to the material's structural integrity, fatigue rupture, stress cracks, corrosion cracks, and structural discontinuities, which disrupt the flow of fluids or gases.However, intramolecular interactions or chemical bonding of the fluid can help maintain a consistent flow.Thus, for the fluid to keep its flow consistent inside the pipeline the molecule of the fluid will exert pressure on the position of pipeline structural discontinuity or leak which will result in a short, rapid release of energy, in the form of an elastic wave.Acoustic Emission (AE) events are transient sound waves generated by changes in the material's structural integrity.These waves can indicate potential leaks due to various forms of damage and can be detected by AE sensors.
The Continuous Wavelet Transform (CWT) is obtained by summing scaled and shifted versions of the wavelet function over the complete time.The continuous wavelet transform (CWT) is used to turn the timedomain AE signal into 3D scalograms by the following definition (Hübner et al., 2020).
The CWT of a signal yields coefficients that depend on the scale and translation parameters 's' and 'τ ', respectively.'τ ' indicates the wavelet's position in the time domain while 's' determines its central frequency and window length.Larger scales correspond to lower frequencies, providing global information about the signal, whereas smaller scales reveal finer details of the signal by corresponding to higher frequencies.The scalograms of CWT are shown in Figure 2(a).There are two main research problems, (1) CWT scalograms are often used in leak detection models that use AE sensors in pipelines.However, these scalograms may contain noise that can affect the accuracy of leak detection.When noise is present, it can be difficult for the model to accurately distinguish between leak and non-leak signals.
Additionally, the presence of noise can make it more challenging for the model to classify leak and normal conditions distinctly.Therefore, it is important to develop strategies to reduce the impact of noise on the model's performance, such as pre-processing the scalogram images before analysis.
(2) Pipeline networks provide transportation medium for various types of liquids and gases.The statistical properties of the AE event resulting from a leak in the pipeline can be affected by different transportation mediums.Thus, it is important to develop a common model that can be used explicitly for pipeline leak detection irrespective of the fluid pressure and its chemical nature.
To address these two problems, the following proposed model is used.

Proposed method
The graphical flow of the proposed method is presented in Figure 1.The proposed approach consists of the following steps: Step 1: AE signals are acquired from the pipeline under normal and leak operating conditions.The discharge of gas or fluid due to a leak in a pipeline causes a stress wave that travels through the surface of the pipeline toward the AE sensors.The stress wave generated by the AE events is detected by the AE sensor in the form of an AE hit.These AE hits in the signal can be utilized for leak detection in the pipeline.
Step 2: The continuous wavelet transform (CWT) is used to turn the time-domain AE signal into 3D scalograms.These pictures show the changes in energy levels over different time-frequency scales by using different intensities of colour.In this study, the scalograms capture the energy shift, and different colours are used to depict different energy levels.
Step 3: The CWT scalograms are then preprocessed and the Leak augmented scalograms are obtained.The steps involved in obtaining the leak-augmented scalograms are as follows.First, a Gaussian filter is applied to the CWT scalogram to smooth it out and minimize the amount of noise that is present in it.After that, an edge detector, called a Laplacian filter, is applied to the CWT scalogram.Because of this, it is possible to identify the edges more accurately in the CWT scalogram.The classification accuracy of a classification model is strongly dependent on the discriminant of the input.For this reason, the scalogram is preprocessed and leakaugmented scalograms are obtained.Figure 2 shows the leak-augmented scalograms which show a clear energy distribution over the entire time scale as compared to that of a traditional scalogram.
Step 4: It is possible to extract holistic and susceptible information from leak-augmented scalograms using a convolutional neural network (CNN) equipped with a convolutional autoencoder (CAE).The leak-susceptible indicators capture changes in energy at a more precise level, such as fluctuations in AE amplitude over time, while the leak holistic indicator gives information about the general properties of the AE signal, such as the frequency distribution and AE intensity.After that, these factors are used as inputs for analysis to find out what makes a pipeline leak different.
Step 5: A complete feature vector is produced after combining the holistic and susceptible properties extracted from the Leak augmented scalograms.This feature vector is then used to find pipeline leaks.After that, this feature vector is sent into an artificial neural network (ANN) with one hidden layer, which helps determine the health state of the pipeline.
Step 6: To assess the performance of both models, the recommended model was compared to the two best models using the performance measures of accuracy, precision, recall, and the F1 score (Das et al., 2019).

Gaussian and Laplace filter
A Gaussian filter is a linear filter that uses a Gaussian function to smooth out an image.The Gaussian function is a curve that looks like a bell, and the mean and standard deviation tell us what it looks like.The amount of smoothing that is done to an image is based on the standard deviation of the Gaussian function.Gaussian filters are commonly used to eliminate noise from an image because they can smooth out small, random fluctuations in pixel intensity while preserving the overall structure of the image (Gupta, 2022).The Laplacian filter is a type of edge detection filter that makes the edges of an image stand out (Ullah et al., 2022).It works by calculating the difference between a specific pixel's intensity and the average intensity of its surrounding pixels.The result is a high-pass filter that makes the edges of the image stand out by making the difference in brightness between neighbouring pixels stronger.This can be used to identify edges and provide more accurate findings in pipeline leak detection.
In this work, acoustic emission (AE) signals were obtained from pipelines and processed using the continuous wavelet transform (CWT) coupled with a Morse wavelet with a symmetry parameter equal to 3.This allowed for the generation of scalograms (Zhao et al., 2022).Figure 2 clearly illustrates different energy zones and pipeline conditions through the use of leakaugmented scalograms.The application of Gaussian and Laplacian filters revealed that the high-energy components in Figure 2a occurred at different times and frequencies compared to those in Figure 2b.Images of acoustic emission (AE) that are taken when pipeline leaks occur usually show high-energy parts that are thought to be caused by AE events that are caused by stress waves coming from the leak.Depending on the type and size of the AE hits caused by the pipeline break, the timings and frequencies of these high-energy parts can change.

Autoencoder for feature extraction
Autoencoders are a kind of neural network used for feature extraction and dimensionality reduction of input data.Encoding and decoding are their two essential components.The encoder compresses the input data into a representation with fewer dimensions, known as the latent space.Using this latent space, the decoder then reconstructs the original input data.The goal of the autoencoder is to find a compressed version of the input data that captures its most important features while leaving out noise and other un-important information (Goodfellow et al., 2016).
Three layers make up an autoencoder: the input layer, the hidden layer, and the out-put layer.Combining the input and hidden layers, the encoder unit of the autoencoder compresses the input data and provides a lowerdimensional representation.The decoder component, composed of the output layer and the same hidden layer, is responsible for reconstructing the original input data from this compressed form.The fundamental objective of the encoder is to discover the most informative hidden representation of the input data, which is subsequently used by the decoder to reconstruct the input data (Bergmann et al., 2018).
The encoder component performs a series of mathematical operations on the input x within its hidden layers to generate latent representations h using Equation (2).These hidden layers contain a bottleneck layer with a weight matrix W1 and bias vector b1, which are used in conjunction with a nonlinear activation function, denoted as 'phi', to transform the input data into the desired output.
where X is the data that has been put back together, h is the input to the autoencoder, phi is ReLu, and W 2 and b 2 are the right weight and bias vectors in Equation ( 3).An autoencoder has three layers: the hidden layer, the output layer, and the input layer.Compressing the input data and generating a lower-dimensional representation are functions of the input and hidden layers of the encoder unit of the autoencoder.To obtain the lowest possible reconstruction error, the autoencoder learns a latent representation that is both discerning and of excellent quality.In the context of this research, the autoencoder was leveraged to derive holistic characteristics from the pipeline AE signals, as it is characterized by its ability to distinguish between different states of pipeline leakage (Bergmann et al., 2018;Zeiler et al., 2011).

Introduction to convolutional autoencoders
To capture the holistic characteristics of AE signal scalograms, a CAE was used.Using the autoencoder's ability to compress data, the CAE got a lot of important information from the leak-augmented scalogram.The CAE model contains an encoder component comprised of a convolutional layer, a pooling layer, and a fully connected layer, as well as a decoder component, comprised of a fully connected layer and a collection of transposed convolutional layers, for decoding the features in the latent space.The scalogram served as the CAE's input (Nguyen et al., 2021).A CAE was utilized to extract holistic characteristics from the leak-augmented scalograms of the AE signals.The CAE was made up of an encoder and a decoder.The main job of the encoder was to pull useful information from the scalograms.The employment of a decoder was necessary for efficient algorithm training.The encoder was made up of convolutional, pooling, and fully connected layers, while the decoder was made up of a fully connected layer and a group of transposed convolutional layers.Table 1 provides a comprehensive outline of the CAE architecture.By using the CAE, better features were found in the form of latent coding, which was then used to rebuild the scalograms.
The encoder architecture used in this study was found to be similar in design to the CNN presented in Table 2.The encoder applied convolution layers to the input data and used max pooling to extract the most relevant information.This process was repeated four times, resulting in a compressed, lower-dimensional latent code.The decoder component of the autoencoder employed four transposed convolution layers (convT) to increase the dimensions of the learned latent code and reconstruct the original input data.Before the convolution process, a stride value was applied to the reconstructed output, with a stride of 2 and padding of 0 being set to enable smooth and accurate reconstruction.
The convT operation is formally represented by Equation (4).The matrix L m , which corresponds to the convolutional layer m, was subjected to a series of convolution and summation operations to generate a set of vectors, denoted as s m−1 .These vectors are then used in subsequent layers of the model to capture higher-level features of the input data.X m is the result in this case.
The convolutional matrix, L m , must pass the transposed convolution to reconstruct the input (Rai & Kim, 2021).The reconstruction process produces an output Here, Y m , with the same dimensions as the output of the transposed convolution layer is represented by the abbreviation S m−1 .

CNN susceptible feature extraction
The susceptible characteristics of the leak-augmented scalograms were retrieved using a CNN.The most significant input properties were extracted using the convolutional and pooling layers of the CNN.An output layer and a fully connected layer processed the convolutional and pooling layers' flattened features afterward.The input leak-augmented scalograms were convolved using a variety of filters in the convolutional layer.The convolution technique was carried out separately for each channel of the input picture because the AE photos were 3D images.A feature map was produced using a convolution method at each layer and an activation function [33].Equation ( 6) presents the principle of the convolutional layer mathematically.
In Equation ( 5), the 2D convolution operation of channel k (where k takes values ranging from 1 to K m−1 ) is performed on the input of the (m-1)th convolutional layer, denoted by x m−1 k using the weights of the Cth filter in layer m represented by W m k,c .The output of this operation is then passed through a nonlinear activation function, denoted as K. a m (.), where (.) refers to the downside sampling process, is used to obtain the feature map.In this process, ReLU serves as a nonlinear activation function.
Max pooling was the pooling technique used in this research.The primary objective of the pooling layer was to recover crucial data while also lowering the computational complexity in terms of time and memory needs.The fundamental organization of the feature maps created by the pooling layer is shown in Equation ( 7).
Here, down(.)stands for the down sampling process, x m c is the pooling layer's out-put, x m−1 k is the output of the last layer and input of the current layer, and x m−1 k shows a multiplicative bias, while b m c is the additive bias.After the data was processed by the convolutional and pooling layers, a feature map was generated.X m = vec(x m−1 ) further flattened the feature map.The completely linked layer received the flattened feature map.The characteristics were weighted in a layer that was completely linked.The output layer of the CNN for this study was the fully connected layer.The mathematical model of the completely linked layer is shown in Equation ( 8): In this case, W m and b m are the weight and bias, respectively, a is the activation function and X m is the output of the fully linked layer.The fully connected layer in the convolutional neural network produced the leaksusceptible indicators for the present research (CNN).
The architecture of the CNN employed in this research is shown in Table 2.A discriminant feature pool was created by combining the leak susceptible indicators from the CNN with the leak holistic indicator from the CAE.There were 8,192 features in the discriminant feature pool.The procedure for extracting both leak holistic and susceptible indicators is shown in Figure 3.

t-SNE
The machine learning technique known as t-SNE, or 't-distributed stochastic neighbor embedding', is frequently used to depict high-dimensional data in a 2D or 3D scatter-plot.As a result, points that are comparable in the high-dimensional space will be represented by points that are close to one another in the low-dimensional space since the technique retains the local structure of the data (Van der Maaten & Hinton, 2008).This allows for patterns and structures in large, complex datasets to be easily discovered and displayed.
In this work, t-SNE is used to analyze the encoded image representations created by the autoencoder neural network.Autoencoders are trained to compress an input image into a more miniature representation, known as an 'encoding', and then use this encoding to reconstruct the original image.By viewing these encodings using t-SNE, it is possible to learn more about the structure of the data and how the autoencoder presents the images.t-SNE can also be used to identify clusters of related images within the encoded space, making it useful for classifying different types of acoustic emission signals, such as those associated with leaks and those that are not.In this context, t-SNE can be especially useful for pipeline leak detection as it allows for the identification of subtle changes in the signals that may indicate the presence of a leak.The two models clearly demonstrate that they were unable to classify the leak and normal conditions as accurately as the proposed model in the first part of Figures 4-6.The t-SNE is used to classify the features of leaks (red) and non-leaks (green) in the figures illustrated in this work.

Identification of pipeline leak states using an ANN
The An artificial neural network, often known as an ANN, is a form of machine learning model.There are three primary layers that make up an ANN: the input layer, the hidden layer, and the output layer.The purpose of the input layer is to collect input features, which are then changed in the hidden layer through a series of linear matrix multiplication operations.These modifications take place after the input characteristics have been collected.After that, the ANN's output layer uses an activation function to put the updated features into groups based on what roles they play (Goodfellow et al., 2016;Hu et al., 2021).In this study, the sigmoid function was chosen to act as an activation function in the output layer of the neural network.The neural network is able to learn complicated connections between the input characteristics and the classes that correspond to them because of the nonlinearity that is introduced by the activation function in the output layer of the neural network (Krizhevsky et al., 2017).By using the sigmoid  function, the neural network was able to make an output in the range of 0-1, which is good for classification tasks.
In this study, the status of the pipeline was ascertained using input characteristics, and the ANN was employed for classification.A cross-categorical entropy loss function was used to categorize the data.Due to the fact that the input and hidden layers did not play any part in the classification process, the activation function was not applied to those layers.It is possible to represent the ANN by using Equation ( 9): Here, X m is the mth layer output, x m−1 is the output, and W m and b m are the mth layer's weight and bias vector.Table 3 illustrates the ANN's architectural design used in this study.

Experimental setup
The experimental design used in this study and some of the schematics are shown in Figure 7(a) and (b), respectively.A stainless-steel water pipeline with an outside diameter of 114 mm and a thickness of 6 mm was outfitted with AE R15I-AST sensors manufactured by Mistras Group, Inc. of New Jersey, United States.This was done in order to mimic the occurrence of pipeline leaks.In order to obtain AE signals at a sampling frequency of 1 MHz, we used a computer and a data-collection apparatus manufactured by National Instruments and given the model number NI-9223.In order to replicate leaks of different sizes in the pipeline, an electrical drill machine was utilized to create a hole.A fluid control valve was then welded onto the pipeline at the location of the hole to regulate the flow of fluid through the hole.To ensure that the experiment was conducted in a safe and environmentally friendly manner, water was chosen as the fluid to flow through the pipeline.Water is a non-toxic and non-hazardous substance, which ensures the safety of both the environment and the operating staff.By using water in our experiment, we were able to simulate a leak scenario in a controlled and safe manner.In particular, a valve was installed on the pipeline so that the usual pressure of 13 bars could be maintained even when the valve was in its closed position.After the first two minutes of taking measurements, the valve was opened to simulate a 1 mm pipeline break.After that, more measurements were taken for another two minutes.Then, data was collected for a further two minutes while the valve was closed, and the pipeline was operated at a pressure of 18 bars.In the last step of the experiment, a pipeline leak of 0.5 mm was purposefully created, and data was collected for 6 min while the valve was left open.The same setup is repeated for a 0.7 mm leak size and 1 mm leak size at 7 bar.For each leak size, a total of 240 samples were taken, 120 of which were taken when the pipeline wasn't leaking and the other 120 when it was leaking.This resulted in a total of 240 samples for each pressure state.The AE signals of the pipeline, both while it was leaking and when it was functioning normally, are shown in Figure 8(a) and  (b) below.For the safety of the operating staff, the fluid from the pipe-line leak was collected in a container with the help of a hose.

Dataset collection and description
In this research, the pipeline acted as a conduit for the transmission of fluid and gas, with the pressure being adjusted to either 13 or 18 bar for each medium using a centrifugal pump.Data was initially gathered when the pipeline was used normally with the valve closed.The leak valve was then opened to 1 mm while maintaining the same pressure, creating Dataset-1.After gathering data at a pressure of 13 bar, the pipeline was run normally at a pressure of 18 bar, and data was collected again.Opening the leak valve to 0.7 mm while changing the pressure 7 bar conditions produced Dataset-3.For Dataset-4 the leak valve was opened to 0.5 mm at the same pressure conditions while collecting data.Table 4 provides a thorough explanation of each data set.

Results and discussion
The way that the training and testing data are set up is an essential part of figuring out how well the suggested method works.For training, datasets for a 1 mm leak size at 13 and 18 bars of fluid pressure were used.For evaluation, three data sets were used with leak sizes of 0.5, 0.7, and 1 mm and 7, 13, and 18 bars of pressure.Four hundred eighty normal samples and 480 leak samples were included in the dataset for this research, which had 960 samples in total.During the model's training, 70% of the samples were randomly selected, and 30% were used for the model's testing.To ensure that the findings would be consistent, each dataset was subjected to the tests 15 times (the number of experiments).

Suggested approach: comparison of performance
The CNN-CAE deep neural network is used in the proposed method to pull features out of leak-augmented scalograms, capturing both holistic and susceptible information.Although leak holistic indicators record patterns and broader characteristics, leak-susceptible indicators detect leak-specific information.A feature vector created from the combination of these characteristics is utilized as the ANN's input to determine pipe-line conditions.Metrics like accuracy, precision, recall, and F1 score are used to compare the suggested method to the reference method.These metrics help measure how well the classification algorithm works and how correct the classification results are.The metrics mentioned were computed by utilizing Equations ( 10) through ( 13).
The terms 'TPa', 'FPa', and 'FNa' stand for 'true-positive', 'false-positive', and 'false-negative' outcomes derived from the characteristics designating class A, respectively.False-positive results relate to the number of samples that were wrongly classified as belonging to class A when they really do not.True-positive results indicate the number of samples that are accurately identified as belonging to class A. False-negative findings are the number of samples that were wrongly assigned to class A even if they do not belong to class A. A denotes the total number of classes in the collection and the variable n α represents all the samples from class A. The combination of TPa and FNa, which represents the total number of samples that belong to class A, is denoted by n α .The combination of FPa and the difference between the total number of data samples and n α , which represents the total number of samples that do not belong to class A, is represented as the total number of samples that are misclassified as belonging to class A. Lastly, in the classification algorithm, the variable N stands for the total number of data samples in the testing sets.It is a crucial parameter in evaluating the performance of the algorithm and determining the accuracy of the classification results.The results will be more accurate for the whole set of data if there are more data samples.So, it's important to make sure that the testing sets have enough data samples to give an accurate picture of how well the classification algorithm works.
To evaluate the effectiveness of the proposed model, the proposed model is compared with two other relevant models used for similar purposes.The first model is a convolutional autoencoder (CAE) and neural network (NN) proposed by Prosvirin et al. (2021), for detecting leaks in centrifugal pumps using Kurtograms and Ahmad et al. (2022) using AE signals and CWT images, and the second model is a convolutional neural network (CNN) used by Shukla et al. (Shukla & Piratla, 2020) in a similar experimental setup.
After applying the proposed method to the AE data obtained from the industrial fluid pipeline, the suggested technique resulted in higher accuracy, precision, recall, and F1 scores of 99.08, 99.12, 99.08, and 99.08 percent, respectively, on all the datasets for pipeline leak detection under different fluid pressures and leak sizes.Table 5 presents the results obtained from the proposed method and the reference methods.It can be seen from Table 5 that the proposed method outperformed the reference methods in terms of classification accuracy.The superior performance of the proposed method can be explained as follows.The proposed method outperforms reference models in all experiments and across all performance parameters because of its core idea of preprocessing the scalogram images and using CNN-CAE for discriminating pipeline health-related features extraction.
Shukla et al. (Shukla & Piratla, 2020) have also used a deep learning-based technique for leak identification.The study they conducted aimed to detect leaks in a leaky plastic tank using hydrophone-collected data.This method is selected for comparison because the experimental setup adopted in this study is the same as that of the reference method.The reference method used a mix of frequency-domain, time-domain, and frequency-time-domain signals for the identification of leaks using the deep-learning CNN model.The study suggested that the deep learning model shows sensitivity toward the frequency spectrum of the AE signals.After applying the steps presented by Shukla et al. (Shukla & Piratla, 2020) to our dataset, the method resulted in accuracy, precision, recall, and F1 scores of 96.51%, 96.33, 96.51%, and 96.51%, respectively.Masoumeh Rahimi   2022) used nearly the same deep neural architecture as presented in this study.After applying the steps proposed in (Prosvirin et al., 2021) to our dataset, the method resulted in accuracy, precision, recall, and F1 scores of 95.83%, 96.37%, 95.83%, and 95.83%, respectively.The AE signals obtained from the pipeline consist of continuous background noises and leak-related AE hits.The AE images, such as scalograms and kurtograms, can represent the variations in the signals, however, they are often affected by the strong background noise.For this reason, it is important to further preprocess these images, such as using the approach of the proposed method.This is the primary reason that the proposed method has higher accuracy than that of the reference method presented in (Prosvirin et al., 2021).
In terms of accuracy, it is clear that the proposed method is better than the reference methods because the characteristics it gathered for pipeline leaks and normal conditions were more accurate and less spread out, as evident in the T-SNE illustrations presented in Figures 4-6.The characteristics produced from the referenced approaches underperformed in identifying pipeline health status compared to the suggested method because they were either dispersed or less discriminating for the pipeline's normal and leak situations.Furthermore, the misclassification of the proposed and the referenced methods are presented in the form of confusion matrices in Figures 9-11.The suggested approach was used to examine the pipeline signal using MATLAB.The process of extracting the characteristics from the leakaugmented scalograms and categorizing them takes less than 1 min for five trials on a computer with a 4.2 GHz processor and 24 GB of RAM.
Overall, the study makes a significant contribution to the field of leak detection in pipelines by introducing a comprehensive approach that effectively addresses the issue of signal noise encountered by deep learning models.By overcoming the limitation of employing separate models for distinct fluids and varied operating conditions, the study offers a unified solution that enhances the overall efficiency and effectiveness of leak detection systems.The obtained results demonstrate the potential of the proposed method to substantially enhance accuracy and reliability in detecting leaks, thereby playing a crucial role in ensuring pipeline safety and environmental preservation.This research holds considerable • A hybrid deep learning CNN-CAE-based model is presented for the autonomous extraction of leaksusceptible indicators and leak holistic indicators from the leak-augmented scalograms for all kind of fluids such as liquids and gases.

Figure 1 .
Figure 1.Proposed fault diagnosis framework for pipeline health diagnosis.
et al. (2020) used the same kind of techniques and the same kind of results were obtained using the said data set.Underperformance was expected because the AE signal obtained from the pipeline is heavily affected by the noise.Furthermore, these signals are highly nonstationary, and the Fourier spectrum may not be able to detect leak-related changes in the signals.The method proposed byProsvirin et al. (2021), utilized kurtograms obtained from preprocessing the vibration signals of the CP and deep architecture based on a CNN-CAE to identify mechanical defects in the CP.This method is selected for comparison because the reference method likeAhmad et al. (

Table 1 .
Configuration of the convolutional autoencoder for feature extraction.

Table 2 .
The CNN architecture.

Table 3 .
Overview of the employed artificial neural network: layers and operations.

Table 4 .
Description of the acquisition configuration for each dataset.