Acoustic feature based geometric defect identification in wire arc additive manufacturing

ABSTRACT In additive manufacturing of metals, numerous techniques have been employed to sense print defects. Among these, acoustic-based sensing has the advantage of low cost and shows the most potential to identify both external and internal defects as an in-situ monitoring system. Using acoustic signals, researchers have broadly investigated non-machine learning and machine learning-based approaches to identify defects like balling, micro defects, lack of fusion pores, keyhole pores, cracks, and porosity. While most of these works have shown promising results for laser-based AM systems, few have explored how acoustic signals can be used effectively for Wire Arc Additive Manufacturing (WAAM) defect detection. This paper proposes a methodology to construct machine learning (ML)-based models on identifying geometrically defective bead segments using acoustic signals during the WAAM process. Geometrically defective bead segment or geometric defect is a defect that causes voids in the final printed part due to incomplete fusion between two non-uniform overlapping bead segments. Such a defect is currently not explored in the literature. The proposed methodology uses a novel dataset labeling approach to identify good and bad bead segments based on an optimal threshold of the range of mean curvature. Furthermore, the methodology targets defective bead segments based on acoustic feature inputs like Principal Components (PC) or Mel Frequency Cepstral Coefficients (MFCC). To understand the resulting performance of the defect identification models constructed based on the proposed methodology, experiments are performed and tested on a variety of ML models (KNN, SVM, RF, NN, and CNN) based on the Inconel 718 material. The results show that the combinatorics of two acoustic input features and five ML models can be able to identify geometrically defective segments accurately with F1 score that ranges from 80% to 85%.


Introduction
Wire Arc Additive Manufacturing (WAAM) is a direct energy deposition process according to ASTM F2792-12a where weld beads are deposited layer-by-layer to form 3D metallic components (Busachi et al. 2015;Ding et al. 2015).WAAM employs an electric arc as a heat source and a metal wire as a feedstock for material deposition (Rodrigues et al. 2019).WAAM is recently becoming popular because of its low equipment cost, low buy-to-fly ratio, high deposition rate, and environmentally friendly approach.However, WAAM suffers from processing-related defects such as porosity, cracks, distortion, oxidation, etc. Xu et al. (2018), Yuan et al. (2020), Wu et al. (2018) and Xia et al. (2020).Sometimes these defects can propagate to the subsequent layers, which reduces the strength (Rajashekar and Rajaprakash 2016;Surovi and Soh 2022) of the final printed part, shortens the product's lifetime, and sometimes cause a collapse of the structure that causes economic loss (Chu and Wang 2016;Vilček et al. 2017).Therefore, it is important to identify print defects as early as possible to take appropriate corrective measures during the printing process to save welding resources and material costs (Surovi, Dharmawan, and Soh. 2021).
Researchers used various non-destructive techniques (NDT) to identify defects, such as laser-based profile scanning, image-based sensing, thermography, acoustic-based sensing, etc.For instance, on laser-based profile scanning, Chen et al. (2021) used a laser profiler, and Huang et al. (2022) used a 3D laser profilometer inspection (3D-LPI) system for the identification of surface defects.Laser-based profilers identify such defects by capturing the reflected laser line from the target surface through a narrow bandpass optical filter.
However, the limitation of this approach is that ambient light may blend with the reflected laser, particularly on metallic surfaces, affecting its accuracy, and it is incapable of identifying internal defects.On image-based sensing, Li et al. (2023) used a CMOS camera and an optical filter for surface defects detection.Similarly, Cho et al. (2022) used a high dynamic range (HDR) camera to identify normal and abnormal beads.In both approaches, images from the object were analysed for defects.The downside of such an approach is that defects can only be identified if visible on the image, and such sensors require high maintenance.For thermography, Kryukov et al. (2014) used a thermal camera to identify defects by studying the intensity profile during the friction weld process.In his approach, internal defects were identified by monitoring the heat dissipation from the object's cooling rate and thermal gradients.However, additional post-processing is required to identify defects due to a wormhole or lack of fusion.In acoustic-based sensing, the acoustic signal generated during the deposition process is filtered and analysed for features using signal processing techniques to detect defects.They have several advantages over the previously discussed sensors due to their low cost, ease of maintenance, simple structure, and highly targeted sensitivity (Kaushik, Nance, and Ahuja 2005), on top of their potential to identify both external and internal defects.
Several works have analysed the acoustic signal from the electric arc to determine the stability of the deposition process.Typically, researchers use acoustic emission (AE) or acoustic sensors to characterise the process stability, metal transfer modes, or defects.AE sensors are attached to the printing surface and used to capture the sound signal waveform through the material during the deposition process.On the other hand, acoustic sensors are mounted near the printing surface and measure the sound pressure waveform in the air.In the context of using acoustic-based sensing to determine process stability and metal transfer mode, Polajnar, Bergant, and Grum (2013) showed that irregularities in the bead geometry are reflected in the intensity of acoustic and current signals.They also found that acoustic signals with high-frequency arc oscillations indicate process instability.Pal, Bhattacharya, and Pal (2010) studied the relationship of welding arc signals with process parameters and metal transfer mode for both continuous gas metal arc welding (GMAW) and pulsed gas metal arc welding (P-GMAW).They found that the RMS and Kurtosis of arc sound can effectively identify metal transfer modes.Grad et al. (2004) investigated acoustic signals and weld process characteristics for GMAW.They found that non-uniform and discontinuous gas metal arc welds can be associated with non-regularities in their arc behaviour, such as long silence between two successive arc re-ignitions.Lv et al. (2014) showed that the height of the arcs constituting the shape of the bead has a linear relationship with the pressure of the measured acoustic signal.The research mentioned above highlighted that recorded acoustic signal contains rich information on the deposition process and can be used to infer the arc conditions, melt pool dynamics, and even internal properties of the welded material.
In the context of defect detection using acoustic signals, researchers have broadly investigated nonmachine learning (non-ML) or machine learning (ML) based approaches to identify different types of defects that occurred in metal AM.Among the non-ML-based approaches, Gaja and Liou (2017) showed that porosity and crack defects of the laser metal deposition (LMD) process could be identified by acoustic signals.They concluded that porosity defect produces acoustic profiles with shorter decay time and less amplitude, while cracks defect triggers acoustic characteristics with short duration and high amplitudes.Ito et al. (2021) showed that AE sensors could pick out the pore location and micro defects occurrence in selective laser melting (SLM).They compared their finding with X-ray computed tomography (CT) and concluded that bursttype AE events correspond to pore and micro defects.Rao et al. (2022) proposed a two-stage ultrasonic waveform tomographic method to detect and characterise porosity defects in multi-material (MM) laser metal deposition (LMD) parts.They concluded that their method could accurately reconstruct the multi-material interface and porosity by lowering the deposition laser power.Recently, Bevans et al. (2023) used a wavelet-integrated graph theory to filter, process, and monitor the acoustic signatures for WAAM.This approach utilised one type of feature, which they term Fiedler number, to pick out defect location accurately.However, it could not classify the specific type of defect that occurs at that location.
On the use of ML approaches for defect detection using acoustic signals, Drissi-Daoudi et al. (2022) explored acoustic features from time, frequency, and time-frequency domains.They used a Convolutional Neural Network (CNN) to identify the lack of fusion pores, conduction mode, and keyhole pores defects for three metallic alloys.They found that the classification of acoustic features is material and defect dependent.They also found that the performance of their process depends on the geometry of the chamber material and the position of the AE sensor.Subsequently, Drissi-Daoudi et al. (2023) developed a robust acoustic-based CNN model to classify the same defects based on spectrograms, where they demonstrated that such defects could be picked out even when caused by other process parameters that were not used in the training process.They also found that the number of parameter sets, the relative normalised distance between the training and testing set, and the position of the training sets in the process map influence the resulting classification accuracy.Similarly, Chen et al. (2023) researched acoustic features such as time, frequency, and time-frequency domain with different traditional machine learning and CNN models to identify crack and keyhole defects in laser direct energy deposition (LDED).They concluded that the Mel Frequency Cepstral Coefficient (MFCC) -CNN model outperformed all the classic ML models in their experiments.However, their approach required longer acoustic signals for defect detection than others in the literature.Wasmer et al. (2023) compared the performance of seven supervised ML algorithms to identify the lack of fusion and conduction modes of functionally graded materials based on time and frequency domain features.They found that the resulting classification accuracy was low, at around 73%, due to noise created by the gas flow surrounding the laser deposition process zones.Shevchik et al. (2019) combined a Fiber Bragg Grating (FBG) as an AE sensor with a spectral convolutional neural network to classify three quality categories based on porosity.Pandiyan et al. (2021) proposed a semi-supervised approach where two generative Convolutional Neural Networks based on Variational Auto-Encoder, and General Adversarial Networks were used to differentiate balling, lack of fusion (LOF) pores, and keyhole pores during the conduction mode melting process for LPBF.However, their model cannot be generalised across different powder distributions, process parameters, and compositions.Tempelman et al. (2022) extracted time and frequency series features from acoustic signals using Support Vector Machines (SVM) to classify keyhole pores defects in the LPBF process.Their approach identified the keyhole location using an X-ray measurement and correlated it with an acoustic signal.They found that high-frequency signals are an indication of keyhole defect formation.Recently, Kononenko et al. (2023) analysed the AE signal in the LPBF process and, based on a certain threshold with their proposed ML model, separated the AE signature of crack from noise.However, in practical application, if the noise level is high, the threshold would be high, which limits such an approach.A summary of the investigated defects and their limitations for the above-mentioned ML and non-ML papers are shown in Table 1.
The research, as mentioned above, highlighted that time domain, frequency domain, and wavelet analysis are popular techniques to process acoustic signals to determine defects.Specifically, features such as peak amplitude, kurtosis, energy, number of counts, duration, rise, peak amplitude, frequency, and Mel Frequency Cepstral Coefficient have been used to identify various defects (Surovi, Dharmawan, and Soh. 2021;Surovi, Hussain, and Soh 2022).However, while most of the published works of both ML and non-ML-based approaches show promising results for in-situ monitoring systems in laser-based AM applications, few have explored how acoustic signals can be used effectively for WAAM defect detection.Furthermore, when comparing ML and non-ML-based approaches, we believe that the ML approaches have benefits over the non-ML approaches because they can be scaled easily using a unified framework to include more defect categories if appropriate data are obtained.They also show the potential to yield a higher accuracy because they can consider multiple features in their model, unlike non-ML-based approaches, where only a single feature is considered.This is because non-ML-based approaches often require domain-expert knowledge in the form of handcrafted rules involving process parameters.Defect detection algorithms using such hand-crafted rules are highly sensitive to the rules.For instance, new rules need to be adopted when introducing a new material or part geometry (Goh, Sing, and Yeong 2021).On the other hand, an ML-based approach (e.g.neural networks) can learn the rules as long as sufficient and accurate data is provided.However, the accuracy of an ML model depends on how accurately the data set is labeled for training.Currently, the datasets reported in the literature are labeled based on visual inspection, which is subjective and error-prone.
In this paper, a methodology for constructing MLbased models to identify geometrically defective bead segments using acoustic sensing during the WAAM process is proposed.This approach targets geometric defects, a defect unique to the WAAM process, due to its sensitivity to process variation.A bead produced using a constant WAAM process is known to produce segments of varying geometrical shapes due to unaccountable process variation.Such defects are not explored in literature as the defects do not cause issues to a single bead segment element itself.However, it has implications when two overlapping bead segments are required to fuse.It contributes to voids in the final printed part due to the incomplete fusion between two overlapping bead segments, a common problem when performing multi-bead prints as shown in Figure 1.
The ML-based construction methodology proposed in this paper leverages a novel approach toward geometric defect detection through bead segment monitoring and dataset labeling to separate good and bad bead segments based on an optimal threshold of the range of mean curvature.By discretizing defect detection, it has the benefit of identifying a localised defect more accurately so that early intervention can be possible.Currently, labeling of AM defects is highly manual and is based on a visual inspection approach.This is errorprone and time-consuming, especially when the dataset is large.Here an approach to determine an optimal threshold to separate good and bad segments based on overlapping areas of KDE distribution is proposed.To understand the resulting performance of the defect detection model constructed based on the proposed methodology, experiments are performed and tested based on Inconel 718 material.(Palaz and Collobert 2015;Zhao et al. 2010).The data set labeling uses a curvature-based labeling approach that heuristically finds a range curvature threshold to separate good and bad segments.Subsequently, the acoustic features and labels are used to train five ML models, namely, the K-Nearest Neighbor (KNN), Support Vector Machine (SVM), Random Forest (RF), Neural Network (NN), and Convolutional Neural Network (CNN), to evaluate the best out of the ten defect identification models for identifying geometrically defective bead segments.

Experimental setup and data collection
Experiments were conducted on the robotic WAAM system at the Singapore University of Technology and Design (SUTD) to construct the geometric defect detection frameworks, as shown in Figure 3.
The system consists of a robot manipulator (ABB IRB1660ID), a welding power source (Fronius TPS 400i) equipped with a welding torch (Fronius WF 25i Robacta Drive), a Cartesian coordinate robot made up of three linear rails (PMI KM4510) powered by three servos (SmartMotor SM34165DT).In addition, a microphone (UMIK-1 miniDSP) is positioned at about 80 cm above the substrate for acoustic data measurement.The UMIK-1 acoustic microphone is an omnidirectional microphone that is pre-polarised.Its frequency response is 20 Hz-20 kHz and can capture sound signal sampling rates up to 48 kHz.

Data collection
Thirty-three weld beads of length 100 mm were printed using Inconel 718 (BÖHLER 3Dprint AM 718) wires.Each bead was printed using a different torch speed and wire feed rate combination to obtain different weld bead geometry.The torch speed (TS) and wire feed rate (WFR) were chosen based on the material process map that yielded good bead  the temporal resolution of all beads is not the same.Among all the bead segments, the minimum temporal resolution is around 230ms and the maximum temporal resolution is around 5 s.Therefore, downsampling (Surovi, Dharmawan, and Soh. 2021) was performed to make all the acoustic signal segments of equal length and at a sampling rate of 44100 Hz.Thus, a total of 660 acoustic signal segments for different torch speeds and wire feed rates were obtained.
Similarly, the GOM ATOS III Triple scanner was used for collecting point clouds measurement of the printed bead geometry to determine each bead segment curvature range.The point cloud data consists of both vertex and face information.Examples of the measured bead  point cloud are shown in Figure 3.Each bead was separated from its substrate using RANSEC plane segmentation and divided into 20 segments to yield 660 bead profile segments.The length of each bead segment is 5 mm.Together with the acoustic signal segments, the bead profile segments form a set of datasets for feature extraction, labeling, and construction of the defect identification model.

Acoustic feature extraction
The acoustic features consist of the Principle Components and Mel Frequency Cepstral Coefficients of the acoustic signal.In the following the mathematical background for each feature extraction technique and implementation details will be explained.Before extracting the features, the acoustic segments are denoised by wavelet threshold de-noising (Zhang, Wen, and Chen 2017).

Principle component analysis (PCA)
PCA is a popular statistical transform technique for multidimensional data.It is used extensively in the image and audio processing for reducing dimensionality and identifying signals of interest.PCA uses singular value decomposition and projects the high dimensional data into a lower dimension space (Tipping and Bishop 1999).The PCA feature extraction procedure consists of 5 steps and is given below: . Step 1: Compute Discrete Fourier Transform (DFT) on the acoustic signal x(t) using the following equation: where N is the number of points used to compute the DFT. .Step 2: Assemble the DFT signal of each segment y(k) into a matrix.
Step 3: Calculate the covariance matrix Step 4: Calculate the eigenvectors and eigenvalues of the covariance matrix using the following equation: where l i is the eigenvalue associated with the eigenvector e i . .Sort the eigenvectors in the descending order of eigenvalues.The eigenvector with the highest eigenvalue forms the 1st principal component, the eigenvector with the second-highest eigenvalue forms the 2nd principal component, the eigenvector with the third-highest eigenvalue forms the 3nd principal component, and so on.
The dimension of the DFT features used for each bead segment is N = 10495.An example of the DFT feature of a signal segment based on this dimension is shown in Figure 4 (left).The number of principal components required as a feature depends on the ML model employed for geometric defect detection.In this paper, 4 principal components yields optimal results for KNN, SVM, NN and CNN models.As for RF, it is found that 6 principal components yield the best results.

Mel frequency cepstral coefficients (MFCC)
MFCC is a popular acoustic feature extraction technique and is used for speech and emotion recognition (Sato and Obuchi 2007).Mel frequency refers to human audible range frequency.MFCCs are short-term power spectrum-based features (Logan 2000) which capture the distinguishing characteristics of sound.The MFCCs feature extraction procedure (Rao and Manjunath 2017) from acoustic signal consists of 7 steps and is given below: (1) Step 1: Convert each segmented signal into frames.
(2) Step 2: On each frame, apply a window function (e.g. Hamming window) to get a windowed signal.(3) Step 3: Compute Discrete Fourier Transform (DFT) on the windowed signal using Equation (1).(4) Step 4: Compute the Mel-filter bank, which is a set of band-pass filters.The filter bank is a nonlinear-scale filter bank that imitates a human's audible system.Most of the filter shape is triangular.The filter banks are implemented in the frequency domain for MFCC computation.The conversion of physical frequency to Mel frequency is given by where f is the physical frequency in Hz and f Mel is the human perceived frequency.(5) Step 5: Calculate filter-bank energies by multiplying the square of the magnitude spectrum of the DFT signal with each filter-bank: where M is the total number of triangular Mel filters The MFCC features can be computed using the librosa python package 1 .There are three parameters whose value needs to be set: frame size, hop length, and the number of filter banks, and those used in this paper are 2048, 512, and 13, respectively.The reason for setting the frame to 2048 is that if a frame size were smaller than this, the number of samples would not have been enough to get a reliable spectral estimate for defect detection.Similarly, if the frame size is larger than 2048, it may cause frequent changes in the information inside the frame, making the signal not stationary.As for the hop length, 512 is chosen to avoid any information loss at the beginning and end of the frame.Since the first few MFCC coefficients contain the majority of sound signal information compared to the rest (Gupta et al. 2013), 13 filter banks that give 13 MFCC coefficients are deemed sufficient for feature extraction.An example of the resulting MFCCs feature of a signal segment based on these parameters is shown in Figure 4 (right).

Dataset labeling based on range of mean curvature
The curvature at a point of a bead segment surface indicates how much it curves.The higher the curvature, the sharper the curve 'turns' at that point (Do Carmo 2016).The range of "mean curvature" of a segment, which we term range curvature, indicates how much the curvature changes in a bead segment.If the range curvature is small, the bead segment has a smooth or uniform surface.Otherwise, it indicates a non-uniform bead segment with some geometric defects.See Figure 5. Notice that the good bead segment has a smaller range curvature at 0.45 mm while the bad segment with a geometric defect (circled in red) has a higher range curvature at 1.2 mm.
To label the dataset, the range curvature of all bead segments needs to be computed and followed by a heuristic search to determine an optimal threshold to separate the good and bad segments.This can be achieved by searching through a set of range curvature threshold values.The optimal threshold value is such that the overlapping area formed by the Kernel Density Estimates (KDEs) of the good and bad bead segments yields minimal overlap.Based on this optimal threshold value, any bead segments above this threshold are labeled as bad.Otherwise, they are good.

Heuristic-based search for a range of mean curvature threshold
In the following, the details of the heuristic-based range curvature threshold search technique will be explained.Let S = (s 1 , s 2 , . . ., s i , . . ., s n ) be the sequence of bead segments, and p i1 , p i2 , . . ., p ij , . . ., p im be the set of points on a particular bead segment s i .The parametric spherical surface at each point on segment s i is approximated using the moving least squares (MLS) method (Guennebaud and Gross 2007).
To compute the mean curvature, κ, at each point of this parametric surface, let k ij be the mean-curvature at jth point p ij on segment s i .The mean curvature (Jia 2020) at this point, can be found as the average of the principal curvatures, k 1 and k 2 .
Such computation can be easily performed using MeshLab (Cignoni et al. 2008) with a colourise curvature (APSS) filter.The steps to determine the optimal range curvature threshold consist of 3 steps, which are given below.They consist of Range Curvature Parameter Metric Computation, Density Estimation, and Heuristic Search.
(1) Step 1: Let RC i be the range of mean-curvatures of segment s i .Compute range of mean-curvatures (RC 1 , RC 2 , . . ., RC n ) for n segments using (2) Step 2: Let G i = {s j [ S:RC j ≤ RC i } be the set of good bead segments and B i = {s j [ S:RC j .RC i } as the set of bad segments for range curvature RC i .Estimate the distribution of feasibly good segments ˆf (G i ) and bad segments ˆf (B i ) using the kernel density estimates function provided by SciPy 2 library in python.(3) Step 3: To find the optimal RC i * , heuristically search through various RC i thresholds.The optimal RC i * is the value that minimises the overlapping area between two KDE distributions of ˆf (G i ) and ˆf (B i ).
An example of the resulting labeled data is shown in Figure 6.Note that the identified range curvature threshold is 0.85 (Section 3.1) and bead segments 1-3 and 18-20 are deemed defective segments.

ML models for geometric defect identification
The proposed geometric defect identification models built on the five ML models: K-Nearest Neighbor (KNN), Support Vector Machine (SVM), Random Forest (RF), Neural Network (NN), and Convolutional Neural Network (CNN).In the following, the background for each model and implementation details will be explained.

K-Nearest neighbor (KNN)
KNN is a supervised learning technique.The algorithm selects the Kth nearest points of a new example by calculating the distance between all existing examples with the new example.The shortest distance from the new example determines the Kth nearest neighbours.The class of a new example is assigned to the majority class of Kth nearest points.For this application, the number of nearest neighbours is proposed to be K = 11 for PCA features and K = 31 for MFCC features.These are selected as they yield minimum mean validation loss over the range of K explored.See Section 3.2.1 for the experiments leading to such a choice.

Support vector machine (SVM)
SVM is a discriminative classifier that creates a line or a hyper-plane to separate data points into different classes.A non-linear SVM was employed to classify the bead segments into good and bad based on their input features and corresponding labels.The goal of SVM is to search a function f (x) with parameters a i and b (Platt 1999), which can predict defective segments of beads.A general format of SVM is shown and the RBF kernel (radial basis function kernel) used is For this application, the hyperparameter γ is selected to be 0.7 for PCA features, and 0.01 for MFCC features.See Section 3.2.2 for the experiments leading to such a choice.

Random forest (RF)
RF is a supervised ML technique with many decision trees as its building blocks.RF is suitable for modeling non-linear and complex systems.It is generally unaffected by outliers and noise and involves a faster training process.RF learns to build the relationships between the inputs and outputs in the training stage.
Once the training is done, a new example is presented for testing.Each tree in the trained RF then votes for classification (Breiman 2001).The classification with the most votes is used as the predicted classification.For this application, the number of trees is selected as 150 for the PCA features and 60 for the MFCC features.See Section 3.2.3 for the experiments leading to such a choice.

Neural network (NN)
NN combines input layer neurons, hidden layer neurons, and output layer neurons, where two neurons are connected by weight.A set of input data is used to train the NN to optimise its desired output.The training process is done by adjusting the weights between the connected neurons to minimise the loss (Wang 2003).
The architecture of the proposed NN for defective segment identification comprises one input layer, four hidden layers and one output layer, as shown in Figure 7. Inputs to the NN are the extracted PCA features or MFCC features.A Rectified Linear Unit (ReLU) activation function (Nair and Hinton 2010) is used in all hidden layers.The L 2 regularisation is chosen as 0.001 and is used in every hidden layer to avoid the overfitting problem (Cortes, Mohri, and Rostamizadeh 2012).See Section 3.2.4 for the experiments leading to such a framework.

Convolutional neural network (CNN)
CNN is analogous to traditional NN, comprised of convolutional, pooling, and fully connected layers.The main difference between CNN and traditional NN is that the input of CNN comprises images, and the neurons of CNN layers are three-dimensional-height, width, and depth information to the images (O'Shea and Nash 2015).
The architecture of the proposed CNN for defective segment identification comprises one convolutional layer, one pooling layer, one fully connected layer, and one output layer, as shown in Figure 8.Both the PCA and MFCC features are converted into input images.The convolutional layer consists of 13 and 5 filters for PCA and MFCC feature-based models, respectively.The size of each filter is 3 × 3. The Fully Connected (FC) layer contains 64 neurons to connect all the activation of previous layers.One dropout (0.3) is added after the FC layer to avoid overfitting.The final architecture is optimised with a suitable stride of 2 × 2 and regularisation (ReLU) of 0.001.One convolutional layer was sufficient for this application, as having two yields no significant accuracy improvement.See Section 3.2.5 for the experiments leading to such a framework.

Experimental results
In this section, experiments are conducted to determine the range curvature threshold for dataset labeling and the hyperparameters for the various proposed ML models for identifying geometrically defective bead segments during the WAAM process.Using the hyperparameters, a set of ML models is trained based on the identified feature parameters.A comparative study of all the trained models is subsequently evaluated for their defect identification performance.

Threshold selection for dataset labeling
The dataset labeling seeks to separate the dataset into good and bad bead segments based on their range curvature.As explained previously, this can be achieved by searching through a set of range curvature threshold values that yield minimal overlapping area formed by the Kernel Density Estimates (KDEs) of the resulting good and bad bead segments.Figure 9 shows the KDE distribution plots for range curvature thresholds of 0.55, 0.85, and 0.95, and the relationship of different range curvature thresholds with their associated overlapping areas.Notice that each of the KDE plots yields different overlapping areas.The overlapping area is minimal when the range curvature threshold is 0.85 mm.Thus, based on this threshold, any bead segments above this value are considered bad, and vice versa.
The dataset after the curvature-based labeling consists of 415 (62%) good bead segments and 245 (38%) bad bead segments.

Choice of hyperparameters
In the following, various experiments are conducted to determine the set of hyperparameters suitable for the KNN, SVM, RF, NN, and CNN defect detection model.

Hyperparameters for KNN
Figure 10 shows the effects of the KNN hyperparameters on the resulting classification error for both the PCA and  MFCC features.The hyperparameters include the number of nearest neighbours K and the weight function.To find the optimal K and required weight function, experiments are conducted to determine the classification error over a range of K and weight function.Two weight functions, the 'uniform' or 'distance', are explored.Five-fold cross-validation is performed on the training dataset for K = 1, . . ., 50 with the two weight functions, and the average validation classification error based on the PCA and MFCC features is plotted.The results show that the lowest classification error with PCA features is around 0.156 when K = 11 with a 'uniform' weight function.Similarly, the lowest classification error with MFCC features is around 0.138 when K = 31 with a "distance-weight function.

Hyperparameters for SVM
Figure 11 shows the effects of the SVM hyperparameters on the resulting classification error for both the PCA and MFCC features.The hyperparameters include a parameter γ that determines how far the influence of a single training example reaches, the regularisation parameter C, and the kernels.To find the optimal γ, C,

Hyperparameters for RF
Figure 12 shows the effects of the RF hyperparameters on the resulting classification error for both the PCA and MFCC features.The hyperparameter consists of the number of trees used.Similarly, to find the optimal number of trees, five-fold cross-validation is performed on the training dataset as the number of trees varied from 1 to 350.The average validation classification error is plotted using PCA and MFCC features.The results show that the lowest classification error with a PCA feature is around 0.160 with 150 trees.Similarly, the lowest classification error with MFCC features is around 0.149 with 60 trees.

Hyperparameters of NN architecture
Figure 13, Tables 2, and 3 show the effects of the NN hyperparameters on the resulting classification accuracy for both the PCA and MFCC features.The hyperparameters consist of the number of layers and neurons in each layer.Experiments are conducted to determine the validation accuracy as the number of layers and neurons varies to find the optimal number of layers and study the impact of the number of neurons in each layer.Five-fold cross-validation is performed on the training dataset as the number of layers varied 12. RF classification error based on the number of trees for PCA features (left plot) and MFCC features (right plot).
from 1 to 5. The average validation classification accuracy is plotted based on PCA and MFCC features.The results show the 4-layer NN provides the highest classification accuracy based on the PCA and MFCC features.
Similarly, different combinations of neurons within a 4-layer NN are studied.In general, it is found that the validation accuracy is insensitive to neuron changes, with about 1% change based on the combinations tried.For PCA features, the 4-layer neurons combination of (256,128,64,32) provides the highest classification accuracy.Similarly, for MFCC features, the 4-layer neurons combination of (256,128,64,16) provides the highest classification accuracy.

Hyperparameters of CNN architecture
Figure 14 shows the effects of the CNN hyperparameters: 'the number of filters in the convolutional layer' on the resulting classification accuracy for both the PCA and MFCC features.To find the optimal number of filters, experiments are conducted to determine the classification accuracy as the number of filters varies.Fivefold cross-validation is performed on the training data set as the number of filters varied from 1 up to 40.The average validation classification error is plotted based on PCA and MFCC features.The results show that the highest classification accuracy with a PCA feature occurs at 13 filters.Similarly, the highest classification accuracy with MFCC features occurs at 5 filters.

Training mechanism
With the set of hyperparameters for the various ML models determined in the earlier sections, a five-fold cross-validation (stratified k-fold) (Cawley and Talbot 2003) is used to train the defect detection models based on PCA and MFCC features, using the Inconel 718 bead segment dataset.For training on the NN and CNN model, adam optimiser (Kingma and Ba 2014) with sparse categorical cross-entropy loss, and the value of 0.0001 is used as the learning rate.

Defect identification models training based on PCA features
Figure 15 shows the average validation accuracy for each of the defect identification models as the principal components of the acoustic signal varied from 1 to 25.
Note that for the number of principal components taken as a feature, the average accuracy is based on   the average of 30 randomly selected dataset of 60% training, 25% testing, and 15% validation.As shown from the results, the average validation accuracy increases at the beginning and reaches around 87% to 90% (depending on the type of ML model used).It shows that the first four components of PCA when combined as a feature, yield the most accurate prediction for KNN, SVM, NN, and CNN models.Similarly, the first six components of PCA when combined as a feature, yield the most accurate prediction for RF model.
Figure 16 shows the loss curve for NN and CNN models.Notice that both the training and testing loss decrease the number of epochs increase.It seems that after 35 epochs, both training and validation loss become stable with a small gap, indicating that the model had been trained without over-fitting.Thus, the training stops at 35 epochs.

Defect identification models training based on MFCCs features
Figure 16 shows the loss curve for the NN and CNN models.Notice that both training and testing loss decrease as the number of epochs increases.Similar to the PCA features, both training and testing loss become stable with a minimum gap after 35 epochs, and no overfitting are observed.Thus, the training stops at 35 epochs.

Performance evaluation of defect detection model based on F1 score, precision, recall and confusion matrix
To test the resulting trained defect detection model performance, the various models are evaluated based on  their F1 score, precision, recall, and confusion matrix.The reason for using these metrics is that they are useful in evaluating the performance of an imbalanced dataset.Table 4 shows the comparative study performed on the resulting defect identification models based on F1 score, precision, and recall.
It is observed that the MFCC feature with an SVM model (MFCC+SVM) has the highest F1 score among all the defect identification models, at around 84%.
Table 5 shows the comparative study performed on the resulting defect detection models based on the confusion matrix, which measures the truly good and bad segments picked out by the models.It shows that the PCA feature with a KNN model (PCA+KNN) can pick out the most number of true bad bead segments at 186.

Conclusion
In this paper, a methodology for constructing the geometric defect detection model to identify defective bead segments using acoustic sensing for the WAAM process is proposed.The performance of ten of such models is evaluated, and the results have demonstrated they can able to identify geometrically defective segments accurately (80% to 85%) when tested on the Inconel 718 dataset.Among the various trained defect detection models, it is found that having MFCC acoustic features with support vector machine performs the best in terms of F1 score, and having principal components as an acoustic feature with K nearest neighbour performs the best based on the confusion matrix.The first novelty of this research lies in the defects that are targeted.Geometric defects are a unique problem in WAAM due to their sensitivity to process variation.A bead produced using a constant process parameter would produce bead segments of different geometrical shapes due to environmental factors.This would lead to voids and affect the final part performance if left untreated.The second novelty lies in the segment-wise geometric defect detection approach.By discretizing defect detection, it has the benefit of identifying a localised defect more accurately so that early intervention can be possible.The third novelty lies in the dataset labeling approach of geometric defects, where good and bad bead segments are separated based on a threshold of the range of mean curvature determined through a heuristic search.Currently, labeling for AM defects is based on a visual inspection approach, which is error-prone and time-consuming when the dataset is large.Here an approach to determine an optimal threshold for labeling based on overlapping areas of KDE distribution is proposed to separate good and bad segments.For future work, the authors intend to explore the inclusion of other features, such as the WAAM process parameters, to further increase the defect detection performance.The goal is to implement this approach for defect detection during real-life printing.

Figure 2
Figure 2 shows the methodology of constructing the geometric defect detection model.The approach consists of three elements: acoustic feature extraction, dataset labeling, and defect detection modeling.Two feature extraction techniques: the Principal Component Analysis (PCA) and Mel Frequency Cepstral Coefficients (MFCCs) are explored for their suitability in identifying geometric defects due to their performance in solving other domain-specific problems (Palaz and Collobert 2015; Zhao et al. 2010).The data set labeling uses a

Figure 1 .
Figure1.Schematic representation of a multi-beads printing (left) and Actual printing (right) as seen from the build direction.Geometrically defective segments lead to voids between two successive beads will affect the final printed part strength and quality.
formation and was found to be in the range of [1,20] mm/s and [2,8] m/min, respectively.The shielding gas used a composition of 70% Ar and 30% He, and at a constant gas flow rate of 25 L/min.The nozzle-to-work distance was set at around 15mm.For each bead print, acoustic signals were acquired at 44k Hz and divided into 20 signal segments.Since the different combinations of torch speed and wire feed rate produce different acoustic signals length,

Figure 2 .
Figure 2. The methodology for constructing the geometric defect detection Frameworks.

Figure 3 .
Figure 3. Experimental setup of SUTD Robotic WAAM for Bead Geometry and Acoustic Data Collection (left).(a) Original beads with plate (b) Mesh representation of beads with plate (right).
and H m (k) is the weight given to the kth energy spectrum bin contributing to the mth output band.(6) Step 6: Calculate the logarithm of the energies.(7) Step 7: Apply Discrete Cosine Transform (DCT) to the log filter bank energies and produce the cepstral coefficients for each frame.Traditionally, 8 to 13 coefficients are selected.The equation for getting coefficients is given below: s(m)) cos (pn(m − 0.5)/M) (6)where n = 0, 1, 2, . . ., C − 1, c(n) are the cepstral coefficients, and C is the number of MFCCs.

Figure 4 .
Figure 4. DFT features of a segmented signal (left) MFCC features of a segmented signal (right).

Figure 5 .
Figure 5. Range of mean curvature of geometrically good (left) and bad (right) bead segment.The red circle indicates a geometric defect.

Figure 6 .
Figure 6.Bead with defective and non-defective segments (Top) with corresponding acoustic time-domain waveforms (Middle) and range curvature (Bottom).

Figure 7 .
Figure 7.The proposed NN Architecture for defective bead segments identification.

Figure 8 .
Figure 8.The proposed CNN Architecture for defective bead segments identification.

Figure 9 .
Figure 9. Distribution plots for good and bad segments of beads for different thresholds (First three plots).Different thresholds vs overlapping area plot(last plot).
and required kernel function, experiments are conducted to determine the classification error over a range of γ, C, and kernels.Two kernels based on Radial Basis Function (RBF) and Sigmoid are explored.Fivefold cross-validation is performed on the training dataset for g = 0.01 . . .1.5 and C = 0.01 . . .1.5 with the two kernel functions.The average validation classification error is plotted using PCA and MFCC features.The results show that the lowest classification error with a PCA feature occurs when g = 0.7 and C = 1.1 with an RBF kernel.Similarly, the lowest classification error with an MFCC feature occurs when g = 0.01 and C=1.48 with an RBF kernel.

Figure 10 .
Figure 10.KNN classification error based on the number of nearest neighbour K and weight function for PCA features (top plot) and MFCC features (bottom plot).

Figure 11 .
Figure 11.SVM classification error based on the choice of γ and C for the RBF and Sigmoid kernels with PCA features (top) and MFCC features (bottom).

Figure 13 .
Figure 13.NN Classification accuracy based on the number of layers of a NN architecture for PCA features (left plot) and MFCC features (right plot).

Figure 14 .
Figure 14.CNN classification accuracy based on the number of filters within its architecture for PCA features (left plot) and MFCC features (right plot).

Figure 15 .
Figure15.Average validation accuracy for the different defect identification models vs the number of principal components used as a feature.

Figure 16 .
Figure 16.The loss curve for PCA+NN and MFCC+NN (top) The loss curve for PCA+CNN and MFCC+CNN (bottom).

Table 1 .
Prior work concerning acoustic defect detection in metal AM.

Table 2 .
Different Combination of number of neurons for PCA +NN.

Table 3 .
Different Combination of number of neurons for MFCC +NN.

Table 5 .
Comparative study of all of the proposed defect identification model based on Confusion Matrix.