Ply-by-ply inline thermography inspection for thermoplastic automated tape layup

Abstract Automated tape layup (ATL) largely employs post manufacturing manual visual inspection techniques for defect detection, which severely affects the productivity. Inline monitoring and defect prediction can help in making the process faster and more reliable. The presented work details the use of thermography as an inspection tool for thermoplastic tape material. A new online monitoring system is developed containing Infrared camera integrated on a purpose build ATL test rig. The capability of the tool to identify various defects is analyzed. Moreover, detailed temperature and cooling behavior analysis is done for defect prediction. Graphical Abstract


Introduction
Growing demand for composite manufacturing requires processes with high productivity. Automated tape layup (ATL) is one such process that promises high level of automation and cost effectiveness. The process has been widely used in high performance industries like aerospace for production of composite laminates from unidirectional prepregs [1][2][3]. Traditionally used for thermosets, thermoplastics based flat laminates have continuously been researched due to their easy formability, enhanced mechanical properties and the possibility to have in-situ consolidation, with no requirement for post processing [4].
Even though the process is highly automated, defects are a common occurrence whether due to material inconsistency or manufacturing variability.
To maintain constant quality, defects have to be recognized and remedied. State of the art for detection is limited to manual visual inspection, which is labour and time intensive. One study showcased the downtime comprising of material refilling, error correction and cleaning, and can be as high as 50% [1]. Another study reported that the downtime for inspection and repair is between 32% and 63% of the overall production time [2]. According to another study [5], of the total machine cycle time required for a fuselage section, only 32% comprises of deposition time, 41% of the overall process duration is needed for manual quality assurance and rework. This is the case when the defects can be detected before curing/consolidation. Of course, not all flaws can be detected visually, undetected flaws lead to higher repair costs and in worst case, rejection of the part causing further material, monetary and productivity loss [2,3,[6][7][8][9].
A monitoring tool, capable of in-situ inspection with early detection, quantification and localization capabilities is needed for process monitoring and quality assurance. Our research group has developed a ply-by-ply online monitoring and inspection tool and the presented work elaborates the effort to check the capability and feasibility of the tool.

Defect types
A multitude of defects can arise during the layup which can be categorized either by cause of occurrence or method of detection. Common defect types classified by cause of incidence and crucial for quality assurance according to DIN29971 [10] are: positioning defects like gaps, overlaps, missing tows, twisted tows; bonding defects such as bridging, air pockets; foreign object defects (FOD) such as fuzzball and tow defects such as splice. A summary of defects being capable of detected inline are shown in Figure 1.
A review of defects is also provided by [3,5,11]. While [3] talks about selective defects, effects and their identification [5], divides the defects into primary and secondary and summarizes the potential secondary effects that may accompany or result from of manufacturing induced primary imperfections. There are defects that cannot be seen by naked eyes and hence cannot be detected during manual inspection. Some examples being, uneven bonding/de-bonding, voids, degradation, residual stress and surface roughness variation. Some of them are discussed in [3]. The cause, anticipation, existence and significance entailing the defects are discussed in [11].
Positioning defects, causes and their influence mechanical properties including the tensile, compression, shear, fatigue and vibration properties are discussed in detail in [12]. Experimental and simulative study of tow steering defects is detailed in [13]. A review of the effects of defects on mechanical properties can be found in [3,11,12]. While Oromiehie et al. [3] discuss the individual effect of each processing parameter on the quality of the laminate along with the review of experimental and analytical results of such defects, the significance and progression of each defect is described by Harik et al. [11]. Sun et al. [12] present a review of positioning defects in conventional and variable stiffness laminate and various methods to control the defects. The experimental investigation of effects of gaps and overlaps is conducted by Woigk et al. [14], with a distinction between lamina and laminate level by Croft et al. [15]. Zenker et al. [16] conducted statistical analysis with multiple null-hypotheses investigating the effects of consolidation process, defect type, defect size and stacking sequence on the mechanical performance of the thermoplastic laminate. Although the consensus is that defects are detrimental to the mechanical performance, some experimental results point otherwise. In some cases, a defect can improve one property while retarding the other. Therefore, based on the mechanical requirements and ease of controllability, defects can either be selectively placed or avoided altogether.
The quantitative and qualitative characterization of defects caused during processing, their significance and progression analysis, will help in prognosis analysis to enable damage progression models capable of predicting remaining life accurately. For delamination defects, depth analysis is indispensable, as depth is directly related to damage progression [17]. A thorough ply by ply inspection having size Figure 1. Summary of defects capable of being detected inline via surface or edge detection. and depth information related to defects must be acquired for such an analysis. Based on the significance and severity of progression of the defect, damage mitigation strategies can then be adopted.
Most of the defects are caused by optimized parameter selection, machine variability and material variability. The defects arising due to machine and material variability itself can be characterized but are hard to control. Parameters on the other hand can be selected based on desired output, degree of consolidation, crystallinity level, degradation limit, residual stress limit, given a good understanding about parameter relation and the output. Process temperature history also influences crystallinity, residual stress, void content and degradation [18,19]. This is especially important if considering insitu consolidation for thermoplastics. Temperature history a direct influence on consolidation homogeneity and internal part quality. There are some commercial software solutions that help in selecting optimum parameters. These solutions are however based on static offline programming of process equipment, with no feedback control. Since the control is based on kinematics of the layup head, they are not flexible enough to have a material adaptive temperature control [20,21]. Temperature history based control will therefore increase both productivity and quality of the laminate [20].
Improved automated inspection enabling the manufacturer in finding flaws and reworking on them if required prior to placing another layer will help in cost, material and time saving [6]. Implementing reliable process window based on advanced temperature prediction will also help in improving part quality [2,20].

Defect detection
Over the years, many different online defect detection methods have been researched upon, but as discussed earlier, most of the present systems still employ time and cost intensive visual inspection [6]. Based on the defect type defect detection techniques can broadly be divided into two groups, surface defect detection and internal defect detection.
For internal defects, especially residual stress, a number of embedded sensor techniques have been used. Some of them being, embedding metallic particles in combination with X-ray diffraction [22], using strain gauges or stress wave [23,24]. Optical coherence tomography (OCT) can detect both surface and internal defects [12]. More recently, fibre bragg gratings (FBG) have been demonstrated to be able to detect gaps and overlaps based on temperature and strain detected by the sensor. They can be used for both online monitoring and structural health monitoring (SHM) but, the embedding difficulties, cross-sensitivity to temperature and, strain and low temperature resistance reduce the ease and range of operation [25].
Surface detection is more common, the simplest utilization of which is a camera system mounted on the ATL head, which continuously takes images and analyses them. The reliability of such a system is dependent on intensive illumination to differentiate between black single tow and already laid up plies, complex image analysis and concentration of the operator. Since the detection quality depends on ambient conditions, the reliability is questionable [2]. A slightly different approach makes use of laser profilometry or laser triangulation sensors. They rely on edge detection and can easily localize the position of the tow, but do not deliver as much information about the layup quality as the camera [2,6]. A combination of the above two results into a laser-vision inspection system. A laser projector scans and then projects predefined geometrical shapes on the inspection area. This is captured in an image by the vision system. 3D ply location, gaps and the fiber orientation can be detected by such a system. Similar to a camera, the usage is limited by the reflective properties of tapes and precise image projection onto highly contoured parts [26]. Digital image correlation (DIC) is another technique used to inspect surface defects [27,28]. It was demonstrated that 3D DIC can be used for ply inspection during manufacturing for defects like gaps, laps, twisted tow and damaged tow. However, an optical random pattern projector needs to be developed for this purpose [27].
Most of the above mentioned techniques are either intrusive or require special working environment and tools. Infrared (IR) thermography is however, a non-invasive, non-contact technique and thermal contrast helps in easy identification of inhomogeneity during layup, thus helping in identifying the defects with ease. Large aerospace components made up of aluminum, composites and hybrid fibre metal laminates make use of such IR inspection. The technology has been shown capable of detecting a variety of defects like delamination, porosity, fibre/matrix cracking, thermal stress cracking, interlaminar disbond and impact damage for carbon fibre reinforced composites (CFRP) [29,30]. Other advantages of this method are, coverage of large surface area and ease of inspection without the need to couple the whole volume [31]. The fast pace and accuracy makes it suitable for ATL process. The most prominent disadvantage is difficulty in identifying deeply embedded defects, which will be deemed unnecessary if using ply-by-ply online inspection.
Patents have been filed for online thermographic tool (US patent 7513964 B2 [32], by Boeing), but practical realization has only been achieved at individual research institutes. No industrial use has been reported so far [2,6]. Research done by Gregory et al. [6,7], Schmidt and Denkena et al. [2,9,10] use similar approach adopted in this paper but lack either holistic data analysis or system behavior identification. This paper tries to fill in that gap that can serve to actively control the process.

Process description
This research work focuses on thermoplastic tape laying. The major control variables that influence the quality of the layup are heat, speed of layup and pressure [33]. ATL requires continuous layup of material with simultaneous application of force and heat. Tapes are brought together and compressed, resulting into intimate contact. With continuous application of heat, healing of the tape in molten state results into intermolecular diffusion. As the tapes cool down, consolidation follows [34]. Based on the material and processing requirements, a variety of heat sources are employed for ATL processes like hot gas torch [25], laser [35] and LED heating [36] among others. The present work utilizes, a radiation based pulsed light energy system, humm3 as the main heat source.

Monitoring tool
The thermal camera records the radiation of the specified region. The thermal contrast of the surroundings and the lay-up process provides a visual representation of surface temperature of the part. This surface history when collected over time, with a priori knowledge of heating conditions, provides temperature information about the volume [5]. The thermal contrast is affected by compaction force, lay-up speed, heat input by the heat source, material and temperature of compaction roll and layup tool material and temperature. Other influences include process kinematics and environmental effects [21].
According to method of detection, defect recognition can be categorized as, edge detection and surface detection. Edge detection helps in identifying positioning defects like end of tow, twisted tow, gaps and overlaps using tow geometry details. The research facility at Montanuniversit€ at Leoben (MUL), utilizes a laser triangulation sensor for edge detection. Along with the defects detected by edge detection, surface detection can further be used for recognizing bonding defects, tow defects and foreign bodies [2]. The IR camera takes use of surface detection.
As the compaction roll is cold compared to the substrate/incoming tow, the temperature contrast is highest right behind the compaction roll, rendering it most suitable for tow localization.
If all the process parameters and machine kinematics remain constant, any inhomogeneity in the surface temperature would be the result of a defect. This would influence the heat transfer from the substrate to the tow surface and in turn the bonding. Same applies for foreign bodies. As different objects have different optical and thermal properties, any inclusion of foreign bodies will result in a change of surface temperature [2]. By defining a region of interest (ROI), and a threshold based on average surface temperature, temperature anomalies as hot or cold spots can be detected and localized [37].
A new online monitoring system is developed by MUL containing Infrared camera integrated on a purpose build ATL test rig. Post consolidation, inline ply-by-ply inspection is done. Surface thermal history for the layup course over time is recorded, which is then extracted along the width of the tape (ROI). The end result is a single image containing sequence of the extracted line detailing the temperature data over the length of a single tape. Temperature gradient throughout the layup is then used to recognize positioning, foreign bodies and bonding defects. The presented work details the capability and feasibility of this tool.
The IR camera is mounted behind the compaction roll, having horizontal view as shown in Figure  2 (right). This position was most suitable to avoid excessive reflections. The setup is shown in Figure  2. Inline measurement is done in the online mode by accessing the camera data directly via a Software Development Kit (SDK), which was provided with the camera. The recorded infrared images are analyzed by the software developed by MUL, in order to locate defects. The software can be used during the measurement (online) or the recorded infrared sequences can be evaluated later on in offline mode. The details of the evaluation technique and the Graphics User Interface (GUI) in both online and offline mode are explained in detail in [38]. Some of the features used in this work are described below.
1. Localization: To automatically locate the defects. Two consecutive images are extracted directly after the consolidation roll and their difference image (x-gradient) is used for object location/segmentation. The objects are located using edge detection (Canny operator). The accuracy relies on the contrast between the two consecutive images. This is demonstrated in Figure 3, where the objects are automatically located and the shape is outlined with a red boundary. 2. Rectification: To analyze the cooling behavior as a function of distance from the compaction roll, the conic field of vision from the camera has to be made rectangular. Rectification is used to remodel the perspective. The extracted images are then shifted accordingly to align at the same x position (length). The points for such rectification can be chosen interactively, and the user sets the dimensions of the rectangle. It should be noted here that the accuracy of the software is directly related to accurate measurement of the distance of camera from the consolidation roll. An example of this feature can be seen in Figure 4.
3. Statistics: Temperature history is recorded in the form of statistics (mean, standard deviation, skewness, kurtosis) and is used to analyze anomaly in temperature. Temperature history is calculated across the y-position (width) for all x-positions (length). The temperature history of an ideal layup (free of defects) can also be used for defect prediction.

Process specifications
The monitoring unit comprises of an InfraTec camera, with a 25 mm lens. The camera captures images at 640 Â 512 pixel and has a typical resolution of 20 mK for temperature (Noise Equivalent Temperature Difference, NETD). For the present work, the spatial resolution is $4.8 pixel/mm based on the position of the camera. The measurements for presented work are recorded at 100 Hz and 200 ls integration time. Thermoplastic unidirectional CF-PA6 tape manufactured by SGL Carbon SE having a width of 25.4 mm and a melting point of 220 C is used. A soft conformable silicon roll having a diameter of 50 mm and 70 shore A hardness is used. The layup speed is kept constant at 50 mm/s. A compaction force of 200 N is applied throughout the process. The heat output of the humm3 system is maintained at 2.9 KW. The layup tool is made out of aluminium and is temperature controlled.
All results shown are with emissivity set to 1. Extensive experiments were done to find the true emissivity value for the carbon fibre tape and the following relationship is found as shown below. Where e denotes emissivity and T denotes surface temperature recorded with emissivity set to 1, in C e ¼ 1:0026 À 0:0003 Ã T This emissivity value can be used to find the exact temperature values. Since the aim of this research is to get comparative data, to find temperature anomalies, this conversion is not done. The average emissivity over 30 C to 150 C (temperature range for 200 ls integration time) is 0.91. To determine emissivity, a layer of CF-PA6 tape was placed on the layup tool, which was set to a starting temperature of 30 C. A fine wire K type thermocouple with a diameter of 0.05 mm was soldered into the surface of the tape to ensure the best possible contact with the surface. The temperature of the layup tool was increased in steps of 20 C (up to 150 C) and the subsequent temperature readings from thermal camera (with emissivity set to 1) and thermocouple were recorded. A linear regression between temperature values gives the relation for emissivity.
For the present research, a single ply is used to denote a single tape and tow defects for automated fibre placement (AFP) are analogous to tape defects where applicable, and the terms are used interchangeably.
A remark should be made here, that no interference was observed due to the humm3 system. The CF-tape and consolidation roll blocked direct effect of the pulsed light. Moreover, the spectral range for humm3 system is between 200 and 1000 nm range which is out of the range of the IR camera, having spectral range between 2000 and 5000 nm.

Artificial defect detection
Artificial defects are introduced in unidirectional ply-on-ply layup. A description of the defects is given below. A comparison is also made between post-consolidation volume active thermography and ply-by-ply inline thermography and to assess their ability to recognize various defects and validate the tool.
For this research work pulse/flash active thermography is used. Inline thermography does not require any external stimulus as enough thermal contrast is available during the process itself. Active thermography requires an external thermal stimulus to produce thermal contrast to visualize the defects. Short powerful flash with broad frequency range can be used to retrieve information at different depths [7]. The detection technique is sensitive to surface geometry variation and reflectivity from the environment [9]. The camera used for analysis has a resolution of 640 Â 512 pixel/ 20-50 mK. Typically, 6 mm Â 6 mm defects can be detected. Depending on camera position and sample thickness, detection of smaller defects is also possible.
Threshold of large and small defects according to aerospace standards are 7.35 mm for large defects for both gaps and overlaps, 2.5 mm for small gap and 1 mm for small overlap [16]. The foreign object inclusions for defect introduction are chosen to conform to those standards.
For artificial defect inclusion, a single layer of tape is laid on layup tool maintained at 30 C. Defects are placed on top and another tape is laid. The defects introduced from left to right are. 2.5 mm wide carbon fibre tape, 6 mm Â 6 mm Kapton tape, 6 mm Â 6 mm flash tape and 7.35 mm wide carbon fibre tape, as shown in Figure 3. The occurrence of these defects is probable during the running process.

Temperature analysis
Ideally, the temperature over the course of the layup having constant process parameters and machine kinematics should stay constant. Due to the resistance in heat flow through thickness (and ATL head heating up with each layup) of a composite laminate, the substrate temperature depends highly on the previously laid layers. For a laminate with several plies, this effect will be pronounced and can eventually lead to hot spots or degradation in worst case. It is therefore necessary to take thermal prediction or experimental thermal history into account for path planning and homogenous laminate quality [21].
Experiments are conducted to analyse temperature progression through thickness over the course of the layup to be used for temperature prediction and defect control. Cooling behaviour as a function of distance from the compaction roll is also analyzed.

Foreign object inclusions
For a constant heat flux, homogenous temperature over the layup is expected unless some flaw is encountered. High thermal conductivity and large surface are of the tool helps it in dissipating heat quite quickly. When substrate is laid on this tool, the composite layer having comparatively lower thermal conductivity will restrict the heat flow. As the laminate thickness builds up, this leads to a non-linear increase in the surface temperature of the subsequent layers. Therefore, when defects like tape slit, missing tape, and gap appear as cold spots as the substrate is colder. Splice and overlap further restrict heat flow to the tool and lead to hot spots. Based on the conductivity, foreign object inclusion can appear as either cold spots or hot spots.
The carbon fibre tape here acts as splice/overlap and the temperature is $20 C higher than the surrounding tape, shown in Figure 4. This is analogous to placing a third layer and the temperatures are in accordance with Figure 7 (left image). The Kapton tape and flash tape also show up as hot spots and have temperature >10 C than the composite tape. The high temperature of FOD compared to the laid tape, eases localization, shown in Figure 3.
The cooling behavior of defects compared to the composite tape for different time stamps is shown above in Figure 4. The highest contrast is right after detection, 0.01 s (top image). The contrast sharply reduces till 0.4 s (20 mm from the roll) and after 2 s (100 mm from the roll) no demarcation could be made between the defects and the tape. As also seen in Figure 8 (left image), the temperature gradient is steepest until 0.4 s and gradually recedes off to 2 s, where the temperature stabilizes.
Another use of such inline monitoring tool is to verify structural integrity of the laminate after sensor embedment. As discussed above FBG have found extensive use for structural health monitoring (SHM). Absence of hot/cold spot despite foreign object inclusion (FBG sensor in this case), indicates good adhesion to the substrate. The data received from such a sensor is reliable.
In Figure 5, polyimide coated FBG is integrated in the laminate. Except the coating from the tail (190 mm), the sensor itself (200 mm-350 mm) cannot be located. Placement of the tail inside the laminate should be avoided to ensure perfect bonding between the sensor and composite.

Active thermography: validation and comparison
The defects shown in Figure 6, from left to right are as follows: extra tape (twice), Kapton film, washer and flash tape (twice). The defects are introduced on single tape substrate and then another tape is laid on top of the defects, as shown from 50 mm to 250 mm tape length. A stark phase difference can be seen from consolidated region (50 mm-450 mm) to unconsolidated region/lose tape (450 mm-600 mm). A remark should be made here; the defects introduced in this case have different size (bigger) than the ones for inline thermography.
Active thermography results are in agreement with inline thermography results (although both analyze different size of defects). Both inline and active thermography can detect foreign body inclusions to varying degree of clarity. Inline thermography for this particular setup had a better resolution and could detect defects smaller than 2.5 mm. Inhomogeneity in temperature over the length can be seen in inline thermography, while volumetric analysis using active thermography shows quite homogenous quality throughout the layup. The depth of defect cannot be determined if using only active thermography in transmission mode. Since inline thermography relies on continuous monitoring, depth probing is dispensed with. Since post-manufacturing volumetric defects cannot be envisioned with inline thermography, post quality checks can be done with active thermography.

Temperature analysis for online defect control
As discussed earlier, most simulation tools make use of static modelling without considering material and test rig configuration effects, for example, a setup using a silicon compaction roll. This decreases accuracy of temperature prediction. Therefore, to be able to predict the temperatures for individual layers reliably, following experimental results are deemed necessary. This historical temperature data is used as a threshold for mean temperature for the ROI to detect defects online. With a-priori knowledge of the mean temperature, regions exceeding the threshold (mean temperature ± SD) are localized and marked as defects.
The compaction roll completes approximately 3 rotations during one layup for the present setup. As the silicon roll gets heated over the course of layup and affects the temperature distribution, the layup course and temperature profiles are divided into three regions. The temperature statistics shown below (Figure 7) are averaged out for the green box (10 mm Â 40 mm) shown in Figure 5. It should be noted here, that the box is positioned approximately in the middle of the heated region and is so chosen to ease the software for online control and better accuracy. Changing the position from bottom to middle or top gave similar results. The standard deviation over length (green box, 40 mm) is within 2 C for the first two layers. This then goes on to be within 2 C from third layer onwards. The standard deviation over width (green box, 10 mm) is within 2 C throughout. Using this information, the statistics can be extended for the whole layup length.
As mentioned earlier, cold tool helps in faster heat dissipation for the substrate. The higher the tool temperature, the hotter the substrate. A positive temperature gradient exists through thickness (laying tapes on top of the substrate) affecting all subsequent layers until a steady state is reached. This can also be seen in Figure 7 (left image). The standard deviation is within 2 C. The gradient is similar for both tool temperatures, indicating that the curve can be shifted (except the first layer) to predict temperatures for tools with higher temperature. Although more experiments at different tool temperatures are required to confirm this.
Temperature difference for adjacent tapes compared to the first laid tape is shown in Figure 7 (right image). For the first tape, there is no surrounding material retarding the heat flow in transverse direction and hence it is colder compared to the rest of the tapes. There is a gradual rise in temperature of the adjacent tape because of difficulty of heat dissipation in transverse direction because of the previously laid tape and the compaction roll (and ATL head) gradually becoming hotter. The standard deviation here as well is within 2 C. This way thermal analysis of the whole laminate having multiple layers (on top of each other) and each layer having multiple plies/tapes (adjacent to each other) can be mapped.
To have the best definition of ROI, temperature and thereby contrast reduction for different layers is studied. As expected, the cooling behavior through thickness in Figure 8 (left image) shows results in agreement to the discussion above. The cooling rate is almost same for all the layers. After 2 s of layup, all layers above layer no. 3, cool down to the same temperature. The substrate can be assumed to have stable temperature before subsequent layup. This is confirmed by temperatures shown in Figure 7 (let image) for tool temperature 30 C. The through thickness temperature is almost same from third layer onwards.
Temperature as a function of distance to the compaction roll is also shown in Figure 8 (right image). Here right most point at 200 mm is closest to the roll, and thereby to the nip-point. If the ROI is chosen far behind the compaction roll, the temperature contrast might not be sufficient to detect temperature anomaly. Also, the temperature would have stabilized by then, as seen in cooling behavior through thickness, Figure 8 (left image). Therefore, to have highest contrast and reliable results, ROI should be chosen near the compaction roll.

Summary and outlook
The monitoring tool detailed in the research work shows capability to detect surface defects. FOD arising in a single ply can be detected and localized inline with ease. The heat absorption and cooling behavior of defects can be further analyzed. These results are also in agreement with active thermography results. Based on the resolution, bonding defects and therefore, integrity of bonded sensor for SHM can also be checked.
Early detection helps with early rectification, and saving the whole part from possible rejection during post manufacturing checks. Depending on the severity of defect on the laminate properties, the ply can be peeled off or a heat pass can be applied. This also helps in avoiding expensive and extensive post manufacturing efforts for defect removal. Overall productivity increase is ensured. To further improve the process productivity, based on the requirements, a trade-off between inline monitoring and post manufacturing NDT can be made. Here, inline monitoring can significantly reduce the effort for post manufacturing quality checks. Since most of the defects can be detected inline, less time is required for other inspections.
The present work illustrates how temperature of first tape within a layer can be extended to subsequent tapes and layers covering the whole laminate. The tool also serves as a basis for temperature prediction and defect control. Historical temperature statistics from previous experiments are used to predict temperature of the incoming tape, given a set of processing parameters. Using this a priori knowledge, any anomaly is identified as a defect. Since the temperature analysis is done on the present set up configuration, material and environment variability is already accounted for and the results are accurate having a small standard deviation. Instead of having a constant mean temperature as threshold, using the temperature statistics available, a continuously adaptive mean temperature threshold can be built. Further experiments can be done to have information about parametric influence on temperature variation and use this to expand the scope of prediction and control.