Online optical inspection of electrolytic copper plate based on digital moiré

Abstract The application of automatic inspection plays an important role in improving the quality of industrial products. Aiming at nodules on the surface of electrolytic copper plate, this article introduces a comprehensive optical online inspection method. In order to obtain dynamic information of product and overcome the interference of random factors such as large-curvature deformation, position deviation and tilt, the system integrates a variety of single technologies including digital moiré, image recognition and spatial transformation. The online test results show that the system can effectively extract the information of nodules and automatically classify the products according to the set threshold.


Introduction
Automatic optical inspection (AOI) is widely used for product quality control in modern industry. [1,2] As an important non-contact three-dimensional measurement method, structured-lighting technology [3] had been continuously developed. The moir e fringe technology had evolved from optical moir es (including shadow moir e [4,5] and projection moir e [6,7] ) to digital moir e [8,9] to meet the needs of online dynamic inspection. [10][11][12] Since phase-shifting is performed as a post-process, and only one frame of object image with fringe pattern needs to be captured, digital moir e is very suitable for the measurement of moving objects. [8] In the production of electrolytic copper, there may be nodules on the surface of some copper plates. Large nodules can reduce the quality of the product. The copper plates to be inspected on the production line are shown in Figure 1(a), attached to the stainless steel cathode core and with a size of 1020 mm Â 1080 mm. Figure 1(b) shows nodules on surface of some copper plates. If the size of nodule exceeds the set value, the copper plate will be judged as unqualified product. In this study, when the horizontal cross-sectional area of nodules with a height greater than 5 mm exceeds 10% of the area of the copper plate, the copper plate will be picked out as a nonconforming product.
Obviously, the manual inspection cannot adapt to modern industrial production. It is easy to cause misjudgments and lead to poor quality control. Therefore, it is necessary to develop a corresponding online detection device to replace manual inspection. Digital moir e is the first choice for this task, but digital moir e alone cannot meet all the technical requirements of online copper plate inspection. Different from static detection in laboratory, the online detection faces many interference factors. For example, the large-curvature deformation of the copper plate can lead to serious errors in nodule height measurements. This paper proposes a comprehensive inspection system that integrates digital moir e, laser ranging, image recognition, and spatial transformation. The actual test results show that this system can effectively remove interference and extract nodule height information, which can well meet the requirements of online inspection of electrolytic copper.

Principle and method
The electrolytic copper plates on the production line are in motion and the area to be inspected is very large (with meter size). Digital moir e was selected as the basic inspection method. Via performing phase-shift processing on the captured image, the relative height of the measured object can be obtained. [13][14][15][16] In this work, the phase map was usually wrapped between -p and p, so a temporal phase unwrapping algorithm was employed to remove the phase ambiguity. [17][18][19][20] The result given by digital moir e is the height of the object relative to the reference plane. In order to obtain the actual information of nodules, the copper plate image needs to be captured on the preset reference plane. Besides a projector and a camera, a laser rangefinder was also employed in optical device, which triggers the camera when it detects the copper plate has reached the preset position (reference plane). The schematic diagram of inspection system is shown in Figure 2. Figure 3 includes a schematic diagram of the optical inspection device and a photo of the site installation. The inspection system was installed on the outside of the production line. It does not interfere with normal production activities even during the initial installation.
Referring to Figure 4, the height h of the object at point B is given by the basic equation of digital moir e:    where H is distance between the baseline and the reference plane, d is projector-camera distance and p is the pattern pitch (period width), u is the phase modulated by object. The inspection process flow is shown in Figure 5. The dot line box shows the pre-preparation process. The short dash line box shows the inspection process and long dash line envelops digital moir e and plane fitting process.
The inspection method employed in this study consists of two steps: preparation and measurement. They are respectively described in detail below.

System preparation
For digital moir e, computer-generated phase-shift fringes are usually used to obtain moir e contours. However, for large inspection objects, this approach introduces artifacts because the farther away from the projector, the wider the projected fringes. In order to solve this problem, a special strategy is employed in this paper. A qualified electrolytic copper plate was selected as the reference plane, and then four sinusoidal gratings with a phase-shift of p/2 were projected on its surface in sequence. These grating images are recorded and stored. Using them to obtain moir e contours, the artifacts can be fundamentally avoided. The plate should remain stationary during the capture of 4 reference images. The fringe images recorded can be described by Equations (2)-(5): Figure 5. The inspection process of electrolytic copper plate.
where R(x, y) represents the reflectivity of the reference copper plate, a denotes the background component and b reflects the fringe contrast, f x and f y are the frequency of the fringe pattern in x and y direction, u 0 x, y ð Þ is the phase distribution modulated by the reference copper plate. The corresponding images are shown in Figure 6(a). Two alternating components (AC) of reference pattern with 90-degree phase difference can be extracted as Equations (6) and (7). They are stored in computer as the reference pattern for post-process.
In order to eliminate environmental interference and improve the efficiency of image post-processing, the system also prepares a digital mask in advance. By filling the plate area with white and other area with black, a mask indicating reference plate area can be obtained, as shown in Figure 6(b).

Image capture
The laser rangefinder in inspection system is in continuous operation. When the copper plate reaches the set distance, the projector projects a grating pattern with a phase shift of 0-degree on the surface of copper plate, and the camera captures the image at the same time. It can be expressed as the following: where R 0 x, y ð Þ denotes reflectivity of the measured copper plate, and u x, y ð Þ is the phase modulated by the measured copper plate.

Data preprocessing
The pattern described by Equation (8) is then transformed into the frequency domain and filter out the AC component using a well-known Butterworth filter given by the following: where x c denotes cut-off frequency, n is the order of filter. The DC component can be expressed as the following: Using the Otsu Method, [21] the gray threshold of I DC o x, y ð Þ can be calculated, which is a number between 0 and 1. We denote this gray threshold value as "GTH," and convoluting I DC o x, y ð Þ with Sobel matrices respectively, the edge information can be extracted as the following: where S x and S y are Sobel matrices in x and y direction:

Plate margin and possible nodules area detection
In order to reduce the error caused by large-curvature deformation (see Section 2.2.5 for details), image recognition technology was used to identify areas that may have nodules before the phase calculation.
Using the results of I edge o x, y ð Þ and GTH, and taking the mask as a reference, the margin of inspected copper plate can be found by standard image processing such as dilating and filling. I edge o x, y ð Þ is converted to a binary data using the following function: where GTH was calculated from the image of copper plate. GTH/3 gives the best result for edge detection. As an example, the detection process of the nodular area is presented in Figure 7. The edge information I edge o x, y ð Þ and plate mask are shown in Figure 7(a,b). The areas where nodules may exist is detected with a similar process. The detection process of nodules area is shown in Figure 7 According to the property of nodular area, the isolated area should be a trace of electrolyte solution or a nodule with very small height. An experimentally obtained dilation size was used to connect isolated nodule edges together. The regions which area not increased will be removed. The size 20 Â 20 was found suitable for most circumstance, as we see in current Figure 7(c)-(e). We change the dilation size from 20 Â 20 to 200 Â 200, in order to take most nodular area together and make the range big enough to cover most of nodules.

Phase and height calculation
Subtracting Equations (8) with (10), the AC component of plate pattern can be expressed as the following: This AC component is multiplied by the two saved AC components (Equations (6) and (7)) respectively, the results can be expressed by Equations (15) and (16): Equations (15) and (16) are the moir e fringes combined with a frequency doubling carrier. After filtering out the carrier with Butterworth low-pass filter, the moir e fringes with 0 and p=2 phase shift of copper plate are the following: The phase wrapped within (Àp, p) can be obtained by: Figure 8 shows the phases of the copper plate in Figure 7. In this phase map, it can be found that there is a significant tilt and positional deviation between the copper plate and the reference plane.
The phase unwrapping method [22] is used to get the unwrapped phase / x, y ð Þ and the mapping relation between the phase and the height can be obtained by calibration: [23] where b x, y ð Þ is the parameter obtained by calibration, h x, y ð Þ is the height of the detected object. The copper plate surface can be reconstructed by this mapping relationship.

Spatial transformation and nodule evaluation
The projector and camera are triggered by the laser rangefinder at a distance range of 2100 ± 10 mm, in addition, the copper plate on the production line swings randomly. Therefore, there are inevitably random deviation D 1i and tilt D 2i between the captured image and the reference plane. As shown in Figure 9, they can introduce serious artifacts in nodule inspection. A spatial transformation algorithm is performed to fit the phase map to the reference plane. [24] The errors caused by random deviation and tilt can be eliminated by this algorithm.
In actual inspection, in addition to random deviation and tilt, some copper plates may have large-curvature deformation, which will introduce another error D 3i : Obviously, it is impossible to make a curved surface fit the reference plane well. So, D 3i cannot be completely eliminated via spatial transformation. However, by reducing the spatial transformation area, D 3i can be limited to an acceptable range. In this paper, sub-regions where nodule may exist were individually transformed (see Section 4).  Figure 7, the reference plane is set to zero. The spatial transformation process is described as follows: 1. Starting from the four corners of the detection area, search along the diagonal of each corner until four points are found on the copper plate image. 2. Based on these points, four rectangle areas with 10 Â 10 pixels are marked (please see Figure  10). The coordinate (x, y, u) of points within the four rectangles are used to fit a plane by least squares. [24] The equation is shown as follows: 3. The transformation is made as the following: In actual inspection, in order to simplify the operation, the height h x, y ð Þ in Equation (1) is not calculated. The transformed phase u 0 x, y ð Þ is directly used to evaluate the nodules. In this paper, the phase threshold corresponding to a height of 5 mm is 0.82 radians (see Section 3).
As a continuation of the example in Figure 7, Figure 10 shows the data processing results of bounding box in Figure 7(h). Figure 10(a) shows the wrapped phase. Figure 10(b) shows the unwrapped phase and four reference areas. Since the copper plate image does not fill the upper right corner of the bounding box, the AREA 4 is not around the corner. Figure 10(c) is the phase after spatial transformation (this operation push the phase plane back to the reference plane, z ¼ 0). Figure 10(d) shows the nodules higher than the inspection threshold.

System
The inspection system was composed of an optical device and a Hewlett-Packard Graphic workstation (Z8G4:4214/64G/512SSD þ 2T/1450W/RTX4000). The optical device was used to project binary fringe and capture single frame of electrolytic copper plate. It consists of a DLP projector (DPM-E4500LIRMKII On Axis IRL), a LUSTER CCD camera (LBAS-U3200-14M, 3648 Â 5472 Resolution) with FA1615A lens, and a OLAY laser rangefinder (A090, 1 mm Resolution, up to 30 Hz working Frequency). The projector and the camera maintain an angle of 8.5 degrees. The distance from the optical device to the reference plane was designed to be 2100 mm. Regarding the movement of the copper plate and the working cycle of the rangefinder, the trigger distance was set to 2100 ± 10 mm to ensure each copper plate can be captured. Since the shape of the copper plate to be inspected is approximately square, the image resolution actually used by the system is 3648 Â 3648.
The distance from the projector to the reference plane was 2100 mm, and the period width of the projected pattern was 5.83 mm. According to Equation (1), the height h corresponding to one phase period can be calculated as 38.46 mm, and then the phase threshold corresponding to any height can be obtained. A set of corresponding values is given in Table 1.
Four in advance captured images of a standard copper plate was saved in system to generate moir e contour at post-process, in each image the phase of projected fringe was set as 0, p=2, p, and 3p=2, respectively. Before performing further analysis on the moir e patterns to extract the height information, the fringe-pattern projected on object must be removed as noise. [25] In this work, a Butterworth filter of 5 orders was used to remove the high-frequency noise, [13] the cutoff frequency was 85 pixels. The OLAY laser rangefinder operates at a frequency of 30 Hz.

Result and discussion
The system had been tested in laboratory and on electrolytic copper production line, respectively. In laboratory, samples with height of 3 mm, 5 mm, 8 mm, and 10 mm were randomly placed on the reference plane, the measurement results show that the maximum error was less than 0.5 mm, which meet the inspection requirements. Online test results show that the system can accurately identify nodules and limit tilt errors within acceptable ranges. The inspection for each product was completed within 6 seconds. Representative examples are expressed below. Figure 11 shows the nodule detection results of three different samples. In Figure 11(a), there is no marked area, it means that the copper plate is of good quality. For such a result, the system does not perform subsequent calculations and returns directly with "pass." In Figure 11(b,c), some bounding box can be seen, but the system cannot differentiate between nodules and greasy dirt at this stage. If any areas are marked, digital moir e process must be applied to calculate the phase shift or height difference of copper plate. As mentioned above (see Figure 5), the inspection system runs digital moir e only on copper plates that may have nodules. The digital moir e analysis results of Figure 11(b,c) are shown in Figures 12 and 13, respectively.  Figure 11. Samples of nodule detection. Figure 12 shows the results of the sample in Figure 11(b). Figure 12(b) shows the calculated phase of full plate, from which positional deviation and tilt can be seen. Figure 12(c)-(f) shows the results of further processing of bounding box. The final results Figure 12(e) show that there is no nodule higher than 5.00 mm (corresponding to phase threshold of 0.82 radians).
The system has good flexibility, by changing the phase threshold, nodules with different heights can be screened out. As an example, Figure 12(f) gives nodules higher than 2.45 mm (corresponding to phase threshold of 0.40 radians). Figure 13 shows the results of another sample in Figure 11(c). There are 3 areas where nodules may be present, and it can be seen that this copper plate has large-curvature deformation from Figure 13(b). Figure 13(c)-(e) shows the results for further processing of AREA1, (f)-(h) for AREA2 and (i)-(k) for AREA3. The final results show that only AREA 3 has nodules with a height greater than the threshold (see Figure 13(k)).
Nodule inspection seems to be an easy-to-implement routine measurement, but in the real factory environment, various interference factors such as large-curvature deformation, greasy dirt, random deviation, and plate tilt make the detection very difficult. In practice, height artifacts caused by random positional deviation and tilt are often larger than the real height of the nodule.  Therefore, the actual height of nodule cannot be obtained by digital moir e alone. To solve this problem, image recognition and spatial transformation are introduced in post-processing. Here, the main purpose of using image recognition techniques is to divide the copper plate image into smaller sub-region that can be better fitted to the reference plane relative to the entire copper plate, thereby minimizing the tilt errors.

Conclusion
In this research, the detection target (nodule) is severely interfered by both 3 D signals (caused by deformation, random deviation and tilt) and 2 D signals (caused by oil stain and shadow), and in most cases the target signal is lower than the interference signal. Technology mix has play a vital role in removing artifacts. By introducing image recognition and spatial transformation, the practical application range of digital moir e has been greatly expanded.
Practical results show that the inspection scheme proposed in this paper is simple, effective and easy to implement, which can effectively improve the inspection efficiency of electrolytic copper.

Disclosure statement
No potential conflict of interest was reported by the author(s).