Exploiting Image Assisted Total Station in Digital Image Correlation (DIC) displacement measurements: insights from laboratory experiments

ABSTRACT The paper addresses a relevant subject in structural health monitoring for civil engineering applications such as bridges or other large structures. Experimental laboratory program on the accuracy and precision of remote displacement measurements was reported. Could an onboard optical camera of a Total Station (Image-Assisted Total Station (IATS) Leica Nova) be exploited to integrate other measurement techniques or even temporarily replace ordinary instrumental observations in case of instrument failure? The experimental campaign was carried out in the Institute of Engineering Geodesy and Measurement Systems Laboratory at Graz University of Technology (Austria). In particular, pictures of a predetermined moving target, acquired from the IATS’ onboard optical camera (focal length of 231 mm), have been processed through Digital Image Correlation (DIC) to get displacements measures. The obtained results were validated using the “ground truth” assessed by the laboratory equipment. The same was done, capturing pictures with a Digital Single-Lens Reflex (DSLR) consumer camera (focal length of 55 and 85 mm) and using the same experimental setup. Once validated, a comparison between IATS’ optical camera and DSLR observations returned very similar accuracy and precision. The experimental outcomes suggest that comparable results can be achieved by processing pictures from both IATS’ onboard camera.


Introduction
Image-based methods used in remote sensing techniques for monitoring analyse a displacement/deformation pattern on an object surface by tracking distributed points or correlating the available images. Over the last few decades, remote sensing techniques have been recognised as a key tool in monitoring activities. They became a valuable and complementary tool to standard contact monitoring solutions, e.g., inclinometers, extensometers, crack-gauges, and others (Meng, 2002;Psimoulis & Stiros, 2013;Roberts et al., 2004). SAR Interferometry, terrestrial SAR Interferometry (TInSAR), terrestrial laser scanning (TLS), and Global Navigation Satellite System (GNSS) are often used as solutions for structural and infrastructural monitoring (Fagandini et al., 2017;Luzi et al., 2004;Mazzanti, 2012;Selvakumaran et al., 2018;Tapete et al., 2013;Wang et al., 2018) and geotechnical investigations (Bozzano et al., 2017;Delacourt et al., 2007;Mugnai et al., 2019;S.-E. Chen et al., 2016;Scaioni et al., 2014). Thanks to their high precision and accuracy, Robotized Total Station and Levelling techniques have been adopted in a considerable amount of structural monitoring applications such as bridges and buildings (Merkle & Myers, 2004;Murai et al., 2004). Among these remote sensing techniques, during the 2000s-2010s image processing methodologies have developed and increased their potential simultaneously with the progress of sensors, algorithms and computing power (J. G. Chen et al., 2017;Hild & Roux, 2006; J.-J. Lee & Shinozuka, 2006a, 2006bWahbeh et al., 2003).
Digital image processing can be considered one of the most innovative branches among remote sensing techniques. It can identify and detect structure surface changes (Wen et al., 2021). Generally, the Digital Image Correlation (DIC) technique, which is an optical-numerical measurement technique, can provide full-field 2D and 3D displacements of any type of object surface (Bing Pan et al., 2008;Sutton et al., 2009).
Some authors have used ground-based optical camera approaches for displacement monitoring (Bruno et al., 2020;Buckner, 1983;Caporossi et al., 2018;Debella-Gilo & Kääb, 2011;Ehrhart & Lienhart, 2015a;Leprince, Barbot, etal., 2007). According to (Stumpf, 2013), digital image processing can be an essential tool (or video-cameras) for geotechnical and structural monitoring purposes thanks to their low costs and high-quality attitudes (Farquharson et al., 2015;J. G. Chen et al., 2017;James et al., 2006;Lava et al., 2009;Travelletti et al., 2012). In various studies and applications concerning infrastructure monitoring such as stress and strain measurements, crack propagation and static and dynamic displacement, DSLR cameras have been vastly used (Fayyad & Lees, 2014;Ab Ghani et al., 2016;Hammoudi et al., 2021;Hoult et al., 2013;Murray et al., 2015;Pritchard et al., 2013;Yoneyama & Ueda, 2012). Some algorithms and software, e.g., IRIS developed by Nhazca (Hermle et al., 2021) or the Digital Image Correlation routine (Bickel et al., 2018) have been developed. Furthermore, through the image processing and correlation methodologies, it is now possible to measure displacement and deformation with subpixel accuracy (Lin & Lan, 2010;Ma et al., 2018;B Pan et al., 2013). In the last few years, cameras with telescope lenses were used as image-based systems to allow for more considerable distances between the camera and the monitored structure (Martins et al., 2013). Some authors have proposed using the camera to be integrated into the modern total station telescope, namely, the imageassisted total station (IATS; Ehrhart, 2017;Ehrhart & Lienhart, 2015c, 2015bWagner, 2016). Although this camera was widely used only for documentation and report purposes, it is a promising sensor that allows accurate measurements from long ranges since the angular resolution of the resulting image data benefits from the optical magnification of the telescope (Omidalizarandi et al., 2018).
Monitoring activities can last for long periods; a monitoring system must guarantee good reliability and durability for the entire operating period. Even if the operating monitoring system is well designed with a tough sheltering and good redundancy for data transmission and power supply, it can deal with adverse environmental conditions or unexpected drop in data communication and power service providers; some unforeseen variables can lead to loss or useless data. Moreover, a monitoring system with reference targets as a Robotised Total Station (RTS) can require frequent maintenance interventions for target loss. If, for instance, a landslide exceeds a certain speed of deformation, the line of sight could be lost, or due to the changing behaviour of the phenomenon, the interest for a specifically monitored area decreases and a new one raise for another area. In all these cases, data loss is an almost unavoidable consequence. Having an alternative method to maintain the system efficiency and gain a certain degree of flexibility and adaptability to deal with evolving phenomena such as landslides, densely populated areas, and active construction sites could improve monitoring and modelling. To process the 24.2 MPx data at high speed, an imageprocessing engine EXPEED 3, optimised for DSLRs, is incorporated. A rich gradation with less noise is achieved with its high-performance noise-reduction function even at a high sensitivity range. Indeed, a wide sensitivity range from ISO 100 to 6400 is provided (Nikon Inc, 2018;Nikon Inc Tokyo, 2018).

IATS Leica Nova MS60 -Telescope camera
Digital imaging using the Leica Nova MS60 enables enhanced image assisted surveying and image documentation. Generally, digital imaging improves the measurement efficiency, quality and documentation of the field measurements. The Leica Nova Table 1. Summary of the technical specification of the imagebased sensors used in the experimental tests (Grimm & M, 2013;Nikon Corporation, 2012 MS60 incorporates two cameras: the overview camera (OAV) with a fixed focal length above the telescope and the on-axis camera (OAC) located in the telescope optical path. The telescope camera (OAC) was designed so that a focused image is simultaneously available on the live video stream and the optical path through the ocular ( Figure 1). Hence, the image data captured by the OAC benefits from the full 30x magnification of the telescope.
The optical sensor embedded in the IATS telescope camera is an ON Semiconductor MT9P031 with a 1/2.5− inch CMOS active−pixel digital image sensor and an active imaging pixel array 2592 H x 1944 V.
The 5 MPx CMOS image sensor features ON Semiconductor low−noise CMOS imaging technology that achieves medium-high image quality (based on the signal−to−noise ratio and low−light sensitivity) while maintaining the inherent size and cost and integration advantages of the CMOS. In particular, the speckle pattern was created with a Matlab function through a random matrix and different size such as 2000X2000, 1000X1000, 800 × 800. Some tests have been conducted by varying distance from target and matrix (speckle pattern), to check how the DIC algorithm worked. In this way, an optimal speckle pattern was identified.

Measurement laboratory
According to (Kovačič et al., 2011), a high quality and reliability comparison and analysis can be achieved in laboratories since they provide ideal environmental and boundary conditions that are hard to be guaranteed during field measurement surveys. The DIC experimental tests were performed in the Institute of Engineering Geodesy and Measurement Systems (IGMS) measurement laboratory at Graz University of Technology (Austria). This measurement centre was selected because of its intrinsic characteristics, optimal for a rigorous study and investigation of image-based displacement analyses' theoretical accuracy and precision.
The IGMS laboratory is characterised by a regularshaped structure, with an overall size of approximately 6.3 m x 33.2 m x 3.5 m (WxDxH). The laboratory foundations are separated from the rest of the building, thus guaranteeing total isolation from the ground vibration effects. Full air conditioning and lighting systems are installed, preserving the temperature level at 20°C ± 0.5°C and relative humidity at 40% ± 10% (Woschitz & Schauer, 2017). The laboratory consists of a primary calibration and testing facility that includes climate chambers, gyroscope testing, a vertical comparator, sensors' calibration bench with fibre optic strain, and a horizontal comparator for electronic distance measurement calibration, among other features, thus guaranteeing an almost perfect stable atmospheric and optimal lighting condition.

Experimental setup
Designed experiments aim to explore the performances of the above-described optical sensors in DIC applications. Carrying out some set of DIC experimental tests using different geometrical configurations for each optical sensors, accuracy and precision were investigated as the two parameters well represent the effectiveness and reliability of a measurement system (Jones & Iadicola, 2018). Even if accuracy and precision in the scientific community are well-known terms, reminding here their meaning could help in the reading; in particular, "accuracy" is the systematic difference between a measured quantity and the real values; "precision" refers to the random difference between multiple measurements of the same quantity, thus providing a value of the measurements' repeatability (David J White et al., 2005;Hope & Dawe, 2015;D J White et al., 2003;Mazzanti, 2012;Mucchi et al., 2018).
The proposed experimental test was arranged tailoring parameters and laboratory features to the investigated sensors. The target consists of a metal plate with a random speckle pattern. It was fixed vertically on a linear positioning stage Physik Instrumente M-410.DG, which has a position repeatability of 2 μm (Physik Instrumente, 2008). The target was moved perpendicularly to the cameras' line of sight, along the x-axis of the linear positioning stage in steps of 0.25 mm within a range of ± 8 mm (Figure 2), covering 65 predetermined reference positions. No movement was induced along the vertical direction (Y-axis).
The optical sensors were installed on surveying pillars (Figure 3) at the height of 150 cm (pillar coordinates are known with an accuracy better than 0.1 mm).
A high-contrast medium coarse-grained random speckle pattern was applied on the metal plate surface (Figure 3). By adopting this element, at each distance, the target surface was visible in the images.
DIC measurements have been carried out from four different distances between target and sensors, precisely 6.3 m, 12.6 m, 18.9 m and 26.5 m, producing a variation in the collected images' pixel size (Figure 4-Figure 4, Table 2).  Two different NIKKOR focal lenses (FL) have been mounted on the camera body within this study framework at 55 mm and 85 mm.
Twenty pictures were collected by IATS and DSLR Nikon D3200 at each distance for each of the 65 positions the target covered on the horizontal plan. Accordingly, approximately 5000 frames were collected by each sensor to test the multi-imaging effect on the displacement measurements.
Several sources of error were considered during all experimental stages to give the observations strong reliability. Systematic influences (such as the camera optical axis perpendicularity with the speckle pattern realised through auto-collimation with the small mirror and IATS laser pointer) were addressed optimizing position of the sensors and the size of the random speckle pattern by identifying the configuration that guarantee best results in terms of best correlation. Thanks to the controlled laboratory environment, some noise sources, instrumental and environmental warm-up and atmospheric condition effects were adequately avoided.

Data analysis
Image pre-processing Based on the instrumentations' technical specification and the equipped Focal Length (Niemann, 2019), the images collected from both sensors have been retro deformed using PTLens software to decrease the lens distortion effect.
The multi-imaging effect, on the measurements' accuracy and precision as retrieved by DIC analyses on 55 mm FL images was investigated.
An ad hoc MATLAB v. R2018a written code was used to perform image-averaging operation on the collected images (The MathWorks, 2018). According to Mazzoleni (2013) and Turrisi (2017), the displacement evaluation sensitivity to the image acquisition noise such as digitisation, read-out noise, black current noise, and photon noise represents an essential topic in DIC accuracy and precision measurements. The standard deviation of the displacement error is proportional to the standard deviation of the image noise and it is inversely proportional to the squared grey level gradients' average and to the subset size. Furthermore, the DIC measurements may be affected by variations in the measurement uncertainties because of the noise in the images caused by imaging equipment, poor lighting set up or lower camera gain (Reu, 2013).

DIC measurements and statistics
To evaluate the accuracy and precision of the DIC analyses, the subpixel image correlation algorithm COSI-Corr (Coregistration of Optically Sensed Images and Correlation; Ayoub et al., 2009; Ayoub,   Leprince et al., 2007), which is available as an open-source plug-in for the ENVI® software package, was used (Harris® Geospatial Solutions, I, 2018). The best DIC parameters used in each analysis were identified through a trial and error approach. The parameters that restituted the highest correlation coefficients have been chosen, and a correlation threshold of 0.9 was set. The decorrelation signal occurrence ruled the selection of this specific windowsize and step-size parameters on the DIC maps. Even if the different DIC parameters were used, by achieving similar displacement values in all cases, those DIC parameters, as reported in Table 3,were set as the best choice; thus, they did not allow a wide spreadout decorrelation signal. Afterwards, the statistical analyses were performed using ENVI® v. 5.4 and MATLAB v. R2018a; they aimed to determine the sensors' accuracy and precision. Further detailed descriptions of these software programs' algorithms and technical specifications are available on their official websites. The workflow of the DIC measurements carried out in this study is outlined in Figure 5.

Measurement precision
The displacements detected by both sensors were compared to the reference movements of the linear positioning stage. Figure 6 and 7 show the mean residuals of the measured displacement and the measured displacements' residuals to every positioning step reference positions. In particular, in Figure 6 displacements are referred to as the IATS (A) and DSLR Nikon D3200, equipped with 55-mm (B) and 85-mm (C) lenses. The thin black dashed lines represent the displacement magnitude of 0,25 mm induced by the linear positioning stage Physik Instrumente M-410.DG.
In Figure 7 residuals of the measured displacements to the reference positions have been here reported for different distances. They are referred to the IATS (A) and DSLR Nikon D3200, equipped with 55-mm and 85-mm lenses.
First, the achievable precision was analysed by computing the standard deviation of the measured displacement residuals. At each distance, DIC measurements, as performed by IATS, were characterised by a better precision than those retrieved by DSLR (equipped by both lenses). As reported in Figure  7, at 26.5 meters, a linear increase in the precision of 0.05 mm was measured for the IATS-based measurements, while the increase was approximately 0.13 mm for the DSLR-based measurements. The figure shows, as expected, an inverse proportionality between the increase of the distance of investigation and precision.
This result introduced a general consideration concerning using optical-based sensors in displacement measurements as employed, for example, in geotechnical and structural monitoring purposes. Notably, the DIC analyses were performed in a fully controlled environment by adopting the same image-processing algorithms, statistics operators and a replicable measuring methodology. Hence, the measured displacement values could be adequately compared, even if different optical sensors retrieved them. The IATS is provided by a low-resolution small-sized sensor that can collect images for exploiting the 30x magnification of its telescope camera. Conversely, the DSLR camera results are generally provided by high-resolution big-sized sensors that can collect images based on the used FL (in this study framework, 55 mm and 85 mm lenses).

Measurement accuracy
In Figure 8, the mean residuals of the measured displacements are reported for each distance. IATS measured a residual of approximately 0.002 mm at 6 meters, while the DSLR Nikon measured residuals of 0.05 mm and 0.04 mm, with 55-mm and 85-mm lenses, respectively. At a distance of 26.5 meters, the IATS was characterised by higher values, 0.1 mm, while measuring approximately 0.03 mm for both lenses of the DSLR Nikon.  For each DIC analysis, the correlation coefficient (CC) was also computed: the CC referring to the DSLR Nikon D3200 (with both lenses) was higher (>99%) than that measured by the IATS (~97-98%) at each distance.
We also investigated the vertical component using an appropriate arrangement of the linear positioning stage, and similar results were obtained. In addition to the DIC analyses described previously, a further investigation was performed to analyse the measurement results' multi-imaging effect as computed through the DSLR 55-mm lens images. Nevertheless, there have not been reasonable improvements in the achieved residuals of displacement to the reference positions. However, the multi-imaging effect allowed an increase in the quality of DIC measurements.

Discussion
The IATS system is a promising sensor since it allows accurate measurements from long ranges using the total station on-axis camera. This function is allowed by its ability to exploit the optical magnification of the telescope. In some previous studies reported in the introduction section, the IATS systems were employed in very accurate displacement and vibration measurements of bridges for using edge detection methods on circular markers.
Here, we investigate the accuracy and precision in displacement measurements through the DIC technique employment to images collected by the IATS Leica Nova MS60 and DSLR Nikon D3200 optical sensors. We evaluated and compared the potentials and limits of both sensors related to different variables such as the camera settings, distances from the target, and displacement rate.  The high precision linear displacement control system can perform a step-by-step analysis of the measurement detection capabilities in different laboratory configurations. The displacements detected with both sensors were then compared to the reference movements. The terms "residuals" and "standard deviation of the residuals" are hereafter considered as "accuracy" and "precision", respectively.
The accuracy at a distance of 12.5 meters is higher for IATS (with values up to 0.004 mm) than for DSLR with both optical configurations (ranging between 0.04 and 0.05 mm; Figure 9). However, at 20 meters, an inversion in accuracy occurs. At 26.50 meters, IATS' accuracy is up to 0.09 mm, while DSLR accuracy is 0.03 mm. Hence, unlike the DSLR sensors, a significant decrease in IATS sensor accuracy occurs at higher distances.
This feature may be related to the different optical sensors' different features that affect the image quality and, consequently, the DIC measurements. Specifically, IATS' lower quality and smaller optical sensor size are considered key factors. The size, as well as the technical specifications of the sensors (reported in Table 1), indeed influenced the quality of the collected images (by generating blurriness and colour aberrations) and, thus, the assessed displacement accuracy (Figure 10). At a distance of 26,5 meters (in light blue), the accuracy of the DIC measurements retrieved from the IATS was lower than that referred to the DSLR (with both lenses) (A). On the contrary, the IATS was more precise (~0,05 mm) than the DSLR, which significantly decreased its precision up to 0,13 mm (B).
According to (Cheremkhin et al., 2017), sensor noise can be divided into random and pattern components, thus representing one of the main factors limiting camera performance. Temporal noise varies randomly, whereas spatial noise has a pattern that arises due to sensor non-uniformities. Furthermore, the sensors' image compression mode can also impact the measurements' accuracy.
In an uncontrolled environment (i.e., outside of a perfectly controlled laboratory), different sources of noise, which were not considered in this study, may affect the results such as changes in lighting condition, humidity, pressure, wind, air refraction and density, atmospheric haze, dust, shadows, overheating, natural speckle pattern, camera motion, selection of the correlation algorithm and the related parameters (subset and step-size, correlation and shape functions, subpixel algorithm; Handel, 2007;Hijazi et al., 2011;Lava et al., 2010Lava et al., , 2013Yoneyama et al., 2006).
There seems to be a general rule that lowerresolution small-sized sensors, compared to highresolution big-sized sensors, are affected by a more relevant accuracy with the distance.

Conclusions
In this study, an experimental DIC investigation was carried out for laboratory displacement analyses. The accuracy and precision of commercial optical sensors (DSLR Nikon D3200) and a state-of-the-art Image Assisted Total Station (IATS) Leica Nova MS60 was assessed and compared.
Precision and the accuracy in DIC displacement measurements were decidedly not coincidental for all investigation distances, as they have assumed different values and trends. For the IATS-based DIC analyses and those for the DSLR, a general linear decrease in the precision occurred by increasing the target distance. Precision values ranged from ~0.001 mm at 6.3 m up to 0.05 mm at 26.5 m for the IATS, while for DSLR, the values ranged from 0.002 mm at 6.3 m up to 0.12 mm at 26.5 m.
Furthermore, the IATS-based DIC analyses were characterised by a higher accuracy at short distances (approximately 0.002 mm at 6.3 m) than at long distances (up to 0.1 mm at 26.5 m). However, the DSLRbased DIC analyses showed an accuracy ranging between 0.04 mm and 0.05 mm at all distances. At a distance of approximately 20 m, an accuracy magnitude inversion occurred between the used sensors, thus having low (for IATS) and high (for DSLR) values.
This result refers to the low-resolution smallsized optical sensor of the IATS. This type of camera is commercially mounted on the IATS systems, but it is mainly still used for documentation and report purposes. One interesting option is integrating a high-resolution camera, as provided by a bigsized optical sensor, to find interesting image correlation results. Compared to a standard camera with telescope lenses, the usage of an IATS has several advantages. IATS can precisely measure the distance to arbitrary objects by electronic distance measurement (EDM). The continuous EDM mode enables recurring to autofocus for image and video capture. Moreover, distance information links the computed pixel displacements to metric values.
For the averaging image effect, a general increase of the precision occurred for all the distances, thus decreasing the signal uncertainty by approximately 50%. In contrast, the DIC measurements' accuracy was not improved and instead grew slightly worse, which grew as the distance increased. However, in the authors' opinion, a completely different result may be achieved in "uncontrolled laboratory" environments where an averaging may support the analyses in reducing several additional noise sources.
This sensitivity and reliability study on commercially available image-based sensors can be fundamental in correctly designing the measurement campaigns. Indeed, the possible assessed movements' achievable accuracy can be evaluated a-priori once the real field measurements have been correctly planned.
Image-based sensors are expected to progressively increase their potential and helpfulness in geotechnical and structural monitoring purposes in the near future. Furthermore, thanks to the continuous technological progress, optical sensors embedded in the IATS systems may also become more appealing and suitable for high accuracy displacement measurements of observed structures at both short and long ranges. Additionally, the increasing development of image-based sensors, which can collect pictures at an optical wavelength (e.g., infrared (IR), microwaves or terahertz), is expected to open new perspectives in remotely sensed geotechnical and structural monitoring. These remote sensing advancements would allow for the study and analyses of different phenomena that are generally not investigable with commonly used optical sensors (e.g., crack opening with IR sensors or the humidity effect on structural behaviour).
Performing tests within a controlled environment, which allows to remove variables such as temperature, humidity, lighting, is the main added value of this work. In this way, results are directly controlled by distance, sensor quality and DIC algorithm.
Although ground truth measurements under laboratory conditions are useful information, and the presented results of the measurement technique can be exploited for outdoor measurement, accuracy and precision could be degraded by the moving atmosphere, vibrations at the sensor, and other possible errors. Thus, the presented results can benefit from further field test to identify the influence of experiment boundary conditions. This study results seem adequate for an estimate of the accuracy at the same distance. For this purpose, additional experiments with more extensive and more extended sets of distances and focal lengths would be necessary, especially for achieving reliable results in real-world applications.