Reliability of a low-cost webcam recording system for three-dimensional lower limb gait analysis

ABSTRACT The purposes of this study were to develop and evaluate the test-retest reliability of a specific low-cost three-dimensional webcam recording system (3D-WCRS) and compare its reliability to a standard motion analysis system. Twenty healthy volunteers comprised of 5 males and 15 females with a mean age of 22.90 years and mean BMI of 22.72 kg/m2 were investigated for angles of hip, knee and ankle joints in three planes while walking at a self-selected speed. Intraclass correlation coefficients (ICCs) were used to evaluate as well as compare the test-retest reliability of the 3D-WCRS and standard motion analysis system. Standard error of measurement (SEM) was also analyzed for the purposes of the study. The results exhibited excellent test-retest reliability for the 3D-WCRS (ICCs ranged between 0.93 and 0.99, p = 0.001) in the three joints and planes. The standard motion analysis system demonstrated excellent reliability for all joints and planes (ICCs ranged between 0.99 and 1.00, p = 0.001). Minimal SEM values were observed in both the 3D-WCRS and standard motion analysis systems. Therefore, the developed low-cost 3D-WCRS exhibits good to excellent test-retest reliability. The test-retest reliability of the 3D-WCRS is likely to be comparable to a standard motion analysis system.


Introduction
Walking is an essential human activity for daily living (Kirtley 2006). If a person's gait is abnormal, it will affect their quality of life and may also result in high expenditure for rehabilitation. To evaluate abnormal gait patterns, a gait analysis measurement system is an important tool for the specific treatment planning of each patient (Barak et al. 2006;Kirtley 2006). Presently, significant evidence exists to suggest that clinicians typically assess the gait of patients through visual observation due to a lack of necessary equipment, as well as time consumption and financial support (Eastlack et al. 1991;Coutts 1999;Watelain et al. 2005;Williams et al. 2009;Carse et al. 2013). However, visual observation alone is not reliable and has low accuracy with disagreement in the descriptive terminology used for gait analysis among observers (Watelain et al. 2005;Williams et al. 2009). Observers may not be able to accurately memorize the gait patterns of patients because of the complexity of the motions.
A camera-based three-dimensional (3D) motion analysis system has been developed to assess the complexity of kinematic gait parameters such as joint angles, joint position, body alignment, and gait speed. The advantages of this system are accuracy, reliability, and providing a range of gait parameters within one analysis (Simon 2004;Miller and Callister 2009). While 3D motion analysis systems have been widely used in laboratory research, they are rarely used in clinical settings due to limitations including the expense of the system, sophisticated tool kits, timeconsuming processes, and the need for specialists.
Nowadays, a webcam is efficacious to use instead of an expensive camera (Simon 2004). Wang et al. (2013) developed a low-cost video system in 2013 to assess the gait of elderly people. Their results showed that the developed low-cost system was able to assess spatiotemporal gait parameters in the elderly; however, the joint angles of lower extremities were not measured in their study. In 2010, Kongkhiaw (2010) developed a real-time gait assessment tool with a webcam recording system. This novel system showed reliable output for lower extremity joint angles. Nevertheless, the motion system from this study performed two-dimensional analysis only. Therefore, the current study aims to evaluate the test-retest reliability of a low-cost three-dimensional webcam recording system (3D-WCRS) compared to the standard 3D motion analysis system. The current study intends to develop the 3D-WCRS to reliably measure the joint angles for hips, knees and ankles. This developed low-cost 3D motion analysis system might be promoted for future use in clinical settings.

Participants
Twenty healthy volunteers (5 males, 15 females, mean age of 22.90 ± 2.20 years, BMI of 22.72 ± 3.38 kg/m 2 ) were recruited in this study using convenience sampling. Inclusion criteria were individuals aged between 21 and 40 years who were healthy and had no deformities or abnormalities of the joints in the lower limbs, such as bowed legs, club foot or other obvious and debilitating deformities. Participants were excluded if they had suffered any major orthopedic injury, such as a meniscus tear, ligament tear, and/or recent muscle strain or tendon sprain, fracture or any surgery to the lower extremities within the last year before participating in this study. In addition, participants were excluded if they had recently experienced abnormal orthopedic conditions, such as hip pain or knee osteoarthritis and/or neuromuscular deficits which could considerably affect their gait. Pregnant or menstruating females were likewise excluded. The experimental protocol was approved by the Naresuan University Institutional Review Board (IRB No. 0616/60). Participants signed a written informed consent before joining the tests.

3D-WCRS
The developed 3D-WCRS consisted of three webcams (OKER, 2.0 MP, 30 frames per second; fps and infrared), a personal computer (LG Intel® Core™ i5-470, CPU 3.20 GHz), reference frame, 14 mm spherical markers, tape rule and tripod kits. The software for calculating joint motion in this study was developed by a researcher with LabVIEW and NI Vision development module. The developed software extracted images from three video files that were previously recorded by three webcams simultaneously. The software transformed the extracted images into Hue Saturation Luminance (HSL) color format. Then, the sets of pixels of markers in the extracted images were obtained using image segmentation. The ranges of HSL values were set as H (0-255), S (232-255) and L (0-46) respectively. After that, the center pixels of markers of images from cameras were used to calculate their positions in a Cartesian coordinate system (x, y, z) using stereo-vision. The stereovision technique is used for processing the inputs from digital images from more than two cameras. It matches pixels according to the principle of triangulation and constructs a three-dimensional estimation of an object's position, including length, width and depth (Bebeselea-Sterp et al. 2017). The joint angles (hip, knee and ankle angles) were calculated by the co-ordinates of the related markers of the joints. Because of the limitations of laboratory space and 3D-WCRS camera setting, only the angles of joint motion in right lower limbs were selectively analyzed in the current study (Figure 1(b)).
Prior to collecting the data, the 3D-WCRS was calibrated with a reference frame with reflective markers. The reference frame consisted of a rectangular chart (60 × 60 cm) with nine reflective (14-mm diameter) markers with known x, y, z co-ordinate positions. The reference frame was put in the middle of the capture area and was seen clearly by three webcams. The calibration was analyzed by determining differences between the x, y, z coordinate positions of all markers between the reference frame and the recorded values from the 3D-WCRS. The calibration was accepted when the averaged errors of the x, y, z co-ordinate positions from all markers were less than 5% of the actual positions.
The standard motion analysis system: Vicon TM motion analysis system The standard motion analysis system referenced in the current study is the Vicon TM Motion Analysis System. This system consists of a 10-camera Vicon™ Nexus system (Oxford Metrics, Oxford, UK) at 200 Hz, a personal computer, calibration kits, and reflective spherical markers. Plugin Gait Model software (Oxford Metrics, Oxford, England) was used to construct a three-dimensional model of lower extremities (Dempster 1955). Only right hip, knee and ankle joint angles of participants were analyzed in the gait cycle. Kinematic data were filtered with a 4th order zero-lag low-pass Butterworth filter at 6 Hz.

Experimental procedure
Test-retest reliability of 3D-WCRS system Prior to the start of the study, the researchers gained experience in the use of the 3D-WCRS, the standard motion analysis system and the method of marker attachment. Researchers initially collected participant joint motion in the right hip, knee and ankle joints at the Faculty of Engineering, Naresuan University. The laboratory settings are detailed in Figure 1(a). The participants were assessed for hip, knee and ankle joint angles for the test-retest reliability of the 3D-WCRS for two sessions. On the test date, participants wore a black fitted t-shirt and shorts during testing in the laboratory. Next, a set of 14 mmdiameter spherical reflective markers were attached with adhesive tape on the participants' bony landmarks, including the anterior superior iliac spine (ASIS), posterior superior iliac spine (PSIS), greater trochanter, lateral aspect of the mid-thigh, middle of the lateral knee joint line, lateral malleolus, base of the fifth metatarsal bone, and head of the second metatarsal bone. Five-centimeter wand markers were placed on the lateral sides of the mid-thigh and shank ( Figure 1(b)). The maker set used in the 3D-WCRS was designed to represent the segments of the lower limb. The joint angles were calculated from one segment relative to nearby segments and also calculated in relation to the joint angles in static trial. The wand markers were used to calculate joint angles in the horizontal plane. After obtaining the marker locations, the joint angles were calculated using Microsoft Excel function as follows: In sagittal plane; hip angle = ATAN2( In horizontal plane, hip angle = ATAN2(Z MT -Z MT1 , X MT - Where; X, Y and Z were co-ordinates, and the letters following the co-ordinates were the marker name (Figure 1(b)).
Later, participants practiced walking along a specific test path to gain familiarity before the recording of data. In the actual tests, participants were initially recorded from the static trial in an upright standing position, which was used for initial reference angles when analyzing hip, knee and ankle joints during walking. Afterward, individual participants were assessed for gait and recorded using the 3D-WCRS while walking at their natural self-selected speed along the test path (5 m) for 10 trials. A day after the first session, gait assessments with 3D-WCRS were recollected for all participants in a second session. Lastly, the researchers manually digitized and tracked the exported information of the gait cycle including hip, knee and ankle joint angles.
Test-retest reliability of the standard three-dimensional motion analysis system When data collection at Naresuan University was completed, participants were appointed to test again at another laboratory at the Faculty of Physical Therapy, Mahidol University, to investigate the test-retest reliability of the standard motion analysis system. First, the standard motion analysis system was calibrated, followed by measurement of weight, height, leg length, knee width, and ankle width for individual participants before data collection. Individual participants had spherical reflective markers (14 mm in diameter) attached on their bony landmarks with adhesive tape according to the modified Helen Hayes marker set (Christopher 2006). The locations of marker placement were bilateral ASIS and PSIS, lateral aspects of the thighs, lateral condyle of the femurs, mid-tibias, lateral malleoli, and the heels and heads of the second metatarsal bones (Figure 1(c)). Afterward, participants practiced walking along the specific test path to gain familiarity before the recording of data. Then, participants were also recorded starting from a standing position. Next, individual participants were recorded for their gait using the standard motion analysis system while walking at their natural selfselected speed along the testing path (10 m) for 10 trials. Lastly, Plug-in Gait was used to calculate the angles of the lower limbs during the gait cycle.

Data acquisition and statistical analysis
Hip, knee and ankle joint angles during eight phases of the gait cycle of the 20 participants were calculated by the same researcher from 3D-WCRS and the standard motion analysis system using the same criteria for data analysis as follows. The researcher selected five successful trials which showed a complete gait cycle and no loss of markers in the recorded pictures. The selected pictures of the gait cycle were separately analyzed into eight events including heel strike, foot flat, midstance, heel off, toe off, initial swing, mid-swing and late swing phases by visual inspection (Nordin and Frankel 2012). Heel strike was the picture where the heel initially contacts the floor. Foot flat was the picture showing the whole plantar surface placed on the floor. Mid stance was the picture where the whole body weight was loaded on the leg and the position of the lower leg was vertically arranged to the floor. Heel off and toe off were defined when the heel and the toe were initially lifted from the floor, respectively. The picture of the initial swing was selected from the middle picture recorded between toe off and mid-swing phases. The picture of mid-swing was chosen if the analyzed leg was under the midline of the body. The picture of late swing was picked out from the middle picture between midswing and a second heel contact phases. The researcher randomly selected successful gait trials if there were more than five successful trials. In the manual digitization process, the researcher visually selected the pictures of each phase of the gait cycle and transferred the x, y, z coordinate positions from the recorded picture to calculate the joint angle relatively to the static trial. Mean, standard deviation (SD) and standard error of measurement (SEM) for hip, knee and ankle joint angles were averaged from five trials. There were 160 gait data samples (8 phases × 20 participants) per session for each joint and plane.
Intraclass correlation coefficient (2-way mixed effects, absolute agreement) was used to analyze the test-retest reliability of the 3D-WCRS and the standard motion analysis system. Test-retest reliability was analyzed with a single joint angle across five trials within each session, whereas the averaged joint angle was used to determine test-retest reliability between sessions. ICC values were identified as excellent (ICC > 0.90), good (ICC between 0.75 and 0.90), moderate (0.50-0.75) and poor (ICC < 0.50) reliability (Portney and Watkins 2009). Average walking speeds over the gait cycle were manually calculated after data collection for both measurement systems using right ASIS marker, measured in the direction that volunteers traveled. One-way repeated measures with a Bonferroni's post hoc analysis were used to determine the differences between average walking speeds between session 1 (from the 3D-WCRS), session 2 (from the 3D-WCRS) and the standard 3D motion analysis system. Moreover, the mean differences of joint angles between the 3D-WCRS and the standard motion analysis system were also descriptively reported. Statistical analyses were performed using SPSS statistical software and a p-value of 0.05 was set for all statistical analyses.

Results
The test-retest reliability of both within and between sessions of the 3D-WCRS expressed as ICC is provided in Table 1. The ICC values of both within and between sessions were good to excellent for each plane of hip, knee and ankle joints (ICC of first session = 0.89-0.99, ICC of second session = 0.86-0.99, p = 0.01). Test-retest reliability between the first and second sessions was excellent for all joints and planes (ICC = 0.93-0.99, p = 0.01). The test-retest reliability in a single session of the standard 3D motion analysis system expressed as ICCs is provided in Table 2. The ICC values were excellent for all joints and planes (ICC = 0.99-1.00, p = 0.01). The joint angles (mean ± SD) obtained by the 3D-WCRS are presented in the supplementary information.
The averages of SEM for hip, knee and ankle joints from eight consecutive phases during the gait cycle in three planes as evaluated by the 3D-WCRS and the standard 3D motion analysis system are presented in Table 3. The SEM values for hip, knee and ankle joints in the sagittal plane were minimal (SEM of the first session ranged between 0.09º and 0.83º; 0.19º and 2.33º; 0.11º and 1.52º and SEM of the second session ranged between 0.43º and 2.02º; 0.43º and 2.76º; 0.22º and 2.31º). The SEM values for hip, knee and ankle joints in the frontal (SEM of the first session ranged between 0.28º and 1.42º; 0.08º and 1.33º; 0.39º and 2.03º and SEM of the second session ranged between 0.25º and 1.50º; 0.23º and 5.60º; 0.38º and 1.90º) and horizontal planes (SEM of the first session ranged between 0.35º and 1.45º; 0.55º and 2.99º; 0.30º and 3.10º and SEM of the second session ranged between 0.20º and 1.48º; 0.43º and 1.98º; 0.26º and 1.91º) were also minimal. In addition, smaller SEM values from the standard 3D motion analysis system were observed (SEM of hip, knee and ankle joints in three planes ranged between 0.08º and 0.86º; 0.03º and 0.28º; 0.10º and 0.45º).
The trajectory of hip, knee and ankle joint motions in three planes of 3D-WCRS against the standard motion analysis system were plotted and are shown in Figure 2. Most of the joint trajectory patterns obtained by 3D-WCRS looked similar to the motions observed by the standard motion analysis system, especially in the sagittal plane.
The average walking speeds from 3D-WCRS were 0.89 ± 0.08 m/s (session 1), 0.92 ± 0.10 m/s (session 2) and the average walking speed from the standard 3D motion analysis system was 1.21 ± 0.08 m/s. Average walking speeds were significantly different (F (2, 28) = 133.86, p = 0.001) between the 3D-WCRS and the standard 3D motion analysis system in session 1 (mean difference = 0.32 m/s, p = 0.001) and session 2 (mean difference = 0.29 m/s, p = 0.001). There was no difference in average walking speeds between session 1 and 2 with the 3D-WCRS (mean difference = −0.03 m/s, p = 0.44). Means, minimums and maximums of differences of joint angles in the hip, knee and ankle between 3D-WCRS and the standard 3D motion analysis system are shown in Table 4. The average joint angle differences were smaller for all joints in the sagittal and frontal planes when compared with those angles in the horizontal plane.

Discussion
The aims of this study were to evaluate the test-retest reliability of a low-cost 3D-WCRS, both within and between sessions, and to compare its reliability to a standard 3D motion analysis system.
Generally, the results demonstrated that the test-retest reliability coefficients, both within and between sessions of the 3D-WCRS, were good to excellent for all joints and planes. Additionally, the SEM values observed by both the 3D-WCRS and the standard motion analysis system were found to be low (<5º) (HCW et al. 2006;Macionis 2013).  Table 2. Test-retest reliability coefficients for hip, knee and ankle joints in three planes obtained by standard 3D motion analysis system (n = 160).  Moreover, the trajectories for lower limb motion during the gait cycle from the 3D-WCRS and the standard motion analysis system were similar. This indicated that the developed 3D-WCRS is adequately reliable for determining hip, knee and ankle joint angles during gait in three planes. Therefore, the developed 3D-WCRS is likely to be as reliable as the standard 3D motion analysis system. Although the ranges of mean differences of joint angles during walking between the 3D-WCRS and the standard 3D motion analysis system were large, these differences would not be suitable for representing the similarity between the two measurement systems as measurements of joint motions taken by the two systems were taken on different days and in different places. The SEM of knee and ankle joints in all planes during toe off, initial swing and mid-swing phases in both sessions were larger as above quantify when compared with the hip joint and other phases. The decreased recording frame rate of the 3D-WCRS may have caused some variability. It was possible that motions of knee and ankle in these phases were rapidly moved and some of the captured pictures from webcams were obscured. This means that the webcams could not effectively record rapid motions. During the recording of data, the decreasing of frame rate Figure 2. Trajectory of average hip, knee and ankle joint motions in three planes during 100% gait cycle analyzed using the 3D-WCRS (session 1 and 2) and a standard motion analysis system. Table 4. Means, minimums and maximums for mean differences in hip, knee and ankle joint angles between 3D-WCRS and standard 3D motion analysis system of eight consecutive phases during a gait cycle in three planes (n = 20). for webcams was observed (from 30 fps to approximately 20 fps). This was one limitation of the 3D-WCRS in that frame rate of the webcams was automatically lowered while simultaneously recording motion. Accordingly, the number of webcam frame rates would be a source of variability in terms of joint angles in those phases. In addition, the inconsistency of walking speed among participants may cause variability in measurements. Although participants may already have been familiar with the specific test path before data collection and asked to walk at their natural speed, it could not be assured whether participants walked with a consistent speed. The results demonstrated that walking speeds from the 3D-WCRS (in session 1 and 2) were significantly different from the standard 3D motion analysis system. When the walking speed of individual participants was different, it affected their gait patterns and created inconsistency (McGinley et al. 2009;Kwon et al. 2015). Joint angle errors between trials in the current study were similarly observed in previous studies which demonstrated variability in motion analysis measurement. A systematic review of McGinley et al. in 2009McGinley et al. in (2009 reported that most studies which measured joint angles of the lower limbs during walking by three-dimensional gait analysis systems demonstrated small errors (<4°) of joint angles between trials except the angles in hip and knee rotation. Fosang et al. (2003) and McDowell et al. (2000) also found that the variation of flexion and extension angles in the lower limbs could range from 5 to 10 degrees. The range of SEM observed with the 3D-WCRS ranged within 5 degrees. It could be said that the variability from 3D-WCRS is acceptable when compared to the standard 3D motion analysis system. The 3D-WCRS was deliberately developed to be a lowcost motion analysis system for convenient use in clinical settings. The 3D-WCRS was preliminarily established as a prototype recording system. The cost of equipment and software for developing the 3D-WCRS was quite low (approximately 1,500 US dollars) when compared to the standard 3D motion analysis system (at least 60,000 US dollars) (Carse et al. 2013). However, there were some limitations in the current study. As a prototype, the 3D-WCRS cannot report in real-time for joint angle or gait speed. Furthermore, it entailed time-consuming manual analysis, visual inspection of gait in different phases and only one side of the lower limb could be measured due to a restriction in the field of view. In addition, the frame rates for webcams were decreased during recording and timeconsumption as well as skill-dependence for calculating data was high. These limitations could be solved and evolved in a subsequent study. In future studies, the 3D-WCRS should synchronously investigate gait analysis with the standard motion analysis system to prove its concurrent validity. Furthermore, participants with abnormal gait should be included for investigating the performance of the 3D-WCRS, such as individuals with osteoarthritis and neuromuscular deficits. In conclusion, the 3D-WCRS exhibited mostly good to excellent test-retest reliability and is likely to be as reliable as a standard 3D motion analysis system but at a much lower cost. The 3D-WCRS is a more cost-effective recording system when compared to a standard 3D motion analysis system and can reliably analyze the angles of the lower limbs in three dimensions during walking.