Measuring wearing time of knee-ankle-foot orthoses in children with cerebral palsy: comparison of parent-report and objective measurement

Background Orthotic wearing time may be an important confounder in efficacy studies of treatment in 2 children with spastic cerebral palsy (SCP). Most studies measure parent-reported wearing time with 3 questionnaires, but it is questionable whether this yields valid results. This study aims to compare 4 parent-reported wearing time (WT parent ) with objectively measured wearing time (WT obj ) in children 5 with SCP receiving orthotic treatment. Methods Eight children with SCP participated in this observational study. For one year, they received 7 knee-ankle-foot orthosis (KAFO) treatment. WT parent was measured using questionnaires. WT obj was 8 measured using temperature sensor-data-loggers that were attached to the KAFOs. The 2.5 th and 97.5 th 9 percentiles and median of differences between methods (per participant) were used to calculate limits 10 of agreement and systematic differences.


Introduction 1
According to the International Society for Prosthetics and Orthotics consensus document 1;2 , orthotic 2 treatment is applied in efforts to prevent deformities and muscle shortening in children with spastic 3 cerebral palsy (SCP). Applying a high strain onto muscles by stretching the muscle tendon complex 4 using orthoses is presumed to increase joint range of motion or prevent its loss 2 . Wearing time is an 5 important factor in determining effectiveness, as efficacy of treatment with orthoses is suggested to be 6 dependent on duration that muscles are strained 3 . Two recent studies 4-6 , investigating the efficacy of 7 ankle-foot orthoses (AFO) and knee-ankle-foot orthoses (KAFO) straining the calf muscles, used 8 questionnaires filled in by the participant's parents to assess wearing time. However, it is questionable 9 whether such parent-reported wearing time is a valid measure. 10 Research focusing on objective methods to measure wearing time has been reported for 11 patients with scoliosis, wearing spine orthoses. Several objective methods are available, such as data 12 acquisition using temperature sensors 7-10 or using force transducers 9;11 attached to the orthoses. These 13 methods yielded reliable and valid wearing time estimates 7-11 . Temperature sensors have been 14 mentioned as most feasible with good agreement with actual wearing time, with no reported 15 mechanical problems and the sensors being easy to use 9 . 16 Objective measures have been used for comparison of self-reported wearing time and actual 17 wearing time (further referred to as objectively measured wearing time). Previous studies assessing the 18 reliability and validity of self-reported wearing time measurements in patient with spine orthoses 19 showed that, in general, self-reports by patients or parents led to overestimation 8; [12][13][14][15][16] . In particular 20 cases, patient-reported wearing time was twice the objectively measured wearing time 14;16 . Also 21 children with clubfoot deformities, who were wearing a foot abduction orthosis, patient-reported 22 wearing time (by parents) was overestimated 17 . In efficacy studies, such overestimation may lead to 23 erroneous conclusions if orthoses were not worn long enough to be effective. 24 To the best of our knowledge, in children with SCP who wear KAFOs or other orthoses to 25 prevent loss of ankle-foot range of motion, parent-reported wearing time has not been compared to 26 objectively measured wearing time. Because wearing instructions, aim of treatment and design of 27 orthoses are different from patients wearing spine orthoses, validity of parent-reported orthotic 28 wearing time may vary from the above described studies. Therefore, we aim to compare parent-29 reported wearing time in children with SCP, collected using questionnaires, and objective wearing 30 time, measured using temperature sensor-data-loggers attached to the KAFO. It is hypothesised that 31 parent-reported wearing time differs from objective measurements. 32

Methods 1
This observational study is part of a randomized controlled trial that tested the efficacy of KAFO 2 treatment. Approval for this study was obtained from the Medical Ethics Committee of VU University 3 Medical Center 4;5 . All participants (if older than 12 years) and their parents signed an informed 4 consent form. 5 Participants and study design 6 All children participating in this study were also participants in the experimental group of a multi-7 center study investigating the efficacy of KAFOs to prevent equinus in children with SCP (Splint 8 study 4;5 ). In the Splint study, 30 children were recruited and 17 of them were randomized to the KAFO 9 group. They were recruited between February 2010 to April 2012. The KAFOs of children who were 10 recruited in Amsterdam or Venlo (8 children, allocation was determined by the child's residence) were 11 equipped with a temperature sensor-data-logger to measure wearing time ( Figure 1). These 8 children 12 were included in the present study. Although the sensor-data-logger was visible for the participants, 13 they were not informed about the actual aim of the device. Besides that wearing time was measured 14 objectively, parents of participants were asked to report KAFO wearing time for their child. The 15 participants were aged between 4 and 16 years and were able to walk independently (gross motor 16 function classification system (GMFCS) level of I-II) or with walking aids (GMFCS level III) 18 . All 17 participants were instructed to wear KAFOs for at least 6 hours every other night. parents did not respond. The following multiple choice question was asked: 'How many hours did 27 your child wear his/her KAFO?'. Answers were given as seven categories, ranging from 0-1 hour to 6 28 hours or more. 29 Objectively measured wearing time (WT obj ) in hours/week, was measured with a temperature sensor- time-temperature data. An increase of at least 3°C in temperature within 45 minutes, indicated that the 34 KAFO was put on. As KAFO temperature is not expected to increase over normal body temperature 1 (37 o ), periods with higher temperatures were ignored, because it was assumed that these temperature 2 increases were artefacts caused by external heating (e.g. KAFO exposed to direct sunlight or heater 3 radiation). The KAFO was assumed to be removed if temperature decreased to values similar to 4 previous periods that KAFO was not worn. 5 Analyses 6 For each participant, WT parent and WT obj per week were calculated for all weeks that data of 7 both methods were available. A week was only analysed if WT parent and WT obj measurements were 8 available for at least 4 days. If only 4-6 days of a week were available, wearing time of those days was 9 used to estimate total wearing time of 7 days. For WT parent , the centre value of each time category (i.e. 10 0.5 for 0-1 hours, 1.5 for 1-2 hours up to 6.5 for 6 hours or more) was used to calculate total wearing 11 time of the week. 12 Agreement between the two methods was assessed according to the non-parametric Bland and 13 Altman method 19 . The non-parametric method was used because of violation of normal distribution of 14 data due to some extreme values for differences in wearing time between measurement methods. For 15 each participant, median of the differences between both measurement methods per week was 16 determined. In addition, the group median of differences and 2.5 th and 97.5 th percentile score were 17 determined, reflecting the systematic difference between methods and limits of agreement 18

respectively. 19
Results 20 Temperature sensor data were available for 8 participants of the Splint study (mean age: 8.7±1.9 years; 21 gender: 6 males, 2 females; GMFCS: level I (n=2), level II (n=4), III (n=2)). The period of KAFO 22 treatment of 4 participants was shorter than one year because other treatment (e.g. botulinum toxin 23 treatment) was indicated clinically (n=3) or the KAFO caused too much discomfort (n=1) and 24 therefore the protocol was not finished. As a consequence, KAFO treatment varied from 2 months to 1 25 year. Individual response to questionnaires ranged from 22 to 100% with a median of 87%. For about 26 25% of the sent questionnaires reminders had to be sent. Acceptable time-temperature data that was 27 available per participant ranged from 10 to 100% (median: 66%). Missing time-temperature data was 28 caused by 1) too high air temperature because of weather conditions, leading to a too small increase in 29 temperature if the KAFO was worn (3% of total missing time-temperature data), or 2) technical 30 problems with sensor-data-loggers (due to sensor failure that was not related to KAFO wearing), 31 leading to empty data-files (97% of total missing time-temperature data). The period that data of both 32 WT parent and WT obj were available ranged from 1 to11 weeks per participant, with a median of 3 33 weeks. 34

<< Figure 2 >> 1
Individual median, interquartile range, minimum and maximum values of WT parent and WT obj 2 are presented in Fig. 2. Temperature data show that the KAFO wearing time was lower than prescribed 3 and in one case, KAFO wearing time was 0 hours. In addition, data show that, in some participants, 4 there were large differences between WT parent and WT obj . The median of these differences per week are 5 shown per participant in Fig. 3. This figure shows that there was no systematic difference between 6 both methods (median: 0.13 hours per week), but that differences between methods show large 7 variability between participants (ranging from -2.0 to 12.9 hours per week), resulting in large limits of 8 agreement (lower limit/2.5 th percentile: -1.7 hours per week; upper limit/97.5 th percentile: 11.1 hours 9 per week), This indicates that agreement between methods is low. 10 << Figure 3 >> 11 Comparing measurements within participants, a high variability in differences between the 12 two methods was shown as well. Individual data of weekly measurements of two participants is shown 13 in Fig. 4. For participant 102, WT parent ranged from 0 to 12 hours per week, while WT obj ranged from 0 14 to 8.5 hours per week (Fig. 3A). Differences between WT parent and WT obj ranged from -5.5 to 5.5 hours 15 per week with a median of 0 hours per week (Fig 3B). For participant 207, WT parent ranged from 0 to 16 16.5 hours per week, while WT obj range from 0 to 8.5 hours per week). Differences between WT parent 17 and WT obj ranged from -1 to 10 hours per week with a median of 2.5 hours per week. The wide range 18 of within participant difference between WT parent and WT obj was also expressed in high individual 19 limits of agreement (see figure 3B and 3D, participant 102: lower limit/2.5th percentile: -5.4 hours per 20 week, upper limit/97.5th percentile: 5.2 hours per week; participant 207: lower limit/2.5th percentile: -21 -0.9 hours per week, upper limit/2.5th percentile: 9.4 hours per week). 22

<< Figure 4 >> 23 24
Discussion 25 The present study shows that parent-reported wearing time of KAFOs differs from objective 26 measurements. The differences in reported wearing time between methods vary considerably per 27 participant: some parents reported higher wearing times than what was measured by temperature 28 sensor-data-loggers while others reported lower wearing times. In literature regarding spine orthoses, 29 mostly overestimation of self/parent-reported wearing time is reported (differences range from 0-30 150%) 8; [12][13][14][15][16] . It should be noted that these conclusions are often based on means of all participants. In 31 accordance with the reported range of differences between wearing time measurement methods, the 32 present study found no systematic difference or at most a small overestimation of parent-reported 33 wearing time. The large range of overestimation reported in literature may indicate a similar trend as 1 found in this study that self/parent-reported wearing time and objectively measured wearing time 2 varies considerably between participants. Therefore it may be concluded that parent-reported wearing 3 time and objectively measured wearing time are not comparable. 4 In the present study, high values of limits of agreement between parent-reported and 5 objectively measured wearing times were found. If the objectively measured wearing time method is 6 considered as a golden standard, the limits of agreement reflect the error of parent-reported wearing 7 time. In the present study, the limits of agreement indicate that only if parents reported that the KAFO 8 was worn for more than 11 hours per week, it can be concluded with certainty that the KAFO was 9 worn for at least a short period. However, participants often did not wear their KAFO for such long 10 period in general (i.e. Fig. 2 shows that participant 102 and 207 did wear their KAFO not more than week, agreement between methods is still very low. 17 The response rate of questionnaires was relatively high (approximately 85%). Additional 18 analysis showed that there was no correlation between response rate and agreement between parent-19 reported and objectively measured wearing times (Spearmans rho=0.252, p=0.5). Another factor 20 possibly affecting agreement was delayed response to questionnaires: it was often necessary to send 21 reminders to parents after 2 weeks. This may indicate that parents did not complete questionnaires on 22 the day of KAFO wearing and may explain the variability in response. However, uncertainty about 23 delayed responses and therefore variable recall time will always be part of any parent report method. 24 Overestimation was larger compared to under estimation: some parents reported KAFO wearing for 25 periods that the temperature-data-logger did not register any higher temperatures. This may be the 26 result of a more general phenomenon that people adjust their answers according to socially desired 27 answers. It is unknown whether other methods of self-reporting may yield more accurate results. 28 Because of validity problems of parent-reported wearing time, we recommend the use of 29 objective techniques to measure KAFO wearing time. We agree with Hunter 9 et al, that it is feasible to 30 use temperature sensor-data-loggers to measure KAFO wearing time. However, the high percentage of 31 missing data with the current device indicates that technical failures can occur and need to be solved. 32 In this study, we were able to collect data for each participant after replacing the non-functioning 33 sensors. In most datasets, it was straightforward to distinguish KAFO wearing from non-KAFO 34 wearing using criteria described above. However, at high ambient temperatures (about 25 o C), it is 35 more difficult to determine KAFO wearing time because differences in temperature with or without a 1 leg in the KAFO was smaller. Comparison of data from temperature sensors inside the KAFO and 2 additional outside temperature sensors 20 may solve this problem. 3 Limitations of the study 4 A limitation of this study is the low number of participants. Non-parametric Bland Altman 5 plots are less reliable in small sample sizes 19 . However, because of the large intra-individual variation, 6 we believe that research using larger samples will yield similar conclusions. 7 A second limitation is that parent-reported wearing time was assessed by only one method and 8 cannot be generalized to other self/parent-reporting wearing time methods. We chose to send online 9 questionnaires during each day. This was only done for every 4 th week of the month to decrease 10 burden on participants. 11

Applications to clinical practice 12
Measuring parent-reported orthotic wearing time with online daily questionnaires for one 13 week per month does not show to agree with objective measurements using a temperature sensor-data-14 logger. Outcome of parent-reported questionnaires has to be interpreted with extreme care. 15

Conclusion 16
Wearing time of a knee-ankle-foot orthosis assessed by questionnaires is not in agreement with of 17 wearing time assessed by temperature sensor-data-loggers and should therefore be interpreted with 18 utmost care. Using objective measurement methods are preferred. Regarding temperature sensors, we 19 recommend to use a more sophisticated method then used in this study to measure orthotic wearing 20 time with temperature sensor-data-loggers, like the method described by Bus e.a. 20 , in order to 21 overcome difficulties in determining orthotic wearing time at high ambient temperatures. 22

Acknowledgement 24
We thank all participants and their parents for their participation in this study. We also thank 25 Pottstown, Pennsylvania, USA. The "temperature sensor-data-loggers" were partly sponsored by 1 PEDAK Meettechniek BV Heythuysen, The Netherlands. The authors declare that there is no conflict 2 of interest. Although we received financial support from Ultraflex Europe, Ultraflex Systems Inc. and 3 PEDAK Meettechniek, they did not have authority over this study. The KAFO as shown in the figure is configured with a fixed knee angle and a variable ankle angle 5 using a spring that provides variable ankle dorsiflexion.The temperature sensor-data-logger was 6 attached to the KAFO with a nut (upper inset) and bolt (lower inset). The flat side (lower inset) of the 7 sensor-data-logger touched the participants skin during wearing of the KAFO.