Telerehabilitation for people with physical disabilities and movement impairment: development and evaluation of an online toolkit for practitioners and patients

Abstract Purpose Telerehabilitation has increasingly been used since the COVID-19 pandemic but with limited guidance available on undertaking physical assessments using remote methods. We aimed to provide such guidance by developing a Telerehab Toolkit, an online information and training resource for practitioners, patients, and carers on telerehabilitation for people with physical disabilities and movement impairment. Materials and methods Development and evaluation of the toolkit were informed by the Knowledge to Action framework and took place iteratively in two phases—knowledge creation and action. Information was collated from various sources including literature review, online survey, service evaluation, and focus group discussions. The toolkit has been evaluated using think-aloud interviews, e-mail and social media feedback from users, and analytics data on user engagement with the website. Results The Telerehab Toolkit focuses on remote physical assessments, and contains information on technology, digital skills, remote assessment tools, information governance, and safety for telerehabilitation. Resources include top tips from practitioners and patients, how-to guides, checklists, videos, and links to evidence. Conclusions The Telerehab Toolkit has been well-received by practitioners, healthcare students, patients, and carers, is being disseminated widely, and is freely available (www.plymouth.ac.uk/research/telerehab). IMPLICATIONS FOR REHABILITATION Telerehabilitation has been increasingly used since the COVID-19 pandemic, but with limited guidance and training for practitioners on undertaking safe and effective remote physical assessments. The Telerehab Toolkit has been developed iteratively using the Knowledge to Action framework; it is a free online resource for practitioners and patients with specific guidance on telerehabilitation for physical disabilities and movement impairment. It is anticipated that the resource will help to improve the knowledge, skills, and confidence of the current and future rehabilitation workforce.


Background and objectives
Telerehabilitation is defined as the delivery of rehabilitation via information and communication technologies [1]. With the COVID-19 pandemic acting as a catalyst, the use of remote methods (including telephone and video-based consultations) has greatly increased [2][3][4]. Telerehabilitation is associated with various benefits for patients, including reduced stress and anxiety and improved accessibility [5]. There are also potential financial savings for service providers including reduced costs associated with practitioners' time and practitioner and patient travel to clinics [6], in addition to lower outpatient resource use [7].
Despite the widespread use of telerehabilitation, there has been a lack of guidance, information, and training on how to safely and effectively undertake remote consultations for people with physical disabilities and movement impairment. Most studies and guidance documents provided generic advice and information on aspects of telehealth, such as communication in videobased consultations; although this is important, there was little or no specific guidance on movement-related assessments [8]. Consequently, there were large variations in the approaches taken, with professional bodies expressing concerns about inequity and inefficiency in telerehabilitation, and recognising a need for clear and standardised guidance [9][10][11].
The aim of this project was to produce an information and training resource that could be used by practitioners and patients, to improve knowledge, skills, and confidence in telephone and video-based consultations for physical disabilities and movement impairment. To achieve this, we established the knowledge gaps and training needs, explored experiences (including challenges and facilitators), and collated evidence and recommendations regarding best practices in telerehabilitation. The process of development and evaluation of the resource-the Telerehab Toolkit-is described in this paper.

Theoretical basis and overview
Implementation science is a relatively new field that aims to enhance the uptake and impact of resources, interventions, or services in healthcare [12]. The Knowledge to Action (KTA) framework [13] provided the overarching conceptual framework for the project. The KTA framework seeks to synthesise knowledge about a particular issue and refine and implement this knowledge in an iterative process [13]. Development of the Telerehab Toolkit took place in two main phases, knowledge creation and action ( Figure 1). Reflecting the iterative, non-linear development process, knowledge creation informed action and vice versa. For example, the literature and early service evaluation discussions informed the initial content, and later discussions and user feedback resulted in revisions and additions to the content. Consultations with experts (clinical and academic specialists in rehabilitation and physical disabilities), patients and carers, and Patient and Public Involvement (PPI) representatives took place throughout the knowledge creation and action phases. � A service evaluation involving discussions with 53 practitioners, patients, and carers [14] � Focus group discussions with 49 undergraduate physiotherapy students Through triangulation of knowledge from the different sources, a clear need for a novel educational resource in telerehabilitation was identified. Although most practitioners felt competent in undertaking video-based and telephone consultations, they reported low confidence in carrying out remote physical assessments. Only 19% of practitioners surveyed had received formal training in telerehabilitation, and some felt like they were "feeling their way in the dark" [4]. Patients and carers generally reported positive experiences of telerehabilitation but also identified a need for further guidance.
The areas in which guidance was most needed were established in the knowledge creation phase. For example, practitioners and students were concerned about the validity, reliability, and safety of remote physical assessments. They also wanted guidance on information governance (e.g., ensuring privacy and consent) and telerehabilitation with special groups (e.g., patients with sensory, cognitive, and communication impairments). Patients and practitioners wanted information on technology and links to resources for improving their digital skills.
We also collated information that was directly used as content within the toolkit. This included patient and carer accounts of telerehabilitation experiences, case reports from practitioners, and top tips for successful remote physical assessments.

Action phase
The action phase involved the iterative development and implementation of the Telerehab Toolkit. The steps within this phase were as follows: Planning of format, structure, and core sections. Based on the needs of practitioners and patients that were identified in the knowledge creation phase, it was decided that the format of the toolkit would be a web-based resource (rather than an app). An outline plan for the structure of the resource was produced, with core sections (pages) to meet the needs of the intended users.
Drafting of content, review, and revisions. Following the outline plan and using the information gathered in the knowledge creation phase, the content of each section was drafted. Taking a section-based approach, this phase involved internal review by the core research team, in addition to external feedback from collaborators, survey and service evaluation participants, educationalists, and PPI representatives. Individuals were approached for feedback according to their areas of expertise; for example, information governance managers were consulted regarding the information governance section, and digital health researchers provided feedback on the digital skills and technology sections. Patients, carers, and PPI representatives gave feedback on the patients' section, which was also reviewed by practitioners. This was an iterative process with �2-5 versions of each section being developed and refined as a result of feedback.
Implementation: website setup, population, and launch. When the content of each section was finalised, the researchers worked closely with the University's digital team to set up and populate the online resource. The website was launched in May 2021.
User feedback: think-aloud interviews. Individual think-aloud interviews [15] were undertaken via videoconferencing with five practitioners, five patients, and two carers. Interviews were carried out in two rounds (May to June, and July to September 2021) to facilitate further iterative development of the toolkit. The participants interacted with the website in the presence of the researcher (SAB or KA) and concurrently provided feedback on usability (ease of use and navigation), perceived usefulness (including relevance and value of information), and presentation (including layout and quantity of information). This led to minor revisions to several sections of the website; examples of these are given in the Results section.
Addition of audio-visual content. When the core content had been developed, audio-visual content was added to improve engagement with the resource. This included videos (how to navigate the website; how to get the most out of remote appointments; and a discussion between practitioners on person-centred care), case reports (written and video), and infographics (checklists and how-to guides for patients and practitioners).

Dissemination and feedback.
Dissemination of the resource is ongoing and has been carried out in two phases to date. In phase one (August 2021), e-mails introducing the toolkit were sent to 35 relevant clinical, professional, and educational networks, and six health and social care organisations in South West England. The resource was also promoted to contacts in the project mailing list and via social media, with encouragement to disseminate this widely using a snowballing campaign. In phase two (December 2021), 77 academic institutions (universities throughout the UK) were contacted via named individuals based in health and social care faculties. Feedback was requested within the dissemination e-mails; further minor revisions to the presentation and content were made as a result.

Monitoring and evaluation.
To monitor the use and usefulness of the resource, two sources of information are being captured on an ongoing basis. Data on user engagement with the website is collected using Google Analytics [16], with a monthly report provided by the University's digital team. To complement this data, qualitative feedback from practitioners and patients is continually being obtained via e-mail (with a feedback request on the homepage). This has resulted in further revisions and additions to the presentation and content.

Revisions
Revisions to presentation and content were made at various stages of development as a result of feedback from practitioners, students, patients, and carers ( Figure 1). Feedback on the presentation included making it easier to navigate the website, improving the conciseness of certain sections, highlighting key points, and minor changes to the layout. Feedback on content included making the resource more applicable to a wide range of occupations, the addition of further specific guidance, and changes to wording to improve clarity, accessibility, and engagement. Some examples of feedback and revisions made are given in Table 1.

The developed resource
The developed resource is available from www.plymouth.ac.uk/ research/telerehab [17]. It is composed of core sections for practitioners, patients, and a digital skills section for practitioners and patients (Textbox 1). The practitioners' section consists of eight sub-sections covering different topics in telerehabilitation, such as remote measures and assessment tools, and information governance and safety. There are two additional sections-"About the research" and five individual case reports supplied by practitioners (e.g., telerehab for COVID -19). The toolkit contains a range of resources including: � Top tips and quotes from practitioners and patients; � Accounts and experiences; � Common questions and answers; � How-to guides to telephone and video-based consultations; � Checklists of equipment and resources; � A training checklist for practitioners and students; � Infographics; � Videos; � Links to evidence-based papers; � and links to other useful resources.
The Telerehab Toolkit as it appears currently (January 2022) is shown in Figure 2.

Analytics data
Analytics data demonstrates that the resource is being used and is engaging for its users. Since dissemination began in August 2021, there have been �1750 total views (and 1250 unique viewing sessions) of the website per month. The most frequently visited sections and the average time spent viewing each page are shown in Table 2. The average viewing time varies between sections and over time, but ranges from �1 min for shorter sections and navigation pages (e.g., resources for practitioners) to almost 8 min for more detailed sections (e.g., remote measures and assessment tools). Most users access the website via a direct link, and a high proportion access it from search engines, such as Google. Although the toolkit was designed as a UK-based resource, its reach has been wide with access from Europe, Asia, North America, and Australasia. Examples of when to use video consultations were added for a range of professions, including occupational therapy, podiatry, dietetics, prosthetics and orthotics, and clinicians. Quotes and case reports from a range of practitioners were added. Some changes to wording were suggested in the patients' section-e.g., "appointments" may be a more familiar term than "consultations." "Consultations" was changed to "appointments" throughout the patients' section.
Think-aloud interviews It is not obvious how to return to the homepage-can a navigation button be added?
A toolkit navigation menu including a "Telerehab homepage" button was added to each of the patient and practitioners' sections. A specific guide for family members/carers would be useful.
A list of ways in which the carer can be involved was added to the patients' section. Carers' needs were addressed in the questions and answers. The web links do not stand out very well.
All links were highlighted in bold font. In the specialist guides section, there is too much text and it is difficult to focus.
Quotes were placed in separate boxes. Key points were emphasised in bold font. Sub-sections were reorganised and broken down into smaller chunks of text. It would be good to have a news or updates section or banner, to see what has changed since the last visit.
Twitter feed was embedded in the homepage-this is regularly updated with information about new content. In "Remote vs. face-to-face appointments" in the patients' section, the wording sounds negative towards remote appointments (e.g., "Are remote appointments a poor substitute for face-to-face?").
Sentence was phrased more positively: "Are remote appointments as good as face-to-face ones?"

Dissemination and feedback
There is no specific guidance on telerehabilitation for people with anxiety and depression.
Recommendations for telerehabilitation for people with anxiety and depression were added to the specialist guides section.

Monitoring and evaluation
From analytics data, the COVID page had fewer views compared with the other sections.
A central link to the COVID section was added to the practitioner resources gallery.
Note. This is only a small selection of feedback and revisions to indicate the types of changes that were made.

Qualitative feedback
Qualitative feedback from users has been predominately positive. Practitioners and patients reported that they found the toolkit "easy to use," "practical," "clearly presented," and "well laid out." Practitioners commented on the value and usefulness of the toolkit due to it being a novel resource with information that was not available elsewhere. They perceived the toolkit as comprehensive and useful for their current and future practice, and believed that the resource would help to improve their knowledge of, and confidence in, telerehabilitation. For example: Such an excellent resource for clinicians completing telerehabilitation, I really like the section with advice for groups and cognitively impaired patients -thank you.
(Comment from a practitioner on Twitter)

It is very comprehensive -it includes everything I hoped it would. It feels like everything has been considered from a practitioner's perspective …
We're all trained to see patients face-to-face -I haven't received any training in remote consultations. It's like retraining us to be clinicians in a virtual world. Current evidence is really important. The toolkit will be really valuable going forward.
(Practitioner feedback from think-aloud interview) Patients thought that the patient guides within the toolkit would be most useful for people who were less familiar with technology and that these would have been useful for them before they had their first video appointment: It is all good information, especially for those who are not familiar with technology. It would have been useful when first starting to learn how to do video appointments.
(Patient feedback from think-aloud interview) Overall, they perceived the toolkit as a useful resource for people having remote appointments, now and in the future: The information is excellent … It is such an invaluable toolkit for people in the future. It is an asset and a resource for the future.

Early evidence of reach and potential impact
In addition to presentations at academic and professional conferences and other fora, the toolkit has been disseminated via a range of channels. The resource has reached the health and social care professionals and students, the technology industry, patients, carers, and the public. This includes local, regional, national, and international audiences. Some selected examples of reach and potential impact are provided in Table 3.

Discussion
In this paper, we have described the development of the Telerehab Toolkit -a novel, evidence-based resource to share best practices and provide guidance on telerehabilitation for people with physical disabilities and movement impairment, including those recovering from COVID-19. Other publications detail the methods and results of the associated scoping review [8], national survey [4], service evaluation [14], and an exploratory study of technologies to support movement assessment in video consultations [18].
The approach taken has several strengths, including the multistage, iterative development process, which enabled the toolkit to be continually evaluated and refined, and which involved the end users at each stage. Analytics data have indicated that the resource has had a wide reach and good user engagement. The average viewing time of most pages has been consistently high, with certain sections (such as remote measures and assessment tools) viewed for 3-8 min. This is encouraging as it is higher than the 2-3 min that is considered a good standard metric [19], and comparable to other web-based health interventions evaluated using Google Analytics [16], where the total average viewing time ranged from 3 to 6 min [20,21]. Complementing this data,  qualitative feedback (captured via e-mail and think-aloud interviews) has shown that the resource has been well-received by practitioners, students, patients, and carers. This feedback has provided evidence for high perceived ease of use and perceived usefulness, two central concepts used to measure the acceptance of new technology or digital intervention [22]. Based on our scoping review results [8], unlike earlier resources that provided generic guidance on remote consultations, the Telerehab Toolkit focuses on the physical and movement assessment aspects of consulting via video. As the COVID-19 pandemic progresses and practitioners seek solutions to assessing physical status and movement remotely, other guidance documents have been produced which provide valuable and complementary information [23][24][25][26]. The Telerehab Toolkit is comprehensive and widely applicable to a range of professions within health and social care. It is freely and publicly available, and designed to be used as a training resource for students, as well as more experienced practitioners. Although the resource is web-based, it has been optimised for mobile access, to meet the preferences of different users.
The KTA framework provided a useful basis for the development and evaluation of our resources, facilitating knowledge creation and action in iterative and closely linked phases. Additional components of the KTA framework are worthy of discussion; for simplification, these were not included in the flowchart (Figure 1). Adapting knowledge to the local context and tailoring the intervention were important considerations. The toolkit was designed to be used primarily by UK health and social care professionals, so it was ensured that the information it contained was relevant to this group. For example, guidance on information governance in telerehabilitation was based on information provided by the National Health Service (NHS) including NHS Digital [27], and links to UK organisations that could help to improve digital skills were included. We also explored whether tailoring of any information would be necessary for the survey and service evaluation, but found that practitioners throughout the UK had consistent information and training needs, and recommendations for best practice were based on common themes. Instead of tailoring different sections of the toolkit to specific groups of practitioners or patients, the approach taken was to ensure that the resource was as comprehensive as possible, to enable the user to focus on the sections that they found most interesting or useful. Assessing barriers and facilitators to knowledge use is another key component of the KTA framework. We explored barriers and facilitators throughout the development process, including factors influencing the use of telerehabilitation and potential factors that might influence the use of the toolkit. In line with other research findings [5, 28,29], a lack of digital skills and access to technology were recognised as barriers for both of these, and hence the decision was made to include some printable documents. For example, the checklists and how-to guides for telephone and video-based rehabilitation were included as downloadable PDFs, which could be printed by a practitioner and posted to a patient before their first remote appointment.
Monitoring of the impact of the Telerehab Toolkit is ongoing, but early evidence of reach and potential impact has been promising, with the resource having been disseminated via a range of regional, national and international channels. The next steps for the toolkit are further dissemination to UK healthcare Trusts, and continuing to capture feedback and subsequently refine the resource. Sustained knowledge use is an important stage of the KTA framework; the research team is currently working on a sustainability plan that considers issues, such as how frequently the toolkit should be updated, and how to sustain the engagement of users in the longer term.

Limitations
There were two main limitations. Firstly, detailed survey data was collected from practitioners throughout the UK, but the service evaluation and student discussions focused on South West England. This may limit generalisability, although there was an excellent triangulation of findings from the various sources of knowledge. Secondly, the Telerehab Toolkit website is hosted by the University of Plymouth; this was associated with some design limitations, for example, it was not possible to include a search bar that was independent of the main University website. Despite this, the University was the most feasible provider for the website, and also deemed the most appropriate as it was where the resource was developed.

Conclusions
As the COVID-19 pandemic continues, it is important to educate and upskill the current and future health and social care workforce in remote care. This includes the safe and effective use of telerehabilitation for people with physical disabilities and movement impairment, a topic on which guidance on best practice was previously lacking. The Telerehab Toolkit is a newly developed, freely available, online resource, with the aim to fill this knowledge gap by providing guidance on a range of aspects of telerehabilitation. The resource will be useful for practitioners and students, patients, and carers. This paper describing the development and evaluation of the resource may also be of interest to researchers or professionals looking to produce educational or training resources in other fields of health and social care, who might like to follow the approach taken.