Leveraging existing education innovations to establish a community of practice to promote medical education scholar development

ABSTRACT Problem While some academic health centers have organizational structures to support educational scholarship, such as Medical Education Research Units (MERU), many education scholars may lack access to such institutional resources to support their research agenda and professional growth. Approach In 2014, as part of an externally funded education grant, three faculty educators established a unique education data management system Research & Evaluation Data for Educational Improvement (REDEI). Lacking an institutional MERU, they established an educational research community of practice (CoP) using REDEI as the research dataset. The senior faculty member’s effort to facilitate the group is funded by the Dean’s Office. Outcomes The CoP meets every 2 weeks to generate research ideas, discuss analytic approach and strategy, review analyses designed to address or explore a research question, and plans for manuscript development. Our CoP has grown from 3 to 18 members representing faculty educators, administrators, and staff across many departments in the School of Medicine. As of 2021, the REDEI system contains performance data on 1,246 students across all years of undergraduate medical education. To date, we have published 11 peer-reviewed educational research manuscripts. Five learners have served as coauthors (three medical students and two residents), three of whom were first authors. Eleven additional papers are in process. This community of practice supports productivity, provides mentorship, overcomes barriers, and is flexible enough for people to join when they can or when an area of interest is actively under development. Next Steps We are working on educational interdisciplinary research grant submission and creating collaborations with other institutions. Our focus remains on honing skills in grantsmanship, identification of impactful research questions, application of rigorous methods and instrumentation to address them, and refining process of budget development, timelines, and other planning strategies.


The problem
Participating in the scholarship of medical education enables educators to learn from each other, build on their achievements, and career advancement. While many medical education professionals derive deep and intrinsic rewards from their scholarly work, those who must also publish for promotion and tenure may also face external pressures to pursue a high level of scholastic output without the support that researchers in clinical medicine or basic sciences often have. Regardless of motivation for performing medical education scholarship, scholars in medical education face a myriad of obstacles to successfully advance educational research. These include, but are not limited to: a lack of training in educational research methods, design, and statistical analyses; lack of appreciation for educational rigor and scholarship; limited opportunities for extramural funding; time demands due to competing responsibilities and lack of protected time; and a need for mentorship, community, accountability, and rewards [1][2][3].
Varying organizational structures have been developed to support the academic mission of educational researchers, such as the creation of formal offices or departments of medical education or medical education research units (MERU) [4]. In the USA, only 34% of US-based medical schools having an active medical MERU as recognized by the Society of Directors of Research in Medical Education [5]. Another contemporary model called Health Professions Education Scholarship Units (HPESU) has emerged as a less formal structure relative to offices or departments of medical education. HPESUs are a discrete organizational structure sponsored by a parent organization and typically are constituted primarily with three professional HPES roles: (1) clinician educators, (2) research scientists, and (3) administrative leaders [6,7] who work together to produce health professions education scholarship. While MERUs and HPESUs offer for support for scholarship, what should education scholars do if their parent organization lacks this organized framework?
Originally proposed in 1991 by Lave and Wenger, communities of practice (CoP) represents a conceptual framework that describes a group of 'people who share a concern or a passion for something they do and learn how to do it better as they interact regularly' [8]. Over time, a CoP builds collective growth among members through mutual engagement, joint enterprise, and shared repertoire [9]. Thus, a CoP aims to foster learning and growth via peer mentorship and shared practice among a group of people who share a common passion [9]. This is contrasted against a typical workgroup or committee, which typically focus on productivity as a chief outcome. Early experiences with CoPs formed among medical educators show promise for breaking down silos and bringing medical educators from various backgrounds together [10,11]. Moreover, purposeful cultivation of CoPs have been proposed as a model to address many lacking aspects of traditional educator development efforts [12,13]. At Oregon Health & Science University (OHSU), there is no MERU nor a formally recognized HPESU. However, building off of previous education innovation work, namely, receipt of an American Medical Association Accelerating Change in Medical Education Grant, we created an educator's CoP to foster educator development toward supporting research and scholarship skills. In this innovation report, we describe how we formed a CoP, its associated costs, benefits to its members, as well as scholarly outcomes.

Approach
We received funding to transform OHSU's undergraduate medical education (UME) curriculum from a time-based to time-variable competency-based program from the AMA [14]. The support included the development of a relational data system with an Internet-based learner portfolio viewing portal designed to allow students, faculty educators, advisors, and educational leaders to monitor the progression of student development in as close to real time as possible. Fed by multiple data streams, such as ExamSoft, MedHub, Anatomy Hive and USMLE/ NBME, the Research & Evaluation Data for Educational Improvement (REDEI) system houses student performance on a variety of formative and summative educational metrics. REDEI's dashboards include performance on weekly quizzes, end of block examinations (e.g., OHSU final exams, clinical skills and performance exams, USMLE/NBME assessments, and health systems science exams) [15], as well as workplace-based assessments (WBAs) aligned with the 13 Core Entrustable Professional Activities (EPAs) for entering residency [16]. Scholarly projects, student artifacts, preceptorship, and remediation data are also included in REDEI.
When REDEI was initially developed, we applied for and received Institutional Review Board (IRB) and Family Education Rights and Privacy Act (FERPA) Approvals for use of REDEI Data for research purposes [17] and additionally received a Federal Certificate of Confidentiality to protect the privacy of research subjects by prohibiting disclosure of identifiable, sensitive research information to anyone not connected to the research except in a few specific situations [18]. Now well established, the REDEI system refreshes every two hours in its data processing and viewing systems.
Development of the REDEI system involved a senior faculty member and educational researcher (Author PAC) team leader, a web developer, and an assistant dean for student affairs. This team, known as the REDEI team, met every other week beginning in 2014 to develop, review, and revise REDEI features as ongoing needs were identified. By 2016, the REDEI Data system had captured performance data on 453 medical students across a variety of measures, and data were stable enough to be used for educational research. As the structural and operational features of the portfolio system matured (Table 1), the REDEI team began to grow and was replaced with the REDEI User Scholarship Hopefuls (RUSH) team.
The RUSH team made this transformation to address the institution's desire to develop educators interested in learning educational research and academic writing skills. Accordingly, we intentionally developed RUSH as an educator's CoP using the definition espoused by McGrath et al., as discussed in "The Problem" section [9]. With the data supplied by the REDEI system as the convening platform, the RUSH team founders sought to build a community in which members' skill development was the primary objective and scholastic output regarded as a secondary outcome.

Operational features of our community of practice
CoPs aim to foster learning and growth via peer mentorship and shared practice among a group of people who share a common passion [9]. To create a CoP, the RUSH team developed and came to consensus on a shared goal. Namely, that we would be dedicated to helping all team members' skills to rise together toward (1) identifying relevant educational research questions; (2) providing input into preliminary and final analyses; and (3) ongoing manuscript development while engaging and including learners and junior faculty whenever possible. Given our CoP's roots, our shared task is to research questions arising from or in relation to the evolving REDEI dataset.
The initial core REDEI group of 3 grew to 9 members in 2016, to now 18 core members of the RUSH team who attend bi-weekly meetings when possible (Table 1). RUSH has amassed a broad array of educators from various departments and of differing ranks who would not otherwise collaborate in educational research, including medical student and resident members who are working on scholarly projects related to their training. Membership has been variable but steady with some standing members, while others have come and gone depending on their interest areas and the CoP focus.
RUSH team meetings begin with a Research Jeopardy game where participants choose correct findings among incorrect ones from recently published educational research projects. The game is designed to help members recognize the oftenwrong assumptions commonly made about educational research outcomes and to support critical thinking and assessment of research rigor. The remainder of the time is typically spent identifying and discussing potential projects, reviewing data/ making decisions on data quality (e.g., inclusion/ exclusion criteria, management of outliers), discussing the results of analyzed data, and identifying and planning next steps for manuscript development. We use a research idea/manuscript tracking system to manage progress. Participants sign up in the tracking system to work on a project of interest, which includes manuscript title, lead and senior authors, writing group, date of data maturity, target journal, and submission timeline. The tracking database provides the entire group with a shared mental model from which to work, starting with the research idea and following through to manuscript development phases, submission status, and revising/resubmission management. The tracking database helps the CoP maintain a shared vision.
We use the International Committee of Medical Journal Editors' (ICMJE) four-point authorship criteria to guide author responsibilities, where all authors must meet all four criteria: (1) substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; (2) drafting the work or revising it critically for important intellectual content; (3) final approval of the version to be published; and (4) agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved [19].
After projects and team members are identified, and next steps delineated, team members work on manuscript development using a two-group approach: Group 1 produces the first rough draft while Group 2 undertakes critical revisions. We have found it helpful to save fresh eyes for Group 2 work, as it serves as a 'mini review. ' We aim for no more than four drafts prior to submission, and the objective for each review is clearly conveyed prior to work on the respective draft. We have also found that everyone gains valuable experience by taking turns on each writing group, according to their interest and available bandwidth during the production of the manuscript. If someone signs up for an authorship role but is unable to fulfill the four responsibilities, the lead and senior authors remind them of the criteria and ask if they feel they have met them. Thus far, this approach has not changed the dynamic of the CoP, and the authors who have not met criteria have either stepped up to meet them or have agreed to be removed from the paper.

Institutional support and sustainability
The educational researcher team lead, a senior faculty member who devotes 0.20 FTE to the effort, is funded by the Dean's Office. This a funding mechanism which bypasses more traditional departmental or office funding models. This individual (1) manages the agenda, develops Research Jeopardy questions, and facilitates RUSH meetings; (2) oversees and conducts data hygiene activities; (3) conducts data analyses; (4) develops the research methods and results sections of manuscripts; (5) provides mentoring on manuscript development and publishing ethics; and (6) assists with grant development and submissions to advance the educational research effort. The A novel WBA system designed for handheld devices that load data to a repository using the internet for EPA competency assessment. Is described along with initial results from our inaugural medical student cohort. a Denotes author was a student or resident at time of work.
Web-designer/programmer who works on data capture and structures and creates analytic files dedicates 1.0 FTE.

Benefits of the RUSH Community of practice
RUSH participants receive mentoring in educational research that is often not available in their departments due to a lack of expertise or competing demands. As part of this mentoring, their skills in manuscript development, revision, and resubmission improve and they make meaningful progress in their scholastic mission. Similarly, they receive assistance with educational grant development, submission, and, if funded, project management that further advances their skills as educational researchers. The collaborative ideas and newly formed partnerships across departments and divisions have been instrumental in contributing to a highly functional CoP.
Since 2016, members of our CoP have published 11 peer-reviewed educational research manuscripts, with an additional one currently in review (Table 2). Five learners have served as coauthors on these papers (three medical students and two residents), three as first authors. We began publishing with REDEI data in 2017, as it took about three years for the REDEI system to be mature enough to use data for research purposes. We are able to produce one to two papers each year. We have an additional 11 papers in process between the stages of conceptualization and statistical analyses. Table 3 provides additional metrics from the RUSH group, including conference proceedings, program evaluations, quality improvement/curricular revision, and grant submissions and receipts. We have succeeded in establishing a growing community of scholars who fit this work into their schedules because it fosters and supports productivity, helps to overcome barriers by working together so everyone's skills improve, and is flexible enough for people to join when they can or when an area of interest is actively under development.

Next steps
As the sophistication of our CoP has grown, we have begun collaborating on educational research grant submissions, many of which are multi-institutional and interdisciplinary. This provides additional skills in grantsmanship, including networking and relationship development, to advance educational research beyond a single setting. In addition to fostering advancement of junior faculty, success in grant attainment provides protected time and other resources to conduct studies that are currently added on to busy schedules. To conduct more impactful research than we are currently able to do at our own institution, we have also requested datasets from the AAMC to conduct larger studies, such as identifying predictors of success among students who are underrepresented in medicine. Finally, we are exploring how to leverage our CoP process and shared expertise to support projects outside of our original focus of REDEI platform research questions. We have found that the CoP approach has fostered educational scholarship effectively at our institution and hope that other academic institutions in medicine may benefit from the community of practice model we have implemented, as it may be generalizable to other settings. The Health Professions Education Scholarship Units (HPESU), another possibility, would require more institutional support than is currently available at OHSU.