Trends in… Modernization at the Nation’s Medical Library

Abstract This manuscript describes the ongoing efforts to meet ever-changing patron needs by modernizing operations, infrastructure, workforce, and customer service at the National Library of Medicine (NLM). Charged with collecting, organizing, preserving, and disseminating biomedical and life sciences information to the public, the NLM constantly adapts to evolving technologies and scientific advancements in order to better fulfill its mission. Historical context and an overview of operational decision-making offer a window into the workings of our nation’s medical library.


Introduction
Striving to be an engine for innovation and discovery, 1 the National Library of Medicine (NLM) has taken great strides in recent years to modernize its user-centered approach for operations, infrastructure, workforce, and customer service. We are guided by the NLM Strategic Plan 2017-2027 that affirms our commitment to data-driven science, engaging closely with stakeholders, and building a workforce for the future. 2 Our plan strives to accomplish all of this while preserving our essential mission. First and foremost, we are a National Library that collects, organizes, preserves, and disseminates biomedical and life sciences information to the public.
As part of our mission to advance science and improve public health, we recognize the importance of connecting users to NLM resources.
Ultimately, resources must be easily accessible and fulfill users' needs. To this end, we continually strive to understand our diverse range of users ( Figure 1) and how their needs vary over time in order to create new and better ways to deliver the right information at the right level at the point of need. Technological advancements and user expectations are constantly evolving, especially after the COVID-19 pandemic. With users primarily accessing NLM resources through virtual media, the need to understand and improve the NLM online user experience is increasingly integral to achieving our mission. Putting users first has become a centerpiece of how we operate and a catalyst for organizational change. Figure 1 illustrates the various self-identified NLM User Groups.
NLM's modernization journey began with a change in leadership with Patricia Flatley Brennan, RN, PhD, becoming the first female Director of NLM, one of the 27 Institutes and Centers of the National Institutes of Health (NIH). Since assuming the directorship in 2016, Dr. Brennan has positioned the Library to be the hub of data science at NIH, and an international leader in biomedical discovery and data-powered health. With data science and digital application on the rise across every sector, Dr. Brennan challenged NLM to improve their core database, products, and services to better integrate and add value for users in an increasingly data-driven world.

NLM modernization principles
In 2019, NLM began what has been termed a product-oriented transition in order to focus on developing products that fulfill the NLM/NIH mission and meet or exceed customer expectations and deliver those products via an iterative, data-driven, user-centered approach. The desired outcomes of this transition are worth repeating here: Focus on developing and delivering products that fulfill the NLM mission and meet customer expectations. NLM provides critical data, products, and services used daily by policymakers, clinicians, researchers, librarians, educators, and the publicthis means we need to deliver on both the organizational mission and goals and the needs of this diverse group of users. For example, librarians are a user group but not the only, nor the largest, user group for PubMed. (Figure 2). This requires a balancing of priorities. Enable iterative delivery by creating cross-functional product teams empowered to make optimal decisions. Following the CIO Playbook 3 from the U.S. Digital Service (USDS), we aspire to deliver the products users expect via cohesive, collaborative teams that deliver new functionality to users on a regular basis. As part of a continuous cycle, frequent feedback from a variety of users and sources of information are then fed back into the development of subsequent features. Focus on outcomes, not outputs (not simply applying the scrum methodology but rather by working with an agile mindset and delivering with agility). "Agile transformations" are increasingly pursued by traditional organizations wishing to offer greater value for users of digital products and services. Often, organizations that attempt to complete an agile transformation end up delivering the wrong things, or things users don't need, in incremental chunks. To ensure this does not happen at NLM, product teams are required to develop a vision and goals for their productsoutcomes that solve existing user challenges, needs, or problems, and measure progress toward closing the gap between current product capabilities and what users want and need.
In our experience, we learned that the most effective way to develop goals for a product is to follow the series of questions: Who is it for? What can they do? How will we know? Who is it for? As mentioned, NLM products serve a variety of users and user types (Figure 1). It is a reality that we cannot serve every need of every user, so on whom should our focus be? To answer that, we created a list of "design targets," that is, particular user groups that want or need what our products can deliver. Focusing on a particular set of users does not have to come at the expense of other user groups, because often the needs of different groups will overlap. When there are widely different needs for different groups, product teams can be more successful if they work on the groups and their needs in some priority order. What can they do? Given a specific design targetthe "who" defined abovewhat features, functions, data, or capabilities are needed? What problems or challenges do the target users face? What are they prevented from doing by the limitations or missing features of our products? Product teams usually prepare the list of needs as a "problem statement"in other words things that users cannot do now or are prevented from doing, or ultimately need to do. How will we know? Once we have identified our design target and decided which of their problems need solving, we need a way to measure what we do. Here it can be tempting to measure in terms of outputhow many lines of code, or webpages, or database records, or site features did we produce? Our focus on outcomes, not output, really comes into focus here. We seek measures of customer success, called success metrics, that indicate whether users can perform the desired action, solving their challenge in the process.
Taking these outcomes and questions into account, let us next look at a product-related inflection point in NLM's journey with the modernization of PubMed and the MEDLINE subset-NLM's flagship product. As illustrated in the following case studies, NLM embraced new ways of working, such as data-driven decision-making, and positioned itself to lead transformative change.

Pubmed modernization: A case study
PubMed is a free, life sciences literature database that, on a typical weekday, provides search features and access to biomedical information to more than 3.5 million individuals. But success did not happen overnight, and it hinged on delivering value to the scientific community by connecting them to the literature. Even before PubMed's inception in 1996, NLM sought to engage directly with users to respond to ever-changing needs. 4 In thinking about how to approach the challenge of improving upon a heavily-used, well-liked biomedical resource such as PubMed, NLM found inspiration in the example of the U.S. Digital Service (USDS), a group comprising the "best engineering, design, and government talent to change our government's approach to technology." 5 Founded in the wake of the challenging launch of Healthcare.gov, USDS focused on core values, including bringing users into the product development process and optimizing for "results, not optics". 5 The PubMed team is comprised of medical librarians, software architects and developers, product managers, project managers, and scientists. In the mid-2010s this team, working within NLM's National Center for Biotechnology Information (NCBI), began to explore ways to enlist users in the design and development process for its services and databases. Their approach combined user research techniques, such as interviews and usability testing, with small incremental improvements in site features and functionality via a series of experiments, collectively known as "NCBI Labs". The NLM envisioned NCBI Labs as a sort of idea and testing incubator 6 . where changes small and large could be released publicly so that end users could try them and provide feedback to NCBI product teams. This approach was inspired in part by the work of the USDS, along with the General Service Administration's (GSA) 18 F product and technology consultancy and earlier, very similar work being done by the government of the United Kingdom (uk.gov).
NCBI Labs provided a series of early user-focused web experiments, like PubMed Also Viewed, 7 a small improvement to PubMed that allowed users to see biomedical citations related to the current citation. NLM built upon these initial user feedback and small-idea experiments with the release of PubMed Journals, 8 a sort of preview of future work on PubMed that allowed users to follow journals of interest and keep track of recent updates within those publications. In around two years, nearly 20,000 users followed 10,453 distinct journals, with each customer following three journals on average. Eventually PubMed Journals was retired, 9 part of the natural life cycle of an NCBI Labs experiment, but the user testing and usage analysis of PubMed Journals helped prove some initial design and technology approaches that later became part of the modernized PubMed. One of the best examples of this was a series of experiments on the "cite" button that is used to display a formatted literature citation in PubMed. In the experiment, the performance (click-through rate) of different color and text combinations was analyzed to determine the best option for users ( Figure 3).
Based on the success of the early NCBI Labs experiments, and with a growing focus on improving PubMed, NLM began PubMed modernization efforts in earnest. In 2018, NLM began to increase the number and type of experiments related to PubMed, starting with a series of feature experiments and culminating in the creation of an experimental website, PubMed Labs, 10 to provide a more wholistic platform on which to prototype new designs, gather insights and feedback, and rigorously test the Best Match relevance search algorithm. 11 Best Match was based on a machine learning technology and was designed to offer a faster way for users to get to the most relevant citations based upon their queries. Placed at the heart of PubMed Labs, the Best Match search ranking option demonstrated that this new sort order resulted in more citations selected for review from PubMed's search results page. 11 Yet, a new and powerful search option was only one of several improved experiences the team was committed to delivering. To help organize the improvements around PubMed product principles, the team developed the following list: Put users first. Decisions and prioritizations were to be considered in the context of PubMed users' needs. This required a large amount of user research to gather data and build empathy. Make it fast. PubMed users were already accustomed to getting their results quickly, and the new version had to improve upon that. Deliver 24/7 access. PubMed is a mission-critical resource of the National Library of Medicine that needs to be consistently and continuously available. Do the most for the most people. As a public service, it was the team's aim to do the most good for target user groups and for the majority of PubMed's user base, as illustrated in Figure 2. Provide an error-free database. Corrections were prioritized and a database free of errors was the goal.
The team used a multitude of user research techniques including surveys, user personas, user interviews, website analytics, prototype testing, task analysis, use case development, and usability testing to support and guide the modernization effort. Research plans which outlined the background, goals, questions, methods, participants, and outcomes sought were established for each data gathering effort.
One of the major objectives of the PubMed modernization effort was to eliminate the separate, mobile version of the website in favor of a single, responsive website that could provide comprehensive functionality to all patrons, regardless of what type of device they were using. This strategy not only followed website best practices, 12 but was also important for compliance with Congressional legislation aimed at improving the digital experience for all individuals. 13 Using multiple data approaches enabled us to develop an understanding of what the PubMed user experience was like for individuals with mobile devices who were redirected to the separate, mobile site. Fully understanding the disparity in experiences between desktop and mobile users guided the product development and gave us a basis by which to measure improvement.
NLM runs surveys on its websites routinely to gauge and monitor customer sentiment toward its products. In the early research phases of PubMed modernization, a survey was conducted on both the desktop and mobile versions of PubMed. The final question of the survey asked respondents, "How likely are you to recommend this website to a friend or colleague?" which is an industry-standard question that is used to develop a metric known as a "Net Promoter Score" (NPS). 14 Survey takers are prompted to respond with a whole number value ranging from 0 (Not at all likely to recommend) to 10 (Extremely likely to recommend). Respondents who give scores ranging from zero to six are classified as "detractors," those that give scores of seven or eight are classified as neutral, and those giving scores of nine or ten are classified as "promoters." The percentage of promoters minus the percentage of detractors gives a final score from À100 to 100. The creators of NPS, Bain and Company, classify NPS scores as follows: Above zero is good, above twenty is favorable, above fifty is excellent, and above eighty is "world class." 15 For reference, the median NPS for technology companies, according to SurveyMonkey.com, is þ40 with the top quartile scoring þ64 and higher. 16 The PubMed FY19 baseline survey gave a composite NPS of 62.3 from 2,610 respondents which would put the site comfortably in the "excellent" category. However, when separating the responses according to device and calculating the NPS, desktop users gave a score of 75.3 (n ¼ 1,662) and users responding from a mobile device gave score of 37.4 (n ¼ 948), which was a red flag signaling the disparity in experiences. Furthermore, website analytics revealed that desktop visitors to PubMed were much more likely to run a search than visitors using a mobile device (36.8 vs. 8%). The PubMed mobile pages were designed with a hyperlink at the bottom of every page that would bring the user to the desktop version of PubMed when clicked. Just over 10% of PubMed mobile users opted for the desktop version of the site, and of that population 20.6% ran a PubMed search. The lower NPS for mobile users of PubMed, combined with the website analytics that revealed user behavior, demonstrated that full functionality was desired by a significant portion of the mobile device population. The separate mobile implementation of PubMed was not delivering on this need.
The updated version of PubMed which was the result of user interviews, usability tests, design studios, product meetings, presentations at conferences and medical libraries, and over a year of beta testing, was set as the default version for the general public in May 2020. Users of PubMed on a mobile device now were able to enjoy the same experience and tools as everyone else: filters, saved searches, collections, "cite" feature (used in the writing of this article), clipboard, and the PubMed advanced search builder. In fact, the mobile experience tested so well that the separate, mobile-only version of legacy PubMed was retired in March 2020, a full 2 months prior to the general launch. Several months after the full release, a follow up survey was run to measure how well the new version was received by the public. The aggregate NPS increased from 62.3 (2019) to 73.1 (2020). The NPS for desktop users remained steady, improving slightly from 75.3 (2019) to 76.2 (2020). What drove the change in the aggregate score? The mobile population, which surged from an NPS of 37.4 (2019) to 66.7 (2020), clearly showed a tremendous improvement for that segment of users.
On an average workday, there are roughly double the number of users accessing PubMed via a desktop computer versus a mobile device. However, a curious change to this ratio can be observed with regularity when the typical academic institution is on recess: over the summer months and over the winter break. During these periods, PubMed usage via a mobile device often overtakes desktop usage on the weekends and holidays (Figure 4). Whereas typical desktop usage will show a large variation during the weekday compared to the weekend (roughly halved), mobile usage of PubMed exhibits a much less pronounced change, more on the order of 20% fewer users.
The importance of mobile devices and their growth in use over the past decade cannot be understated. Smart phones are now ubiquitous and are used to quickly look up information while working in the lab or clinic, in a lecture, or on holiday. Medical libraries responded to seismic shifts in service delivery and user expectations in the 1990s with the advent of the internet age. The rise of the smart phone beginning in the 2010s has had a similar impact upon daily life and can be considered a second wave within that revolution. As library professionals, while we may still be accustomed to doing the majority of our work on desktop computers, we should not overlook the experience of our patrons which may be markedly different from our own.

From PubMed labs to a product-centered organization
Building the modernized, mobile-friendly PubMed was transformative for NLM. Leadership saw the success of the user-centered, data-driven approach to product improvement that PubMed delivered and began to explore how to capture that approach and apply it to other NLM products and services. Luckily for NLM, USDS has encapsulated its recommendations into a clear and easy-to-follow Digital Services Playbook, 17 covering everything from data-driven decision-making to understanding what users need and delivering it. But one "play" in particular helped NLM to deliver PubMed effectively: to build "using agile and iterative practices."  Perhaps no recent topic in product and technology development has been the subject of more debate, writing, and misunderstanding than Agile product development, but it is outside the scope of this present work to explore in anything more than a cursory way. What can be said is that developing in small pieces of user-facing functionality, pieces that users can see and use and test and break, makes for better outcomes. We can say this with confidence based on the PubMed modernization experience and that of other NCBI Labs experiments. By breaking down the development of large, complex systems into a series of smaller features and functionality, NLM was able to learn throughout the development process, adroitly modifying features to meet user demands even as those features were rolled out.
It is also worth noting that PubMed modernization would not have happened without careful stewardship of the time and effort spent by developers, subject matter experts, and others. NLM has a massive number of production systems and services, so every resource, human or technical, is precious and needs to be used with rigorous efficiency. PubMed Labs had very clear targets and goals, as mentioned previously, and we would argue that it was able to meet those goals all the way to a successful launch in large part due to the clear identification of those goals and the application of agile development with data-driven decision making and prioritization.
As NLM management saw these outcomes, they realized that many NLM products and teams could take advantage of these "plays." We began to think increasingly of our work, particularly in the context of planning and execution, as a series of products. Each product is a real thing that users see and interact with; it has its own users with needs, goals, and ways to measure success.
While developing the new version of PubMed, we knew from long experience that some users perform simple searches while a subset of more sophisticated users require an advanced search. We focused on the simple search case first because it is easier and quicker to build, but also because even sophisticated searchers need a simple search function. Thus, we started with the basics, and only when that was refined via user feedback did we move on to building a modernized advanced search. At each step, whether in individual user tests or in surveys on PubMed Labs, we looked to see if users found the new search easier to use. In this case, as noted previously, we found NPS to be a useful success metric, particularly in looking at users accessing PubMed via a mobile device. When mobile users gave the new search higher marks, we knew we were on the right track.
We encourage product teams to limit the number of specific metrics tied to a particular goal and to focus efforts on those specific outcomes. Given that many NLM resources consist of databases that can be searched, a common product goal is to increase the percentage of users who indicate "very easy" or "easy" to the question "how easy was it to find what I was looking for" on our standard survey. A rising proportion of "very easy" responses to that question is a clear indicator of improvement. This is not to say that this one metric, the success metric, would be the only measure used to evaluate a product. All products have a wider set of Key Performance Indicators (KPI) which can be used to evaluate the health of a product or to establish that it is meeting certain minimum or expected standards. For example, speed of page load, speed of returning search results, and overall system availability, also called uptime, are all important for PubMed users. Pages and search results should load quickly, and the service needs to be "up" essentially all the time. But the difference with KPIs, compared to success metrics, is that as long as we are hitting our thresholds (sometimes called service-level objectives or SLOs), we wouldn't expect or try to deliver continual improvement in those numbers. Once we have achieved a reasonable search result load time, say under one second, we would not optimize for ever-shorter load timesthat would have diminishing returns for users and higher costs for NLM. If the load time and availability are within our target, we can focus on the things we want to improvethe things that users want and neednamely, an easier and more successful search experience.
This focus on success metrics allows product teams to prioritize their work by focusing on those features and improvements most likely to drive the success metric in the right directionin other words, the outcome users need.

From goals to products
To start the process, product teams establish a document we call the "Vision/Goals Canvas" containing a long-term product vision, design targets, problem statements, goals, and success metrics, plus one additional element: initiatives. Initiatives are the "how"how we will address the problem statement to reach our goals. For example, if a design target for, say, PubMed is clinicians in a clinical setting such as a hospital, the problem might be that it's difficult to use (legacy) PubMed on a mobile device. Then the goal is to make it easier to use PubMed on mobile devices and the associated success metric is a mobile NPS on the new PubMed. Framed that way, the PubMed team came up with an initiative to develop a better, more responsive (i.e. flexible display on multiple devices) version of PubMed. Each product has a set of goals and a related set of initiatives by which the teams hope to enable their products to meet those goals, all measured and confirmed via the appropriate success metric.
Once a team has completed its Vision/Goals Canvas, they work to break down the initiatives into manageable pieces. The most successful NLM teams address initiatives via the "user story" format, to wit: "as a [type of user], I want to [complete a task] so that [I get the outcome I want]". A specific example for PubMed would be, "As a hurried clinician in a clinical setting, I want to find PubMed citations quickly so that I can get information immediately relevant to my work at hand." This example is what most agile practitioners would call a "saga", meaning quite a substantial body of work, taking weeks or months to complete. Sagas can be broken into smaller "epics" consisting of still-fairly-large feature sets. For example, if we consider the PubMed home page with its search form in isolation from the actual searching process, the search results, and the display of citation information, an epic might be, "as a PubMed user with a phone, I want to run a PubMed search so that I can see the results and choose one to view." In practice, epics are often the logical unit that is built and delivered to users for evaluationanother way to think of "features".
Product planning at NLM combines the goals and initiatives from the Vision/Goals canvas and the user story/saga approach to create a paired set of planning artifacts: a long-range plan ("strategic roadmap") showing the initiatives in rough time and priority order over a multi-year period, and a shorter-term "release plan" that lays out the likely epics to be delivered in the next few months and succeeding calendar quarters. Often, release plan item scheduling is based on product team priorities, but as part of a public agency, sometimes NLM product teams need to incorporate specific policy and management directives, prompting an update to the release plan or strategic roadmap. These tools provide excellent transparency for management and allow product teams to clearly articulate the impacts of mandates on product feature releases and the related success metrics, often leading to a robust discussion of alternatives and contingencies. A recent specific example was the COVID-19 pandemic, which required a substantial realignment of resources to handle large amounts of new data and associated data, and reporting requirements across multiple products. Accommodating this was a challenge, but the challenge was manageable because we had clearly prioritized existing work plans already laid out; this made it easier to move work around on the release plan to accommodate both the new COVID requirements as well as existing priorities.
NLM's dozens of product teams took several calendar quarters to build out Vision/Goals Canvases, Strategic Roadmaps, and Release Plans for every product. We held an initial set of quarterly business reviews with each product team: first to review the Vision/Goals canvas, then the Strategic Roadmap and Release Plan, before settling into regular cycle of reviewing progress each quarter. In the reviews, we looked at any changes that have been made, most often to the Release Plan, as epics are delivered and detailed monthly planning is conducted, along with a review of progress toward success metric objectives. Product teams also use the quarterly review to communicate any accomplishments, risks, and opportunities that might require a change in budgets or staffing.

Automating MEDLINE indexing in PubMed
MEDLINE, developed in 1971, contains more than 29 million references in life sciences, concentrating on biomedicine, from more than 5,200 worldwide journals in about 40 different languages. A valuable feature of MEDLINE is that the citations are indexed with NLM's Medical Subject Headings (MeSH), a controlled, hierarchical biomedical vocabulary for indexing, cataloging, and searching biomedical information. MEDLINE is the largest subset of PubMed which, as stated previously, is accessed by more than 3.5 million users daily for search and retrieval of biomedical and life sciences literature.
Medical research and associated biomedical literature have increased exponentially in volume and scope, resulting in an ever-increasing backlog of citations needing to be indexed in MEDLINE. 18 In 2021, over 1.2 million citations were indexed and added to MEDLINE, and the National Center for Science and Engineering Statistics predicts a steady increase of about 4% annually ( Figure 5). 19 With an ever-increasing volume of citations in MEDLINE and flat resources, it was imperative to ensure we were meeting our customers' needs. In December 2016, NLM launched a MeSH Indexing Assessment Project to evaluate MeSH terms currently assigned to MEDLINE citations, to examine what is the impact of indexing? We specifically wanted to know who are using MeSH citations, how are they using them, and for what purpose.
The project findings confirmed the value of MEDLINE indexing and the value of applying selected non-subject metadata, such as publication type, Implement fully automated MEDLINE indexing with human quality controlcontinue MeSH indexing through a fully automated process with human quality control, using the Medical Text Indexer (MTI), PubTator, and other tools developed by the Lister Hill Center and NCBI. Apply human curation and automation to improve the discoverability of chemical and gene information in MEDLINEimprove the discoverability of chemical and gene information in MEDLINE citations by developing specialized automated methods for identifying and indexing chemical entities and gene names. Merge the Indexing Management System (IMS) with the PubMed Data Management system (PMDM) to support full automation and chemical and gene curationcreate a unified application to load, store, and update PubMed citations that will optimize data processing, eliminate errors associated with storing data in separate applications, and increase efficiencies. Investigate the automation of data review and develop an implementation plan if feasibleevaluate, after the planned merger of systems, the need for manual data review for metadata such as links to errata, adding research funding support, adding databank names and identifiers like ClinicalTrials.gov, and decide whether any aspects of the process can be automated.
Automation efforts are not new to NLM. In fact, NLM was already spearheading an Indexing Initiative project in 1996 through research conducted in one of NLM's divisions, the Lister Hill National Center for Biomedical Communications, to respond to the increasing output of biomedical literature and lack of resources. 20 This early team was charged with automating indexing methodologies on MEDLINE citations while maintaining the high-quality associated with human subject matter experts. There has been some form of automation used for the past 20 years through the NLM Medical Text Indexer (MTI), the main product of the Indexing Initiative. 21 MTI has provided MeSH vocabulary recommendations to human indexers since 2002. Furthermore, since 2011, NLM expanded the use of machine-only indexing on a select set of journals as a new product, MTI first-line (MTIFL) was implemented. 22 As a result of automating MEDLINE indexing, NLM can now maintain the increasing number of new publications each year, deliver high quality curation services within days (precision rates track at 90% overall 23 , with MeSH 90%, Check Tags 92%, and Supplementary Concept Records or SCRs 93%) not months, and focus on adding additional data access points (chemical, drug, and gene data) to MEDLINE to make those citations more discoverable. 24 That said, automated indexing can miss concepts, particularly for chemicals. This is why articles involving such concepts are specifically curated by humans so that such terms are added, and why we are focusing our efforts on improving the algorithm's performance for chemicals.
We are expanding automation efforts to include accurately identifying chemicals, drugs and genes used in articles, to allow users to locate literature focusing on these topics. Our goal is to focus on how users function and use products (e.g., literature searches) and to streamline the processes involved to facilitate the efficient and timely identification of relevant literature and linkages across products for our users to discover. To do this, we must lean into seeking and collecting relevant data and information about how our products (e.g., databases) are being used and how the experience could be improved, and connect our users to other related biomedical information. It is only through seeking these types of data from our users that products like MEDLINE have continued to flourish for over 50 years. 25

Modernization moving forward
NLM is now about two years into the product transition of PubMed, and this year MEDLINE has literally flipped the switch to fully automated indexing. Using our product principles, we have identified goals and success metrics to determine product status and progress toward meeting user needs and goals. NLM looks to continued change and modernization as a way of life within our organization. NLM has been evolving and setting the standard throughout its history. For example, the inception of using the Medical Literature Analysis and Retrieval System (MEDLARS) in the early 1960s, to automate records for new books, was an important first step for reimagining medical libraries. No longer just a place to store books and journals, medical libraries are active information centers, using television, telephone, and computer technology to develop networks for communicating biomedical information quickly and to evaluate and develop new systems for information storage and retrieval. 26 More recently, the COVID-19 pandemic has given us the opportunity to rethink, redesign and rebuild for a better future. As we continue to modernize, we are thinking beyond our systems, databases, products, and resources, to our number one asset, our talented staff. To attract and retain an engaged, productive, and diverse workforce that makes us the global information powerhouse that our users rely on, we must ensure that staff are provided every opportunity to develop in their careers and expertise and feel supported through new workplace flexibilities that will enable them to thrive. The use of our physical spaces will need to continue to change over time as we adapt to evolving physical and virtual spaces and user expectations, while supporting a diverse and inclusive workforce.
As NLM Director Dr. Patricia Flatley Brennan explained, "Data is not enough. Learning how to organize and share data is not an intuitive skill that most scientists have, however it is a skill that our librarians have". 27 With a sharp focus on changing user needs, we are modernizing our operations and physical plant to be able to curate at scale, optimizing for continuous increase in biomedical literature and data, and providing users easy, cross platform access to our products, services, and data. As one of the 27 institutes of NIH and a national library, NLM continually adapts to such changes in the research ecosystem, and moreover, serves as an engine for innovation and scientific discoverability.

Funding
This work was supported by the National Library of Medicine at the National Institutes of Health.

Notes on contributors
Dianne Babski, MIM (dianne.babski@nih.gov), is the Associate Director for Library Operations at the National Library of Medicine, National Institutes of Health, US Department of Health and Human Services, Bethesda, Maryland, USA.