Lessons Learned from Reevaluating Big Deals with Unsub

Abstract The value of big deals is increasingly unclear. This article briefly discusses factors others have considered in evaluating big deals and covers the four factors that should be considered moving forward: open access, interlibrary loan, post-termination access, and a-la-carte costs. Unsub, a tool for reevaluating big deals created by the nonprofit OurResearch, is introduced. Lessons learned are shared from two years of helping libraries reevaluate big deals to provide insight into the complexities and tradeoffs involved in evaluating big deals across many libraries.


The value of the big deal is declining
Bundles of journals started to be sold to libraries by Academic Press (now part of Elsevier) in 1996 (Coghill, 2019;Galbraith & Hess, 2020) and came to be known by 2001 as the "Big Deal" (Frazier, 2001). The impetus for the big deal appears to have been to reduce costs, but due to a variety of reasons big deal costs have only increased (Frazier, 2005;Galbraith & Hess, 2020) while library budgets continue shrinking (Bosch et al., 2022;Coghill, 2019) leading many to the conclusion that the value of the big deal is declining (Schonfeld, 2019). When considering whether to renew or cancel a big deal, we should not do so blindly, but instead let data drive the decision making process.

Reevaluating big deals
Learning from and building on past work on evaluating big deals, what factors should be considered when reevaluating big deals? In evaluating big deals for renewal or cancelation, the following are a sampling of different approaches: McLean and Ladd (2021). Five factors: cost per use, pdf downloads, authorships, citations, faculty and graduate student survey scores. Li and Eggleston (2017). Six factors: cost per use, cost per citation, impact factor, impact per paper, usage, and citations. Arnold and Mays (2019). Four factors: cost per use, usage, authorships, citations. Johnson and Cassady (2018). Survey of librarians involved in assessing big deals; the survey found two clusters of librarians, those that favored more subjective factors (faculty consultation, subject knowledge, importance to discipline) and those that favored more quantitative factors (cost, usage, faculty survey data). Nabe and Fowler (2012). Three factors: cost per use, usage, impact factor.
The above are representatives of the factors considered when evaluating big deals. There is clear indication of the factors librarians think are important to consider; for example, cost per use is clearly the biggest factor. However, there are a number of factors not included or at least very rarely included in reevaluating big deals.
First, let's consider open access. The fraction of newly published scholarly articles that are open access is increasing every year. Thus, open access is a factor that can not be ignored. Second, interlibrary loan (ILL) is generally not considered in published works about reevaluating big deals. ILL can be an important factor replacing licensed access with relationships with other institutions or document delivery through third parties (e.g., RapidILL). Third, when a big deal is canceled, some titles that were part of the big deal include post-termination access (PTA). This access is a crucial missing piece that isn't often considered and is especially important in the first few years after big deal cancelation when the library still has access to a majority of the new articles from a title with PTA.
There are a few important changes to cost per use that should be made. First, when including open access and PTA it makes more sense to use cost per controlled use so that the CPU value only includes uses that are paywalled. Second, CPU can incorporate citations and authorships into the calculation of the use component of CPU, with citations and authorships optionally weighted according to their perceived importance.
A web application Unsub (https://unsub.org/) takes into account these four factors and creates a customized forecast over the next five years (Figure 1). Unsub is a dashboard that helps a librarian reevaluate their big deal's value and understand their cancelation options-whether you want to get a better deal, cancel a big deal, or only model title by title subscriptions

Lessons learned
The following are lessons learned from a few years of reevaluating big deals with Unsub:  (Piwowar et al., 2018). We now have high quality data on Open Access through Unpaywall (https://unpaywall.org/).

Post-termination access and interlibrary loan.
There's a "masking" effect of post-termination access (PTA) on interlibrary loan (ILL), at least for journal titles that have PTA following the cancelation of a big deal. That is, directly after cancelation of a big deal, the library has access to all newly published articles for the title. As time goes on, the library has a decreasing proportion of the newly published articles. We know from Unpaywall what people actually read-and what they read in most disciplines are the newest articles from the most recent few years. So, right after cancelation, the library has what people want to read-new articles. But as time goes on the library has fewer and fewer of the newest articles, leading to a potential increase in ILL. In summary, right after cancelation a potential increase in ILL is held off because of PTA. 3. Institution size correlates with cancelation savings.
Publishers generally do not charge different amounts for journals at the title level, but they do for big deals (Bergstrom et al., 2014). Thus, smaller schools save less money when canceling a big deal, while larger schools save more money when canceling, even if the percent savings is the same. The percent savings is likely the most important for the schools, but the absolute amount is an important indicator of the amount of profits steered away from the publisher. 4. No easy answer to how many titles.
When unbundling from a big deal, there will in all likelihood be a la carte subscriptions as a result. The next question is then how many titles to subscribe to. This is a difficult question to answer and depends on a lot of factors. There is a tradeoff between cost and convenient access for library users. 5. Saving money on a big deal is a big win.
Many people in the scholarly community are sympathetic to the goals of accelerating the move to open access publishing and decreasing dominance by the big publishers. Thus, although canceling big deals is great for both of these goals, renegotiating a better deal on a big deal is still a big win toward these two goals. 6. Tracking journal metadata is not easy.
Journal metadata is crucial to the forecasting that Unsub performs. Journal metadata is needed to make sure a journal is still being published, can be subscribed to (a journal that has transitioned to fully open access no longer has a subscription option), and more. Making sure we have up to date and correct journal metadata is rather complex because journals are often changing publishers, transitioning from paywalled to open access, and metadata can sometimes be wrong if publishers do not handle metadata correctly.
7. Factors we do not directly account for are still important. Any model (including that used in Unsub) makes a tradeoff between the number of parameters and parameter quality-do you use the few parameters with really high quality, or as many parameters as possible regardless of quality. As Unsub has to operate "at scale" (i.e., work for a lot of users), we don't use every factor individual schools may use. For example, McLean and Ladd (2021) described their approach to unbundling two big deals. The parameter they used that Unsub does not is survey data (of faculty and graduate students). In addition, there's some subset of titles a library will want to subscribe to regardless of cost per use or other metrics if they're deemed important for a focus area like a law or nursing school. 8. Documentation is crucial.
This point is not germane to the readers of this journal, but bears repeating here nonetheless. Unsub is a complex tool for good reasons-it is modeling a complex problem and there are many details involved. We are striving to increase the quality of our documentation to help Unsub users cancel big deals, get better big deals, and manage their a la carte subscriptions.

Conclusion
Serials collection management is in a moment of flux. It can be difficult to make decisions about serials collections when there are so many factors at play. Unsub makes it relatively easy to incorporate many factors to get a forecast of future cost and access.