Study design

To estimate the perfect- and typical-use effectiveness of the Dot app for pregnancy prevention, we adapted guidelines for contraceptive effectiveness studies [8], including FABMs [9], to the context of a fertility app [7]. The study design reflects relevant points included in a recent systematic review of FABM effectiveness studies [10]. This study was approved by Georgetown University’s Institutional Review Board. The protocol is available at ClinicalTrials.gov (NCT02833922).

Participant recruitment

We estimated that we needed to recruit over 700 women in order to achieve a sample size of at least 255 women completing 13 cycles. This sample size provided 90% power to detect a 6% decrease in the 1-year pregnancy rate of app users, with one-sided type I error at 5% [11]. When the test was used for claiming effectiveness, the more stringent one-sided significance level of 5% was used [11]. Study participants were recruited between 7 February 2017 and 10 August 2017 from a pool of new Dot users who lived in the continental United States. Prior to enrolment, participants had downloaded the app on their Android phones, chosen to prevent pregnancy, entered their second menstrual period start date, met initial recruitment criteria and replied affirmatively to a pop-up message describing the study and asking if they were interested in participating [7,11,12].

Inclusion criteria

We aimed to recruit a study population of fertile women who were at risk of pregnancy and wanted to avoid pregnancy for at least 1 year. As described by Shattuck et al. [12], 42,788 women downloaded the Dot app for pregnancy prevention during the recruitment period, of whom 3014 responded to an in-app survey indicating an interest in participating in the Dot study. Some 2932 (97.3%) women responded to a pre-eligibility survey asking whether they were 18–39 years old, intended to use Dot for pregnancy prevention for at least 1 year and had not used hormonal contraception in the last three cycles. The 1759 (60.0%) who were interested and pre-eligible were further screened to determine whether they were sexually active with a male partner (or partners), had consistent cycles between 20 and 40 days long with less than 10 days’ variation and had had at least three menstrual periods following the most recent (if any) pregnancy [12]. Pre-eligible women completed the full eligibility screening, received further information about the study and completed an in-app informed consent process. Due to multiple steps required in the original enrolment process [12], enrolment was low during the early months of recruitment. With a streamlined process, enrolment increased to 718 (40.8% of those interested and pre-eligible) participants [7,12].

Data collection

Study data were collected between 7 February 2017 and 15 October 2018. To facilitate data collection and study management, we developed Proofmode, a multi-component research platform that was integrated into the Dot app to securely capture, store and send study data to the research team [12]. Proofmode stored data on each participant, including their responses to an initial sociodemographic questionnaire, their self-reported menstrual period start dates and Dot-calculated fertile window dates for each cycle. Proofmode also collected other study-related data including daily coital diary and use of other contraceptives (i.e., condoms, withdrawal and emergency contraception). At the beginning of each cycle, Proofmode sent pop-up questions inquiring about pregnancy intention and importance of preventing pregnancy at that time. Proofmode also facilitated periodic surveys and an exit survey [12].

We also used Amplitude app analytics software (https://amplitude.com) to review participants’ app use. We combined information from Proofmode, Amplitude and interviews (in the case of pregnancies) to categorise pregnancies resulting from ‘correct’ or ‘incorrect’ use of the app. Proofmode and Amplitude collected data including, but not limited to, how many times the app was opened, entry of a menstrual period start date and sexual history data. Both Proofmode and Amplitude are described in more detail elsewhere [7,12].

Determining pregnancy

According to the study protocol [11], a participant’s entry of a menstrual period start date was an indicator that she had completed the prior cycle without becoming pregnant. If she reached her 41st cycle day without entering a menstrual period start date, she received a pop-up message inquiring if she was still interested in continuing the study. If she indicated yes, she was asked to enter her menstrual period start date. A woman whose menstrual period start date fell after the 40th day was withdrawn from the study (see inclusion criteria). If she did not respond, a subsequent pop-up message asked if she thought she might be pregnant, and the study team actively followed up [7]. On any day of the cycle, if a participant reported that she thought she might be pregnant, we directly contacted her via phone or email to schedule an interview regarding her pregnancy. We asked: (1) if the pregnancy was confirmed by a urine pregnancy test and/or a health care provider; (2) whether the participant had changed her pregnancy intention since last reporting; and (3) questions around method use and sexual activity during the cycle that resulted in a pregnancy.

With the participant’s consent, we express-mailed potentially pregnant participants two urine pregnancy tests with instructions to confirm the pregnancy, either by digital image or free return shipment [7,11]. Women who returned a positive pregnancy test (n = 10) or confirmed verbally by phone (n = 5), email (n = 8) or Google survey (n = 2) that they were pregnant were considered pregnant.

Study discontinuation

All women who exited the study were classified as pregnant, not pregnant or lost to follow-up. There were a variety of reasons why women exited the study other than pregnancy, including being ineligible because: (1) they had long (>40 days), short (<20 days) or variable (>9 days) cycles; (2) they experienced a technical or phone issue; (3) they were no longer using the Dot app or were not using Dot to prevent pregnancy; or (4) they no longer wanted to be in the study [7]. Women we classified as lost to follow-up after they stopped using the app and no longer responded to study representatives, in which case they were categorised as possibly pregnant or very unlikely to be pregnant based on their coital data and the timing of their discontinuation as described below [7]. Women who discontinued using the app and did not respond to the in-app messaging were actively followed up via email, phone or Google survey. In addition, women who reported having a long cycle (>40 days) were followed up in the same process as women who reported a potential pregnancy.

Data analysis

We calculated Dot’s 13 cycle typical- and perfect-use effectiveness and related confidence intervals (CIs) using multi-censored Kaplan–Meier life-tables. We also calculated cycle-by-cycle effectiveness failure rates. We descriptively analysed our study sample and study participants’ reasons for discontinuation. Statistical analyses were carried out using Stata, version 15 (StataCorp LLC, College Station, TX, USA). To determine typical-use effectiveness, we conducted life-table analysis using all pregnancy cycles and correct and incorrect use cycles in which intercourse was reported at any point in the cycle. We defined correct use as either use of a condom or not having intercourse during the days that Dot identified as potentially fertile. We defined incorrect use as cycles with one or more instances of unprotected intercourse during the days that Dot identified as potentially fertile; unprotected intercourse included the use of withdrawal, emergency contraception, some ‘other’ unspecified method and/or no method [7,11].

To determine method effectiveness, or, the perfect-use pregnancy rate, we considered only cycles in which Dot was used correctly. We assessed adherence to the method by calculating the proportion of completed perfect-use cycles to all completed cycles in the study. For both the typical- and perfect-use calculations, we censored cycles in which: (1) no sexual history data were entered; (2) no intercourse was reported; or (3) the participant exited the study prior to cycle completion (e.g., self-exit, lost to follow-up, ineligibility) [7].

Additionally, we conducted a sensitivity analysis to assess potentially unaccounted-for pregnancies. Women who discontinued use after the onset of the fertile window and reported unprotected intercourse one or more times during the fertile window were classified as ‘possibly pregnant’. Women who discontinued prior to the onset of the fertile window or did not report unprotected intercourse during that fertile window were classified as ‘unlikely to be pregnant’ [7]. Using the categorisations above (‘pregnant’, ‘possibly pregnant’ and ‘very unlikely pregnant’), we calculated separate life-table estimates and CIs for the ‘pregnant’ and ‘possibly pregnant’ scenarios. Given the definition of ‘very unlikely pregnant’, we did not conduct analyses for this scenario. A survival analysis was conducted to identify associations between risk of pregnancy and demographic factors such as relationship type, age group and educational level [7].

Finally, we conducted both attrition and descriptive analyses of study discontinuation. Comparisons considered age, ethnicity, type of relationship with partner(s), prior pregnancy, coital behaviour and use of condoms, withdrawal and emergency contraception during the pregnancy cycle.

Participant profile

The average age of women in the Dot study was 29 (n = 718), with 83% of participants almost evenly divided among 18–24 (27.6%), 25–29 (32.0%) and 30–34 (23.3%) age groups. A smaller proportion (15.7%) were aged 35–39. As shown in Table 1, more than half of the participants identified their race as Caucasian/White (55.2%), and about one-fifth identified as African American/Black (18.7%) or Hispanic/Latino (16.4%). Almost half of participants described their relationships as long term (46.9%), with approximately a quarter reporting that they were married (27.0%). The others reported a mix of relationship statuses. Nearly 75% (n = 535) of study participants reported having ever-used hormonal birth control, with a majority reporting pills (64.3%), followed by injections (13.1%), implants (9.0%), intrauterine devices (4.5%), vaginal ring (4.1%), patch (2.2%), and two or more methods (2.8%) (data not shown). Slightly over half of participants reported having previously been pregnant (52.2%). A Fisher’s exact comparison showed no statistical differences in age (p = .892), race (p = .591), relationship status (p = .764) or prior pregnancy (p = .023) between participants who became pregnant and the entire cohort.

Perfect- and typical-use effectiveness of the Dot fertility app over 13 cycles: results from a prospective contraceptive effectiveness trial

All authors

Victoria Jennings, Liya T. Haile , Rebecca G. Simmons, Jeff Spieler & Dominick Shattuck

https://doi.org/10.1080/13625187.2019.1581164

Published online:

18 March 2019

Overview of pregnant participants

Over the course of 13 cycles, 25 women became pregnant, of whom 24 reported having intercourse without a condom during the fertile window. One pregnancy occurred in a cycle in which the participant used Dot correctly, reporting three acts of intercourse during the fertile window, all with condom use. Fifteen of the 25 pregnancies occurred within the first six cycles of the study. As shown in Table 1, 40% (10/25) of women who became pregnant were aged 25–29, 52% (13/25) were White/Caucasian and 88% (22/25) were married or in a long-term relationship. Slightly more than half (56%, 14/25) had previously been pregnant. There were no statistical differences in the proportion of women who become pregnant across age, ethnicity, relationship status or ever-pregnancy.

Typical- and perfect-use effectiveness and discontinuation

The cohort of 718 women aged 18–39 provided 6616 cycles used in the analyses. The vast majority of participants (99.0%) who completed a cycle completed 100% of their daily coital diary. On average, study participants reported having intercourse 5.2 times per cycle (±4.9) across 13 cycles of use. Censoring cycles for which no sexual history data were entered, no intercourse was reported, or exiting the study prior to cycle completion (e.g., self-exit, lost to follow-up, ineligibility) resulted in the exclusion of 1219 cycles. We found a 13 cycle typical-use life-table failure rate of 5.8% (95% CI: 3.57%, 8.09%) (Table 2). All 25 pregnancies identified in these data were reported as unintended. Twenty-four percent of cycles (n = 1609) were identified as perfect use, and one perfect-use pregnancy was reported in cycle 11. Thus, the 13 cycle perfect-use failure rate was 1.0% (95% CI: 0.9%, 2.9%).

Perfect- and typical-use effectiveness of the Dot fertility app over 13 cycles: results from a prospective contraceptive effectiveness trial

All authors

Victoria Jennings, Liya T. Haile , Rebecca G. Simmons, Jeff Spieler & Dominick Shattuck

https://doi.org/10.1080/13625187.2019.1581164

Published online:

18 March 2019

We analysed the reported reasons for exit for the 718 participants. Three hundred and thirty-six participants (347 minus the 11 who exited) completed 13 cycles in the study. The most common reasons for early exits included women no longer being eligible to use Dot for pregnancy prevention (long or short cycles, >10 days’ variation in cycle length), being lost to follow-up, and no longer wanting to use Dot to prevent pregnancy (Table 3).

Perfect- and typical-use effectiveness of the Dot fertility app over 13 cycles: results from a prospective contraceptive effectiveness trial

All authors

Victoria Jennings, Liya T. Haile , Rebecca G. Simmons, Jeff Spieler & Dominick Shattuck

https://doi.org/10.1080/13625187.2019.1581164

Published online:

18 March 2019

Sensitivity analysis

Among women categorised as lost to follow-up (n = 92), 33 were further categorised as possibly being pregnant. Although pregnancy status among these participants was undetermined, including these possible pregnancies in a sensitivity analysis resulted in a worst-case typical-use life-table failure rate of 12.8% (95% CI: 9.66%, 5.99%). We also identified 59 lost to follow-up who were categorised as very unlikely to be pregnant. In 52 of these cycles women reported that they did not have unprotected intercourse during the fertile window prior to becoming lost to follow-up and seven women stopped using the app prior to the onset of the fertile window, which we were able to detect via Amplitude and Proofmode data.

Additional analyses

Findings and interpretation

Our findings indicate that Dot is effective for pregnancy prevention and most women are able to use it successfully according to their intention to prevent pregnancy. With only one perfect-use pregnancy (and thus a perfect-use failure rate of 1.0%) and a typical-use failure rate of 5.8% for women aged 18–39, Dot’s effectiveness is comparable to that of a number of other user-controlled contraceptive methods and other FABMs [1,13,14]. This is important given that Dot requires a user to enter only her menstrual period start date, making it less burdensome for the user than other FABMs. As shown in the survival analysis, there were no statistical associations between sociodemographic characteristics and pregnancy, suggesting that Dot can be used successfully for pregnancy prevention by women of different ages, ethnicities and relationship types.

The coital frequency of study participants was similar to that of other FABMs. While there were no racial/ethnic or relationship-type differences, the attrition analysis found that younger women were more likely to exit the study prior to completion compared with older women, though both age groups had similar reasons for discontinuation (change in their purpose for using Dot, cycle length and variability, and loss to follow-up). Indeed, age is often associated with higher rates of method switching [15]. Furthermore, the average monthly coital frequency of Dot study participants (5.2 ± 4.9) is similar to the mean monthly coital frequency from Demographic and Health Survey data and that of other women who use FABMs [16].

Importantly, the design of apps provides for new opportunities to avoid gaps in contraceptive coverage or facilitate pre-pregnancy health by alerting women when they no longer meet the required inclusion or eligibility criteria (for pregnancy prevention).

Strengths and weaknesses

This study has several strengths that distinguish it from studies of other app-based FABMs [14,17,18]. The prospective 13 cycle cohort design, use of multi-censored life-table analysis, clear inclusion and exclusion criteria, a defined protocol for classifying pregnancies and real-time data collection were critical components of the study. Daily in-app data collection, use of app analytics and rigorous follow-up strategies allowed us to monitor the study progress in real time and resulted in very little missing data and low loss to follow-up (12.8%). The percentage of participants lost to follow-up was well below the 20% suggested by several contraceptive effectiveness experts [19,20]. The study includes coital behaviour, which participants were prompted (via app notification) to record daily. Our participant cohort reflected the general sociodemographic breakdown of the broader US population [21] and consisted of women who were likely to be at risk of pregnancy and prospectively declared their pregnancy intention and the importance of avoiding pregnancy.

While self-reported coital data are impossible to confirm, our use of technology allowed us to collect these data relatively unobtrusively and to capture the time and date data were entered, increasing our confidence in the reports [7]. Additionally, we used a combination of study data and analytical information reported in the app to determine possible or potential pregnancy among women who were lost to follow-up, which strengthens confidence in the study results.

This study also has some limitations. Traditional effectiveness studies use physical sites for participant reporting, which usually includes physical examinations that might reveal comorbidities that could result in subfertility or an undetected pregnancy. As our study was virtual, it was not possible to conduct such examinations and tests, so we relied on self-report. Further, one pregnancy occurred during a cycle in which the participant reported using a condom on all days of the fertile window she had intercourse. While our definition of correct use (abstaining or using a condom on fertile days) conforms to Dot’s instructions, it is impossible to know whether this pregnancy occurred because Dot incorrectly identified days of ‘low’ or ‘high’ risk or whether the condom failed. Additionally, our participant interaction was limited to phone, chat and email, which may not be the most effective mode of study. The use of Proofmode facilitated the ease of data entry but did not change the typical use of the app. It is possible that interactions from the study team members could have changed the user’s interaction with the app. Given this possibility, the study team had minimal contact with participants.

While it is assumed that women who enrol in contraceptive effectiveness studies of any kind are more motivated than the general public and may offer bias, this is balanced by the enrolment of users who self-selected to participate after downloading the app, selecting prevent pregnancy, and entered their second menstrual period start date, the very low burden of data entry and the diversity of the study sample.