2021 Nudge Symposium Abstracts

Abstract: Laboratory studies and field tests of interventions to increase COVID-19 vaccination intentions or uptake of other vaccines suggest that behavioral science can contribute to reaching population immunity. The first opportunities to field test such interventions in the COVID-19 context are with healthcare workers (HCWs), who are among the first to be offered COVID-19 vaccines and are important ambassadors for vaccine acceptance in the general population. While most HCWs employed by Geisinger, a large Pennsylvania health system, reported intentions to get vaccinated, many wanted to wait due to concerns about vaccine side effects and unknown risks. Geisinger’s initial communication strategy entailed sending 36 vaccine-related mass messages to all employees over five weeks, following which around 60% had scheduled their vaccinations.

We evenly randomized all 9,723 employees who had not scheduled a COVID-19 vaccination to a delayed control condition or to receive one of two individually-addressed emails that: a) explained that Pennsylvania would soon expand vaccine eligibility, reducing employees’ access to appointments; b) encouraged them to schedule an appointment; and (c) framed the decision either by 1) noting that many Americans and employees have chosen to be vaccinated (social norms) or 2) favorably juxtaposing the vaccine’s risks with those of COVID-19 itself (reframing risk). Both emails encouraged an active choice to receive a vaccine (automatically sending employees to a scheduling portal) or decline (automatically sending them to a survey soliciting their primary reason for hesitancy).

Geisinger employees are 73% female and 89% white, with an average age of 43. Both emails led to significantly more registrations than the delayed condition (social norms: OR = 2.11, 95% CI [1.65–4.21], p < .001; reframing risk: OR = 2.36, 95% CI [1.77–2.87], p < .001). There was no significant difference in registrations between the two email conditions, OR = 1.07, 95% CI [0.88–1.30], p = .498. Among 1,281 HCWs who declined to register and completed the survey, the most cited reasons were unknown vaccine risks (35%) and pregnancy-related concerns (13%).

Over three days, a single individual email nudge caused over twice as many HCWs to schedule their COVID-19 vaccination compared with HCWs in the control condition. This is consistent with findings that reminders can increase influenza vaccine uptake but the effect was larger than for most other reminder-based and health-related interventions. Given the immense amount of media attention and previous COVID-19 vaccine promotion to HCWs, it may seem counterintuitive that one reminder could increase vaccination by late adopters. However, competing demands on attention, behavioral inertia, and “sludge” that makes it hard to follow through on intentions likely conspire to make a single, timely, targeted reminder helpful. Our messages’ behavioral features—active choice, appeal to authority, and emphases on scarcity, social norms, and risk recalibration—may have contributed to their effect. Sending targeted messages designed with behavioral science can be a cost-effective way of encouraging “late-adopters” to initiate COVID-19 vaccination.

Abstract: Failure to get vaccinated for COVID-19 has a tremendous impact on how the virus curtailed. Even when the COVID-19 vaccines are available and free, many fail to get vaccinated. In this study, based on data of members at Israel’s largest health care organization, we aimed to assess the most effective behaviorally reminder to increase vaccination. The current study clarifies the value of two methods to encourage COVID-19 vaccination: Simple nudge reminders, and regulation that increases the benefit from vaccination. Our analysis of reminders is based on a controlled experiment. The results reveal that in the current context a reminder that emphasizes personal benefit is 9% more effective than a social benefit nudge. Our analysis of regulations can be described as a case study. It suggests that the green pass policy implemented in Israel on February 21, 2021 (and relaxes the social distance constraints for vaccinated individuals) was associated with an increase of 20% in the vaccination rate. We conclude with a discussion of the conditions that determine the relative effectiveness of personal benefit nudges.

Background: Providing appropriate, safe analgesia during the opioid crisis remains a challenge for Emergency Departments (EDs). Prescription Drug Monitoring Programs (PDMPs) have potential as a tool to improve prescribing decisions but are underutilized. Electronic health record (EHR) based clinical decision support (CDS) represents a patient-specific and scalable intervention to promote behavior change by modifying clinical workflows. Our objective is to formatively assess the development and early deployment of CDS tools designed to facilitate ED PDMP use.

Methods: This is an IRB-approved user-centered CDS development in a healthcare system with 10 EDs and 495,000 ED visits/year. Using published best practices and opioid risk factors, we developed two logic-driven patient-specific interruptive CDS alerts: 1) using PDMP risk criteria alone and 2) PDMP + EHR risk criteria. The alert logic was triggered for discharge opioid or benzodiazepine e-prescription orders and was suppressed if the PDMP was already reviewed during the encounter or for hospice, cancer, or sickle cell diagnoses. Iterative CDS modifications were informed by interviews with target adopters and organizational decision makers, including observations of prototype use. We then randomized providers to receive an interruptive CDS alert using one of the two logic profiles or a control alert (fired for any opioid and benzodiazepine e-prescription). The alert itself included patient- and logic-specific risk factors plus a link to the single sign-on PDMP activity. Alerts were activated in the system’s main academic ED (156 providers and 88,500 unique patients in 2020) for approximately one month prior to activation in all system EDs. Education was disseminated through meeting presentations, emails from departmental leadership, and a featured article in the health system EHR newsletter.

Results: Qualitative feedback from 20 interviews and 10 workflow observations identified concerns for workflow interruptions, alert fatigue, and alert suppression. To address these and quantify impact of alerts, we completed 3 months of silent testing on ED discharges: control alert triggered 8.1% (95%CI 7.9-8.3%) of all discharges, with variability across EDs (5.1-11.3%). PDMP only alert triggered 2.8% (95%CI 2.7-3.0%) before modification and 2.0% (95%CI 1.9-2.1%) afterward. PDMP + EHR alert triggered 4.2% (95%CI 4.1-4.4%) before modification and 3.7% (95%CI 3.5-3.9%) afterward. Early deployment in the main academic ED resulted in firing rates of 1.19% (95%CI 0.96-1.46%) for the control alert, 0.29% (95%CI 0.18-0.45%) for the PDMP alone alert, and 0.44% (95%CI 0.30-0.62%) for the PDMP+EHR alert. The median number of risk factors recognized by the PDMP and PDMP+EHR alerts were two per patient (n=55, range 1-5). Systemwide CDS rollout occurs this month, so firing rates, adoption, and PDMP use for all 3 CDS tools in EDs, inpatient facilities and ambulatory clinics are anticipated to be available soon.

Conclusions: User-centered design with key stakeholder input and pilot testing on a large set of target patients helped identify and address barriers to CDS adoption. Silent alert firing data can inform implementation decisions, validate monitoring tools, and give context to CDS education and messaging.

Background: Hand hygiene is a preventive approach that can prevent healthcare-associated infections, patient morbidity, and hospital length-of-stay. Many hospitals have tried multidimensional interventions to promote hand hygiene, but there are no studies in Saudi Arabia testing the effect of simple, low-cost interventions on hand hygiene performance. The objective of this study was to measure the impact of hand hygiene zones on emergency department healthcare workers’ compliance to performing hand hygiene before touching patients.

Methods Recruiting all emergency departments (EDs) that are under the Ministry of Health’s administration in Riyadh (N = 15), we used a computer program to randomize 14 EDs into treatment and control groups at a 1:1 ratio after stratifying by location (urban n = 6, rural n = 8). The intervention was sticker-created hand hygiene zones around each hospital bed in the treatment group. Hand hygiene compliance rates were measured for three and four weeks pre and post intervention, respectively, by covert observers. Recruited volunteer observers were trained by the General Directorate of Infection Prevention and Control to recognize and record the first moment of hand hygiene which occurs before touching a patient. Hand hygiene opportunities that occurred within sight of the observer were recorded as “performed hand hygiene properly,” “attempted to perform hand hygiene,” or “did not attempt to perform hand hygiene.”. Proper compliance was calculated by dividing the number of hand hygiene opportunities that were performed properly over the total number of hand hygiene opportunities. Observation began in May and ended in July 2019. McNemar's test was used to compare compliance at baseline and post-intervention within each intervention arm, and chi-square was conducted for between-group differences.

Results: During the entire study period, 1,658 hand hygiene opportunities were observed. Overall, 59% of them were attributed to nurses, 39% to physicians, and 2% to the third option “Other/I don’t know.” Proper compliance showed a significant increase in both treatment and control groups. Proper compliance increased in the control group from 10% to 19% at p < .001, and in the treatment group from 3% to 30% at p < .001. The difference between the increase in proper compliance in control and treatment hospitals was also significant at p < .001.

Conclusion: Hand hygiene zones show effectiveness in increasing compliance among ED staff. Future studies can replicate the design using material that is more durable to determine whether it is the effect of the zones or if it is temporary due to the recent change in environment. This cost-effective approach can improve compliance to hand hygiene and help control a pressing public health issue.

Background: Cardiac injury is a widespread cause of morbidity and mortality worldwide. Cardiac troponin is an available and accurate marker that aids in rapid diagnosis and management of cardiac injury. Troponin outperforms other blood tests at discerning cardiac injury, yet other tests are often performed leading to extraneous blood draws, diagnostic confusion, and unnecessary healthcare expenditure. We evaluated creatine kinase (CK), an alternative cardiac biomarker, because it is diagnostically inferior to troponin yet commonly ordered. We examined cardiac testing in a large cardiovascular care center and implemented a behaviorally informed quality improvement initiative to reduce unnecessary testing. 

Methods: We revised institutional protocols to change behavioral defaults for the decision to test and provided individualized peer-comparison feedback at the level of clinical unit and physician subspecialty. Our interventions preserved clinician autonomy with no prohibitions on testing and no financial incentives. 

Results: Our baseline audit of existing practice showed CK testing remained stable in the year prior to intervention. At six months from the start from intervention, CK testing decreased from 3963 to 2111 per month, amounting to a 46.7% reduction (95% CI 33.2 to 60.2; p<0.001) equaling 61 fewer tests per hospital day. Ratio of CK-to-troponin tests decreased from 0.91 to 0.49 (p<0.001). Troponin testing remained stable throughout. The reduction in CK testing extended to all hospital departments, sustained at one-year follow-up, and translated to about $28,000 annualized savings for the institution.

Conclusions: Our intervention was well-received and demonstrated that a low-cost, behaviorally informed intervention may lead to significant reduction in unnecessary CK testing for patients with suspected cardiac injury. This approach is feasible, pragmatic, and may be applied to achieve substantial cost savings for laboratory testing across diverse healthcare situations.

Background: Over the past decade, health systems have begun to recognize applications of behavioral insights as powerful tools to influence decisions. Multiple provider systems have developed internal nudge units to design, deploy, and evaluate the effects of behavioral insight-informed interventions. To our knowledge, no health plan had done so, despite the potential to improve the effectiveness of key programs.

Methods: In May 2020, Blue Cross Blue Shield of Massachusetts (BCBSMA) formed an internal nudge unit, led by its Performance Measurement and Improvement division, and assisted by consultants from the Behavioral Insights Team and academic institutions. Early goals included helping key operational units (e.g., medical management, member and  provider  communications,  contracting and network management) recognize and submit opportunities to apply behavioral insights to improve member and provider decision making, developing behavioral insights-informed versions of submitted interventions (or developing new interventions where none had existed before), and deploying these interventions to maximize their evaluability and value to the health plan and its members.

Results: In its first eight months, the BCBSMA nudge unit deployed 9 interventions targeting member and provider decisions. Of these, eight were randomized to maximize evaluability, and 4 have produced final results. 30 additional interventions are in production. Targeted provider decisions included COVID-19 antibody testing, statin use in diabetes, metabolic testing for children on psychotic medication, and outlier prescribing, test ordering, and billing patterns. Targeted member decisions included colorectal cancer screening via fecal immunohistochemistry test (FIT), survey response decisions (aiming to improve response rates), and COVID-19 vaccine uptake. Among completed randomized interventions, effects of behavioral insights informed (“nudgified”) interventions have ranged from 0.2 to 4.7 percentage points, in the intended directions: -0.2 percentage points for COVID-19 antibody testing (p=0.005), +1.8 percentage points with mailed messages (p=0.11) and +3.9 percentage points (p=0.02) via fax for statin prescribing in diabetes, +4.7 percentage points for metabolic monitoring (p=0.10), and +3.0 percentage points for FIT test uptake (p<.001). An additional non-randomized intervention applied behavioral insights to BCBSMA’s member app interface to encourage members to report their race and ethnicity, to support the company’s efforts to measure and address inequities in care. Post-implementation, the rate of race and ethnicity data submission has increased by approximately 10,000 members per week, with zero complaints to date, as assessed via call center and social media monitoring.

Conclusions: With guidance and modest resource investment, it is feasible to launch a health plan nudge unit that can produce rapid results. Successful behavioral insights-informed interventions can be replicated at scale at low marginal cost. Based on the rapidly expanding nudge unit pipeline at BCBSMA, demand for nudge unit services within health plans appears to be strong.

Background: We first aim to confirm using data whether or not variety can indeed be associated with favorable health outcomes, and if so, whether variety metrics serve as helpful information for recommendation system design. Second, we aim to demonstrate how these health-minded behavioral marketing insights can indeed be used to help guide recommendation system design, and ultimately lead to improvements in system performance. In other words, can (and should) behaviorally motivated criteria pertaining to variety be included in the design of recommendation systems?

Methods: We used large-scale food entry data from a popular mHealth app designed to help users keep track of calories. These data provide us a granular view of eating habits (i.e., what collection of foods they eat each day) for a large number of users over time. Using these data, we explore the implications of consumption variety (i.e., eating a diverse set of foods) on both calorie consumption/composition and recommendation system design. Using these unique data, we first investigate the empirical relationship between variety and health-related outcomes (i.e., calorie consumption/composition). After establishing these empirical results, we then design a novel behaviorally-informed multi-criteria recommendation system for mHealth that can accommodate for variety in recommended foods (FOODVAR).

Results: Our empirical analysis reveals that consumption variety is indeed linked to lower calorie consumption as well as greater consumption of vegetables (as opposed to snacks). Furthermore, we demonstrate that our FOODVAR system outperforms an alternative system that does not make use of variety as a criterion.

Conclusion: The research shows that consumption variety in food recommendation can be motivated by health considerations, and that the incorporation of variety might yield benefits to recommendation system design performance in itself. We used large-scale food entry data from a popular mHealth app designed to help users keep track of calories. These data provide us a granular view of eating habits (i.e., what collection of foods they eat each day) for a large number of users over time. Using these data, we explore the implications of consumption variety (i.e., eating a diverse set of foods) on both calorie consumption/composition and recommendation system design. Using these unique data, we first investigate the empirical relationship between variety and health-related outcomes (i.e., calorie consumption/composition). After establishing these empirical results, we then design a novel behaviorally-informed multi-criteria recommendation system for mHealth that can accommodate for variety in recommended foods (FOODVAR).

Background: Strategies targeting unhealthy beverage consumption have mostly been explicit, involving policy change or regulation (introducing a sugar tax, restricting access or using plain packaging), which have had limited success. Instead, the present study investigated whether implicit interventions based on nudging principles could promote healthier beverage choices from vending machines.

Methods: Two experiments manipulated the artwork on a vending machine display. Study 1 (n = 144) and Study 2 (n = 235) compared the effect of seven artwork displays (beverage branded, red, blue or black coloured, or featuring a glass of water or coke) on beverage choice. Participants also indicated how much they liked and how often they the beverages (Study 1), or rated the taste, healthiness, energy and refreshing value of each beverage (Study 2).

Findings: The black vending machine significantly influenced caffeine-based choices in Study 2. Other significant predictors of beverage choice were how much participants liked the beverage and how often they consumed it (Study 1), as well as their perception of the health and taste of the beverage (Study 2).

Discussion: Colour and artwork nudges have been shown to be effective in previous food research. Study 2 showed that colour may also influence beverage choices. The colour of the black machine may have implicitly led participants to think of coffee, causing them to choose caffeine-based beverages. These findings suggest that subtle changes such as the colour of the vending machine can influence beverage choices, and thus is an important factor to consider in promoting healthy choices.

Background: The rural Transitions Nurse Program (TNP) is a care coordination intervention for high-risk veterans. An interactive dashboard was used to provide real-time performance metrics to sites as an audit and feedback tool to support changes in processes and behavior as well as increase veteran enrollment numbers. Despite this, one year post-implementation, enrollment goals consistently went unmet. Control theory suggests that feedback using diverse methods can positively influence performance. Nudge emails can draw attention to performance metrics to improve awareness of current state. This study evaluated whether Veteran enrollments and site communication increased when feedback occurred through a dashboard plus weekly nudge email versus dashboard alone. 

Settings/Population: This observational study reviewed enrollment counts of urban, rural, and highly rural Veterans who were hospitalized and discharged from four VA hospitals participating in TNP. Transitions nurses and site champions implementing TNP at each site were sent the nudge intervention and surveyed. 

Methods: Veteran enrollment counts between the dashboard phase and dashboard plus weekly nudge email phase were compared. Prior to the nudge intervention, sites were required to report enrollment data through a VA-hosted database, with data reports available for sites to retrieve and review within secure folders. The nudge intervention was designed according to evidence-based best practices and included salience and default/positioning effects. Rather than having to retrieve their data, nudge emails delivered data weekly to transition nurses and site champions via bright and colorful run charts to grab recipients’ attention. The difference of means for weekly enrollment between the two phases were calculated using Poisson distribution. After 3 months of nudge emails, a survey assessing TNP transitions nurse and physician champion perceptions of the nudge emails was distributed. 

Results: Our sample included four VA medical centers with one transition nurse and site champion at each site. The average enrollment for the four TNP sites during the ~20-month dashboard only phase was 4.23 veterans/week. The average during the 3-month dashboard plus nudge email phase was 4.21 veterans/week. The difference in means was -0.03 (p=0.73). Adjusting for time trends had no further effect. Four nurses responded to the survey. Two reported neutral and two reported positive perceptions of the nudge emails. No site champions responded to the survey. 

Conclusion: Drawing attention to metrics, through nudge emails, maintained, but did not increase TNP veteran discharges compared to dashboard feedback alone. While we may have increased awareness to enrollment goals, enrollments continued lower than program expectations. Given the challenges of influencing behavior, we recommend frequent, rapid testing of evidence-based audit and feedback methods within programs. Inclusion of qualitative approaches should be considered to better understand the underlying mechanisms that may be driving performance.

Abstract: Previous attempts to promote healthy eating using explicit techniques have not been consistently successful. We therefore investigated an implicit strategy (item placement techniques) based on the principles of nudging to encourage healthy food choices in the context of snack menus. Two experimental studies compared presentation of healthy items in the top, middle, and bottom sections of a snack menu. Study 1 compared these presentations in a physical menu, while Study 2 used an online menu. Menus consisted of eight unhealthy and four healthy items, arranged in three rows of four in Study 1, and one column of 12 in Study 2. In each study, participants selected one food item from one of the three experimental menus, before completing the Revised Restraint Scale (to determine dietary restraint status). In Study 1 (n = 172), item placement condition did not predict healthiness of food choice. In Study 2 (n = 182), healthy items were most popular from the first section of the menu, in comparison to the middle or last sections. The manipulation may have been more effective in Study 2 because the top section of the menu was viewed first, the middle section second, and the bottom section last in Study 2, whereas all items were viewed at once in Study 1. Dietary restraint did not moderate the effect of item placement condition on food choice. In line with nudging principles, results suggest that item placement techniques could be a potentially powerful tool in promoting healthy choices from online snack menus. Added up over time on a population level, consistent healthier choices could result in general health benefits such as better nutrient intake and physical wellbeing.

Background: Worldwide, over two billion people are infected with hepatitis B virus (HBV). HBV is commonly spread through infected blood or bodily fluid. Childbirth is an important route of transmission because the earlier the disease is contracted, the more likely the disease will progress, placing newborns at risk. The hepatitis B (HepB) vaccine is the most effective method in reducing and eradicating the spread of HBV. The CDC recommends administration of the newborn HepB vaccine within the first 24 hours of life. In a large teaching hospital located in the Northeast, the yearly newborn HepB vaccine rate within 24 hours of birth was about 30% and 50% before discharge. The newborn HepB vaccine rate is well below the Healthy People 2020 goal of 85%. This quality improvement project aimed to improve the newborn HepB vaccine administration rate within 24 hours of birth by 10% in eligible newborns in the well-baby nursery by implementing an electronic medical record embedded best-practice alert.

Methods: A quality improvement project using a pre/post design was conducted. An electronic medical record best practice alert was implemented using the Plan-Do-Study-Act model.  All newborns admitted to the well-baby nursery were included. A total of 120 newborns' electronic medical records were randomly selected to extract the HepB vaccine administration status within 24 hours of birth and before hospital discharge, the newborn average length of stay, and nurse vaccine activity documentation. Four weeks of post-intervention data were analyzed and compared to four weeks of pre-intervention data.

Results: Implementation of a best practice alert resulted in a 17% increase in the HepB vaccination rate within 24 hours of birth, compared to pre-intervention (45% vs. 28%, p = 0.058). The vaccine rate before discharge increased by 5% compared to pre-intervention (50% vs. 55%, p = 0.583). Finally, vaccination activity documentation in the electronic medication administration record increased by 12% compared to pre-intervention vaccination activity documentation (88% vs. 100%, p = 0.013). The length of stay was similar among newborns whose nurses did not receive the best practice alert (M = 2.2 days) compared to newborns whose nurses did receive the best practice alert (M = 2 days), indicating increased vaccination counseling did not delay discharge (p = 0.487).

Conclusion: A newborn HepB vaccine best practice alert embedded in the electronic medical record has improved compliance with the CDC guidelines in administering the vaccine to newborns admitted to the well-baby nursery within 24 hours of birth. The best practice alert also increased vaccine activity documentation among Obstetrical nurses. Although not statistically significant, our goal of improving newborn HepB vaccine administration within 24 hours of life was achieved. An electronic medical record embedded best-practice alert may be effective for other vaccine administration reminders and help integrate health promotion activities in acute care settings.

Abstract: The omnipresence of unhealthy food cues in our environment is a major contributor to poor dietary behaviours. Emerging research has shown that changing the food environment through the co-presentation of a healthy food cue may subtly ‘nudge’ individuals towards making healthier choices. Accordingly, the present study aimed to investigate the effect of priming with a healthy food cue on subsequent food and drink choices from an online fast-food ordering menu. An analogue of the online ordering menus used by fast-food retailers was created. Participants (N = 291 women) were first presented with a prime displaying either a healthy or unhealthy meal, or no prime (control). They were then shown a pictorial menu with items presented in one of two orders (menu 1 or 2) and asked to choose one item from each of three sections (mains, drinks, desserts). Participants also completed a questionnaire measure of dietary restraint. Overall, participants made more healthy choices from menu 1 than 2. For menu 1, there was a significant interaction between prime condition and restraint status, whereby restrained eaters made relatively more healthy choices following the healthy prime. This was particularly the case for main meal items. The findings have real-world implications for digital businesses on how to present food and drink items to nudge individuals towards healthier choices.

Background: Gamified interventions might have the potential to promote regular physical activity. To this end, numerous smartphone app-delivered interventions have implemented gamified affordance, with promising findings. However, to date, there are no systematic reviews and meta-analyses that investigate the effects of standalone gamified smartphone apps for physical activity. This study aims to 1) identify gamified affordances implemented in standalone gamified smartphone apps by previous researchers and 2) examine the effects of gamified apps on promoting physical activity.

Methods: The systematic review and meta-analysis follow the PRISMA guidelines. Web of Science, Scopus, PubMed, PsycINFO, and ACM Digital Library were searched for publications from January 2008 to December 2020. Based on the predetermined PICOS-criteria, randomized controlled trials (RCT) or single-armed pre-to-post interventions, delivered by standalone gamified smartphone apps and targeting physical activity, were eligible for inclusion. Gamified affordances implemented in the smartphone apps were summarized. Study-specific results were analyzed using random effects meta-analysis, with standardized mean difference (SMD). Meta-regressions, subgroup analyses, and sensitivity analyses were performed. The review protocol is registered with the PROSPERO (CRD42020209502).

Results: 18 studies were eligible and 15 studies were included in the meta-analyses. In-game rewards, leaderboards, virtual teams, social networking, and virtual challenges were the top five most frequently used gamified affordances (overall frequency from 1 to 13, median of 6). In most studies, multiple affordances were adopted (from 1 to 10, median of 4.5). Gamified apps had a small-to-moderate effect on physical activity in both between-group RCTs (n = 11 apps, SMD = .37, 95% CI [.06, .69], I2 = 74%, p < .01; GRADE: Moderate) and within-group pre-to-post interventions (n = 17 apps, SMD = .40, 95% CI [.17, .63], I2 = 75%, p < .01; GRADE: Very low). In between group meta-regressions and subgroup analyses, the effects were moderated by the duration of the intervention (n = 11, SMD = .10, p = .009) and among active-control groups (vs. waitlist, n = 8, SMD = .60, p = .03), as well as gender and population. Within group analyses identified a modifying effect of duration, as well as a larger effect on step counts (n = 7 apps, SMD = .80), compared to moderate-to-vigorous physical activity (n = 10 apps, SMD = .20, p = .03). Differences in effects between three types of apps (immersion vs. social vs. achievement/progression) were not significant for within- but significant for between-group studies. Leave-one-out sensitivity analyses sustained the main effects with lower heterogeneity (I2 of 37.7% and 50.4%, respectively).

Conclusion: Using gamified smartphone apps as standalone interventions can increase physical activity. Future research may investigate the effects of gamified apps on physical activity by isolating the effects of specific types of affordances to rule out potential confounding influence by multiple affordances within one investigation.

Abstract: We draw from longitudinal data to exploit the effect of two exogenous shocks in pro-social behaviours in the Netherlands, namely a tax reform introducing tax breaks for donations to culture and arts, and a humanitarian aid campaign for the Syrian refugee crisis and typhoon in the Philippines set up in May 2013. We find effects consistent with the hypothesis of an ‘unbounded altruism budget’ hypothesis. The culture-related tax breaks lead to an increase in monetary donations culture and arts in 60€, but importantly and consistently with an altruism budget, we find an increase in donations to health in 23€ but a 3 percental point decrease in the probability of blood donation. Similarly, the unanticipated effects of the Giro555 campaign led to a reduction in volunteering time in 4.5 hours per month. Such positive spillovers are driven by increased trust in philanthropic organizations and are consistent with an unbounded altruism budget.

Background: The Get Waivered website (GW) was originally launched to assist clinicians with obtaining their DEA-X waiver with the use of traditional, in-person courses. On April 23, 2020, GW implemented a digital campaign for its remote course for physicians and healthcare providers via three channels: direct (emails), Social (Instagram and Twitter campaigns), and Organic (Google search). “Web analytics'' involves the analysis of qualitative and quantitative data from websites, which can be used to guide improvements in the user experience to better achieve desired outcomes. Many educational conferences, previously delivered in-person, have moved to utilize remote formats, but nationwide training sessions are not always filled. This paper provides guidance on improving attendance and engagement in virtual training courses. On May 20, 2020, a total of 814 providers completed the Get Waivered Remote Course. 

Objectives: To evaluate healthcare users obtained from three marketing channels implemented for the GW site: direct, social, and organic along with their engagement on the course’s website. Methods: We utilized Google Analytics (Commercial®) to evaluate sources of traffic to the GW website including Direct (visitors who arrive directly, including by clicking a link in an email), Social (e.g. Facebook, Twitter, Instagram), Organic (arrive via a search results page), and Referral traffic (arrive via another site). Engagement metrics included the number of pages per session and the time spent on each page. 

Results: 70% of traffic to the site was Direct, 13% was Social; 13% was Organic; and 4% was Referral traffic. Users who arrived by Organic and Referral sources visited 2.3 and 2.8 pages per session, and spent 2:17 and 4:37 minutes per session, on average. Users from Direct and Social sources visited 1.6 and 1.6 pages per session and spent 1:16 and 0:52 minutes per session on average. 

Conclusion: Although top sources of traffic were Direct and Social, which would encompass GW’s email and Instagram/Twitter campaigns, respectively, higher user engagement came from Organic and Referral channel traffic. Understanding these analytics will help guide us to reach more clinicians who will attend the courses in the future. Moreover, improvements in user attendance and engagement in nationwide virtual training courses may result by creating more opportunities from Organic and Referral traffic for physicians and other healthcare providers to find and use the website.

Background: In response to COVID-19, governments around the world have imposed quarantine and isolation, which have been associated with mental and emotional distress because of limited social interaction. Therefore, it is essential to monitor individuals in such conditions and provide continuous psychological support. The aim of this study was to explore the effect of providing virtual psychoeducational group therapy on confined individuals’ mental wellbeing.

Methods: The Behavioral Insights Unit in the Ministry of Health in Saudi Arabia (MOH) facilitated virtual psychoeducation group therapy sessions for individuals in quarantine or isolation due to COVID-19. A text message was sent to all patients or quarantined individuals with a link inviting them to register for the therapy sessions. Registered individuals (N = 20) received schedules and instructions to access the session via Microsoft Teams. Mental health counselors (N = 8) were recruited via MOH registry to voluntarily facilitate each session. The sessions were sequential in content and used materials designed to teach and train participants healthy coping techniques.

• Duration of each session: 2 hours
• Number of sessions: 3-4 sessions per group
• Number of participants: 4-6 individuals per group lead by two therapists

A qualitative content analysis was conducted with all eight therapists post-intervention to explore the perceived effectiveness of the group sessions.

Results: Four themes were discovered: “novelty of virtual psychoeducation group therapy”, “design and materials for the intervention”, “challenges”, and “groups engagement”. These themes provided insights on the impact of the sessions and the improvement opportunities that exist. For the “novelty” theme, therapists reported no participants knew what they registered for and what psychoeducation group therapy meant. “Many participants said this therapy has positively changed my perception about therapy”, a therapist relayed. Regarding the “design and materials” theme, all therapists agreed that the design of the intervention program and materials were clear and easy to implement. For example, while one therapist commended, “I liked how clear the design was but at the same time there was room for me to add”, some suggested adding more sessions and focusing on stress management. According to therapists, one of the biggest “challenges” was commitment to attending all sessions; many participants registered and did not attend any sessions or attend only one session. One of the groups had to cancel a session because only one participant attended. Other challenges included managing the group to ensure everyone’s participation, the length of each session, and technical difficulties. The groups that consisted of male patients were more interactive whereas it took more efforts from therapists to engage female attendees. One therapist commented, “I noticed men in the group got comfortable with each other immediately”.

Conclusions: Findings of this study suggest that therapists agreed on the positive psychological effect of virtual psychoeducation group therapy on those quarantined or isolated due to COVID-19. However, due to the novelty of this type of therapy, there exist some challenges that could improve the dynamic of participants once addressed.

Abstract: Antimicrobial resistance (AMR) is a major public health threat that limits treatment of infectious diseases. AMR-related mortality is predicted to surpass cancer-related mortality by 2050, with up to 10 million lives lost annually. A global call for a multifaceted approach to combat AMR includes reducing inappropriate use, as antibiotics are commonly prescribed, but up to one-half are prescribed inappropriately. This is problematic because antibiotic use is the main driver of antibiotic resistance, rendering antibiotics less effective over time. Modification of the microbiology culture and susceptibility results reports, which clinicians review prior to prescribing an antibiotic is a strategy to influence and encourage appropriate antibiotic use. We performed a recent scoping review of nudges on microbiology reports, which is a rapidly emerging area of interest. Opportunities to further this work include widening the breadth of nudging interventions evaluated and incorporating them into prospective study designs. Our objective was to develop and apply a framework to systematically identify and classify nudging strategies targeting antimicrobial prescribing behaviour.  

Methods: We assembled a multi-disciplinary team of experts in infectious diseases, medical microbiology, antimicrobial stewardship, medical decision-making, and epidemiology to identify infectious and non-infectious scenarios in which nudging in microbiology reports could be applied. For comparison, we analyzed current baseline components of microbiology reports including report style, default comments, and antibiotic susceptibility results reported. Nudging strategies were categorized as:

• Selective reporting: Change the default reporting to only displaying appropriate antibiotics for a specific scenario, rather than displaying all antibiotics tested (current practice)
• Framing: Adding or adjusting report comments to guide decision-making
• Eye level: Making desirable prescribing options more salient by moving them to the top of the list instead of alphabetical order (current practice)

Different potential objectives of nudging were also categorized: 

• Start appropriate antibiotics
• Stop inappropriate antibiotics
• Escalate (e.g., using broader spectrum, greater number of antibiotics)
• De-escalate (e.g., using narrower spectrum, lesser number of antibiotics)

Results: We identified 36 nudging strategies for microbiology report nudges in 15 case scenarios. Selective reporting nudges typically reduced the default reporting of antibiotics from a median of six visible options to a median of 2 visible options. Report comments were created to guide appropriate prescribing by providing additional context (recommending preferred therapy was most common in 10 scenarios, followed by comments to help differentiate true infection from contamination or colonization, the latter of which don’t usually require treatment, in five scenarios). An eye-level nudging strategy could be applied in 10 of 15 scenarios and changed the order of reporting from alphabetical in 9 of 10 scenarios. 

Conclusion: We present a team-based approach to designing nudging interventions in a complex healthcare environment. This approach systematically identifies categories of nudging strategies in a microbiology laboratory setting that have broad applicability to many patient scenarios. Future research will evaluate the impact of these strategies in a vignette-based randomized controlled trial targeting antibiotic prescribers and ultimately prospective in-practice trials. Our approach may help to form the basis for designing and evaluating future healthcare nudging interventions.

Background: While the rest of the world is tackling the COVID-19 pandemic, the U.S is also tackling another public health crisis: the opioid epidemic. According to the Center for Disease Control and Prevention (CDC), approximately 450,000 people died from opioid overdoses between 1999-2018 1). Most often, the data that is available to the states are outdated and prevent public health officials from creating effective policies to tackle this epidemic. Impact-oriented modeling helps guide health care officials, policymakers, and the general public to make informed behavioral health decisions, similar to how the public has responded to public dashboards for COVID incidence rates or weather reports. Such models require criteria like accessibility, ease of use, adaptability, transparency, agility, responsiveness for informed behavioral health decisions, such as carrying Narcan kits at specific incidences of opioid overdoses in regional locations 2). The Autoregressive Integrated Moving Average (ARIMA) model is a digital predictive tool that creates an impact by predicting surges in opioid overdose deaths in particular geographical communities based on past data trends 3). This can act as behavioral nudges to help public health officials and the general population be better prepared to effectively respond to the crisis.

Methods/Results: The Autoregressive Integrated Moving Average (ARIMA) model is a forecasting algorithm based on information in the past values of the time series, a sequence recorded over regular time intervals (3). As a linear regression model, it uses its own lags as predictors of future values (3). In order to forecast rates of drug overdose deaths, the ARIMA model was fitted to the CDC dataset from January 2015 to July 2020. In addition, total population data and time periods for all 50 states was collected from the U.S Census Bureau. Based on the available data, the forecasted values in a recent 12-month period showed a 24.2% increase in overdose deaths from 69,266 projected deaths in July 2019 to 86,001 projected deaths in July 2020 (4). However, the actual data shows an increase in opioid overdose deaths from July 2019 with 68,023 projected deaths to July 2020 with 83,544 actual deaths, an overall increase of 22.8% (4).

Conclusion: Though a difference between the predicted and reported values exists, the accuracy of the ARIMA model to forecast overdose deaths will become evident as additional 2020 data becomes available. The ARIMA model can be an essential tool to address this public health emergency of opioid use disorder.

Background: High no-show rates often waste valuable resources in the healthcare system. Preliminary field visits to Primary Healthcare Centers (PHCs) confirmed that a considerable proportion of the no-shows’ appointments were false due to staff registering patient arrival inappropriately. The objective was to investigate whether visual cue reminders will prime staff to register patients’ arrival properly.

Methods: In 2019, an 11-week cluster randomized controlled trial was conducted on 35 PHCs in Riyadh, Saudi Arabia, and were randomly allocated into treatment (n=18) and control (n=17) groups. Visual cues were installed wherever patients’ arrivals were registered. The primary outcome was patient no-show appointment as reflected in the national appointment booking system, Mawid. A multilevel logistic regression estimated the odds ratio (OR) of no-shows in the treatment compared to control group. Analysis was also conducted by types of clinics and seasonality of Ramadan, where Muslims fast and public activities are usually disrupted. 

Results: The total appointments booked were 152,247. Treatment had lower odds of no-shows than control group, although not significant (OR=0.81; 95%CI=0.50-1.31). Ramadan month had higher no-shows than pre-Ramadan (OR= 1.60; 95%CI=1.55-1.66). Compared to family medicine clinics, smoking cessation appointments had the highest no-show rate (OR=3.95; 95%CI=3.43-4.54), followed by dental appointments (OR=2.14; 95%CI=1.97-2.32). Dressing and pharmacy appointments had lower odds of no-shows than family medicine appointments. 

Conclusions: The effect of the visual cue was promising in reducing the no-show, presuming this effect was the false-no shows appointments. The discrepancies between false and true no-shows can be addressed using patient to self-check-in at PHCs, which would provide an accurate assessment that can help with resources and planning.

Keywords: No-show, visual cue, Nudge, PHC, Riyadh, Saudi Arabia

Background: The COVID-19 pandemic has stressed health systems because its novelty, speed, and magnitude forced critical operational and clinical decisions despite incomplete and shifting information, insufficient resources, and tension between patient care and clinician safety. For example, every system has had to contend with the use of personal protective equipment (PPE), treatment of patients with COVID-19, and a strategy for re-opening services. Historically, safety net systems, such as the Los Angeles County Department of Health Services (LACDHS), would be considered amongst the most challenged by situations fraught with such ambiguity, risk, and conflict because of limited resources and governmental oversight. 

Intervention: LACDHS has managed the pandemic’s challenges in part through its adaptation of and reliance on Expected Practices (EPs) as a nimble, multidisciplinary decision-making strategy. EPs were originally created as a clinical decision support tool crafted based on existing evidence, real-world practice conditions, available resources, and the principle of patient equity (1).  EPs represent neither policies nor guidelines but nudge providers to follow a standard approach except when there is a compelling justification to deviate. EPs are living documents, updated as evidence and resources change, making them ideal for the fluxes experienced during the COVID-19 pandemic. The core principles of creating an EP have remained the same, a dependence on data-driven practice, consensus decision making, feasibility of implementation, cognizance of resource limitations, and in this context, equity for patients as well as for clinical and non-clinical workforce. EPs have been an essential tool for LACDHS during the pandemic and used in three new domains: 1) decision support beyond clinical care, 2) navigation of treatment controversies, and 3) rapid, flexible re-opening strategies. As the intended audience for the EPs expanded during the pandemic, a new strategy for dissemination was also successfully deployed with hundreds of downloads daily of the COVID EPs from the internal SharePoint by a broad range of employees including environment services, clinical and administrative workforce. 

Conclusion: While originally envisioned as a clinical decision support tool, the EP framework was transformed into a powerful instrument that has facilitated a successful response to this great health crisis. By relying on the core principles that underlie the creation of an EP, LACDHS was able to tackle a plethora of questions that arose during this, hopefully, once in a lifetime global pandemic including human resource and supply chain struggles, controversial treatment protocols, and elastic reopening strategies. We anticipate that uses for the EP will continue to evolve and meet other novel system needs.

Background: Many conditions related to physical and cognitive diseases can be detected, prevented, and treated at an early age. Importantly, late detection affects not only children’s health but also their academic achievements, behaviors, and development. Annually, the Saudi Ministry of Health (MOH) runs a national school-based screening program for 1st and 4th graders. The screening includes physical, dental, and mental health check-ups. However, due to COVID-19 pandemic and school closures, screening now is only offered at primary health care centers (PHC), for which parents must book an appointment and accompany their children. Such a process is not mandatory and there is low adoption. The objective of this study is to examine the effect of SMS reminders sent to parents to increase the rate of appointments booked.

Methods: Three consecutive SMS have been created using insights from behavioral economics (e.g., social norms and messenger effects) to nudge parents to book an appointment for their child at student screening clinics. A public elementary school in Riyadh has been selected as it has its own communication platform with families. Parents of all children enrolled in first and fourth grades (N = 117) were sent three unique messages with one-week interval in February 2021. All massages concluded with a link that directed parents to the national health appointment system, Sehhaty, where they could schedule an appointment at healthcare centers near them. The primary outcome of this study is the number of participating parents who booked an appointment for their children in the students’ health screening clinics. This outcome was extracted from the national health appointment system, Sehhaty, by checking the national ID of each student. National IDs were cross-referenced in Sehhaty to determine the counts of parents who already had booked an appointment at baseline. McNemar test was used to check the statistical significance in the count of booked appointments before and after the intervention.

Results: Of the 117 participants, 56.4% had already booked an appointment at baseline, and this percentage increased to 65.8% after intervention. There is a statistically significant percent increase by 9.4% (p < .001) in the number of participating parents who booked an appointment after receiving the three SMS reminders.

Conclusion: SMS reminders hold a promising effect in increasing parents’ compliance to school screening visits. Given the low cost of such an intervention, it is recommended that school administrators and state officials take an active approach to nudge citizens beyond traditional means (e.g., awareness campaigns and memoranda) to close any intention-to-action gaps. Future studies might investigate demographics of those who are more “nudgeable”, using a larger national sample, an ongoing endeavor by the current research team.

Background: Using Advanced Care Planning (ACP) to help patients make decisions about their future care has proven benefits for patients, caregivers, and health systems, yet these conversations often occur too late, if at all. To improve the quality of ACP for patients system-wide, a large health system adopted the Serious Illness Care Program. It provides a standardized method to discuss and document patients’ values and goals related to end-of-life care. Since launching the program training in 2014, an affiliated community hospital has successfully trained over 95% of its primary care providers (PCPs). Despite the rigor of the program and training, the percentage of providers who document serious illness conversations (SICs) remains low. This study utilizes a behavioral science approach to uncover barriers to initiating and documenting SICs in primary care, and to test innovative low-cost solutions. 

Methods: We used a two-phase mixed-methods research approach to examine the behavioral barriers among PCPs to initiating and documenting SICs. The first phase consisted of a literature review, quantitative analysis of administrative data, two observations of virtual training, and a behavioral audit of the existing SIC processes and materials. Insights from these activities informed a behavioral map of the decisions and actions that PCPs must complete to document an SIC. In the second phase, we used the map to uncover potential barriers and evaluated our hypotheses via 12 semi-structured interviews with primary care providers (eight physicians, two social workers, one registered nurse, and one nurse practitioner). Finally, we conducted a series of individual and group design activities to build an intervention that we will pilot starting Spring 2021. Four Primary Care Practices will receive the treatment intervention and four matched comparison practices will follow business as usual. 

Results: We uncovered six main behavioral barriers to conducting and documenting SICs among PCPs: time scarcity discourages providers from prioritizing early SICs, uncertainty around prognosis and conversation timing discourages initiation, discomfort with the new process and anxiety around SICs prompts avoidance, diffusion of responsibility between specialists and PCPs exacerbates inaction, established habits challenge the adoption of the new process, and documentation hassles in the electronic health record hinder follow-through. We utilized the identified barriers to design a behavioral intervention with three main components: 1) care team notifications sent via EPIC for patients with upcoming PCP visits that would be appropriate for SICs, 2) a pre-chart note for the patient visit, and 3) a simplified documentation template accessible through a SmartPhrase. The notifications are sent from a clinician two days before the patient’s visit. They prompt the PCP to ask the patient permission to schedule a separate SIC follow-up, provide support materials, and outline clear next steps. The pre-chart note serves as a timely reminder. It includes a script to ask permission and an active choice for scheduling an SIC follow-up. Lastly, the documentation template contains a hyperlink to easily record notes in the appropriate flow sheet, as well as the relevant billing codes and key information to include. The results of the evaluation are forthcoming.

Conclusions: Applying a behavioral science approach can reveal novel insights about persistent barriers to documenting SICs in primary care, identify ways to optimize the existing SIC program, and uncover new low-cost opportunities to assist PCPs with these important conversations. 

Background: Health care associated infections (HAI) are estimated to affect up to 15% of hospital inpatients in low-income countries (LICs). A critical but often neglected aspect of HAI prevention is basic environmental hygiene, particularly surface cleaning and linen management. TEACH CLEAN is a multimodal intervention aimed at improving environmental hygiene which was hypothesized to work mainly via the following behaviour change strategies: monitoring behaviour, mobilising managers, providing knowledge, increasing instrumental and normative beliefs. We evaluated the effectiveness of this intervention in a pilot study in three high-volume maternity and newborn units in Dar es Salaam, Tanzania. 

Methods: This study design prospectively evaluated the intervention as a whole (monitoring behaviour, mobilising managers), and offered a before-and-after comparison of the impact of the main training (knowledge and beliefs). We measured changes in microbiological cleanliness (Aerobic Colony Counts (ACC) and presence of S. aureus) using dipslides, and physical cleaning action using gel dots. These were analysed with descriptive statistics and logistic regression models. We used qualitative (focus group discussions, in-depth interviews, and semi-structured observation) and quantitative (observation checklist) tools to measure why and how the intervention worked. We describe these findings across the themes of adaptation, fidelity, dose, reach, and context. 

Results: Microbiological cleanliness improved during the study period (ACC pre-training: 19%; post-training: 41%). The odds of cleanliness increased on average by 1.33 weekly during the pre-training period (CI=1.11-1.60), and by 1.08 (CI=1.03-1.13) during the post-training period. Cleaning action improved only in the pre-training period. Detection of S.aureus on hospital surfaces did not change substantially. Knowledge improved over three indicators. The intervention was well received and considered feasible in this context. The major pitfalls in the implementation were the limited number of training sessions at the hospital level and the lack of supportive supervision. A systems barrier to implementation was lack of regular cleaning supplies. 

Conclusions: The evaluation suggests that improvements in microbiological cleanliness are possible using this intervention and can be sustained. Pre-training improvements were most likely due to a combination of early intervention activities including hospital management engagement, as well as the intense evaluation schedule. The discrepancy between cleaning action and microbiological cleanliness in the post-training period suggests that the training itself improve cleaning quality (through improve knowledge) which a key step on the pathway to infection prevention in hospitals. Future research should assess whether this bundle is cost-effective in reducing bacterial and viral transmission and infection using a rigorous study design.

Abstract: Current statistics on organ donation point to an ever-increasing demand yet inadequate supply of available donors. For example, in the United States, there are over 113,000 individuals currently on the transplant waiting list, and 22 people die each day waiting. Concerningly, the gap between those needing and those receiving transplants continues to widen. One way to address the ever-growing demand is to increase the number of individuals registered. Yet while the vast majority support organ donation, many do not act and register.  Low registration rates are especially common in countries with explicit consent policies—individuals must opt in to become donors—compared to countries with presumed consent policies—where individuals are registered by default but can opt-out. Although some suggest changing the default may be a promising intervention, the realized impact on actual donations has been mixed, in part due to uncertainties about the deceased person’s preferences. Furthermore, changing policies involves implementation challenges and ethical considerations surrounding informed consent. To date, most countries maintain their existing policies, thus prompting the question, what can be done within explicit consent systems to improve registrations? Prior research provides us with a good understanding of predictors of organ donation attitudes and intentions, yet little is known about how to increase actual registrations.  To address these limitations, we conducted a field experiment in the Province of Ontario (N = 3,330) to test behavioral interventions, targeting information and altruistic motives, to increase new organ donor registrations in a prompted choice context. We supported our interventions with improvements to streamline the registration process (i.e., additional time to review the materials, intercepting customers at the time of decision, and a simplified form). Our paper contributes to the limited evidence for low-cost and scalable solutions to increase organ donor registrations within the current explicit consent systems. Our field experiment demonstrates how intercepting customers with promotional materials at the right time (an information brochure and perspective-taking prompts), along with process improvements, can increase registrations. Specifically, we find that our best performing condition, prompting perspective-taking through reciprocal altruism (“If you needed a transplant would you have one? If so, please help save lives and register today.”) significantly increased actual registration rates from 4.1% in the control condition to 7.4%, an 80% increase. We were able to do so without imposing on the freedom of individuals, raising ethical concerns (i.e., changing defaults), or passing new legislation. To illustrate the potential impact of our findings, if everything held constant over time and we introduced our best performing intervention (reciprocal altruism) Ontario-wide, we could expect roughly 225,000 additional new registrations annually. Given that one donor can save up to eight lives, and enhance 75 others, such an increase could make a meaningful impact on the lives of many. By leveraging behavioral science to design our interventions, we contribute to the understating of how to reduce the intention-action gap in the context of organ donation, improve public policy, and enhance societal welfare. 

For more information, access the full paper.

Abstract: Nudging techniques such as visual cues and primes can be used to subtly encourage healthier consumption. Two experiments tested the effects of four cues/primes on choices from a vending machine display. Participants (17-25 years) were randomly assigned to view a general health cue (image of a person running along a beach), a water prime or soft drink prime (image of water or soft drink poured into a glass), or a control cue before selecting an item from the vending machine. In Experiment 1 (n = 138) the machine included only beverages; in Experiment 2 (n = 643) it also included snacks. The experimental manipulation did not predict choice in Experiment 1 but did in Experiment 2. Specifically, the general health cue nudged people away from healthier beverages compared to all other conditions, while the water prime nudged participants towards healthier beverages compared to the general health and control conditions. In the soft drink prime condition, compared to the general health cue, participants who made a healthier choice were more likely to select a beverage. In Experiment 2 females were more likely to select a healthier food over any beverage compared to males. In both experiments, overall liking and habitual consumption of the chosen items also predicted choice, as did ratings of hunger and thirst in Experiment 2. 

These findings demonstrate that visual cues and primes can be effective for nudging choices from a combined food and beverage vending machine. This research has the potential to inform strategies for encouraging healthier consumption in a vending machine environment.

Abstract: Due to increasing global rates of obesity, there is a demand for strategies and nudges to encourage healthier eating. One such method is menu labelling, with past research and policy focused largely on calorie content and labelling. This study aimed to examine the impact of a simpler colour coded labelling system, and its impact on food and drinks ordered from a café-style menu. The colours red and green were chosen due to the implicit association of red with ‘stop/danger’ and green as the counterpoint to red. A mock menu was developed with food and drink options (mains, desserts, drinks) evenly split into healthy ‘green’ and unhealthy ‘red’ categories. Green items were lower in fat, salt or sugar, compared to ‘red’ ones. Participants (n=508) were randomly assigned to 1 of 4 menu conditions (control, red and green, red only, green only). The control condition presented all items with no labels and a neutral café logo placeholder. The ‘red and green’ condition provided either a small red or green circle next to the image of the food/drink and a legend explaining the labels. In the ‘red only’ condition only the unhealthy items were labelled with a red circle, and in the ‘green only’ condition only the healthy items were labelled with a green circle. Participants completed an online survey where they were asked to imagine themselves in a café setting before making an imagined choice of a main, dessert, and drink from the randomized menu. 

Preliminary analyses using one-way ANOVAs showed a significant effect of menu condition for the total number of red choices F (3, 504) = 3.20, p = .02. Post-hoc tests indicated that participants in the ‘red and green’ menu condition were less likely to select unhealthy (i.e., red) items (M= 1.27, SD= .89) compared to the control condition (M= 1.59, SD=.93, p =.04). Although not statistically significant, participants in the other ‘red only’ (M= 1.30, SD =.88) and ‘green only’ (M=1.34, SD =.98) menu conditions also made fewer red choices compared to the control condition.

The tentative results of the study aim to contribute to the current knowledge about nudging and menu labelling and explore the feasibility of encouraging healthier food and drink choices with nudges in the population. At present the results suggest that providing an overall indicator of health as a colour coded circle may be effective in nudging healthier choices. Future research should examine the application of labelling across different contexts (e.g., vending machines, online ordering) or the use of other nudges combined with labelling (e.g., adjusting proportions). Other options to explore could include the use of backgrounds or borders around images or descriptions of food.

Background: Hospital inpatients often undergo frequent pathology tests, especially blood tests, which are not always necessary or useful. Unnecessary tests are associated with unfavourable outcomes including iatrogenic anemia (1,2) and reduced patient satisfaction and may lead to overdiagnosis and overtreatment. Additionally, excessive testing is a potential source of healthcare cost savings. The task of ordering blood tests is often assigned to junior medical officers (JMOs), who may order more tests as a precaution since this brings no immediate repercussions. The Developing Rational Approaches to pathology ordering: Curbing Unnecessary Laboratory Analyses (DRACULA) project uses individualised feedback reports and department-specific guidelines to promote rational pathology test ordering by JMOs at a teaching hospital in Sydney, Australia.

Methods: Phase one of the DRACULA project comprised of individualised feedback reports sent to JMOs twice per clinical rotation. These reports provided anonymous feedback on their ordering practices. Reports included the number of tests ordered in comparison to the average of all JMOs over the period and the average of JMOs on the same rotation over the past year. They also provided specific data on ordering practices for common blood tests. The reports were treated confidentially and there were no consequences for ordering a large number of tests. They were intended to inform JMOs how their ordering practices compared to their cohort to encourage self-reflection and discussion around appropriate pathology ordering. The second phase of the project consisted of department-specific pathology ordering guidelines developed by senior clinicians. These included information about appropriate tests in commonly encountered clinical situations and the usual frequency of testing required. To evaluate the impact of these two initiatives, data from June to November 2017 was compared to the same six-month period in 2018, after introduction of the feedback reports. We examined the number of tests ordered per patient per day in hospital, as well as length of stay and rates of 30-day readmission, unplanned intensive care unit admission, and inpatient mortality. Independent t-tests were used for parametric data and Mann-Whitney U tests were used for non-parametric data. Two-tailed p <0.05 was considered significant. Additionally, the number of tests per patient-day in hospital for two departments during one rotation in 2018 and 2019, before and after the introduction of pathology ordering guidelines, was compared. 

Results: After the introduction of feedback reports, the median number of tests ordered per patient-day in hospital decreased from 2.3 to 2.0, a 10.3% reduction (p < 0.001). Across the same period, there was no significant difference in median length of stay, 30-day readmission, intensive care unit admission or inpatient mortality. Colorectal surgery and Haematology inpatients experienced a 13% and 35% reduction in tests per patient-day in hospital respectively following implementation of pathology ordering guidelines.

Conclusion: Individualised feedback reports and pathology ordering guidelines are a simple and effective nudge to encourage a more rational approach to test ordering and reduce excess investigations without compromising quality of care.

Background: Many of the ongoing issues in surgery remain related to errors in human judgment and decision-making. We sought to identify the use of choice architecture or “nudges” in surgery.

Methods: A critical review of the key findings from behavioural economics was performed. The key principles of choice-architecture design were mapped onto recent advances in surgery. Additionally, we performed a scoping review of electronic databases to identify articles that have explicitly referenced “behavioural economics”, “nudges” or “nudging”, “choice architecture” AND “surgery”.

Results: Thaler & Sunstein suggest key principles in the design of good choice-architecture through their mnemonic, “NUDGES”  (iNcentives, Understanding mappings, defaults, Give feedback, Expect error, Structure complexity). Choice-architecture, on critical review, was found to be widespread within surgery. The salience of an incentive can be just as important as the incentive itself. In surgery, incentives have been used to motivate surgical teams to improve their performance. A neurosurgical unit that used a display board lowered time to discharge, decreased the number of Clostridium difficile infections, and increased patient satisfaction. Understanding mappings is the concept that decision-makers can be helped by making the consequences of their decisions more explicit. clear. Surgeons are called upon daily to help patients make the decision that best aligns with the patient’s own wishes and desires. Patient decision aids, “worst case, best case” scenarios and other methods are important tools in shared decision-making in the perioperative setting. Defaults are a powerful component of modern surgical care, such as the World Health Organization (WHO) Safe Surgical Checklist. Defaults manifest themselves in surgery in many other ways, such as through Enhanced Recovery After Surgery (ERAS) programs. A good nudge should make feedback more visible to decision-makers. This is found in anesthetic machines alarms, morbidity & mortality rounds, and more explicitly, feedback through video-based feedback or coaching. Expecting error is the concept that systems should expect humans to make errors. Instrument counts in the operating room avoid post-completion errors, and surgical staplers will not fire if loaded incorrectly. Many surgeons utilize cognitive methods to help to structure complexity. In laparoscopic cholecystectomy, the critical view of safety has helped to mitigate common bile duct injuries through a structured approach.

Review of Behavioural Economics in Surgical Literature: From our scoping review of publications on electronic databases that utilized the terms “nudges AND surgery”, we found 11 articles that specifically discussed behavioural economics, nudges, or choice architecture. Overall themes included improving surgical decision-making, patient decision-making, or general reviews of the concepts of behavioural economics in surgery.  

Limitations: It was not possible to perform a systematic review of all possible uses of “nudges” in surgery, given the heterogeneity of the interpretation of choice architecture utilization. 

Conclusions: While surgery clearly is replete with examples of choice architecture, few surgeons have utilized the findings from behavioural economics to guide the rollout of interventions. There is a significant opportunity to increase the collaboration between surgeons and behavioural economists to improve the effectiveness of surgical quality improvement projects. 

Abstract: For many vaccine-preventable diseases such as influenza, vaccination rates are lower than optimal to achieve community protection. Those at high risk for viral infection and serious related complications are especially advised to be vaccinated to protect themselves. Using influenza as a model, we studied one method of increasing vaccine uptake: informing high-risk patients, identified by a machine learning model, about their risk status. Patients with primary care providers at Geisinger who were identified as in the top 10% of risk for flu infection and complications (N=39,717) were evenly randomized to 1) a control condition (exposure only to standard direct mail or patient portal vaccine promotion efforts); or to be informed via direct mail, patient portal, and/or SMS that they were 2) at high risk for influenza and its complications if not vaccinated; 3) at high risk according to a review of their medical records; or (4) at high risk according to a computer algorithm analysis of their medical records. Patients in the three treatment conditions were 5.7% more likely to get vaccinated during the 112 days post-intervention than those in the control group (42.6% vs. 40.3% absolute vaccination rates, p < .001), and did so an average of 1.4 days earlier (p < .001). The impact of our messages declined rapidly: the treatment effect was a 9.8% relative increase during the first 30 days of the campaign, when all messages were being distributed to patients, compared to a 1.2% relative decrease during the ensuing 82 days. Improving timing, adding reminders, and incorporating specific reasons patients are at risk could increase impact in future implementations. Some laboratory research has documented a pattern of “algorithm aversion,” in which people choose human over superior algorithmic forecasts, especially after they observe the algorithm make an error. Yet other laboratory research has found instances of “algorithm appreciation,” in which people followed advice more when they thought it came from algorithms than when they thought it came from humans. However, there have been no field experiments directly comparing the effects of disclosing versus not disclosing to individuals the fact that computer algorithms were used to create a specific prediction or rationale for a recommended action. Here we observed no significant differences among risk messages (p > .30 in all cases), suggesting that at least in the context of AI/ML for risk stratification and communication, patients are neither especially averse to nor uniquely appreciative of learning their medical records have been reviewed or that computer algorithms were involved. Identifying patients who are at high risk for serious COVID-19 disease does not require novel or customized machine-learning models, so similar approaches to ours should be considered for encouraging COVID-19 vaccination among high-risk patients.

Abstract: Healthcare patient portals such as MyChart have been shown to improve patient satisfaction and engagement while decreasing administrative burden and reclaiming physician time for patient care activities. In addition, healthcare organizations receive “meaningful use” incentives for certain types of adoption and use of electronic portals by their patients and providers. One attractive benefit of Geisinger's patient portal (myGeisinger, a local instance of MyChart) is that it allows patients to view results from procedures and tests when they are available in the electronic health record (EHR). Otherwise, patients have to wait up to several days for results to arrive in the mail or for a physician to call, if they are notified about their results at all. Nonetheless, in September 2019, 58% of patients seen at Geisinger were not enrolled in the patient portal. In the present study, we used the fundamental nudge principle of timeliness to encourage patient portal enrollment by highlighting this specific benefit of the portal at a time when patients would be best situated and motivated to take advantage of it.

Over 15 days in May–June 2020, all Geisinger patients who had new test results available in the EHR but who were not currently enrolled in myGeisinger (N = 4605) were randomized to either receive or not receive an email within 24 hours of their result being posted. The emails informed patients that they had test results available and that these could be accessed immediately via the portal. Patients who received a notification were 3.1 times as likely to enroll in the portal as those who did not receive one, with one-week enrollment rates of 12.1% compared with 3.9%, respectively (p < 0.001). Over the ensuing six months the patients we nudged used the portal at a similar rate to the non-nudged patients (21.4 vs. 23.9 total logins, p = .54), suggesting that our nudge converted non-users into typical rather than “one and done” users.

Two versions of emails were tested, one suggesting that patients could view their results directly through myGeisinger by clicking an embedded hyperlink, the other suggesting that patients could view their results after a quick myGeisinger registration process by clicking the same link. Both links took patients to the same page, where they could register within a few minutes. Follow-up analyses revealed that patients receiving the message implying that they could access their results with one click were significantly more likely to click the registration link than those told there was a registration process required to see their results (35% vs. 23%, respectively, p < 0.001) and they were nominally more likely to complete enrollment (14% vs. 12%, p = 0.154).

These findings illustrate the value of nudging patients when the information presented is maximally relevant to a temporally proximal potential decision point (and thus increases motivation to engage with the nudge) and demonstrate how such a technique can be harnessed to increase adoption of a service that improves patients' access to their healthcare.

Background: Care in the medical intensive care unit (MICU) is complex and has resulted in the incorporation of rounding checklists to increase adherence to evidence-based care. Studies examining rounding checklists have shown mixed results. There is randomized control trial evidence that checklists can increase uptake of certain care elements despite lack of improvement in mortality, infection rates, or length of stay. We used the Mindspace framework with an emphasis on the messenger during rounds and changes in default care to perform a nudge intervention. We sought to develop and implement a rounding checklist to standardize and systematically address evidence-based care recommendations identified from the literature and self-identified by staff as commonly missed during MICU rounds.

Methods: We administered a Likert scale and free response survey to MICU providers including attending physicians, fellows, residents, and advanced practice providers to identify frontline staff interest in the use of a rounding checklist, the format of questions preferred, and perceptions of commonly missed clinical care details. We developed a modified checklist based on a published, validated ICU rounding checklist, incorporating practices felt to be particularly high priority based on survey responses. Internal EHR-based baseline data from January 1, 2021, through April 30, 2021, was analyzed to assess pre-implementation process and outcome measures of interest including use of urinary catheters and central lines, VTE prophylaxis adherence, glycemic control, total routine lab order volume (hemograms, CBC, BMP, serum lactate), and SBT adherence. Summary statistics were used to evaluate survey results and to assess changes in process and outcome measures. 

Results: Our pre-implementation survey had 29 responses (30% response rate) composed of 18% attending physicians, 25% fellows, 39% residents, and 18% advanced practice providers. When asked to rate the utility of a checklist for improving MICU rounds on a scale from 1 (not helpful) to 5 (very helpful) the trend was towards favoring use with 59% responding ‘somewhat helpful’ or ‘very helpful.’ The three care items providers felt were missed most frequently included: discontinuing unnecessary lab orders (86%), removing unnecessary lines (76%), and removing unnecessary urinary catheters (66%). 79% of respondents preferred yes/no questions while 21% preferred selecting from a list of responses. Notably, 59% of respondents emphasized the need for brevity in the rounding checklist. Baseline data from 1426 MICU patient-days showed 59.7% of patient-days with a urinary catheter in place, 54.4% of patient days with central lines in place, 29.5% of patient-days without VTE prophylaxis, 42.4% of patient-days outside of glycemic control parameters (<80mg/dL or >180mg/dL), 3.72 routine lab orders per patient-day, and SBT performed on 28.1% of 612 ventilator-days. Our checklist is currently in the initial implementation stage.

Conclusions: Using stakeholder input, we designed and implemented a MICU rounding checklist as a nudge towards improving MICU care via adherence to evidence-based guidelines. We found providers prefer a brief, simplified checklist and self-identified removing lines, urinary catheters, and unnecessary lab orders as the most pressing issues in our MICU. We are currently implementing this checklist and will review process and outcome measures on a regular basis.

Background: The ability to learn a motor skill and effective motor performance is vital for surviving and successful living, with skilled movement being critical in many activities. But cognitive declines associated with aging negatively affect the ability to learn motor skills. Given the ever-growing population of older adults, it has become imperative to develop interventions that will assist them to live independent lives for as long as possible. In our study, we examined the effectiveness of an intervention based on the OPTIMAL theory of motor learning and performance on motor skill acquisition and the ability to manage proactive interference, in both younger and older adults. 

Methods: The experiment was designed as a mixed factorial study. We recruited 39 younger adults and 30 older adults and randomly assigned them to either the experimental group or to the control group.  Both groups were given a novel visuomotor task to perform on robotic equipment. While the experimental group received optimized instructions that were a combination of the motivational and attentional paradigms (enhanced expectancies, external focus of attention, and autonomy support), the control group received standard instructions. Executive function and emotional intelligence measures were also collected. 

Results: We found that the intervention affected the two groups differentially. On the whole, it was relatively more successful at improving learning in older adults. The older adults also displayed an enhanced memory trace of the task compared to their younger counterparts, indicating better learning/retention. It appears that the effectiveness of the optimization might have been mediated by various cognitive, emotional, biological, demographical factors, and the type of motor task involved. While in the younger adult groups, executive function was observed to be highly associated with their motor performance, in the older adults, both, executive function and emotional intelligence abilities (emotion management and emotion regulation by reappraisal) were correlated with motor performance. Additionally, the older adults also displayed a positive mindset throughout their process of learning a novel motor skill, while their younger counterparts were more prone to appeared to negative affect, which appeared to have impeded their learning ability.

Conclusion: The idea of the study was to provide evidence of a novel intervention method that was both effective and simple and could be easily employed to enhance motor skill acquisition among older adults in the short term. The hope is, that such findings would pave the way for future work on the application of these techniques across various fields including rehabilitation, therapy, training, education, and sports across different age groups, populations, and health conditions.

Background: Opioid administration and prescribing in Emergency Departments (EDs) has been identified as a contributor to the opioid epidemic. Best practices for the use of alternatives to opioids (ALTs) exist, however utilization of ALTs remains low, and opioids remain a primary analgesic choice by ED providers. Nudges have been identified as a potential tool to influence provider behavior, due to their non-intrusive and low-workload nature. Our objective is to increase the use of alternatives to opioids by iteratively developing and implementing a package of electronic health record (EHR) nudges encouraging provider use of ALT clinical care pathways and changing order choice architecture to guide providers towards ALT medications.     

Methods: This is a program implementation project, with elements of quality improvement, implemented at a large healthcare system with 12 EDs and 450,000 visits per year. Two separate types of nudges were created with the goal of decreasing opioid use and increasing use of ALTs. The first was a non-interruptive reminder for providers to access existing condition-specific clinical care pathways which prioritize ALTs. These pathways had already been created but existed outside the usual workflow and suffered from low utilization despite the high prevalence of their applicable clinical conditions (back pain, headache, vomiting/abdominal pain, dental pain, musculoskeletal pain, and nephrolithiasis). Users are prompted within their usual workflow to “skip to XXX pathway” based on chief complaint and discrete patient factors. The second nudge is a change in ordering search scope and choice architecture where providers ordering an opioid not from an ALT pathway are preferentially presented with ALTs in their search results. This nudge is also non-interruptive and appears within the usual order entry/selection screens and will be implemented within the next two months.  

Results: After the implementation of nudges, we expect to see a decrease in opioid utilization and a corresponding increase in ALTs. Data from pre-implementation will be compared to process and patient-centered outcomes after implementation of both the medication order nudges and the care pathway nudges. Data collection is ongoing. Preliminary data from all EDs over the past two months since pathway reminder implementation shows no change in ALT pathway utilization or opioid administrations per 1000 patients (pre: mean 238.7, post: mean 231.7, median difference: -5.1 CI [-19.7, 6.6]). There is a significant difference in number of opioid prescriptions per 1000 patients (pre: mean 73.8, post: mean 66.0, median difference: -11.3 CI [-14.7, -6.5])

Conclusion: EHR order nudges offers a promising way of creating behavior change among providers. Successful implementation of workflow nudges with real-time monitoring of changes in process (utilization) and patient-centered outcomes (medication use) will quantify the impact of nudges on behavior change regarding choice of analgesics in the ED.

Background: Optimal blood product utilization requires a balance between clinical benefit, costs, and risks associated with transfusions. Other institutions have improved red blood cell utilization through Electronic Health Record (EHR) Clinical Decision Support (CDS) interventions implemented as a bundle of changes with most including interruptive alerts. However, it is unclear which modifications to the user interface led to the improvements. Interruptive alerts may contribute to overall alert fatigue, so it is critical to understand which elements of CDS are effective at increasing appropriate transfusion behavior. Incorporating end-users can yield insight into user experience and contextual awareness. As part of a larger effort to improve blood transfusion practices at our institution, we implemented a user-centered design (UCD) protocol to evaluate proposed changes to the blood transfusion ordering interface. 

Methods: We developed three different versions of blood transfusions orders: 

• General improvements including removal of extraneous material (e.g. extra icons), up-to-date laboratory information, and incorporation of behavioral economic “nudges” such as pre-populated selections and limited checkboxes 
• The general improvements in #1 as well as non-interruptive CDS alerting users when the last hemoglobin was outside the guideline transfusion threshold
• The same general improvements in #1 with the same CDS information as #2, but displayed as an interruptive, “pop-up” known as a “Best Practice Alert” (BPA)

We interviewed 14 providers (residents, advanced practice providers, and attending physicians) from different high transfusing specialties across the institution. We asked providers to “think aloud” while using both the baseline orders as well as the versions described above in response to specific and deliberate design elements embedded in the three versions. We also asked semi-structured questions about usability and user preferences. 

Results: The blood transfusion order preferred by a majority of end-users was the BPA version (n=8/14, 57%) which users rationalizing that the in-line alert was not visually effective in capturing their attention, while the interruptive BPA forced a brief stop in the workflow to consider the guidelines. This was supported by the finding that 42% of the users unknowingly scrolled past the in-line CDS because the text was not stylized, and the lab values did not seem specific to their patient. All users supported the general improvements, though with a wide range of comments. To this end, users universally valued the incorporation of up-to-date lab information as a design feature that made workflow more efficient. Further changes to the user interface were informed by the feedback obtained during these sessions with users. 

Conclusion: The user experience (UX) uncovered through the “think aloud” approach produced a clear and rich understanding of different providers' needs and considerations when ordering to transfuse blood. End-users shared their annoyances and affordances with deliberate design choices. Input from end-users guided the creation of an order-set that was suitable for a spectrum of providers while also nudging adherence to transfusion guidelines to address the institutional goal of improving blood transfusion practices.

Background: Code status orders serve an important role in ensuring patients’ resuscitation preferences are honored regardless of care setting, however, efforts to improve code status documentation have largely focused on the inpatient setting alone. Promotion of appropriate ambulatory do-not-resuscitate (DNR) order utilization is of particular interest to our cancer center due to evidence of poorer post-cardiopulmonary resuscitation outcomes in the oncology population and institutional policies that (1) allow ambulatory code status to persist across ambulatory encounters and (2) require the presence of a valid physician orders for life-sustaining treatment (POLST) or pre-hospital DNR form to support ambulatory DNR designations.  

Methods: We progressively implemented clinical decision support (CDS) within our electronic health record (EHR) to increase the entry of ambulatory DNR orders supported by legal documentation. First, we introduced an interruptive alert to notify clinicians attempting to sign an ambulatory DNR order in the absence of a POLST or pre-hospital DNR to encourage adherence to institutional policy. Nine months later, we introduced a second interruptive alert to suggest entry of an ambulatory DNR order in the presence of a POLST or pre-hospital DNR to encourage code status capture consistent with patients’ documented wishes. Ambulatory DNR order volume and alignment with legal documentation were then examined retrospectively using descriptive and inferential statistics, by comparing the nine-month periods before and after each alert implementation.  

Results: During the evaluation timeframe, 127,968 patients were seen in the ambulatory setting.  Within this population, 504 patients (<1%) had a POLST specifying DNR or pre-hospital DNR on file.  The proportion of ambulatory DNR orders supported by a legally valid POLST or pre-hospital DNR increased from 25.7% (n1 = 35) to 65.6% (n2 = 31, p < 0.05) following introduction of the first alert.  Median monthly ambulatory DNR order utilization remained low at three per month before and after introduction of the first alert.  Median ambulatory DNR order utilization increased to 30 orders per month following introduction of the second alert and the proportion of these orders supported by a legally valid POLST or pre-hospital DNR was 95.6% (n3 = 274, p < 0.05).   

Conclusion: Our CDS framework increased the quantity and quality of ambulatory DNR orders, demonstrating that thoughtful interruptive alerts can improve goal concordant EHR documentation. Layering alerts resulted in a synergistic outcome; while the proportion of legally supported DNR orders increased following implementation of the first alert, DNR order volume did not increase until implementation of the second alert. This CDS framework may be useful to other institutions interested in improving documentation of ambulatory DNR status supported by legal documentation. Opportunities exist to further extend this framework to promote increased POLST completion and utilization of ambulatory DNR orders; for example, a suggestion to complete a POLST upon inpatient DNR designation could increase POLST prevalence and elicit entry of ambulatory DNR orders during future post-admission encounters.

Abstract: Behaviorally-informed direct communications represent a clear tool for practitioners to use to address vaccination uptake, including as part of the unprecedented efforts to combat the COVID-19 pandemic. This paper discusses eight randomized evaluations of such interventions conducted by the United States General Services Administration’s Office of Evaluation Sciences (OES) in collaboration with private and public health facilities, Veterans Health Affairs health care systems, and the US Department of Health and Human Services. These evaluations had a median sample size of 55,000, deployed a variety of light-touch behaviorally informed direct communications, and used administrative data to measure actual vaccination uptake. The interventions were varied in terms of both the target populations as well as the behavioral insights utilized -- they ranged from disseminating planning prompts among Medicare beneficiaries to communicating social norms to schools and daycare centers.  A meta-analysis shows changes in vaccination rates of roughly .2 percentage points, with two OES studies yielding statistically significant increases in vaccination uptake. In light of these findings, this paper offers four key lessons for future vaccination efforts, including the COVID-19 vaccination program: 1) Behaviorally-informed direct communications can increase vaccination rates at scale but may have smaller, less reliable effects than the published literature suggests; 2) Rapid evaluations of vaccination-uptake interventions in real-world contexts at scale are essential for learning what works in specific contexts for populations of interest; 3) Designing vaccination administration systems to support randomized evaluations can make evidence building easier and cost-effective; and 4) Additional evidence is needed to evaluate the cost-effectiveness of behaviorally-informed direct communications relative to other interventions. Taken together, this work highlights the effects of implementing communications interventions at scale and demonstrates the value of conducting rapid evaluations to enable evidence building and support ongoing vaccination efforts.

Background: Cardiac lab testing, including Troponin I [$2.7M/yr], Brain Natriuretic Peptide (BNP) [$620,000/yr] and Creatine Kinase MB (CK-MB) [$300,000/yr] represent the most costly lab tests performed in our health system. In addition, there is variability in how these tests are ordered by providers leading to waste and low-value care. Legacy Health’s High-Value Care Committee aimed to decrease total cost of cardiac testing by 10% relative to baseline by decreasing variability and removing low-value tests.   

Methods: We identified eleven separate cardiac order-sets displaying eleven different ways to order Troponin I varying in test frequency (i.e., every 4 hours x3, every 6 hours x2, every 8 hours without stop, etc.) or test type (i.e., Troponin alone or in combination with CK/CKMB). The most common way to order Troponin I was via a stand-alone single order containing three tests (Troponin/CK/CKMB). A health system standard was developed that two negative Troponin I drawn 6 hours apart is sufficient to rule out coronary syndrome in patients presenting with low-risk chest pain.  We supported this standard with an educational campaign with key provider stakeholders (i.e., Hospitalists, Housestaff, Cardiologists, and ED providers). Local practice patterns showed that the first Troponin is obtained in the ED; therefore, all admission cardiac order-sets were standardized to list Troponin I x 1 as the orderable choice and removed the various other ways. The heart failure order set contained a pre-checked order for BNP to be ordered automatically every 3 days; a practice that is considered low value. The pre-checked BNP order was changed to be de-selected but remained in the order set for providers to opt-in if needed. In the current era of more sensitive and specific Troponin testing, CKMB is no longer recommended as a first line cardiac test. The Troponin/CK/CKMB “triple” order was removed from the electronic medical record. The direct cost of each test was determined and held constant during the study period as follows: Troponin I $100/test, BNP $73/test, CKMB $23/test. Total cost was calculated as the direct cost of each test multiplied by its volume.  Cost-savings was determined by comparing the total costs of tests from the pre-intervention baseline (April 2017-March 2018) to costs post-intervention (April 2018 through Dec 2020). We developed monthly usage reports for each hospital in our health system. The data was reviewed monthly by a project steering committee and shared quarterly with stakeholder groups to reinforce educational efforts.  

Results: In the first-year post-intervention, we achieved a 10% reduction in Troponin ordering, a 30% reduction in BNP ordering, and a 90% reduction in CK-MB ordering with savings of $288,000, $222,000, and $280,000 for each test respectively for a total savings of $790,000. From April 2018 to December 2020, we have saved $2.22M relative to pre-intervention cost patterns.

Conclusion: Coupling provider education with strong interventions to standardize order sets and removing options for low-value testing, we were able to reduce total cost of cardiac testing by 21%, surpassing our goal of 10%. These interventions created sustainable improvements over three years with significant cost savings.

Background: As the opioid crisis records its highest mortality rate to date amid parallel pandemics, physicians are presented with a unique opportunity to lead an industry-wide call to action to save lives. Medication-assisted treatment (MAT) interventions have significantly evolved, yet physician prescribing of these interventions has lagged behind, adversely affecting clinical decision- making and health outcomes. Emergency departments (ED) have been disproportionately impacted, as they have continued to represent the primary touchpoints for individuals suffering from opioid use disorder (OUD). Federal regulations restricting prescriptive authority of the most advanced evidence-based therapy falls under Section 3502 of the Children's Health Act, the Drug Addiction Treatment Act, which mandates obtaining a DEA-X waiver to administer buprenorphine for OUD, a partial agonist that allows for the controlled tapering of opioid dependency. While this increased governmental oversight was intended as a mitigation tool against potential drug diversion, its unintended consequences have contributed to less than 5% of U.S. physicians currently holding a waiver, thereby critically impeding access to life-saving treatment. The DEA-X waiver process’s reputation has long preceded itself and has historically been a barrier to implementing change at the organizational level. To inspire collective behaviors in the direction of becoming waivered, decision-makers must perceive the action’s utility benefits as outweighing its associated costs. Get Waivered, a psychosocial-driven initiative, was founded in 2017 to help address these issues at both the macro and micro levels. Successes from its implementation at Massachusetts General Hospital (MGH) in Boston provide empirical evidence in support of strategic, value-added nudges with the potential to pave the way for OUD treatment reform.

Methods: To identify the disincentives to DEA-X waiver acquisition among emergency physicians, we used a semi-structured interview process, facilitated through two-way dialogue. The extrapolated data was then standardized, revealing trends on which we developed corresponding interventions rooted in behavioral economics principles. The interventions were further classified according to the ADKAR Change Management Model, establishing a structured template to systematically guide future propositions.

Results

Conclusion: Lessons learned from this case study have contributed to a scalable framework for subsequent mobilization solutions at the state, city, and community-based levels, culminating in an integrative effort to change the trajectory of the resurging opioid epidemic. Although the science clearly validates MAT interventions, they diverge from the traditional abstinence-based treatment schema and present several barriers. To address these bottlenecks, behavioral economic theory can be engaged to drive specific practices conducive to improved clinical decision-making and OUD patient outcomes, and therefore warrants further study.