VoteAmerica June 2020 ballot chase program.

Prepared by: Rachael Firestone, M.A. and Christopher Mann, Ph.D. with Annie Wang, J.D. Originally published on September 28, 2020

Summary:

In large-scale randomized controlled tests during the June primaries in five states, VoteAmerica demonstrated that treatments using MMS text messaging and robocalls generate significant increases in the return of mail ballots. Message testing indicates that multiple message strategies are successful, with no statistically significant differences among messages. Messages are drawn from pre-pandemic SMS mobilization tests or carefully framed pandemic-related messages. A single round of treatment increased turnout by one-third of a percentage point in a test for June 2nd primaries. Two rounds of treatment increased turnout by nearly one percentage point in a test for June 9th primaries.

Background:

Due to the challenges of voting in person during the COVID-19 pandemic, many individuals have turned to casting a ballot by mail in 2020. However, relatively little research has been devoted to encouraging voters to return mail ballots. The goal of this program was to inform future mail ballot chase programs by shedding light on the impact of receiving encouragement to return mail ballots and the most effective message for doing so.

This testing was conducted in five states during primaries conducted in June 2020. The three states differed in their past VBM use and approach to VBM in the primary. Maryland and Nevada mailed ballots to every registered voter, which was a large administrative change from relatively low levels of no-excuse mail ballots in prior elections in each state. The Georgia Secretary of State sent mail ballot applications to every registered voter in Georgia for the primary election, a significant change from prior practice although no-excuse mail ballots have been available in Georgia for some time. New Mexico election officials and civic organizations strongly encouraged voters to request mail ballots under long-standing rules to allow no-excuse mail ballots. Pennsylvania allowed no-excuse mail ballots for the first time in the 2020 primary election. Originally scheduled for April 28th, PA’s primary was postponed to June 2nd due to public health concerns and use of mail ballots was strongly encouraged in the interim.

This program targeted individuals who had requested (GA, NM & PA) or received (MD & NV) mail ballots but not yet returned them.

Targets:

Voters of color in Georgia (37,164), Nevada (74,260), New Mexico (3,883), Pennsylvania (9,944), and Maryland (247,444) with high quality cell phone and/or landline phone numbers. (High quality phone numbers were defined using the TargetSmart phone confidence score. Based on prior testing, only scores of 1, 2, or 3 were used (approximately 2/3rds of available phone numbers).

Randomized Controlled Tests:

Records were randomly assigned to receive no contact from VoteAmerica (control group) or one of several treatments.

Random assignment provides clear, causal evidence of the effect of the treatment delivered by VoteAmerica. This research design, known as field experiments, is considered the gold standard for voter mobilization evaluation by academic researchers and the Analyst Institute.

Treatment:

The treatment consisted of text messages if a record had a high-quality cell phone, automated phone calls (robo-call) if a record had a landline high quality phone, or both a robo- call and a text message if a record had both types of phone numbers.

In the first experiment in Maryland, New Mexico, and Pennsylvania, targeted voters received one wave of contact encouraging them to return their mail ballots for the June 2nd primary election (time constraints for program execution prevented a second wave of contact). All targeted voters received the same message in these states.

In the second experiment in Georgia and Nevada, targeted voters received two waves of contact encouraging them to return their mail ballots for the June 9th primary election. Additionally, four different messages were tested in Georgia and Nevada (see Message Test section).

The text messages were delivered using MMS technology (the default for group text messaging on phones) because MMS allows the delivery of images as well as text. Records were randomly assigned to receive either a plain text MMS or an MMS with an image. The images used in MMS messages are included in the figure below.

It is worth noting that past field experiments on using robo-calls for voter mobilization have shown no impact on turnout. However, they may be more effective when used to remind voters to return mail ballots than in mobilizing the more difficult task of going to a polling place.

Message Test:

(see Appendix for copies of each message)

Social Pressure: This treatment successfully increased turnout in a large-scale SMS ballot chase program in 2018. The key mechanism is telling voters that their pro-social behavior (voting) is being monitored: “who you vote for is secret but whether you vote is public record.”

Friends & Family: This treatment also successfully increased turnout in a large-scale SMS ballot chase program in 2018. The key mechanism is a reminder that voting is a social activity, and indirectly that others they care about (friends & family) will monitor this pro-social behavior.

A direct comparison of Friends & Family and Social Pressure messages in SMS voter mobilization for in-person voting in 2019 found Friends & Family to be more effective. Therefore, it was selected for the first experiment where only one message was used.

Anti-coronavirus: This treatment was developed specifically for the pandemic and successfully increased turnout in earlier ballot chase pilot tests in special elections. Motivation to return the ballot is increased by framing voting by mail as an action against the coronavirus and/or the uncertainty created by the pandemic.

Anticipatory benefits: This treatment was drawn from past research on encouraging early in- person voting. It is based on well-developed social psychology research about increased motivation to complete an action when it facilitates looking forward to a future event (e.g. making reservations for a vacation, purchasing tickets to a concert or sporting event). Completing the voting process and looking forward to the outcome has increased early voting in past research.

Random assignment:

The randomization used blocking to increase the probability of pre-treatment balance between randomly assigned conditions for voter characteristics that are correlated with voting behavior. Blocking variables were state, district, number of people of color in household, turnout score, race, number of cell phones in target universe in household, voted by mail in 2018, voted by mail in 2016, voted in 2018, voted in 2016

Timing:

New Mexico, Pennsylvania, and Maryland

Text messages and robo-calls were sent in New Mexico and Pennsylvania on May 29th, 2020 because ballots had to be received by election offices on Election Day (June 2, 2020).

Text messages and robo-calls were sent in Maryland on May 31st because ballots had to be postmarked by Election Day (June 2, 2020).

Georgia and Nevada

Wave 1 text messages and robo-calls were sent in Georgia and Nevada on June 3rd prior to the June 9 primary election.

In Georgia, Wave 2 messages and robo-calls were sent on June 5th because ballots had to be received by election offices on Election Day. In Nevada, messages and robo-calls were sent on June 7th because ballots had to be postmarked by Election Day.

Results:

In the first experiment, one round of treatment generated an increase in turnout of 0.31 percentage points. This effect is consistent with voter mobilization experiments using a single round of text messaging, and in particular with the effects seen from using SMS for ballot chase in 2018. (NB: (The increase in turnout in the first experiment was just outside conventional standards of statistical significance with p = 0.056.)

In the second experiment, three rounds of treatment generated an increase of 0.97 percentage points. This effect was consistent with past voter mobilization experiments in finding that multiple text messages appear to generate more impact on turnout than a single round of text messages.

The second experiment also suggests there is little difference across the tested messages. Three messages were based on messages found to be effective in tests in previous elections, and the fourth was based on poll message testing. Therefore, the absence of statistically significant differences across these messages is unsurprising. Each message caused a statistically significant increase in ballots returned —

• Social Pressure by 1.0 percentage point (p=0.007)
• Friends and Family by 1.1 percentage points (p=0.004)
• Anti-Coronavirus by 1.1 percentage points (p=0.006)
• Anticipatory by 0.74 percentage points (p=0.048).

Although the Anticipatory Benefits treatment is notably lower, it should not be ruled out for use in future elections because it has more effective in high contested close elections in past programs – and neither the Nevada nor Georgia primaries featured such contests.

Adding the images to the MMS messages did not make a significant contribution to the effect on turnout in either test. If anything, this addition made the messages with images very slightly less effective than the plain text messages.

Future Steps:

These results indicate that text messaging and robo-calls are an effective method of increasing turnout among people who are voting by mail. While it can often feel like the pandemic ‘changes everything’, the psychological mechanisms from delivering low-cost messages about return ballots remains an effective way to increase participation.

The negligible, and even very slightly negative, impact of adding images is surprising. Different images might have a larger positive impact. And it is possible that the images somehow reduced delivery. But whether the mechanism is behavioral or technological, these tests strongly indicate that the added cost for these messages is not a wise investment.

Appendix: Treatment Messages

In New Mexico, Pennsylvania, and Maryland, records were randomly assigned to receive no contact from VoteAmerica versus a Friends and Family message via robo-call and text.