2016 the effects of electronic signatures on voter turnout

Program conception, design, and fundraising by Debra Cleaver (VoteAmerica). Funding for the initial pilot by the Knight Foundation. Analysis by Pantheon Analytics. Originally published on June 26, 2017. Program implemented by a national civic organization (NCO).

Executive Summary

In analyses of five states where NCO’s E-Sign feature was implemented, circumstantial evidence suggests that E-Sign may have had some positive effects on voter participation – though the results are not conclusive, and there was state-by-state variation. In North Carolina and Florida, E-Sign users were much more likely than non-users to have successfully voted absentee, though this could be due to self-selection bias. E-Sign users in these two states, as well as in Nebraska and Arizona, were also more likely than non-users to have voted at all. In Georgia E-Sign users were less likely to have voted than non-users. Overall, however, E-Sign users voted at a right five points higher than those who did not choose E-Sign. This finding is especially promising, given that E-Sign users were younger and had lower vote propensities.

1. Background

Throughout the 2016 election cycle, NCO helped thousands of voters fill out applications to vote via mail-in/absentee ballots. While generally this meant users had to print out their own form and mail it in, in October of 2016 NCO added an “E-Sign” feature giving users the option to take a picture of their signature and submit their absentee application electronically. This feature was made available to users in seven states.

Of these seven states, five produced sufficient sample sizes for analysis: North Carolina, Florida, Georgia, Arizona, and Nebraska. Though E-Sign was also rolled out in Kansas and Idaho, usage numbers were too low to produce meaningful data for analysis.

2. Procedures and Methodology

Of the seven states where E-Sign was enabled, data from Arizona, Florida, Georgia, Nebraska and North Carolina are included in this report. Data from NCO’s absentee ballot request form users were matched with TargetSmart’s national voter file. The matched file contained many appended variables, the most important of which enumerated whether (and by what method) each person voted in the 2016 General election. However, each state reported vote method in slightly different ways. North Carolina and Florida reported absentee, early in-person, and Election Day distinctly.

Georgia, on the other hand, reported votes as advance (i.e. early in-person ​or vote-by-mail) versus some other method (i.e. in-person Election Day voting). Nebraska simply reported whether a vote had been cast, with no mention of vote method. Therefore, this report is only able to examine vote-by-mail data in North Carolina and Florida, and not in Georgia and Nebraska. In Arizona many counties appeared to have misattributed some mail in votes as early in-person voters making it difficult to combine with the other states.

Overall turnout, however, is analyzed across all five states.

E-Sign was rolled out on slightly different days in these five states. In North Carolina, Florida, and Arizona the rollout was on 10/15. In Georgia and Nebraska the rollout appears to have been slightly later on 10/18. The methodology for the analysis in this report is a simple comparison of users in the seven days immediately before and immediately after these respective debuts. While the pre-debut pool cannot be considered a perfect (i.e. randomized) control group for the post-debut pool, those audiences should be similar enough that informative comparisons can be made. Comparisons are also made between users who chose E-Sign versus those who did not during the comparable period.

3. State by State Results

3.1 North Carolina Analysis

The E-Sign feature rolled out on North Carolina on 10/15. In the seven days after E-Sign rollout there were 984 successful users of the absentee ballot request form page, of which 180 (18%) used E-Sign versus the print-and-mail option.

Overall, users were more likely to vote absentee after E-Sign’s deployment (36.3% absentee rate versus 32.1% in the week before E-Sign debuted). As mentioned in the Methodology section above, the users from these two time frames cannot be considered perfect analogues in the way that randomized groups can. But, for the sake of illustration, if these groups ​were​ analogues, then this difference between 36.3% and 32.1% would be considered statistically significant.

Figure 3.1a also shows that E-Sign users voted absentee at a much higher rate than other users during this same time frame (50.6% versus 33.1%). Conversely, E-Sign users voted in-person (early or on Election Day) at lower rates than other users.

Figure 3.1b illustrates the absentee voting rates for users grouped by day of usage. Though there was some day-by-day variation, E-Sign users consistently voted absentee at higher rates than those who did not choose E-Sign. Of course, we cannot discount the possibility that selection bias is at work here. It is possible that the type of person already predisposed to vote absentee was more likely to choose E-Sign in the first place.

3.2 Florida Analysis

The E-Sign feature rolled out in Florida on 10/15/2016. In the seven days after it’s rollout (10/15/2016-10/21/2016) there were 5,095 successful users of NCO’s absentee ballot request form page. Of those, 1,045 (21%) used the E-Sign feature.

Regardless of method, E-Sign users in Florida voted at a higher rate than non-users. Only 24.2% of E-Sign users did not vote, while 30.6% of other users from the same time period did not vote. This contrasts with North Carolina, where there were no differences in overall turnout rates between E-Sign users and others.

As in North Carolina, E-Sign users in Florida ended up voting in very different ways from other users. In the week after E-Sign’s debut, the absentee voting rate for E-Sign users was much higher than the absentee rate for other users (59.4% versus 38.6%). The rate of early in-person voting was lower among E-Sign users, as was Election Day in-person voting to a lesser extent.

Unlike in North Carolina, however, we see an overall dip in absentee voting for Florida’s users (E-Sign or otherwise) during the week after E-Sign’s debut, relative to the week before: 42.8%, down from 47.5%. This is largely because of the low rate of absentee voting by users who ​did not​ use E-Sign. This finding is perplexing, and prompts the question: did the presence of E-Sign somehow discourage voting among the people who did not choose E-Sign? We cannot discount the possibility, though it seems improbable.

One possible explanation is that absentee voting rates were already trending downward during this same two-week period. Figure 3.2b illustrates the absentee voting rates for Florida users grouped by day of usage.

The Florida results are a good reminder of the limits of the methodology used in these analyses. Users from the seven days before E-Sign’s debut may not, in fact, be ideal comparisons to users in the seven days after E-Sign. If late-coming users had a fundamentally lower baseline likelihood of successful absentee voting, then E-Sign may simply have helped stem the tide. It is also possible that E-Sign appealed to those who were already more likely to vote absentee anyway.

3.3 Georgia Analysis

In Georgia, E-Sign was deployed on 10/18/2016. There were 3,569 successful users of NCO’s absentee ballot request form in the week (10/18/2016-10/24/2016) following E-Sign’s deployment, of which 335 (9%) used E-Sign, a somewhat lower rate than previously examined states.

The data provided in Georgia were less granular than in North Carolina or Florida. Votes were recorded as either Advance or Other. “Advance” appears to cover both mail-in voting and early in-person voting, while “Other” likely represents Election Day in-person voting. Since mail-in/absentee voting is not reported distinctly from other methods of voting early, this unfortunately means that we can’t determine whether the vote-by-mail rate was affected by E-Sign.

Overall turnout among E-Sign users was lower as compared to those who did not use E-Sign. Assuming that “Other” represents in-person Election Day voting, we can see that E-Sign users were less likely to vote on E-Day than other users. This contrasts with all four of the other states in this study.

3.4 Nebraska Analysis

E-Sign debuted in Nebraska on 10/18/2016. In the week that followed there were 498 users. Of that, 37 (7%) used E-Sign.

In Nebraska, vote method was not reported. Instead, there was only an indicator for overall voting rate, so it is not possible to compare E-Sign’s effect on vote-by-mail behavior specifically. However, E-Sign users in the sample week voted at rate 14 percentage points higher than non-users. While this is certainly a positive sign, the sample size of users for this state is very low, so we cannot draw strong conclusions from these figures.

3.5 Arizona Analysis

The E-Sign feature rolled out in Arizona on 10/15. For this analysis we looked at the seven days before and after this date. In the seven days after E-Sign rollout there were 1,166 successful users of NCO’s absentee ballot request form, of which 313 (27%) used E-Sign.

Users in Arizona the week after ESign’s deployment had lower turnout than those who used the form the week before (65% versus 73%). However, in the week where E-Sign was available, E-Sign users voted at a 7% higher rate than other users, a substantial improvement.

Interestingly, while it’s true that E-Sign users voted by absentee at slightly higher rates, they voted early-in-person at ​much​ higher rates. This is a counterintuitive result, given that E-Sign is a tool to make voting by mail easier. We suspect, however, that this discrepancy may be due to differing reporting methods by county. For instance, there were almost no absentee ballots reported in Phoenix--but a lot of reported early voting. It is possible that absentee ballots were counted as early votes in some counties. Since we have reason to believe many “Early In Person” votes may in fact be absentee votes, Figure 3.5b combines the early and absentee categories.

Figure 3.5c illustrates the absentee voting rates for users grouped by day of usage – in this case reverting to the strict definition of absentee voting and not including the other Early category. Unlike in other states, there is not a consistent trend showing E-Sign users voting by mail at a higher rate. However, much of this could be attributed to very low sample size on some days and not indicative of any underlying trend.

4. Multi-State Aggregated Results

In the five states that we have analyzed, approximately 17% of a total of 11,262 users chose E-Sign during the week after the feature was introduced. In this section we have aggregated states together where appropriate to identify broader trends.

4.1 Turnout Analysis

Aggregating the results from North Carolina, Georgia, Florida, Nebraska, and Arizona together, we can see tat E-Sign users voted at rate five percentage points higher than users who did not choose E-Sign during that same period.

This elevated rate of voting among E-Sign users is particularly interesting in light of their demographic and behavioral characteristics. As the following section shows, E-Sign users draw disproportionately from groups that are less likely to vote. While this is not conclusive evidence that E-Sign caused more people to vote, it is certainly a positive sign.

4.2 Population Analysis

In the week after E-Sign debuted, E-Sign users were a younger population, with a much higher share of users under 35 and a much lower proportion of users over 45 as compared to the other users. This is especially intriguing given the turnout rates illustrated in the previous section. It is possible – perhaps even likely – that E-Sign helped increase turnout among a cohort that is less likely to vote in the first place, though further randomized controlled testing will be needed to establish a causal relationship.

Figures 4.2b and 4.2c show differences in voter history and turnout predictions. E-Sign users were less habitual voters as compared to the other users, and were therefore predicted to turn out at lower rates than those who did not use E-Sign. This is what makes the gap in actual turnout so notable; E-Sign users voted at higher rates in 2016.

4.3 Vote-by-mail versus other methods

Figure 4.3 combines data for North Carolina and Florida, the only two states where absentee/mail-in-ballot status was clearly defined. These combined results largely mirror those seen in the Florida analysis, since Florida’s user base is about five times the size of North Carolina’s.

5. Discussion

Though there was some state-by-state variation, E-Sign users across the five states in this report voted at a rate five points higher than users over the same period. This is especially notable given that E-Sign users were younger and less habitual voters, which would have predicted a lower turnout than for other users.

In states where the data has been clearly defined, Florida and North Carolina, the rate at which E-Sign users voted by mail was substantially higher, though we cannot rule out self-selection as an explanation. These various findings, while not unimpeachably conclusive, suggest that E-Sign was probably a positive method of stimulating voter participation in 2016.

However, it remains a puzzling finding that, in Florida, users from the week after E-Sign’s debut were less like to vote absentee than users from the week before E-Sign’s debut. There is not a logical explanation for this, except perhaps that absentee rates were already trending downward. Likewise, in Georgia E-Sign users voted at slightly lower rates than other users, bucking the trend. These variations add nuance to the overall findings, which were positive.

6. Next Steps

Additional retroactive research is less important than implementing planned statistically controlled research in the future. When NCO implements new versions of E-Sign, it should run true randomized controlled trials. This will help answer questions about E-Sign’s effects on turnout and vote method much more definitively.

NCO should also run A/B tests to improve E-Sign’s uptake rate beyond the 17% rate seen across the four states in this study. An elevated uptake rate will, numerically, be a huge benefit to any other experiments undertaken with E-Sign – and likely be a boon to users as well.

Appendix