Massachusetts Message Testing · Wave 1
Massachusetts Wave 1: What the test found
October 2–7, 2026 · 3,600 respondents (3,564 weighted) · six treatment videos plus placebo
Five of six videos showed detectable ballot choice differences; none showed detectable comprehension differences
- Population
- Massachusetts general adults — not a verified likely-voter sample
- Fielded
- October 2–7, 2026
- Sample
- 3,600 respondents · 3,564 weighted
- Control
- Placebo video · 494 weighted
- Threshold
- 90% statistical threshold
1 · Support and understanding are different outcomes.
Five videos received the positive persuasion classification. All six overall comprehension results were “no measurable effect.” This study does not show why the two outcomes diverged.
2 · Danielle has the largest reported overall lift estimate.
Her +18–point estimate is higher than the other displayed estimates. The supplied results do not establish that every difference between videos is statistically significant.
3 · No format is shown to perform better in general.
Each video bundles its speaker, wording, duration, production and format. Comparing individual videos cannot isolate a format effect.
4 · Four of five Parallax-produced videos showed measurable positive lift.
Danielle, Littlewood, Donut and Dustin received Grow Progress’s positive persuasion classification versus placebo. Rodriguez did not show a measurable effect. Three Parallax videos had higher reported lift estimates than the official campaign ad; Dustin matched its rounded +11 estimate. Descriptive comparisons of reported estimates: statistical superiority over the official campaign ad has not been established.
Ballot choice
Comprehension
The 90% statistical threshold is a statistical decision rule. It does not mean there is a 90% chance any particular person changed their mind.
The six videos and their results
01 · Danielle Allen
Largest reported lift estimate · 35s street interview
02 · Littlewood
Only negative comprehension estimate — not detectable
03 · Donut
Positive support result; no detectable comprehension difference
04 · Official campaign ad
Tied at +11 with Dustin
05 · Dustin
Tied at +11 with the official ad
06 · Rodriguez
Only video without a detectable support difference
Results are versus placebo. A difference between two point estimates does not establish a statistically significant head-to-head difference.
Ten video–audience pairings to test
Adults 18–34 and 2022 midterm non-voters
Her highest levels: 75% vs 46% placebo among 18–34s and 73% vs 41% among 2022 non-voters. Significant in 29 of 36 subgroups — the safest broad default.
High school or less and $0–$49K households
The only significant video among high school or less (72% vs 51%); 70% vs 42% in $0–$49K households.
Dustin (46%) and Rodriguez (44%) trail placebo with the high-school-or-less group.
$100K+ households, 35–54s and Independent/Other
Highest level of any video in each: 70% vs 50%, 69% vs 51% and 67% vs 45%.
Adults 55+
The only two significant videos with 55+: Littlewood 62% and Danielle 60% vs a 46% placebo.
The official ad does nothing here (47%).
Parents of children under 18
72% vs 50% placebo — the highest level of any video among parents.
Flat with non-parents (54% vs 46%, not significant).
2024 Republican voters
Dustin 69% and Danielle 67% vs a 53% placebo — the only two significant videos in that group.
Urban and $0–$49K audiences
63% vs 44% among urban respondents and 62% vs 42% in $0–$49K households.
Not significant with suburban (57% vs 51%) or $50K–$99K (54% vs 50%).
Latino respondents
Donut and the official ad are the only significant videos here, both 72% vs a 54% placebo.
Low-turnout audiences
60% vs 41% among 2024 non-voters, 63% vs 41% among 2022 non-voters and 66% vs 46% among 18–34s.
Keep away from 55+ (47% vs 46%).
Conservatives and registered Republicans
No video is significant with either group. Best untested bets: Dustin (63% among Republicans) and Danielle (61% among conservatives).
Hold Rodriguez out of rotation — no measurable effect overall.
Significant = video vs placebo inside one subgroup at the 90% statistical threshold, not videos vs each other. 36 subgroups × six videos with no documented multiplicity adjustment, so some marks are likely chance.
Subgroup results
Gender and age
Race/ethnicity and education
Household income and parental status
Ideology and party identification
Community type and 2024 presidential vote choice
2022 midterm turnout and media influence
What the study can and cannot establish
Supported by this study
- Randomized comparisons of the tested videos with a placebo.
- Immediate stated Yes vote choice and correct-answer comprehension outcomes.
- Five positive overall vote-choice classifications under the platform’s 90% threshold.
- No statistically detectable overall comprehension difference for the six videos.
Comprehension caveat
What “no measurable effect” means
Weighted counts by arm
| Placebo | 494 |
| Danielle Allen | 512 |
| Littlewood | 484 |
| Donut | 518 |
| Official campaign ad | 491 |
| Dustin | 520 |
| Rodriguez | 546 |
Methodology notes
- Usual significance threshold: 90%. “Directional” classifications sit above 80% but below 90%.
- Exact interval endpoints and the error-bar confidence level are not supplied, so no numerical confidence intervals are shown.
- The weighted total is not an effective sample size.
- These are immediate survey responses. They do not establish actual election results, lasting opinion change or the outcome of the separate MA paid-flight precinct RCT.
- Weighted on age, gender, race, education, party and 2024 presidential vote.
- The report’s “82% probability Danielle is best” is a platform-reported ranking statistic with an unspecified calculation; it is not used as a headline here.
- Rounded arm counts sum to 3,565; the reported weighted total is 3,564. Both are shown as published.
Limitations
- Green = Grow Progress “Persuasive” (positive at its 90% threshold). Gray = “No measurable effect.” Ordered by reported ballot-choice lift; numbers show display order, not six statistically distinct ranks. Higher bars do not establish statistically significant head-to-head differences.
- The 90% threshold is a statistical decision rule. It does not mean there is a 90% chance any particular person was persuaded. The official campaign ad is a tested video; the placebo video is the experimental control.
- Reported lifts in percentage points versus placebo, as published by Grow Progress. Lifts are calculated before rounding, so differences between displayed whole numbers can be off by a point.
- Ordered by reported ballot-choice lift; numbering is display order, not six statistically distinct ranks. Card notes are descriptive observations, not reasons to deploy a particular video. Production source is bundled with speaker, message and format — this test does not isolate the effect of who produced a video. Placebo arm: n = 494 weighted. Danielle is the confirmed 35-second person-on-the-street clip; the official campaign ad replaced Harris, who is not part of this lineup.
- Results are versus placebo. A difference between two point estimates does not establish a statistically significant head-to-head difference.
- Hypotheses to test, not proven matches. Significance compares each video with placebo inside one subgroup — it does not show one video beats another for that audience. With 36 subgroups × 6 videos, some patterns will be chance. Validate in market: rotate two videos per audience against a holdout.
- These bars are levels, not lifts. Each shows the % selecting Yes in one subgroup and arm. Significance markers compare one video with placebo inside that subgroup — not videos with each other, and not subgroups with each other. Lifts are not derived by subtracting these rounded levels.
- Starting levels differ. A high Yes level is not automatically a large effect.
- Many comparisons. 36 subgroup panels × 6 videos invite chance findings; the report warns about this but documents no adjustment.
- Small cells. The flag marks actual cell counts below 50; displayed sample counts are weighted. An absent flag does not guarantee power.
- Description only. Nothing here identifies audiences to target or creative to serve to a group.
Questions for the research debrief
- Can Grow Progress provide exact intervals and direct video-to-video comparisons?
- Which subgroup differences were prespecified and supported by adequate samples?
- How should the questionnaire’s written explanation affect interpretation of comprehension?
Source: Grow Progress Wave 1 report, fielded October 2–7, 2026. Grow Progress subgroup views, “Overall Percentage” · Moving people toward Yes. Rebuilt from supplied screenshots; the source’s “Percentage points” axis label is replaced with % selecting Yes.