Massachusetts Message Testing · Executive summary
13 of 16 treatment videos showed detectable ballot support differences
Every wave · Massachusetts general adults — not verified likely voters · separate samples
Ballot support, every wave against its own placebo
Separate studies, own placebos
Published versus approximate
The campaign ad is a treatment
No confidence intervals are shown: none are in the data.
Wave 3: one of six videos showed detectable comprehension differences
- 1Wave 1: none of six videos detectable.Lifts from −3 to +3 pp, all inconclusive.
- 2Wave 2: three of four videos detectable.Pahlow, DiZoglio and Matt Wilson.
- 3Wave 3: one of six videos detectable.Dustin Cathcart.
Wave 1 comprehension lifts, for context — all inconclusive
Wave 2 comprehension lifts, for context
The card iteration
Limitation
Different videos and separate samples; no cards-versus-no-cards randomized comparison.
Three questions, two outcomes
- 01Answer background and media-trust questions
- 02Randomly assigned to a group
- 03Watch a test or comparison video
- 04Read the ballot explanation
- 05Answer support and understanding questions
Which format do you trust?
People chose the format they trust most for political or public information. · TV or broadcast ad · Someone speaking to camera · Podcast or interview clip This tells us their preference before the test.
Would you vote Yes?
People said how they would vote on Question 3. Yes · No · Would not vote We compared the percentage choosing Yes across groups.
Did you understand the change?
People selected what Question 3 would change about primary elections. We compared the percentage choosing the correct answer. Asked after everyone read the ballot explanation.
Shares of respondents choosing each answer — not ballot support or persuasion results.
Video–audience pairings to test
Adults 18–34 and 2022 midterm non-voters
Her highest levels: 75% vs 46% placebo among 18–34s and 73% vs 41% among 2022 non-voters. Significant in 29 of 36 subgroups — the safest broad default.
High school or less and $0–$49K households
The only significant video among high school or less (72% vs 51%); 70% vs 42% in $0–$49K households.
Dustin (46%) and Rodriguez (44%) trail placebo with the high-school-or-less group.
$100K+ households, 35–54s and Independent/Other
Highest level of any video in each: 70% vs 50%, 69% vs 51% and 67% vs 45%.
Adults 55+
The only two significant videos with 55+: Littlewood 62% and Danielle 60% vs a 46% placebo.
The official ad does nothing here (47%).
Parents of children under 18
72% vs 50% placebo — the highest level of any video among parents.
Flat with non-parents (54% vs 46%, not significant).
2024 Republican voters
Dustin 69% and Danielle 67% vs a 53% placebo — the only two significant videos in that group.
Adults 55+ and high school or less
The only significant video in each: 54% vs 41% placebo among 55+ and 61% vs 45% among high school or less. Significant in 24 of 38 subgroups — the broadest in Wave 2.
$100K+ households and 2024 non-voters
The only significant video among $100K+ (60% vs 48%); the highest level among 2024 non-voters (65% vs 31%).
Pahlow (56%) and Matt Wilson (55%) are also significant with non-voters.
Latino respondents
75% vs 53% placebo — the highest level of any video in this group. Green (71%) is the only other significant video.
Among Black respondents: 44% vs 52%, not significant (small sample).
Women
The only two significant videos: 61% and 63% vs a 50% placebo.
Pahlow (56%) and DiZoglio (57%) are not significant.
Conservatives
57% and 55% vs a 41% placebo — the only two significant videos.
DiZoglio (46%) and Matt Wilson (48%) sit near placebo.
Respondents who trust social video most
69% and 71% vs a 46% placebo — the highest levels in any media-trust group. Green (63%) is also significant.
Self-selected group; describes stated trust, not a format effect.
Seen in both waves