Massachusetts Message Testing · Wave 2
Massachusetts Wave 2: What the test found
October 12–15, 2026 · 2,400 respondents (2,371 weighted) · four treatment videos plus placebo
Four of four videos showed detectable ballot support differences; three showed detectable comprehension differences
- Population
- Massachusetts general adults — not verified likely voters
- Fielded
- October 12–15, 2026
- Sample
- 2,400 respondents · 2,371 weighted
- Control
- Placebo video · 443 weighted
- Threshold
- Source’s 90% statistical criterion
1 · All four videos showed detectable ballot-support differences.
Four treatment videos were compared with one placebo (47% Yes). Each meets the source’s 90% criterion.
2 · Three of four showed detectable comprehension differences.
Pahlow, DiZoglio and Matt Wilson meet the criterion. Green’s ≈ +5 pp is inconclusive — not evidence of no effect.
3 · The two outcomes order the videos differently.
Green has the largest displayed ballot-support estimate; Pahlow the largest displayed comprehension estimate. Neither is shown to differ statistically from the other videos.
4 · Whole videos were tested.
Each video bundles presenter, script, on-screen cards, format and length. Randomizing complete videos cannot isolate which element produced a difference.
Ballot support
Comprehension
The source’s 90% statistical criterion is a statistical decision rule. It does not mean there is a 90% chance any particular person changed their mind.
The four videos and their results
01 · Pahlow
Largest displayed comprehension estimate in Wave 2.
02 · DiZoglio
Detectable differences on both outcomes.
03 · Matt Wilson
Detectable differences on both outcomes.
04 · Green
Largest displayed ballot-support estimate in Wave 2; comprehension inconclusive.
Frames supplied by the team
Results are versus placebo. A difference between two point estimates does not establish a statistically significant head-to-head difference.
How the test worked
- 1Entry questionsIdeology and partisanship, education and presidential vote, plus one custom media-trust question.
- 2Random assignmentEach participant was randomly assigned to one of five arms.
- 3ExposureOne of four treatment videos — Pahlow, Dizoglio, Matt Wilson or Green — or a placebo video. The placebo is the control, not a message.
- 4Written explanationEvery arm read the same written summary of Question 3, shown right, before either outcome question — so comprehension reflects recognition after reading, not video recall alone.
- 5Two outcomesBallot choice first, then comprehension. Yes and the correct description were the source’s desired outcomes; “Would not vote” was anchored last.
Shown to every arm after the video · as printed in the source
Two outcomes in detail
Reading a lift
Between videos
Which of the following best describes the main change Question 3 would make to Massachusetts state primary elections?
- CorrectCreate one primary with all candidates on the same ballot, with the top two vote-getters overall advancing to the general election
- Keep separate party primaries, but allow every voter to choose which party’s primary to vote in
- Create one primary ballot, with the top candidate from each political party advancing to the general election
- Eliminate primary elections and put all candidates directly on the general-election ballot
Ten video–audience pairings to test
Adults 55+ and high school or less
The only significant video in each: 54% vs 41% placebo among 55+ and 61% vs 45% among high school or less. Significant in 24 of 38 subgroups — the broadest in Wave 2.
$100K+ households and 2024 non-voters
The only significant video among $100K+ (60% vs 48%); the highest level among 2024 non-voters (65% vs 31%).
Pahlow (56%) and Matt Wilson (55%) are also significant with non-voters.
Latino respondents
75% vs 53% placebo — the highest level of any video in this group. Green (71%) is the only other significant video.
Among Black respondents: 44% vs 52%, not significant (small sample).
Women
The only two significant videos: 61% and 63% vs a 50% placebo.
Pahlow (56%) and DiZoglio (57%) are not significant.
Conservatives
57% and 55% vs a 41% placebo — the only two significant videos.
DiZoglio (46%) and Matt Wilson (48%) sit near placebo.
Respondents who trust social video most
69% and 71% vs a 46% placebo — the highest levels in any media-trust group. Green (63%) is also significant.
Self-selected group; describes stated trust, not a format effect.
Moderates, some college and $0–$49K households
The highest level in each: 65% vs 48%, 64% vs 49% and 60% vs 49%.
Leads by one point or less. Green is also significant in all three; Matt Wilson in the first two.
2022 midterm non-voters
64% vs 40% placebo — the highest level. Matt Wilson (61%) and Green (62%) are also significant.
Turnout is self-reported.
Independent/Other party ID
Every video is significant: 55% to 63% vs a 38% placebo, the lowest party placebo level.
Matt Wilson is highest (63%).
Democrats, Republicans, liberals and parents
No video is significant in any of these groups. Highest levels: Green at 63% among Democrats (vs 54%) and Republicans (vs 51%).
Placebo levels start higher here (51%–58% Yes).
Significant = video vs placebo inside one subgroup at the source’s 90% statistical criterion, not videos vs each other. 38 subgroups × four videos with no documented multiplicity adjustment, so some marks are likely chance.
Subgroup results
Levels, not lifts. Each bar is the % selecting Yes in one subgroup and arm.
Many comparisons. 38 subgroups × 4 videos; no multiplicity adjustment is documented.
Small cells. Hatched, italic = source flag for fewer than 50 actual responses.
Description only. No interaction tests; nothing here identifies audiences to target.
Gender, age, race and education
Household income, parental status, political ideology and party ID
Urbanicity, 2024 presidential vote choice, 2022 midterm turnout and media market
What the study can and cannot establish
Supported by this study
- Three complete videos produced detectable recognition gains versus placebo — consistent with the aim of the added cards.
What it cannot establish
- Attribute the gains to the cards rather than script, presenter, format or length. Green’s ≈ +5 pp is inconclusive, not evidence of no effect.
Comprehension caveat
What “no measurable effect” means
Weighted counts by arm
| Placebo | 443 |
| Pahlow | 482 |
| DiZoglio | 484 |
| Matt Wilson | 506 |
| Green | 457 |
Methodology notes
- No confidence intervals are shown because exact intervals were not supplied. Comprehension lift is the same comparison for the share correctly identifying the change Question 3 would make. Wave 1 names link to the review copies embedded in the original Wave 1 report. Wave 2 names link to the tested MP4 listed in the Grow Progress report. Sources: Grow Progress Wave 1 and Wave 2 reports; verified results register.
- Levels are rounded displayed percentages from the Grow Progress Wave 2 comprehension topline; ≈ lifts are arithmetic differences between them. Detectable = meets the source’s 90% criterion versus placebo; inconclusive is not evidence of no effect. Wave 1 lifts are published values versus the Wave 1 placebo.
- Comparison video = study placebo. Source: Grow Progress original reports, W1 pp. 11/18, W2 pp. 10/17. Weighted counts: W1 1,919 / 855 / 790 (total 3,564); W2 1,290 / 581 / 501 (total 2,372; reported 2,371, a rounding difference). Stated trust is not a measure of persuasion effectiveness. Exact questions and detailed methods appear later in the report.
- Levels are rounded displayed percentages; ≈ lifts subtract the placebo level. Dashed tick = placebo. Shown in source order.
- Card notes are descriptive, not reasons to deploy a video. “Watch tested MP4” opens the file listed in the Grow Progress report (verified publicly playable on October 15, 2026). “Review copy” opens the current Frame.io copy; rounded durations match, but exact-cut identity is not established. Durations are from the tested MP4 metadata; all four are 1440 × 2560 vertical.
- General adult population, Massachusetts; in field Monday, October 12 to Thursday, October 15, 2026. Weighted on age, gender, race, education, party and 2024 presidential vote choice. Rounded arm counts sum to 2,372; the published weighted total is 2,371 — both shown as published. Weighted counts are not effective sample sizes.
- Share selecting Yes on Question 3 after each arm. Recreated from the source values; no intervals are drawn because their confidence level is not documented.
- The ballot topline chart in the Grow Progress PDF (page 3) renders visually collapsed and is not reproduced. Values are recovered from the dashboard screenshot above (2026-10-15, 9.08.50 AM). That crop is unlabeled; its attribution to Ballot Vote Choice is inferred from the surrounding supplied report context and because its values (47 / 57 / 58 / 58 / 61) differ from the comprehension toplines (45 / 56 / 54 / 51 / 50). All four treatment bars carry the source’s significance outline.
- Share choosing the correct description of the change, after the written explanation. Recreated from the source’s rounded values.
- Comprehension outlines in the source apply to Pahlow, DiZoglio and Matt Wilson only. The source’s “probability to be best” figures (70% for Pahlow on comprehension, 6% on ballot choice) use an undocumented calculation and are not used as findings. Comprehension subgroup charts (source PDF pages 11–17) were not supplied legibly and are not reproduced.
- Calculated approximations from rounded weighted counts (denominator 2,372). No Wave 1 distribution is available in the supplied report.
- Green bar with dark outline = significant versus placebo inside that group at the source’s 90% threshold; gray = not marked; lavender = placebo; dashed line = placebo level. Rebuilt on a 0–100% scale from the Grow Progress Media Influence breakout (screenshot 9.10.58 AM).
- Rebuilt on a common 0–100% scale from the supplied Grow Progress ballot-choice subgroup screenshots (9.09.04–9.09.27 AM); the source’s “Percentage points” axis label is replaced with % selecting Yes. Significance compares one video with placebo inside one subgroup — not videos with each other, and not subgroups with each other. Weighted counts by cell are not reported.
- Rebuilt on a common 0–100% scale from the supplied Grow Progress ballot-choice subgroup screenshots (9.09.34–9.10.01 AM); the source’s “Percentage points” axis label is replaced with % selecting Yes. Significance compares one video with placebo inside one subgroup — not videos with each other, and not subgroups with each other. Weighted counts by cell are not reported.
- Rebuilt on a common 0–100% scale from the supplied Grow Progress ballot-choice subgroup screenshots (9.10.17–9.10.41 and 9.11.04 AM); the source’s “Percentage points” axis label is replaced with % selecting Yes. Significance compares one video with placebo inside one subgroup — not videos with each other, and not subgroups with each other. Weighted counts by cell are not reported.
Limitations
- Different videos and separate samples; no cards-versus-no-cards randomized comparison.
- Unchanged frames from Wave 2 creative. Examples only — not proof that the cards caused the results, and not attributed to individual videos.
- These are shares of respondents choosing each answer — not ballot support or persuasion results.
- Hypotheses to test, not proven matches. Significant = video vs placebo within one subgroup (90% threshold), not video vs video; with no documented multiplicity adjustment, some patterns will be chance. Full pairings: pages 9 and 23.
- Green bar = detectable at the source’s 90% criterion. Gray = inconclusive. All lifts (≈) are differences between rounded displayed percentages; order is the source’s listing order, not a ranking.
- The 90% criterion is the source’s statistical decision rule. It does not mean there is a 90% chance any particular person changed their mind.
- Unchanged frames. Examples of the added cards — not proof of card causality, and not attributed to individual videos.
- It describes who respondents were at the start. It is not an outcome of the videos, and it is not an experimental test of which medium works best.
- These are % selecting Yes inside each self-selected group — not the share choosing each trust response.
- Hypotheses for further testing, not targeting recommendations. Significant = video vs placebo inside one subgroup at the source’s 90% threshold — not videos vs each other. 38 subgroups × 4 videos with no documented multiplicity adjustment, so some marks are likely chance. Ballot-choice levels only.
Source: Grow Progress, “Rapid Message Test Results: Massachusetts nonpartisan primary round 2,” fielded October 12–15, 2026. Source record, unchanged: Grow Progress dashboard, key experiment details (screenshot 2026-10-15, 9.13.59 AM; labeled Wave 2 by the team). Sources: Grow Progress Wave 1 and Wave 2 reports; verified results register.
Seen in both waves