Open Primaries Message Test · Wave 1
Open Primaries Message Test: what the test found
October 26–29, 2026 · 2,503 respondents (2,503 weighted) · four treatment videos plus placebo
One of four videos showed detectable ballot support differences; three showed detectable comprehension differences
- Population
- Respondents who completed the test, weighted to Massachusetts general adults (18+)
- Fielded
- October 26–29, 2026
- Sample
- 2,503 respondents · 2,503 weighted
- Control
- Placebo video · 497 weighted
- Threshold
- 90% criterion (two-sided two-proportion test)
Ballot support
The weighted percentage selecting Yes, compared with the placebo arm.
Comprehension
The weighted percentage choosing the correct description, compared with the placebo arm.
The 90% criterion (two-sided two-proportion test) is a statistical decision rule. It does not mean there is a 90% chance any particular person changed their mind.
The four videos and their results
46 s
01 · Why independents are locked out — Gaby
501 actual respondents
Ballot support61.4%
vs 48.7% placebo · +12.7 ppDetectable
Comprehension56.3%
vs 43.7% placebo · +12.6 ppDetectable
n = 485 weighted
39 s
02 · Why independents are locked out — MSNOW
501 actual respondents
Ballot support53.9%
vs 48.7% placebo · +5.2 ppInconclusive
Comprehension48.3%
vs 43.7% placebo · +4.6 ppInconclusive
n = 518 weighted
21 s
03 · Why independents are locked out (21 s)
500 actual respondents
Ballot support51.9%
vs 48.7% placebo · +3.1 ppInconclusive
Comprehension55.2%
vs 43.7% placebo · +11.4 ppDetectable
n = 486 weighted
41 s
04 · Why independents are locked out (41 s)
500 actual respondents
Ballot support50.2%
vs 48.7% placebo · +1.4 ppInconclusive
Comprehension50.7%
vs 43.7% placebo · +7 ppDetectable
n = 518 weighted
Results are versus placebo. A difference between two point estimates does not establish a statistically significant head-to-head difference.
Two outcomes in detail
Which of the following best describes the main change the measure would make?
- Each party would keep its own primary, but independent voters could choose one to vote in
- CorrectAll candidates would run in one primary, and the two with the most votes overall would advance
- Primaries would be eliminated, and every candidate would go straight to the general election
- General elections would be decided by ranked choice voting
Subgroup results
Bars are levels — the % selecting Yes in one subgroup and arm — not lifts. Significant vs placebo inside that subgroup (90% criterion (two-sided two-proportion test)) Not marked Hatched, italic = source small-sample flag Placebo level in that subgroup
Political ideology and party ID
Education and 2024 presidential vote choice
What the study can and cannot establish
Supported by this study
- Randomized comparison of each video arm with the placebo arm on stated ballot choice and comprehension.
What it cannot establish
- Comparisons between video arms: each arm is tested only against the placebo.
- A verdict from any arm or subgroup with fewer than 50 actual respondents: those comparisons are inconclusive by rule.
What “no measurable effect” means
Inconclusive means the comparison did not meet the criterion; it is not evidence of zero effect.
Weighted counts by arm
| Placebo | 497 |
| Why independents are locked out — Gaby | 485 |
| Why independents are locked out — MSNOW | 518 |
| Why independents are locked out (21 s) | 486 |
| Why independents are locked out (41 s) | 518 |
Arm counts sum to the reported weighted total, 2,503.
Methodology notes
- Counted: 2503 complete records, the latest per panel id, env development. Not counted: 0 complete(s) from another env, 600 earlier duplicate(s), 0 with an arm outside the test, 0 screenout(s); 3 start record(s) seen.
- Raked to education, party, vote2024 in 3 iterations (largest remaining gap 7.1e-7); weights normalized to mean 1, range 0.306 to 4.627, capped at 5 (0 capped). Targets: Massachusetts general adults (18+), derived from the audience store by scripts/results/targets.mjs.
- Weighting target, education (as raked): hs-or-less 31.2%, some-college 21.5%, bachelors-plus 47.3%. American Community Survey 5-year estimates (U.S. Census Bureau); education_4 @25, count; period 2020-01-01 to 2024-12-31; universe population 25+; proxy: population 25+; derivation audience-store. B15003 covers ages 25 and over; the store has no 18–24 education for the state, so the 25+ distribution stands in for adults 18+ and likely overstates bachelor's-or-higher for them.
- Weighting target, party (as raked): democrat 37.3%, republican 14.6%, independent-other 48.2%. Cooperative Election Study common content 2020, 2022, 2024 (Harvard University); pid3 @25, 2024, pct (commonweight); period 2024-01-01 to 2024-12-31; universe adults 18+ (CES state respondents, commonweight); derivation audience-store. Party identification, a survey estimate: n 1119, margin of error up to ±4 points. The printed shares sum to 100.1 and are renormalized to 100. Identification, not enrollment: the state's party enrollment measures registration, a different construct on a different universe. 2024 is used to match the 2024 vote target (--party-year picks another); the store also holds 2023 (n 383, ±5) and 2025 (n 272, ±6.3).
- Weighting target, vote2024 (as raked): democrat 37.4%, republican 22.0%, did-not-vote 38.2%, other 2.4%. Official ballot-measure results (state election offices); American Community Survey 5-year estimates (U.S. Census Bureau); presidential_vote @25, 2024, count; ACS age_bucket @25 (adults 18+); period 2024-11-05 to 2024-11-05; universe adults 18+; derivation audience-store. 3,512,930 ballots cast ÷ 5,679,875 adults 18+; "Did not vote" is the adults who cast no ballot (38.15%), non-citizen adults included. Blank presidential lines (a ballot cast, no presidential choice) count with "other" under the pax-entry map. Respondents over-report voting, so raking to the true non-vote share up-weights the non-voters a panel yields.
- Declared but not asked by this test, so not raked: age, gender, race.
- Levels are weighted percentages. Lifts are arm level minus placebo level, in points, with a margin of 1.645 standard errors. Each comparison is a two-sided two-proportion z-test with Kish effective sample sizes; detectable means p < 0.1 and at least 50 actual respondents in both groups.
Limitations
- No multiplicity adjustment is applied across subgroups.
Source: pax-trinity-test-responses:open-primaries-1, computed 2026-10-29 (env development).