Reading results

Lifts, margins, and what a result can and can't tell you.

What a lift means

The difference, in percentage points, between how the people who saw a video answered and how the people who saw the placebo answered. The panel is randomized, so the gap is the video's effect alone. A lift of +4 on ballot support means 4 more points of support among the video's viewers than among the placebo's.

Detectable and inconclusive

A lift is detectable when it meets the study's stated threshold against the placebo, a 90% threshold for the Massachusetts waves. Inconclusive means the test didn't detect an effect at that threshold; it isn't proof the effect is zero. A lift that doesn't clear its test reads gray.

The margin

In Lift Analysis, a lift's margin is a 90% interval. A margin that crosses zero means the test couldn't tell the effect from none.

The RCT badge

A green check and RCT on a clip mean it ran as an arm in a randomized controlled test and its result is on file. A clip without it hasn't been tested, and its panel's figures are illustrative.

Resonance

A 0 to 100 score: where a video's lift on its main outcome ranks among every lift on file for that outcome. 90 means it beat nine in ten. It shows once at least five lifts are in the pool.

Why Wave 3 says simulated

Massachusetts Wave 3 was fielded to a small test panel to prove the sends end to end, and its results were simulated. The report says so.

Are the numbers weighted?

A test's respondents are weighted to its population's targets; for Massachusetts adults, education, party and 2024 vote. The Open primaries survey's topline is self-selected and unweighted.