Protobi's primary bar chart is designed to compare subsets of respondents, but sometimes you need to compare respondents to themselves. Consider the following:
- are they more likely to choose this answer than other answers
- are they more likely to answer yes to this answer than other questions
Then again, maybe you just want to align a grid of questions neatly in a table.
Using a PairedTable
A PairedTable allows you to neatly display collections of questions together, and run pairwise comparisons for stronger statistical tests.
Below is an example from Pew Research's Gender and Generations survey. Each respondent rated how similar they perceive different groups:
In the example above, 28% of respondents report strong conflicts between "Young people and older people." 57% of respondents report strong conflicts between "Poor people and rich people."
The difference in responses seems sizable, but is it statistically significant? That's where PairedTables can be useful.
Paired table
Below is the same data organized as a paired table. The data organized with each child as a column and each value as a row.
PairedTables: statistics
PairedTable uses a paired difference t-test to compare means, and McNemar's test (with small sample corrections) to compare percentages.
In a PairedTable, each column is compared with each other individual column. Blue column labels highlight statistically significant difference between questions.
Cells show counts/frequencies for all respondents, but each statistical test to compare whether differences between two questions are significant considers only respondents who answered both questions.
Turn statistics on/off
You can turn pairwise testing off in "Additional options", press the green checkmark to deselect "testPairwiseSignificance". A blue x indicates an option has been deselected.
Create a paired table
To create a paired table, press the blue square edit icon and select "Chart type," select "paired":
Alternatively, press the blue square edit icon and select "Edit JSON..." and in the JSON, specify a chartType attribute:
{
"chartType": "PairedTable"
}Transpose a PairedTable
The default is to show each variable as a column and each response value as a row. This makes the view look more like a traditional crosstab.
However, if you have many variables and fewer response options you can transpose the table by choosing swap under "Chart options" in the chart dialogue.
This will yield a transposed table with one row per question and one column per response option.
Here, the statistical comparisons are between rows rather than columns:
Stats testing
Protobi can optionally compare columns for significantly significant differences. To enable stats tests, open the Properties Dialog and select “showSignificance”.
Independent vs Paired samples
Protobi can compare columns as either “Independent samples” or “Paired samples”. Paired sample testing is more sensitive (i.e. it is more likely to show a small difference as “significant”). But it only applies when both columns have the same “base”, in other words, when every respondent who answers one column also answers the other.
Independent sample testing is more conservative but it applies when two questions are answered by different respondents. To enable paired testing, open the Properties Dialog and select “testPairwise”.
In the example below we see two questions where the result is 50% green in one and 60% green in the other. But in the first case, we don't know who is who, all we know is that it question says 50% is green, and in the other says 60% is green. In the second case, we can “pair” the samples, and can see if someone switched values or stayed the same.
Stats tests for independent samples
The Z tests is the most common test for proportions between independent samples:
This statistic theoretically has a standard normal distribution for large samples.
There are many variations of this test, particularly if either the sample size or the population size is small. Protobi uses the standard Z test, when both n1 and n2 exceed a minimum sample size, which defaults to 20 for most projects.
Stats tests for paired samples
McNemar’s test is designed to test differences in sample proportions in paired samples. This statistic theoretically has a Chi-Square distribution for large samples. This test also has variations for sample and population sizes.
Protobi uses McNemar’s test with Edward’s Correction for Continuity, which is the variant most commonly today, including the language R :
Examples
The following example shows three variations of two questions, which we might call “before” and “after” .
Ex#1: Independent samples
In the first example, the samples are independent. We can't say who's who or who switched. All we can say is that out of 150 respondents in each sample, in the first one 50% were green, and in the second 60% were green:
In this example
- z = 1.74
- p = 0.081 so “not significant” since p > 0.05
Ex #2: Paired samples
This example shows the same 10% point increase from 50% to 60% but here all of that difference is due to 15 blues switching to green:
- b =15 (discordant pairs in one direction, e.g. green to blue)
- c = 0 (discordant pairs the other direction, e.g. blue to green
- X^2 = ( |15 – 0 | – 1)^2 / (15 + 0) = 13.1
- p = 0.0003 so “significant”, p < 0.05
Ex #3: Paired samples
This example shows the same 10% point increase from 50% to 60% but here all of that difference is due to 15 blues switching to green
Image Caption
- b = 40 (discordant pairs in one direction)
- c = 25 (discordant pairs the other direction)
- X^2 = ( |40 – 25 | – 1)^2 / (40 + 25) = 3.01
- p = 0.082 so “not significant”, p > 0.05
This tutorial describes paired tables and the optional stats tests that can be applied. For your project, we'll be happy to help you set up the appropriate settings for your project at support@protobi.com