What is Significance testing?
In technical terms, a significance test is used to determine whether to accept or reject the null hypothesis. In layman’s terms, it’s the likelihood that the result did not occur randomly and is likely attributable to a specific cause. That means, if a result is shown to be significantly higher or lower, we believe the difference is larger than can be explained by chance.
How Methodify Uses Significance Testing
Significance testing is used within Crosstab reporting and bar charts to identify which data points are significantly higher lower than each other. For our purposes of analyzing data in Methodify crosstab we are using the Test Statistic z (z-test). The z-test is used to calculate a value and then is compared to a value determined by the Confidence Level selected and if greater than this value (or less in some cases), the result is usually deemed significant.
Within the crosstab table and bar chart, each variable is compared against itself to see if significantly higher or lower. If yes, an uppercase letter is placed under the cell to signal that it is significantly higher than another column.