Meaningfulness refers to the practical, real world application of a statistic. If a statistically significant correlation between variables, for instance, has meaningfulness that correlation says something to the real world and understanding that correlation can have an impact on how people adjust to the situation from here on out. Therefore, while statistical significance is helpful, it is not the end-all-be-all for research: a research finding has to be meaningful for it to have importance. It has to have some sort of impact in the real world for it to be meaningful.

For the researchers who stated that “given this research was exploratory in nature, traditional levels of significance to reject the null hypotheses were relaxed to the .10 level,” I would first suggest that statistical significance is not something that necessarily has to be considered of paramount importance in exploratory research. Statistical significance is more appropriate for testing a hypothesis. For an exploratory research study, the researcher can get by with simply identifying possible connections.

Moreover, if the levels of statistical significance have to be lowered from their standard or traditional thresholds in order to convey significance, one should really think about whether the significance is really there or not. Plus, the main thing is whether or not the relationship is meaningful. For the researchers, predictor and response variables can be helpful in identifying patterns of behavior for whatever the subject is. However, it does not do much good to lower significance thresholds in order to get that “statistically significant” label applied to the findings in order to sell them the way one might sell a can of peas to GMO-hating consumers by labeling the can “organic”.

The researcher of an exploratory study should focus more on explaining why the relationship is meaningful in this case and less on worrying about statistical significance. Instead, of moving the threshold, look at why the findings did not express significance according to traditional levels. Ask yourself, “Was the sample to small?” “Was the test appropriate?” There could be many different reasons for the finding.

As the American Statistical Association (2016) points out, “a p-value, or statistical significance, does not measure the size of an effect or the importance of a result.” In other words, p-value only tells one about the nature of the strength of the relationship between variables. It is not an indicator of meaningfulness. In order for meaningfulness to be explained, the researcher has to look at method, research design, the problem the research is supposed to be addressing, context, sample, theoretical application, and so on. The researcher must be able to explain why a relationship is meaningful or why it applies to the real world problem.

Even if there is no statistical significance between variables, this does not invalidate the research. For example, say a researcher is conducting an exploratory survey. The researcher does not have a hypothesis but may have an assumption—i.e., the researcher may believe that a predictor and a response are likely to be found or that they are meaningful for x, y, z. If the researcher conducts a study with a suitable sample and everything is appropriate, including method, design and analysis, the findings should be examined as is because these findings will help the researcher to refine his thinking about the subject. The findings may reveal something other than what the researcher assumed to be true or expected to find. This is perfectly okay—so long as the data collection and so on were all performed appropriately.

Research is not about always proving one’s assumptions true or about always have statistical significance. If statistical significance is lacking, it is better to accept this and explain what was found by concluding that there was no clear or convincing relationship between the variables in this study. It does not make the study suddenly more valuable to relax the significance standards. The study will have meaning in and of itself if it has been conducted with validity and reliability. If the findings have credibility and trustworthiness, this is what makes the study meaningful—and the researcher just has to be able to explain why it is meaningful.

So a researcher has to be careful about carrying bias with him into a study. That bias if not recognized and accounted for can get the researcher to do a great many silly things—such as relaxing the significance threshold just to be able to slap that “significant” label on the findings. If the standards for “organic” were relaxed to include anything grown in the same neighborhood as an organic farm, the producer would lose all credibility and no one would believe his products to be reliable. The same concept applies to research. Labels are not as important as the actual underlying meaning that they convey.

For that reason, the researcher should focus more on supplying the reader with a sense of why the research has meaning, even if the findings do not corroborate what the researcher expected to find. The researcher is simply there to apply the right method, etc., to a problem and report the findings. The researcher is not on the hook for reporting a certain type of finding or a specific result. The researcher is there to report the results plain and simple and to make sure the study is valid and reliable, credible and trustworthy.

References

American Statistical Association (2016). American Statistical Association Releases

Statement on Statistical Significance and P-Values. Retrieved from http://www.amstat.org/newsroom/pressreleases/P-ValueStatement.pdf