Statistical Significance and Clinical/Practical Significance

At the conclusion of this module, participants will be able to:

• • Evaluate the role of statistical findings in practice improvement.
  • Differentiate between clinical and statistical significance.
  • Determine clinical significance.
  • Translate clinical/practical details for general audiences.
  • Identify implications for practice based on findings.

Definitions:

Statistical Significance- Mathematical conclusion indicating the degree of magnitude of a numeric finding resulting from scholarly investigation.

Clinical/Practical Significance- Practical determination of whether a result is substantive enough to change the activities of practice.

Clinical/Practical Significance

The power in practicality- Practicality is important in practice improvement. The goal of the endeavor, after all, is to provide a real solution for a real environment. It may be easy for some to lose their way when analyzing data. It is quite common for practice improvement projects to generate data and present findings pertaining to statistical significance. However, it is essential that the investigator also present information in a practical light. 

Envisioning your findings at work in the practice setting- When determining appropriate data analysis and presentation techniques, it is useful to first consider how the data will be used in the practice setting. 

• How is the data currently understood? For example: Are you discussing aggregate fall rates, and if so, how is the metric discussed at the organization? 
• How often is the data reported on at the site?
• What language is commonly used at the site when discussing the values being represented in the project? 
• What is the intended use for the data? 
• What format is needed by the stakeholders?

Essentially, stakeholders and project site leaders should not be left to convert the data to their own language or metric style. Practice improvement projects should generate usable findings that are easily communicated to the audience. 

Statistical Significance

Though focus may be on clinical and practical significance for practice improvement projects, it is also useful to consider statistical findings and how significance can be helpful in final presentation of a project outcome. Statistical significance offers a more objective and general measure of worth. It is not to be solely relied on for determining how to use the data but could offer a common language when comparing the finding to works in other settings or locations. Even when findings are statistically significant in number, there are other elements such as small sample size that may make the statistical thresholds less meaningful. The investigator should never blindly claim significance based on analytical results.

Translating the clinical/practical details for general audiences: The so what technique- Implications are amongst the most important constructs authored by the investigator. In the end, there must be conclusions that clearly establish what the data means. Imagine presenting all your information in a presentation to stakeholders. When the presentation is concluded, the audience asks these questions:

• So what?
• What does this information mean for me and for my interests?
• So, what should we do now that we have this information?
• So, what is recommended for future projects? 
• So, what does this mean for the clients/population we serve?

Even if the questions are not actually asked in-person, the investigator should be mindful of them when concluding the work. Reflecting on these questions will help the investigator complete a more robust report that closes the loop on the purpose of the project, and utility of the findings. 

What to do when findings are not significant- In the event findings are not clinically or statistically significant, the recommendations for future projects becomes critical. Lack of significance can still inform practice as it often indicates where resources should not be allocated. Changes in practice can be costly, and it is useful to understand the resource benefit balance. When findings are not significant, go back to the “so what” questions. Also ask yourself:

• What did we learn from this project?
• Were any decisions made based on the findings
• Should the project be repeated, and if so then what should change?

Suggested readings

University of Ottowa. (n.d.) Statistical Significance and Clinical Importance. Retrieved from: https://www.med.uottawa.ca/sim/data/Statistical_significance_importance_e.htm

http://archive.ahrq.gov/research/findings/factsheets/translating/tripfac/trip2fac.pdf 

https://www.uws.edu/wp-content/uploads/2013/10/P_Values_Statistical_Sig_Clinical_Sig.pdf 

Reference 

Page, P. (2014). Beyond statistical significance: clinical interpretation of rehabilitation research literature. International journal of sports physical therapy, 9(5), 726-36.

University of Ottowa. (n.d.) Statistical Significance and Clinical Importance. Retrieved from: https://www.med.uottawa.ca/sim/data/Statistical_significance_importance_e.htm

http://archive.ahrq.gov/research/findings/factsheets/translating/tripfac/trip2fac.pdf 

Zaccagnini, M. & White, K. (2017). The Doctor of Nursing practice essentials: A new model for advanced practice nursing. Burlington, MA: Jones and Bartlett 

Banerjee, A., & Chaudhury, S. (2010). Statistics without tears: Populations and samples. Industrial psychiatry journal, 19(1), 60-5.