II. Definition: Statistical Significance
- Statistical Significance is the probability that study findings are due to chance
- The risk of rejecting the null hypothesis when it is actually true
- In planning a study, a level of risk is chosen (e.g. 5% risk or P-Value of 0.05)
- P-Value is used to denote significance
- P-Value < 0.05: <5% that findings due to chance
- Reflects reproducibility of the study findings only
- Does not predict individual patient's effect
- Confidence Interval
- Provides a range of possible outcomes
- Example: Number Needed to Treat ranges from 20 to 100
- More clinically relevant
- Statistical Significance does not mean clinically useful
- See Clinical Significance below
III. Definition: Clinical Significance
- Reflects how much of an effect a patient sees
- Example:
- Study shows drug x significantly improves Hair Growth
- Reality: Even the patient cannot see the difference
IV. Definition: Heterogeneity
- Step 1: Clinical Heterogeneity (study similarity)
- Assess similarity of who and what was evaluated across pooled, meta-analyzed studies
- Evaluates similarity between studies (systematic review, meta-analysis)
- Combined studies in meta-analysis should have similar protocols (e.g. inclusion and exclusion criteria)
- Step 2: Statistical Heterogeneity
- Assess similarity among study results (were they consistent)
- Assign a P-Value to a combined group of studies that reflects the difference in their results and the likelihood that this is due to random chance
- Large spread of data across more than one study (i.e. greater heterogeneity) suggests the studies should not be combined in meta-analysis
- References
- Newman in Majoewsky (2013) EM:Rap 13(7): 7