We often hear results of research studies that contradict earlier studies. These misunderstandings and misuses are often passed down from teacher to student or from colleague to colleague. Some practices based on these misunderstandings have become institutionalized.

This course will discuss some of these misunderstandings and misuses. Topics covered include the File Drawer Problem (a.k.a. Publication Bias), Multiple Inference (a.k.a. Multiple Testing, Multiple Comparisons, Multiplicities, or The Curse of Multiplicity), Data Snooping, and ignoring model assumptions.

To aid understanding of these mistakes, about half the course time will be spent deepening understanding of the basics of statistical inference beyond what is typically covered in an introductory statistics course. Participants in this course should gain understanding of these common mistakes, how to spot them when they occur in the literature, and how to avoid them in their own work. Many participants will also gain deeper understanding of basic statistical concepts such as p-values, confidence intervals, sampling distributions, robustness, and model assumptions.

Target Participants:

Middle and Senior Level Officers in the department of planning, research and statistics. This course is also intended for a wide audience, including: graduate students who read or do research involving statistical analysis; workers in a variety of fields (e.g., public health, social sciences, biological sciences, public policy) who read or do research involving statistical analysis; faculty members who teach statistics, read or do research involving statistical analysis, supervised graduate students who use statistical analysis in their research, peer review research articles involving statistical analysis, review grant proposals for research involving statistical analysis, or are editors of journals that publish research involving statistical analysis; and people with basic statistical background who would like to improve their ability to evaluate research relevant to medical treatments for themselves or family members.

Course Description:

Topics covered include mistakes involving

• Uncertainty, probability, or randomness
• Biased sampling
• Problematical choice of measures
• Misinterpretations and misuses of p-values
• Mistakes involving statistical power
• The File Drawer Problem (AKA Publication Bias)
• Multiple Inference (AKA Multiple Testing, Multiple Comparisons, Multiplicities, or The Curse of Multiplicity)
• Data Snooping
• Ignoring model assumptions.

To aid understanding of these mistakes, about half the course time will be spent deepening understanding of the basics of statistical inference (model assumptions, sampling distributions, p-values, significance levels, confidence intervals, Type I and II errors, robustness, power) beyond what is typically covered in an introductory statistics course.