The sixth meeting of our Phil Stat Forum*:
The Statistics Wars
and Their Casualties
18 February, 2021
TIME: 15:00-16:45 (London); 10-11:45 a.m. (New York, EST)
For information about the Phil Stat Wars forum and how to join, click on this link.
“Testing with Models that Are Not True“
ABSTRACT:The starting point of my presentation is the apparently popular idea that in order to do hypothesis testing (and more generally frequentist model-based inference) we need to believe that the model is true, and the model assumptions need to be fulfilled. I will argue that this is a misconception. Models are, by their very nature, not “true” in reality. Mathematical results secure favourable characteristics of inference in an artificial model world in which the model assumptions are fulfilled. For using a model in reality we need to ask what happens if the model is violated in a “realistic” way. One key approach is to model a situation in which certain model assumptions of, e.g., the model-based test that we want to apply, are violated, in order to find out what happens then. This, somewhat inconveniently, depends strongly on what we assume, how the model assumptions are violated, whether we make an effort to check them, how we do that, and what alternative actions we take if we find them wanting. I will discuss what we know and what we can’t know regarding the appropriateness of the models that we “assume”, and how to interpret them appropriately, including new results on conditions for model assumption checking to work well, and on untestable assumptions.
Christian Hennig is a Professor in the Department of Statistical Sciences,“Paolo Fortunati”, at the University of Bologna since November 2018. Hennig’s research interests are cluster analysis, multivariate data analysis incl. classification and data visualisation, robust statistics, foundations and philosophy of statistics, statistical modelling and applications. He was Senior Lecturer in Statistics at UCL, London, 2005- 2018. Hennig studied Mathematics in Hamburg and Statistics in Dortmund. He was promoted at the University of Hamburg in 1997 and habilitated in 2005. In 2017 Hennig got his Italian habilitation. After having obtained his PhD, he worked as research assistant and lecturer at the University of Hamburg and ETH Zuerich.
Mayo D. (2018). “Section 4.8 All Models Are False” excerpt from Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars, CUP. (pp. 296-301)
Slides and Video Links:
Christian Hennig’s slides: Testing In Models That Are Not True
Christian Hennig Presentation
- Link to paste in browser: https://philstatwars.files.wordpress.com/2021/02/hennig_presentation.mp4
Christian Hennig Discussion
- Link to paste in browser: https://philstatwars.files.wordpress.com/2021/02/hennig_discussion.mp4
Mayo’s Memos: Any info or events that arise that seem relevant to share with y’all before the meeting. Please check back closer to the meeting day.
*Meeting 14 of our the general Phil Stat series which began with the LSE Seminar PH500 on May 21
National Institute of Statistical Sciences (NISS): The Statistics Debate (Video)
The second meeting of our New Phil Stat Forum*:
The Statistics Wars
and Their Casualties
September 24: 15:00 – 16:45 (London time)
10-11:45 am (New York, EDT)
“Bayes Factors from all sides:
who’s worried, who’s not, and why”
Richard Morey is a Senior Lecturer in the School of Psychology at the Cardiff University. In 2008, he earned a PhD in Cognition and Neuroscience and a Masters degree in Statistics from the University of Missouri. He is the author of over 50 articles and book chapters, and in 2011 he was awarded the Netherlands Research Organization Veni Research Talent grant Innovational Research Incentives Scheme grant for work in cognitive psychology. His work spans cognitive science, where he develops and critiques statistical models of cognitive phenomena; statistics, where he is interested in the philosophy of statistical inference and the development of new statistical tools for research use; and the practical side of science, where he is interested in increasing openness in scientific methodology. Morey is the author of the BayesFactor software for Bayesian inference and writes regularly on methodological topics at his blog.
Relevant background readings for this meeting covered in the initial LSE 500 Phil Stat Seminar can be found on the Meeting #4 blogpost
SIST: Excursion 4 Tour II Megateam: Redefine Statistical Significance:
Information and directions for joining our forum are here..
Slides and Video Links:
Morey’s slides “Bayes Factors from all sides: who’s worried, who’s not, and why” are at this link: https://richarddmorey.github.io/TalkPhilStat2020/#1
Video Link to Morey Presentation: https://philstatwars.files.wordpress.com/2020/09/richard_presentation.mp4
Video Link to Discussion of Morey Presentation: https://philstatwars.files.wordpress.com/2020/09/richard_discussion.mp4
Mayo’s Memos: Any info or events that arise that seem relevant to share with y’all before the meeting.
*Meeting 9 of our the general Phil Stat series which began with the LSE Seminar PH500 on May 21
VI. (June 25) BONUS MEETING: Power, shpower, severity, positive predictive value (diagnostic model) & a Continuation of The Statistics Wars and Their Casualties
There will also be a guest speaker: Professor David Hand:
“Trustworthiness of Statistical Analysis”
Recommended (if time) What Ever Happened to Bayesian Foundations (Excursion 6 Tour I)
Mayo Memos for Meeting 6:
-Souvenirs Meeting 6: W: The Severity Interpretation of Negative Results (SIN) for Test T+; X: Power and Severity Analysis; Z: Understanding Tribal Warfare
-JSM session I am part of on August 6: P-Values and “Statistical Significance”: Deconstructing the Arguments — Topic Contributed Panel
–Nature paper just came out on which I’m a co-author: “Five Ways to Ensure Models Serve Society”
-Selected blogposts on Power
- 05/08/17: How to tell what’s true about power if you’re practicing within the error-statistical tribe
- 12/12/17: How to avoid making mountains out of molehills (using power and severity)
Slides & Video Links for Meeting 6:
Slides: Mayo 2nd Draft slides for 25 June (not beautiful)
Video of Meeting #6: (Viewing Videos in full screen mode helps with buffering issues.)
VIDEO LINK: https://wp.me/abBgTB-mZ
VIDEO LINK to David Hand’s Presentation: https://wp.me/abBgTB-mS
David Hand’s recorded Powerpoint slides: https://wp.me/abBgTB-n4
AUDIO LINK to David Hand’s Presentation & Discussion: https://wp.me/abBgTB-nm
Another link is here.
V. (June 18) The Statistics Wars and Their Casualties
-Amrhein, V., Greenland, S., & McShane, B., (2019). Comment: Retire Statistical Significance, Nature, 567: 305-308.
-Ioannidis J. (2019). “The Importance of Predefined Rules and Prespecified Statistical Analyses: Do Not Abandon Significance.” JAMA. 321(21): 2067–2068. doi:10.1001/jama.2019.4582
-Mayo, DG. (2019), P‐value thresholds: Forfeit at your peril. Eur J Clin Invest, 49: e13170. doi: 10.1111/eci.13170
Mayo Memos for Meeting 5:
–Bonus meeting: 25 June: See General Schedule
-Selected blogposts on Significance Test Wars from March 2019:
- March 25, 2019: “Diary for Statistical War Correspondents on the Latest Ban on Speech.”
- June 17, 2019: “The 2019 ASA Guide to P-values and Statistical Significance: Don’t Say What You Don’t Mean” (Some Recommendations)(ii)
- July 19, 2019: The NEJM Issues New Guidelines on Statistical Reporting: Is the ASA P-Value Project Backfiring? (i)
- November 4, 2019: On some Self-defeating aspects of the ASA’s 2019 recommendations of statistical significance tests
- November 14, 2019: The ASA’s P-value Project: Why it’s Doing More Harm than Good (cont from 11/4/19)
- November 30, 2019: P-Value Statements and Their Unintended(?) Consequences: The June 2019 ASA President’s Corner (b)
- December 13, 2019: “Les stats, c’est moi”: We take that step here! (Adopt our fav word of Phil Stat!) iii
-ASA 2016 Guide’s Six Principles (PDF)
Slides & Video Links for Meeting 5:
Slides: Draft Slides for 18 June (not beautiful)
Video Meeting 5: (Viewing in full screen mode helps with buffering issues.)
VIDEO LINK: https://wp.me/abBgTB-n8