LSE Research Seminar PH500 21 May – 25 June, 2020

Mayo

 General Schedule  PDF

CLICK ON READINGS

Topic: Current Controversies in Phil Stat (LSE, Remote 15:00 – 17:00; Thursdays 21 May-18 June)

Main Text SIST: Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars CUP, 2018)*:

Slides for each meeting will be on individual meeting posts on this blog.

I. (May 21)  Introduction: Controversies in Phil Stat:  

SIST*: Preface, Excursion 1

Preface
Excursion 1 Tour I
Excursion 1 Tour II

*Note: The above are from proofs, participants have a copy of the book  Notes/Outline of Excursion 1 Postcard

II. (May 28) N-P and Fisherian Tests, Severe Testing:

SIST: Excursion 3 Tour I (focus on pages up to p. 152): 3.13.23.3

Recommended: Excursion 2 Tour II pp. 92-100

Optional: I will (try to) answer questions on demarcation of science, induction, falsification, Popper from Excursion 2 Tour II

HandoutAreas Under the Standard Normal Curve

III. (June 4) Deeper Concepts: Confidence Intervals and Tests: Higgs’ Discovery:

SIST: Excursion 3 Tour III
(Strongly) Recommended (as much as interests you) Excursion 3 Tour II: It’s the Methods Stupid: Howlers and Chestnuts of Tests

IV. (June 11) Rejection Fallacies: Do P-values exaggerate evidence? Jeffreys-Lindley paradox or Bayes/Fisher disagreement:

SIST: Excursion 4 Tour II

Recommended (if time): Excursion 4 Tour I: The Myth of “The Myth of Objectivity”

 

V. (June 18) The Statistics Wars and Their Casualties:

SIST: Excursion 4 Tour III pp. 267-286; Farewell Keepsake pp. 436-444.  

-Amrhein, V., Greenland, S., & McShane, B., (2019). Comment: Retire Statistical Significance, Nature, 567: 305-308.

-Ioannidis J. (2019). “The Importance of Predefined Rules and Prespecified Statistical Analyses: Do Not Abandon Significance.” JAMA. 321(21): 2067–2068. doi:10.1001/jama.2019.4582

-Mayo, DG. (2019), P‐value thresholds: Forfeit at your peril. Eur J Clin Invest, 49: e13170. doi: 10.1111/eci.13170

Recommended (and fun) P-values on Trial: Selective Reporting of (Best Practice Guides Against) Selective Reporting

 

VI. (June 25) BONUS MEETING: Power, shpower, severity, positive predictive value (diagnostic model) & a Continuation of The Statistics Wars and Their Casualties

There will also be a guest speaker: Professor David Hand: “Trustworthiness of statistical analysis” (abstract).

SIST Excursion 5 Tour I (pp. 323-332; 338-344; 346-352),Tour II (pp. 353-6; 361-370), and

Recommended: What Ever Happened to Bayesian Foundations (Excursion 6 Tour I)

 

Information Items for SIST

-References: Captain’s Bibliography

Souvenirs

-Summaries of 16 Tours (abstracts & keywords)

Excerpts & Mementos on Error Statistics Philosophy Blog (I will link to items from excerpted proofs for interested blog followers as we proceed)

-SEV apps: Richard Morey. Newly updated Richard Morey SEV app.

-SIST Errata

DELAYED: JUNE 19-20 Workshop: The Statistics Wars and Their Casualties

 

Meeting 6 (June 25)

VI. (June 25) BONUS MEETING: Power, shpower, severity, positive predictive value (diagnostic model) & a Continuation of The Statistics Wars and Their Casualties

There will also be a guest speaker: Professor David Hand:
      “Trustworthiness of Statistical Analysis”

Reading:

SIST Excursion 5 Tour I (pp. 323-332; 338-344; 346-352),Tour II (pp. 353-6; 361-370), and Farewell Keepsake pp. 436-444

Recommended (if time) What Ever Happened to Bayesian Foundations (Excursion 6 Tour I)


Mayo Memos for Meeting 6:

-Souvenirs Meeting 6: W: The Severity Interpretation of Negative Results (SIN) for Test T+; X: Power and Severity Analysis; Z: Understanding Tribal Warfare

-JSM session I am part of on August 6: P-Values and “Statistical Significance”: Deconstructing the Arguments — Topic Contributed Panel

Nature paper just came out on which I’m a co-author: “Five Ways to Ensure Models Serve Society”

-SEV apps: Richard Morey. Newly updated Richard Morey SEV app.

-Selected blogposts on Power

  • 05/08/17: How to tell what’s true about power if you’re practicing within the error-statistical tribe
  • 12/12/17: How to avoid making mountains out of molehills (using power and severity)

Slides & Video Links for Meeting 6:

Slides: Mayo 2nd Draft slides for 25 June (not beautiful)

Video of Meeting #6: (Viewing Videos in full screen mode helps with buffering issues.)
VIDEO LINK:
https://wp.me/abBgTB-mZ

VIDEO LINK to David Hand’s Presentation: https://wp.me/abBgTB-mS
David Hand’s recorded Powerpoint slides: https://wp.me/abBgTB-n4
AUDIO LINK to David Hand’s Presentation & Discussion: https://wp.me/abBgTB-nm

Another link is here.

 

Meeting 5 (June 18)

.

V. (June 18) The Statistics Wars and Their Casualties

Reading:

SIST: Excursion 4 Tour III: pp. 267-286; Farewell Keepsake pp. 436-444

-Amrhein, V., Greenland, S., & McShane, B., (2019). Comment: Retire Statistical Significance, Nature, 567: 305-308.

-Ioannidis J. (2019). “The Importance of Predefined Rules and Prespecified Statistical Analyses: Do Not Abandon Significance.” JAMA. 321(21): 2067–2068. doi:10.1001/jama.2019.4582

-Mayo, DG. (2019), P‐value thresholds: Forfeit at your peril. Eur J Clin Invest, 49: e13170. doi: 10.1111/eci.13170

Recommended (and fun) P-values on Trial: Selective Reporting of (Best Practice Guides Against) Selective Reporting


Mayo Memos for Meeting 5:

-Souvenirs Meeting 5: S: Preregistration and Error ProbabilitiesT: Even Big Data Calls for Theory and FalsificationZ: Understanding Tribal Warfare

Bonus meeting: 25 June: See General Schedule

-Selected blogposts on Significance Test Wars from March 2019:

  • March 25, 2019: “Diary for Statistical War Correspondents on the Latest Ban on Speech.”
  • June 17, 2019: “The 2019 ASA Guide to P-values and Statistical Significance: Don’t Say What You Don’t Mean” (Some Recommendations)(ii)
  • July 19, 2019: The NEJM Issues New Guidelines on Statistical Reporting: Is the ASA P-Value Project Backfiring? (i)
  • November 4, 2019: On some Self-defeating aspects of the ASA’s 2019 recommendations of statistical significance tests
  • November 14, 2019: The ASA’s P-value Project: Why it’s Doing More Harm than Good (cont from 11/4/19)
  • November 30, 2019: P-Value Statements and Their Unintended(?) Consequences: The June 2019 ASA President’s Corner (b) 
  • December 13, 2019: “Les stats, c’est moi”: We take that step here! (Adopt our fav word of Phil Stat!) iii

-ASA 2016 Guide’s Six Principles (PDF)


Slides & Video Links for Meeting 5:

Slides: Draft Slides for 18 June (not beautiful)

Video Meeting 5: (Viewing in full screen mode helps with buffering issues.)
       VIDEO LINK: https://wp.me/abBgTB-n8

Meeting 4 (June 11)

getting beyond…

IV. (June 11) Rejection Fallacies: Do P-values exaggerate evidence? Jeffreys-Lindley paradox or Bayes/Fisher disagreement:

Reading:

SIST: Excursion 4 Tour II

Recommended (if time): Excursion 4 Tour I: The Myth of “The Myth of Objectivity” 


Mayo Memos for Meeting 4

–Souvenirs  Meeting 4: Q: Have We Drifted From Testing Country? (Notes From an Intermission); R: The Severity Interpretation of Rejection (SIR)

FUN! Take a look at Richard Morey’s newly updated SEV app. It will display P-values, power and SEV (click display options). You can change the default by clicking the tab details and then using that link. Don’t forget to change the range of parameter values. If you change n to 25, you’ll get the answers to the example I gave in meeting #2.

  1. Solutions to problems given in Meeting #2: With X̅ =154 (PDF); with X̅ = 152 (PDF)
  2. Using the app for simple P-values: I wasn’t able to use the board to draw the curves for different P-values in meeting #2. Here’s how you can view them using Morey’s app for simple P-values. 

How do you interpret it? This just came out in NEJM (in defending policies based on antibody tests). “In the world of randomized clinical trials, statisticians test scientific hypotheses by requiring a probability of less than 5% that the observed result could have occurred by chance.” (Waiting for Certainty on Covid-19 Antibody Tests — At What Cost?)  https://www.nejm.org/doi/full/10.1056/NEJMp2017739?source=nejmtwitter&medium=organic-social

-See details on Bonus Meeting: June 25.


Slides & Video Links for Meeting

Slides: (PDF)

Meeting 3 (June 4)

III. (June 4) Deeper Concepts: Confidence Intervals and Tests: Higgs’ Discovery:

Reading:

SIST: Excursion 3 Tour III 

(Strongly) Recommended (as much as interests you) Excursion 3 Tour II:It’s the Methods Stupid: Howlers and Chestnuts of Tests


General Info Items:  

-References: Captain’s Bibliography

–Souvenirs  Meeting 3: N (Rule of Thumb for SEV), O(Interpreting Probable Flukes), [L (Beyond Incompatibilist Tunnels),M(Quicksand Takeaway]

-Summaries of 16 Tours (abstracts & keywords)

Excerpts & Mementos on Error Statistics Philosophy Blog


Mayo Memos for Meeting 3:

-5/30 I blogged on nearly every topic in SIST as I was writing it at errorstatistics.com. Using the search on errorstatistics.com for a topic, you’ll often discover the development of ideas and discussion from readers. Reader discussion often saved me from blunders!

-A review essay of SIST by particle physicist Bob Cousins is relevant for the topics of meeting #3: Cousins, R. (2020). “Connections between statistical practice in elementary particle physics and the severity concept as discussed in Mayo’s Statistical Inference as Severe Testing” (Draft February 22, 2020), arXiv:2002.09713v1 [stat.OT].


Slides & Video Links for Meeting 3:

Slides:

at-home seminar room w/ blackboard

(Thebes) conference room for seminar

amp & mike

on the desk outside conference room

Meeting 2 (May 28)

.

II. (May 28): N-P and Fisherian Tests, Severe Testing: How to avoid fallacies of tests

Reading:

SIST: Excursion 3 Tour I (focus on pages up to p. 152): 3.13.23.3

Recommended: Excursion 2 Tour II pp. 92-100 (Sections 2.4-2.7)

Optional: I will (try to) answer questions on demarcation of science, induction, falsification, Popper from Excursion 2 Tour II (Section 2.3)

(Use comments on this blog for queries we don’t get to in the seminar. The first comment you write is sent to moderation to be approved; after that it’s automatic.)

HandoutAreas Under the Standard Normal Curve

5 minute refresher on means, variance, standard deviations, and the Normal distribution, standard normal


General Info Items:  

-References: Captain’s Bibliography

SouvenirsMeeting 1: A-D;  Meeting 2 Souvenirs: (E) An Array of Questions, Problems, Models, (I) So What Is a Statistical Test, Really?, (J) UMP Tests, (K) Probativism

[Souvenirs from optional pages–they’re free: (F) Getting Free of Popperian Constraints on Language, (G) The Current State of Play in Psychology, (H) Solving Induction Is Showing Methods with Error Control]

-Summaries of 16 Tours (abstracts & keywords)

Excerpts & Mementos on Error Statistics Philosophy Blog

-Mementos from Excursion 2 Tour II: Falsification, Pseudoscience, Induction 2.3-2.7


Mayo Memos for Meeting 2:

5/27 Today (27 May) is the statistician Allan Birnbaum’s birthday. I put up a blogpost (on my Error Statistics Philosophy blog) with a volume on foundations of statistics that Synthese published in his honor in 1977.

5/27 Sam Fletcher’s review essay of my book SIST is up at the journal Philosophy of Science


Slides & Video Links for Meeting 2:

Slides:
Meeting #2 main slides (PDF)

Supplemental slides (Likelihoodist vs. Significance Tester w/ Bernoulli Trials) (PDF)

Meeting 1 (May 21)

.

I. (May 21)  Introduction: Controversies in Phil Stat  

Reading:

SIST*: Preface, Excursion 1
Preface
Excursion 1 Tour I

Excursion 1 Tour II

*Note: The above are from proofs, participants should have a copy of the book

Notes:
NOTES on Excursion 1
Postcard

Please ask any questions from the First Meeting in the comments of this blog.


General Info Items:  

-References: Captain’s Bibliography

Souvenirs  Meeting 1: Souvenir A Postcard to Send, Souvenir B Likelihood versus Error Statistical, Souvenir C A Severe Tester’s Translation Guide, Souvenir D Why We Are So New

-Summaries of 16 Tours (abstracts & keywords)

Excerpts & Mementos on Error Statistics Philosophy Blog


Slides & Video Links for Meeting 1:

Slides: (PDF)

Intro video from July 28, 2019
(Viewing in full screen mode helps with buffering issues.)

The Statistics Wars
and Their Casualties

19-20 June 2020 Delayed (a covid 19 casualty)

London School of Economics (CPNSS)

Alexander Bird (King’s College London), Mark Burgman (Imperial College London),
Daniele Fanelli (London School of Economics and Political Science),
Roman Frigg (London School of Economics and Political Science),
David Hand (Imperial College London), Christian Hennig (University of Bologna), Katrin Hohl (City University London), Daniël Lakens (Eindhoven University of Technology), Deborah Mayo (Virginia Tech), Richard Morey (Cardiff University),
Stephen Senn (Edinburgh, Scotland), Jon Williamson (University of Kent)*

While the field of statistics has a long history of passionate foundational controversy the last decade has, in many ways, been the most dramatic. Misuses of statistics, biasing selection effects, and high powered methods of Big-Data analysis, have helped to make it easy to find impressive-looking but spurious, results that fail to replicate. As the crisis of replication has spread beyond psychology and social sciences to biomedicine, genomics and other fields, people are getting serious about reforms.  Many are welcome (preregistration, transparency about data, eschewing mechanical uses of statistics); some are quite radical. The experts do not agree on how to restore scientific integrity, and these disagreements reflect philosophical battles–old and new– about the nature of inductive-statistical inference and the roles of probability in statistical inference and modeling. These philosophical issues simmer below the surface in competing views about the causes of problems and potential remedies. If statistical consumers are unaware of assumptions behind rival evidence-policy reforms, they cannot scrutinize the consequences that affect them (in personalized medicine, psychology, law, and so on). Critically reflecting on proposed reforms and changing standards requires insights from statisticians, philosophers of science, psychologists, journal editors, economists and practitioners from across the natural and social sciences. This workshop will bring together these interdisciplinary insights–from speakers as well as attendees.

Organizers: D. Mayo and R. Frigg

Logistician (chief logistics and contact person): Jean Miller 

*We expect one or more additional participants