LSE PH500 Meeting

November 19: “Randomisation and control in the age of coronavirus?” (Stephen Senn)

The third meeting of our New Phil Stat Forum*:

The Statistics Wars
and Their Casualties

November 19: 15:00 – 16:45  (London time)
10-11:45 am (New York, EST) 

“Randomisation and Control in the Age of Coronavirus

Stephen Senn

ABSTRACT: Many critics of randomisation have assumed that it is supposed to guarantee balance of prognostic factors, proceeded to show that this is impossible and then concluded that the theory is flawed. However, the shocking truth about randomisation is exactly the opposite of what they suppose. If we knew that all prognostic factors in a randomised clinical trial were balanced, the standard analysis of such trials would be wrong. The analysis that Fisher proposed for randomised experiments makes an allowance for factors being unbalanced. I shall show how this fundamental misunderstanding of how the randomisation and analysis combination deals with error is the origin of a serious error in interpreting trials. I shall illustrate the points with a game of chance and an actual trial. I conclude by recommending that would-be commentators should not presume to analyse the logic of trials until they have analysed some results.

Stephen Senn is a consultant statistician in Edinburgh. His expertise is in statistical methods for drug development and statistical inference. He consults extensively for the pharmaceutical industry in the UK, Europe and the USA on: planning of clinical trials and drug development programmes, project evaluation and prioritization, regulatory advice and representation, data safety monitoring board advice, specialist analyses, and statistical training. Stephen Senn has worked as a statistician but also as an academic in various positions in Switzerland, Scotland, England and Luxembourg. From 2011-2018 he was head of the Competence Center for Methodology and Statistics at the Luxembourg Institute of Health in Luxembourg. He was a Professor in Statistics at the University of Glasgow (2003) and University College London (1995-2003). He received the George C Challis Award of the University of Florida for contributions to biostatistics, 2001 and the PSI Award for most interesting speaker in 25 years of PSI in 2002. In 2009, he was awarded the Bradford Hill Medal of the Royal Statistical Society. In 2017 he gave the Fisher Memorial Lecture. He is an honorary life member of PSI and ISCB.

Information about the Phil Stat Wars forum and how to join is here. 


Readings:

For related posts on randomization by Stephen Senn, see these guest posts from the Error Statistics Philosophy blog:

Slides and Video Links:

Stephen Senn’s slides: Randomisation and Control in the Age of Coronavirus

Stephen Senn’s presentation:

Discussion on Senn’s presentation:

 


Mayo’s Memos: Any info or events that arise that seem relevant to share with y’all before the meeting.

*Meeting 11 of our the general Phil Stat series which began with the LSE Seminar PH500 on May 21

September 24: Bayes factors from all sides: who’s worried, who’s not, and why (R. Morey)

The second meeting of our New Phil Stat Forum*:

The Statistics Wars
and Their Casualties

September 24: 15:00 – 16:45  (London time)
10-11:45 am (New York, EDT) 

“Bayes Factors from all sides:
who’s worried, who’s not, and why”

Richard Morey

.

Richard Morey is a Senior Lecturer in the School of Psychology at the Cardiff University. In 2008, he earned a PhD in Cognition and Neuroscience and a Masters degree in Statistics from the University of Missouri. He is the author of over 50 articles and book chapters, and in 2011 he was awarded the Netherlands Research Organization Veni Research Talent grant Innovational Research Incentives Scheme grant for work in cognitive psychology. His work spans cognitive science, where he develops and critiques statistical models of cognitive phenomena; statistics, where he is interested in the philosophy of statistical inference and the development of new statistical tools for research use; and the practical side of science, where he is interested in increasing openness in scientific methodology. Morey is the author of the BayesFactor software for Bayesian inference and writes regularly on methodological topics at his blog.

Readings:

R. Morey: Should we Redefine Statistical Significance

Relevant background readings for this meeting covered in the initial LSE 500 Phil Stat Seminar can be found on the Meeting #4 blogpost 
     SIST: Excursion 4 Tour II    Megateam: Redefine Statistical Significance: 

Information and directions for joining our forum are here..

Slides and Video Links:

Morey’s slides “Bayes Factors from all sides: who’s worried, who’s not, and why” are at this link: https://richarddmorey.github.io/TalkPhilStat2020/#1

Video Link to Morey Presentation: https://philstatwars.files.wordpress.com/2020/09/richard_presentation.mp4

Video Link to Discussion of Morey Presentation: https://philstatwars.files.wordpress.com/2020/09/richard_discussion.mp4


Mayo’s Memos: Any info or events that arise that seem relevant to share with y’all before the meeting.

*Meeting 9 of our the general Phil Stat series which began with the LSE Seminar PH500 on May 21

Meeting 7 (July 30)–Discussion of JSM 2020 Panel on P-values & “Statistical Significance”

All: On July 30 (10am EST) I will give a virtual version of my JSM presentation, remotely like the one I will actually give on Aug 6 at the JSM. Co-panelist Stan Young may as well. One of our surprise guests tomorrow (not at the JSM) will be Yoav Benjamini!  If you’re interested in attending our July 30 practice session* please follow the directions here. Background items for this session are in the “readings” and “memos” of session 5.

Members: Materials resulting from Meeting 7:

“Work of renowned UK psychologist Hans Eysenck ruled ‘unsafe’”, The Guardian (Oct 11, 2019) (LINK).

*unless you’re already on our LSE Phil500 list

JSM 2020 Panel Flyer (PDF)
JSM online program w/panel abstract & information):

Slides & Video Links for Meeting 7:

DRAFT OF Mayo JSM 2020 SLIDES (PDF)

FINAL Mayo JSM 2020 SLIDES (PDF)

 

Meeting 4 (June 11)

getting beyond…

IV. (June 11) Rejection Fallacies: Do P-values exaggerate evidence? Jeffreys-Lindley paradox or Bayes/Fisher disagreement:

Reading:

SIST: Excursion 4 Tour II

Recommended (if time): Excursion 4 Tour I: The Myth of “The Myth of Objectivity” 


Mayo Memos for Meeting 4

–Souvenirs  Meeting 4: Q: Have We Drifted From Testing Country? (Notes From an Intermission); R: The Severity Interpretation of Rejection (SIR)

FUN! Take a look at Richard Morey’s newly updated SEV app. It will display P-values, power and SEV (click display options). You can change the default by clicking the tab details and then using that link. Don’t forget to change the range of parameter values. If you change n to 25, you’ll get the answers to the example I gave in meeting #2.

  1. Solutions to problems given in Meeting #2: With X̅ =154 (PDF); with X̅ = 152 (PDF)
  2. Using the app for simple P-values: I wasn’t able to use the board to draw the curves for different P-values in meeting #2. Here’s how you can view them using Morey’s app for simple P-values. 

How do you interpret it? This just came out in NEJM (in defending policies based on antibody tests). “In the world of randomized clinical trials, statisticians test scientific hypotheses by requiring a probability of less than 5% that the observed result could have occurred by chance.” (Waiting for Certainty on Covid-19 Antibody Tests — At What Cost?)  https://www.nejm.org/doi/full/10.1056/NEJMp2017739?source=nejmtwitter&medium=organic-social

-See details on Bonus Meeting: June 25.


Slides & Video Links for Meeting

Slides: (PDF)

Meeting 3 (June 4)

III. (June 4) Deeper Concepts: Confidence Intervals and Tests: Higgs’ Discovery:

Reading:

SIST: Excursion 3 Tour III 

(Strongly) Recommended (as much as interests you) Excursion 3 Tour II:It’s the Methods Stupid: Howlers and Chestnuts of Tests


General Info Items:  

-References: Captain’s Bibliography

–Souvenirs  Meeting 3: N (Rule of Thumb for SEV), O(Interpreting Probable Flukes), [L (Beyond Incompatibilist Tunnels),M(Quicksand Takeaway]

-Summaries of 16 Tours (abstracts & keywords)

Excerpts & Mementos on Error Statistics Philosophy Blog


Mayo Memos for Meeting 3:

-5/30 I blogged on nearly every topic in SIST as I was writing it at errorstatistics.com. Using the search on errorstatistics.com for a topic, you’ll often discover the development of ideas and discussion from readers. Reader discussion often saved me from blunders!

-A review essay of SIST by particle physicist Bob Cousins is relevant for the topics of meeting #3: Cousins, R. (2020). “Connections between statistical practice in elementary particle physics and the severity concept as discussed in Mayo’s Statistical Inference as Severe Testing” (Draft February 22, 2020), arXiv:2002.09713v1 [stat.OT].


Slides & Video Links for Meeting 3:

Slides: (PDF)

Video: https://videos.files.wordpress.com/Xmn6q0iz/lecture_3_ph500_trimmed_fmt1.ogv

at-home seminar room w/ blackboard

(Thebes) conference room for seminar

amp & mike

on the desk outside conference room

Meeting 2 (May 28)

.

II. (May 28): N-P and Fisherian Tests, Severe Testing: How to avoid fallacies of tests

Reading:

SIST: Excursion 3 Tour I (focus on pages up to p. 152): 3.13.23.3

Recommended: Excursion 2 Tour II pp. 92-100 (Sections 2.4-2.7)

Optional: I will (try to) answer questions on demarcation of science, induction, falsification, Popper from Excursion 2 Tour II (Section 2.3)

(Use comments on this blog for queries we don’t get to in the seminar. The first comment you write is sent to moderation to be approved; after that it’s automatic.)

HandoutAreas Under the Standard Normal Curve

5 minute refresher on means, variance, standard deviations, and the Normal distribution, standard normal


General Info Items:  

-References: Captain’s Bibliography

SouvenirsMeeting 1: A-D;  Meeting 2 Souvenirs: (E) An Array of Questions, Problems, Models, (I) So What Is a Statistical Test, Really?, (J) UMP Tests, (K) Probativism

[Souvenirs from optional pages–they’re free: (F) Getting Free of Popperian Constraints on Language, (G) The Current State of Play in Psychology, (H) Solving Induction Is Showing Methods with Error Control]

-Summaries of 16 Tours (abstracts & keywords)

Excerpts & Mementos on Error Statistics Philosophy Blog

-Mementos from Excursion 2 Tour II: Falsification, Pseudoscience, Induction 2.3-2.7


Mayo Memos for Meeting 2:

5/27 Today (27 May) is the statistician Allan Birnbaum’s birthday. I put up a blogpost (on my Error Statistics Philosophy blog) with a volume on foundations of statistics that Synthese published in his honor in 1977.

5/27 Sam Fletcher’s review essay of my book SIST is up at the journal Philosophy of Science


Slides & Video Links for Meeting 2:

Slides:
Meeting #2 main slides (PDF)

Supplemental slides (Likelihoodist vs. Significance Tester w/ Bernoulli Trials) (PDF)

Video: https://videos.files.wordpress.com/YgQcHOn3/zoom-_lecture_2_fmt1.ogv

Meeting 1 (May 21)

.

I. (May 21)  Introduction: Controversies in Phil Stat  

Reading:

SIST*: Preface, Excursion 1
Preface
Excursion 1 Tour I

Excursion 1 Tour II

*Note: The above are from proofs, participants should have a copy of the book

Notes:
NOTES on Excursion 1
Postcard

Please ask any questions from the First Meeting in the comments of this blog.


General Info Items:  

-References: Captain’s Bibliography

Souvenirs  Meeting 1: Souvenir A Postcard to Send, Souvenir B Likelihood versus Error Statistical, Souvenir C A Severe Tester’s Translation Guide, Souvenir D Why We Are So New

-Summaries of 16 Tours (abstracts & keywords)

Excerpts & Mementos on Error Statistics Philosophy Blog


Slides & Video Links for Meeting 1:

Slides: (PDF)

Intro video from July 28, 2019
(Viewing in full screen mode helps with buffering issues.)

LSE Research Seminar PH500 21 May – 25 June, 2020

Mayo

 General Schedule  PDF

CLICK ON READINGS

Topic: Current Controversies in Phil Stat (LSE, Remote 15:00 – 17:00; Thursdays 21 May-18 June)

Main Text SIST: Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars CUP, 2018)*:

Slides for each meeting will be on individual meeting posts on this blog.

I. (May 21)  Introduction: Controversies in Phil Stat: [Meeting 1 Blog Post; Slides: (PDF)]

SIST*: Preface, Excursion 1

Preface
Excursion 1 Tour I
Excursion 1 Tour II

*Note: The above are from proofs, participants have a copy of the book  Notes/Outline of Excursion 1 Postcard

II. (May 28) N-P and Fisherian Tests, Severe Testing: [Meeting 2 Blog Post; Meeting #2 Main slides (PDF) & Supplemental slides (Likelihoodist vs. Significance Tester w/ Bernoulli Trials) (PDF)]

SIST: Excursion 3 Tour I (focus on pages up to p. 152): 3.13.23.3

Recommended: Excursion 2 Tour II pp. 92-100

Optional: I will (try to) answer questions on demarcation of science, induction, falsification, Popper from Excursion 2 Tour II

HandoutAreas Under the Standard Normal Curve

III. (June 4) Deeper Concepts: Confidence Intervals and Tests: Higgs’ Discovery: [Meeting 3 Blog Post; Slides (PDF)]

SIST: Excursion 3 Tour III
(Strongly) Recommended (as much as interests you) Excursion 3 Tour II: It’s the Methods Stupid: Howlers and Chestnuts of Tests

IV. (June 11) Rejection Fallacies: Do P-values exaggerate evidence? Jeffreys-Lindley paradox or Bayes/Fisher disagreement: [Meeting 4 Blog Post; Slides: (PDF)]

SIST: Excursion 4 Tour II

Recommended (if time): Excursion 4 Tour I: The Myth of “The Myth of Objectivity”

 

V. (June 18) The Statistics Wars and Their Casualties: [Meeting 5 Blog Post; Draft Slides (PDF)]

SIST: Excursion 4 Tour III pp. 267-286; Farewell Keepsake pp. 436-444.  

-Amrhein, V., Greenland, S., & McShane, B., (2019). Comment: Retire Statistical Significance, Nature, 567: 305-308.

-Ioannidis J. (2019). “The Importance of Predefined Rules and Prespecified Statistical Analyses: Do Not Abandon Significance.” JAMA. 321(21): 2067–2068. doi:10.1001/jama.2019.4582

-Mayo, DG. (2019), P‐value thresholds: Forfeit at your peril. Eur J Clin Invest, 49: e13170. doi: 10.1111/eci.13170

Recommended (and fun) P-values on Trial: Selective Reporting of (Best Practice Guides Against) Selective Reporting

 

VI. (June 25) BONUS MEETING: Power, shpower, severity, positive predictive value (diagnostic model) & a Continuation of The Statistics Wars and Their Casualties: [Meeting 6 Blog Post; Mayo Slides (PDF) & Hand Slides (recorded power point)]

There will also be a guest speaker: Professor David Hand: “Trustworthiness of statistical analysis” (abstract).

SIST Excursion 5 Tour I (pp. 323-332; 338-344; 346-352),Tour II (pp. 353-6; 361-370), and

Recommended: What Ever Happened to Bayesian Foundations (Excursion 6 Tour I)

 

Information Items for SIST

-References: Captain’s Bibliography

Souvenirs

-Summaries of 16 Tours (abstracts & keywords)

Excerpts & Mementos on Error Statistics Philosophy Blog (I will link to items from excerpted proofs for interested blog followers as we proceed)

-SEV apps: Richard Morey. Newly updated Richard Morey SEV app.

-SIST Errata

DELAYED: JUNE 19-20 Workshop: The Statistics Wars and Their Casualties