The Order of the Statistical Jedi
Preface
Introduction
The power of repetition (and my…umm…
complicated
history with statistics)
But there’s a better way
The Curriculum Hasn’t Changed in 50 Years!
The General Linear Model Approach
Ethics
History of the Replication Crisis
Dederick Stapel
Darryl Bem
The “P-Hacking” Article
P-hacking
The Scientific Method Movement
Values versus Ethics
The Open Science Values
1. Protecting humanity
2. Seek truth
3. Openness and transparency.
4. Humility and skepticism.
5. Dissemination.
Making Change
Further data analysis ethics.
Measurement
Why am I talking about measurement?
Constructs
Operational Definitions
Validity
Evaluating Validity
Reliability
Evaluating reliability
Increasing Reliability
Variable types
Predictor versus Outcome Variables
Measurement scales
Take-home message
Univariate Distributions
Categorical Variables
Column Sorting
Visualizing
Interpreting Bar Charts
Numeric Variables
What to Look Out For
Univariate Estimates
Central Tendency: What’s the most likely score?
Mean
Mode
Median
Central Tendency in JASP
Central Tendency in R
Variability: How precise are the scores?
Range
Deviations, Standard Deviation, and Variance
Median Absolute Deviation
Variability in JASP
Variability in R
Z-Scores and Probability
Bivariate Visualizations
Avengers Dataset
Visualizing bivariate relationships in R using Flexplot
Visualizing bivariate relationships in JASP using Visual Modeling
Scatterplots: Numeric on numeric
What to look for
Problems to look out for
Practice
Beeswarm plots: Categorical on Numeric
What to look for
Problems to look out for
Other Bivariate Plots
Logistic Plots: Numeric on Binary
Association Plots: Categorical on Categorical
Bivariate Estimates
Statistics Help Us Predict Things
Conditional Estimates
Estimates for Numeric Predictors
Slopes and Intercepts
Making Predictions
When Slopes/Intercepts Don’t Make Sense
Correlation Coefficients
Estimates for Categorical Predictors
Slopes and Intercepts for Categorical Predictors?
Cohen’s
\(d\)
Diagnostics
Models are tools. And they don’t have feelings.
Residuals
Diagnostic tool # 1: Histogram of the residuals
Sensitivity Analyses
Diagnostic tool # 2: Residual Dependence (RD) Plot for Linearity
Statistical Models are Lazy
Residual Dependence Plots
How to Fix Nonlinearity
How to tell if nonlinearity is a problem?
How much nonlinearity is too much?
Diagnostic tool # 3: Scale-Location (SL) Plots for Homoscedasticity
Spread-Location (SL) Plots
Outliers
Independence
Why do models assume independence?
What happens if you violate the independence assumption?
How to detect and handle dependent data?
Summary
The General Linear Model
Wax on, wax off
What is a model
What is the general linear model
What makes a good statistical model?
Prediction Versus Group Differences
Out with the old, in with the shiny
One-Sample T-Test
Traditional Analysis
One-Sample T-Test as a LM
Independent Sample T-Test
Preparing Data for a t-test
Traditional t-test Analysis
LM Approach
Related t-test
Traditional Related t-test Analysis
LM Analysis of a Related t-test
ANOVA
Traditional Analysis of ANOVA
ANOVA as a LM
Regression
Traditional Regression Analysis
LM Approach
Categorical Outcome Variables
It’s All the Same!
Multivariate General Linear Models
What is a multivariate relationship?
Reasons to use multivariate GLMs
Visualizing Multivariate Relationships in Flexplot
Encoding Additional Dimension Using Colors/Lines/Symbols or Panels
Encoding Additional Dimensions Using Added Variable Plots
Summary
Practice
Multivariate GLMs: Conditioning Effects
Multicollinearity
Controlling by conditioning
Conditioning is just residualizing
All the ways of thinking about “conditioning”
Be careful about conditioning! (And using multiple regression)
1. Conditioning will not prove causation.
2. Be Careful what you condition on
3. Only study and interpret the effects of the interest variable
4. Conditioning with interaction effects.
Additional Estimates of Interest
Slopes
R squared.
Semi-Partial
\(R^2\)
Applied Analyses
ANCOVA
Multiple Regression
Multivariate GLMs: Interaction Effects
Visualizing interaction effects
A simple visual trick to tell if there’s an interaction
Interactions between numeric variables
Simple Slopes Analysis
The Flexplot Approach to Interpreting Interactions
The GLM for interaction effects
Common things people screw up in the literature
Gripe #1. Interpreting main effects when interactions exist
Gripe #2: Failing to check whether interactions exist when doing an ANCOVA
Estimates for interactions
Applied Analyses
Factorial ANOVA
Multiple Regression
Mediation Analysis
Probability
Why and when we need probability?
Finite Samples
Infinite sets
Infinite Sets and Sampling
How to ensure a representative sample
Probability Density Functions
Computing Probabilities From PDFs
Chapter Summary
Probability Two: Bayesian Probabilities (Versus Frequentist Approaches)
A Tale of Two Roomates
Tom’s Approach
Egon’s Approach
What do they conclude?
The Bayesian Approach
Strengths of the Bayesian approach
Weaknesses/Objections to the Bayesian Approach
Frequentist/Likelihood Description
Strengths
Weaknesses
Doing Bayesian Analyses in R
Probability 3: The Central Limit Theorem
Groundhog Day
The Central Limit Theorem
Implications of the CLT
Implication #1: Normality doesn’t really matter
Implication #2. If your sample size is large enough, it’s quite likely your estimate is close to the true value
Implication #3: We can (kinda sorta) make inferences about the population
Using the CLT to Making Inferences
Confidence intervals
p-values
Learning Objectives
Model Comparisons
Nested versus non-nested Models
The Fit/Complexity Tradeoff
Model comparisons are tools, not procedures
Visual model comparisons
The
model.comparison
function
AIC/BIC
Bayes Factors
p values
\(R^2\)
The predicted values
What if the statistics disagree?
Hierarchical Regressions: Nested Model Comparisons in SPSS
Non-nested models
Summary
Published with bookdown
The Order of the Statistical Jedi:
Responsibilities, Routines, and Rituals
The Order of the Statistical Jedi:
Responsibilities, Routines, and Rituals
Dustin Fife
2022-04-18
Preface
